Friday, February 02, 2007

New Flickrfs release v1.3.9

After some long time, I again had a chance to work upon my favourite filesystem after reiserfs, flickrfs. This release doesn't add nice cool features, this one is targeted towards robustness, resource utilization and efficiency. The previous versions of flickrfs had various issues, namely:
  • Slow internet connection, time-outs or other errors, would grind flickrfs to a halt; forbidding it to download information about sets, and stream.
  • If the number of images grow, in order of thousands, flickrfs would start consuming tons of RAM and eventually swap space; and would literally make my old thinkpad R40 crash.
  • Retrieval of full metadata information for each image was time and network resource consuming, and mostly not required.
  • Sometimes flickr response would return error, even if the uploading of image succeeds, making flickrfs generate false error report; and confusing the end-user, who would then end up uploading the same image twice. (confusing huh? ;-))
  • For developers, a.k.a. me, flickrfs code was huge, all in one file, with lot of boilerplate code sections.
This version v1.3.9 of flickrfs tackles all these issues (also see features list):
  1. Robustness - Flickr operations, as in, interactions with flickr server are now fail-safe. flickrfs handles the URLError exceptions thrown by the operation, checks its result, and retries the operation multiple times in case of failure. Thus, even if connection is lost temporarily, flickrfs would continue its operations unaffected.
  2. Efficiency - flickrfs retrieves the sets information in parallel, background threads. Hence, the directory structure is created quickly, allowing users to start working. If some sets couldn't be retrieved the first time, they'd be taken care of when *syncing* kicks in. Secondly, full metadata information of images would only be retrieved when asked for; though they'll always be listed in the directory. Its just like the way flickrfs handles images.
  3. Resource Saving - flickrfs would now store the image information in Berkeley db database, instead of storing it in memory. Let come millions of images, flickrfs would still consume only negligible amounts of RAM; regarding hard disk space, it would be in 10s of Megabytes; not an issue for today's computers.
  4. Refactoring of code - Developers, again a.k.a. me, the code is now divided into 3 different files and... wait! hmm.. you shouldn't care about it. Just know that its more beautiful :-).
So, just give it a whirl. I'm sure you'd like it. More about flickrfs on its homepage.

5 comments:

Goran said...

The latest Flickrfs is a thing of beauty and I'm using it as my only Flickr manager / uploader / backuper now. Thank you, kind Sir, for keepin' it real in this horrid age of dragging things onto square windows.

mann said...

@goran - Its good to know you like it and are using it full time. While there would be lots of people who'd like GUI frontend, I myself am more close to CLI, and it seems like we have similar interests.

Josh Glover said...

Thanks for writing flickrfs. I just added it it Gentoo Portage, so Gentooists can install it simply by running:

  sudo emerge -av flickrfs

A package entry should show up in the Gentoo Package Directory soon.

mann said...

That's great Josh, thanks :-).

rschu said...

Josh, your ebuild is not working, see http://bugs.gentoo.org/show_bug.cgi?id=189000