Open source project: zmugfs

In April 2007, I bought my wife a digital SLR for her birthday. I also took this chance to install Fedora 7 on her computer. I am familiar with using digital cameras under Linux and had no desire to figure it out on Windows. Within a couple hours, Fedora 7 was installed, and I was able to download images from the camera to her computer. We tried several photo management programs like gthumb and f-spot. We stuck with gthumb, using the standard directories, as this was more natural for how we worked.

We use smugmug.com to host our photos. It is a subscription-based site geared towards professional photographers. We chose smugmug because it offered unlimited storage, backup DVDs of all our photographs, and their excellent API.

On a warm day in August, I observed my wife uploading picture to smugmug.com. The upload screen is a basic file interface with text boxes and browse buttons that allow users to process up to 10 images at a time. This can be quite cumbersome if an average photo shoot yields 200 or more pictures.

I noticed she had Nautilus open so she could choose which photos to upload, then used the browse button to find the image’s filename. I asked her, “Wouldn’t it be awesome if you could select the photos you wanted to upload in Nautilus and drag them to another folder all at once?”

That’s when I truly saw the light bulb turn on over my head. I could write a FUSE-based filesystem that looked to the user like a regular file window… but was actually able to communicate to smugmug.com via their API.

On August 19th, 2007, I announced my project on my blog.

I remember hearing about FUSE, Filesystem in Userspace . FUSE is an open source project that allows one to create virtual filesystems for use by non-privileged users without having to recompile the kernel.

There are many projects that use FUSE today including zmugfs, EncFS, flickrfs, and many others. What makes FUSE so attractive is that the API is simple. It offers a low barrier-to-entry, as well as numerous language bindings. It works with popular languages such as C, Python, and Java as well as with more obscure ones like Haskell, Erlang, and Lua.

Before implementing zmugfs, I searched for FUSE based filesystems for smugmug. I found none, but I found two others, one for flickr and one for gmail, both written in Python. After 10 years of writing Java, I wanted to learn a new language, and Python fit the bill. I needed a language that was easy to understand, fast to develop with, and well supported by FUSE.

After working in the evenings for 3 months, I released zmugfs 0.1 on October 31, 2007.

How does it work?

Zmugfs consists of the FUSE-based filesystem (zmugfs.py), some configuration files (zmugfsrc), and a library wrapper for the smugmug.com Javascript Object Notation (JSON) API (zmugjson).

The FUSE functionality is all captured within zmugfs.py. You extend a class called Fuse from the python-fuse library, implementing the methods you need to support the operations you wish to support. For my first release, I planned read-only support of the galleries and photos.

The categories or albums on smugmug are mapped to directories in zmugfs, while photos are mapped to files. I implemented 5 methods: getattr(), opendir(), open(), read() and release(). The bulk of the work was in the JSON API wrapper.

Zmugjson uses simplejson to parse the JSON responses into dictionaries and objects. It also creates a server proxy in a way similar to how the Python xmlrpc library works. It allows dynamic methods to be called; for instance, to call the method smugmug.user.getDetails you can say:

client.smugmug.user.getDetails()

Zmugjson will then generate the proper URL to communicate with smugmug. To avoid exposing too many of the API details to the callers, zmugjson attempts to encapsulate the details with a smugmug wrapper class that handles the gory details of parsing the JSON response and dealing with status codes. This approach made it much easier to write zmugfs.

        sm = zmugjson.Smugmug(apikey)
        sessionid = sm.loginWithPassword(self._config['smugmug.username'],
                                         self._config['smugmug.password'])
        tree = sm.getTree(sessionid, 1)

The final piece of zmugfs is its configuration file. By default, the configuration file is stored in $HOME/.zmugfs/zmugfsrc. It contains 4 entries:

smugmug.username=
smugmug.password=[plain text]
# specify image.disk.cache using megabytes
# 100MB ~ 50 images
image.disk.cache=100
image.memory.cache=5

The image.*.cache controls how much disk and memory to consume when downloading image data. The smugmug.* configuration entries are simply your smugmug.com account information and plain text password.

Installation

The simplest way to install zmugfs is using the pre-made rpms which can be found at Sourceforge.

Download the zmugfs and zmugjson rpms, then install them using this command:

sudo yum install –nogpgcheck zmugfs-0.1-2.fc7.noarch.rpm zmugjson-0.1-1.fc7.noarch.rpm

Yum will take care of the installation, including checking any dependencies. Once installed, add the user that will be using zmugfs to the fuse group:

sudo /usr/sbin/usermod -a -Gfuse zmuguser

Now run /usr/bin/zmugfs [mount directory] where “mount directory” is the directory where you want to store your smugmug albums locally. At this point, you should be able to browse your albums and photos using Nautilus or the command line.

As you can see, zmugfs allows you to treat your online smugmug.com albums and photos as if they were local to your machine.

Future versions of zmugfs will allow you to create and upload your images, but version 0.1 is limited to read-only commands.

Got a open source project you’d like to introduce to the world? Tell us about it.

This entry was posted by Jesus M. Rodriguez on Tuesday, January 29th, 2008 at 5:47 pm and is filed under Fedora, technical. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.