waves in the aether

The holidays are coming and my wonderful wife, Adrienne, has started thinking about gifts for the family. One of the problems we have is that our entire life is digital, but it is not a promiscuous one. No facebook, flickr, twitter or any of the other numerous places on the net that give you a bit of convenience in return for turning you into a product. We are digital homebodies.

One of the inconvenient consequences of being a digital homebody is that grandparents do not have access to photos of their grand children. None of those services would allow us to share videos and photos with the analog generation either. So Adrienne decided to start creating a photobook on lulu.

She spent some time culling our giant photo library down to only about 80 images then tried to upload them to lulu. Only 40 of them made it.

Many years ago when I was a poor student wandering the country I was crashing at a friends house. She was trying to change a light bulb, with her room mate, in a chandelier (technically an electrolier) that was not built to have its light bulbs changed. Ladders, chairs, tables, screwdrivers, hammers and loud swearing were all part of this elaborate process. I was leaning against a wall watching with great amusement as this scene unfolded before me. Being simultaneously relieved and embarrassed at my lack of participation, I was befuddled when my friend thanked me for not interfering. Indicating that was an unusual but welcome trait.

The extreme juxtaposition between her perception of the situation and mine made that stay with me. Not only because it has often been my job to interfere when people are having problems, but because like so many geeks it is in my nature to solve problems wherever they occur.

Combine the accidental lesson learned so many years ago with the inherently tense situation caused when trying to help a frustrated person you can perhaps understand the trepidation I felt when I offered assistance to my wife.

Lulu’s book creation software is a giant flash monstrosity. As such it does not perform well on under powered ‘eco’ laptops. Another side effect is that all UI widgets must be non-native and enter into the uncanny valley. This combination makes all flash apps cumbersome at best and unusable at worst, lulu is no exception.

After trying to upload the images a couple times, which created duplicates of the same 40 I wondered if the images were corrupt. Investigation demonstrated that numerous image viewer apps could load them just fine without issue. Trying another book service, blurb, had the same problem, the same images did not get uploaded. What the hell was going on?

I can’t remember the exact moment I saw this, the file dialog box had ‘*.jpg, *.jpeg, *.png’ as a file filter. Inspecting the files we were trying to upload I noticed the following extensions, ‘*.jpg’, ‘*.JPG’. Spot the problem?

No?

It took me a while. Longer than I care to admit.

The filesystem on my wife’s laptop is case sensitive. Unlike FAT32 where jpg is the same as JPG, jPg, Jpg, etc. jpg is NOT the same as JPG. In fact they are fundamentally different extensions.

With numerous cell phones and digital cameras all creating various file naming conventions one of them creates JPG instead of jpg. I have yet to track down the culprit.

Now that I have found the reason why half the images were not uploading. How to solve this? I couldn’t imagine going through 40+ files and renaming them individually. Being a professional geek I pulled out my trusty bash shell.

Being the good geek I write my bash one liners incrementally, so the first attempt is a simple loop:

for f in `find -name '*.JPG'`; do echo $f; done

As any seasoned bash scripter can tell, there is only one thing that could have made this approach untenable, spaces. Yes, spaces. For time immemorial I and every person who has ever written a shell script has despised spaces in file names. There are ways to deal with this, but it was approaching 11pm, I was tired and I just wanted to get this done.

For the past couple of weeks I had been steeped neck deep in erlang. While erlang is probably the last language one should think of when trying to solve this problem, I couldn’t help it. In less than a minute I whipped up the following:

#!/usr/bin/env escript
%%! -noshell -noinput
main(_) ->
    filelib:fold_files(".",".*\.JPG$", false,
                       fun(F,A) ->
                               Base = filename:basename(F),
                               Ext = filename:extension(F),
                               file:rename(F,Base++Ext)
                                   end,[]).

This worked!

It does have one minor bug that did not affect the desired outcome, e.g. changing extension from JPG to jpg. I will leave this as an exercise for the reader to find the bug.

When all 80 of the images were finally uploaded to lulu, my wife asked a very reasonable question: “And I was suppose to do that how?”

A very reasonable question.

• you would first have to understand the quirks of whatever file system you are using.
• you would have to notice that the file selection dialog box was not giving you the ‘All Files’ option.
• you would then have to be able and/or willing to rename 40 files

I have been lucky enough to spend the last ten years creating and solving these problems. Since these problems are so prolific there is a pattern recognition machine in my head that can wade through the confusing errors and silent failures.

Software is incomprehensibly complex. No one has figured out how to manage that complexity yet. There are best practices, tools and methodologies available to mitigate the failures caused by software’s inherent complexity. However, they are all just that, a mitigation not a solution.

Given software’s predilection towards failure and its near ubiquity in modern life I am left to wonder how non-developers cope. My heart truly goes out to them.