Monday, October 8, 2012

A family photo server

Earlier this year I asked for suggestions on Google+ about dealing with increasingly large image collections. In our house, we have two DSLRs, four phones that take pictures, and two point and shoot cameras. The images from these are scattered across several hard drives and online backup accounts; over the past several years they've been inconsistently backed up. We have a network attached storage device that houses all images, but due to poor backup processes in the past, we have several cases of duplicate images.

Adding to the complexity, we paid Scandigital to scan years of print photos – everything from our honeymoon to our first cross-country drive to our first house. This added several thousand images to our archive – a good thing, to be sure, as we now had electronic copies of pictures we hadn't looked at in years. But the challenge of managing those images – now numbering close to 50,000 – was getting insurmountable.

I hadn't gotten around to actually implementing a solution – we had a busy summer and I wasn't convinced I really wanted to tackle this. Then my son had a school assignment last week requiring him to find a dozen pictures to share with classmates from his childhood... and actually finding pictures for him was a nightmare. After more than an hour of poking through our archive, we hadn't found more than 5 he was happy with. I was frustrated, he was annoyed, and it became clear this wasn't sustainable. It was time to dive in.

The solution I more or less settled on was what I documented this summer: dedicate one desktop computer to organizing the image catalog. This past weekend, I picked up a computer at Best Buy, and I'm already happy with the progress (though I expect it'll be months before I will feel like I'm done). Here's what I did:

The computer

Bought a Gateway desktop with a dual core Intel processor and a 1TB hard drive. Total cost? $350. (Twenty years-ago me will stare at that line for a long, long time. It's OK, me-from-the-past; computers are commodities but gas is now $5/gallon.) It features an HDMI port, so I parked the computer behind the television in our family room and plugged it into one of the TV's available HDMI connections; when I want to display photo albums easily, I can just pull them up on the TV. (Note: when I first connected the computer to the TV, the computer's display extended beyond the boundaries of the screen. This blog post helped me figure out the problem: I had to adjust the TV's settings to stop zooming in; once I did that, I was all set.)

I added a Logitech wireless keyboard to the computer so I could operate the computer from the couch; it includes an integrated trackpad, and so far I'm pretty happy with it.

The images

Copied over all of the images from the NAS drive to the PC. Installed Picasa, and let it find all of the pictures. All told, there are slightly over 50,000 images taking up 200 gigs of disk space (I think there might be more, actually, but I haven't finished confirming that everything made it over yet). Thanks to the fast processor, indexing these images took Picasa just a few minutes; last time I tried this with a laptop it took hours and didn't complete. Hardware matters!

The faces

This is where it started getting magical: after just a couple hours, Picasa had found thousands of faces across our images, and grouped them very accurately. All of a sudden, I could see photos of my six year-old daughter, from her birth to this past summer vacation. There's my twelve year-old son – at his third birthday party, on his first day of kindergarten, leaving for his first overnight Scout camp – in one place. And my ten year-old son – the day he was born, his first airplane ride, the day he learned to ride a two-wheeler. It wasn't just the kids: my wife and I are there too, as are the grandparents (including my grandparents, both of whom have died), extended family, and friends.

What's next

Like I said, I'm nowhere near done. This is a solid foundation, but I have a long way to go. Here's what I think I need to do to get this under control:

  • De-dupe the catalog. Picasa has a nice "show duplicates" feature, but since it shows both copies of the picture that's duplicated, removing the dupes while leaving one copy is a time-consuming affair. This article from Digital Inspiration looks like it'll help; according to Picasa I have more than 4,000 duplicates.
  • Confirm I have all the pictures. I haven't done a full audit of where all of the family's pictures are hiding; in my Picasa account, in my wife's, on the kids' SD cards, etc.
  • Simplify synchronization from those sources. Once I have all of the images, the next step is to ensure that going forward the new images will get included in the master Picasa collection. Crashplan on the Mac will likely satisfy this for both my wife and I; I'm looking into solutions for Android (Dropbox with its instant-upload option may be a good go-between here, though I haven't started looking at how best to do this across several devices).
  • Install VNC on the photo server. While I'm able to operate the computer from the couch, that's not the most useful way to do actual work. It's great for lean-back viewing of the pictures, but doing lots of manipulation can get tedious. I'm going to install VNC so that I can access the computer from my laptop when I'm at home, which should make it easier to do the heavy lifting when needed.
  • Turn on cloud sync. I've got a lot of unused disk space on my Google Drive account, so once I have the local catalog in a good place, I'm going to enable Picasa's cloud synchronization, which will not only give me reliable backup of all images, it'll also give me an easy way to share all of these images. For the most part that means sharing with my wife, but I'll probably also share with family who may like the ability to browse through all of our images.
PS: The assignment

Even without all of that yet-to-be-done work completed, when it came time to find pictures for my son's assignment, it took all of about 10 minutes. The combination of a fast computer, large display, reliable face-tagging, and simple interface meant that we were able to very quickly find a handful of pictures from nearly 10 years in a matter of minutes. I'm encouraged, and feel like I've got a pretty good path forward.

9 comments:

  1. Quite a relevant posting (again)!.. Thanks!
    I wonder if/when/(how) Lightroom will add the face recognition I'm sorely missing . I suppose I could just dump lowres versions to a Picasa folder and somehow get the results written back as tags once grouped. Hmmmm.

    ReplyDelete
  2. I don't know that I could convince myself to use a windows machine and Picassa instead of a Mac and iPhoto, even if it seemed to make more sense. I feel like my future in personal computing is going to be tied very closely to Apple's operating systems, and it just feels risky to allow part of my system to exist outside of that..

    I'll be staying tuned to your exploits, though. This is a problem that needs solving.

    ReplyDelete
  3. @Erik: I replied on Twitter too, but for consistency: AFAIK, that doesn't work for multiple users, multiple devices, multiple computers. And what do you use for iPhoto backup and multi-user access?

    ReplyDelete
  4. Rick, great post, as usual. I'm curious as to your use of Lightroom now. I know you previously moved from Picasa to Lightroom. Have you given up on the Lightroom idea? If not, where does that fit in?

    Thanks...John

    ReplyDelete
  5. @John - Thanks! Still use Lightroom for photo processing; I will likely export processed photos to an album that gets absorbed into the master catalog, and may or may not independently store all photos (as opposed to just the processed 'keepers') on the NAS drive.

    ReplyDelete
  6. Rick, I was wondering if you would ever tell someone you didn't know a really great idea without having a conversation with them first?

    ReplyDelete
  7. I've been after doing something like this for years, this post is very encouraging. I've a daughter who is now 15, she had a similar assignment to your son, from her school requesting finding some old photos. Thanks Rick :)

    ReplyDelete
  8. @rklau, I have been wanting to do this for years and will use your article as a starting place to get this project off of the ground. My preference would be for a cloud solution so that I can stream this from anywhere to any device, but the cost for that much online space is prohibitive at this point so local might be the way to go in the interim. Again, thanks for an informative article.

    ReplyDelete