Sub Ubi

The last few weeks have been… action packed. This would seem to be as good a description as any.

The shipping pod arrived on Tuesday, and we had it packed by dinner time.

On Wednesday, the pod went away, and so did we. What followed was an excessive amount of driving over the next few days that left us wondering just how many weeks (so it seemed) that we had been in transit from our comfy basement in Needham. We drove from Boston to the Atlantic Highlands, NJ, then up to Rosseau, Ontario, down to Waterloo, and finally to Columbia Station, Ohio.

Our first destination was a trip to northern New Jersey to be with family. I got a very nice picture of Carrie while we were all gathered together.

We then traveled on to Rosseau, Ontario to see many friends and bear witness to the Christening of Dave and Teena’s new daughter. While leaving at 05:30 did mean that we’d miss traffic north into New York city, we failed to consider that it would mean we’d hit heavy traffic at the border (people from Buffalo apparently like to go up into Canada for the weekend), and we didn’t know that Toronto at rush hour is absolutely foul. Our drive took 17 hours, which is only 1.7 times longer than Google said it would take. We no longer have any reason to ever want to drive into Toronto again.

We had one day “off” on Saturday, and on that day Dave and I went sailing in a small sailboat. This was my first time sailing, and it was a lot of fun. Dave smashed both of his big toes, and as soon as we launched off the dock, we were dumped in the lake and lost our tiller. Dave re-mounted the tiller in the wind-whipped chop, we climbed back in, and successfully tacked along the shore to our picnic destination. Of course, the picnic site was in a sheltered area, so we just dumped ourselves from the boat again to park it… or whatever you call the destination-management process with little sailboats.

Sunday saw us driving down to visit Robin and Meg and the newest member of their family. Monday we then embarked on the final leg of the journey, which took us from Waterloo back to my parents’ house in the Cleveland area. All told, it was 1400 miles of driving in six days.

At this point, our little Toyota Echo (which was getting upwards of 40 MPG on the highway! hooray!) is now in the shop. On the drive, it developed a bit of a growl in the front end, and it looks like both the driver- and passenger-side wheel bearings want replacing. Badly. And, the drive shafts are probably going to go, too, just because it’s a good idea, as they’re tired too, and they had to be removed as part of the operation anyway.

Today, the pod arrived, and we unpacked it. Now, we reserve a truck, and hopefully conduct a move at the end of the month.

We have two trips to make out to Meadville before we move, so there’s still more driving to be done. All told, I’d say that buying a house and making a move in under two months is a reasonably distracting and time-consuming thing, and fitting ceremonies of both death and birth into one week can be a bit draining. But we’re home now, and there’s ice cream to be had.

PS. If you’re a friend or family, and you’d like to be notified when I post new pictures to the photo galleries I’ve set up, please drop me an email and let me know. My photos are password protected (to keep them off the Internets at large), but they’re not terribly top secret. The galleries are located at http://sububi.smugmug.com/, a photo hosting/sharing site that I’ve decided to experiment with for the next year.

PPS. If you’re of the European persuasion, here’s a route of equivalent distance in the UK. It takes you from Canterbury, up around London, to Cambridge, Birmingham, down to Oxford, Reading, Bristol, down through Exeter and Plymouth, up and around to Southampton, up to Bath and over to Cardiff, up along the Welsh coast through Aberystwyth to Bangor, through Colwyn Bay on to Crew, up to Liverpool, on to Manchester and Leeds, up along the east coast to Edinburgh, over to Glasgow, and then ending on the coast (because we like the seashore) in Prestwick. Admittedly, it would be much prettier from Leeds to go up through the Lake District; it comes out roughly the same, but we’d end on the east coast of Scotland, which is rather striking.

I have no idea what I’m doing with my camera.

Robin points out that I should put the thing on P, ISO 100, and go outside and learn how to compose photos. I’ll get around to this. For the moment, I’ve had the camera for about two days, and I’ve had limited opportunity to use it. I did take it to dinner last night; a bunch (like, twenty) of Olin students gathered for dinner at Vinnie T’s, and I joined. It was tasty, and it was fun to see so many people I knew.

What amazed me about shooting at 8MP RAW was what could be done after the picture was taken. Vinnie T’s was very orange. Everything had a yellow/orange/red cast to it.

Last night, I downloaded a beta of Adobe’s Lightroom 2.0. This is kinda like iPhoto on steroids. The above picture was taken by Nik while he was poking around with the camera; nothing he did on the shot could have compensated for the lighting in the restaurant. (At least, I don’t think you can easily compensate for that kind of lighting at the moment that you want to take the picture.)

After playing with the hue, saturation, and luminosity across the entire color space, I came up with something vaguely human looking:

To me, this kind of technology is freaking magic. Of course, I have no idea what I’m actually doing. I mean, I have a rough understanding of color theory, and can make some sense of what the color curves in Lightroom mean, but mostly, I’m just looking at the photo, looking at the sliders, and moving them to reflect where I think I want that particular color to “go.” I assume someone who really knows what they’re doing could look at the “before” picture and see a path through the color space to a more natural coloration—but I don’t have that ability at this point.

Back to packing.

I’ve started using my del.icio.us bookmarks regularly now. I’m pretty sure there’s only two or three people reading this (hi mom!), but I thought I’d mention it just the same.

In particular, I’ve made a point of using the notes field. Every time I post a bookmark, I do write a bit about why I’m posting it. This is partially to improve my own searching of the links later as well as to make it clear to anyone following along as to why I’m posting it.

In short, it’s a link blog.

So, if you’re interested in the things I’m bookmarking, you can grab the RSS feed for my del.icio.us bookmarks on the right hand side of the page. Mom, since I don’t think you’re using a news reader yet, this probably doesn’t apply to you. But then, you’ve surprised me before…

you say “no thanks.”

http://www.youtube.com/watch?v=eELH0ivexKA

A rap about tea. Rather good, actually. I mean, in a silly sort of way. From Rick, by way of kent-fun.

I’ve been slowly making progress on a new server for Untyped, and moving sububi.org is part of that process. Because the domain only makes use of the web, it’s a “lightweight” domain. However, it forced me to get a few things updated with Apache and MySQL.

So, it seems like the blog has successfully moved. I’ll worry about themes and other niceness later. For now, infrastructure is continuing to fall into place, which makes me happy.

In related news, I received my fit-PC, and will be getting our backup server running 24/7 momentarily. That will get its own post in due course. So, all of our servers will have a geographically isolated backup, but not quite yet a RAIDed backup. One step at a time.

I’ve switched almost entirely to mobile phones for communications; we haven’t had a landline all year, and don’t expect to this next year, either.

The most problematic thing about this would be the 2W dissipated by the phone’s cellular radio. That, and handsfree units are expensive—anywhere from $20 to $100 or more.

An expensive headset.

Then I remembered eBay! I was able to buy three handsfree cables for our phones as well as two for my father’s phone for less than the cost of a single Bluetooth headset. Granted, they’re cables as opposed to wireless cyborg implants, but frankly, I don’t care. At least the cable never needs to be recharged…

So, in the midst of travel, taxes, and many other exciting things, I lost my hard drive two weeks ago. And not only that, but it was a surprise, overnight failure. Friday night it worked, Saturday it was dead, and I had no warning.

So, be warned, and backup! You can loose your HD at any time, without warning!

It turns out that my MacBook restore CD fails to boot, and as a result, I’ve installed Ubuntu 7.04 “Feisty Fawn” on my laptop. It works, and lets me do what I need to do… email and some writing, mostly.

WARNING: This document assumes you are familiar with installing software, the notion of remote servers, and have some experience mounting remote servers and storage facilities for archiving or otherwise backing up your data. If any of that sounds completely bonkers to you, then it is possible the rest of this document will, too.

Amazon does more than sell books. Perhaps their most valuable assets are the information regarding the trends of book buyers. Reselling access to this information is one way Amazon makes money off the fact that you like to buy books from them. That said, Amazon clearly needs lots of computers and disk drives to store all the data they have regarding books, your interactions, and so on. Given that they have already invested in this infrastructure (in the form of hardware, people to manage that hardware, etc.), they have also begun to sell access to the very infrastructure they use to run their business.

For the busy worker bee, there is one Amazon service that should be very interesting: the Amazon Simple Storage Service (S3). In the simplest possible terms, Amazon is selling disk space. Now, they’re not selling 1GB of disk space, or 10GB… they’re selling you as much disk space as you would like to use. That might be a few megabytes, or it might be a few gigabytes. That’s important, because you only pay for what you use.

Pricing

Amazon has set a very simple pricing structure: you pay for the transfer of data to their servers, you pay to copy data off of their servers, and you pay to leave data on their disks. Their prices are per gigabyte. To copy 1 gigabyte of data to Amazon, you pay $0.20. To copy that same 1GB back from Amazon, they’ll charge you another $0.20. For every month that you leave that 1GB on their disks, they’ll charge you $0.15.

To put these numbers in perspective, I’m going to give you some concrete examples:

• A 128MB memory stick

  Lets say you have a 128MB memory stick. It can hold (perhaps) a book you might be editing, or a dissertation you are writing, or whatever. Copying that entire 128MB to Amazon’s S3 service would cost you (0.128 * 0.20) cents, and it would cost you (0.128 * 0.15) cents per month to store it. That’s 3 cents to copy the data to their server, and 2 cents per month to store the data.

• A CD

  A CD holds roughly 700MB of data. That would cost (0.7 * 0.20) cents to copy to Amazon’s servers, and (0.7 * 0.15) cents per month to store. Or, 14 cents to copy an entire CD to Amazon’s server, and 10 cents per month to store it.

• A DVD

  I include a DVD worth of data only because you might want to store photos on Amazon’s servers. A DVD is roughly 5GB. This means it costs $1.00 to copy a DVD to S3, and it costs $0.75 per month to keep the pictures there.

I’ve beaten the horse quite badly, I’m afraid. But I think my point is made: this is the cheapest way to back up your data. I will belabor this point just a bit. If you go out and buy a hard drive, it will be cheaper. However, that hard drive will be in your house. If your house burns down, the drive is destroyed. If your house is burgled, the drive is gone. If the drive fails, you loose the data. So, now you need two drives… in two places. Your time is too valuable to be copying data from one drive to another, and driving it back and forth between home and your safe-deposit box in the bank. Given that Amazon already has multiple, redundant servers in multiple geographic locations (your data is in more than one place)… you simply cannot go wrong with Amazon’s pricing and services.

How to use it

If you’re still reading, and you agree with me, then you’d probably like to know how to take advantage of Amazon’s services. There are a number of ways, some more realistic than others.

• Scripting

  Technically, the service is open, so you could write your own software (in Perl, Python, Scheme, or any other language) to access Amazon’s S3 service. While you might not, it is good to know that others can. This is not a closed service. It is privately owned, but anyone in the world can write software to interact with their servers.

• “FTP”

  It is possible to use the program Interarchy to transfer files between your computer and the Amazon S3 service. I say “FTP” because it isn’t technically FTP. However, Interarchy makes it look like FTP, and works reasonably well. Interarchy costs around $50.

• JungleDisk

  At the moment, this last option is free, but may cost money in the future. I’ll spend more time describing it below.

If you think that this might be the way for you to cheaply backup data, photos, and the like, then proceed. Keep in mind that, as time goes on, more and more people will be reselling Amazon’s services. This means, for example, that someone will come out with a simple way for you to do all the things I’m describing here, charge you a bit more than Amazon, and make a fortune. However, that ease of use (someday, who knows when) might be worth the extra pennies.

In the meantime, if you want to make use of Amazon’s S3 service yourself, you’ll need to take a few steps yourself.

1. Register with Amazon

   You first need to register for the S3 service with Amazon. You’ll need to create a Web Services account, and in particular, activate the S3 service. This shouldn’t be too impossible a task—if you do it, and decide it was very hard, please drop me a note, and I’ll expand this section.

   You will need a credit card handy at this point, as Amazon wants a way to bill you every month for services rendered. Remember, if you register, and don’t use it, you won’t be billed… so you are simply promising to pay for the service you use, not some flat rate, etc.

2. Obtain your Access and Secret Keys

   When you are subscribed, you’ll be able to view your Access Key and Secret Keys for the S3 service. FOR THE LOVE OF ALL THINGS GOOD, DON’T LOOSE THESE OR GIVE THEM AWAY. Really. Don’t send it in an email to a friend in plain text, don’t keep it in plain text on your laptop… if you want your data on Amazon’s servers to even be partially secure, don’t loose or release that key into the wild. Really. Honest.

3. Download JungleDisk

   Now download JungleDisk for your platform (Windows or Mac). After downloading, install.

   NOTE: This program is currently free, but might become non-free sometime in the future. Be prepared, that if you like this solution to backing things up, you may have to pay for the software. (Personally, I’m all about supporting independent software developers, so if the author wants $20 or so for his software, and it makes my life better, great.)

4. Follow the instructions

   There are instructions to help you out in terms of installation and use. Read them.

Once you have JungleDisk installed and working, you should have a new drive icon somewhere in your system. Under windows, you might have a new drive mapped in the Explorer; under Mac OS X, it mounts it as a new network drive (much like .Mac or other remote servers).

NOTE: Security? You can enable security in JungleDisk, and it will encrypt your data before shipping it to Amazon. This is not terribly robust, in the long run, but it is good enough that someone snooping the data in-between cannot read the data. I recommend you enable this, even for the casual user. However, in doing so, MAKE SURE YOUR SECRET KEY IS SAFE. If you loose it, you won’t be able to decrypt the data.

Backup for Brainiacs

Note: this section is Mac specific. I don’t own a Windows computer, and I don’t intend to ever own one again. If you need help, at this point, with Windows-specific concerns, I’ll happily obtain a machine and do the work to help you… but I will charge you significant amounts of money in consulting and training fees.

At this point, you have a lot of options with respect to how you use your S3 account.

One thing you can do is simply drag files and folders onto the JungleDisk manually. By doing this, it will copy them to Amazon’s servers. This is OK; I mean, you get to selectively decide where things go, and what to back up. But what most people don’t know is that irregular, manual backups are usually no good. What you want is something mostly automatic, so you can (with the press of a button) backup everything that matters to you.

What I recommend is that you investigate one of three programs:

1. PsyncX, David Baker

   PsyncX is free/open-source software. It builds on UNIX tools for scheduling and performing backups. It will allow you to choose source and destination folders (like MimMac), and run them (potentially) on a regular basis. For example, you can have PsyncX automatically back up your important documents every night. The nice thing is, it will synchronize, so if a file hasn’t changed, it won’t copy it. Again, you’ll need to investigate it more, on your own, to decide if this is for you.

2. MimMac, Ascendant Softworks

   I’ve not used MimMac before, but it is in the space of programs that you might consider. It lets you define source folders and destination folders, and it will backup or synchronize those folders for you. You’ll need to read more about it specifically to see if it works for you.

The point of programs like these is that you should be able to (1) press a single button, and (2) have one or more directories on your computer automagically copied to your your S3 account by way of JungleDisk. Software like PsyncX will make sure that the version it is copying is newer (or modified) than the version on your S3 account, and only copy things over if they have chagned. This way, you don’t have to go digging and say “What files did I change today?” With almost 100% certainty, you’d miss one.

There are other backup programs out there. You may find the ones I’ve listed are not to your liking, or you might not like the idea of backup at all. However, if your backups are not (a) one-touch, or (b) automatic, they’re not good. Or, put another way, they’re unreliable, because you are unreliable. I know, I know, you have a mind like a steel trap… but trust me, your trap might become rusted, and then you can’t be trusted.

That’s all, folks

I’ve moved quickly through some bits, slowly through others, but in all I’ve tried to cover things end-to-end. Good backups are hard to come by, and this is something that, if it matters to you, you’ll need to work at. I’ve spent hours working on a robust backup strategy for a server I help manage, and I know that every hour I spend on that backup strategy is ten hours saved in recovery. In fact, some data I have can’t be recovered, no matter how hard I work. Photos? Old documents? Email? These things need to be backed up regularly, off-site, and with care if they’re going to be available to you after machine theft or failure.

Please feel free to leave comments and feedback below; I’ll evolve this note as people point out flaws, omissions, or desirable additions.

It’s always cool when you wake up in the morning and discover that your backpack (containing your Powerbook, iPod, and passport) and your wife’s iBook and iPod, are gone.

But, no one in the house was harmed, and life goes on. I could be much worse off; because of a bit of a scare earlier in the week, I backed up the entire hard drive. As in the entire drive, bit-for-bit. And I have a complete image of Carrie’s machine from no more than two or three weeks ago.

Now, I’ve had to cancel every single card I have in my possession, and am proceeding to change every single password I have, but that’s OK. Inconvenience I can live with.

Hopefully, the insurance peeps will hook us up. That phone call will happen Monday.

This video at YouTube is a really sweet juggling segment by Chris Bliss, recorded at Pickwick and Frolick in Cleveland.

Sweetness.