There are four things that I commonly need reference material for, at present: the Boost library, the wxWidgets library, the SQLite library, and the C++ Standard Template Library. The first three all have downloadable HTML reference material, so I’ve long had them on call.

The last one… there’s a lot of reference material on it, but most of it is in book form, which isn’t very convenient. The dead-tree format is the worst: I’ve got an excellent reference book, but it weighs at least twice what my laptop computer does, and it’s not very efficient to look things up in either, compared to a hyperlinked format. I’ve also got an e-book version of it, which is a lot lighter, but which still suffers in the lookup-efficiency department.

When I’m in the office, I generally use two reference sites, which I won’t name here. I generally know the name of the function I want to use, and am just looking for the proper syntax, so the hyperlinked format that they provide is perfect for quick lookups. Their reference material was not, however, available in downloadable form… until I ran across this article on Lifehacker.

I’d used wget before on occasion, to download large single files, but I’d never seen it used for anything else. The article showed me that it was useful for a lot more than that — and the thing that got my attention, that it was useful for downloading entire swathes of a site too.

It took me a while to come up with the proper invocation, because the site I was trying it on has a lot of stuff I don’t want too, including a link that appears everywhere on the site with different parameters, and which wget was downloading each time. I would have just let it, but one of the places that link appears is in its own page, recursively and with different parameters each time, and it simply never ends. Anyway, the final result looks something like this, with some names changed:

wget -m -k -X forum,userprofiles --reject='garbagelink*' http://www.example.com

The -m says to “mirror” the site to my local hard drive; the -k says to change all links to refer to the local copy. The -X forum,userprofiles tells it to ignore those directories on the site, and the --reject='garbagelink*' says to ignore the problematic link, with the asterisk indicating that it should ignore any URL containing the link regardless of what comes after it. The last piece is, of course, the site to download from.

The result: everything I wanted was pulled to my local hard drive, available for reference regardless of whether I’m connected to the ‘net or not, exactly as I’d hoped.

It’s a nice tool to have. I don’t really have any other sites I need to use it on, at present, but if I find one, I now know how to get it.

When BigCo bought it, the first thing they did was slap a new name on it and hire a guy to build a new interface for it. I wasn’t enamored of the “new name” idea, because it had a good reputation and a following of thousands of customers under its original name, but I was all for a nicer interface.

After six months, the guy had built a very pretty interface. Unfortunately, it was crap. It only really worked at 1024×768, the resolution that that developer preferred (I was stuck at 800×600 on one of my machines at that point, as many people were, and it was all but unusable at that resolution). It simplified some options to the point of uselessness. But worst of all, under the hood, the guy had taken so many shortcuts that fixing its shortcomings would have required completely rewriting it from scratch.

Many customers who tried both preferred to use the original interface, despite its ugliness — a damning indictment.

Easy isn’t always the best answer. The original program was complex, and required the person using it to study the options available and make choices; simplifying it reduced the choices and made it easier to use, but also destroyed a good portion of its usefulness. And as another facet, by taking so many “easy” shortcuts in the new interface’s design, the developer had made it all but impossible to improve it.

There are trade-offs in everything. An oak tree takes a long time to grow, but its wood is dense; a pine tree can grow quickly by not spending so much energy on its structure. Furniture made of pine is much lighter, easier to build, easier to move, and less expensive, but furniture made of oak lasts. You’ll notice that oak furniture is still around.

There’s a reason why doing something the hard way is often preferable.

It was inevitable that I would upgrade my current system (a mid-2009 model MacBook Pro) eventually, but I wanted to put it off for a while. I’ve talked before about my love/hate relationship with upgrades; they’re often a big improvement, but at the cost of a lot of unnecessary grief, and I have work to do. But Lion included some features that sounded very nice, and one that I found irresistable: new versions of FileVault (the disk-encryption software) and Time Machine (the automatic backup program) that work properly together, rather than the Snow Leopard versions where Time Machine was crippled if you used FileVault as well.

As mentioned before, I do very little on the Mac OS side of things; the only programs I generally run on it are VMware Fusion, Skype (because it doesn’t work well in virtual machines), and a to-do list program called Things. Most of my work is done in Linux or Windows virtual machines. That turned out to be a major benefit for this upgrade, because there was far less to go wrong; so long as those three programs continued to operate, my work wouldn’t be disrupted, and reports indicated that the latest versions shouldn’t have any problems that would affect me.

Despite that, I waited until Friday (yesterday), so I’d have three days to recover if anything went seriously wrong, and began.

(Of course, Friday morning a friend of mine — who I won’t name, but who comments on this blog regularly — helpfully sent me a link to a bunch of complaints that Lion caused major instability in many systems. And I’d recently seen the same complaint on The Register’s review of it, though that one was fixed by a clean reinstall. The clean reinstall route was my plan B if the system couldn’t manage an upgrade; plan C was to go back to Snow Leopard via my Time Machine backups, but I hoped I wouldn’t have to use it.)

I’d read somewhere that in order to get the FileVault benefits of Lion, you had to turn off the old FileVault and then turn on the new one, so I decided to turn off the old one before I upgraded. That required a great deal of free disk space, so I had to move about 150GB worth of virtual machines to my NAS drive (which took several hours), then decrypt it (which took several more, and I couldn’t use the machine while it worked).

That done, I followed the steps here, including uninstalling VMware Fusion (because VMware recommended reinstalling it after the upgrade anyway, for stability), fired up the Mac App Store (for the first and possibly only time), bought it, and downloaded it. The download took quite a while; it’s nearly 4GB of data, which is twice what we normally use in a full month, but we have a fairly fast connection and plenty of bandwidth allowance that we never use, so I wasn’t worried. I didn’t install it immediately; instead I made a backup of the installer and created a clean-install boot disk on an SD card, as described here. Then disconnected my Time Machine drive (just in case) and let the installer rip.

It took about 40 minutes. I sat through the last three minutes of it, then crossed my fingers while it rebooted and I logged in…

It came up without any problem.

Turning the new FileVault encryption on was a lot easier than turning the old one off — it worked in the background, and apparently encrypted everything in place, so it didn’t need all the room that the old one required. I’ve temporarily lost my third monitor, because the USB video adapter I use for it doesn’t have a stable Lion driver yet, but I knew about that ahead of time. And every hour now, if I’m in the office, Time Machine automatically backs up all the changes that I’ve made since the last backup.

Unless something goes wrong in the next week or so, I’m calling it a success.

But as an introduction to pointing out some curious facts about the world today, it works. And yes, I’m quite familiar with the fact that this is a blog post about another blogger.

Believe me, if you don’t have a backup system in place, you will regret it.

Today’s 3D printers are the direct precursor to science fiction devices like Star Trek’s replicators and the replicator-like nanotechnology device in Neal Stephenson’s The Diamond Age. It’s fascinating to think of a day when your kids can be kept occupied by an endless number and variety of toys cheaply created by your own living room 3D printer, or the structure of a house can be created in twenty-four hours with a large but fairly simple machine and almost no manpower.

The “no manpower” part also concerns me a little, but that’s for another post.

EDIT, 2011-06-02: 3D printers can now print working keys too.

I’ve written a post a day for nearly two years straight, missing only a handful of days. I like that pace, and I plan to resume it, but not until I’ve got the bulk of Project M finished. That may be a few weeks or a few months, I won’t know until I’m a lot closer to that point.

I’ll still post on occasion if I find something interesting, but I haven’t had a lot of time to keep up with the news recently. (In fact, I’ve got nearly two months’ worth of news backed up to scan through… my blog entries have been relying on a cache of articles that I’d previously bookmarked as comment-worthy, which is nearly exhausted.)

For those keeping score at home, Project M is going quite well, but ridiculously slowly. In my previous projects, I sometimes took design shortcuts that I later regretted, and rarely tested things as well as I should have. This time, I’ve committed to designing the bones of the program right, from the ground up,¹ and writing automated testing suites for any components where they make sense. And any code that I come up with that looks usefully reusable is getting put into a library of cross-platform classes and functions, with at least a minimum of documentation, which will make it much easier for me to write new programs and much easier to get others (or myself, after time away from it) productive on the code in a minimum amount of time.

It’s a lot of extra time and effort up front, but it also means far fewer bugs later, and that I shouldn’t have to go back to redesign and rewrite large portions of code in order to add new features. That means less time later that I’ll have to spend on older programs, and more time I’m free to work on new ideas, which should directly improve the bottom line in the long run.

Software developers rarely get the opportunity to think long-term and do code right. Management always wants things done yesterday, and it doesn’t matter how much of a nightmare the code is behind the scenes because neither they nor the customer will ever see it. There are cases where that’s necessary, but I believe that investing extra in the design step will pay off big in the long run. We’ll see.

¹ For instance, I’ve spent the last several weeks coming up with a correct and easy-to-use design for handling Unicode strings in various encodings. One that makes it all but impossible to confuse, for example, variables holding ASCII strings and ones holding UTF-8-encoded strings, even though the two use the same underlying storage type. And makes it trivial to convert as needed between ASCII, UTF-8 for Linux and for storing Unicode in single-byte strings, the UTF-16 used for Windows Unicode, and UTF-32/UCS-4 for single-symbol-per-character string manipulation. That kind of design work fascinates me, and the classes I’ve come up with are a true joy to use.