Geek Drivel

Anything that helps deal with distasteful chores is welcome.

Last Friday, I discovered that I needed a 64-bit version of Windows for development purposes. I had a 64-bit version of Windows XP on a spare machine, but I haven’t set it up since the recent move, and until we get more stuff organized, I frankly don’t have the room. But it occurred to me that I’m running a 64-bit machine now, so at least in theory, I should be able to install a 64-bit version of Windows in a virtual machine under VMware Fusion. So I figure, what the heck, let’s give it a try. And while I’m at it, I might as well finally take a look at Windows 7 too.

MSDN supplied a 64-bit copy of Windows 7 “Ultimate”, and a key for it. When I created a virtual machine for it, VMware Fusion obligingly recognized the downloaded ISO file and installed it for me with no further instruction on my part. After the obligatory round of updates and reboots, I settled down to figure out how to use it.

In any new Windows OS, I have to figure out how to change a few things immediately. The “don’t show hidden files” option has to come off, and especially the “hide extensions on known files types” option — I’m a big boy, I can stand seeing file extensions and files that Microsoft thinks are too dangerous or scary for delicate eyes. I do not want it to automatically reboot if it crashes — I want to see that it has crashed, and any diagnostics that it produces, before I (manually!) reboot it. And turn off those damned animations, I don’t care about them and you’re running in a virtual machine so don’t waste the CPU cycles.

Since Microsoft always moves the settings around between versions, I had to poke around awhile to find them, and I got a good look at the OS as I did so. And I have to say, I’m impressed. The user interface is a lot better than it was before — easier to use, easier to navigate around, and it mostly stays out of your way until you need it. The system is pretty responsive, something that you couldn’t always say about earlier incarnations. It’s using roughly half of the one gigabyte of memory that VMware Fusion allocated to it, leaving half a gigabyte of memory for programs to use without swapping. Even the security stuff didn’t cause me much trouble, I just had to tell it that it was okay to run programs from a network drive without panicking and warning me about them each time.

But it wasn’t until I installed Visual Studio that I really saw the biggest difference. All of my programs compiled 33% to 50% faster! I’d anticipated that the compiler would be a little faster under a 64-bit operating system, despite the fact that it’s only a 32-bit program itself, but that’s a lot more than I’d expected!

All told, it was one of the most painless Windows installs I’ve ever done, and it didn’t take me too long to decide that this was going to be my new development VM. I did run into one minor but baffling glitch, which I still haven’t determined the cause of (could be Windows 7, could be Visual Studio 2005, might even be something else entirely), but I found a work-around for it, so everything is working fine now.

I’m still not going to use Windows for anything Internet-related, only for games and software development. Linux is a lot safer on the ‘net at present. But for the foreseeable future, when I use Windows, I’ll definitely be using Windows 7.

During my most recent software development project, which I’ve already blogged about extensively, I finally learned to use a version control system (VCS from here on) right.

To fully understand this, you have to know where I was coming from. I learned to program in the early-to-mid eighties, and never even heard of a VCS until about 1998, when I picked up a copy of Visual C++ 6.0. I couldn’t see much use for it — I was archiving a zip-file copy of the source code for my software at every version release, and it seemed to me that VCS did the exact same thing.

At that point, I viewed the “working copy” of the source code as sacred. I’d quickly learned that whenever I made a change to it, I had to be able to reverse it if it didn’t work out, so for small changes, old code was just commented out and left there — that, and the zipped-up backups, were my VCS. It made for some messy code, because I rarely went back to remove the old commented-out stuff, but it worked.

When I sold my earlier software company in 2003 and went to work for the new company for a while, I was shocked at just how casually developers at the new company treated source code. They made changes to it willy-nilly, without leaving the old code intact! What if they screwed something up? The five-minute introduction to CVS that I got explained how to check in and check out source code, but nothing about the big picture of how or why to use it. To me, it was still just a glorified backup program.

Fast forward to a few weeks ago, when I started making some extreme source code changes to a large twelve-year-old project. That’s when I started really learning how to use version control.

The first lesson: the working copy is temporary. The sacred version of the source code is the one in the version-controlled repository, and if you’re using it properly, you can always fall back on it if your changes have introduced problems. You can also use it on multiple machines if necessary, with no problems, the source of a future compile-speed-related blog entry.

There’s a corollary to this: check in early, check in often. Make small and focused changes, get them working, and check them in as soon as they’re done. That way, if a change introduces a bug, you can easily locate the exact source-code alterations that caused it. And if all else fails, you can easily throw it away and revert to the last checked-in copy without losing valuable work. If I’d known this a few weeks ago, and practiced it, I could have saved several days of debugging huge check-ins when they later turned out to have problems.

And finally, if you have to make large, sweeping changes that must all be completed before you can see whether they’ll work, make a separate branch until it’s finished. Then you can keep checking in the smaller changes as they’re completed, even though they haven’t been tested, and all the check-ins on the main branch are known to always contain good working copies of the program. With a modern version-control system like Git, you can easily merge the branch into the main code once it’s done.

Now that I see the big picture, VCSs make a lot of sense. I’ll be using mine a lot more in the future, and quite differently.

I never needed to be convinced of this, but it’s nice to know nonetheless.

I’ve spent the last few days integrating my new math library into the Windows project I started coding it for. Yesterday morning I was ready to try running the integrated copy, but I had some kind of problem starting it up. I couldn’t track it down very easily though, due to how long it took to do a full compile of that project.

As you might remember, I’m now using a MacBook Pro, and doing Windows development under VMware Fusion. I expected that things would be a little slower under VMware Fusion than on a bare-metal machine, and for the most part I didn’t really notice the difference… until I started working with this very large project again. I remembered it taking a long time to compile this one from scratch, but I remembered that “long time” as being closer to ten minutes. It was taking well over an hour now, which was simply ridiculous.

I’m also still running my Linux virtual machine at the same time, for web access and such. I had some trouble getting the two virtual machines to play well together… whenever I started the Windows VM, the Linux one suddenly started losing memory, for reasons I couldn’t understand. I tweaked the memory usage on both of them… it helped, but they still seemed to be fighting over memory at times, even though the system itself had plenty of memory for both of them and itself too, without swapping.

I finally found out the reason for all of that when I went to locate a fix for the slow Windows compiles yesterday. It turns out that VMware Fusion, unbeknownst to me, was sharing memory between the two, allowing the Windows machine to steal the Linux machine’s resources. (Just what I’d expect a Microsoft product to do. ). On VMware’s site, I discovered a page with a note at the bottom:

Adding the following settings to a virtual machine can reduce the I/O load on the hard disk, however these adjustments require additional memory on the host. Only add these settings if there is sufficient free memory on the host to accommodate all the memory allocated to the virtual machine, otherwise you may cause a memory starvation condition that can reduce performance of all the running virtual machines or possibly affect the host operating system.

I had plenty of memory, so bring it on! After shutting down the Windows VM, I added the four lines that the page suggested to the virtual machine’s vmx file:

    MemTrimRate = "0"
    mainMem.useNamedFile=false
    sched.mem.pshare.enable = "FALSE"
    prefvmx.useRecommendedLockedMemSize = "TRUE"

That helped, a lot. Not only with the speed of the compiles, either — the Windows system was no longer stealing the Linux machine’s memory. But the compiles were still pretty slow, and it took me a while to discover why.

During a compile, I popped up the Windows Task Manager program, to see whether the system was swapping a lot of memory or something. And lo, what do I see, but that there’s another process who’s memory and CPU usage were going up while the compiler was trying to run. It turned out to be Microsoft Security Essentials, the free anti-malware program that I’d installed a little while back because the VPN software that one of the companies I work with uses insisted that I have some kind of virus protection. It was supposed to have no noticeable impact on the machine. Well, it had a very noticeable impact in this case.

After turning off its “real-time protection” option (and ignoring the dire warnings about how my computer was now At Risk!), full compiles were much faster. Still noticeably slower than on a similarly-equipped bare-metal system, but a lot more tolerable. I can’t say exactly how fast yet because I’m still improving it as I work on the code; I’ll run a thorough test once I’m done.

So equipped, I finally tracked down that startup problem — it was an instance of the static initialization order fiasco, which I should have thought of a lot sooner since I’d run into it just last week too. Oh well, live and learn. For now it’s back to the real task, developing the software.

Microsoft Visual C++ 2005 (i.e. v8) is a decent compiler, with a decent IDE. But there’s one “feature” that I dislike passionately: Intellisense.

The idea behind it is a good one: that the IDE scans your source code every so often and figures out how certain things will be compiled so that it can tell you about parts of it when you ask — the type of a variable, the parameters to a function call, the members of a class, etcetera. And it works very well… on smaller projects, anyway.

On larger ones, like Project Badger, those every-so-often scans become productivity-sucking nightmares. During a scan, the IDE sucks up all of the CPU power of the machine, and it can take several minutes to complete. On my machine, the user interface becomes practically unusable during that time. And it scans fairly often… whenever you start up the IDE, whenever you switch from one project to another, or from one configuration to another (like debug to release), and sometimes with no apparent provocation at all.

I’ve tried to find a way to disable it before, using the oft-recommended method of renaming a DLL. It doesn’t work. It might eliminate some of the scans, but the thing still scans way too often. Yesterday, after I’d had to switch my project from release to debug (and suffer through a scan) once too often, I decided to check into it again.

Lo and behold, I found a solution!

In a nutshell: with the IDE closed, replace the *.NCB file in your project directory with a zero-byte file, and make it read-only. (You may also have to copy that read-only file to your temporary directory; I did so, but I don’t know whether it’s necessary.) The IDE will complain about it when it starts up, and offer to fix the “problem;” don’t let it. Without write-access to that file, the IDE won’t even try to scan, ever!

Of course, you won’t have access to Intellisense features on that project. But I usually find them superfluous anyway, and definitely not worth the time-cost in terms of productivity — I can look up the things that I might use it for myself and save all that unproductive scanning time.

Life is good again.

One of the repeating items on my to-do list is cleaning off my office desk. It’s something I committed to recently, to try to keep my working area tidy; so long as I keep on top of it every few days, it only takes a few minutes each time. But I’d found myself putting it off for several days, and yesterday I decided to figure out why.

It didn’t take much detective work. We’ve moved most of our stuff into the new place now, but we’re still unpacking and organizing it all, and a lot of things don’t have places to go yet. One of those things was presently-unused computer cables and parts, which comprised the majority of the items on my desk.

Once I determined what to do with them, it was child’s play to clean off the desk. Total elapsed time: less than five minutes, including the time needed to set up the place.

Why is this worthy of a blog post? Because it’s a common productivity trap: you repeatedly procrastinate on something because you don’t know something about it, when it would only take a few minutes of thinking or research to figure it out. It’s easy to do once you realize that that’s the problem, but it can take some time to achieve that realization.

To reiterate the mantra of productivity:

1. Identify your goal and determine the measurable outcome that embodies it;
2. identify the next physical step toward that outcome that’s small enough to accomplish;
3. execute that step;
4. repeat steps 2 through 4 until the goal is achieved.

Or, in the oversimplified words of the shoe company Nike (named after the Greek goddess of victory): just do it.

As a programmer, I’ve always found that the more monitor space you’ve got, the more productive you can be. And multiple monitors are even better than a single huge monitor, in a lot of ways. When you’re referring to online documentation while you’re programming, for example, it’s a lot easier to just glance from one monitor to another than it is to switch between windows, or try to somehow fit all of both windows on a single screen at once.

I’ve been using a dual-monitor setup for years, but earlier this week I ran into a situation where even two monitors simply weren’t enough: coding in a Windows VM while having a Linux VM running a proxy and web browser, and needing to simultaneously monitor a program under Mac OS X. I managed to get by, but it was extremely irritating. And that was only the first part of that project, I’m going to have to revisit it for at least one more extended period in the near future.

I had an extra monitor lying around unused (an early-tech 4:3 17″ LCD one with an annoying flicker problem), but no way to connect it — this MacBook Pro only has one external monitor port in addition to the built-in monitor panel, and it’s already in use. And being a notebook computer, there was no way to add another video card. I’d seen USB video adapters before, but I’d never been able to use them because none of the manufacturers seemed to support Linux. But I suddenly realized a couple days ago that I was running a far more mainstream system now, and that the USB video adapter manufacturers who ignored Linux would almost certainly support Mac OS X.

Sure enough, that was the case. I found a fairly cheap representative of the species at our local Best Buy (for the curious, a Diamond BVU195 “USB Display Adapter Pro”), which claimed that OS X was supported. It turns out that it was, but I had to track down a hard-to-find page on their website to get the drivers; they weren’t included on the driver disk in the box, and whoever wrote the skimpy little manual had apparently never heard of an Apple computer. But they installed with no further hassle… and I had a three-monitor system.

It’s really a beautiful thing.

The third monitor isn’t quite as snappy as the first two — there’s a small but noticeable delay when you try to scroll a window on it, for example — but I expected that. I had some trouble getting it on my desk as well; even though I’d cleaned the desk off recently, and had kept it meticulously neat since, there simply wasn’t any place where it wouldn’t block something I didn’t want to give up. I finally plunked it down on the left side of the desk, in front of the window… it’ll do for now. It still has that annoying flicker too, but as it’s off to the side and I don’t have to stare at it continuously for long periods, I should be able to ignore that.

All in all, I’m happy with the setup. It should make further development work a lot easier. Now if the 8GB memory expansion that I need for this system would just drop to an affordable level, so my virtual machines could run simultaneously without a lot of really slow disk-swapping, I’d be all set.

I’ve mentioned before that I no longer run antivirus software on most of my Windows machines, but Bruce Schneier has just posted a persuasive argument in favor of it in most cases, despite the flaws.

But persuasive or not, it doesn’t change my opinion or my stance on the systems I control. Antivirus is generally a good thing, but on several CPU-challenged systems that I run, it’s more of a hassle than a help. If malware manages to penetrate one of those systems at some point, I may re-think that. Until then, it stays off.

(For the curious, and those about to flame me in the comments, please read the original article that Schneier is responding to first. Like Ranum, I am using other — and more effective — forms of security on those machines, just not an antivirus program.)

Joel On Software, on one of the few similarities between student projects and professional ones:

It’s taken me a while, but I finally learned that long-term deadlines (or no deadlines at all) just don’t work with professional programmers, either: you need a schedule of regular, frequent deliverables to be productive over the long term.

That applies to things like Project X too, a fact I’ll have to keep in mind.