Obligatory science-fiction link: in Harry Harrison’s The Stainless Steel Rat’s Revenge, the main character poses as a representative of an intergalactic weapons manufacturer to infiltrate the militant world of Cliaand:

I unlocked the case and flipped back the lid. The armament specialist glared down at the various components in their padded niches. I explained.

“My firm is the originator and sole manufacturer of the memory line of proximity fuses. No other line is as compact as ours, none as versatile.” I used tweezers to take a fuse from a holder. It was no larger than a pinhead. “This is the most minuscule, designed to be used in a weapon as small as a handgun. Firing activates the fuse which will then detonate the charge in the slug when it comes near a target or predetermined size. This other fuse is the most intelligent, designed for use in heavy weapons or missiles.” They all leaned forward eagerly when I held up the wafer of the Mem-IV and pointed out its singular merits.

“All solid-state construction, capable of resisting incredible pressures, thousands of G’s, massive shocks. It can be preset to detonate only when approaching a specific target, or can be programmed externally and electronically at any time up to the moment of firing. It contains discrimination circuits that will prevent explosion in the vicinity of friendly equipment. It is indeed unique.”

There was no talk of his wares guiding the explosive bullet/warhead/whatever, simply adding some intelligence to when, where, and whether it detonates, but I can foresee this real-world example adding such capabilities in the future as well.

How painful? The first compile of a small C++ source file took roughly three minutes and forty-eight seconds, almost all of it spend waiting for the hard drive. Subsequent ones (if I’d logged in no more than a few hours ago) took only sixteen seconds. If I did anything between compiles — switched to a Firefox window to do some research, for instance — then the time for the next compile started climbing toward the initial mark pretty quickly. And if I left it logged in overnight, a compile of the same file never took less than a full minute, until I logged out and back in again.

I don’t know why the hard drive on this system seems so slow. I can’t even figure out if what I’m seeing is normal for this machine’s specifications (and I really hope it isn’t). If the hard drive were noticeably faster, the IOWait bottleneck wouldn’t be as much of a problem, but replacing it to find out isn’t a viable option at the moment, so it was time to look for alternative ways to improve it.

The first thing you always look at in such cases is giving the machine more memory. That was problematic here though: I need to run at least two virtual machines (Linux and Windows) almost full-time, often with a third (an older version of Windows) as well. The third virtual machine can get away with only a gigabyte of memory, but the Linux system requires at least two gigabytes with the workload I use it for, and the other Windows one nearly as much. The host machine is maxes out at 8GB, and it’s fully loaded. Gritting my teeth, I decided to sacrifice the third VM and bumped the Linux VM up to 2.5GB. There was no noticeable change.

(I’d already ensured that both the host machine and the Linux VM were running fully in memory, without swapping.)

The next avenue to explore was figuring out what the compiler was spending its time reading. If I knew that, maybe I could do something to streamline it. But all attempts at identifying that have failed — GCC’s bewildering array of debugging options doesn’t seem to include anything that provides that information, and though I’m sure there are programs to log exactly what files are being accessed when under Linux, I haven’t been able to locate them.

Okay, plan C: maybe there was some error or setting on the disk that was slowing down the reads? fsck gave the Linux virtual drives a clean bill of health, and using noatime made no noticeable difference either. Disk Utility on the host machine claimed there were many permissions problems, so I let it grind away at it for more than an hour until it was satisfied, but that produced no change. Scanning the host disk for any problem areas took a while longer, and was equally fruitless. I even tried resetting the PRAM, though I’m not sure what that is or does; no effect.

Plan D involved digging into all the information Google could provide on Linux file system speeds. Maybe an alternate file system would help? From everything I was able to find, the only one that might help significantly was ReiserFS, and only if the files the compiler was spending its time on were small ones. Experimenting with that felt like it would waste more time than I’d save by solving the problem (assuming it did, which wasn’t assured), so scratch that idea.

On to plan E (and some concern that I’d run out of letters before this was through): maybe there’s a cache setting to improve things?

Paydirt! Or rather, something slightly better than just dirt. After maybe a couple days of work on that, spread out over several weeks, I finally found one page a couple days ago that described the only option that made a significant difference: the cache-pressure setting. Essentially it tells the caching system whether to prefer to keep the contents of files in the cache, or the file-system information that lets it find files. The default setting is 100, which means keep both equally; a higher setting favors the contents of files, a lower one favors the file-system info. Some experimentation with it (using the command sudo sysctl -w vm.vfs_cache_pressure=XX, where XX is the number to set it to) showed that a setting of 10 kept the compile times and IOWait to a minimum — success!

Or was it? That worked well if I’d logged into the machine within the last few hours, but after it had been running for a while, compiles started slowing down again — to the point that, after leaving it running overnight, that file took more than a minute to compile, no matter how many times I tried it or how close together they were. Better that it was previously, but was there any way to improve it further?

What was happening overnight that could affect it that way? What did logging out and back in change that fixed it? The only answer seemed to be memory, again — as the VM ran longer, the memory in use grew, until it stabilised at between 600MB and 700MB (closer to 1GB if Firefox, with my current set of must-have extensions, were also running). That left a gigabyte for caching — surely that was sufficient for whatever GCC needed to look at? But there was no other difference I could find.

Maybe 2.5GB just wasn’t enough? I couldn’t imagine why that might be the case, but I bumped it up to a full 3GB.

It worked. The first compile after rebooting the machine still took the same amount of time, and subsequent compiles remained at about 16 seconds — but the next morning, after leaving it running overnight, compiles after the first one stayed at about 16 seconds. The IOWait was over!

I’m not real happy about that solution. The machine is responsive, but it’s operating perilously close to its memory limit: there isn’t enough room left to sneeze in without forcing it to start swapping to disk. When it was running three VMs, I could always shut down the third one if I needed to free up some RAM; now that safety valve is gone. Even upgrading to more recent hardware wouldn’t help; the current crop of MacBook Pro machines also maxes out at 8GB.

I would really like to stay with a Mac, for the convenience of having all three major OSes available simultaneously. I must stay with a notebook system. I hope Apple’s next crop of MacBook Pro machines increases the memory limit, or I’ll have to look at non-Mac alternatives.

I’ve said it before, in all-caps and bold italics: DO NOT USE AN INSECURE FREAKIN’ CONSUMER OS LIKE WINDOWS ON VITAL CONTROL SYSTEMS!

At least a few of the higher-ups in the military seem to have learned to listen to what their technical people have likely been saying for years. With any luck it’ll filter out to the private sector too, and sooner rather than later.

EDIT, three hours later: in an ironic twist, it turns out that today is the tenth anniversary of the Bill Gates’ “trustworthy computing” memo. While the change in focus has been welcome, it hasn’t really hardened Windows, just elminiated the most blatant insecurities. Windows remains basically a single-user consumer OS, and still tries to be consumer-friendly at the expense of security. So long as Microsoft refuses to require people to learn anything in order to use Windows, it will never be secure.

This may sound strange, but that isn’t necessarily problem. A consumer OS should be easy to use, and shouldn’t require the user to learn any more than he could pick up by sitting at the keyboard and playing with it. At the same time, such an OS should never be used for anything vital — leave it to what it’s good for, which doesn’t include anything that requires security.

There was a near-future SF novel I read many years ago (I don’t recall which one) that posited that actors would be replaced almost entirely by computer-generated models. We’re already seeing the precursors of it — all the Pixar releases, for instance. And characters in games are constantly getting more realistic; in portions of Final Fantasy X, for instance, you would swear that you’re looking at people, rather than computer-generated images.

Get some strong AI, and give them a couple decades of practice, and you may well start to see the occasional real movies — not just cartoon-like ones — made entirely in a computer.

Here is a short clip of a toad (frog?) playing the same game. Be sure to watch it through to the end, where the toad expresses his own disgust in a rather more active way.

(Around Halloween this year, I suggested that GoddessJ should get a program we saw for her iPad that showed a spider that ran away from wherever you touched, and let the cats play with it. She declined because she was afraid our lard-arsed felines would break the iPad.)

That brings up a fact well-known in security circles: an attacker only needs to find a single vulnerability, while the defender must block all the possible attacked vectors. Miss even one, and you might as well not have wasted your time protecting any of them.

I hope there weren’t any really important secrets in that drone.

UPDATE: It seems that smart guys are saying this is implausible because it would be hard to do. Hate to break it to you, guys, but “hard to do” just means that nobody on your side has put in the effort to figure out how to do it yet. As such, it may still be how it was done.

Then there are non-android uses, like simply controlling machinery or computers without need of physical contact. Repetitive Stress Injury could become a thing of the past when all you have to do is think about hitting the keys. I imagine you’d be able to “type” a whole lot faster that way too.

And of course, it will pave the way for the inevitable merging of mind and machine.