When I was a kid, I went to school with a girl with dyslexia. She tried to explain it to me once, but I wasn’t really able to figure it out. After reading this article, it makes a lot more sense.
It also makes sense of the condition from an evolutionary perspective, for me — the ability to take in an entire scene at once would be a very useful trait for our ancestors, to the point that it’s still very common, despite the problems that it causes in today’s world.
WARNING! WARNING! WARNING!
The changes below are NOT RECOMMENDED! See part III for details!
WARNING! WARNING! WARNING!
It’s been several weeks since I touched on this subject last, and I’m sorry to say that the solution I came up with in the earlier article just wasn’t good enough. It was forcing the system to swap, which slowed things down even more. I had to back the Linux virtual machine back down to 2.5GB of memory, which was just barely enough to keep up the fast compiles, if I made sure to always log out and back in each morning and always have my browser shut down before I started compiling.
Yesterday afternoon, while pondering this problem, it suddenly occurred to me that I might already know of a solution! The problem is the slow hard drive, and the amount of memory needed by the file-system cache to mitigate that. But if those files took up less space in the file-system, then presumably they would also take up less space in the cache as well, right? So why not compress the files?
Disk compression is a classic trade-off. You spend more CPU time in exchange for saving disk space. It’s less well known, but if you have a slow drive and a fast CPU, that can also mean faster access to the compressed data.
The next question was, how to do it? Compressing the host drive was chancy… it might help the speed, but it might equally hurt it, and I didn’t want to waste a lot of time on this on no more than a guess. Compressing the entire Linux system drive might be an option too, but I didn’t want to mess with the boot system if I didn’t have to. Then I realized that I didn’t have to — I could just store the files that the compiler would presumably need on a new and compressed virtual drive, and mount it on the file system where the original, uncompressed files were.
Okay, how do I turn on compression on EXT4…? Hm, slight problem: EXT4 doesn’t have any compression. Okay, not a big deal. Linux has lots of file systems. Surely some of them would handle compression.
Some do, but not many. According to Wikipedia’s comparison of file systems page, the only Linux file systems that support transparent compression are Reiser 4 and BTRFS. (There were a few others that support it and could be gotten to work, but they were user-space, or required extra drivers, or were read-only, or weren’t native to Linux.) I’ve considered Reiser 4 before, but last I heard it’s not under development anymore, and its namesake developer is in jail for the foreseeable future, which rather tarnishes it.
BTRFS is another story. I’ve been impatiently waiting for it to mature enough to safely use. After some consideration, I decided that it was sufficient for a mostly-read-only portion of the system.
I still don’t know what files gcc is using, but some poking around the file system suggested that most of them are likely to be under /usr. There were somewhere between 4.2GB and 4.4GB in that tree, depending on what program you believed. To be safe, I made a new 15GB virtual drive, formatted it to BTRFS (via gparted — yeah, I cheated, so sue me 😉 ), and mounted it with the “compress” switch by using a line like this in /etc/fstab…
/dev/sdb1 /mnt/btrfs btrfs compress,noatime 0 2…copied the entire contents of the /usr directory to it (which took less than 15 minutes). df reported that the entire 4.2 or 4.4GB was actually using only 2.5GB, so the compression is working. 🙂 Then changed the /etc/fstab to this instead…
/dev/sdb1 /usr btrfs compress,noatime 0 2…and rebooted. Everything came up properly, and mount reported that /usr was pointing to the new virtual drive, as it should.
Now for the tests.
Initial compile: 3:03 (down from 3:48)! Subsequent compiles, after changing a single source file: 0:17 (a slight increase over the 0:16 listed in the earlier entry, but well within measurement tolerance, as well as my own). And the kicker: after clearing out the entire cache (with sync then sudo su then echo 3 > /proc/sys/vm/drop_caches), and confirming that it was cleared out, I modified the same source file and started a compile. The cache when it was done: 1.1GB!
I hadn’t thought to drop the cache and test it with the older setup, but if it dropped below 1.4GB or 1.5GB, I’d start getting IOWait delays again. So it looks like that gives me a respectable amount of breathing room — enough, at least, that I can safely run a browser while compiling code. 😀
I suspect I could do better than that, if I could figure out what other files to compress. There are /lib, /lib32, and /lib64 directories off of the root, but they contain less than 500MB total. Poking around other directories didn’t show me any other obvious targets. In any case, what I have now should be better than before. We’ll see how it goes.
A science-fiction film actually made in orbit. Talk about… wait for it… a match made in heaven. 😉
Good God… someone actually advocating an intelligent solution to movie piracy, in a mainstream business magazine?! Unbelievable!
Listen up, Hollywood. As an independent software developer, I’ve been dealing with Internet piracy for longer than you — without a single lawyer, let alone an army of them, and often on a shoestring budget. The facts in that article are pretty much true, especially the part about people paying for convenience. No matter how many battles you might win against the pirates, you can never win the war — not through technology, not through legislation, not even through Mafia-style brute force. If you want sales, you have to make the process of buying the product less painful and more pleasant than stealing it.
I buy DVDs, for three reasons: I have the money, I believe in rewarding creators, and I like to ensure that I have a copy of the movies and TV shows that I like to watch whenever I decide to watch them. Like most people, I’ll play by your rules — but only so long as you don’t make them too annoying. If you get arrogant and start trying to push me around (as you did with the Shrek 4 DVD that GoddessJ and I finally watched recently, with its unskippable previews for other movies whether I want to sit through them or not, as well as the usual unskippable FBI threats) I’ll be sorely tempted to rip your DVD and watch it on the computer instead, where I’m in control. It’s a lot easier and more convenient to just pop the purchased DVD into the player and hit play, but that convenience is dramatically diminished when you pull crap like that.
As the article ends:
Treat your customers with respect, and they’ll do the same to you. And that is how you fight piracy.
It’s never going to kill it off completely. People with more time and energy than money are still going to steal your stuff. Learn to live with that, because you can’t stop it, and concentrate on making the purchased experience so much better that those people will buy your wares as soon as they can afford them.
Just my two cents’ worth.
(Via Schneier on Security)
This sort of thing might have come directly from a science-fiction action story, where the hero is patched up in an unbelievably short time between epic space battles. Who says the future is always far away? 😉
I’ve often wondered if there might be a way to harness the placebo effect to actually help people get better. It seems that I’m not the only one, either.
“Designer ice cubes“? More proof that life imitates art, I guess. 😉
I can’t tell whether this is the funniest thing I’ve ever heard, or the most appalling. Or maybe both. 🙂
Now why hasn’t anyone thought of something like this before? Practically everyone uses earphones these days, and I for one always have to look at the blasted things to figure out which one is supposed to go in which ear. Maybe most people just don’t bother?
Of course, from the consumer’s end this will probably be a very pricey convenience for a long time, once it’s available at all. But from a business perspective, it just goes to prove that there are still little things right under your nose that you could improve, and likely make a bundle doing so.
“US, UK research: Thick kids more likely to become bigots“.
Ouch. I’ve often thought that today’s conservatives weren’t the brightest bulbs on the whole, and wondered whether there was some causal link there, but to have it bluntly spelled out like that is pretty damning.
Maybe these studies should be publicized a lot more. Nobody wants to be labeled a dunce, so it might cut down on racism, homophobia, and Tea Party Republicans all at once. 😉
(I have to point out that this refers only to today’s conservatives. Conservatism was a respectable position twenty or thirty years ago, it has just been hijacked more recently by the anyone-different-must-be-destroyed crowd, and the politicians willing to pander to them.)