A little while back, I ran across a rant about multithreaded programming on The Register (NSFW due to strong language), which starts out with this paragraph:
I don’t know about you, but every time I have to program with threads and shared resources, I want to remove my face incrementally with a salad fork. Locks, mutexes, the synchronized keyword; all of these things can strike fear into the heart of a green developer. Most seasoned developers just fall into a rut of depression when it’s time for multi-threading. Developers like me simply talk our way out of it. It’s easier than thinking.
I was rather bemused by this, but I figured that some developers probably didn’t like writing multithreaded programs, just as some passionately despise pointers or date calculations.
But a few days ago, I was reading a FAQ page and discovered the comment “Threads are evil. Avoid them.” At this point, I decided I had to speak out.
Multithreaded programming is not that hard to master. There are a few extra things you have to do, and there are a few new problems to watch out for, but if you follow a straightforward set of rules, it’s easily possible to write good — and safe — multithreaded programs.
I can’t offer a complete list of these rules. I learned them the hard way; they’re embedded in the way I write code. But here are a few of them off the top of my head, geared toward C++ code:
-
Know when not to use it. Multithreading is powerful, but so is dynamite — and like dynamite, you have to treat it with respect, or it’ll blow your arm off. Multithreaded programming can be very useful in some cases, but most of the time it’s not called for.
-
Have each thread do a completely separate task. If your program can’t be split up into tasks that can be run mostly independent of one another, don’t even attempt it.
-
Don’t pass automatic variables (by reference or pointer) between threads. Automatic variables (in C and C++, at least) are created on the current thread’s stack; when they go out of scope in that thread, the memory is no longer safe to access in any thread. Passing non-array variables of built-in data types by value is generally safe, but that’s all I can promise.
-
For any variable that uses allocated memory and crosses threads, use reference-counting, with automatic cleanup in its destructor. You don’t know when it’s safe to delete such a variable. The Boost library’s
shared_ptr, or something similar, is the way to go for that kind of thing. -
Whenever possible, use a specific resources (file, block of memory, whatever) in one thread only. Try to avoid having two or more threads operating on the same block of data or the same file simultaneously. If you have to share a resource between threads, create a thread-safe interface for it — a C++ class that encapsulates it, for example. Then you can control all access to it, and use critical section or mutexes to prevent threads from trying to use it simultaneously.
-
Be careful with libraries. There are a lot of useful code libraries out there that can’t safely be used by multiple threads simultaneously. A lot of programmers these days rely heavily on bits of code written by other people. That’s fine — if you know exactly what’s going on in it. If you don’t, and the author doesn’t provide any assurance that it’s thread-safe, then you have to assume that it isn’t.
- Watch out for cyclical dependencies. If a thread locks a resource, make sure it can’t wait on another resource that might be locked by a different thread that could also be waiting on the first resource — that will lock your program up tighter than Fort Knox. I can’t completely articulate the way I avoid this problem, but it involves knowing exactly what each thread in your program is responsible for, and what resources each one uses.
Basically, you just have to know your code and how the different threads might interact. It takes some practice to get it right — and it’s not the kind of thing you want to practice on with a program that other people are going to use — but it’s really not difficult once you get the hang of it.
One additional point: multithreaded programming will be a must in the relatively near future, given the arrival of multicore processors. Unless the app is not at all compute-intensive, to improve response time the app will have to be multithreaded.
The silicon got here a bit ahead of the software tools to leverage it, but the good news is that there’s an increasing array of options: Intel’s TBB, OpenMP, Cilk++, Microsoft PLINQ and TPL, of course Pthreads and WinAPI threads, and others. (For what it’s worth, we outline some of the issues around multithreading in our e-Book, http://www.cilk.com/multicore-e-book/)
As a developer, I find that most programs have a ridiculous amount of power at their disposal already — far more than they really have any use for. That may change as more CPU-intensive design methods appear (artificial intelligence, anyone?), but in general, multiple CPU cores simply make running multiple programs better, rather than improving most individual programs.
In other words, most programs simply won’t need multithreading, or will only need a very simple multithreading system (maybe one thread for the interface and another for the actual work), for the foreseeable future. It’s only the developers of the really powerful programs (graphics programs, 3D games, and the like) that will really need to understand multithreading.
My chance for fanboi-ism again!
Apple is developing “Grand Central” for Snow Leopard which is supposed to allow programs and the operating system to utilize multiple cores with less difficulty.
http://www.apple.com/macosx/snowleopard/
here’s one way to assess whether multithreading will be needed by the masses: ask users how many of them are satisfied with the performance of their favorite apps.
My experience is that a large fraction is NOT satisfied. Now – is it possible to deliver some performance without multithreading? Of course. Moving to an improved serial algorithm is a good thing. But multithreading offers another dimension to push performance.
While I agree with your general points and often promote those very ideas myself (particularly partitioning of the problem) I disagree with your comment that if you cannot divide the problem in such a way that each thread does a completely separate task “don’t even attempt it.” Sure, that’s great advice for a novice and for code that will be maintained by various people of unknown skill levels, but it doesn’t show a ‘master’ level of knowledge. You wouldn’t, for example, be able to take advantage of a cell processor with that approach.
“Multithreaded programming is not that hard to master.” — Yes it is. Again, the advice given above is not a master level of multi-threaded programming. The more you learn about mutli-threaded programming, the more you realize that it truly is a more difficult problem than you ever could have imagined.
@Ilya: yes, it’s worth the extra effort for some programs. I’m saying that it’s not worth the extra programmer effort for most programs though — it falls into the 20% of code that takes 80% of the developer’s time and attention.
@Rex Kerr: The “completely separate task” comment in the original post is just my rule of thumb, and as such is ridiculously oversimplified, but it’s still valid in most cases. If two threads are operating on the same data simultaneously, a multi-core x86 CPU has to coordinate their access to the data (such as re-fetching updated cache memory on the separate cores), in addition to the overhead and developer-bother of mutexes or critical sections. That increases the code complexity and decreases the potential speed improvements, making it less useful than it might seem otherwise.
As with everything, there are exceptions to every rule. An adept developer can squeeze a lot more performance out of a multi-core CPU, but it’s going to cost a lot more in terms of time and effort — it’s only worth it in certain cases.
I wouldn’t claim to know everything about multithreaded programming, but I’ll stand by my assertion that it isn’t that hard to master, if someone is willing to experiment and make mistakes. Sure, there are some truly devilish multithreading problems out there — but most developers are never going to run into them.
@Ilya: One additional point: multithreaded programming will be a must in the relatively near future, given the arrival of multicore processors. Unless the app is not at all compute-intensive, to improve response time the app will have to be multithreaded.
Not at all true. Using multiple processes is a perfectly reasonable solution, as are programming models that abstract threading out of the equation.
Christopher – good point. I should have been more specific. Yes, when the computational challenge involves increasing throughput, and running multiple instances of the app works – yes, we can avoid multithreading.
But for the scenario of wanting to increase response time for a given app – say, a finite element analysis of a complex assembly, or some linear algebra operation on a given data set, or many recursive algorithms, my sense is that these will benefit greatly from multithreading the app, as they don’t lend themselves nicely to breaking up into multiple processes.