Is Parallel Programming Just Too Hard?

pcause writes "There has been a lot of talk recently about the need for programmers to shift paradigms and begin building more parallel applications and systems. The need to do this and the hardware and systems to support it have been around for a while, but we haven't seen a lot of progress. The article says that gaming systems have made progress, but MMOGs are typically years late and I'll bet part of the problem is trying to be more parallel/distributed. Since this discussion has been going on for over three decades with little progress in terms of widespread change, one has to ask: is parallel programming just too difficult for most programmers? Are the tools inadequate or perhaps is it that it is very difficult to think about parallel systems? Maybe it is a fundamental human limit. Will we really see progress in the next 10 years that matches the progress of the silicon?"

Re:Nope.

[dch24]

Parallel programming isn't all that hard

Then why is it that (as of right now) all the up-modded posts are laid out sequentially down the comment tree?

Obligatory

[Architect_sasyr]

Windows Vista

Re:our brains aren't wired to think in parallel

[Anonymous Coward]

Our brains are massively parallel, but we do not consciously attend to more than a couple of things at a time.

Speak for yourself, I forget to breathe whenever I'm doing someth...

Re:Two words: map-reduce

[pchan-]

I guess we can make all future computer science courses one week then, since every problem can be solved by map-reduce!

What do you mean? I just wrote a clone of Half-Life 2 using only Map-Reduce. It works great, but it requires 600 exabytes of pre-computed index values and data sets and about 200 high end machines to reach decent frame rates.

Re:our brains aren't wired to think in parallel

[Lars512]

I think the biggest reason why it is difficult is that people tend to process information in a linear fashion. I break large projects into a series of chronologically ordered steps and complete one at a time. Sometimes if I am working on multiple projects, I will multitask and do them in parallel, but that is really an example of trivial parallelization.

That's because you have a global interpreter lock.

Re:our brains aren't wired to think in parallel

[Hoi Polloi]

Actually I parallel process when I read most Slashdot comments. Right now for instance I was reading your comment but thinking about cupcakes. Ummm, cupcakes...

Re:Actually it's just pipelined

[Mattintosh]

Again, you only need to look at the fuzzy-brain effect of bad Powerpoint presentations to see just that in practice. Forced to try to process two streams at the same time (speech and text), people just make a hash of both.

There's way more to it than that. The brain is an efficient organizer and sorter. It also tends to be great at estimating the relative importance of things and discarding the lesser ones it can't deal with. Thus, 99.999999999% of the time, I ignore Powerpoint presentations. It goes in my eyes, the brain deciphers the signal, decides that the Powerpoint stuff is useless drivel, and it continues processing the audio (in parallel!) and terminates the visual processing thread. Shortly thereafter, the audio signal is also determined to be of minimal benefit and is discarded as useless drivel as well, leaving more processing time for other things. Things like pondering the answer to the age-old question, "What's for lunch?"

BINGO!

[Anonymous Coward]

"There has been a lot of talk recently about the need for programmers to shift paradigms and begin building more parallel applications and systems. The need to do this and the hardware and systems to support it have been around for a while, but we haven't seen a lot of progress. The article says that gaming systems have made progress, but MMOGs are typically years late and I'll bet part of the problem is trying to be more parallel/distributed. Since this discussion has been going on for over three decades with little progress in terms of widespread change, one has to ask: is parallel programming just too difficult for most programmers? Are the tools inadequate or perhaps is it that it is very difficult to think about parallel systems? Maybe it is a fundamental human limit. Will we really see progress in the next 10 years that matches the progress of the silicon?"

BULLSHIT!

Anyone remembers the bullshit bingo [google.com] game? :)