What Makes Parallel Programming Difficult? 196
An anonymous reader writes "Intel's Aater Suleman writes about why parallel programming is difficult. ... I was unaware ... that a major challenge in multi-threaded programming lies in optimizing parallel programs not just getting them to run. His analysis is insightful and the case study is very enlightening if you are unfamiliar with parallel code debugging. "
Efficiency is hard (Score:5, Informative)
Using locks and the like make it very easy to do multithreaded and parallel programs.
The big problem comes when you need multiple locks because you find your program is waiting more on locks than anything else which is gumming up the whole works, and that can easily lead to deadlocks and other fun stuff.
Another way is to consider lockless algorithms, which don't have such blocking mechanisms. However, then you get into issues where atomicity isn't quite so atomic thanks to memory queues and re-ordering done in the modern CPU, and thus have to start adding memory barriers before doing your atomic exchanges.
Raymond Chen (of Microsoft) did a nice write up of the lockfree ways to do things and what Windows provides to accomplish them.
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/05/10149783.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/06/10150261.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/06/10150262.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/07/10150728.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/08/10151159.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/08/10151258.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/12/10152296.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/13/10152929.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/14/10153633.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/15/10154245.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/19/10155452.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/20/10156014.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/21/10156539.aspx [msdn.com]
http://blogs.msdn.com/b/oldnewthing/archive/2011/04/22/10156894.aspx [msdn.com]