Fundamentals of Parallel Programming



In this chapter we will look at some of the fundamental concepts and techniques specific to parallel programming. We do not aim at providing an all-inclusive detailed description of parallel computing as that would require much more than the total length of this book; rather we aim at presenting a theoretical framework on which later chapters, dealing with programming many-core chips can build on. We will consider, on conceptual level, the fundamental types of parallelism, the synchronization concepts as well as how these are implemented and supported in modern computing systems. Finally, we describe the basic patterns for architecting and designing parallel software.


Shared Memory Message Passing Processor Core Critical Section Transactional Memory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    The OpenMP Architecture Review Board (2008) The OpenMP Application Program Interface. Accessed 10 January 2011
  2. 2.
    Frigo M, Leiserson C E, Randall K H (1998) The implementation of the Cilk-5 Multithreaded Language. Proceedings of the ACM SIGPLAN 1998 conference on Programming Language Design and Implementation, 212-223Google Scholar
  3. 3.
    Reinders J (2007) Intel Thread Building Blocks: Outfitting C++ for Multi-core Processor Parallelism. O’Reilly MediaGoogle Scholar
  4. 4.
    Mars J, Williams D, Upton D, Ghosh S, Hazelwood K (2008) A Reactive Unobtrusive Prefetcher for Multicore and Manycore Architecture. Proceedings of the Workshop on Software and Hardware Challenges of Manycore Platforms 2008, 41-50Google Scholar
  5. 5.
    Nellans D, Sudan K, Balasubramonian R, Brunvand E (2010) Improving Server Performance on Multi-Cores via Selective Off-loading of OS Functionalility. Proceedings of the 10th Workshop on Interaction between Operating Systems and Computer ArchitectureGoogle Scholar
  6. 6.
    Apple Corporation (2009) Grand Central Dispatch : A Beter Way to Do Multicore. Accessed 11 January 2011Google Scholar
  7. 7.
    Gropp W, Lusk E, Skjellum A (1994) Using MPI: Portable Parallel Programming with the Message Passing Interface. MIT Press Scientific And Engineering Computation Series, Cambridge MA, USAGoogle Scholar
  8. 8.
    Dijkstra E (1974) Over seinpalen (in Dutch). EWD 1974. Accessed 11 January 2011
  9. 9.
    Hoare, C A R (1974) Monitors: an Operating System Structuring Concept. Communications of the ACM 17(10):549-557zbMATHCrossRefGoogle Scholar
  10. 10.
    Hansen B P (1975) The Programming Language Concurrent Pascal. IEEE Transactions on Software Engineering 2:199-206CrossRefGoogle Scholar
  11. 11.
    Herlihy M, Moss J E B (1993) Transactional Memory: Architectural Support for Lock-free Data Structures. Proceedings of the 20th International Symposium on Computer Architecture: 289-300Google Scholar
  12. 12.
    Yalcin G, Unsal O, Hur I, Cristal A, Valero M (2010) FaultTM: Fault Tolerance using Hardware Transactional Memory. Proceedings of the 3rd Workshop on Parallel Execution of Sequential Programs on Multi-Core ArchitecturesGoogle Scholar
  13. 13.
    Vajda A (2010) Handling of Shared Memory in Many-Core Processors without Locks and Transactional Memory. Proceedings of the 2010 Workshop on Programmability Issues for Multi-Core ComputersGoogle Scholar
  14. 14.
    Suleman M A, Mutlu O, Qureshi M K, Patt Y N (2009) Accelerating Critical Section Execution with Assymetric Multi-Core Architectures. Proceedings of the International Conference on Architectural Support for Programming Languages and Operating SystemsGoogle Scholar
  15. 15.
    Vajda A (2010) The Case for Coherence-less Distributed Cache Architecture. Proceedings of the 4th Workshop on Chip Multi-processor Memory Systems and InterconnectsGoogle Scholar
  16. 16.
    TIPC Working Group (2010) TIPC: Transparent Inter Process Communication Protocol. Accessed 11 January 2011Google Scholar
  17. 17.
    Multicore Association (2010) Multicore Communication API, available from Accessed 11 January 2011
  18. 18.
    Friedman D, Wise D (1976) The impact of Applicative Programming on Multiprocessing. Proceedings of the International Conference on Parallel Processing: 263-272Google Scholar
  19. 19.
    Gamma E, Helm R, Johnson R, Vlissides J M (1994) Design Patterns: Elements of Reusable Object-Oriented Software. Addison Wesley ProfessionalGoogle Scholar
  20. 20.
  21. 21.
    Mattson T G, Sanders B A, Massingill B L (2004) Patterns for Parallel Programming. Addison WesleyGoogle Scholar
  22. 22.
    Mattson T (2010) Our Pattern Language (OPL). /patterns/opl_pattern_language-feb-13.pdf. Accessed 11 January 2011Google Scholar
  23. 23.
    Asanovic K et al (2006) The Landscape of Parallel Computing Research: A View from Berkeley. University of California, Berkeley Technical reportGoogle Scholar
  24. 24.
    Tasharofi S, Johnson R (2010) Patterns of Optimized Loops, Proceedings of the Workshop on Parallel Programming PatternsGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Oy L M Ericsson AbJorvasFinland

Personalised recommendations