Abstract
Parallel systems have evolved to become highly hierarchical and heterogeneous, requiring tools to program at the level of clusters, nodes, vectors, and accelerators such as GPUs and FPGAs. This chapter describes existing methodologies to program across these multiple levels and then proceeds to describe in detail the features of three modern parallel programming environments that focus on node and SIMD level parallelism, namely OpenMP, OmpSs-2, and XiTAO. The chapter starts by presenting a taxonomy of parallel programming models and then describes the three focus programming models in sequence.
This is a preview of subscription content, log in via an institution to check access.
References
Blumofe RD, Joerg CF, Kuszmaul BC, Leiserson CE, Randall KH, Zhou Y (1995) Cilk: an efficient multithreaded runtime system. In: Proceedings of the fifth ACM SIGPLAN symposium on principles and practice of parallel programming, PPOPP’95, pp 207–216
Duran A, Perez JM, Ayguadé E, Badia RM, Labarta J (2008) Extending the openmp tasking model to allow dependent tasks. In: Eigenmann R, de Supinski BR (eds) OpenMP in a new era of parallelism. Springer, Berlin/Heidelberg, pp 111–122. ISBN 978-3-540-79561-2
Duran A, Ayguadé E, Badia RM, Labarta J, Martinell L, Martorell X, Planas J (2011) OmpSs: a proposal for programming heterogeneous multi-core architectures. Parallel Process Lett 21 (2): 173–193
Karrenberg R (2015) Automatic SIMD vectorization of SSA-based control flow graphs. Springer, Wiesbaden
Kennedy K, Allen JR (2001) Optimizing compilers for modern architectures: a dependence-based approach. Morgan Kaufmann Publishers Inc., San Francisco
Kurzak J, Ltaief H, Dongarra J, Badia RM (2010) Scheduling dense linear algebra operations on multicore processors. Concurr Comput Pract Exp 22 (1): 15–44. ISSN 1532-0626
Larsen S, Amarasinghe S (2000) Exploiting superword level parallelism with multimedia instruction sets. In: Proceedings of the ACM SIGPLAN 2000 conference on programming language design and implementation, PLDI’00. Association for Computing Machinery, New York, pp 145–156. ISBN 1581131992. https://doi.org/10.1145/349299.349320
Maroñas M (2021) On the design and development of programming models for exascale systems. PhD thesis, Universitat Politècnica de Catalunya
Maroñas M, Sala K, Mateo S, Ayguadé E, Beltran V (2019) Worksharing tasks: an efficient way to exploit irregular and fine-grained loop parallelism. In: 2019 IEEE 26th international conference on high performance computing, data, and analytics (HiPC). IEEE, pp 383–394
Maroñas M, Teruel X, Bull M, Ayguade E, Beltran V (2020) Evaluating worksharing tasks on distributed environments. In: 2020 IEEE international conference on cluster computing (CLUSTER). IEEE. Pending publication
Maroñas M, Ayguadé E, Beltran V (2021) Mitigating the NUMA Effect on Task-Based Runtime Systems. Submitted to the Journal of Supercomputing. ACM
OpenMP Architecture Review Board (1997) OpenMP Fortran Application Programming Interface 1.0. Accessed: 18 Feb 2021
OpenMP Architecture Review Board (1998) OpenMP C and C++ Application Programming Interface 1.0. Accessed: 18 Feb 2021
OpenMP Architecture Review Board (2005) OpenMP Application Programming Interface 2.5. Accessed: 18 Feb 2021
OpenMP Architecture Review Board (2008) OpenMP Application Programming Interface 3.0. Accessed: 18 Feb 2021
OpenMP Architecture Review Board (2013) OpenMP Application Programming Interface 4.0. Accessed: 18 Feb 2021
OpenMP Architecture Review Board (2020) OpenMP Application Programming Interface 5.1. https://www.openmp.org/wp-content/uploads/OpenMP-API-Specification-5-1.pdf. Accessed: 18 Feb 2021
Pallares F, Mateo S, Beltran V, Ayguadé E (2019) Master Thesis: extending OmpSs-2 with flexible task-based array reductions. https://upcommons.upc.edu/handle/2117/129246. Accessed: 01 Mar 2021
Perez JM, Beltran V, Labarta J, Ayguadé E (2017) Improving the integration of task nesting and dependencies in OpenMP. In: 2017 IEEE international parallel and distributed processing symposium (IPDPS). IEEE, pp 809–818
Pericàs M (2018) Elastic places: an adaptive resource manager for scalable and portable performance. ACM Trans Archit Code Optim 15 (2). ISSN 1544-3566. https://doi.org/10.1145/3185458
RISC-V Community (2021) RISC-V vector extension intrinsic document. https://github.com/riscv/rvv-intrinsic-doc. Accessed: 25 Feb 2021
Soomro PN, Abduljabbar M, Castrillon J, Pericás M (2021) An online guided tuning approach to run CNN pipelines on edge devices. In: Proceedings of the 18th ACM international conference on computing frontiers (CF’21), New York. Association for Computing Machinery (ACM) pp 45–53. ISBN 9781450384049. https://doi.org/10.1145/3457388.3458662
Swamy H (2012) Structured parallel programming patterns for efficient computation by michael mccool, arch d. robison and james reinders. ACM SIGSOFT Softw Eng Notes 37: 43. https://doi.org/10.1145/2382756.2382773
The XiTAO development team (2021) XiTAO. https://github.com/CHART-Team/xitao.git. Accessed: 26 Feb 2021
Acknowledgements
This work has been developed with the support of the Spanish Ministry of Science and Innovation (Computacion de Altas Prestaciones VIII: PID2019-107255GB). This work has received funding from the European Union Horizon 2020 research and innovation program under the LEGaTO project with grant agreement No. 780681 (https://legato-project.eu/). This work is supported by the European Union’s Horizon 2020 research and innovation program under the grant agreement No 754304 (DEEP-EST). This work has been done as part of the European Processor Initiative project. The European Processor Initiative (EPI) (FPA: 800928) has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement EPI-SGA1: 826647
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Singapore Pte Ltd.
About this entry
Cite this entry
Farooqi, M.N. et al. (2022). Parallel Programming Models. In: Chattopadhyay, A. (eds) Handbook of Computer Architecture. Springer, Singapore. https://doi.org/10.1007/978-981-15-6401-7_30-1
Download citation
DOI: https://doi.org/10.1007/978-981-15-6401-7_30-1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-6401-7
Online ISBN: 978-981-15-6401-7
eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering