Abstract
In the aerospace sciences we produce huge amounts of data. This data must be arranged in a meaningful order, so that we can analyze or visualize it. In this paper we focus on data that is distributed among computer processes and then needs to be sorted by a single root process for further analysis. We assume that the memory on the root process is too small to hold all sorted data at once, so that we have to perform the sorting and processing of data chunk-wise. We prove the efficiency of our approach in weak scaling tests, where we achieve a near constant bandwidth. Additionally, we obtain a considerable speed up compared to the standard parallel external sort. We also demonstrate the usefulness of our algorithm in a real-life aviation application.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Data may also reside on a subset of the mesh elements, in which case gaps in the Ids I occur.
- 2.
A related problem is the so called external sort problem. Here the data resides unsorted on a hard drive and has to be written back to the hard drive in sorted order, while only a limited amount of data can fit into the memory of the calculating process [10].
References
Bitton, D., DeWitt, D.J., Hsiao, D.K., Menon, J.: A taxonomy of parallel sorting. Technical report, Cornell University (1984)
CASE-2: SGI ICE X, Intel Xeon E5–2695v2 12C 2.4 GHz, Inifiniband, FDR. https://www.top500.org/system/178196. Accessed 16 Apr 2019
Friedland, D.B.: Design, analysis, and implementation of parallel external sorting algorithms. Ph.D. thesis (1981). aAI8206830
JUWELS: http://www.fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/JUWELS/JUWELS_node.html. Accessed 15 Apr 2019
Karypis, G., Schoegel, K., Kumar, V.: ParMETIS - parallel graph partitioning and sparse matrix ordering library, version 3.1 (2013)
Kernighan, B.W., Lin, S.: An efficient heuristic procedure for partitioning graphs. Bell Syst. Tech. J. 49(2), 291–307 (1970)
Knuth, D.E.: The Art of Computer Programming: Sorting and Searching, vol. 3. Pearson Education, London (1997)
Langr, D., Tvrdík, P., Dytrych, T., Draayer, J.P.: Algorithm 947: Paraperm-parallel generation of random permutations with MPI. ACM Trans. Math. Softw. 41(1), 51–526 (2014). https://doi.org/10.1145/2669372. http://doi.acm.org/10.1145/2669372
Leicht, T., et al.: DLR-Project Digital-X - Next generation CFD solver ‘Flucs’. Deutscher Luft- und Raumfahrtkongress 2016, February 2016. https://elib.dlr.de/111205/
Leu, F.C., Tsai, Y.T., Tang, C.Y.: An efficient external sorting algorithm. Inf. Proces. Lett. 75(4), 159–163 (2000)
Meinel, M., Einarsson, G.O.: The FlowSimulator framework for massively parallel CFD applications. In: PARA 2010 Conference: State of the Art in Scientific and Parallel Computing. Citeseer (2010)
Reimer, L.: The FlowSimulator–a software framework for CFD-related multidisciplinary simulations. In: European NAFEMS Conference Computational Fluid Dynamics (CFD) - Beyond the Solve, December 2015. https://elib.dlr.de/100536/
The HDF Group: Hierarchical Data Format, version 5 (1997-NNNN). http://www.hdfgroup.org/HDF5/
Acknowledgments
This research was carried out under the project Virtual Aircraft Technology Integration Platform (VicToria) by the German Aerospace Center (DLR).
The authors gratefully acknowledge the Gauss Centre for Supercomputing e.V. (www.gauss-centre.eu) for funding this project by providing computing time through the John von Neumann Institute for Computing (NIC) on the GCS Supercomputer JUWELS at Jülich Supercomputing Centre (JSC).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Holke, J., Rüttgers, A., Klitz, M., Basermann, A. (2020). Data-Adapted Parallel Merge Sort. In: Schwardmann, U., et al. Euro-Par 2019: Parallel Processing Workshops. Euro-Par 2019. Lecture Notes in Computer Science(), vol 11997. Springer, Cham. https://doi.org/10.1007/978-3-030-48340-1_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-48340-1_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-48339-5
Online ISBN: 978-3-030-48340-1
eBook Packages: Computer ScienceComputer Science (R0)