Skip to main content

Didal: Distributed Data Library for Development of Parallel Fragmented Programs

  • Conference paper
  • First Online:
Parallel Computing Technologies (PaCT 2023)

Abstract

Nowadays with rapid evolution of high-performance computing systems it’s becoming essential to have tools to simplify development of efficient portable parallel programs for these systems. Fragmented programming is a technology where parallel program is represented as a collection of pieces of data (data fragments) and computations on these pieces (computation fragments), able to be tuned to the resources of a computing system and automatically provide such facilities as dynamic load balancing. Didal is a distributed data library to support development of efficient parallel fragmented programs on distributed memory supercomputers. The library contains facilities for data partitioning, distribution and load balancing. In this paper foundations of the library are explained and applicability of the library is demonstrated with Particle-in-Cell (PIC) method implementation, which shows performance comparable to conventional parallel programming tools.

This work was carried out under state contract with ICMMG SB RAS 0251-2022-0005.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Kireev, S., Malyshkin, V.: Fragmentation of numerical algorithms for parallel subroutines library. J. Supercomput. 57, 161–171 (2011). https://doi.org/10.1007/s11227-010-0385-3

    Article  Google Scholar 

  2. Malyshkin, V.: Active knowledge, LuNA and literacy for oncoming centuries. LNCS 9465, 292–303 (2015)

    MathSciNet  Google Scholar 

  3. Valkovsky, V.A., Malyshkin, V.E.: Synthesis of parallel programs and systems on the basis of computational models, Nauka, Novosibirsk, pp. 128 (1988). In Russian: Valkovsky, V.A., Malyshkin, V.E. (eds.) Sintez parallelnykh program i system na vychislitelnykh modelyah. Nauka, Novosibirsk, 128 str. (1988)

    Google Scholar 

  4. Acun, B., et al.: Parallel programming with migratable objects: Charm++ in practice. In: SC ’14: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 647–658. IEEE Press (2014). https://doi.org/10.1109/SC.2014.58

  5. Chamberlain, B.L., Callahan, D., Zima, H.P.: Parallel programmability and the chapel language. Int. J. High Perform. Comput. Appl. 21(3), 291–312 (2007). https://doi.org/10.1177/1094342007078442

    Article  Google Scholar 

  6. Bauer, M., Treichler, S., Slaughter, E., Aiken, A.: Legion: expressing locality and independence with logical regions. In: SC ’12: International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 1–11. IEEE Computer Society Press, Washington, DC (2012). https://doi.org/10.1109/SC.2012.71

  7. Lastovetsky, A.: Adaptive parallel computing on heterogeneous networks with mpC. J. Parallel Comput. 28(10), 1369–1407 (2002). https://doi.org/10.1016/S0167-8191(02)00159-X

    Article  MATH  Google Scholar 

  8. Bakhtin, V.A., Krukov, V.A.: DVM-approach to the automation of the development of parallel programs for clusters. J. Program. Comput. Softw. 45, 121–132 (2019). https://doi.org/10.1134/S0361768819030034

    Article  Google Scholar 

  9. Fürlinger, K., Fuchs, T., Kowalewski, R.: DASH: A C++ PGAS library for distributed data structures and parallel algorithms. In: IEEE 18th International Conference on High Performance Computing and Communications (HPCC). IEEE Press (2016). https://doi.org/10.1109/HPCC-SmartCity-DSS.2016.0140

  10. Brock, B., Buluç, A., Yelick, K.: BCL: a cross-platform distributed data structures library. In: ICPP ’19: 48th International Conference on Parallel Processing, pp. 1–10. Association for Computing Machinery Press, New York (2019). https://doi.org/10.1145/3337821.3337912

  11. Buss, A., et al.: STAPL: standard template adaptive parallel library. In: SYSTOR ’10: 3rd Annual Haifa Experimental Systems Conference, pp. 1–10. Association for Computing Machinery Press, New York (2010). https://doi.org/10.1145/1815695.1815713

  12. Beckman, P.H., Gannon, D., Johnson, E.: HPC++: experiments with the parallel standard template library. In: ICS ’97: 11th International Conference on Supercomputing, pp. 124–131. Association for Computing Machinery Press, New York (1997). https://doi.org/10.1145/263580.263614

  13. Bachan, J., et al.: UPC++: a high-performance communication framework for asynchronous computation. In: 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 963–973. IEEE Computer Society Press (2019). https://doi.org/10.1109/IPDPS.2019.00104

  14. Kireev, S.: A parallel 3D code for simulation of self-gravitating gas-dust systems. In: Malyshkin, V. (ed.) PaCT 2009. LNCS, vol. 5698, pp. 406–413. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03275-2_40

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Georgy Schukin .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Malyshkin, V., Schukin, G. (2023). Didal: Distributed Data Library for Development of Parallel Fragmented Programs. In: Malyshkin, V. (eds) Parallel Computing Technologies. PaCT 2023. Lecture Notes in Computer Science, vol 14098. Springer, Cham. https://doi.org/10.1007/978-3-031-41673-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-41673-6_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-41672-9

  • Online ISBN: 978-3-031-41673-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics