Parallel Superposition for Bulk Synchronous Parallel ML

  • Frédéric Loulergue
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2659)

Abstract

The BSMLlib is a library for Bulk Synchronous Parallel programming with the functional language Objective Caml. It is based on an extension of the λ-calculus by parallel operations on a parallel data structure named parallel vector, which is given by intention.

Those operations are flat and allow BSP programming in direct mode but it is impossible to express directly divide-and-conquer algorithms. This paper presents a new construction for the BSMLlib library which can express divide-and-conquer algorithms. It is called parallel superposition because it can be seen as the parallel composition of two BSP threads which can each use all the processors. An associated cost model derived from the BSP cost model is also given.

Keywords

Bulk Synchronous Parallelism Functional Programming Divide-and-Conquer Cost model 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    G. Akerholt, K. Hammond, S. Peyton-Jones, and P. Trinder. Processing transactions on GRIP, a parallel graph reducer. In Bode et al. [3].Google Scholar
  2. 2.
    W. Bäumker, A. adn Dittrich and F. Meyer auf der Heide. Truly effiient parallel algorithms: c-optimal multisearch for an extension of the BSP model. In 3 rd European Symposium on Algorithms (ESA), pages 17–30, 1995.Google Scholar
  3. 3.
    A. Bode, M. Reeve, and G. Wolf, editors. PARLE’93, Parallel Architectures and Languages Europe, number 694 in Lecture Notes in Computer Science, Munich, June 1993. Springer.MATHGoogle Scholar
  4. 4.
    O. Bonorden, B. Juurlink, I. von Otte, and I. Rieping. The Paderborn University BSP (PUB) Library-Design, Implementation and Performance. In Proc. of 13th International Parallel Processing Symposium & 10th Symposium on Parallel and Distributed Processing (IPPS/SPDP), San-Juan, Puerto-Rico, April 1999.Google Scholar
  5. 5.
    O. Bonorden, F. Meyer auf der Heide, and R. Wanka. Composition of Efficient Nested BSP Algorithms: Minimum Spanning Tree Computation as an Instructive Example. In International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA), 2002.Google Scholar
  6. 6.
    G.-H. Botorog and H. Kuchen. Efficient high-level parallel programming. Theoretical Computer Science, 196:71–107, 1998.MATHCrossRefGoogle Scholar
  7. 7.
    M. Cole.Algorithmic Skeletons: Structured Management of Parallel Computation. MIT Press, 1989.Google Scholar
  8. 8.
    M. Danelutto and D. Ratti. Skeletons in MPI. In S.G. Aki and T. Gonzalez, editors, International Conference on Parallel and Distributed Computing and Systems, Cambridge, USA, November 2002. ACTA Press.Google Scholar
  9. 9.
    J. Darlington, A. J. Field, P. G. Harrison, P. Kelly, D. Sharp, Q. Wu, and R. While. Parallel programming using skeleton functions. In Bode et al. [3].Google Scholar
  10. 10.
    P. de la Torre and C. P. Kruskal. Submachine locality in the bulk synchronous setting. In L. Bougé, P. Fraigniaud, A. Mignotte, and Y. Robert, editors, Euro-Par’96. Parallel Processing, number 1123-1124 in Lecture Notes in Computer Science, Lyon, August 1996. LIP-ENSL, Springer.Google Scholar
  11. 11.
    C. Foisy and E. Chailloux. Caml Flight: a portable SPMD extension of ML for distributed memory multiprocessors. In A. W. Böhm and J. T. Feo, editors, Workshop on High Performance Functionnal Computing, Denver, Colorado, April 1995. Lawrence Livermore National Laboratory, USA.Google Scholar
  12. 12.
    A. V. Gerbessiotis and L. G. Valiant. Direct Bulk-Synchronous Parallel Algorithms. Journal of Parallel and Distributed Computing, 22:251–267, 1994.CrossRefGoogle Scholar
  13. 13.
    G. Hains. Subset synchronization in BSP computing. In H.R. Arabnia, editor, PDPTA’98 International Conference on Parallel and Distributed Processing Techniques and Applications, volume I, pages 242–246, Las Vegas, July 1998. CSREA Press.Google Scholar
  14. 14.
    Y. Hayashi. Shaped-based Cost Analysis of Skeletal Parallel Programs. PhD thesis, University of Edinburgh, 2001.Google Scholar
  15. 15.
    J.M.D. Hill, W.F. McColl, and al. BSPlib: The BSP Programming Library. Parallel Computing, 24: 1947–1980, 1998.CrossRefGoogle Scholar
  16. 16.
    K. Hinsen. Parallel Programming with BSP in Python. Technical report, Centre de Biophysique Moléculaire, 2000.Google Scholar
  17. 17.
    F. Loulergue. BSλp: Functional BSP Programs on Enumerated Vectors. In J. Kazuki, editor, International Symposium on High Performance Computing, number 1940 in Lecture Notes in Computer Science, pages 355–363. Springer, October 2000.Google Scholar
  18. 18.
    F. Loulergue. Parallel Composition and Bulk Synchronous Parallel Functional Programming. In S. Gilmore, editor, Trends in Functional Programming, Volume 2, pages 77–88. Intellect Books, 2001.Google Scholar
  19. 19.
    F. Loulergue. Implementation of a Functional Bulk Synchronous Parallel Programming Library. In 14 th IASTED International Conference on Parallel and Distributed Computing Systems, pages 452–457. ACTA Press, 2002.Google Scholar
  20. 20.
    F. Loulergue, G. Hains, and C. Foisy. A Calculus of Functional BSP Programs. Science of Computer Programming, 37(1–3):253–277, 2000.MATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    J. Martin and A. Tiskin. BSP Algorithms Design for Hierarchical Supercomputers. submitted for publication, 2002.Google Scholar
  22. 22.
    W. F. McColl. Scalability, portability and predictability: The BSP approach to parallel programming. Future Generation Computer Systems, 12:265–272, 1996.CrossRefGoogle Scholar
  23. 23.
    Q. Miller. BSP in a Lazy Functional Context. In Trends in Functional Programming, volume 3. Intellect Books, may 2002.Google Scholar
  24. 24.
    P. Panangaden and J. Reppy. The essence of concurrent ML. In F. Nielson, editor, ML with Concurrency, Monographs in Computer Science. Springer, 1996.Google Scholar
  25. 25.
    S. Pelagatti. Structured Development of Parallel Programs. Taylor & Francis, 1998.Google Scholar
  26. 26.
    J. Serot and D. Ginhac. Skeletons for parallel image processing: an overview of the skipper project. Parallel Computing, 28(12):1685–1708, 2002.MATHCrossRefGoogle Scholar
  27. 27.
    D. B. Skillicorn. Multiprogramming BSP programs. Department of Computing and Information Science, Queen’s University, Kingston, Canada, October 1996.Google Scholar
  28. 28.
    D. B. Skillicorn, J. M. D. Hill, and W. F. McColl. Questions and Answers about BSP. Scientific Programming, 6(3), 1997.Google Scholar
  29. 29.
    A. Tiskin. The Design and Analysis of Bulk-Synchronous Parallel Algorithms. PhD thesis, Oxford University Computing Laboratory, 1998.Google Scholar
  30. 30.
    A. Tiskin. A New Way to Divide and Conquer. Parallel Processing Letters, (4), 2001.Google Scholar
  31. 31.
    Leslie G Valiant. A bridging model for parallel computation. Communications of the ACM, 33(8): 103, August 1990.CrossRefGoogle Scholar
  32. 32.
    A. Zavanella. Skeletons and BSP: Performance Portability for Parallel Programming. PhD thesis, Universita degli studi di Pisa, 1999.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Frédéric Loulergue
    • 1
  1. 1.Laboratory of Algorithms, Complexity and LogicUniversity Paris XIICréteil cedexFrance

Personalised recommendations