Systematic extraction and implementation of divide-and-conquer parallelism

  • Sergei Gorlatch
Concurrent and Parallel Programming
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1140)


Homomorphisms are functions that match the divide-and-conquer paradigm and thus can be computed in parallel. Two problems are studied for homomorphisms on lists: (1) parallelism extraction: finding a homomorphic representation of a given function; (2) parallelism implementation: deriving an efficient parallel program that computes the function. A systematic approach to parallelism extraction proceeds by generalization of two sequential representations based on traditional cons lists and dual snoc lists. For some non-homomorphic functions, e.g., the maximum segment sum problem, our method provides an embedding into a homomorphism. The implementation is addressed by introducing a subclass of distributable homomorphisms and deriving for them a parallel program schema, which is time optimal on the hypercube architecture. The derivation is based on equational reasoning in the Bird-Meertens formalism, which guarantees the correctness of the parallel target program. The approach is illustrated with function scan (parallel prefix), for which the combination of our two systematic methods yields the “folklore” hypercube algorithm, usually presented ad hoc in the literature.


Auxiliary Function Associative Operation Parallel Prefix Equational Reasoning Parallelism Implementation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    K. Achatz and W. Schulte. Architecture independent massive parallelization of divide-and-conquer algorithms. In B. Moeller, editor, Mathematics of Program Construction, Lecture Notes in Computer Science 947, pages 97–127, 1995.Google Scholar
  2. 2.
    D. Barnard, J. Schmeiser, and D. Skillicorn. Deriving associative operators for language recognition. Bulletin of EATCS, 43:131–139, 1991.Google Scholar
  3. 3.
    J. Bentley. Programming pearls. Communications of the ACM, 27:865–871, 1984.Google Scholar
  4. 4.
    R. S. Bird. Lectures on constructive functional programming. In M. Broy, editor, Constructive Methods in Computing Science, NATO ASO Series F: Computer and Systems Sciences. Vol. 55, pages 151–216. Springer Verlag, 1988.Google Scholar
  5. 5.
    G. Blelloch. Scans as primitive parallel operations. IEEE Trans. on Computers, 38(11):1526–1538, November 1989.Google Scholar
  6. 6.
    B. Carpentieri and G. Mou. Compile-time transformations and optimizations of parallel divide-and-conquer algorithms. ACM SIGPLAN Notices, 20(10):19–28, 1991.Google Scholar
  7. 7.
    M. Cole. Parallel programming with list homomorphisms. Parallel Processing Letters, 5(2):191–204, 1994.Google Scholar
  8. 8.
    A. Gibbons and W. Rytter. Efficient Parallel Algorithms. Cambridge Univ. Press, 1988.Google Scholar
  9. 9.
    J. Gibbons. The third homomorphism theorem. J. Fun. Programming. To appear.Google Scholar
  10. 10.
    J. Gibbons. Upwards and downwards accumulations on trees. In R. Bird, C. Morgan, and J. Woodcock, editors, Mathematics of Program Construction, Lecture Notes in Computer Science 669, pages 122–138, 1992.Google Scholar
  11. 11.
    J. Gibbons. The third homomorphism theorem. Technical report, Univ. of Auckland, 1994.Google Scholar
  12. 12.
    S. Gorlatch. Constructing list homomorphisms. Technical Report MIP-9512, Universität Passau, 1995.Google Scholar
  13. 13.
    S. Gorlatch. Stages and transformations in parallel programming. In M. Kara et al., editors, Abstract Machine Models for Parallel and Distributed Computing, pages 147–162. IOS Press, 1996.Google Scholar
  14. 14.
    S. Gorlatch. Systematic optimal parallelization of scan and other list homomorphisms. In Proceedings of the Euro-Par'96. LNCS, to appear, 1996.Google Scholar
  15. 15.
    Z. Grant-Duff and P. Harrison. Parallelism via homomorphisms. Parallel Processing Letters. To appear.Google Scholar
  16. 16.
    B. Heinz. Lemma discovery by anti-unification of regular sorts. Technical Report 94-21, TU Berlin, May 1994.Google Scholar
  17. 17.
    J. Kornerup. Mapping a functional notation for parallel programs onto hypercubes. Information Processing Letters, 53:153–158, 1995.Google Scholar
  18. 18.
    H. Kuchen, R. Plasmeijer, and H. Stolze. Distributed implementation of a dataparallel functional language. In PARLE'94, LNCS 817, pages 464–477, 1994.Google Scholar
  19. 19.
    K. Kumar and D. Skillicorn. Data parallel geometric operations on lists. Parallel Computing, 21(3):447–459, 1995.Google Scholar
  20. 20.
    R. Ladner and M. Fischer. Parallel prefix computation. J. ACM, 27:831–838, 1980.Google Scholar
  21. 21.
    J. Misra. Powerlist: a structure for parallel recursion. ACM TOPLAS, 16(6):1737–1767, 1994.Google Scholar
  22. 22.
    Z. G. Mou. Divacon: A parallel language for scientific computing based on divide and conquer. In Proc. 3rd Symposium on the Frontiers of Massively Parallel Computation, pages 451–461, October 1990.Google Scholar
  23. 23.
    J. O'Donnell. A correctness proof of parallel scan. Parallel Processing Letters, 4(3):329–338, 1994.Google Scholar
  24. 24.
    F. Preparata and J. Vuillemin. The cube-connected cycles: A versatile network for parallel computation. Communications of the ACM, 24(5):300–309, 1981.Google Scholar
  25. 25.
    M. J. Quinn. Parallel Computing. McGraw-Hill, Inc., 1994.Google Scholar
  26. 26.
    D. Skillicorn. Foundations of Parallel Programming. Cambridge Univ. Press, 1994.Google Scholar
  27. 27.
    D. Skillicorn and W. Cai. A cost calculus for parallel functional programming. Journal of Parallel and Distributed Computing, 28:65–83, 1995.Google Scholar
  28. 28.
    D. Smith. Applications of a strategy for designing divide-and-conquer algorithms. Science of Computer Programming, (8):213–229, 1987.Google Scholar
  29. 29.
    D. Swierstra and O. de Moor. Virtual data structures. In B. Moeller, H. Partsch, and S. Schuman, editors, Formal Program Development, Lecture Notes in Computer Science 755, pages 355–371.Google Scholar
  30. 30.
    D. Walker. The design of a standard message passing interface for distributed memory concurrent computers. Parallel Computing, 20:657–673, 1994.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1996

Authors and Affiliations

  • Sergei Gorlatch
    • 1
  1. 1.University of PassauPassauGermany

Personalised recommendations