# Systematic extraction and implementation of divide-and-conquer parallelism

## Abstract

Homomorphisms are functions that match the divide-and-conquer paradigm and thus can be computed in parallel. Two problems are studied for homomorphisms on lists: (1) parallelism *extraction*: finding a homomorphic representation of a given function; (2) parallelism *implementation*: deriving an efficient parallel program that computes the function. A systematic approach to parallelism extraction proceeds by generalization of two sequential representations based on traditional *cons* lists and dual *snoc* lists. For some non-homomorphic functions, e.g., the maximum segment sum problem, our method provides an embedding into a homomorphism. The implementation is addressed by introducing a subclass of distributable homomorphisms and deriving for them a parallel program schema, which is time optimal on the hypercube architecture. The derivation is based on equational reasoning in the Bird-Meertens formalism, which guarantees the correctness of the parallel target program. The approach is illustrated with function *scan* (parallel prefix), for which the combination of our two systematic methods yields the “folklore” hypercube algorithm, usually presented *ad hoc* in the literature.

### Keywords

Prefix Shoe Folk## Preview

Unable to display preview. Download preview PDF.

### References

- 1.K. Achatz and W. Schulte. Architecture independent massive parallelization of divide-and-conquer algorithms. In B. Moeller, editor,
*Mathematics of Program Construction*, Lecture Notes in Computer Science 947, pages 97–127, 1995.Google Scholar - 2.D. Barnard, J. Schmeiser, and D. Skillicorn. Deriving associative operators for language recognition.
*Bulletin of EATCS*, 43:131–139, 1991.Google Scholar - 3.J. Bentley. Programming pearls.
*Communications of the ACM*, 27:865–871, 1984.Google Scholar - 4.R. S. Bird. Lectures on constructive functional programming. In M. Broy, editor,
*Constructive Methods in Computing Science*, NATO ASO Series F: Computer and Systems Sciences. Vol. 55, pages 151–216. Springer Verlag, 1988.Google Scholar - 5.G. Blelloch. Scans as primitive parallel operations.
*IEEE Trans. on Computers*, 38(11):1526–1538, November 1989.Google Scholar - 6.B. Carpentieri and G. Mou. Compile-time transformations and optimizations of parallel divide-and-conquer algorithms.
*ACM SIGPLAN Notices*, 20(10):19–28, 1991.Google Scholar - 7.M. Cole. Parallel programming with list homomorphisms.
*Parallel Processing Letters*, 5(2):191–204, 1994.Google Scholar - 8.A. Gibbons and W. Rytter.
*Efficient Parallel Algorithms*. Cambridge Univ. Press, 1988.Google Scholar - 9.J. Gibbons. The third homomorphism theorem.
*J. Fun. Programming*. To appear.Google Scholar - 10.J. Gibbons. Upwards and downwards accumulations on trees. In R. Bird, C. Morgan, and J. Woodcock, editors,
*Mathematics of Program Construction*, Lecture Notes in Computer Science 669, pages 122–138, 1992.Google Scholar - 11.J. Gibbons. The third homomorphism theorem. Technical report, Univ. of Auckland, 1994.Google Scholar
- 12.S. Gorlatch. Constructing list homomorphisms. Technical Report MIP-9512, Universität Passau, 1995.Google Scholar
- 13.S. Gorlatch. Stages and transformations in parallel programming. In M. Kara et al., editors,
*Abstract Machine Models for Parallel and Distributed Computing*, pages 147–162. IOS Press, 1996.Google Scholar - 14.S. Gorlatch. Systematic optimal parallelization of scan and other list homomorphisms. In
*Proceedings of the Euro-Par'96*. LNCS, to appear, 1996.Google Scholar - 15.Z. Grant-Duff and P. Harrison. Parallelism via homomorphisms.
*Parallel Processing Letters*. To appear.Google Scholar - 16.B. Heinz. Lemma discovery by anti-unification of regular sorts. Technical Report 94-21, TU Berlin, May 1994.Google Scholar
- 17.J. Kornerup. Mapping a functional notation for parallel programs onto hypercubes.
*Information Processing Letters*, 53:153–158, 1995.Google Scholar - 18.H. Kuchen, R. Plasmeijer, and H. Stolze. Distributed implementation of a dataparallel functional language. In
*PARLE'94, LNCS 817*, pages 464–477, 1994.Google Scholar - 19.K. Kumar and D. Skillicorn. Data parallel geometric operations on lists.
*Parallel Computing*, 21(3):447–459, 1995.Google Scholar - 20.R. Ladner and M. Fischer. Parallel prefix computation.
*J. ACM*, 27:831–838, 1980.Google Scholar - 21.J. Misra. Powerlist: a structure for parallel recursion.
*ACM TOPLAS*, 16(6):1737–1767, 1994.Google Scholar - 22.Z. G. Mou. Divacon: A parallel language for scientific computing based on divide and conquer. In
*Proc. 3rd Symposium on the Frontiers of Massively Parallel Computation*, pages 451–461, October 1990.Google Scholar - 23.J. O'Donnell. A correctness proof of parallel scan.
*Parallel Processing Letters*, 4(3):329–338, 1994.Google Scholar - 24.F. Preparata and J. Vuillemin. The cube-connected cycles: A versatile network for parallel computation.
*Communications of the ACM*, 24(5):300–309, 1981.Google Scholar - 25.M. J. Quinn.
*Parallel Computing*. McGraw-Hill, Inc., 1994.Google Scholar - 26.D. Skillicorn.
*Foundations of Parallel Programming*. Cambridge Univ. Press, 1994.Google Scholar - 27.D. Skillicorn and W. Cai. A cost calculus for parallel functional programming.
*Journal of Parallel and Distributed Computing*, 28:65–83, 1995.Google Scholar - 28.D. Smith. Applications of a strategy for designing divide-and-conquer algorithms.
*Science of Computer Programming*, (8):213–229, 1987.Google Scholar - 29.D. Swierstra and O. de Moor. Virtual data structures. In B. Moeller, H. Partsch, and S. Schuman, editors,
*Formal Program Development*, Lecture Notes in Computer Science 755, pages 355–371.Google Scholar - 30.D. Walker. The design of a standard message passing interface for distributed memory concurrent computers.
*Parallel Computing*, 20:657–673, 1994.Google Scholar