Supporting coarse and fine grain parallelism in an extension of ML

  • Peter Bailey
  • Malcolm Newey
  • David Sitsky
  • Robin Stanton
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 854)


We have built an extension of Standard ML aimed at multicomputer platforms with distributed memories. The resulting language, paraML, differs from other extensions by including and differentiating both coarse-grained and fine-grained parallelism.

The basis for coarse-grained parallelism in paraML is process creation where there is no sharing of data, with communication between processes via asynchronous message passing to typed ports. Fine-grained parallelism may be introduced in a number of ways; we have investigated data parallelism and light-weight threads as per Concurrent ML. Our experiments with data parallelism and algorithmic skeletons illustrate that libraries can be constructed to hide process creation and message passing from the user. Furthermore, since processes may be arbitrarily nested, such libraries can be linked together in program hierarchies, allowing the integration of a range of parallel granularities to be used within a program in a modular and efficient manner.

Performance analysis of the system is encouraging, with good speedup for a range of problems. Optimisation of process creation and message passing have lowered the size at which paraML becomes cost-effective against sequential implementations of ML for scientific problems.


parallel granularity processes ML algorithmic skeletons data parallelism 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Bailey, P. and Newey, M. Implementing ML on Distributed Memory Multicomputers. ACM SIGPLAN Notices, 28 (1), 59–63 (1993).Google Scholar
  2. [2]
    Bailey, P. and Newey, M. An Extension of ML for Distributed Memory Multicomputers. In Proceedings of the Sixteenth Australian Computer Science Conference, pp. 387–396, Brisbane, Australia (1993).Google Scholar
  3. [3]
    Bertomieu, B. and le Sergent, T. Programming with behaviours in an ML framework: the syntax and semantics of LCS. In Lecture Notes in Computer Science, 788, pp. 89–104, Springer Verlag, (1994).Google Scholar
  4. [4]
    Blelloch, G., Chatterjee, S., et al. Implementation of a Portable Nested Data-Parallel Language. In Proceedings of the Fourth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 102–111, San Diego, (1993).Google Scholar
  5. [5]
    Cole, M. Algorithmic Skeletons: Structured Management of Parallel Computation. MIT Press, (1989).Google Scholar
  6. [6]
    Foisy, C., Vachon, J. and Hains, G. DPML: de la Sémantique á l'Implantation. Journées Francophones des Langages Applicatifs, (1994).Google Scholar
  7. [7]
    Foster, I. and Chandy, K. M. Fortran M: A Language for Modular Parallel Programming. Argonne National Labs.Google Scholar
  8. [8]
    Hains, G. and Foisy, C. The Data-Parallel Categorical Abstract Machine. In Proceedings of the PARLE-93, (1993).Google Scholar
  9. [9]
    Halstead, R. Multilisp: A Language for Concurrent Symbolic Computation. ACM ToPLaS, 7 (4), 501–538 (1985).Google Scholar
  10. [10]
    High Performance Fortran Forum High Performance Fortran Language Specification. (January 1993).Google Scholar
  11. [11]
    Huelsbergen, L. and Larus, J. Dynamic Program Parallelization. In Proceedings of the ACM Conference on Lisp and Functional Programming, pp. 311–323, San Francisco, (1992).Google Scholar
  12. [12]
    Jagannathan, S. and Philbin, J. A Foundation for an Efficient Multi-Threaded Scheme System. In Proceedings of the ACM Conferenc on Lisp and Functional Programming, pp. 345–357, San Francisco, (1992).Google Scholar
  13. [13]
    Kranz, D. A. and Halstead, R. Mul-T: A high-performance parallel Lisp. In Proceedings of the SIGPLAN'89 Conference on Programming Language Design and Implementation, pp. 81–90, (1988).Google Scholar
  14. [14]
    Krumvieda, C. D. DML: Packaging High-Level Distributed Abstractions in SML. In Proceedings of the Third International Workshop on Standard ML, (1991).Google Scholar
  15. [15]
    Lin, C. and Snyder, L. A Comparison of Programming Models for Shared Memory Multiprocessors. In Proceedings of the International Conference on Parallel Programming, pp. 163–170, (1990).Google Scholar
  16. [16]
    Matthews, D. A Distributed Concurrent Implementation of Standard ML. University of Edinburgh, LFCS, (August 1991).Google Scholar
  17. [17]
    Milner, R., Parrow, J. and Walker, D. A Calculus of Mobile Processes I & II. Information and Computation, 100 (1), 1–77 (1992).Google Scholar
  18. [18]
    Milner, R., Tofte, M. and Harper, R. The Definition of Standard ML. MIT Press, Cambridge, Massachusetts (1989).Google Scholar
  19. [19]
    Morrisett, J. G. and Tolmach, A. Procs and Locks: A Portable Multiprocessing Platform for Standard ML of New Jersey. In Proceedings of the Fourth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 198–207, San Diego, (1993).Google Scholar
  20. [20]
    Quinn, M. J. and Hatcher, P. J. Data-parallel programming on multicomputers. IEEE Software, 7 (5), 69–76 (1990).Google Scholar
  21. [21]
    Reppy, J. H. Higher-Order Concurrency. PhD Thesis, Cornell University (1992).Google Scholar
  22. [22]
    Sitsky, D. The Application of Functional Languages in Scientific Computations. Honours Thesis, Australian National University, (1993).Google Scholar
  23. [23]
    Subhlok, J., Stichnoth, J. M., et al. Exploiting Task and Data Parallelism on a Multicomputer. In Proceedings of the Fourth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 13–22, San Diego, (1993).Google Scholar
  24. [24]
    von Eicken, T., Culler, D., et al. Active Messages: a Mechanism for Integrated Communication and Computation. In Proceedings of the 19th Annual International Symposium on Computer Architecture, pp. 256–267, (1992).Google Scholar
  25. [25]
    Wagner, D. B. and Calder, B. G. Leapfrogging: A Portable Technique for Implementing Efficient Futures. In Proceedings of the Fourth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 208–217, San Diego, (1993).Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1994

Authors and Affiliations

  • Peter Bailey
    • 1
  • Malcolm Newey
    • 1
  • David Sitsky
    • 1
  • Robin Stanton
    • 1
  1. 1.Department of Computer ScienceThe Australian National UniversityAustralia

Personalised recommendations