Skip to main content
SpringerLink
Log in

Automatic Parallelization for Distributed-Memory Systems: Experiences and Current Research

  • Conference paper
Europäischer Informatik Kongreß Architektur von Rechensystemen Euro-ARCH ’93

Part of the book series: Informatik aktuell ((INFORMAT))

Abstract

Distributed-memory systems (DMMPs) are powerful tools for solving large-scale scientific and engineering problems. However, these machines are difficult to program since the data must be distributed across the processors and message-passing operations must be inserted for communicating non-local data. In this paper, we discuss the automatic parallelization of Fortran programs for DMMPs, based on the programming paradigms associated with Vienna Fortran and High Performance Fortran. After introducing the state of the art, as represented by currently implemented systems, we will identify a number of limitations of this technology. In addition to insufficient functionality for handling many real applications, a major deficiency of current systems is the lack of intelligence in selecting good transformation strategies. We argue that a knowledge-based approach to compiling will contribute to more powerful and intelligent automatic parallelization systems in the future.

The work described in this paper was carried out as part of the ESPRIT projects GENESIS and PREPARE funded by the Austrian Research Foundation (FWF) and the Austrian Ministry for Science and Research (BMWF)

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. S. Ahuja, N. Carriero, and D. Gelernter. Linda and friends. IEEE Computer, 19: 26–34, August 1986.

    Article  Google Scholar 

  2. F. André, J.-L. Pazat, and H. Thomas. PANDORE: A system to manage data distribution. In International Conference on Supercomputing, pages 380–388, June 1990.

    Google Scholar 

  3. Marc Baber. Hypertasking support for dynamically redistributable and resizeable arrays on the iPSC. In Proceedings of the Fifth Distributed Memory Computing Conference, 59–66, 1990.

    Google Scholar 

  4. P. Bose Interactive Program Improvement via EAVE: an Expert Adviser for Vectorization. In: Proc. Int. Conf. on Supercomputing, St. Malo, pp.119–130(July 1988 ).

    Google Scholar 

  5. L.C.Breebaart,E.M.Paalvast,H.J.Sips. A Rule Based Transformation System for Parallel Languages. In: Proc.Third Workshop on Compilers for Parallel Computers, Vienna, Austria. Technical Report ACPC/TR 93-8, Austrian Center of Parallel Computation,pp. 13–21 (July 1992).

    Google Scholar 

  6. P. Brezany, H.M. Gerndt, V. Sipkova and H. Zima. SUPERB support for irregular scientific computations. In Proceedings of the Scalable High Performance Computing Conference, 1992.

    Google Scholar 

  7. P. Brezany, V. Sipkova, R. Das, J. Saltz. Compilation of Vienna Fortran Forall Loops. Paper submitted to the Conference SUPERCOMPUTING’93, Portland, Oregon, November 1993.

    Google Scholar 

  8. D.Callahan and K.Kennedy. Compiling programs for distributed-memory multiprocessors. Journal of Supercomputing, 2 (2), 151–69 (October 1988).

    Article  Google Scholar 

  9. B.M.Chapman,T.Fahringer, and H.P.Zima. Automatic Support for Data Distribution on Distributed Memory Multiprocessor Systems. In: Proc.Sixth Annual Workshop on Compilers for Parallelism, Portland,OR (August 1993) (to appear).

    Google Scholar 

  10. B.M. Chapman, P. Mehrotra, and H.P. Zima. Programming in Vienna Fortran. Scientific Programming 1, 1

    Google Scholar 

  11. M. Chen and J. Li. Optimizing Fortran 90 programs for data motion on massively parallel systems. Technical Report YALE/DCS/TR-882, Yale University, January 1992.

    Google Scholar 

  12. R. Das, J. Saltz. A manual for Parti runtime primitives - revision 2. Internal Research Report, University of Maryland, 1992.

    Google Scholar 

  13. T.Fahringer,R.Blasko, and H.P.Zima. Automatic Performance Prediction to Support Parallelization of Fortran Programs for Massively Parallel Systems. Proc.6th ACM Int.Conf.on Supercomputing,Washington D.C.,July 1992.

    Google Scholar 

  14. T.Fahringer and H.P.Zima. A Static Parameter Based Performance Prediction Tool for Parallel Programs. Proc.7th ACM Int.Conf.on Supercomputing,Tokyo,July 1993.

    Google Scholar 

  15. I. Foster and S. Taylor. Strand: New Concepts in Parallel Programming. Prentice- Hall, Englewood Cliffs, NJ, 1990.

    Google Scholar 

  16. G. Fox, S. Hiranandani, K. Kennedy, C. Koelbel, U. Kremer, C. Tseng, and M. Wu. Fortran D language specification. Department of Computer Science Rice COMP TR90079, Rice University, March 1991.

    Google Scholar 

  17. I.T.Foster and M.Chandy. Fortran M: A Language for Modular Parallel Programming. Technical Report, Department of Computer Science, California Institute of Technology (June 1992).

    Google Scholar 

  18. H. M. Gerndt. Automatic parallelization for distributed-memory multiprocessing systems. Ph.d. Thesis, university of bonn, december 1989.

    Google Scholar 

  19. H.M. Gerndt. Work Distribution in Parallel Programs for Distributed Memory Multiprocessors. In Proceedings of the International Conference on Supercomputing, 1991.

    Google Scholar 

  20. A.Gupta and B.E.Prasad.(Eds.). Principles of Expert Systems, IEEE Press, 1988.

    Google Scholar 

  21. M. Gupta, and P. Banerjee. Automatic Data Partitioning on Distributed Memory Multiprocessors. In Proceedings of the Sixth Distributed Memory Computing Conference., 1991.

    Google Scholar 

  22. M.T.Harandi and J.Q.Ning. Knowledge-Based Program Analysis. IEEE Software, January 1990, pp. 74–81.

    Google Scholar 

  23. P.J.Hatcher and M.J.Quinn. Data-Parallel Programming on MIMD Computers. MIT Press Scientific and Engineering Computation Series, 1991.

    Google Scholar 

  24. S.Hiranandani, K.Kennedy and C.-W.Tseng. Compiling Fortran D for MIMD Distributed-Memory Machines. Comm. ACM Vol. 35,No. 8, pages 66–80, August 1992.

    Article  MathSciNet  Google Scholar 

  25. High Performance Fortran Forum. High Performance Fortran Language Specification. Version 1.0, May 3, 1993.

    Google Scholar 

  26. David E. Hudak and Santosh G. Abraham. Compiler Techniques for Data Partitioning of Sequentially Iterated Parallel Loops. In Proceedings of the ACM International Conference on Supercomputing, 187–200 (1990)

    Google Scholar 

  27. K. Ikudome, G. Fox, A. Kolawa, and J. Flower. An automatic and symbolic parallelization system for distributed memory parallel computers. In Proceedings of the The Fifth Distributed Memory Computing Conference, pages 1105–1114, Charleston, SC, April 1990.

    Chapter  Google Scholar 

  28. Kathleen Knobe, Joan D. Lukas, and Guy L. Steele. Data Optimization: Allocation of Arrays to Reduce Communication on SIMD Machines. Journal of Parallel and Distributed Computing 8, 102–118 (1990)

    Article  Google Scholar 

  29. C. Koelbel. Compiling programs for nonshared memory machines. Ph.D. Thesis, Purdue University, August 1990.

    Google Scholar 

  30. C. Koelbel, P. Mehrotra, and J.Van Rosendale. Semi-automatic Process Partitioning for Parallel Computation. International Journal of Parallel Programming, 16 (5),pp. 365–382, 1987.

    Article  Google Scholar 

  31. J.Liebowitz,D.A.De Salvo (Eds.). Structuring Expert Systems, Prentice-Hall Inc. (1989).

    Google Scholar 

  32. J.Li. Compiling Crystal for Distributed-Memory Machines. Ph.D. Thesis, Yale University, Technical Report DCS/RR-876, October 1991.

    Google Scholar 

  33. J. Li and M. Chen. Index Domain Alignment: Minimizing Cost of Cross reference between Distributed Arrays. In: Frontiers’90: The Third Symposium on the Frontiers of Massively Parallel Computation, October 1990.

    Google Scholar 

  34. J. Li and M. Chen. Compiling Communication-Efficient Programs for Massively Parallel Machines. IEEE Transactions on Parallel and Distributed Systems Vol. 2 (3), 361–376, July 1991.

    Article  Google Scholar 

  35. P. Mehrotra and J. Van Rosendale. Programming distributed memory architectures using Kali. In A. Nicolau, D. Gelernter, T. Gross, and D. Padua, editors, Advances in Languages and Compilers for Parallel Processing, pages 364–384. Pitman/MIT- Press, 1991.

    Google Scholar 

  36. J. H.Merlin. ADAPTing Fortran 90 Array Programs for Distributed Memory Architectures. In H.P.Zima, editor,Parallel Computation. Proc.First International ACPC Conference, Salzburg, Austria, pages 184–200. Lecture Notes in Computer Science 591, Springer Verlag, 1991

    Google Scholar 

  37. MIMDizer User’s Guide, Version 8.0. Applied Parallel Research Inc., Placerville, CA., 1992.

    Google Scholar 

  38. E. Paalvast and H. Sips. A high-level language for the description of parallel algorithms. In Proceedings of Parallel Computing 89, Leyden, Netherlands, August 1989.

    Google Scholar 

  39. D. Pase. MPP Fortran programming model. In High Performance Fortran Forum, Houston, TX, January 1992.

    Google Scholar 

  40. J.Ramanujam, and P.Sadayappan. Compile-Time Techniques for Data Distribution in Distributed Memory Machines. TR-90-09-03, Dept. Electrical and Comp. Engineering, Louisiana State University (Sept. 1990).

    Google Scholar 

  41. A. Rogers and K. Pingali. Process decomposition through locality of reference. In Conference on Programming Language Design and Implementation, pages 69–80. ACM SIGPLAN, June 1989.

    Google Scholar 

  42. R. Rühl and M. Annaratone. Parallelization of Fortran code on distributed-memory parallel processors. In Proceedings of the ACM International Conference on Super-computing, June 1990.

    Google Scholar 

  43. J. Saltz, H. Berryman, and J. Wu. Multiprocessors and Run-Time Compilation: Concurrency, Practice and Experience, Vol. 3 (6), 573–592 (December 1991).

    Article  Google Scholar 

  44. J. Saltz, K. Crowley, R. Mirchandaney, and H. Berryman. Run-time scheduling and execution of loops on message passing machines. Journal of Parallel and Distributed Computing, 8 (2): 303–312, 1990.

    Article  Google Scholar 

  45. J.Subhlok,J.M.Stichnoth,D.R.O’Hallaron, and T.Gross. Exploiting Task and Data Parallelism on a Multicomputer. Proc.Forth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP’93),pp.13–22.

    Google Scholar 

  46. Vienna Fortran Compilation System Version 1.0 User’s Guide. Department of Statistics and Computer Science, University of Vienna, Austria (January 1993).

    Google Scholar 

  47. K.Y.Wang,D.Gannon. Applying Al Techniques to Program Optimization for Parallel Computers. In: K.Hwang,D.DeGroot(Eds.): Parallel Processing for Supercomputers and Artificial Intelligence. McGraw-Hill Pub. Company, Chap.12., pp. 441–485 (1989).

    Google Scholar 

  48. K.Y.Wang. Intelligent Program Optimization and Parallelization for Parallel Computers. Tech. Rept. No. CSD-TR-91-030, Department of Computer Science, Purdue University (April 1991).

    Google Scholar 

  49. H.P. Zima, H. Bast, and H.M. Gerndt. Superb: A tool for semi-automatic MIMD/SIMD parallelization. Parallel Computing, 6: 1–18, (January 1988).

    Article  Google Scholar 

  50. H.P. Zima, P. Brezany, B.M. Chapman, P. Mehrotra, and A. Schwald. Vienna Fortran — a language specification. ICASE Internal Report 21, ICASE, Hampton, VA, 1992.

    Google Scholar 

  51. H.P. Zima and B.M. Chapman. Supercompilers for Parallel and Vector Computers. ACM Press Frontier Series, Addison-Wesley, 1990.

    Google Scholar 

  52. H.P. Zima and B.M.Chapman. Compiling for Distributed-Memory Systems. Invited Paper, Proceedings of the IEEE, Special Section on Languages and Compilers for Parallel Machines. To appear (February 1993). Also: Technical Report ACPC/TR 92-16, Austrian Center for Parallel Computation (November 1992 ).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Software Technology and Parallel Systems, University of Vienna, Brünner Strasse 72, A-1210, Vienna, Austria

    Hans P. Zima, Peter Brezany, Barbara M. Chapman & Jan Hulman

Authors
  1. Hans P. Zima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter Brezany
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Barbara M. Chapman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jan Hulman
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut für Informatik, Technische Universität München, Arcisstraße 21, D-80290, München, Germany

    Peter Paul Spies

Rights and permissions

Reprints and permissions

Copyright information

© 1993 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zima, H.P., Brezany, P., Chapman, B.M., Hulman, J. (1993). Automatic Parallelization for Distributed-Memory Systems: Experiences and Current Research. In: Spies, P.P. (eds) Europäischer Informatik Kongreß Architektur von Rechensystemen Euro-ARCH ’93. Informatik aktuell. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-78565-8_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-78565-8_43

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-57315-9

  • Online ISBN: 978-3-642-78565-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation