Some Perspectives on High-Performance Mathematical Software

  • Daniela di Serafino
  • Lucia Maddalena
  • Paul Messina
  • Almerico Murli
Part of the Applied Optimization book series (APOP, volume 24)

Abstract

In this paper we trace the state of the art of high-performance mathematical software, that is, mathematical software for high-performance computing environments. Our overview is not meant to be exaustive; rather, we provide examples of software products and related projects, that are representative of the evolution aimed at exploiting the new features of advanced computing environments. We also discuss some issues concerning the design and implementation of mathematical software, that are introduced by the complex and highly varied nature of advanced computer architectures. Special attention is given to high-performance software for nonlinear optimization.

Keywords

Coherence Gall Tate Sorting Nash 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    E. Anderson, Z. Bai, C. Bishof, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, S. Ostrouchov, and D. Sorensen, LAPACK Users’ Guide,, Second Edition, SIAM, 1994. Software available at: http://www.netlib.org/lapack/.Google Scholar
  2. 2.
    T.E. Anderson, D.E. Culler, D.A. Patterson, and the NOW Team, A Case for Networks of Workstations: NOW, IEEE Micro, Feb. 1995. NOW Project home page: http://now.cs.berkeley.edu/. mcs.anl.gov/summar ies/minpack93/summary.html.Google Scholar
  3. 3.
    B.M. Averick and J.J. More, User Guide for the MINPACK-2 Test Problem Collection, Argonne National Laboratory, Mathematics and Computer Science Division, Tech. Memorandum no. 157, 1991. Information available at: http: //www.mcs.anl.gov/summar ies/minpack93/summary.html.Google Scholar
  4. 4.
    B.M. Averick and J.J. More, Evaluation of Large-Scale Optimization Problems on Vector and Parallel Architectures, SIAM J. on Optimization, vol. 4, pp. 708 - 721, 1994.MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    S. Balay, W. Gropp, L.C. Mclnnes and B. Smith, PETSc 2.0 Users Manual,Argonne National Laboratory, Argonne IL, 1996. Software available at: http://www.mcs.anl.gov/petsc/petsc.html.Google Scholar
  6. 6.
    BLAS Technical Forum, Document for the Basic Linear Algebra Subprograms (BLAS) Standard. Available at: http: //www.netlib.org/utk/papers/blast-forum.html.Google Scholar
  7. 7.
    BlockSolve. Information available at: http: //www.mcs.anl.gov/ sumaa3d/BlockSolve/ index.html. Software available by anonymous ftp from info.mcs.anl.gov in the directory pub/BlockSolve95.Google Scholar
  8. 8.
    N.J. Boden, D. Cohen, R.E. Felderman, A.E. Kulawik, C.L. Seitz, J.N. Seizovic, and W-K. Su, Myrinet: A Gigabit-per-Second Local-Area Network, IEEE-Micro, vol.15, no.l, pp.29-36, Feb. 1995. Information available at: http://www.myri.com.Google Scholar
  9. 9.
    R.H. Byrd, C.L. Dert, A.H.G. Rinnoy Kan, and R.B. Schnabel, Concurrent Stochastic Methods for Global Optimization, Math. Prog., vol. 46, pp. 1 - 29, 1990.MATHCrossRefGoogle Scholar
  10. 10.
    CAPSS. Software available at http://www.netlib.org/scalapack.Google Scholar
  11. 11.
    J. Choi, J. Demmel, I. Dhillon, J. Dongarra, S. Ostrouchov, A. Petitet, K. Stanley, D. Walker, and R.C. Whaley, ScaLAPACK: A Portable Linear Algebra Library for Distributed Memory Computers - Design Issues and Performance, Tech. Rep. CS-95-283, Computer Science Dept., Univ. of Tennessee, Knoxville, (LAPACK Working Note no. 95), 1995. Software available at http://www.netlib.org/scalapack/.Google Scholar
  12. 12.
    J. Choi, J. Dongarra, S. Ostrouchov, A. Petitet, D. Walker, and R.C. Whaley, A Proposal for a Set of Parallel Basic Linear Algebra Subprograms, Tech. Rep. CS-95-292, Computer Science Dept., Univ. of Tennessee, Knoxville, (LAPACK Working Note no. 100 ), 1995.Google Scholar
  13. 13.
    W.J. Cody, Observations on the Mathematical Software Effort, in W.R. Cowell (ed.), Sources and Development of Mathematical Software, Prentice-Hall, Inc., 1984.Google Scholar
  14. 14.
    A.R. Conn, N.I.M. Gould and Ph.L. Toint, Large-Scale Nonlinear Constrained Optimization: a Survey, in E. Spedicato (ed.), Algorithms for Continuous Optimization: the State of the Art, Series C: Mathematical and Physical Sciences, Kluwer Academic Publishers Group, vol. 434, pp. 287 - 332, 1994.Google Scholar
  15. 15.
    D.E. Culler, A. Arpaci-Dusseau, R. Arpaci-Dusseau, B. Chun, S. Lumetta, A. Mainwaring, R. Martin, C. Yoshikawa, and F. Wong. Parallel Computing on the Berkeley NOW, Proceedings of JSPP’97 (9th Joint Symposium on Parallel Processing), Kobe, Japan, 1997.Google Scholar
  16. 16.
    J. Czyzyk, M. Mesnier, and J. More, The Network-Enabled Optimization System (NEOS) Server, Preprint MCS-P615,-1906, 1996. NEOS Server home page: http://www.mcs.anl.gov/home/otc/Server/Google Scholar
  17. 17.
    R.D. da Cunha and T. Hopkins, PIM 1.1 - The Parallel Iterative Methods Package for Systems of Linear Equations User’s Guide, Computing Laboratory, Univ. of Kent at Canterbury, UK, 1994.Google Scholar
  18. 18.
    M. D’Apuzzo, V. De Simone, M. Marino, and G. Toraldo, Parallel Computational Issues for Box-Constrained Quadratic rogramming, Ricerca Operativa, vol. 27, no. 81, 1997.Google Scholar
  19. 19.
    J.M. Dembo and S.A. Zenios, GENOS 1.0 User’s Guide: A Generalized Network Optimization System, Report 87-13-03, Dept. of Decision Science, The Wharton School, Univ. of Pennsylvania, Philadelphia PA, USA, 1987.Google Scholar
  20. 20.
    J.W. Demmel, M.T. Heath, and H.A. van der Vorst, Parallel Numerical Linear Algebra, Acta Numerica, pp. 111 - 197, 1997.Google Scholar
  21. 21.
    D. di Serafino, L. Maddalena, and A. Murli, PINEAPL: A European Project to Develop a Parallel Numerical Library for Industrial Applications, in C. Lengauer, M. Griebl, and S. Gorlatch (eds.), Euro-Par’97 Parallel Processing, Lecture Notes in Computer Science, Springer, 1997. PINEAPL Project home page: http://www.nag. co.uk/projects/PINEAPL.html.Google Scholar
  22. 22.
    J.J. Dongarra, J.R. Bunch, C.B. Moler, and G.W. Stewart, LINPACK Users’ Guide, SIAM, Philadelphia, 1979. Software available at: http://www.netlib.org/linpack/.Google Scholar
  23. 23.
    J. Dongarra, J. Du Croz, S. Hammarling, and R.J. Hanson, An Extended Set of Fortran Basic Linear Algebra Subroutines, ACM Trans. Math. Soft., vol.14, no.l, pp. 1-17, 1988. Software available at: http://www.netlib.org/blas/.Google Scholar
  24. 24.
    J. Dongarra, J. Du Croz, I. Duff, and S. Hammarling, A Set of Level 3 Basic Linear Algebra Subprograms, ACM Trans. Math. Soft., vol.16, no.l, pp. 1-17, 1990. Software available at: http://www.netlib.org/blas/.Google Scholar
  25. 25.
    J.J. Dongarra and D.W. Walker, Software Libraries for Linear Algebra Computations on High-Performance Computers, SIAM Review, vol. 37, no. 2, pp. 151 - 180, 1995.MathSciNetCrossRefGoogle Scholar
  26. 26.
    J. Dongarra and R.C. Whaley, A User’s Guide to the BLACS vl.O, Tech. Rep. CS-95-281, Computer Science Dept., Univ. of Tennessee, Knoxville. (LAPACK Working Note no. 24), 1995. Software available at: http://www.netlib.org.Google Scholar
  27. 27.
    E. Eskow and R.B. Schnabel, Mathematical Modeling of a Parallel Global Optimization Algorithm, Par. Comp., vol. 12, pp. 315 - 325, 1989.Google Scholar
  28. 28.
    I. Foster and C. Kesselman, The Globus Project: A Status Report, Proc. IPPS/SPDP ‘98 Heterogeneous Computing Workshop, pp.4-18, 1998. Globus Project home page: http://www.globus.org.Google Scholar
  29. 29.
    G.C. Fox, Achievements and Prospects for Parallel Computing, in P. Messina and A. Murli (eds), Practical Parallel Computing. Status and Prospects, Special Issue of Concurrency: Practice and Experience, vol.3, no.6, pp. 725 - 739, 1991.Google Scholar
  30. 30.
    GAMS: Guide to Available Mathematical Software, National of Standards and Technology (NIST). GAMS home page: http://gams.nist.gov.Google Scholar
  31. 31.
    A. Griewank and Ph.L. Toint, Numerical Experiments with Partially Separable Optimization Problems, in A. Dold and B. Eckmann (eds.), Numerical Analysis, Lecture Notes in Mathematics, vol. 1066, Springer-Verlag, 1984.Google Scholar
  32. 32.
    A.S. Grimshaw, Wm.A.Wulf, and the Legion Team, The Legion Vision of a Worldwide Virtual Computer, Comm. of the ACM, vol.40, no.l, Jan. 1997. Legion Project home page: http: //www.cs.Virginia.eduAegion/Google Scholar
  33. 33.
    E.N. Houstis, J.R. Rice, S. Weerawarana, A.C. Catlin, P. Papachiou, K. -Y. Wang, and M. Gaitatzes, Parallel (//) ELLPACK: A Problem Solving Environment for PDE based Applications on Multicomputer Platforms. Available at: http://www.cs.purdue.edu/research/cse/pellpack/paper/pellpack-paper-1.html.Google Scholar
  34. 34.
    HPC-Netlib, National HPCC Software Exchange (NHSE). Available at: http://nhse.cs.utk. edu/hpc-netlib/.Google Scholar
  35. 35.
    High Performance Fortran (HPF). Software available at: http://www.netlib.org/hpf/.Google Scholar
  36. 36.
    Hewlett-Packard, HP MPL Document available at: http: // www.hp. com/wsg/ssa/mpi/mpihome.htmlGoogle Scholar
  37. 37.
    S. A. Hutchinson, J. N. Shadid, and R. S. Tuminaro, Aztec User’s Guide: Version 1.1, Sandia National Laboratories Technical Report, SAND95-1559 Oct., 1995. Software available at: http: //www.cs. sandia. gov/CRF/aztecl.html.Google Scholar
  38. 38.
    IBM Engineering and Scientific Subroutine Library for AIX (ESSL). Information available at: http: //www.rs6000. ibm. com/ software/Apps/essl.html.Google Scholar
  39. 39.
    C. Lawson, R. Hanson, D. Kincaid, and F. Krogh, Basic Linear Algebra Subprograms for Fortran Usage, ACM Trans. Math. Soft., vol.5, pp.308-323, 1979. Software available at: http://www.netlib.org/blas/.Google Scholar
  40. 40.
    Macsyma, Macsyma Inc., 20 Academy St., Arlington, MA 02174-6436, USA. http: //www.macsyma. com.Google Scholar
  41. 41.
    R.P. Martin, A.M. Vahdat, D.E. Culler, and T.E. Anderson, Effects of Communication Latency, Overhead, and Bandwidth in a Cluster Architecture, ISCA 24, Denver, USA, June 1997.Google Scholar
  42. 42.
    K. Maschhoff and D. Sorensen, A Portable Implementation of ARPACK for Distributed Memory Parallel Architectures, Preliminary proceedings, Copper Mountain Conference on Iterative Methods, 1996. Information available at: http: //www.caam.rice. edu/kristyn/parpack_home.html. Software available at: ftp://ftp.caam.rice.edu/pub/software/ARPACK.Google Scholar
  43. 43.
    Mathematical Software collection, available through the Math-Net Links to the Mathematical World. Home page: http:// e l i b. z i b - b e r l i n. d e:88/Math-Net/Links/math-web. soft.html.Google Scholar
  44. 44.
    MatLab: Matrix Laboratory, The MathWorks, Inc., 24 Prime Park Way, Natick, MA 01760. http://www.mathworks.com.Google Scholar
  45. 45.
    P. Messina, High-Performance Computers: The Next Generation (Part I), Computers in Physics, vol. 11, no. 5, pp. 453 - 466, 1997.Google Scholar
  46. 46.
    P. Messina and T. Mihaly-Pauna (eds), CAS A Gigabit Network Testbed: Final Report, Tech. Rep. CACR-123, July 1996. Available at: http: //www.cacr. caltech.edu/publications/techpubs/.Google Scholar
  47. 47.
    P. Messina, S. Brunett, T. Gottschalk, and C. Kesselman, Implementing Large-Scale Distributed Synthetic Forces Simulations on Top of Metacomputing Software Infrastructure, Proceedings of 7th Heterogeneous Computing Workshop, IEEE Computer Society Press, Mar. 1998.Google Scholar
  48. 48.
    J.J. More and S.J. Wright, Optimization Software Guide, SIAM, 1993.Google Scholar
  49. 49.
    NAG Fortran Library Manual, Mark 16, Numerical Algorithms Group Ltd., Oxford, England, UK, 1994. NAG home page: http://www.nag.co.uk/.Google Scholar
  50. 50.
    NAG Parallel Library Manual, Release 2, NAG ltd., Oxford, England, UK, 1997.Google Scholar
  51. 51.
    S.G. Nash and A. Sofer, Block Truncated-Newton Methods for Parallel Optimization., Math. Prog., vol. 45, pp. 529 - 546, 1989.MathSciNetMATHCrossRefGoogle Scholar
  52. 52.
    S.G. Nash and A. Sofer, BTN: Software for Parallel Unconstrained Optimization, ACM Trans. Math. Soft., vol. 18, pp. 414 - 448, 1992.MATHCrossRefGoogle Scholar
  53. 53.
    Netlib Repository, University of Tennessee - Knoxville and Oak Ridge National Laboratories. Home page: ttp://www.netlib.org.Google Scholar
  54. 54.
    NHSE: National HPCC Software Exchange. Home page: http://www.nhse.org.Google Scholar
  55. 55.
    S.S. Nielsen and S.A. Zenios, A Massively Parallel Algorithm for Nonlinear Stochastic Network Problems, Operations Research, vol. 41, no. 2, pp. 319 - 337, Mar.-Apr. 1993.MathSciNetMATHGoogle Scholar
  56. 56.
    ParPre: Parallel Preconditioners for Iterative Methods. Information available at: http: //www.math.ucla. edu/eijkhout/parpre.html.Google Scholar
  57. 57.
    PDELab: An Object-Oriented Framework for Building Problem Solving Environments for PDE Based Applications, Tech. Rep. CSD-TR-94-021, Dept. of Computer Science, Purdue University, 1994. PDELab Project home page: http: //www.cs.purdue. edu/research/cse/pdelab/pdelab.html.Google Scholar
  58. 58.
    Parallel ELLPACK (//ELLPACK). Home page: http: //www.cs.purdue. edu/research/cse/pellpack/pellpack.html.Google Scholar
  59. 59.
    IBM Parallel Engineering and Scientific Subroutine Library (Parallel ESSL) for Advanced Interactive Executive (AIX*). Information available at: http:///www.rs6000. ibm. com/software/sp_products/esslpara.html.Google Scholar
  60. 60.
    Parallel Optimization and Automatic Differentiation (POAD) Project, CRPC Parallel Optimization group, Rice University. Home page: http: //www.crpc. r i c e. edu/CRPC/brochure/res_poad.html.Google Scholar
  61. 61.
    Paragon System Parallel Fast Fourier Transform Library Manual, Intel, 1995.Google Scholar
  62. 62.
    PSEware, NSF. PSEware Project home page: http: //www.extreme. indisoia.edu/pseware/.Google Scholar
  63. 63.
    J.R. Rice, Announcement and Call for Papers, Mathematical Software, SIGNUM Letter 4 (3): 7, 1969.Google Scholar
  64. 64.
    J.R. Rice, Mathematical Software, Academic Press, Inc., 1971.Google Scholar
  65. 65.
    J.R. Rice and R.F. Boisvert, Solving Elliptic Problems Using ELLPACK, Spring-Verlag, New York, 1985. Information available at: http://www.cs.purdue.edu/ellpack.Google Scholar
  66. 66.
    J.R. Rice, Numerical Methods, Software and Analysis, Second Edition, Academic Press, Inc., 1993.Google Scholar
  67. 67.
    J.R. Rice and R.F. Boisvert, From Scientific Software Libraries to Problem Solving Environments, IEEE Compu. Science and; Eng., Fall 1996.Google Scholar
  68. 68.
    D. Ridge, D. Becker, P. Merkey, and T. Sterling, Beowulf: Harnessing the Power of Parallelism in a Pile-of-PCs, Proceedings, IEEE Aerospace, 1997. Beowulf Project home page: http://cesdis.gsfc.nasa.gov/beowulf.Google Scholar
  69. 69.
    R.B. Schnabel, A View of the Limitations, Opportunities, and Challenges in Parallel Nonlinear Optimization, Par. Comp., vol. 21, pp. 875 - 905, 1995.Google Scholar
  70. 70.
    Silicon Graphics, Message-Passing Toolkit (MPT), Document available at: http: //www.sgi. com/Products./software/mpt.html.Google Scholar
  71. 71.
    B.T. Smith, J.M. Boyle, B.S. Garbow, Y. Ikebe, V.C. Klema, and C.B. Moler, Matrix Eigensystem Routines - EISPACK Guide, Springer-Verlag, New York, 1974. Software available at: http://www.netlib.org/eispack/.Google Scholar
  72. 72.
    M. Snir, S. Otto, S.H. Lederman, D. Walker, and J. Dongarra, MPI: The Complete Reference, MIT Press, 1996.Google Scholar
  73. 73.
    Transactions on Mathematical Software, ACM. Software available at: http://www.netlib.org/toms/.Google Scholar
  74. 74.
    V. Torczon, PDS: Direct Search Methods for Unconstrained Optimization on Either Sequential or Parallel Machines. Information available at: http://softlib.rice.edu/softlib/catalog/pds.html.Google Scholar
  75. 75.
    P. J.M. van Laarhoven, Parallel Variable Metric Algorithms for Unconstrained Optimization, Math. Prog., vol.33, no.l, pp. 68 - 81, 1985.Google Scholar
  76. 76.
    M.H. Wright, Some Linear Algebra Issues in Large-Scale Optimization, Proceed of the NATO ASI Conference Linear Algebra for Large-Scale and Real-Time Applications, Kluwer Academic Publishers, 1993.Google Scholar
  77. 77.
    S. Wolfram, Mathematica: A System for Doing Mathematics by Computer, Second Edition, Addison-Wesley, Redwood City, CA, 1991. Mathematica home page: http:/ /www.mathematica. com.Google Scholar
  78. 78.
    D. Yang and S.A. Zenios, A Scalable Parallel Interior Point Algorithm for Stochastic Linear Programming and Robust Optimization, Computational Optimization and Applications, vol.7, no.l, pp. 143 - 158, Jan. 1997.MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers, Boston 1998

Authors and Affiliations

  • Daniela di Serafino
    • 1
    • 2
    • 5
  • Lucia Maddalena
    • 2
    • 5
  • Paul Messina
    • 3
  • Almerico Murli
    • 2
    • 3
    • 4
  1. 1.The Second University of NaplesCasertaItaly
  2. 2.Center for Research on Parallel Computing and SupercomputersNaplesItaly
  3. 3.Center for Advanced Computing ResearchCalifornia Institute of TechnologyPasadenaUSA
  4. 4.University of Naples “Federico II”NaplesItaly
  5. 5.Complesso Monte S. AngeloCentro di Ricerche per il Calcolo Parallelo e i Supercalcolatori (CPS-CNR)NaplesItaly

Personalised recommendations