Formal Aspects of Computing

, Volume 25, Issue 6, pp 933–945 | Cite as

Deriving dense linear algebra libraries

  • Paolo Bientinesi
  • John A. Gunnels
  • Margaret E. Myers
  • Enrique S. Quintana-Ortí
  • Tyler Rhodes
  • Robert A. van de GeijnEmail author
  • Field G. Van Zee
Original Article


Starting in the late 1960s computer scientists including Dijkstra and Hoare advocated goal-oriented programming and the formal derivation of algorithms. The chief impediment to realizing this for loop-based programs was that a priori determination of loop-invariants, a prerequisite for developing loops, was a task too complex for any but the simplest of operations. Around 2000, these techniques were for the first time successfully applied to the domain of high-performance dense linear algebra libraries. This has led to a multitude of papers (mostly published in the ACM Transactions for Mathematical Software), a system for the mechanical derivation of algorithms, and a high-performance linear algebra library, \({\tt libflame}\) , that includes more than a thousand variants of algorithms for more than a hundred linear algebra operations. To our knowledge, this success story has unfolded with limited awareness on the part the formal methods community. This paper reports on ten years of experience and is meant to raise that awareness.


Formal derivation Linear algebra libraries Scientific computing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. ABB+99.
    Anderson E, Bai Z, Bischof C, Blackford LS, Demmel J, Dongarra JJ, Du Croz J, Hammarling S, Greenbaum A, McKenney A, Sorensen D (1999) LAPACK users’ guide, 3rd edn. SIAM, PhiladelphiaCrossRefGoogle Scholar
  2. BGvdG.
    Bientinesi P, Gunter B, Vande Geijn RA (2008) Families of algorithms related to the inversion of a symmetric positive definite matrix. ACM Trans Math Softw 35(1): 1–22MathSciNetCrossRefGoogle Scholar
  3. BGM+05.
    Bientinesi P, Gunnels JA, Myers ME, Quintana-Ortí ES, van de Geijn RA (2005) The science of deriving dense linear algebra algorithms. ACM Trans Math Softw 31(1): 1–26CrossRefGoogle Scholar
  4. Bie06.
    Bientinesi P (2005) Mechanical derivation and systematic analysis of correct linear algebra algorithms. PhD thesis, Department of Computer Sciences, The University of Texas. Technical report TR-06-46Google Scholar
  5. BQOvdG05.
    Bientinesi P, Quintana-Ortí ES, van de Geijn RA (2005) Representing linear algebra algorithms in code: the FLAME application programming interfaces. ACM Trans Math Softw 31(1): 27–59CrossRefGoogle Scholar
  6. BvdG11.
    Bientinesi P, van de Geijn RA (2011) Goal-oriented and modular stability analysis. SIAM J Matrix Anal Appl 32(1): 286–308MathSciNetCrossRefzbMATHGoogle Scholar
  7. DDCHD90.
    Dongarra JJ, Du Croz J, Hammarling S, Duff I (1990) A set of level 3 basic linear algebra subprograms. ACM Trans Math Softw 16(1): 1–17CrossRefzbMATHGoogle Scholar
  8. DDCHH88.
    Dongarra JJ, Du Croz J, Hammarling S, Hanson RJ (1988) An extended set of FORTRAN basic linear algebra subprograms. ACM Trans Math Softw 14(1): 1–17CrossRefzbMATHGoogle Scholar
  9. Dij68.
    Dijkstra EW (1968) A constructive approach to the problem of program correctness. BIT 8: 174–186CrossRefzbMATHGoogle Scholar
  10. Dij76.
    Dijkstra EW (1976) A discipline of programming. Prentice HallGoogle Scholar
  11. EBvdG10.
    Eijkhout V, Bientinesi P, van de Geijn R (2010) Towards mechanical derivation of Krylov solver libraries. Procedia Comput Sci 1(1): 1799–1807 Proceedings of ICCS 2010CrossRefGoogle Scholar
  12. FTB11a.
    Fabregat-Traver D, Bientinesi P (2011) Automatic generation of loop-invariants for matrix operations. In: Proceedings of the 11th international conference on computational science and its applications (to appear). Also TR AICES-2010/02-1, AICES, RWTH AachenGoogle Scholar
  13. FTB11b.
    Fabregat-Traver D, Bientinesi P (2011) Knowledge-based automatic generation of partitioned matrix expressions. In: Proceedings of the 13th international workshop on computer algebra in scientific computing (to appear). Also TR AICES-2010/01-3, AICES, RWTH AachenGoogle Scholar
  14. GGHvdG01.
    Gunnels JA, Gustavson FG, Henry GM, van de Geijn RA (2001) FLAME: formal linear algebra methods environment. ACM Trans Math Softw 27(4): 422–455CrossRefzbMATHGoogle Scholar
  15. Gri81.
    Gries D (1981) The science of programming. Springer, New YorkCrossRefzbMATHGoogle Scholar
  16. Gun01.
    Gunnels JA (2001) A systematic approach to the design and analysis of parallel dense linear algebra algorithms. PhD thesis, Department of Computer Sciences, The University of TexasGoogle Scholar
  17. GvdG01.
    Gunnels JA, van de Geijn RA (2001) Formal methods for high-performance linear algebra libraries. In: Boisvert RF, Tang PTP (eds) The architecture of scientific software. Kluwer, Dordrecht, pp 193–210CrossRefGoogle Scholar
  18. GvdG08a.
    Goto K, van de Geijn R (2008) High-performance implementation of the level-3 BLAS. ACM Trans Math Softw 35(1): 1–14CrossRefzbMATHGoogle Scholar
  19. GvdG08b.
    Goto K, van de Geijn RA (2008) Anatomy of high-performance matrix multiplication. ACM Trans Math Softw 34(3): 1–25CrossRefGoogle Scholar
  20. Hoa69.
    Hoare CAR (1969) An axiomatic basis for computer programming. Commun ACM 12: 576–580CrossRefzbMATHGoogle Scholar
  21. Kha02.
    Khalil HK (2002) Nonlinear systems, 3rd edn. Prentice-Hall, Upper Saddle River NYzbMATHGoogle Scholar
  22. LHKK79.
    Lawson CL, Hanson RJ, Kincaid DR, Krogh FT (1979) Basic linear algebra subprograms for Fortran usage. ACM Trans Math Softw 5(3): 308–323CrossRefzbMATHGoogle Scholar
  23. MLB87.
    Moler C, Little J, Bangert S (1987) Pro-Matlab, user’s guide. The Mathworks, Inc.Google Scholar
  24. PvdGB.
    Poulson J, van de Geijn R, Bennighof J (2011) Parallel algorithms for reducing the generalized Hermitian-definite eigenvalue problem. FLAME Working Note #56. The University of Texas at Austin, Department of Computer Science. Technical Report TR-11-05Google Scholar
  25. QOvdG03.
    Quintana-Ortí ES, van de Geijn RA (2003) Formal derivation of algorithms: the triangular Sylvester equation. ACM Trans Math Softw 29(2): 218–243CrossRefzbMATHGoogle Scholar
  26. vdGQO08.
    van de Geijn RA, Quintana-Ortí ES (2008) The science of programming matrix computations.
  27. VZCvdG+09.
    Van Zee FG, Chan E, van de Geijn R, Quintana-Ortí ES, Quintana-Ortí G (2009) Introducing: the libflame library for dense matrix computations. IEEE Comput Sci Eng 11(6): 56–62CrossRefGoogle Scholar
  28. VanZee09.
    Van Zee FG (2009) \({\tt libflame}\) : the complete reference.

Copyright information

© British Computer Society 2012

Authors and Affiliations

  • Paolo Bientinesi
    • 1
  • John A. Gunnels
    • 2
  • Margaret E. Myers
    • 3
  • Enrique S. Quintana-Ortí
    • 4
  • Tyler Rhodes
    • 3
  • Robert A. van de Geijn
    • 3
    Email author
  • Field G. Van Zee
    • 3
  1. 1.RWTH Aachen UniversityAachenGermany
  2. 2.Department of Business Analytics & Mathematical SciencesIBM T.J. Watson Research CenterYorktown HeightsUSA
  3. 3.Department of Computer ScienceThe University of Texas atAustinAustinUSA
  4. 4.Universidad Jaime ICastellónSpain

Personalised recommendations