Skip to main content
SpringerLink
Log in

A Formalisation in HOL of the Fundamental Theorem of Linear Algebra and Its Application to the Solution of the Least Squares Problem

  • Published:
Journal of Automated Reasoning Aims and scope Submit manuscript

Abstract

In this paper we show how a thoughtful reusing of libraries can provide concise proofs of non-trivial mathematical results. Concretely, we formalise in Isabelle/HOL a proof of the Fundamental Theorem of Linear Algebra for vector spaces over inner product spaces, the Gram–Schmidt process of orthogonalising vectors over \(\mathbb {R}\), its application to get the \({ QR}\) decomposition of a matrix, and the least squares approximation of systems of linear equations without solution, in a modest number of lines (ca. 2700). This work intensively reuses previous results, such as the Rank–Nullity theorem and various applications of the Gauss–Jordan algorithm. The formalisation is also accompanied by code generation and refinements that enable the execution of the presented algorithms in Isabelle and SML.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Every finite set can be equipped with a well-order, but they are represented by means of different type classes in the Isabelle library.

  2. The proof can be completed in one single line of Isabelle code, but we usually favour Isar human-readable proofs [54].

  3. Note that the type system takes care of the elements of type vec being of equal size and this assumption can then be avoided in the refinement to immutable arrays.

  4. The benchmarks have been carried out in laptop with an Intel Core i5-3360M processor, 4 GB of RAM, PolyML 5.5.2-3 and Ubuntu 14.04.

References

  1. Adelsberger, S., Hetzl, S., Pollak, F.: The Cayley–Hamilton theorem. Arch. Form. Proofs (2014). http://afp.sf.net/entries/Cayley_Hamilton.shtml, Formal proof development

  2. Aehlig, K., Haftmann, F., Nipkow, T.: A compiled implementation of normalization by evaluation. J. Funct. Program. 22(1), 9–30 (2012)

    Article  MATH  Google Scholar 

  3. Afshar, S.K., Aravantinos, V., Hasan, O., Tahar, S.: Formalization of complex vectors in higher-order logic. In: Watt, S.M., Davenport, J.H., Sexton, A.P., Sojka, P., Urban, J. (eds.) Intelligent Computer Mathematics: CICM 2014. Proceedings, Lecture Notes in Artificial Intelligence, vol. 8543, pp. 123–137. Springer, Berlin (2014)

  4. Aransay, J., Divasón, J.: Formalization and execution of linear algebra: from theorems to algorithms. In: Gupta, G., Peña, R. (eds.) Post Proceedings of the International Symposium on Logic-Based Program Synthesis and Transformation: LOPSTR 2013. Lecture Notes in Computer Science, vol. 8901, pp. 1–19. Springer, Berlin (2014)

    Google Scholar 

  5. Aransay, J., Divasón, J.: Formalisation in higher-order logic and code generation to functional languages of the Gauss–Jordan algorithm. J. Funct. Program. 25, 1–21 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. Aransay, J., Divasón, J.: Generalizing a mathematical analysis library in Isabelle/HOL. In: Havelund, K., Holzmann, G., Joshi, R. (eds.) NASA Formal Methods: NFM 2015, Lecture Notes in Computer Science, vol. 9508, pp. 415–421. Springer, Berlin (2015)

  7. Aransay, J., Divasón, J.: Formalisation of the computation of the Echelon form of a matrix in Isabelle/HOL. Form. Asp. Comput. (accepted for publication) (2016)

  8. Aransay, J., Divasón, J.: Verified Computer Linear Algebra. Accepted for Publication in the Conference EACA 2016 (2016). https://www.unirioja.es/cu/jearansa/archivos/vcla.pdf

  9. Björck, A.: Numerical Methods for Least Squares Problems. SIAM (1996)

  10. Blanchette, J., Haslbeck, M., Matichuk, D., Nipkow, T.: Mining the archive of formal proofs. In: Kerber, M. (ed.) Conference on Intelligent Computer Mathematics: CICM 2015, Lecture Notes in Computer Science, vol. 9150, pp. 3–17. Springer, Berlin (2015). Invited paper

  11. Boldo, S., Jourdan, J., Leroy, X., Melquiond, G.: Verified compilation of floating-point computations. J. Autom. Reason. 54(2), 135–163 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  12. Boldo, S., Lelay, C., Melquiond, G.: Formalization of real analysis: a survey of proof assistants and libraries. Math. Struct. Comput. Sci. FirstView, 1–38 (2016). doi:10.1017/S0960129514000437. http://journals.cambridge.org/articleS0960129514000437

  13. Butler, R.B.: Formalization of the Integral Calculus in the PVS Theorem Prover. Tech. Rep. NASA/TM-2004-213279, L-18391, NASA Langley Research Center (2004). http://ntrs.nasa.gov/search.jsp?R=20040171869

  14. Chang, W., Yamazaki, H., Nakamura, Y.: A theory of matrices of complex elements. Form. Math. 13(1), 157–162 (2005). http://fm.mizar.org/2005-13/pdf13-1/matrix_5.pdf

  15. Chang, W., Yamazaki, H., Nakamura, Y.: The inner product and conjugate of matrix of complex numbers. Form. Math. 13(4), 493–499 (2005). http://fm.mizar.org/2005-13/pdf13-4/matrixc1.pdf

  16. Cohen, C., Dénès, M., Mörtberg, A.: Refinements for free! In: Gonthier, G., Norrish, M. (eds.) Certified Programs and Proofs: CPP 2013, Lecture Notes in Computer Science, vol. 8307, pp. 147–162. Springer, Berlin (2013)

  17. Dahlquist, G., Björck, A.: Numerical Methods in Scientific Computing. SIAM (2008)

  18. Daumas, M., Lester, D., Muñoz, C.: Verified real number calculations: a library for interval arithmetic. IEEE Trans. Comput. 58(2), 226–237 (2009)

    Article  MathSciNet  Google Scholar 

  19. Dénès, M., Mörtberg, A., Siles, V.: A refinement-based approach to computational algebra in COQ. In: Beringer, L., Felty, A. (eds.) Interactive Theorem Proving: ITP 2012, Lecture Notes in Computer Science, vol. 7406, pp. 83–98. Springer, Berlin (2012)

  20. Divasón, J., Aransay, J.: Rank–Nullity theorem in linear algebra. Arch. Form. Proofs (2013). http://afp.sf.net/entries/Rank_Nullity_Theorem.shtml

  21. Divasón, J., Aransay, J.: Gauss–Jordan algorithm and its applications. Arch. Form. Proofs (2014). http://afp.sf.net/entries/Gauss_Jordan.shtml, Formal proof development

  22. Divasón, J., Aransay, J.: Echelon form. Arch. Form. Proofs (2015). http://afp.sf.net/entries/EchelonForm.shtml, Formal proof development

  23. Divasón, J., Aransay, J.: \(QR\) decomposition. Arch. Form. Proofs (2015). http://afp.sf.net/entries/QRDecomposition.shtml, Formal proof development. Updated version available from http://afp.sf.net/devel-entries/QRDecomposition.shtml

  24. Dutertre, B.: Elements of mathematical analysis in PVS. In: von Wright, J., Grundy, J., Harrison, J. (eds.) Theorem Proving in Higher Order Logics: TPHOLs 97. Lecture Notes in Computer Science, vol. 1125, pp. 141–156. Springer, Turku (1996)

    Chapter  Google Scholar 

  25. Gallego-Arias, E.J., Jouvelot, P.: Adventures in the (Not So) Complex Space. The Coq Workshop 2015 (2015). https://github.com/ejgallego/mini-dft-coq

  26. Gonthier, G.: Point-free, set-free concrete linear algebra. In: van Eekelen, M., Geuvers, H., Schmaltz, J., Wiedijk, F. (eds.) Interactive Theorem Proving: ITP 2011, Lecture Notes in Computer Science, vol. 6898, pp. 103–118. Springer, Berlin (2011)

  27. Gonthier, G., Asperti, A., Avigad, J., Bertot, Y., Cohen, C., Garillot, F., Roux, S.L., Mahboubi, A., O’Connor, R., Biha, S.O., Pasca, I., Rideau, L., Solovyev, A., Tassi, E., Théry, L.: A machine-checked proof of the odd order theorem. In: Blanzy, S., Paulin-Mohring, C., Pichardie, D. (eds.) Interactive Theorem Proving: ITP 2013, Lecture Notes in Computer Science, vol. 7998, pp. 163–179. Springer, Berlin (2013)

  28. Haftmann, F.: Code Generation from Isabelle/HOL Theories. http://isabelle.in.tum.de/doc/codegen.pdf (2016)

  29. Haftmann, F., Krauss, A., Kuncar, O., Nipkow, T.: Data refinement in Isabelle/HOL. In: Blazy, S., Paulin-Mohring, C., Pichardie, D. (eds.) Interactive Theorem Proving: ITP 2013, Lecture Notes in Computer Science, vol. 7998, pp. 100–115. Springer, Berlin (2013)

  30. Haftmann, F., Nipkow, T.: Code generation via higher-order rewrite systems. In: Blume, M., Kobayashi, N., Vidal, G. (eds.) Functional and Logic Programming: FLOPS 2010, Lecture Notes in Computer Science, vol. 6009, pp. 103–117. Springer, Berlin (2010)

  31. Haftmann, F., Wenzel, M.: Constructive type classes in Isabelle. In: Altenkirch, T., McBride, C. (eds.) Types for Proofs and Programs: TYPES 2006, Revised Selected Papers, Lecture Notes in Computer Science, vol. 4502, pp. 160–174. Springer, Berlin (2007)

  32. Hales, T., Adams, M., Bauer, G., Dang, D., Harrison, J., Hoang, T.L., Kaliszyk, C., Magron, V., McLaughlin, S., Nguyen, T.T., Nguyen, T.Q., Nipkow, T., Obua, S., Pleso, J., Rute, J., Solovyev, A., Ta, A.H.T., Tran, T.N., Trieu, D.T., Urban, J., Vu, K.K., Zumkeller, R.: A Formal Proof of the Kepler Conjecture. http://arxiv.org/abs/1501.02155 (2015)

  33. Harrison, J.: A HOL theory of Euclidean space. In: Hurd, J., Melham, T. (eds.) Theorem Proving in Higher Order Logics: TPHOLS 2005, Lecture Notes in Computer Science, vol. 3603, pp. 114–129. Springer, Berlin (2005)

  34. Harrison, J.: The HOL light theory of euclidean space. J. Autom. Reason. 50(2), 173–190 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  35. Hölzl, J.: Proving inequalities over reals with computation in Isabelle/HOL. In: Reis, G.D., Théry, L. (eds.) International Workshop on Programming Languages for Mechanized Mathematics Systems: PLMMS’09, pp. 38–45. Munich (2009)

  36. Hölzl, J., Immler, F., Huffman, B.: Type classes and filters for mathematical analysis in Isabelle/HOL. In: Blazy, S., Paulin-Mohring, C., Pichardie, D. (eds.) Interactive Theorem Proving: ITP 2013, Lecture Notes in Computer Science, vol. 7998, pp. 279–294. Springer, Berlin (2013)

  37. HOL Multivariate Analysis Library. http://isabelle.in.tum.de/library/HOL/HOL-Multivariate_Analysis/index.html (2016)

  38. Huffman, B., Kunčar, O.: Lifting and transfer: a modular design for quotients in Isabelle/HOL. In: Gonthier, G., Norrish, M. (eds.) Certified Programs and Proofs: CPP 2013, Lecture Notes in Computer Science, vol. 8307, pp. 131–146. Springer, Berlin (2013)

  39. Klein, G., Andronick, J., Elphinstone, K., Heiser, G., Cock, D., Derrin, P., Elkaduwe, D., Engelhardt, K., Kolanski, R., Norrish, M., Sewell, T., Tuch, H., Winwood, S.: seL4: formal verification of an operating-system kernel. Commun. ACM 53(6), 107–115 (2010)

    Article  Google Scholar 

  40. Lammich, P.: Automatic data refinement. In: Blazy, S., Paulin-Mohring, C., Pichardie, D. (eds.) Interactive Theorem Proving: ITP 2013. Lecture Notes in Computer Science, vol. 7998, pp. 84–99. Springer, Berlin (2013)

    Chapter  Google Scholar 

  41. Lester, D.R.: Real number calculations and theorem proving. In: Mohamed, O.A., Muñoz, C., Tahar, S. (eds.) Theorem Proving in Higher Order Logics: TPHOLs 08. Lecture Notes in Computer Science, vol. 5170, pp. 215–229. Springer, Berlin (2008)

    Chapter  Google Scholar 

  42. Martin-Dorel, É., Melquiond, G.: Proving tight bounds on univariate expressions with elementary functions in Coq. J. Autom. Reason. 1–31 (2015). doi:10.1007/s10817-015-9350-4

  43. Mathematica 10.4. Wolfram Research, Inc. Champaign, IL (2016)

  44. Nipkow, T., Paulson, L.C., Wenzel, M.: Isabelle/HOL: A Proof Assistant for Higher-Order Logic, Lecture Notes in Computer Science, vol. 2283. Springer, Berlin (2002). Updated version available in http://isabelle.in.tum.de/doc/tutorial.pdf

  45. Obua, S., Nipkow, T.: Flyspeck II: the basic linear programs. Ann. Math. Artif. Intell. 56, 245–272 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  46. Pąk, K.: Jordan matrix decomposition. Form. Math. 16(4), 297–303 (2008). doi:10.2478/v10037-008-0036-9

    Google Scholar 

  47. Solovyev, A., Hales, T.: Efficient formal verification of bounds of linear programs. In: Intelligent Computer Mathematics, Lecture Notes in Computer Science, vol. 6824, pp. 123–132. Springer, Berlin (2011)

  48. Solovyev, A., Hales, T.: Formal verification of nonlinear inequalities with Taylor interval approximations. In: NASA Formal Methods, Lecture Notes in Computer Science, vol. 7871, pp. 383–397. Springer, Berlin (2013)

  49. Strang, G.: The fudamental theorem of linear algebra. Am. Math. Mon. 100(9), 848–855 (1993)

    Article  MATH  Google Scholar 

  50. Strang, G.: Introduction to Linear Algebra, 4th edn. Wellesley-Cambridge Press, Cambridge (2009)

    MATH  Google Scholar 

  51. Thiemann, R.: Implementing field extensions of the form \(\mathbb{Q} [\sqrt{b}]\). Arch. Form. Proofs (2014). http://afp.sf.net/entries/Real_Impl.shtml, Formal proof development

  52. Thiemann, R., Yamada, A.: Matrices, Jordan normal forms, and spectral radius theory. Arch. Form. Proofs (2015). http://afp.sf.net/entries/Jordan_Normal_Form.shtml, Formal proof development

  53. Thiemann, R., Yamada, A.: Algebraic Numbers in Isabelle/HOL (2016). Accepted for presentation in ITP 2016

  54. Wenzel, M.: Isabelle/Isar—A Versatile Environment for Human-Readable Formal Proof Documents. Ph.D. thesis, Technische Universität München (2002). https://mediatum.ub.tum.de/doc/601724/601724.pdf

Download references

Acknowledgments

The authors would like to thank the anonymous referees because of their valuable contributions along the review process. Particularly, their suggestions helped us to improve the overall clarity of the presentation and the related work section. This work has been partially supported by the research Grant FPI-UR-12, from Universidad de La Rioja and by the project MTM2014-54151-P from Ministerio de Economía y Competitividad (Gobierno de España).

Author information

Authors and Affiliations

  1. Departamento de Matemáticas y Computación, Universidad de La Rioja, Edif. Luis Vives, c/ Luis de Ulloa n. 2, Logroño, Spain

    Jesús Aransay & Jose Divasón

Authors
  1. Jesús Aransay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jose Divasón
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jesús Aransay.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aransay, J., Divasón, J. A Formalisation in HOL of the Fundamental Theorem of Linear Algebra and Its Application to the Solution of the Least Squares Problem. J Autom Reasoning 58, 509–535 (2017). https://doi.org/10.1007/s10817-016-9379-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10817-016-9379-z

Keywords

Search

Navigation