Skip to main content
SpringerLink
Log in

A Derivative-Free Nonlinear Least Squares Solver

  • Conference paper
  • First Online:
Optimization and Applications (OPTIMA 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 13078))

Included in the following conference series:

Abstract

A nonlinear least squares iterative solver developed earlier by the author is modified to fit the derivative-free optimization paradigm. The proposed algorithm is based on easily parallelizable computational kernels such as small dense matrix factorizations and elementary vector operations and therefore has a potential for a quite efficient implementation on modern high-performance computers. Numerical results are presented for several standard test problems to demonstrate the competitiveness of the proposed method.

Supported by RFBR grant No.19-01-00666.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Armijo, L.: Minimization of functions having Lipschitz continuous first partial derivatives. Pac. J. Math. 16(1), 1–3 (1966)

    Article  MathSciNet  Google Scholar 

  2. Averick, B.M., Carter, R.G., Xue, G.L., More, J.J.: The MINPACK-2 test problem collection (No. ANL/MCS-TM-150-Rev.). Argonne National Lab., IL (United States) (1992)

    Google Scholar 

  3. Ballard, G., Ikenmeyer, C., Landsberg, J.M., Ryder, N.: The geometry of rank decompositions of matrix multiplication II: 3 \(\times \) 3 matrices. J. Pure Appl. Algebra 223(8), 3205–3224 (2019)

    Article  MathSciNet  Google Scholar 

  4. Brent, R.P.: Algorithms for matrix multiplication (No. STAN-CS-70-157). Stanford University CA Department of Computer Science, p. 58 (1970)

    Google Scholar 

  5. Brown, P.N.: A local convergence theory for combined inexact-Newton/finite-difference projection methods. SIAM J. Numer. Anal. 24(2), 407–434 (1987)

    Article  MathSciNet  Google Scholar 

  6. Brown, P.N., Saad, Y.: Hybrid Krylov methods for nonlinear systems of equations. SIAM J. Sci. Stat. Comput. 11(3), 450–481 (1990)

    Article  MathSciNet  Google Scholar 

  7. Brown, P.N., Saad, Y.: Convergence theory of nonlinear Newton-Krylov algorithms. SIAM J. Optim. 4(2), 297–330 (1994)

    Article  MathSciNet  Google Scholar 

  8. Kaporin, I.E.: Esimating global convergence of inexact Newton methods via limiting stepsize along normalized direction, Report 9329, Department of Mathematics, Catholic University of Nijmegen, Nijmegen, The Netherland, p. 8, July 1993

    Google Scholar 

  9. Kaporin, I.E.: The use of preconditioned Krylov subspaces in conjugate gradient type methods for the solution of nonlinear least square problems. (Russian) Vestnik Mosk. Univ. Ser. (Comput. Math. Cybern.) 15(3), 26–31 (1995)

    Google Scholar 

  10. Kaporin, I.E., Axelsson, O.: On a class of nonlinear equation solvers based on the residual norm reduction over a sequence of affine subspaces. SIAM J. Sci. Comput. 16(1), 228–249 (1994)

    Article  MathSciNet  Google Scholar 

  11. Kaporin, I.: Preconditioned subspace descent method for nonlinear systems of equations. Open Comput. Sci. 10(1), 71–81 (2020)

    Article  Google Scholar 

  12. Kazeev, V.A., Tyrtyshnikov, E.E.: Structure of the Hessian matrix and an economical implementation of Newton’s method in the problem of canonical approximation of tensors. Comput. Math. Math. Phys. 50(6), 927–945 (2010)

    Article  MathSciNet  Google Scholar 

  13. Laderman, J.D.: A noncommutative algorithm for multiplying 3 \(\times \) 3 matrices using 23 multiplications. Bull. Am. Math. Soc. 82(1), 126–128 (1976)

    Article  MathSciNet  Google Scholar 

  14. More, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. Argonne National Laboratory. Appl. Math. Division Tech. Memorandum 324, 96 (1978)

    Google Scholar 

  15. Northby, J.A.: Structure and binding of Lennard-Jones clusters: \(13\le N\le 147\). The J. Chem. Phys. 87(10), 6166–6177 (1987)

    Article  Google Scholar 

  16. Oseledets, I.V., Savostyanov, D.V.: Minimization methods for approximating tensors and their comparison. Comput. Math. Math. Phys. 46(10), 1641–1650 (2006)

    Article  MathSciNet  Google Scholar 

  17. Sterck, H.D., Miller, K.: An adaptive algebraic multigrid algorithm for low-rank canonical tensor decomposition. SIAM J. Sci. Comput. 35(1), B1–B24 (2013)

    Article  MathSciNet  Google Scholar 

  18. Toint, P.L.: Some numerical results using a sparse matrix updating formula in unconstrained optimization. Math. Comput. 32(143), 839–851 (1978)

    Article  MathSciNet  Google Scholar 

  19. Wales, D.J., Doye, J.P.K.: Global optimization by basin-hopping and the lowest energy structures of Lennard-Jones clusters containing up to 110 atoms. The J. Phys. Chem. A 101(28), 5111–5116 (1997)

    Article  Google Scholar 

  20. Yu, L., Barbot, J.P., Zheng, G., Sun, H.: Compressive sensing with chaotic sequence. IEEE Sig. Process. Lett. 17(8), 731–734 (2010)

    Article  Google Scholar 

Download references

Acknowledgement

The author thanks the anonymous referee for insightful comments and suggestions which allow to significantly improve the exposition of the paper.

Author information

Authors and Affiliations

  1. Dorodnicyn Computer Center of FRC CSC RAS, Vavilova 40, Moscow, Russia

    Igor Kaporin

Authors
  1. Igor Kaporin
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dorodnicyn Computing Centre, FRC CSC RAS, Moscow, Russia

    Nicholas N. Olenev

  2. Dorodnicyn Computing Centre, FRC CSC RAS, Moscow, Russia

    Yuri G. Evtushenko

  3. University of Montenegro, Podgorica, Montenegro

    Milojica Jaćimović

  4. Krasovsky Institute of Mathematics and Mechanics, Ekaterinburg, Russia

    Michael Khachay

  5. Dorodnicyn Computing Centre, FRC CSC RAS, Moscow, Russia

    Vlasta Malkova

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kaporin, I. (2021). A Derivative-Free Nonlinear Least Squares Solver. In: Olenev, N.N., Evtushenko, Y.G., Jaćimović, M., Khachay, M., Malkova, V. (eds) Optimization and Applications. OPTIMA 2021. Lecture Notes in Computer Science(), vol 13078. Springer, Cham. https://doi.org/10.1007/978-3-030-91059-4_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-91059-4_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-91058-7

  • Online ISBN: 978-3-030-91059-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation