Skip to main content
SpringerLink
Log in

Eigenvalue Analyses on the Memoryless Davidon–Fletcher–Powell Method Based on a Spectral Secant Equation

  • Brief Communication
  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

The subject of this study is the analysis of the spectral secant equation for the well-known Davidon–Fletcher–Powell (DFP) quasi-Newton updating formula. More precisely, we plan to make the memoryless DFP formula well conditioned in several aspects. We first focus our efforts on obtaining eigenvalues of the DFP formula and directly analyzing its spectral condition number. Then, we proceed in a different direction by using the Byrd–Nocedal measure function as well. Lastly, we propose a hybrid adaptive formula for the spectral parameter of the method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Data Availability

Data sharing is not applicable to this manuscript as no new data were created or analyzed in this study.

References

  1. Aminifard, Z., Babaie-Kafaki, S.: Analysis of the maximum magnification by the scaled memoryless DFP updating formula with application to compressive sensing. Mediterr. J. Math. 18(6), 255 (2021). https://doi.org/10.1007/s00009-021-01905-3

    Article  MathSciNet  Google Scholar 

  2. Aminifard, Z., Babaie-Kafaki, S., Ghafoori, S.: An augmented memoryless BFGS method based on a modified secant equation with application to compressed sensing. Appl. Numer. Math. 67, 187–201 (2021). https://doi.org/10.1016/j.apnum.2021.05.002

    Article  MathSciNet  Google Scholar 

  3. Arazm, M.R., Babaie-Kafaki, S., Ghanbari, R.: An extended Dai-Liao conjugate gradient method with global convergence for nonconvex functions. Glas. Mat. 52, 361–375 (2017). https://doi.org/10.3336/gm.52.2.12

    Article  MathSciNet  Google Scholar 

  4. Babaie-Kafaki, S.: On optimality of the parameters of self-scaling memoryless quasi-Newton updating formulae. J. Optim. Theory Appl. 167, 91–101 (2015). https://doi.org/10.1007/s10957-015-0724-x

    Article  MathSciNet  Google Scholar 

  5. Babaie-Kafaki, S., Ghanbari, R.: A linear hybridization of the Hestenes–Stiefel method and the memoryless BFGS technique. Mediterr. J. Math. 15, 1–10 (2018). https://doi.org/10.1007/s00009-018-1132-x

    Article  MathSciNet  Google Scholar 

  6. Barzilai, J., Borwein, J.M.: Two-point stepsize gradient methods. IMA J. Numer. Anal. 8(1), 141–148 (1988). https://doi.org/10.1093/imanum/8.1.141

    Article  MathSciNet  Google Scholar 

  7. Byrd, R.H., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J. Numer. Anal. 26(3), 727–739 (1989). https://doi.org/10.1137/0726042

    Article  MathSciNet  Google Scholar 

  8. Cheng, W.Y., Li, D.H.: Spectral scaling BFGS method. J. Optim. Theory Appl. 146(2), 305–319 (2010). https://doi.org/10.1007/s10957-010-9652-y

    Article  MathSciNet  Google Scholar 

  9. Eldén, L.: Matrix Methods in Data Mining and Pattern Recognition. SIAM, Philadelphia (2007)

    Book  Google Scholar 

  10. Ford, J.A., Moghrabi, I.A.: Multi-step quasi-Newton methods for optimization. J. Comput. Appl. Math. 50(1–3), 305–323 (1994). https://doi.org/10.1016/0377-0427(94)90309-3

    Article  MathSciNet  Google Scholar 

  11. Ford, J.A., Tharmlikit, S.: Three-step fixed-point quasi-Newton methods for unconstrained optimisation. Comput. Math. Appl. 50(7), 1051–1060 (2005). https://doi.org/10.1016/j.camwa.2005.08.007

    Article  MathSciNet  Google Scholar 

  12. Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large-scale optimization. Math. Program. 45(1–3), 503–528 (1989). https://doi.org/10.1007/BF01589116

    Article  MathSciNet  Google Scholar 

  13. Liu, H., Yao, Y., Qian, X., Wang, H.: Some nonlinear conjugate gradient methods based on spectral scaling secant equations. J. Comput. Appl. Math. 35(2), 639–651 (2016). https://doi.org/10.1007/s40314-014-0212-1

    Article  MathSciNet  Google Scholar 

  14. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999). https://doi.org/10.1007/978-0-387-40065-5

    Book  Google Scholar 

  15. Oren, S.S., Luenberger, D.G.: Self-scaling variable metric (SSVM) algorithms. I. Criteria and sufficient conditions for scaling a class of algorithms. Manag. Sci. 20(5), 845–862 (1973). https://doi.org/10.1287/mnsc.20.5.845

    Article  MathSciNet  Google Scholar 

  16. Oren, S.S., Spedicato, E.: Optimal conditioning of self-scaling variable metric algorithms. Math. Program. 10(1), 70–90 (1976). https://doi.org/10.1007/BF01580654

    Article  MathSciNet  Google Scholar 

  17. Powell, M.J.D.: Some global convergence properties of a variable metric algorithm for minimization without exact line searches. In: Cottle, R.W., Lemke, C.E. (eds.) Nonlinear Programming. SIAM–AMS Proceedings, vol. 4, pp. 53–72. American Mathematical Society, Providence (1976)

    Google Scholar 

  18. Pu, D.: Convergence of the DFP algorithm without exact line search. J. Optim. Theory Appl. 112, 187–211 (2002). https://doi.org/10.1023/A:1013004914923

    Article  MathSciNet  Google Scholar 

  19. Pu, D., Tian, W.: A class of DFP algorithms with revised search directions. Numer. Funct. Anal. Optim. 23(3–4), 383–400 (2002). https://doi.org/10.1081/NFA-120006700

    Article  MathSciNet  Google Scholar 

  20. Pu, D., Tian, W.: The revised DFP algorithm without exact line search. J. Comput. Appl. Math. 154(2), 319–339 (2003). https://doi.org/10.1016/S0377-0427(02)00856-7

    Article  MathSciNet  Google Scholar 

  21. Shi, Z.J.: Convergence of quasi-Newton method with new inexact line search. J. Math. Anal. Appl. 315(1), 120–131 (2006). https://doi.org/10.1016/j.jmaa.2005.05.077

    Article  MathSciNet  Google Scholar 

  22. Sun, W., Yuan, Y.X.: Optimization Theory and Methods: Nonlinear Programming. Springer, New York (2006). https://doi.org/10.1007/b106451

    Book  Google Scholar 

  23. Watkins, D.S.: Fundamentals of Matrix Computations. Wiley, New York (2002)

    Book  Google Scholar 

  24. Xu, C., Zhang, J.: A survey of quasi-Newton equations and quasi-Newton methods for optimization. Ann. Oper. Res. 103(1–4), 213–234 (2001). https://doi.org/10.1023/A:1012959223138

    Article  MathSciNet  Google Scholar 

  25. Yabe, H., Ogasawara, H., Yoshino, M.: Local and superlinear convergence of quasi-Newton methods based on modified secant conditions. J. Comput. Appl. Math. 205(1), 617–632 (2007). https://doi.org/10.1016/j.cam.2006.05.018

    Article  MathSciNet  Google Scholar 

  26. Yao, X., Wang, Z.: Broad echo state network for multivariate time series prediction. J. Franklin Inst. 356(9), 4888–4906 (2019). https://doi.org/10.1016/j.jfranklin.2019.01.027

    Article  MathSciNet  Google Scholar 

  27. Yin, F., Wang, Y.N., Wei, S.N.: Inverse kinematic solution for robot manipulator based on electromagnetism-like and modified DFP algorithms. Acta Mech. Solida Sin. 37(1), 74–82 (2011). https://doi.org/10.1016/S1874-1029(11)60199-7

    Article  MathSciNet  Google Scholar 

  28. Zhang, H., Wang, K., Zhou, X., Wang, W.: Using DFP algorithm for nodal demand estimation of water distribution networks. KSCE J. Civ. Eng. 22, 2747–2754 (2018). https://doi.org/10.1007/s12205-018-0176-6

    Article  Google Scholar 

  29. Zhang, J.Z., Xue, Y., Zhang, K.: A structured secant method based on a new quasi-Newton equation for nonlinear least squares problems. BIT Numer. Math. 43(1), 217–229 (2003). https://doi.org/10.1023/A:1023665409152

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors thank the anonymous reviewer and the associate editor for their valuable suggestions that helped to improve the presentation.

Author information

Authors and Affiliations

  1. Department of Mathematics, Semnan University, Semnan, Iran

    Fatemeh Dargahi & Zohre Aminifard

  2. Faculty of Engineering, Free University of Bozen-Bolzano, Piazza Università 5, 39100, Bolzano, Italy

    Saman Babaie-Kafaki

Authors
  1. Fatemeh Dargahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saman Babaie-Kafaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zohre Aminifard
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saman Babaie-Kafaki.

Additional information

Communicated by Giacomo Nannicini.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dargahi, F., Babaie-Kafaki, S. & Aminifard, Z. Eigenvalue Analyses on the Memoryless Davidon–Fletcher–Powell Method Based on a Spectral Secant Equation. J Optim Theory Appl 200, 394–403 (2024). https://doi.org/10.1007/s10957-023-02354-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-023-02354-6

Keywords

Mathematics Subject Classification

Search

Navigation