Skip to main content
SpringerLink
Log in

A limited memory q-BFGS algorithm for unconstrained optimization problems

  • Original Research
  • Published:
Journal of Applied Mathematics and Computing Aims and scope Submit manuscript

Abstract

A limited memory q-BFGS (Broyden–Fletcher–Goldfarb–Shanno) method is presented for solving unconstrained optimization problems. It is derived from a modified BFGS-type update using q-derivative (quantum derivative). The use of Jackson’s derivative is an effective mechanism for escaping from local minima. The q-gradient method is complemented to generate the parameter q for computing the step length in such a way that the search process gradually shifts from global in the beginning to almost local search in the end. Further, the global convergence is established under Armijo-Wolfe conditions even if the objective function is not convex. The numerical experiments show that proposed method is potentially efficient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Mishra, S.K., Ram, B.: Steepest descent method. In: Introduction to Unconstrained Optimization with R, pp. 131–173, Springer, Singapore (2019)

  2. Mishra, S.K., Ram, B.: Newton’s method. In: Introduction to Unconstrained Optimization with R, pp. 175–209, Springer, Singapore (2019)

  3. Mishra, S.K., Ram, B.: Quasi-Newton methods. In: Introduction to Unconstrained Optimization with R, pp. 245–289, Springer, Singapore (2019)

  4. Mishra S.K., Ram B.: Conjugate gradient methods. In: Introduction to Unconstrained Optimization with R, pp. 211–244, Springer, Singapore (2019)

  5. Akaike, H.: On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method. Ann. Inst. Stat. Math. 11, 1–17 (1959)

    Article  MathSciNet  MATH  Google Scholar 

  6. Nash, S.G.: A survey of truncated-Newton methods. J. Comput. Appl. Math. 124, 45–59 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  7. Byrd, R.H., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J. Numer. Anal. 26, 727–739 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  8. Dennis, J.E., Moré, J.J.: A characterization of superlinear convergence and its application to quasi Newton methods. Math. Comput. 28, 549–560 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  9. Powell, M.J.D.: Some convergence properties of a variable mertric algorithm for minimization without exact line search. In: Cottle, R.W., Lemke, C.E. (eds.) Nonlinear Programming, SIAM-AMS Proceedings, vol. IX, pp. 53–72. SIAM, Philadelphia (1976)

  10. Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35, 773–782 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  11. Perry, J.M.: A Class of Conjugate Gradient Algorithms with a Two-Step Variable-Metric Memory. Discussion Paper 269, Center for Mathematical Studies in Economics and Management Science, Northwestern University, Evanston, IL (1977)

  12. Xiao, Y., Wei, Z., Wang, Z.: A limited memory BFGS-type method for large-scale unconstrained optimization. Comput. Math. Appl. 56, 1001–1009 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  13. Li, D.H., Fukushima, M.: A modified BFGS method and its global convergence in nonconvex minimization. J. Comput. Appl. Math. 129, 15–35 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  14. Xiao, Y.H., Li, T.F., Wei, Z.X.: Global convergence of a modified limited memory BFGS method for non-convex minimization. Acta Math. Appl. Sin. Engl. Ser. 29, 555–566 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  15. Shi, Z., Yang, G., Xiao, Y.: A limited memory BFGS algorithm for non-convex minimization with applications in matrix largest eigenvalue problem. Math. Method Oper. Res. 83, 243–264 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  16. Jackson, F.H.: On q-definite integrals. Q. J. Pure Appl. Math. 41, 193–203 (1910)

    MATH  Google Scholar 

  17. Jackson, F.H.: q-difference equations. Am. J. Math. 32, 305–314 (1910)

    Article  MATH  Google Scholar 

  18. Andrews, G.E.: q-Series: Their Development and Applications in Analysis, Number Theory, Combinatorics, Physics and Computer Algebra. CBMS Regional Conference Series in Mathematics, vol. 66. American Mathematical Society, Providence, RI (1986)

  19. Stanković, M.S., Rajković, P.M., Marinković, S.D.: Fractional integrals and derivatives in q-calculus. Appl. Anal. Discrete Math. 1, 311–323 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  20. Zhou, H., Alzabut, J., Rezapour, S., Samei, M.E.: Uniform persistence and almost periodic solutions of a non-autonomous patch occupancy model. Adv. Differ. Equ. 2020, 143 (2020)

    Article  Google Scholar 

  21. Annaby, M.H., Mansour, Z.S.: q-Fractional Calculus and Equations. Springer, Heidelberg (2012)

    Book  MATH  Google Scholar 

  22. Samei, M.E.: Existence of solutions for a system of singular sum fractional q-differential equations via quantum calculus. Adv. Diff. Equ. 2020, 23 (2020)

    Article  MathSciNet  Google Scholar 

  23. Ntouyas, S.K., Samei, M.E.: Existence and uniqueness of solutions for multi-term fractional q-integro-differential equations via quantum calculus. Adv. Differ. Equ. 2019, 475 (2019)

    Article  MathSciNet  Google Scholar 

  24. Bohner, M., Peterson, A.: Dynamic Equations on Time Scales. Birkhäuser, Boston (2001)

    Book  MATH  Google Scholar 

  25. Liang, S., Samei, M.E.: New approach to solutions of a class of singular fractional q-differential problem via quantum calculus. Adv. Differ. Equ. 2019, 14 (2020)

    Article  MathSciNet  Google Scholar 

  26. Kalvandi, V., Samei, M.E.: New stability results for a sum-type fractional q-integro-differential equation. J. Adv. Math. Stud. 12, 201–209 (2019)

    MathSciNet  MATH  Google Scholar 

  27. Hedayati, V., Samei, M.E.: Positive solutions of fractional differential equation with two pieces in chain interval and simultaneous Dirichlet boundary conditions. Bound. Value Probl. 2019, 141 (2019)

    Article  MathSciNet  Google Scholar 

  28. Samei, M.E., Yang, W.: Existence of solutions for k-dimensional system of multi-term fractional q-integro-differential equations under anti-periodic boundary conditions via quantum calculus. Math. Methods Appl. Sci. 43, 4360–4382 (2020)

    MathSciNet  MATH  Google Scholar 

  29. Ernst, T.: The history of q-calculus and a new method (Licentiate Thesis). U.U.D.M, Report (2000)

  30. Ernst, T.: A method for q-calculus. J. Nonlinear Math. Phys. 10, 487–525 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  31. Bettaibi, N., Mezlini, K.: On the use Of the q-Mellin transform to solve some q-heat and q-wave equations. Int. J. Math. Arch. 3, 446–55 (2012)

    Google Scholar 

  32. Sterroni, A.C., Galski, R.L., Ramos, F.M.: The q-gradient vector for unconstrained continuous optimization problems. In: Hu, B., Morasch, K., Pickl, S., Siegle, M. (eds.) Operations Research Proceedings, pp. 365–370. Springer, Heidelberg, Germany (2010)

  33. Gouv\({{\hat{e}}}\)a, E.J.C., Regis, R.G., Soterroni, A.C., Scarabello, M.C., Ramos, F.M.: Global optimization using q-gradients. Eur. J. Oper. Res. 251, 727–738 (2016)

  34. Lai, K.K., Mishra, S.K., Ram, B.: A q-conjugate gradient algorithm for unconstrained optimization problems. Pac. J. Optim, Communicated (2020)

  35. Chakraborty, S.K., Panda, G.: q-Line search scheme for optimization problem (2017). arXiv preprint arXiv:1702.01518

  36. Chakraborty, S.K., Panda, G.: Newton like line search method using q-calculus. In: Giri, D., Mohapatra, R.N., Begehr, H., Obaidat, M. (eds.) International Conference on Mathematics and Computing. Communications in Computer and Information Science, vol. 655, pp. 196–208. Springer, Singapore (2017)

  37. Al-Saggaf, U.M., Moinuddin, M., Arif, M., Zerguine, A.: The q-least mean squares algorithm. Signal Process. 111, 50–60 (2015)

    Article  Google Scholar 

  38. Ahmed, A., Moinuddin, M., Al-Saggaf, U.M.: q-State space least mean family of algorithms. Circuits Syst. Signal Process. 37, 729–751 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  39. Ablinger, J., Uncu, A.K.: q-Functions—a Mathematica package for q-series and partition theory applications (2019). arXiv preprint arXiv:1910.12410

  40. Rajković, P., Stanković, M., Marinković, D.S.: Mean value theorems in q-calculus. Matematicki vesnik 54, 171–178 (2002)

    MathSciNet  MATH  Google Scholar 

  41. Rajković, P.M., Marinković, S.D., Stanković, M.S.: On q-Newton-Kantorovich method for solving systems of equations. Appl. Math. Comput. 168, 1432–1448 (2005)

    MathSciNet  MATH  Google Scholar 

  42. Li, D.H., Fukushima, M.: On the global convergence of the BFGS method for nonconvex unconstrained problems. SIAM J. Optim. 11, 1054–1064 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  43. Shi, Z., Yang, G., Xiao, Y.: A limited memory BFGS algorithm for non-convex minimization with applications in matrix largest eigenvalue problem. Math. Methods Oper. Res. 83, 243–264 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  44. Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10, 147–161 (2008)

    MathSciNet  MATH  Google Scholar 

  45. Dolan, E.D., Morè, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This research was supported by the Science and Engineering Research Board (Grant No. DST-SERB- MTR-2018/000121) and the University Grants Commission (IN) (Grant No. UGC-2015-UTT-59235). The fifth author was supported by Bu-Ali Sina University. The authors would also like to thank the anonymous referees for their valuable comments and suggestions.

Author information

Authors and Affiliations

  1. College of Economics, Shenzhen University, Shenzhen, 518060, China

    Kin Keung Lai

  2. Department of Mathematics, Institute of Science, Banaras Hindu University, Varanasi, 221005, India

    Shashi Kant Mishra

  3. Department of Mathematics, Indian Institute of Technology Kharagpur, Kharagpur, 721302, India

    Geetanjali Panda

  4. Department of Mathematics, Sir Gurudas Mahavidyalaya, Kolkata, 700067, India

    Suvra Kanti Chakraborty

  5. Department of Mathematics, Faculty of Basic Science, Bu-Ali Sina University, Hamedan, Iran

    Mohammad Esmael Samei

  6. DST-Centre for Interdisciplinary Mathematical Sciences, Institute of Science, Banaras Hindu University, Varanasi, 221005, India

    Bhagwat Ram

Authors
  1. Kin Keung Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shashi Kant Mishra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Geetanjali Panda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Suvra Kanti Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mohammad Esmael Samei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bhagwat Ram
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kin Keung Lai.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lai, K.K., Mishra, S.K., Panda, G. et al. A limited memory q-BFGS algorithm for unconstrained optimization problems. J. Appl. Math. Comput. 66, 183–202 (2021). https://doi.org/10.1007/s12190-020-01432-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12190-020-01432-6

Keywords

Mathematics Subject Classification

Search

Navigation