Computational Optimization and Applications

, Volume 71, Issue 1, pp 73–93 | Cite as

Planar methods and grossone for the Conjugate Gradient breakdown in nonlinear programming

  • Renato De Leone
  • Giovanni Fasano
  • Yaroslav D. Sergeyev


This paper deals with an analysis of the Conjugate Gradient (CG) method (Hestenes and Stiefel in J Res Nat Bur Stand 49:409–436, 1952), in the presence of degenerates on indefinite linear systems. Several approaches have been proposed in the literature to issue the latter drawback in optimization frameworks, including reformulating the original linear system or recurring to approximately solving it. All the proposed alternatives seem to rely on algebraic considerations, and basically pursue the idea of improving numerical efficiency. In this regard, here we sketch two separate analyses for the possible CG degeneracy. First, we start detailing a more standard algebraic viewpoint of the problem, suggested by planar methods. Then, another algebraic perspective is detailed, relying on a novel recently proposed theory, which includes an additional number, namely grossone. The use of grossone allows to work numerically with infinities and infinitesimals. The results obtained using the two proposed approaches perfectly match, showing that grossone may represent a fruitful and promising tool to be exploited within Nonlinear Programming.


Conjugate Gradient (CG) method Planar-CG methods Infinities and Infinitesimals Grossone 



G. Fasano thanks the National Research Council-Marine Technology Research Institute (CNR-INSEAN), Italy, for the support received. The work of G. Fasano is partially supported by the Italian Flagship Project RITMARE, coordinated by the Italian National Research Council (CNR) and funded by the Italian Ministry of Education, within the National Research Program 2012–2016. The research of Ya.D. Sergeyev was supported by the Russian Science Foundation, Project No. 15-11-30022 “Global optimization, supercomputing computations, and applications”.


  1. 1.
    Amodio, P., Iavernaro, F., Mazzia, F., Mukhametzhanov, M.S., Sergeyev, Y.D.: A generalized Taylor method of order three for the solution of initial value problems in standard and infinity floating-point arithmetic. Math. Comput. Simulat. 141, 24–39 (2017)Google Scholar
  2. 2.
    Cococcioni, M., Pappalardo, M., Sergeyev, Y.D.: Towards lexicographic multi-objective linear programming using grossone methodology. In: Sergeyev, Y.D., Kvasov, D.E., Dell’Accio, F., Mukhametzhanov, M.S. (eds.) Proceedings of the 2nd International Conference on “Numerical Computations: Theory and Algorithms”, vol. 1776, p. 090040. AIP Publishing, New York (2016)Google Scholar
  3. 3.
    Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-Region Methods. MPS–SIAM Series on Optimization. Society for Industrial and Applied Mathematics, Philadelphia (2000)CrossRefGoogle Scholar
  4. 4.
    D’Alotto, L.: Cellular automata using infinite computations. Appl. Math. Comput. 218(16), 8077–8082 (2012)MathSciNetzbMATHGoogle Scholar
  5. 5.
    D’Alotto, L.: A classification of one-dimensional cellular automata using infinite computations. Appl. Math. Comput. 255, 15–24 (2015)MathSciNetzbMATHGoogle Scholar
  6. 6.
    De Cosmis, S., De Leone, R.: The use of grossone in mathematical programming and operations research. Appl. Math. Comput. 218(16), 8029–8038 (2012)MathSciNetzbMATHGoogle Scholar
  7. 7.
    De Leone, R.: Nonlinear programming and grossone: quadratic programming and the role of constraint qualifications. Appl. Math. Comput. 318, 290–297 (2018)Google Scholar
  8. 8.
    Fasano, G.: Conjugate gradient (CG)-type method for the solution of Newton’s equation within optimization frameworks. Optim. Methods Softw. 3–4, 267–290 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Fasano, G.: Planar conjugate gradient algorithm for large-scale unconstrained optimization, part 1: theory. J. Optim. Theory Appl. 125, 523–541 (2005a)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Fasano, G.: Planar conjugate gradient algorithm for large-scale unconstrained optimization, part 2: application. J. Optim. Theory Appl. 125, 543–558 (2005b)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Fasano, G.: Lanczos-conjugate gradient method and pseudoinverse computation, on indefinite and singular systems. J. Optim. Theory Appl. 132, 267–285 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Fasano, G., Pesenti, R.: Conjugate direction methods and polarity for quadratic hypersurfaces. J. Optim. Theory Appl. (2018) (in press)Google Scholar
  13. 13.
    Greenbaum, A.: Iterative Methods for Solving Linear Systems. SIAM, Philadelphia (1997)CrossRefzbMATHGoogle Scholar
  14. 14.
    Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newtons method. SIAM J. Numer. Anal. 23, 707–716 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Grippo, L., Lampariello, F., Lucidi, S.: A truncated Newton method with nonmonotone line search for unconstrained optimization. J. Optim. Theory Appl. 60, 401–419 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Hager, W., Zhang, H.: The limited memory conjugate gradient method. SIAM J. Optim. 23, 2150–2168 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Hestenes, M.R.: Conjugate Direction Methods in Optimization. Springer, New York (1980)CrossRefzbMATHGoogle Scholar
  18. 18.
    Hestenes, M.R., Stiefel, E.L.: Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49, 409–436 (1952)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Huang, H.Y.: Unified approach to quadratically convergent algorithms for function minimization. J. Optim. Theory Appl. 5, 405–423 (1970)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Iudin, D.I., Sergeyev, Y.D., Hayakawa, M.: Interpretation of percolation in terms of infinity computations. Appl. Math. Comput. 218(16), 8099–8111 (2012)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Iudin, D.I., Sergeyev, Y.D., Hayakawa, M.: Infinity computations in cellular automaton forest-fire model. Commun. Nonlinear Sci. Numer. Simul. 20(3), 861–870 (2015)CrossRefGoogle Scholar
  22. 22.
    Lolli, G.: Infinitesimals and infinites in the history of mathematics: a brief survey. Appl. Math. Comput. 218(16), 7979–7988 (2012)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Lolli, G.: Metamathematical investigations on the theory of grossone. Appl. Math. Comput. 255, 3–14 (2015)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Luenberger, D.G.: Hyperbolic pairs in the method of conjugate gradients. SIAM J. Appl. Math. 17, 1263–1267 (1969)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Margenstern, M.: Using grossone to count the number of elements of infinite sets and the connection with bijections. p-Adic Numbers Ultrametr. Anal. Appl. 3(3), 196–204 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Mazzia, F., Sergeyev, Y.D., Iavernaro, F., Amodio, P., Mukhametzhanov, M.S.: Numerical methods for solving ODEs on the Infinity Computer. In: Sergeyev, Y.D., Kvasov, D.E., Dell’Accio, F., Mukhametzhanov, M.S. (eds.) Proceedings of the 2nd International Conference on “Numerical Computations: Theory and Algorithms”, vol. 1776, p. 090033. AIP Publishing, New York (2016)Google Scholar
  27. 27.
    Nocedal, J., Wright, S.: Numerical Optimization, 2nd edn. Springer, New York (2006)zbMATHGoogle Scholar
  28. 28.
    Oren, S.S.: Planar quasi-Newton algorithms for unconstrained saddlepoint problems. J. Optim. Theory Appl. 43, 167–204 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Rizza, D.: Supertasks and numeral systems. In: Sergeyev, Y.D., Kvasov, D.E., Dell’Accio, F., Mukhametzhanov, M.S. (eds.) Proceedings of the 2nd International Conference on “Numerical Computations: Theory and Algorithms”, vol. 1776, p. 090005. AIP Publishing, New York (2016)Google Scholar
  30. 30.
    Seidenberg, A.: Lecture in Projective Geometry. Dover Publications Inc, Mineola (2005)zbMATHGoogle Scholar
  31. 31.
    Sergeyev, Y.D.: Arithmetic of Infinity, 2nd edn. Edizioni Orizzonti Meridionali, Cosenza (2013)zbMATHGoogle Scholar
  32. 32.
    Sergeyev, Y.D.: Blinking fractals and their quantitative analysis using infinite and infinitesimal numbers. Chaos Solitons Fractals 33(1), 50–75 (2007)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Sergeyev, Y.D.: A new applied approach for executing computations with infinite and infinitesimal quantities. Informatica 19(4), 567–596 (2008)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Sergeyev, Y.D.: Evaluating the exact infinitesimal values of area of Sierpinski’s carpet and volume of Menger’s sponge. Chaos Solitons Fractals 42(5), 3042–3046 (2009)CrossRefGoogle Scholar
  35. 35.
    Sergeyev, Y.D.: Lagrange lecture: methodology of numerical computations with infinities and infinitesimals. Rendiconti del Seminario Matematico dell’Università e del Politecnico di Torino 68(2), 95–113 (2010)zbMATHGoogle Scholar
  36. 36.
    Sergeyev, Y.D.: Higher order numerical differentiation on the Infinity Computer. Optim. Lett. 5(4), 575–585 (2011a)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Sergeyev, Y.D.: Using blinking fractals for mathematical modelling of processes of growth in biological systems. Informatica 22(4), 559–576 (2011b)MathSciNetzbMATHGoogle Scholar
  38. 38.
    Sergeyev, Y.D.: Solving ordinary differential equations by working with infinitesimals numerically on the Infinity Computer. Appl. Math. Comput. 219(22), 10668–10681 (2013)MathSciNetzbMATHGoogle Scholar
  39. 39.
    Sergeyev, Y.D.: Computations with grossone-based infinities. In: Calude, C.S., Dinneen, M.J. (eds.) Unconventional Computation and Natural Computation: Proceedings of the 14th International Conference UCNC 2015, vol. LNCS 9252, pp. 89–106. Springer, New York (2015)Google Scholar
  40. 40.
    Sergeyev, Y.D.: Un semplice modo per trattare le grandezze infinite ed infinitesime. Mat. nella Soc. nella Cult. Riv. della Unione Mat. Ital. 8(1), 111–147 (2015)MathSciNetGoogle Scholar
  41. 41.
    Sergeyev, Y.D.: The exact (up to infinitesimals) infinite perimeter of the Koch snowflake and its finite area. Commun. Nonlinear Sci. Numer. Simul. 31(1–3), 21–29 (2016)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Sergeyev, Y.D.: Numerical infinities and infinitesimals: methodology, applications, and repercussions on two Hilbert problems. EMS Surveys Math. Sci. (2017) (in press)Google Scholar
  43. 43.
    Sergeyev, Y.D., Garro, A.: Observability of turing machines: a refinement of the theory of computation. Informatica 21(3), 425–454 (2010)MathSciNetzbMATHGoogle Scholar
  44. 44.
    Sergeyev, Y.D., Garro, A.: Single-tape and multi-tape turing machines through the lens of the grossone methodology. J. Supercomput. 65(2), 645–663 (2013)CrossRefGoogle Scholar
  45. 45.
    Sergeyev, Y.D., Mukhametzhanov, M.S., Mazzia, F., Iavernaro, F., Amodio, P.: Numerical methods for solving initial value problems on the Infinity Computer. Int. J. Unconv. Comput. 12(1), 3–23 (2016)Google Scholar
  46. 46.
    Trefethen, L.N., Bau, D.: Numerical Linear Algebra. SIAM, Philadelphia (1997)CrossRefzbMATHGoogle Scholar
  47. 47.
    Vita, M.C., Bartolo, S.D., Fallico, C., Veltri, M.: Usage of infinitesimals in the Menger’s Sponge model of porosity. Appl. Math. Comput. 218(16), 8187–8196 (2012)MathSciNetzbMATHGoogle Scholar
  48. 48.
    Žilinskas, A.: On strong homogeneity of two global optimization algorithms based on statistical models of multimodal objective functions. Appl. Math. Comput. 218(16), 8131–8136 (2012)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Scuola di Scienze e TecnologieUniversità di CamerinoCamerinoItaly
  2. 2.Dipartimento di ManagementUniversità Ca’ Foscari VeneziaVeniceItaly
  3. 3.Dipartimento di Ingegneria Informatica, Modellistica, Elettronica e SistemisticaUniversità della CalabriaRendeItaly
  4. 4.Department of Software and SupercomputingLobachevsky State UniversityNizhniy NovgorodRussia

Personalised recommendations