Vectorization of conjugate-gradient methods for large-scale minimization in meteorology

  • I. M. Navon
  • P. K. H. Phua
  • M. Ramamurthy
Contributed Papers


During the last few years, conjugate-gradient methods have been found to be the best available tool for large-scale minimization of nonlinear functions occurring in geophysical applications. While vectorization techniques have been applied to linear conjugate-gradient methods designed to solve symmetric linear systems of algebraic equations, arising mainly from discretization of elliptic partial differential equations, due to their suitability for vector or parallel processing, no such effort was undertaken for the nonlinear conjugate-gradient method for large-scale unconstrained minimization.

Computational results are presented here using a robust memoryless quasi-Newton-like conjugate-gradient algorithm by Shanno and Phua applied to a set of large-scale meteorological problems. These results point to the vectorization of the conjugate-gradient code inducing a significant speed-up in the function and gradient evaluation for the nonlinear conjugate-gradient method, resulting in a sizable reduction in the CPU time for minimizing nonlinear functions of 104 to 105 variables. This is particularly true for many real-life problems where the gradient and function evaluation take the bulk of the computational effort.

It is concluded that vector computers are advantageous for largescale numerical optimization problems where local minima of nonlinear functions are to be found using the nonlinear conjugate-gradient method.

Key Words

Conjugate-gradient methods large-scale minimization vectorization direct minimization meteorological problems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chazan, D., andMiranker, W. L.,A Nongradient and Parallel Algorithm for Unconstrained Minimization, SIAM Journal on Control, Vol. 8, pp. 207–217, 1970.Google Scholar
  2. 2.
    Sloboda, F.,A Conjugate Direction Method and Its Applications, Proceedings of the 8th IFIP Conference on Optimization Techniques, Wurtzburg, Germany, 1977; Springer-Verlag, Berlin, Germany, 1977.Google Scholar
  3. 3.
    Sutti, C.,Nongradient Minimization Methods for Parallel Processing Computers, Part 1, Journal of Optimization Theory and Applications, Vol. 39, pp. 465–474, 1983.Google Scholar
  4. 4.
    Sutti, C.,Nongradient Minimization Methods for Parallel Processing Computers, Part 2, Journal of Optimization Theory and Applications, Vol. 39, pp. 475–488, 1983.Google Scholar
  5. 5.
    Powell, M. J. D.,An Efficient Method for Finding the Minimum of a Function of Several Variables without Calculating Derivatives, Computer Journal, Vol. 7, pp. 155–162, 1964.Google Scholar
  6. 6.
    Lootsma, F. A.,State-of-the-Art in Parallel Unconstrained Optimization, Parallel Computing 85, Edited by M. Feilmeir, E. Joubert, and V. Schendel, North-Holland, Amsterdam, Holland, pp. 157–163, 1986.Google Scholar
  7. 7.
    Nazareth, L.,A Conjugate-Gradient Algorithm without Linear Searches, Journal of Optimization Theory and Applications, Vol. 23, pp. 373–387, 1977.Google Scholar
  8. 8.
    Dixon, L. C. W., Ducksbury, P. G., andSing, P.,A Parallel Version of the Conjugate Gradient Algorithm for Finite-Element Problems, Technical Report No. 1132, Numerical Optimization Centre, Hatfield Polytechnic, 1982.Google Scholar
  9. 9.
    Dixon, L. C. W., andPatel, K. D.,The Place of Parallel Computation in Numerical Optimization, IV: Parallel Algorithms for Nonlinear Optimization, Technical Report No. 125, Numerical Optimization Centre, Hatfield Polytechnic, 1982.Google Scholar
  10. 10.
    Dixon, L. C. W., Patel, K. D., andDucksbury, P. G.,Experience Running Optimization Algorithms on Parallel Processing Systems, Technical Report No. 138, Numerical Optimization Centre, Hatfield Polytechnic, 1983.Google Scholar
  11. 11.
    Patel, K. D.,Parallel Computation and Numerical Optimization, Technical Report No. 129, Numerical Optimization Centre, Hatfield Polytechnic, 1982.Google Scholar
  12. 12.
    Schnabel, R. B.,Parallel Computing in Optimization, Computational Mathematical Programming, Edited by K. Schittkowsky, Springer-Verlag, Berlin, Germany, pp. 357–381, 1985.Google Scholar
  13. 13.
    Straeter, T. A.,A Parallel Variable-Metric Optimization Algorithm, NASA, Technical Note No. D-7329, 1973.Google Scholar
  14. 14.
    Housos, E. C., andWing, O.,Parallel Nonlinear Minimization by Conjugate Gradients, Proceedings of the International Conference on Parallel Processing, Los Alamitos, California, pp. 157–158, 1980.Google Scholar
  15. 15.
    Housos, E. C., andWing, O.,Pseudo-Conjugate Directions for the Solution of the Nonlinear Unconstrained Optimization Problem on a Parallel Computer, Journal of Optimization Theory and Applications, Vol. 42, pp. 169–180, 1984.Google Scholar
  16. 16.
    Mukai, H.,Parallel Algorithms for Solving Systems of Nonlinear Equations, Computers and Mathematics with Applications, Vol. 7, pp. 235–250, 1981.Google Scholar
  17. 17.
    Pierre, D. A.,A Nongradient Minimization Algorithm Having Parallel Structure with Implications for an Array Processor, Computers and Electrical Engineering, Vol. 1, pp. 3–21, 1973.Google Scholar
  18. 18.
    Van Laarhoven, P. J. M.,Parallel Variable-metric Algorithms for Unconstrained Optimization, Mathematical Programming, Vol. 33, pp. 68–81, 1985.Google Scholar
  19. 19.
    Navon, I. M., andLegler, D. M.,Conjugate-Gradient Methods for Large-Scale Minimization in Meteorology, Monthly Weather Review, Vol. 115, pp. 1479–1502, 1987.Google Scholar
  20. 20.
    Shanno, D. F., andPhua, K. H.,Remark on Algorithm 500, ACM Transactions on Mathematical Software, Vol. 6, pp. 618–622, 1980.Google Scholar
  21. 21.
    Liu, D. C., andNocedal, J.,On the Limited-Memory BFGS Method for Large-Scale Minimization, Technical Report No. NAM-03, Department of Electrical Engineering and Computer Science, Northwestern University, 1988.Google Scholar
  22. 22.
    Navon, I. M., andDe Villiers, R.,Combined Penalty-Multiplier Optimization Methods to Enforce Integral Invariants Conservation, Monthly Weather Review, Vol. 111, pp. 1228–1243, 1983.Google Scholar
  23. 23.
    Navon, I. M., andDe Villiers, R.,GUSTAF–A Quasi-Newton Nonlinear ADI FORTRAN IV Program for Solving the Shallow-Water Equations with Augmented Lagrangians, Computers and Geosciences, Vol. 12, pp. 151–173, 1986.Google Scholar
  24. 24.
    Navon, I. M.,FEUDX: A Two Stage, High-Accuracy, Finite-Element FORT-RAN Program for Solving the Shallow-Water Equations, Computers and Geosciences, Vol. 13, pp. 255–285, 1987.Google Scholar
  25. 25.
    Bertsekas, D. P.,Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, New York, 1982.Google Scholar
  26. 26.
    Gustafsson, B.,An Alternating-Direction Implicit Method for Solving the Shallow-Water Equations, Journal of Computational Physics, Vol. 7, pp. 239–254, 1971.Google Scholar
  27. 27.
    Sasaki, J.,Variational Design of Finite-Difference Schemes for Initial-Value Problems with an Integral Invariant, Journal of Computational Physics, Vol. 21, pp. 270–278, 1976.Google Scholar
  28. 28.
    Bertsekas, D. P.,Combined Primal-Dual and Penalty Methods for Constrained Minimization, SIAM Journal on Control and Optimization, Vol. 13, pp. 521–544, 1975.Google Scholar
  29. 29.
    Ramamurthy, M., andCarr, F.,Four-Dimensional Data Assimilation in the Monsoon Region, Part 1: Experiments with Wind Data, Monthly Weather Review, Vol. 115, pp. 1678–1706, 1987.Google Scholar
  30. 30.
    Shanno, D. F.,Conjugate-Gradient Methods with Inexact Searches, Mathematics of Operations Research, Vol. 3, pp. 244–256, 1978.Google Scholar
  31. 31.
    Shanno, D. F.,On the Convergence of a New Conjugate Gradient Method, SIAM Journal on Numerical Analysis, Vol. 15, pp. 1247–1257, 1978.Google Scholar
  32. 32.
    Shanno, D. F., andPhua, K. H.,Matrix Conditioning and Nonlinear Optimization, Mathematical Programming, Vol. 14, pp. 149–160, 1976.Google Scholar
  33. 33.
    Powell, M. J. D.,Restart Procedures for the Conjugate Gradient Method, Mathematical Programming, Vol. 12, pp. 241–254, 1977.Google Scholar
  34. 34.
    Anonymous, N. N.,FORTRAN Library Reference Manual (Mark 12), Numerical Algorithm Group, E04DGF, Oxford, England, 1987.Google Scholar
  35. 35.
    Buckley, A. G., andLenir, A.,QN-Like Variable Storage Conjugate-Gradients, Mathematical Programming, Vol. 27, pp. 155–175, 1983.Google Scholar
  36. 36.
    Buckley, A. G., andLenir, A.,Algorithm 630-BBVSCG: A Variable Storage Algorithm for Function Minimization, ACM Transactions on Mathematical Software, Vol. 11, pp. 103–119, 1985.Google Scholar
  37. 37.
    Beale, E. M. L.,A Derivation of Conjugate Gradients, Numerical Methods for Nonlinear Optimization, Edited by F. A. Lootsma, Academic Press, London, England, pp. 39–43, 1972.Google Scholar
  38. 38.
    Gill, P. E., andMurray, W.,Conjugate-Gradient Methods for Large-Scale Optimization, Technical Report No. SOL 79-15, Systems Optimization Laboratory, Department of Operations Research, Stanford University, 1979.Google Scholar
  39. 39.
    Fletcher, R.,Conjugate Direction Methods, Numerical Methods for Unconstrained Minimization, Edited by W. Murray, Academic Press, London, England, pp. 73–86, 1972.Google Scholar
  40. 40.
    Le, D.,A Fast and Robust Unconstrained Minimization Method Requiring Minimum Storage, Mathematical Programming, Vol. 32, pp. 41–68, 1985.Google Scholar
  41. 41.
    Le Dimet, F. X., andTalagrand, O.,Variational Algorithms for Analysis and Assimilation of Meteorological Observations: Theoretical Aspects, Tellus, Vol. 38A, pp. 97–110, 1986.Google Scholar
  42. 42.
    Legler, D. M., Navon, I. M., andO'Brien, J. J.,Objective Analysis of Pseudostress over the Indian Ocean Using a Direct Minimization Approach, Monthly Weather Review, Vol. 117, pp. 709–720, 1989.Google Scholar
  43. 43.
    Robertson, D. H., Brown, F. B., andNavon, I. M.,Determination of the Structure of Mixed Argon-Xenon Clusters Using a Finite-Temperature, Lattice-Based Monte Carlo Method. Journal of Chemical Physics, Vol. 90, pp. 3221–3229, 1989.Google Scholar

Copyright information

© Plenum Publishing Corporation 1990

Authors and Affiliations

  • I. M. Navon
    • 1
  • P. K. H. Phua
    • 2
  • M. Ramamurthy
    • 3
  1. 1.Department of Mathematics and Faculty Associate, Supercomputer Computations Research InstituteFlorida State UniversityTallahassee
  2. 2.Department of Information Systems and Computer ScienceNational University of SingaporeSingapore
  3. 3.Department of Atmospheric SciencesUniversity of IllinoisUrbana-Champaign

Personalised recommendations