# Vectorization of conjugate-gradient methods for large-scale minimization in meteorology

- 93 Downloads
- 3 Citations

## Abstract

During the last few years, conjugate-gradient methods have been found to be the best available tool for large-scale minimization of nonlinear functions occurring in geophysical applications. While vectorization techniques have been applied to linear conjugate-gradient methods designed to solve symmetric linear systems of algebraic equations, arising mainly from discretization of elliptic partial differential equations, due to their suitability for vector or parallel processing, no such effort was undertaken for the nonlinear conjugate-gradient method for large-scale unconstrained minimization.

Computational results are presented here using a robust memoryless quasi-Newton-like conjugate-gradient algorithm by Shanno and Phua applied to a set of large-scale meteorological problems. These results point to the vectorization of the conjugate-gradient code inducing a significant speed-up in the function and gradient evaluation for the nonlinear conjugate-gradient method, resulting in a sizable reduction in the CPU time for minimizing nonlinear functions of 10^{4} to 10^{5} variables. This is particularly true for many real-life problems where the gradient and function evaluation take the bulk of the computational effort.

It is concluded that vector computers are advantageous for largescale numerical optimization problems where local minima of nonlinear functions are to be found using the nonlinear conjugate-gradient method.

### Key Words

Conjugate-gradient methods large-scale minimization vectorization direct minimization meteorological problems## Preview

Unable to display preview. Download preview PDF.

### References

- 1.Chazan, D., andMiranker, W. L.,
*A Nongradient and Parallel Algorithm for Unconstrained Minimization*, SIAM Journal on Control, Vol. 8, pp. 207–217, 1970.Google Scholar - 2.Sloboda, F.,
*A Conjugate Direction Method and Its Applications*, Proceedings of the 8th IFIP Conference on Optimization Techniques, Wurtzburg, Germany, 1977; Springer-Verlag, Berlin, Germany, 1977.Google Scholar - 3.Sutti, C.,
*Nongradient Minimization Methods for Parallel Processing Computers, Part 1*, Journal of Optimization Theory and Applications, Vol. 39, pp. 465–474, 1983.Google Scholar - 4.Sutti, C.,
*Nongradient Minimization Methods for Parallel Processing Computers, Part 2*, Journal of Optimization Theory and Applications, Vol. 39, pp. 475–488, 1983.Google Scholar - 5.Powell, M. J. D.,
*An Efficient Method for Finding the Minimum of a Function of Several Variables without Calculating Derivatives*, Computer Journal, Vol. 7, pp. 155–162, 1964.Google Scholar - 6.Lootsma, F. A.,
*State-of-the-Art in Parallel Unconstrained Optimization*, Parallel Computing 85, Edited by M. Feilmeir, E. Joubert, and V. Schendel, North-Holland, Amsterdam, Holland, pp. 157–163, 1986.Google Scholar - 7.Nazareth, L.,
*A Conjugate-Gradient Algorithm without Linear Searches*, Journal of Optimization Theory and Applications, Vol. 23, pp. 373–387, 1977.Google Scholar - 8.Dixon, L. C. W., Ducksbury, P. G., andSing, P.,
*A Parallel Version of the Conjugate Gradient Algorithm for Finite-Element Problems*, Technical Report No. 1132, Numerical Optimization Centre, Hatfield Polytechnic, 1982.Google Scholar - 9.Dixon, L. C. W., andPatel, K. D.,
*The Place of Parallel Computation in Numerical Optimization, IV: Parallel Algorithms for Nonlinear Optimization*, Technical Report No. 125, Numerical Optimization Centre, Hatfield Polytechnic, 1982.Google Scholar - 10.Dixon, L. C. W., Patel, K. D., andDucksbury, P. G.,
*Experience Running Optimization Algorithms on Parallel Processing Systems*, Technical Report No. 138, Numerical Optimization Centre, Hatfield Polytechnic, 1983.Google Scholar - 11.Patel, K. D.,
*Parallel Computation and Numerical Optimization*, Technical Report No. 129, Numerical Optimization Centre, Hatfield Polytechnic, 1982.Google Scholar - 12.Schnabel, R. B.,
*Parallel Computing in Optimization*, Computational Mathematical Programming, Edited by K. Schittkowsky, Springer-Verlag, Berlin, Germany, pp. 357–381, 1985.Google Scholar - 13.Straeter, T. A.,
*A Parallel Variable-Metric Optimization Algorithm*, NASA, Technical Note No. D-7329, 1973.Google Scholar - 14.Housos, E. C., andWing, O.,
*Parallel Nonlinear Minimization by Conjugate Gradients*, Proceedings of the International Conference on Parallel Processing, Los Alamitos, California, pp. 157–158, 1980.Google Scholar - 15.Housos, E. C., andWing, O.,
*Pseudo-Conjugate Directions for the Solution of the Nonlinear Unconstrained Optimization Problem on a Parallel Computer*, Journal of Optimization Theory and Applications, Vol. 42, pp. 169–180, 1984.Google Scholar - 16.Mukai, H.,
*Parallel Algorithms for Solving Systems of Nonlinear Equations*, Computers and Mathematics with Applications, Vol. 7, pp. 235–250, 1981.Google Scholar - 17.Pierre, D. A.,
*A Nongradient Minimization Algorithm Having Parallel Structure with Implications for an Array Processor*, Computers and Electrical Engineering, Vol. 1, pp. 3–21, 1973.Google Scholar - 18.Van Laarhoven, P. J. M.,
*Parallel Variable-metric Algorithms for Unconstrained Optimization*, Mathematical Programming, Vol. 33, pp. 68–81, 1985.Google Scholar - 19.Navon, I. M., andLegler, D. M.,
*Conjugate-Gradient Methods for Large-Scale Minimization in Meteorology*, Monthly Weather Review, Vol. 115, pp. 1479–1502, 1987.Google Scholar - 20.Shanno, D. F., andPhua, K. H.,
*Remark on Algorithm 500*, ACM Transactions on Mathematical Software, Vol. 6, pp. 618–622, 1980.Google Scholar - 21.Liu, D. C., andNocedal, J.,
*On the Limited-Memory BFGS Method for Large-Scale Minimization*, Technical Report No. NAM-03, Department of Electrical Engineering and Computer Science, Northwestern University, 1988.Google Scholar - 22.Navon, I. M., andDe Villiers, R.,
*Combined Penalty-Multiplier Optimization Methods to Enforce Integral Invariants Conservation*, Monthly Weather Review, Vol. 111, pp. 1228–1243, 1983.Google Scholar - 23.Navon, I. M., andDe Villiers, R.,
*GUSTAF–A Quasi-Newton Nonlinear ADI FORTRAN IV Program for Solving the Shallow-Water Equations with Augmented Lagrangians*, Computers and Geosciences, Vol. 12, pp. 151–173, 1986.Google Scholar - 24.Navon, I. M.,
*FEUDX: A Two Stage, High-Accuracy, Finite-Element FORT-RAN Program for Solving the Shallow-Water Equations*, Computers and Geosciences, Vol. 13, pp. 255–285, 1987.Google Scholar - 25.Bertsekas, D. P.,
*Constrained Optimization and Lagrange Multiplier Methods*, Academic Press, New York, New York, 1982.Google Scholar - 26.Gustafsson, B.,
*An Alternating-Direction Implicit Method for Solving the Shallow-Water Equations*, Journal of Computational Physics, Vol. 7, pp. 239–254, 1971.Google Scholar - 27.Sasaki, J.,
*Variational Design of Finite-Difference Schemes for Initial-Value Problems with an Integral Invariant*, Journal of Computational Physics, Vol. 21, pp. 270–278, 1976.Google Scholar - 28.Bertsekas, D. P.,
*Combined Primal-Dual and Penalty Methods for Constrained Minimization*, SIAM Journal on Control and Optimization, Vol. 13, pp. 521–544, 1975.Google Scholar - 29.Ramamurthy, M., andCarr, F.,
*Four-Dimensional Data Assimilation in the Monsoon Region, Part 1: Experiments with Wind Data*, Monthly Weather Review, Vol. 115, pp. 1678–1706, 1987.Google Scholar - 30.Shanno, D. F.,
*Conjugate-Gradient Methods with Inexact Searches*, Mathematics of Operations Research, Vol. 3, pp. 244–256, 1978.Google Scholar - 31.Shanno, D. F.,
*On the Convergence of a New Conjugate Gradient Method*, SIAM Journal on Numerical Analysis, Vol. 15, pp. 1247–1257, 1978.Google Scholar - 32.Shanno, D. F., andPhua, K. H.,
*Matrix Conditioning and Nonlinear Optimization*, Mathematical Programming, Vol. 14, pp. 149–160, 1976.Google Scholar - 33.Powell, M. J. D.,
*Restart Procedures for the Conjugate Gradient Method*, Mathematical Programming, Vol. 12, pp. 241–254, 1977.Google Scholar - 34.Anonymous, N. N.,
*FORTRAN Library Reference Manual (Mark 12)*, Numerical Algorithm Group, E04DGF, Oxford, England, 1987.Google Scholar - 35.Buckley, A. G., andLenir, A.,
*QN-Like Variable Storage Conjugate-Gradients*, Mathematical Programming, Vol. 27, pp. 155–175, 1983.Google Scholar - 36.Buckley, A. G., andLenir, A.,
*Algorithm 630-BBVSCG: A Variable Storage Algorithm for Function Minimization*, ACM Transactions on Mathematical Software, Vol. 11, pp. 103–119, 1985.Google Scholar - 37.Beale, E. M. L.,
*A Derivation of Conjugate Gradients*, Numerical Methods for Nonlinear Optimization, Edited by F. A. Lootsma, Academic Press, London, England, pp. 39–43, 1972.Google Scholar - 38.Gill, P. E., andMurray, W.,
*Conjugate-Gradient Methods for Large-Scale Optimization*, Technical Report No. SOL 79-15, Systems Optimization Laboratory, Department of Operations Research, Stanford University, 1979.Google Scholar - 39.Fletcher, R.,
*Conjugate Direction Methods*, Numerical Methods for Unconstrained Minimization, Edited by W. Murray, Academic Press, London, England, pp. 73–86, 1972.Google Scholar - 40.Le, D.,
*A Fast and Robust Unconstrained Minimization Method Requiring Minimum Storage*, Mathematical Programming, Vol. 32, pp. 41–68, 1985.Google Scholar - 41.Le Dimet, F. X., andTalagrand, O.,
*Variational Algorithms for Analysis and Assimilation of Meteorological Observations: Theoretical Aspects*, Tellus, Vol. 38A, pp. 97–110, 1986.Google Scholar - 42.Legler, D. M., Navon, I. M., andO'Brien, J. J.,
*Objective Analysis of Pseudostress over the Indian Ocean Using a Direct Minimization Approach*, Monthly Weather Review, Vol. 117, pp. 709–720, 1989.Google Scholar - 43.Robertson, D. H., Brown, F. B., andNavon, I. M.,
*Determination of the Structure of Mixed Argon-Xenon Clusters Using a Finite-Temperature, Lattice-Based Monte Carlo Method*. Journal of Chemical Physics, Vol. 90, pp. 3221–3229, 1989.Google Scholar