# The Adjoint Newton Algorithm for Large-Scale Unconstrained Optimization in Meteorology Applications

- 227 Downloads
- 15 Citations

## Abstract

A new algorithm is presented for carrying out large-scale unconstrained optimization required in variational data assimilation using the Newton method. The algorithm is referred to as the adjoint Newton algorithm. The adjoint Newton algorithm is based on the first- and second-order adjoint techniques allowing us to obtain the Newton line search direction by integrating a tangent linear equations model backwards in time (starting from a final condition with negative time steps). The error present in approximating the Hessian (the matrix of second-order derivatives) of the cost function with respect to the control variables in the quasi-Newton type algorithm is thus completely eliminated, while the storage problem related to the Hessian no longer exists since the explicit Hessian is not required in this algorithm. The adjoint Newton algorithm is applied to three one-dimensional models and to a two-dimensional limited-area shallow water equations model with both model generated and First Global Geophysical Experiment data. We compare the performance of the adjoint Newton algorithm with that of truncated Newton, adjoint truncated Newton, and LBFGS methods. Our numerical tests indicate that the adjoint Newton algorithm is very efficient and could find the minima within three or four iterations for problems tested here. In the case of the two-dimensional shallow water equations model, the adjoint Newton algorithm improves upon the efficiencies of the truncated Newton and LBFGS methods by a factor of at least 14 in terms of the CPU time required to satisfy the same convergence criterion.

The Newton, truncated Newton and LBFGS methods are general purpose unconstrained minimization methods. The adjoint Newton algorithm is only useful for optimal control problems where the model equations serve as strong constraints and their corresponding tangent linear model may be integrated backwards in time. When the backwards integration of the tangent linear model is ill-posed in the sense of Hadamard, the adjoint Newton algorithm may not work. Thus, the adjoint Newton algorithm must be used with some caution. A possible solution to avoid the current weakness of the adjoint Newton algorithm is proposed.

## Preview

Unable to display preview. Download preview PDF.

## References

- 1.A. Bennett, Inverse Problem in Physical Oceanography, Cambridge University Press, 1992, pp. 346.Google Scholar
- 2.M.S. Berger, Nonlinearity and Functional Analysis, Academic Press: New York, 1977, pp. 417.Google Scholar
- 3.S.R. Caradus, Operator Theory of the Pseudo-Inverse, A Queen’s Papers in Pure and Applied Mathematics, No. 38, Queen’s University, Kingston, Ontario, Canada, pp. 67, 1974.Google Scholar
- 4.G.F. Carey and J.T. Oden, Finite Elements Computational Aspects, Prentice-Hall Press, 1984, vol. 3, pp. 350.Google Scholar
- 5.W.C. Davidon, "Variable metric method for minimization," A.E.C. Research and Development Report, ANL-5990 (Rev.).Google Scholar
- 6.R.S. Dembo, S.C. Eisenstat, and T. Steihaug, "Inexact Newton methods," SIAM Journal of Numerical Analysis, vol. 19, pp. 400-408, 1982.Google Scholar
- 7.J. Dennis and Robert B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall: Englewood Cliffs, NJ, 1983, pp. 378.Google Scholar
- 8.J. Dieudonne, Foundations of Modern Analysis, Academic Press: New York, 1960, pp. 361.Google Scholar
- 9.John Fritz, Partial Differential Equations, 4th edition, Springer-Verlag: New York, 1986, pp. 247.Google Scholar
- 10.P.E. Gill and W. Murray, "Quasi-Newton methods for unconstrained optimization," J. Inst. Maths Applics, vol. 9, pp. 91-108, 1972.Google Scholar
- 11.P.E. Gill and W. Murray, Practical Optimization, Academic Press, 1981, pp. 401.Google Scholar
- 12.G.H. Golub and C.F. Van Loan, Matrix Computations, 2nd edition, The Johns Hopkins University Press: Baltimore and London, 1989, pp. 642.Google Scholar
- 13.A. Grammeltvedt, "A survey of finite-difference schemes for the primitive equations for a barotropic fluid," Mon. Wea. Rev., vol. 97, pp. 387-404, 1969.Google Scholar
- 14.R.N. Hoffmann, "SASS wind ambiguity removal by direct minimization," Mon. Wea. Rev., vol. 110, pp. 434-445, 1982.Google Scholar
- 15.R.N. Hoffmann, "SASS wind ambiguity removal by direct minimization Part II: Use of smoothness and dynamical constraints," Mon. Wea. Rev., vol. 112, pp. 1829-1852, 1984.Google Scholar
- 16.R.N. Hoffmann, "A four dimensional analysis exactly satisfying equations of motion," Mon. Wea. Rev., vol. 114, pp. 388-397, 1986.Google Scholar
- 17.J.F. Lacarra and O. Talagrand, Short-range evolution of small perturbations in a barotropic model," Tellus, vol. 40A, pp. 81-95, 1988.Google Scholar
- 18.F.X. Le Dimet and O. Talagrand, "Variational algorithms for analysis and assimilation of meteorological observations: Theoretical aspects," Tellus, vol. 38A, pp. 97-110, 1986.Google Scholar
- 19.J.L. Lions, Optimal control of systems governed by partial differential equations," Translated by S.K. Mitter, Springer-Verlag: Berlin-Heidelberg, 1971, pp. 404.Google Scholar
- 20.D.C. Liu and Jorge Nocedal, "On the limited memory BFGS method for large scale minimization," Mathematical Programming, vol. 45, pp. 503-528, 1989.Google Scholar
- 21.David G. Luenberger, Linear and Nonlinear Programming, 2nd edition, Addison-Wesley: Reading, MA, 1984, pp. 491.Google Scholar
- 22.S.G. Nash, "Truncated-Newton methods for large-scale function minimization,” in Applications of Nonlinear Programming to Optimization and Control, H.E. Rauch (Ed.), Pergamon Press: Oxford, 1984, pp. 91-100.Google Scholar
- 23.S.G. Nash, "Solving nonlinear programming problems using truncated Newton techniques," Numerical Optimization, P.T. Boggs, R.H. Byrd, and R.B. Schnabel (Eds.), SIAM: Philadelphia, 1984, pp. 119-136.Google Scholar
- 24.S.G. Nash, "Preconditioning of truncated-Newton methods," SIAM J. Sci. Stat. Comput., vol. 6, no.3, pp. 599-616, 1985.Google Scholar
- 25.S.G. Nash and Jorge Nocedal, "A numerical study of the limited memory BFGS method and the truncated-Newton method for large-scale optimization," Tech. Rep. NAM, 02, Department of Electrical Engineering and Computer Science, Northwestern University, 1989, p. 19.Google Scholar
- 26.I.M. Navon and D.M. Legler, "Conjugate gradient methods for large-scale minimization in meteorology," Mon. Wea. Rev., vol. 115, pp. 1479-1502, 1987.Google Scholar
- 27.I.M. Navon, X.L. Zou, J. Derber, and J. Sela, "Variational data assimilation with an adiabatic version of the NMC spectral model," Mon. Wea. Rev., vol. 122, pp. 1433-1446, 1992.Google Scholar
- 28.J. Nocedal, "Updating quasi-Newton matrices with limited storage," Mathematics of Computation, vol. 35, pp. 773-782, 1980.Google Scholar
- 29.D.P. O’Leary, "A discrete Newton algorithm for minimizing a function of many variables," Math. Prog., vol. 23, pp. 20-23, 1983.Google Scholar
- 30.Z. Pu, E. Kalnay, and J. Sela, "Sensitivity of forecast error to initial conditions with a quasi-inverse linear method," Mon. Wea. Rev., 1996, accepted for publication.Google Scholar
- 31.C.R. Rao and S.K. Mitra, Generalized Inverse of Matrices and its Applications to Statistics, John Wiley & and Sons, 1971, p. 240.Google Scholar
- 32.Fadil Santosa and William W. Symes, "Computation of the Hessian for least-squares solutions of inverse problems of reflection seismology," Inverse Problems, vol. 4, pp. 211-233, 1988.Google Scholar
- 33.Fadil Santosa and William W. Symes, "An analysis of least squares velocity inversion," Society of Exploration Geophysicists, Geophysical Monograph #4, Tulsa, 1989.Google Scholar
- 34.T. Schlick and A. Fogelson, "TNPACK-Atruncated Newton minimization package for large-scale problems: I. Algorithm and usage," ACMTOMS, vol. 18, no.1, pp. 46-70, 1992a.Google Scholar
- 35.T. Schlick and A. Fogelson, "TNPACK-Atruncated Newton minimization package for large-scale problems: II. Implementation examples," ACMTOMS, vol. 18, no.1, pp. 71-111, 1992b.Google Scholar
- 36.D.F. Shanno and K.H. Phua, "Remark on algorithm 500-A variable method subroutine for unconstrained nonlinear minimization," ACM Trans. on Mathematical Software, vol. 6, pp. 618-622, 1980.Google Scholar
- 37.J. Stoer and R. Bulirsch, Introduction to Numerical Analysis, 2nd edition, Springer-Verlag: New York, 1976, pp. 659.Google Scholar
- 38.William W. Symes, "A differential semblance algorithm for the inverse problem of reflection seismology," Computers Math. Applic., vol. 22, nos.4/5, pp. 147-178, 1991.Google Scholar
- 39.O. Talagrand and P. Courtier, "Variational assimilation of meteorological observations with the adjoint vorticity equation-Part 1. Theory," Q. J. R. Meteorol. Soc., vol. 113, pp. 1311-1328, 1987.Google Scholar
- 40.Zhi Wang, "Variational data assimilation with 2D shallow water equations and 3DFSU global spectral models," Tech. Rep. FSU-SCRI-93T-149, Florida State University, Tallahassee, Florida, 1993, p. 235.Google Scholar
- 41.Zhi Wang, I.M. Navon, F.X. Le Dimet, and X. Zou, "The second order adjoint analysis: Theory and application," Meteorol. and Atmos. Phy., vol. 50, pp. 3-20, 1992.Google Scholar
- 42.Zhi Wang, I.M. Navon, X. Zou, and F.X. Le Dimet, "A truncated Newton optimization algorithm in meteorology applications with analytic Hessian/vector products," Computational Optimization and Applications, vol. 4, no.3, pp. 241-262, 1995.Google Scholar
- 43.Zhi Wang, Kelvin K. Droegemeier, and L. White, "Application of a New Adjoint Newton Algorithm to the 3-D ARPS Storm Scale Model Using Simulated Data," Accepted for publication by Mon. Wea. Rev., 1997.Google Scholar
- 44.X.L. Zou, I.M. Navon, and F.X. Le Dimet, "Incomplete observations and control of gravity waves in variational data assimilation," Tellus, vol. 44A, pp. 273-296, 1992.Google Scholar