Abstract
A new algorithm is presented for carrying out large-scale unconstrained optimization required in variational data assimilation using the Newton method. The algorithm is referred to as the adjoint Newton algorithm. The adjoint Newton algorithm is based on the first- and second-order adjoint techniques allowing us to obtain the Newton line search direction by integrating a tangent linear equations model backwards in time (starting from a final condition with negative time steps). The error present in approximating the Hessian (the matrix of second-order derivatives) of the cost function with respect to the control variables in the quasi-Newton type algorithm is thus completely eliminated, while the storage problem related to the Hessian no longer exists since the explicit Hessian is not required in this algorithm. The adjoint Newton algorithm is applied to three one-dimensional models and to a two-dimensional limited-area shallow water equations model with both model generated and First Global Geophysical Experiment data. We compare the performance of the adjoint Newton algorithm with that of truncated Newton, adjoint truncated Newton, and LBFGS methods. Our numerical tests indicate that the adjoint Newton algorithm is very efficient and could find the minima within three or four iterations for problems tested here. In the case of the two-dimensional shallow water equations model, the adjoint Newton algorithm improves upon the efficiencies of the truncated Newton and LBFGS methods by a factor of at least 14 in terms of the CPU time required to satisfy the same convergence criterion.
The Newton, truncated Newton and LBFGS methods are general purpose unconstrained minimization methods. The adjoint Newton algorithm is only useful for optimal control problems where the model equations serve as strong constraints and their corresponding tangent linear model may be integrated backwards in time. When the backwards integration of the tangent linear model is ill-posed in the sense of Hadamard, the adjoint Newton algorithm may not work. Thus, the adjoint Newton algorithm must be used with some caution. A possible solution to avoid the current weakness of the adjoint Newton algorithm is proposed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Change history
20 April 2019
It has come to our attention that the ���Adjoint Newton Algorithm��� has been published within the following papers.
20 April 2019
It has come to our attention that the ���Adjoint Newton Algorithm��� has been published within the following papers.
21 August 2019
The original article can be found online at
20 April 2019
It has come to our attention that the ���Adjoint Newton Algorithm��� has been published within the following papers.
21 August 2019
The original article can be found online at
20 April 2019
It has come to our attention that the ���Adjoint Newton Algorithm��� has been published within the following papers.
21 August 2019
The original article can be found online at
References
A. Bennett, Inverse Problem in Physical Oceanography, Cambridge University Press, 1992, pp. 346.
M.S. Berger, Nonlinearity and Functional Analysis, Academic Press: New York, 1977, pp. 417.
S.R. Caradus, Operator Theory of the Pseudo-Inverse, A Queen’s Papers in Pure and Applied Mathematics, No. 38, Queen’s University, Kingston, Ontario, Canada, pp. 67, 1974.
G.F. Carey and J.T. Oden, Finite Elements Computational Aspects, Prentice-Hall Press, 1984, vol. 3, pp. 350.
W.C. Davidon, "Variable metric method for minimization," A.E.C. Research and Development Report, ANL-5990 (Rev.).
R.S. Dembo, S.C. Eisenstat, and T. Steihaug, "Inexact Newton methods," SIAM Journal of Numerical Analysis, vol. 19, pp. 400-408, 1982.
J. Dennis and Robert B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall: Englewood Cliffs, NJ, 1983, pp. 378.
J. Dieudonne, Foundations of Modern Analysis, Academic Press: New York, 1960, pp. 361.
John Fritz, Partial Differential Equations, 4th edition, Springer-Verlag: New York, 1986, pp. 247.
P.E. Gill and W. Murray, "Quasi-Newton methods for unconstrained optimization," J. Inst. Maths Applics, vol. 9, pp. 91-108, 1972.
P.E. Gill and W. Murray, Practical Optimization, Academic Press, 1981, pp. 401.
G.H. Golub and C.F. Van Loan, Matrix Computations, 2nd edition, The Johns Hopkins University Press: Baltimore and London, 1989, pp. 642.
A. Grammeltvedt, "A survey of finite-difference schemes for the primitive equations for a barotropic fluid," Mon. Wea. Rev., vol. 97, pp. 387-404, 1969.
R.N. Hoffmann, "SASS wind ambiguity removal by direct minimization," Mon. Wea. Rev., vol. 110, pp. 434-445, 1982.
R.N. Hoffmann, "SASS wind ambiguity removal by direct minimization Part II: Use of smoothness and dynamical constraints," Mon. Wea. Rev., vol. 112, pp. 1829-1852, 1984.
R.N. Hoffmann, "A four dimensional analysis exactly satisfying equations of motion," Mon. Wea. Rev., vol. 114, pp. 388-397, 1986.
J.F. Lacarra and O. Talagrand, Short-range evolution of small perturbations in a barotropic model," Tellus, vol. 40A, pp. 81-95, 1988.
F.X. Le Dimet and O. Talagrand, "Variational algorithms for analysis and assimilation of meteorological observations: Theoretical aspects," Tellus, vol. 38A, pp. 97-110, 1986.
J.L. Lions, Optimal control of systems governed by partial differential equations," Translated by S.K. Mitter, Springer-Verlag: Berlin-Heidelberg, 1971, pp. 404.
D.C. Liu and Jorge Nocedal, "On the limited memory BFGS method for large scale minimization," Mathematical Programming, vol. 45, pp. 503-528, 1989.
David G. Luenberger, Linear and Nonlinear Programming, 2nd edition, Addison-Wesley: Reading, MA, 1984, pp. 491.
S.G. Nash, "Truncated-Newton methods for large-scale function minimization,” in Applications of Nonlinear Programming to Optimization and Control, H.E. Rauch (Ed.), Pergamon Press: Oxford, 1984, pp. 91-100.
S.G. Nash, "Solving nonlinear programming problems using truncated Newton techniques," Numerical Optimization, P.T. Boggs, R.H. Byrd, and R.B. Schnabel (Eds.), SIAM: Philadelphia, 1984, pp. 119-136.
S.G. Nash, "Preconditioning of truncated-Newton methods," SIAM J. Sci. Stat. Comput., vol. 6, no.3, pp. 599-616, 1985.
S.G. Nash and Jorge Nocedal, "A numerical study of the limited memory BFGS method and the truncated-Newton method for large-scale optimization," Tech. Rep. NAM, 02, Department of Electrical Engineering and Computer Science, Northwestern University, 1989, p. 19.
I.M. Navon and D.M. Legler, "Conjugate gradient methods for large-scale minimization in meteorology," Mon. Wea. Rev., vol. 115, pp. 1479-1502, 1987.
I.M. Navon, X.L. Zou, J. Derber, and J. Sela, "Variational data assimilation with an adiabatic version of the NMC spectral model," Mon. Wea. Rev., vol. 122, pp. 1433-1446, 1992.
J. Nocedal, "Updating quasi-Newton matrices with limited storage," Mathematics of Computation, vol. 35, pp. 773-782, 1980.
D.P. O’Leary, "A discrete Newton algorithm for minimizing a function of many variables," Math. Prog., vol. 23, pp. 20-23, 1983.
Z. Pu, E. Kalnay, and J. Sela, "Sensitivity of forecast error to initial conditions with a quasi-inverse linear method," Mon. Wea. Rev., 1996, accepted for publication.
C.R. Rao and S.K. Mitra, Generalized Inverse of Matrices and its Applications to Statistics, John Wiley & and Sons, 1971, p. 240.
Fadil Santosa and William W. Symes, "Computation of the Hessian for least-squares solutions of inverse problems of reflection seismology," Inverse Problems, vol. 4, pp. 211-233, 1988.
Fadil Santosa and William W. Symes, "An analysis of least squares velocity inversion," Society of Exploration Geophysicists, Geophysical Monograph #4, Tulsa, 1989.
T. Schlick and A. Fogelson, "TNPACK-Atruncated Newton minimization package for large-scale problems: I. Algorithm and usage," ACMTOMS, vol. 18, no.1, pp. 46-70, 1992a.
T. Schlick and A. Fogelson, "TNPACK-Atruncated Newton minimization package for large-scale problems: II. Implementation examples," ACMTOMS, vol. 18, no.1, pp. 71-111, 1992b.
D.F. Shanno and K.H. Phua, "Remark on algorithm 500-A variable method subroutine for unconstrained nonlinear minimization," ACM Trans. on Mathematical Software, vol. 6, pp. 618-622, 1980.
J. Stoer and R. Bulirsch, Introduction to Numerical Analysis, 2nd edition, Springer-Verlag: New York, 1976, pp. 659.
William W. Symes, "A differential semblance algorithm for the inverse problem of reflection seismology," Computers Math. Applic., vol. 22, nos.4/5, pp. 147-178, 1991.
O. Talagrand and P. Courtier, "Variational assimilation of meteorological observations with the adjoint vorticity equation-Part 1. Theory," Q. J. R. Meteorol. Soc., vol. 113, pp. 1311-1328, 1987.
Zhi Wang, "Variational data assimilation with 2D shallow water equations and 3DFSU global spectral models," Tech. Rep. FSU-SCRI-93T-149, Florida State University, Tallahassee, Florida, 1993, p. 235.
Zhi Wang, I.M. Navon, F.X. Le Dimet, and X. Zou, "The second order adjoint analysis: Theory and application," Meteorol. and Atmos. Phy., vol. 50, pp. 3-20, 1992.
Zhi Wang, I.M. Navon, X. Zou, and F.X. Le Dimet, "A truncated Newton optimization algorithm in meteorology applications with analytic Hessian/vector products," Computational Optimization and Applications, vol. 4, no.3, pp. 241-262, 1995.
Zhi Wang, Kelvin K. Droegemeier, and L. White, "Application of a New Adjoint Newton Algorithm to the 3-D ARPS Storm Scale Model Using Simulated Data," Accepted for publication by Mon. Wea. Rev., 1997.
X.L. Zou, I.M. Navon, and F.X. Le Dimet, "Incomplete observations and control of gravity waves in variational data assimilation," Tellus, vol. 44A, pp. 273-296, 1992.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wang, Z., Droegemeier, K. & White, L. The Adjoint Newton Algorithm for Large-Scale Unconstrained Optimization in Meteorology Applications. Computational Optimization and Applications 10, 283–320 (1998). https://doi.org/10.1023/A:1018321307393
Issue Date:
DOI: https://doi.org/10.1023/A:1018321307393