Abstract
It is desirable that a numerical maximization algorithm monotonically increase its objective function for the sake of its stability of convergence. It is here shown how one can adjust the Newton-Raphson procedure to attain monotonicity by the use of simple bounds on the curvature of the objective function. The fundamental tool in the analysis is the geometric insight one gains by interpreting quadratic-approximation algorithms as a form of area approximation. The statistical examples discussed include maximum likelihood estimation in mixture models, logistic regression and Cox's proportional hazards regression.
This is a preview of subscription content, log in via an institution to check access.
References
Andersen, E.B. (1980). Discrete Statistical Models with Social Science Applications, North-Holland, Amsterdam.
Baksalary, J. K. and Pukelsheim, F. (1985). A note on the matrix ordering of special C-matrices, Linear Algebra Appl., 70, 263–267.
Baum, L. E. and Eagon, J. A. (1967). An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology, Bull. Amer. Math. Soc., 73, 360–363.
Collatz, L. (1961). Monotonie und Extremal Prinzipien beim Newtonschen Verfahren, Numer. Math., 3, 99–106.
Cox, D. R. and Oakes, D. (1984). Analysis of Survival Data, Chapman & Hall, London-New York.
Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion), J. Roy. Statist. Soc. Ser. B, 39, 1–38.
Everitt, B. S. and Hand, D. B. (1981). Finite Mixture Distributions, Chapman & Hall, London-New York.
Horst, R. (1979). Nichtlineare Optimierung, Carl Hanser, Munchen-Wien.
Marshall, A. W. and Olkin, I. (1979). Inequalities: Theory of Majorization and Its Applications, Academic Press, New York.
McCullagh, P. and Nelder, J. A. (1983). Generalized Linear Models, Chapman & Hall, London-New York.
Miller, R. G. (1981). Survival Analysis, Wiley, New York.
Moolgavkar, S. H., Lustbader, E. D. and Venson, D. J. (1985). Assessing the adequacy of the logistic regression model for matched case-control studies, Statistics in Medicine, 4, 425–435.
Neter, J., Wasserman, W. and Kutner, M. (1985). Applied Linear Statistical Models, 2nd ed., Homewood, Irwin.
Potra, F. A. and Rheinboldt, W. C. (1986). On the monotone convergence of Newton's method, Computing, 36, 81–90.
Pregibon, D. (1981). Logistic regression diagnostics, Ann. Statist., 9, 705–724.
Sundberg, R. (1976). An iterative method for solution of the likelihood equations for incomplete data from exponential families, Comm. Statist. B—Simulation Comput., 5, 55–64.
Titterington, D. M., Smith, A. F. M. and Makov, U. E. (1985). Statistical Analysis of Finite Mixture Distributions, Wiley, Chichester-New York.
Wu, C. F. (1983). On the convergence properties of the EM algorithm, Ann. Statist., 11, 95–103.
Author information
Authors and Affiliations
Additional information
The second author's research was partially supported by the National Science Foundation under Grant DMS-8402735.
About this article
Cite this article
Böhning, D., Lindsay, B.G. Monotonicity of quadratic-approximation algorithms. Ann Inst Stat Math 40, 641–663 (1988). https://doi.org/10.1007/BF00049423
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF00049423