Abstract
We analyze an algorithm for the problem minf(x) s.t.x ⩾ 0 suggested, without convergence proof, by Eggermont. The iterative step is given by x k+1j =x kj (1-λk▽f(x k)j) with λk > 0 determined through a line search. This method can be seen as a natural extension of the steepest descent method for unconstrained optimization, and we establish convergence properties similar to those known for steepest descent, namely weak convergence to a KKT point for a generalf, weak convergence to a solution for convexf and full convergence to the solution for strictly convexf. Applying this method to a maximum likelihood estimation problem, we obtain an additively overrelaxed version of the EM Algorithm. We extend the full convergence results known for EM to this overrelaxed version by establishing local Fejér monotonicity to the solution set.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cover, T. M.: An algorithm for maximizing expected log investment return,IEEE Trans. Information Theory IT30 (1984), 369–373.
Csiszár, I. and Tusnády, G.: Information geometry and alternating minimization procedures,Statistics and Decisions, Suppl. 1 (1984), 205–237.
Dempster, A. P., Laird, M. N. and Rubin, D. B.: Maximum likelihood from incomplete data via the EM algorithm,J. Royal Statist. Soc. Series B 37 (1977), 1–38.
Dennis, J. E. and Schnabel, R. B.:Numerical Methods for Unconstrained Minimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, 1983.
Eggermont, P. P. B.: Multiplicative iterative algorithms for convex programming.Linear Algebra Appl. 130 (1990), 25–42.
Iusem, A. N.: Convergence analysis for a multiplicatively relaxed EM algorithm,Math. Methods Appl. Sci. 14 (1991), 673–593.
Iusem, A. N.: On the convergence of iterative methods for symmetric linear complementary problems,Math. Programming 59 (1993), 33–49.
Iusem, A. N.: A short convergence proof of the EM algorithm for a specific Poisson model,Rev. Brasileira Probab. Estatistica 6 (1992), 57–67.
Iusem, A. N. and Teboulle, M.: A regularized dual based iterative method for a class of image reconstruction problems,Inverse Problems 9 (1993), 679–696.
Karmarkar, N.: A new polynomial time algorithm for linear programming,Combinatorica 4 (1984), 373–395.
Meng, X.-L. and Pedlow, S.: EM: a bibliographic review with missing articles,ASA Proceedings, 1992.
Ostrowsky, A. M.:Solutions of Equations and Systems of Equations, Academic Press, New York, 1966.
Pang, J. S.: Necessary and sufficient conditions for the convergence of iterative methods for the linear complementarity problem,J. Optim. Theory Appl. 42 (1984), 1–17.
Polyak, B.:Introduction to Optimization, Optimization Software, New York, 1987.
Vardi, Y., Shepp, L. A. and Kaufman, L.: A statistical model for Positron Emission Tomography,J. Amer. Statist. Assoc. 80 (1985), 8–37.
Author information
Authors and Affiliations
Additional information
Research for this paper was partially supported by CNPq grant No 301280/86.
Rights and permissions
About this article
Cite this article
Iusem, A.N. An interior point multiplicative method for optimization under positivity constraints. Acta Appl Math 38, 163–184 (1995). https://doi.org/10.1007/BF00992845
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF00992845