Abstract
A general primal—dual algorithm for linearly constrained optimization problems is formulated in which the dual variables are updated by a dual algorithmic operator. Convergence is proved under the assumption that the dual algorithmic operator implies asymptotic feasibility of the primal iterates with respect to the linear constraints. A general result relating the minimal values of an infinite sequence of constrained problems to the minimal value of a limiting problem (constrained by the limit of the sequence of constraints sets) is established and invoked. The applicability of the general theory is demonstrated by analyzing a specific dual algorithmic operator. This leads to the “MART” algorithm for constrained entropy maximization used in image reconstruction from projections.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
J. Aczél and Z. Daróczy,On Measures of Information and Their Characterizations (Academic Press, New York, 1975).
A. Ben-Israel, “Linear equations and inequalities on finite dimensional, real or complex, vector spaces: A unified theory,”Journal of Mathematical Analysis and Applications 27 (1969) 367–389.
A. Ben-Israel and T.N.E. Greville,Generalized Inverses: Theory and Applications (Wiley, New York, 1974).
L.M. Bregman, “The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming,”USSR Computational Mathematics and Mathematical Physics 7 (1967) 200–217.
Y. Censor, “Row-action methods for huge and sparse systems and their applications,”SIAM Review 23 (1981) 444–466.
Y. Censor, “Finite series-expansion reconstruction methods,”Proceedings of the IEEE 71 (1983) 409–419.
Y. Censor, A.R. De Pierro, T. Elfving, G.T. Herman and A.N. Iusem, “On iterative methods for linearly constrained entropy maximization,” in: A. Wakulicz, ed.,Numerical Analysis and Mathematical Modelling, Banach Center Publications, Vol. XXIV (PWN — Polish Scientific Publishers, Warsaw, Poland, 1990) pp. 145–163.
Y. Censor, A.R. De Pierro and A.N. Iusem, “Optimization of Burg's entropy over linear constraints,”Applied Numerical Mathematicss 7 (1991) 151–165.
Y. Censor and A. Lent, “An iterative row-action method for interval convex programming,”Journal of Optimization Theory and Applications 34 (1981) 321–353.
Y. Censor and A. Lent, “Optimization of ‘logx’ entropy over linear equality constraints,”SIAM Journal on Control and Optimization 25 (1987) 921–933.
Y. Censor and J. Segman, “On block-iterative entropy maximization,”Journal of Information and Optimization Sciences 8 (1987) 275–291.
J.S. Daniel, “On perturbations in systems of linear inequalities,”SIAM Journal on Numerical Analysis 10 (1973) 299–307.
G.B. Dantzig, J. Folkman and N. Shapiro, “On the continuity of the minimum set of a continuous function,”Journal of Mathematical Analysis and Applications 17 (1967) 519–548.
A.R. De Pierro and A.N. Iusem, “A relaxed version of Bregman's method for convex programming,”Journal of Optimization Theory and Applications 51 (1986) 421–440.
J.N. Darroch and D. Ratcliff, “Generalized iterative scaling for log-linear models,”The Annals of Mathematical Statistics 43 (1972) 1470–1480.
T. Elfving, “On some methods for entropy maximization and matrix scaling,”Linear Algebra and its Applications 34 (1980) 321–339.
S. Erlander, “Entropy in linear programs,”Mathematical Programming 21 (1981) 137–151.
H. Everett, III, “Generalized Lagrange multiplier method for solving problems of optimum allocation of resources,”Operations Research 11 (1963) 399–417.
B.R. Frieden, “Statistical models for the image restoration problem,”Computer Graphics and Image Processing 12 (1980) 49–59.
R. Gordon, R. Bender and G.T. Herman, “Algebraic reconstruction techniques (ART) for threedimensional electron microscopy and X-ray photography,”Journal of Theoretical Biology 29 (1970) 471–481.
G.T. Herman,Image Reconstruction from Projections: The Fundamentals of Computerized Tomography (Academic Press, New York, 1980).
G.T. Herman, “Mathematical optimization versus practical performance: A case study based on the maximum entropy criterion in image reconstruction,”Mathematical Programming Study 20 (1982) 96–112.
G.T. Herman, “Application of maximum entropy and Bayesian optimization methods to image reconstruction from projections,” in: C. Ray Smith and W.T. Grandy Jr., eds.,Maximum-Entropy and Bayesian Methods in Inverse Problems (Reidel, Dordrecht, 1985) pp. 319–338.
A.J. Hoffman, “On approximate solutions of systems of linear inequalities,”Journal of Research of the National Bureau of Standards 49 (1952) 263–265.
E.T. Jaynes, “On the rationale of maximum entropy methods,”Proceedings of the IEEE 70 (1982) 939–952.
J.N. Kapur, “Twenty-five years of the maximum-entropy principle,”Journal of Mathematical and Physical Sciences 17 (1983) 103–156.
J.N. Kapur, “A comparative assessment of various measures of entropy,”Journal of Information and Optimization Sciences 4 (1983) 207–232.
B. Lamond and N.F. Stewart, “Bregman's balancing method,”Transportation Research 15B (1981) 239–248.
A. Lent, “A convergent algorithm for maximum entropy image restoration, with a medical x-ray application,” in: R. Shaw, ed.,Image Analysis and Evaluation (Society of Photographic Scientists and Engineers, SPSE, Washington, D.C., 1977) pp. 249–257.
R.D. Levine and M. Tribus, Editors,The Maximum Entropy Formalism (MIT Press, Cambridge, MA, 1978).
D.G. Luenberger,Optimization by Vector Space Methods (Wiley, New York, 1969).
D.G. Luenberger,Introduction to Linear and Nonlinear Programming (Addison-Wesley, Reading, MA, 1973).
G. Salinetti and R.J.-B. Wets, “On the convergence of sequences of convex sets in finite dimensions,”SIAM Review 21 (1979) 18–33. Addendum:SIAM Review 22 (1980) 86.
F. Snickars and J.W. Weibull, “A minimum information principle: Theory and practice,”Regional Science and Urban Economics 7 (1977) 137–168.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Lent, A., Censor, Y. The primal-dual algorithm as a constraint-set-manipulation device. Mathematical Programming 50, 343–357 (1991). https://doi.org/10.1007/BF01594943
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01594943