Abstract
The Legendre transform (LET) is a product of a general duality principle: any smooth curve is, on the one hand, a locus of pairs, which satisfy the given equation and, on the other hand, an envelope of a family of its tangent lines.
An application of the LET to a strictly convex and smooth function leads to the Legendre identity (LEID). For strictly convex and three times differentiable function the LET leads to the Legendre invariant (LEINV).
Although the LET has been known for more then 200 years both the LEID and the LEINV are critical in modern optimization theory and methods.
The purpose of the paper is to show the role of the LEID and the LEINV play in both constrained and unconstrained optimization.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alber, M., Reemtsen, R.: Intensity modulated radiotherapy treatment planning by use of a barrier-penalty multiplier method. Optim. Methods Softw. 22 (3), 391–411 (2007)
Antipin, A.S.: Methods of Nonlinear Programming Based on the Direct and Dual Augmentation of the Lagrangian. VNIISI, Moscow (1979)
Auslender, R., Cominetti, R., Haddou, M.: Asymptotic analysis for penalty and barrier methods in convex and linear programming. Math. Oper. Res. 22 (1), 43–62 (1997)
Bauschke, H., Matouskova, E., Reich, S.: Projection and proximal point methods, convergence results and counterexamples. Nonlinear Anal. 56 (5), 715–738 (2004)
Ben-Tal, A., Nemirovski, A.: Optimal design of engineering structures. Optima 47, 4–9 (1995)
Ben-Tal, A., Zibulevski, M.: Penalty-barrier methods for convex programming problems. SIAM J. Optim. 7, 347–366 (1997)
Bertsekas, D.: Constrained Optimization and Lagrange Multiplier Methods. Academic Press, New York (1982)
Bregman, L.: The relaxation method for finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 7, 200–217 (1967)
Bregman, L., Censor, Y., Reich, S.: Dykstra algorithm as the nonlinear extension of Bregman’s optimization method. J. Convex Anal. 6 (2), 319–333 (1999)
Breitfeld, M., Shanno, D.: Computational experience with modified log-barrier methods for nonlinear programming. Ann. Oper. Res. 62, 439–464 (1996)
Byrne, C., Censor, Y.: Proximity function minimization using multiple Bregman projections with application to split feasibility and Kullback-Leibler distance minimization. Ann. Oper. Res. 105, 77–98 (2001)
Carroll, C.: The created response surface technique for optimizing nonlinear-restrained systems. Oper. Res. 9 (2), 169–184 (1961)
Censor, Y., Zenios, S.: The proximal minimization algorithm with d-functions. J. Optim. Theory Appl. 73, 451–464 (1992)
Chen, C., Mangasarian, O.L.: Smoothing methods for convex inequalities and linear complementarity problems. Math. Program. 71, 51–69 (1995)
Chen, G., Teboulle, M.: Convergence analysis of a proximal-like minimization algorithm using Bregman functions. SIAM J. Optim. 3 (4), 538–543 (1993)
Courant, R.: Variational methods for the solution of problems of equilibrium and vibrations. Bull. Am. Math. Soc. 49, 1–23 (1943)
Daube-Witherspoon, M., Muehllehner, G.: An iterative space reconstruction algorithm suitable for volume ECT. IEEE Trans. Med Imaging 5, 61–66 (1986)
Dikin, I.: Iterative solutions of linear and quadratic programming problems. Sov. Math. Dokl. 8, 674–675 (1967)
Eckstein, J.: Nonlinear proximal point algorithms using Bregman functions with applications to convex programming. Math. Oper. Res. 18 (1), 202–226 (1993)
Eggermont, P.: Multiplicative iterative algorithm for convex programming. Linear Algebra Appl. 130, 25–32 (1990)
Ekeland, I.: Legendre duality in nonconvex optimization and calculus of variations. SIAM J. Control. Optim. 16 (6), 905–934 (1977)
Fiacco, A., Mc Cormick, G.: Nonlinear Programming, Sequential Unconstrained Minimization Techniques. SIAM, Philadelphia (1990)
Frisch, K.: The logarithmic potential method for convex programming. Memorandum of May 13 1955, University Institute of Economics, Oslo (1955)
Goldshtein, E., Tretiakov, N.: Modified Lagrangian Functions. Nauka, Moscow (1989)
Griva, I., Polyak, R.: Primal-dual nonlinear rescaling method with dynamic scaling parameter update. Math. Program. Ser. A 106, 237–259 (2006)
Griva, I., Polyak, R.: Proximal point nonlinear rescaling method for convex optimization. Numer. Algebra Control Optim. 1 (3), 283–299 (2013)
Guler, O.: On the convergence of the proximal point algorithm for convex minimization. SIAM J. Control Optim. 29, 403–419 (1991)
Hestenes, M.R.: Multipliers and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)
Hiriat-Urruty, J., Martinez-Legaz, J.: New formulas for the Legendre-Fenchel transform. J. Math. Anal. Appl. 288, 544–555 (2003)
Ioffe, A., Tichomirov, V.: Duality of convex functions and extremum problems. Uspexi Mat. Nauk 23 (6)(144), 51–116 (1968)
Jensen, D., Polyak, R.: The convergence of a modify barrier method for convex programming. IBM J. Res. Dev. 38 (3), 307–321 (1999)
Kocvara, M., Stingl, M.: PENNON. A code for convex nonlinear and semidefinite programming. Optim. Methods Softw. 18 (3), 317–333 (2003)
Kocvara, M., Stingl, M.: Recent progress in the NLP-SDP code PENNON. In: Workshop Optimization and Applications, Oberwalfach (2005)
Kocvara, M., Stingl, M.: On the solution of large-scale SDP problems by the modified barrier method using iterative solver. Math. Program. Ser. B 109, 413–444 (2007)
Martinet, B.: Regularization d’inequations variationelles par approximations successive. Rev. Fr. Inf. Res. Ofer 4 (R3), 154–159 (1970)
Martinet, B.: Determination approachee d’un point fixe d’une application pseudo-contractante. C.R. Acad. Sci. Paris 274 (2), 163–165 (1972)
Matioli, L., Gonzaga, C.: A new family of penalties for augmented Lagrangian methods. Numer. Linear Algebra Appl. 15, 925–944 (2008)
Melman, A., Polyak, R.: The Newton modified barrier method for QP problems. Ann. Oper. Res. 62, 465–519 (1996)
Moreau, J.: Proximite et dualite dans un espace Hilbertien. Bull. Soc. Math. France 93, 273–299 (1965)
Motzkin, T.: New techniques for linear inequalities and optimization. In: project SCOOP, Symposium on Linear Inequalities and Programming, Planning Research Division, Director of Management Analysis Service, U.S. Air Force, Washington, D.C., no. 10, (1952)
Nash, S., Polyak, R., Sofer, A.: A numerical comparison of barrier and modified barrier method for large scale bound-constrained optimization. In: Hager, W., Hearn, D., Pardalos, P. (eds.) Large Scale Optimization, State of the Art, pp. 319–338. Kluwer Academic, Dordrecht (1994)
Nesterov, Yu., Nemirovsky, A.: Interior Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia (1994)
Nesterov, Yu.: Introductory Lectures on Convex Optimization. Kluwer Academic, Norwell, MA (2004)
Polyak, B.: Iterative methods using lagrange multipliers for solving extremal problems with constraints of the equation type. Comput. Math. Math. Phys. 10 (5), 42–52 (1970)
Polyak, B.: Introduction to Optimization. Optimization Software, New York (1987)
Polyak, R.: Modified barrier functions (theory and methods). Math Program. 54, 177–222 (1992)
Polyak, R.: Log-sigmoid multipliers method in constrained optimization. Ann. Oper. Res. 101, 427–460 (2001)
Polyak, R.: Nonlinear rescaling vs. smoothing technique in convex optimization. Math. Program. 92, 197–235 (2002)
Polyak, R.: Lagrangian transformation and interior ellipsoid methods in convex optimization. J. Optim. Theory Appl. 163 (3), 966–992 (2015)
Polyak, R., Teboulle, M.: Nonlinear rescaling and proximal-like methods in convex optimization. Math. Program. 76, 265–284 (1997)
Polyak, B., Tret’yakov, N.: The method of penalty estimates for conditional extremum problems. Comput. Math. Math. Phys. 13 (1), 42–58 (1973)
Powell, M.J.D.: A method for nonlinear constraints in minimization problems. In: Fletcher (ed.) Optimization, pp. 283–298. Academic Press, London (1969)
Powell, M.: Some convergence properties of the modified log barrier methods for linear programming. SIAM J. Optim. 50 (4), 695–739 (1995)
Ray, A., Majumder, S.: Derivation of some new distributions in statistical mechanics using maximum entropy approach. Yugosl. J. Oper. Res. 24 (1), 145–155 (2014)
Reich, S., Sabach, S.: Two strong convergence theorems for a proximal method in reflexive Banach spaces. Numer. Funct. Anal. Optim. 31, 22–44 (2010)
Rockafellar, R.T.: A dual approach to solving nonlinear programming problems by unconstrained minimization. Math. Program. 5, 354–373 (1973)
Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal points algorithms in convex programming. Math. Oper. Res. 1, 97–116 (1976)
Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)
Teboulle, M.: Entropic proximal mappings with application to nonlinear programming. Math. Oper. Res. 17, 670–690 (1992)
Tikhonov, A.N.: Solution of incorrectly formulated problems and the regularization method. Sov. Math. (Translated) 4, 1035–1038 (1963)
Tseng, P., Bertsekas, D.: On the convergence of the exponential multipliers method for convex programming. Math. Program. 60, 1–19 (1993)
Vardi, Y., Shepp, L., Kaufman, L.: A statistical model for positron emission tomography. J. Am. Stat. Assoc. 80, 8–38 (1985)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Polyak, R.A. (2016). The Legendre Transformation in Modern Optimization. In: Goldengorin, B. (eds) Optimization and Its Applications in Control and Data Sciences. Springer Optimization and Its Applications, vol 115. Springer, Cham. https://doi.org/10.1007/978-3-319-42056-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-42056-1_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-42054-7
Online ISBN: 978-3-319-42056-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)