Optimization Theory

Part of the Statistics and Computing book series (SCO)


This chapter summarizes a handful of basic principles that permit the exact solution of many optimization problems. Misled by the beautiful examples of elementary calculus, students are disappointed when they cannot solve optimization problems analytically. More experienced scholars know that exact solutions are the exception rather than the rule. However, they cherish these exceptions because they form the basis of most iteration schemes in optimization.


Equality Constraint Optimization Theory Tangent Vector Inequality Constraint Loglinear Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arthur D, Vassilvitskii S (2007): k-means++: the advantages of careful seeding. 2007 Symposium on Discrete Algorithms (SODA)Google Scholar
  2. 2.
    Beltrami EJ (1970) An Algorithmic Approach to Nonlinear Analysis and Optimization. Academic Press, New YorkMATHGoogle Scholar
  3. 3.
    Bishop YMM, Feinberg SE, Holland PW (1975) Discrete Multivariate Analysis: Theory and Practice. MIT Press, Cambridge, MAMATHGoogle Scholar
  4. 4.
    Boyd S, Vandenberghe L (2004) Convex Optimization. Cambridge University Press, CambridgeMATHGoogle Scholar
  5. 5.
    Brinkhuis J, Tikhomirov V (2005) Optimization: Insights and Applications. Princeton University Press, Princeton, NJMATHGoogle Scholar
  6. 6.
    Ciarlet PG (1989) Introduction to Numerical Linear Algebra and Optimization. Cambridge University Press, CambridgeGoogle Scholar
  7. 7.
    de Leeuw J (1994) Block relaxation algorithms in statistics. in Information Systems and Data Analysis, Bock HH, Lenski W, Richter MM, Springer, BerlinGoogle Scholar
  8. 8.
    de Souza PN, Silva J-N (2001) Berkeley Problems in Mathematics, 2nd ed. Springer, New YorkMATHGoogle Scholar
  9. 9.
    Everitt BS (1977) The Analysis of Contingency Tables. Chapman & Hall, LondonGoogle Scholar
  10. 10.
    Forsgren A, Gill PE, Wright MH (2002) Interior point methods for nonlinear optimization. SIAM Review 44:523-597CrossRefMathSciNetGoogle Scholar
  11. 11.
    Gabriel KR, Zamir S (1979) Lower rank approximation of matrices by least squares with any choice of weights. Technometrics 21:489-498MATHCrossRefGoogle Scholar
  12. 12.
    Hestenes MR (1981) Optimization Theory: The Finite Dimensional Case. Robert E Krieger Publishing, Huntington, NYGoogle Scholar
  13. 13.
    Kosowsky JJ, Yuille AL (1994) The invisible hand algorithm: solving the assignment problem with statistical physics. Neural Networks 7:477-490MATHCrossRefGoogle Scholar
  14. 14.
    Ku HH, Kullback S (1974) Log-linear models in contingency table analysis. Biometrics 10:452-458Google Scholar
  15. 15.
    Kuhn HW, Tucker AW (1951) Nonlinear programming. in Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability. University of California Press, Berkeley, pp 481-492Google Scholar
  16. 16.
    Lange, K (2004) Optimization. Springer, New YorkMATHGoogle Scholar
  17. 17.
    Luenberger DG (1984) Linear and Nonlinear Programming, 2nd ed. Addison-Wesley, Reading, MAMATHGoogle Scholar
  18. 18.
    Maher MJ (1982), Modelling association football scores. Statistica Neerlandica 36:109-118CrossRefGoogle Scholar
  19. 19.
    Mangasarian OL, Fromovitz S (1967) The Fritz John necessary optimality conditions in the presence of equality and inequality constraints. J Math Anal Appl 17:37-47MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Mardia KV, Kent JT, Bibby JM (1979) Multivariate Analysis. Academic, New YorkMATHGoogle Scholar
  21. 21.
    McShane EJ (1973) The Lagrange multiplier rule. Amer Math Monthly 80:922-925MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Peressini AL, Sullivan FE, Uhl JJ Jr (1988) The Mathematics of Nonlinear Programming. Springer, New YorkMATHGoogle Scholar
  23. 23.
    Ruszczynski A (2006) Nonlinear Optimization. Princeton University Press. Princeton, NJMATHGoogle Scholar
  24. 24.
    Sinkhorn R (1967) Diagonal equivalence to matrices with prescribed row and column sums. Amer Math Monthly 74:402-405MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer New York 2010

Authors and Affiliations

  1. 1.Departments of Biomathematics, Human Genetics, and Statistics David Geffen School of MedicineUniversity of California, Los AngelesLos AngelesUSA

Personalised recommendations