Abstract
This chapter reviews the basic elements of optimization theory and practice, without going into the fine details of numerical implementation. Many UQ problems involve a notion of ‘best fit’, in the sense of minimizing some error function, and so it is helpful to establish some terminology for optimization problems. In particular, many of the optimization problems in this book will fall into the simple settings of linear programming and least squares (quadratic programming), with and without constraints.
We demand rigidly defined areas of doubt and uncertainty!
The Hitchhiker’s Guide to the Galaxy
Douglas Adams
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Or, more generally, Hausdorff, locally convex, topological vector spaces.
- 2.
If Q is not positive definite, but merely positive semi-definite and self-adjoint, then existence of solutions to the associated least squares problems still holds, but uniqueness can fail.
References
B. M. Adams, L. E. Bauman, W. J. Bohnhoff, K. R. Dalbey, M. S. Ebeida, J. P. Eddy, M. S. Eldred, P. D. Hough, K. T. Hu, J. D. Jakeman, L. P. Swiler, J. A. Stephens, D. M. Vigil, and T. M. Wildey. DAKOTA, A Multilevel Parallel Object-Oriented Framework for Design Optimization, Parameter Estimation, Uncertainty Quantification, and Sensitivity Analysis: Version 6.1 User’s Manual. Technical Report SAND2014-4633, Sandia National Laboratories, November 2014. http://dakota.sandia.gov.
E. Bishop and K. de Leeuw. The representations of linear functionals by measures on sets of extreme points. Ann. Inst. Fourier. Grenoble, 9: 305–331, 1959.
S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, Cambridge, 2004.
F. J. Gould and J. W. Tolle. Optimality conditions and constraint qualifications in Banach space. J. Optimization Theory Appl., 15:667–684, 1975.
W. Karush. Minima of Functions of Several Variables with Inequalities as Side Constraints. Master’s thesis, Univ. of Chicago, Chicago, 1939.
S. Kirkpatrick, C. D. Gelatt, Jr., and M. P. Vecchi. Optimization by simulated annealing. Science, 220(4598):671–680, 1983. doi: 10.1126/ science.220.4598.671.
S. G. Krantz. Convex Analysis. Textbooks in Mathematics. CRC Press, Boca Raton, FL, 2015.
M. Krein and D. Milman. On extreme points of regular convex sets. Studia Math., 9:133–138, 1940.
H. W. Kuhn and A. W. Tucker. Nonlinear programming. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, 1950, pages 481–492. University of California Press, Berkeley and Los Angeles, 1951.
M. M. McKerns, P. Hung, and M. A. G. Aivazis. Mystic: A Simple Model-Independent Inversion Framework, 2009. http://trac.mystic.cacr.caltech.edu/project/mystic.
M. M. McKerns, L. Strand, T. J. Sullivan, P. Hung, and M. A. G. Aivazis. Building a Framework for Predictive Science. In S. van der Walt and J. Millman, editors, Proceedings of the 10th Python in Science Conference (SciPy 2011), June 2011, pages 67–78, 2011.
J. Nocedal and S. J. Wright. Numerical Optimization. Springer Series in Operations Research and Financial Engineering. Springer, New York, second edition, 2006.
R. R. Phelps. Lectures on Choquet’s Theorem, volume 1757 of Lecture Notes in Mathematics. Springer-Verlag, Berlin, second edition, 2001. doi: 10.1007/b76887.
K. V. Price, R. M. Storn, and J. A. Lampinen. Differential Evolution: A Practical Approach to Global Optimization. Natural Computing Series. Springer-Verlag, Berlin, 2005.
R. T. Rockafellar. Convex Analysis. Princeton Landmarks in Mathematics. Princeton University Press, Princeton, NJ, 1997. Reprint of the 1970 original.
R. Storn and K. Price. Differential evolution — a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim., 11(4):341–359, 1997. doi: 10.1023/A:1008202821328.
E. Zeidler. Applied Functional Analysis: Main Principles and Their Applications, volume 109 of Applied Mathematical Sciences. Springer-Verlag, New York, 1995.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Sullivan, T.J. (2015). Optimization Theory. In: Introduction to Uncertainty Quantification. Texts in Applied Mathematics, vol 63. Springer, Cham. https://doi.org/10.1007/978-3-319-23395-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-23395-6_4
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23394-9
Online ISBN: 978-3-319-23395-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)