Abstract
We develop criteria for the existence and uniqueness of the global minima of a continuous bounded function on a noncompact set. Special attention is given to the problem of parameter estimation via minimization of the sum of squares in nonlinear regression and maximum likelihood. Definitions of local convexity and unimodality are given using the level set. A fundamental theorem of nonconvex optimization is formulated: If a function approaches the minimal limiting value at the boundary of the optimization domain from below and its Hessian matrix is positive definite at the point where the gradient vanishes, then the function has a unique minimum. It is shown that the local convexity level of the sum of squares is equal to the minimal squared radius of the regression curvature. A new multimodal function is introduced, the decomposition function, which can be represented as the composition of a convex function and a nonlinear function from the argument space to a space of larger dimension. Several general global criteria based on majorization and minorization functions are formulated.
Similar content being viewed by others
References
Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)
Giorgi, G., Guerraggio, A., Thierfelder, J.: Mathematics of Optimization: Smooth and Nonsmooth Case. Elsevier, Amsterdam (2004)
Rapcsák, T.: Smooth Nonlinear Optimization in R n. Kluwer, Dordrecht (1997)
Floudas, C.A.: Deterministic Global Optimization. Kluwer, Boston (2000)
Horst, R.: Deterministic methods in constrained global optimization: some recent advances and new fields of application. Nav. Res. Logist. 17, 433–471 (1990)
Horst , R., Tuy, H.: Global Optimization: Deterministic Approaches, 2nd edn. Springer, Berlin (1993)
Esposito, W.R., Floudas, C.A.: Global optimization in parameter estimation of nonlinear algebraic models via error-in-variables approach. Ind. Eng. Chem. Res. 25, 1841–1858 (1998)
Gau, C.Y., Schrage, L.E.: Implementation and testing of a branch-and-bound based method for deterministic global optimization: operations research applications. In: Floudas, C.A., Pardalos, P. (eds.) Frontiers in Global Optimization, pp. 145–164. Kluwer, Boston (2004)
Floudas, C.A., Akrotirianakis, I.G., Caratzoulas, S., Meyer, C.A., Kallrath, J.: Global optimization in the 21st century: advances and challenges. Comput. Chem. Eng. 29, 1185–1202 (2005)
Demidenko, E.: Is this the least squares estimate? Biometrika 87, 437–452 (2000)
Demidenko, E.: Optimization and Regression. Nauka, Moscow (1989) (in Russian)
Demidenko, E.: On the existence of the least squares estimate in nonlinear growth curve models of exponential type. Commun. Stat. Theory Methods 25, 159–182 (1996)
Mäkeläinen, T., Schmidt, K., Styan, G.P.H.: On the existence and uniqueness of the maximum likelihood estimate of a vector-valued parameter in fixed-size samples. Ann. Stat. 9, 758–767 (1981)
Bates, D.M., Watts, D.G.: Nonlinear Regression and Its Applications. Wiley, New York (1988)
Seber, G.A.F., Wild, C.J.: Nonlinear Regression. Wiley, New York (1989)
Cattle, R.W., Pang, J.S., Stone, R.E.: The Linear Complementarity Problem. Academic Press, New York (1992)
Kamthan, P.G., Gupta, M.: Theory of Basis and Cones. Pitman Advanced Publications, Boston (1985)
Demidenko, E.: Criteria for global minimum of sum of squares in nonlinear regression. Comput. Stat. Data Anal. 53, 1739–1753 (2006)
Jukic, D., Kralik, G., Scitovski, R.: Least-squares fitting Gompertz curve. J. Comput. Appl. Math. 169, 359–375 (2004)
Jukic, D., Scitovski, R.: Least squares fitting Gaussian type curve. Appl. Math. Comput. 167, 286–298 (2005)
Ambrosetti, A., Rabinowitz, P.H.: Dual variational methods in critical point theory and applications. J. Funct. Anal. 14, 349–381 (1973)
Mawhin, J., Willem, M.: Critical Point Theory and Hamiltonian Systems. Springer, New York (1989)
Milnor, J.: Morse Theory. Princeton University Press, Princeton (1963)
Eells, J.: Singularities of Smooth Maps. Gordon and Breach, London (1967)
Orme, C.D., Ruud, P.A.: On the uniqueness of the maximum likelihood estimator. Econ. Lett. 75, 209–217 (2002)
Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970)
Spivak, M.: A Comprehensive Introduction to Differential Geometry, vol. 2. Publish or Perish, Berkeley (1979)
Demidenko, E.: Mixed Models: Theory and Applications. Wiley, New York (2004)
Nemirovsky, A.S., Yudin, D.B.: Problem Complexity and Method Efficiency in Optimization. Wiley, New York (1983)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by B. Polyak.
Rights and permissions
About this article
Cite this article
Demidenko, E. Criteria for Unconstrained Global Optimization. J Optim Theory Appl 136, 375–395 (2008). https://doi.org/10.1007/s10957-007-9298-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-007-9298-6