Verified Global Optimization for Estimating the Parameters of Nonlinear Models

  • Michel Kieffer
  • Mihály Csaba Markót
  • Hermann Schichl
  • Eric Walter
Chapter
First Online:
Part of the Mathematical Engineering book series (MATHENGIN, volume 3)

Abstract

Nonlinear parameter estimation is usually achieved via the minimization of some possibly non-convex cost function. Interval analysis allows one to derive algorithms for the guaranteed characterization of the set of all global minimizers of such a cost function when an explicit expression for the output of the model is available or when this output is obtained via the numerical solution of a set of ordinary differential equations. However, cost functions involved in parameter estimation are usually challenging for interval techniques, if only because of multi-occurrences of the parameters in the formal expression of the cost. This paper addresses parameter estimation via the verified global optimization of quadratic cost functions. It introduces tools for the minimization of generic cost functions. When an explicit expression of the output of the parametric model is available, significant improvements may be obtained by a new box exclusion test and by careful manipulations of the quadratic cost function. When the model is described by ODEs, some of the techniques available in the previous case may still be employed, provided that sensitivity functions of the model output with respect to the parameters are available.

This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    The COCONUT environment. URL www.mat.univie.ac.at/coconut-environment
  2. 2.
    Byrd, R., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16, 1190–1208 (1995)MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Domes, F., Neumaier, A.: Rigorous enclosures of ellipsoids and directed cholesky factorizations (2009). URL http://www.mat.univie.ac.at/˜dferi/publ/Cholesky.pdf. Manuscript
  4. 4.
    Domes, F., Neumaier, A.: Constraint propagation on quadratic constraints. Constraints 10, 404–429 (2010)MathSciNetGoogle Scholar
  5. 5.
    Hansen, E.R.: Global Optimization Using Interval Analysis. Marcel Dekker, New York, NY (1992)MATHGoogle Scholar
  6. 6.
    Hoefkens, J., Berz, M., Makino, K.: Efficient high-order methods for ODEs and DAEs. In: G. Corliss, C. Faure, A. Griewank (eds.) Automatic Differentiation : From Simulation to Optimization, pp. 341–351. Springer-Verlag, New-York, NY (2001)Google Scholar
  7. 7.
    Jacquez, J.A.: Compartmental Analysis in Biology andMedicine. Elsevier, Amsterdam (1972) 8. Kahan, W.: A more complete interval arithmetic. Lecture notes for a summer course, University of Toronto, Canada (1968)Google Scholar
  8. 8.
    Kay, S.M.: Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory. Prentice Hall (1993)MATHGoogle Scholar
  9. 9.
    Kearfott, R.B.: Intlib: a portable fortran 77 elementary function library. ACM Transactions on Mathematical Software 20(4), 447–459 (1994)MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Kearfott, R.B.: Rigorous Global Search: Continuous Problems (Nonconvex Optimization and Its Applications. Springer-Verlag (2010)Google Scholar
  11. 11.
    Land, A.H., Doig, A.G.: An automated method for solving discrete programming problems. Econometrica 28, 497–520 (1960)MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Lin, Y., Stadtherr, M.A.: Validated solution of odes with parametric uncertainties. In: W. Marquardt, C. Pantelides (eds.) Proc. 16th European Symposium on Computer Aided Process Engineering and 9th International Symposium on Process Systems Engineering, Computer Aided Chemical Engineering, vol. 21, pp. 167 – 172. Elsevier (2006). DOI 10. 1016/S1570-7946(06)80041-6. URL http://www.sciencedirect.com/science/article/B8G5G-4P37F2K-S/2/a22680956d2786784b23e88e9b272db6
  13. 13.
    Little, J.D., Murty, K.C., Sweeney, D.W., Karel, C.: An algorithm for the travelling salesman problem. Operations Research 11, 972–989 (1963)MATHCrossRefGoogle Scholar
  14. 14.
    Lohner, R.: Computation of guaranteed enclosures for the solutions of ordinary initial and boundary value-problem. In: J.R. Cash, I. Gladwell (eds.) Computational Ordinary Differential Equations, pp. 425–435. Clarendon Press, Oxford (1992)Google Scholar
  15. 15.
    Makino, K., Berz, M.: Taylor models and other validated functional inclusion methods. International Journal 4(4), 379–456 (2003)MathSciNetMATHGoogle Scholar
  16. 16.
    Mark´ot, M.C., Schichl, H.: Bound constrained optimization in the COCONUT environment (2010). ManuscriptGoogle Scholar
  17. 17.
    Moore, R.E.: Interval arithmetic and automatic error analysis in digital computing. Phd thesis, Stanford University, Stanford, CA (1962). URL http://interval.louisiana.edu/Mooresearlypapers/disert.pdf
  18. 18.
    Mu¨ller, M.: U¨ ber das Fundamentaltheorem in der Theorie der gewo¨hnlichen Differentialgleichungen. Math. Z. 26, 619–645 (1926)Google Scholar
  19. 19.
    Nedialkov, N.S., Jackson, K.R.: Methods for initial value problems for ordinary differential equations. In: U. Kulisch, R. Lohner, A. Facius (eds.) Perspectives on Enclosure Methods, pp. 219–264. Springer-Verlag, Vienna (2001)Google Scholar
  20. 20.
    Neumaier, A.: Interval Methods for Systems of Equations. Cambridge University Press, Cambridge, UK (1990)MATHGoogle Scholar
  21. 21.
    Neumaier, A.: Complete search in continuous global optimization and constraint satisfaction. In: A. Iserles (ed.) Acta Numerica, pp. 271–369. Cambridge University Press (2004)Google Scholar
  22. 22.
    Ratschek, H., Rokne, J.: New Computer Methods for Global Optimization. Ellis Horwood, Chichester, UK (1988)MATHGoogle Scholar
  23. 23.
    Ratz, D., Csendes, T.: On the selection of subdivision directions in interval branch-and-bound methods for global optimization. Journal of Global Optimization 7(2), 183–207 (1995)MathSciNetMATHCrossRefGoogle Scholar
  24. 24.
    Rauh, A., Auer, E., Minisini, J., Hofer, E.P.: Extensions of ValEncIA-IVP for reduction of overestimation, for simulation of differential algebraic systems, and for dynamical optimization. Proc. Appl. Math. Mech. 7(1), 1023,0011023,002 (2007). DOI  10.1002/pamm.200700022
  25. 25.
    Rauh, A., Hofer, E.P., Auer, E.: VALENCIA-IVP: A comparison with other initial value problem solvers. In: Proc. 12th GAMM - IMACS International Symposium on Scientific Computing, Computer Arithmetic and Validated Numerics (SCAN 2006), p. 36. IEEE Computer Society, Duisburg, Germany (2006). DOI  10.1109/SCAN.2006.47
  26. 26.
    Schichl, H.: Global optimization in the COCONUT project. In: Proceedings of the Dagstuhl Seminar “Numerical Software with Result Verification”, Lecture Notes in Comput. Sci., vol. 2991, pp. 277–293 (2004)Google Scholar
  27. 27.
    Schichl, H., Mark´ot, M.C.: Algorithmic differentiation in the COCONUT environment (2010). URL http://www.mat.univie.ac.at/˜herman/papers/griewank.pdf. Manuscript
  28. 28.
    Schichl, H.,Mark´ot,M.C.: Interval analysis on directed acyclic graphs for global optimization. higher order methods (2010). URL http://www.mat.univie.ac.at/˜herman/papers/dag2.pdf. Manuscript
  29. 29.
    Schichl, H., Mark´ot, M.C.: Optimal enclosures of derivatives and slopes for univariate functions (2010). URL http://www.mat.univie.ac.at/˜herman/papers/univar.pdf. Manuscript
  30. 30.
    Schichl, H., Mark´ot, M.C., Neumaier, A.: Exclusion regions for optimization problems (2010). URL http://www.mat.univie.ac.at/˜herman/papers/exclopt.pdf. Manuscript
  31. 31.
    Schichl, H., Neumaier, A.: Exclusion regions for systems of equations. SIAM J. Numer. Anal. 42, 383–408 (2004)MathSciNetMATHCrossRefGoogle Scholar
  32. 32.
    Schichl, H., Neumaier, A.: Interval analysis on directed acyclic graphs for global optimization J. Global Optim. 33, 541–562 (2006)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Shen, Z., Neumaier, A.: The Krawczyk operator and Kantorovich’s theorem. J. Math. Anal. Appl. 149, 437–443 (1990)MathSciNetMATHCrossRefGoogle Scholar
  34. 34.
    Skelboe, S.: Computation of rational interval functions. BIT 14, 87–95 (1974)MathSciNetMATHGoogle Scholar
  35. 35.
    Vu, X.H., Schichl, H., Sam-Haroud, D.: Using directed acyclic graphs to coordinate propagation and search for numerical constraint satisfaction problems. In: Proc. 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2004). Florida (2004)Google Scholar
  36. 36.
    Walter, E., Kieffer, M.: Guaranteed optimisation of the parameters of continuous-time knowledge-based models. In: C. Commault, N. Marchand (eds.) Positive Systems, LNCIS, vol. 341, pp. 137–144. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  37. 37.
    Walter, E., Pronzato, L.: Identification of Parametric Models from Experimental Data. Springer-Verlag, London (1997)MATHGoogle Scholar
  38. 38.
    Wolfe, M.A.: Interval methods for global optimization. Applied Mathematics and Computation 75, 179–206 (1996)MathSciNetMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Michel Kieffer
    • 1
    • 2
  • Mihály Csaba Markót
    • 3
  • Hermann Schichl
    • 3
  • Eric Walter
    • 1
  1. 1.Laboratoire des Signaux et Systèmes - CNRS - SUPELECUniv Paris-SudGif-sur-Yvette cedexFrance
  2. 2.LTCI - CNRS - Telecom ParisTechParisFrance
  3. 3.Fakultät für MathematikUniversität WienWienAustria

Personalised recommendations