Verified Global Optimization for Estimating the Parameters of Nonlinear Models

  • Michel KiefferEmail author
  • Mihály Csaba Markót
  • Hermann Schichl
  • Eric Walter
Part of the Mathematical Engineering book series (MATHENGIN, volume 3)


Nonlinear parameter estimation is usually achieved via the minimization of some possibly non-convex cost function. Interval analysis allows one to derive algorithms for the guaranteed characterization of the set of all global minimizers of such a cost function when an explicit expression for the output of the model is available or when this output is obtained via the numerical solution of a set of ordinary differential equations. However, cost functions involved in parameter estimation are usually challenging for interval techniques, if only because of multi-occurrences of the parameters in the formal expression of the cost. This paper addresses parameter estimation via the verified global optimization of quadratic cost functions. It introduces tools for the minimization of generic cost functions. When an explicit expression of the output of the parametric model is available, significant improvements may be obtained by a new box exclusion test and by careful manipulations of the quadratic cost function. When the model is described by ODEs, some of the techniques available in the previous case may still be employed, provided that sensitivity functions of the model output with respect to the parameters are available.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    The COCONUT environment. URL
  2. 2.
    Byrd, R., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16, 1190–1208 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Domes, F., Neumaier, A.: Rigorous enclosures of ellipsoids and directed cholesky factorizations (2009). URL˜dferi/publ/Cholesky.pdf. Manuscript
  4. 4.
    Domes, F., Neumaier, A.: Constraint propagation on quadratic constraints. Constraints 10, 404–429 (2010)MathSciNetGoogle Scholar
  5. 5.
    Hansen, E.R.: Global Optimization Using Interval Analysis. Marcel Dekker, New York, NY (1992)zbMATHGoogle Scholar
  6. 6.
    Hoefkens, J., Berz, M., Makino, K.: Efficient high-order methods for ODEs and DAEs. In: G. Corliss, C. Faure, A. Griewank (eds.) Automatic Differentiation : From Simulation to Optimization, pp. 341–351. Springer-Verlag, New-York, NY (2001)Google Scholar
  7. 7.
    Jacquez, J.A.: Compartmental Analysis in Biology andMedicine. Elsevier, Amsterdam (1972) 8. Kahan, W.: A more complete interval arithmetic. Lecture notes for a summer course, University of Toronto, Canada (1968)Google Scholar
  8. 8.
    Kay, S.M.: Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory. Prentice Hall (1993)zbMATHGoogle Scholar
  9. 9.
    Kearfott, R.B.: Intlib: a portable fortran 77 elementary function library. ACM Transactions on Mathematical Software 20(4), 447–459 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Kearfott, R.B.: Rigorous Global Search: Continuous Problems (Nonconvex Optimization and Its Applications. Springer-Verlag (2010)Google Scholar
  11. 11.
    Land, A.H., Doig, A.G.: An automated method for solving discrete programming problems. Econometrica 28, 497–520 (1960)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Lin, Y., Stadtherr, M.A.: Validated solution of odes with parametric uncertainties. In: W. Marquardt, C. Pantelides (eds.) Proc. 16th European Symposium on Computer Aided Process Engineering and 9th International Symposium on Process Systems Engineering, Computer Aided Chemical Engineering, vol. 21, pp. 167 – 172. Elsevier (2006). DOI 10. 1016/S1570-7946(06)80041-6. URL
  13. 13.
    Little, J.D., Murty, K.C., Sweeney, D.W., Karel, C.: An algorithm for the travelling salesman problem. Operations Research 11, 972–989 (1963)zbMATHCrossRefGoogle Scholar
  14. 14.
    Lohner, R.: Computation of guaranteed enclosures for the solutions of ordinary initial and boundary value-problem. In: J.R. Cash, I. Gladwell (eds.) Computational Ordinary Differential Equations, pp. 425–435. Clarendon Press, Oxford (1992)Google Scholar
  15. 15.
    Makino, K., Berz, M.: Taylor models and other validated functional inclusion methods. International Journal 4(4), 379–456 (2003)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Mark´ot, M.C., Schichl, H.: Bound constrained optimization in the COCONUT environment (2010). ManuscriptGoogle Scholar
  17. 17.
    Moore, R.E.: Interval arithmetic and automatic error analysis in digital computing. Phd thesis, Stanford University, Stanford, CA (1962). URL
  18. 18.
    Mu¨ller, M.: U¨ ber das Fundamentaltheorem in der Theorie der gewo¨hnlichen Differentialgleichungen. Math. Z. 26, 619–645 (1926)Google Scholar
  19. 19.
    Nedialkov, N.S., Jackson, K.R.: Methods for initial value problems for ordinary differential equations. In: U. Kulisch, R. Lohner, A. Facius (eds.) Perspectives on Enclosure Methods, pp. 219–264. Springer-Verlag, Vienna (2001)Google Scholar
  20. 20.
    Neumaier, A.: Interval Methods for Systems of Equations. Cambridge University Press, Cambridge, UK (1990)zbMATHGoogle Scholar
  21. 21.
    Neumaier, A.: Complete search in continuous global optimization and constraint satisfaction. In: A. Iserles (ed.) Acta Numerica, pp. 271–369. Cambridge University Press (2004)Google Scholar
  22. 22.
    Ratschek, H., Rokne, J.: New Computer Methods for Global Optimization. Ellis Horwood, Chichester, UK (1988)zbMATHGoogle Scholar
  23. 23.
    Ratz, D., Csendes, T.: On the selection of subdivision directions in interval branch-and-bound methods for global optimization. Journal of Global Optimization 7(2), 183–207 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Rauh, A., Auer, E., Minisini, J., Hofer, E.P.: Extensions of ValEncIA-IVP for reduction of overestimation, for simulation of differential algebraic systems, and for dynamical optimization. Proc. Appl. Math. Mech. 7(1), 1023,0011023,002 (2007). DOI  10.1002/pamm.200700022
  25. 25.
    Rauh, A., Hofer, E.P., Auer, E.: VALENCIA-IVP: A comparison with other initial value problem solvers. In: Proc. 12th GAMM - IMACS International Symposium on Scientific Computing, Computer Arithmetic and Validated Numerics (SCAN 2006), p. 36. IEEE Computer Society, Duisburg, Germany (2006). DOI  10.1109/SCAN.2006.47
  26. 26.
    Schichl, H.: Global optimization in the COCONUT project. In: Proceedings of the Dagstuhl Seminar “Numerical Software with Result Verification”, Lecture Notes in Comput. Sci., vol. 2991, pp. 277–293 (2004)Google Scholar
  27. 27.
    Schichl, H., Mark´ot, M.C.: Algorithmic differentiation in the COCONUT environment (2010). URL˜herman/papers/griewank.pdf. Manuscript
  28. 28.
    Schichl, H.,Mark´ot,M.C.: Interval analysis on directed acyclic graphs for global optimization. higher order methods (2010). URL˜herman/papers/dag2.pdf. Manuscript
  29. 29.
    Schichl, H., Mark´ot, M.C.: Optimal enclosures of derivatives and slopes for univariate functions (2010). URL˜herman/papers/univar.pdf. Manuscript
  30. 30.
    Schichl, H., Mark´ot, M.C., Neumaier, A.: Exclusion regions for optimization problems (2010). URL˜herman/papers/exclopt.pdf. Manuscript
  31. 31.
    Schichl, H., Neumaier, A.: Exclusion regions for systems of equations. SIAM J. Numer. Anal. 42, 383–408 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Schichl, H., Neumaier, A.: Interval analysis on directed acyclic graphs for global optimization J. Global Optim. 33, 541–562 (2006)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Shen, Z., Neumaier, A.: The Krawczyk operator and Kantorovich’s theorem. J. Math. Anal. Appl. 149, 437–443 (1990)MathSciNetzbMATHCrossRefGoogle Scholar
  34. 34.
    Skelboe, S.: Computation of rational interval functions. BIT 14, 87–95 (1974)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Vu, X.H., Schichl, H., Sam-Haroud, D.: Using directed acyclic graphs to coordinate propagation and search for numerical constraint satisfaction problems. In: Proc. 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2004). Florida (2004)Google Scholar
  36. 36.
    Walter, E., Kieffer, M.: Guaranteed optimisation of the parameters of continuous-time knowledge-based models. In: C. Commault, N. Marchand (eds.) Positive Systems, LNCIS, vol. 341, pp. 137–144. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  37. 37.
    Walter, E., Pronzato, L.: Identification of Parametric Models from Experimental Data. Springer-Verlag, London (1997)zbMATHGoogle Scholar
  38. 38.
    Wolfe, M.A.: Interval methods for global optimization. Applied Mathematics and Computation 75, 179–206 (1996)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Michel Kieffer
    • 1
    • 2
    Email author
  • Mihály Csaba Markót
    • 3
  • Hermann Schichl
    • 3
  • Eric Walter
    • 1
  1. 1.Laboratoire des Signaux et Systèmes - CNRS - SUPELECUniv Paris-SudGif-sur-Yvette cedexFrance
  2. 2.LTCI - CNRS - Telecom ParisTechParisFrance
  3. 3.Fakultät für MathematikUniversität WienWienAustria

Personalised recommendations