Skip to main content
SpringerLink
Log in

Gauss–Newton Methods for Robust Parameter Estimation

  • Chapter
  • First Online:
Model Based Parameter Estimation

Part of the book series: Contributions in Mathematical and Computational Sciences ((CMCS,volume 4))

Abstract

In this paper we treat robust parameter estimation procedures for problems constrained by differential equations. Our focus is on the l 1 norm estimator and Huber’s M-estimator. Both of the estimators are briefly characterized and the corresponding optimality conditions are given. We describe the solution of the resulting minimization problems using the Gauss–Newton method and present local convergence results for both nonlinear constrained l 1 norm and Huber optimization. An approach for the efficient solution of the linearized problems of the Gauss–Newton iterations is also sketched as well as globalization strategies using line search methods. Two numerical examples are exercised to demonstrate the superiority of the two presented robust estimators over standard least squares estimation in case of outliers in the measurement data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. D.A. Andrews, F. Bickel, F. Hampel, P. Huber, W. Rogers, and J. Tukey, Robust Estimates of Location: Survey and Advanves, Princeton University Press, 1972.

    Google Scholar 

  2. U. Ascher, Collocation for two-point boundary value problems revisited, SIAM Journal on Numerical Analysis, 23(3), pp. 596–609, 1986.

    Article  MathSciNet  MATH  Google Scholar 

  3. I. Barrodale and F.D.K. Roberts, An improved algorithm for discrete l 1 linear approximation, SIAM Journal on Numerical Analysis 10, pp. 839–848, 1973.

    Article  MathSciNet  MATH  Google Scholar 

  4. I. Barrodale and F.D.K. Roberts, Algorithm 478: Solution of an overdetermined system of linear equations in the l 1 norm, Communications of the ACM 17, pp. 319–320, 1974.

    Article  Google Scholar 

  5. I. Bauer, H. G. Bock, S. Körkel, and J. P. Schlöder, Numerical methods for initial value problems and derivative generation for DAE models with application to optimum experimental design of chemical processes, in F. Keil, W. Mackens, H. Voss, and J. Werther, eds., Scientific Computing in Chemical Engineering II, 2, pp. 282–289, Springer, Berlin-Heidelberg, 1999.

    Google Scholar 

  6. I. Bauer, H. G. Bock, S. Körkel, and J. P. Schlöder, Numerical methods for optimum experimental design in DAE systems, Journal of Computational and Applied Mathematics 120, pp. 1–25, 2000.

    Article  MathSciNet  MATH  Google Scholar 

  7. A.E. Beaton and J.W. Tukey, The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data, Technometrics 16, pp. 147–185, 1974.

    Article  MATH  Google Scholar 

  8. A. Ben-Tal, Second-Order and Related Extremality Conditions in Nonlinear Programming, Journal of Optimization Theory and Applications 31(2), pp. 143–165, 1980.

    Article  MathSciNet  MATH  Google Scholar 

  9. H.G. Bock, Numerical treatment of inverse problems in chemical reaction kinetics, in K.-H. Ebert, P. Deuflhard, and W. Jäger, eds., Modelling of Chemical Reaction Systems, volume 18 of Springer Series in Chemical Physics, Springer Verlag, pp. 102–125, 1981.

    Google Scholar 

  10. H.G. Bock, Randwertproblemmethoden zur Parameteridentifizierung in Systemen nichtlinearer Differentialgleichungen, Bonner Mathematische Schriften, Nr. 183, Bonn, 1987.

    Google Scholar 

  11. H.G. Bock, E.A. Kostina, and J.P. Schlöder, Numerical Methods for Parameter Estimation in Nonlinear Differential Algebraic Equations, GAMM Mitteilungen 30(2), pp. 376–408, 2007.

    Article  MathSciNet  MATH  Google Scholar 

  12. H.G. Bock, E. Kostina, and J.P. Schlöder, On the Role of Natural Level Functions to Achieve Global Convergence for Damped Newton Methods, in M.J.D. Powell and S. Scholtes, eds., System Modelling and Optimization. Methods, Theory and Applications, Kluwer, Boston, pp. 51–74, 2000.

    Google Scholar 

  13. A.M. Bruckstein, D.L. Donoho, and M. Elad, From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images, SIAM Review 51(1), pp. 34–81, 2009.

    Article  MathSciNet  MATH  Google Scholar 

  14. R. Bulirsch, Die Mehrzielmethode zur numerischen Lösung von nichtlinearen Randwertproblemen und Aufgaben der optimalen Steuerung, Technical report, Carl-Cranz-Gesellschaft, 1971.

    Google Scholar 

  15. A. Cervantes and L.T. Biegler, Large-scale DAE optimization using a simultaneous NLP formulation, AIChE Journal 44(5), pp. 1038–1050, 2004.

    Article  Google Scholar 

  16. W. Chauvenet, A Manual of Spherical and Practical Astronomy Vol. II - Theory and Use of Astronomical Instruments, J.B. Lippincott & Co., 1868.

    Google Scholar 

  17. J.E. Dennis and R.E. Welsch, Techniques for nonlinear least squares and robust regression, Communications in Statistics - Simulation and Computation 7, pp. 345–359, 1978.

    Article  Google Scholar 

  18. P. Deuflhard, A Modified Newton Method for the Solution of Ill-Conditioned Systems of Nonlinear Equations with Application to Multiple Shooting, Numerical Mathematics 22, pp. 289–315, 1974.

    Article  MathSciNet  MATH  Google Scholar 

  19. R.C. Fair, On the robust estimation of econometric models, Annals of Economic and Social Measurement 3, pp. 667–677, 1974.

    Google Scholar 

  20. A.V. Fiacco and G.P. McCormick, Nonlinear Programming, Sequential Unconstrained Minimization Techniques, SIAM, 1990.

    MATH  Google Scholar 

  21. C.F. Gauss, Theoria Motus Corporum Coelestium in Sectionibus Conicis Solem Ambientium, F. Perthes & J.H. Besser, 1809.

    Google Scholar 

  22. J.E. Gentle, Least absolute value estimation: an introduction, Communications in Statistics - Simulation and Computation 6, pp. 313–328, 1977.

    Article  MathSciNet  Google Scholar 

  23. F.R. Hampel, Robust Estimation: A Condensed Partial Survey, Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 27, pp. 87–104, 1973.

    Article  MathSciNet  MATH  Google Scholar 

  24. M.J. Hinich and P.P. Talwar, A Simple Method for Robust Regression, Journal of the American Statistical Association 70, pp. 113–119, 1975.

    Article  MATH  Google Scholar 

  25. P.E. Holland and R.E. Welsch, Robust regression using iteratively reweighted least-squares, Communications in Statistics - Theory and Methods 6, pp. 813–827, 1977.

    Article  Google Scholar 

  26. P.J. Huber, Robust Estimation of a Location Parameter, The Annals of Mathematical Statistics 35, pp. 73–101, 1964.

    Article  MATH  Google Scholar 

  27. P.J. Huber and E.M. Ronchetti, Robust Statistics - Second Edition, John Wiley & Sons, 2009.

    Google Scholar 

  28. A. Koberstein, The Dual Simplex method, Techniques for a Fast and Stable Implementation, PhD Thesis, U Paderborn, 2005.

    Google Scholar 

  29. S. Körkel and E. A. Kostina, Numerical Methods for Nonlinear Experimental Design, in H. G. Bock, E. A. Kostina, H. X. Phu, and R. Rannacher, eds., Modeling, Simulation and Optimization of Complex Processes, Proceedings of the International Conference on High Performance Scientific Computing, 2003, Hanoi, Vietnam. Springer, 2004.

    Google Scholar 

  30. S. Körkel, Numerische Methoden für Optimale Versuchsplanungsprobleme bei nichtlinearen DAE-Modellen, PhD thesis, U Heidelberg, 2002.

    Google Scholar 

  31. E. Kostina, The Long Step Rule in the Bounded-Variable Dual Simplex Method: Numerical Experiments, Mathematical Methods of Operations Research 3(55), pp. 413–429, 2002.

    Article  MathSciNet  Google Scholar 

  32. E. Kostina, Robust Parameter Estimation in Dynamic Systems, Optimization and Engineering 5(4), pp. 461–484, 2004.

    Article  MathSciNet  MATH  Google Scholar 

  33. E. Kostina and O. I. Kostyukova, A Primal-Dual Active-Set Method for Convex Quadratic Programming, Technical Report, IWR, U Heidelberg, 2003.

    Google Scholar 

  34. S. Krishna, M.H. Jensen, and K. Sneppen, Minimal model of spiky oscillations in NF-κB signaling, Proceedings of the National Academy of Sciences of the United States of America 103(29), pp. 10840–10845, 2006.

    Article  Google Scholar 

  35. P.S. Laplace, Théorie Analytique des Probabilités, V. Courcier, 1814.

    Google Scholar 

  36. A.M. Legendre, Nouvelles méthodes pour la détermination des orbites des comètes, Firmin Didot, 1805.

    Google Scholar 

  37. O.L. Mangasarian and D.R. Musicant, Robust Linear and Vector Support Regression, IEEE Transactions on Pattern Analysis and Machine Intelligence 22(9), pp. 1–6, 2000.

    Article  Google Scholar 

  38. M.R. Osborne, B. Presnell, and B.A. Turlach, On the LASSO and Its Dual, Journal of Computational and Graphical Statistics 9(2), pp. 319–337, 2000.

    MathSciNet  Google Scholar 

  39. B. Peirce, Criterion for the rejection of doubtful observations, The Astronomical Journal 45, pp. 161–163, 1852.

    Article  Google Scholar 

  40. J.R. Rice and J.S. White, Norms for Smoothing and Estimation, SIAM Review 6, pp. 243–256, 1964.

    Article  MathSciNet  MATH  Google Scholar 

  41. J.A. Scales and A. Gersztenkorn, Robust methods in inverse theory, Inverse Problems 4, pp. 1071–1091, 1988.

    Article  MathSciNet  MATH  Google Scholar 

  42. V.H. Schulz, Ein effizientes Kollokationsverfahren zur numerischen Behandlung von Mehrpunktrandwertaufgaben in der Parameteridentifizierung und Optimalen Steuerung, Diploma thesis, U Augsburg, 1990.

    Google Scholar 

  43. S. Waldherr, Uncertainty and robustness analysis of biochemical reaction networks via convex optimisation and robust control theory, PhD thesis, University of Stuttgart, 2009.

    Google Scholar 

Download references

Acknowledgements

This research was supported by the German Federal Ministry for Education and Research (BMBF) through the Programme “Mathematics for Innovations in Industry and Public Services”, as well as by the LOEWE Center for Synthetic Microbiology (SYNMIKRO).

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, University of Marburg, Hans-Meerwein-Strasse, 35032, Marburg, Germany

    Tanja Binder & Ekaterina Kostina

Authors
  1. Tanja Binder
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ekaterina Kostina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ekaterina Kostina .

Editor information

Editors and Affiliations

  1. Interdisciplinary Center for Scientific, Computing (IWR), Heidelberg University, Im Neuenheimer Feld 368, Heidelberg, 69120, Germany

    Hans Georg Bock

  2. Institute for Applied Mathematics, Heidelberg University, Im Neuenheimer Feld 294, Heidelberg, 69120, Germany

    Thomas Carraro

  3. Interdisciplinary Center for Scientific, Computing (IWR), Heidelberg University, Im Neuenheimer Feld 368, Heidelberg, 69120, Germany

    Willi Jäger

  4. Interdisciplinary Center for, Scientific Computing (IWR), Heidelberg University, Im Neuenheimer Feld 368, Heidelberg, 69120, Germany

    Stefan Körkel

  5. Institute for Applied Mathematics, Heidelberg University, Im Neuenheimer Feld 293, Heidelberg, 69120, Germany

    Rolf Rannacher

  6. Interdisciplinary Center for Scientific, Computing (IWR), Heidelberg University, Im Neuenheimer Feld 368, Heidelberg, 69120, Germany

    Johannes P. Schlöder

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Binder, T., Kostina, E. (2013). Gauss–Newton Methods for Robust Parameter Estimation. In: Bock, H., Carraro, T., Jäger, W., Körkel, S., Rannacher, R., Schlöder, J. (eds) Model Based Parameter Estimation. Contributions in Mathematical and Computational Sciences, vol 4. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30367-8_3

Download citation

Publish with us

Policies and ethics

Search

Navigation