Least Squares Parameter Estimation for Dynamic Processes

  • Rolf Isermann
  • Marco Münchhof


The application of the method of least squares to static models has been described in the previous chapter and is well known to scientists for a long time already. The application of the method of least squares to the identification of dynamic processes has been tackled with much later in time. First works on the parameter estimation of AR models have been reported in the analysis of time series of economic data (Koopmans, 1937; Mann and Wald, 1943) and for the difference equations of linear dynamic processes (Kalman, 1958; Durbin, 1960; Levin, 1960; Lee, 1964).


Parameter Estimation Correlation Function Dynamic Process Power Spectral Density Equation Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Albert R, Sittler RW (1965) A method for computing least squares estimators that keep up with the data. SIAM J Control Optim 3(3):384–417zbMATHMathSciNetGoogle Scholar
  2. Åström KJ, Bohlin T (1965) Numerical identification of linear dynamic systems from normal operating records. In: Proceedings of the IFAC Symposium Theory of Self-Adaptive Control Systems, TeddingtonGoogle Scholar
  3. Åström KJ, Eykhoff P (1971) System identification – a survey. Automatica 7(2):123–162zbMATHCrossRefGoogle Scholar
  4. Baur U (1976) On-Line Parameterschätzverfahren zur Identifikation linearer, dynamischer Prozesse mit Prozeßrechnern: Entwicklung, Vergleich, Erprobung: KfK-PDV-Bericht Nr. 65. Kernforschungszentrum Karlsruhe, KarlsruheGoogle Scholar
  5. Becker HP (1990) Beiträge zur rekursiven Parameterschätzung zeitvarianter Prozesse. Fortschr.-Ber. VDI Reihe 8 Nr. 203. VDI Verlag, DüsseldorfGoogle Scholar
  6. Bellmann R, Åström KJ (1970) On structural identifiability. Math Biosci 7(3–4):329–339CrossRefGoogle Scholar
  7. Bombois X, Anderson BDO, Gevers M (2005) Quantification of frequency domain error bounds with guaranteed confidence level in prediction error identification. Syst Control Lett 54(11):471–482zbMATHCrossRefMathSciNetGoogle Scholar
  8. Box GEP, Jenkins GM, Reinsel GC (2008) Time series analysis: Forecasting and control, 4th edn. Wiley Series in Probability and Statistics, John Wiley, Hoboken, NJzbMATHGoogle Scholar
  9. Burg JP (1968) A new analysis technique for time series data. In: Proceedings of NATO Advanced Study Institute on Signal Processing, EnschedeGoogle Scholar
  10. Campi MC, Weyer E (2002) Finite sample properties of system identification methods. IEEE Trans Autom Control 47(8):1329–1334CrossRefMathSciNetGoogle Scholar
  11. Deutsch R (1965) Estimation theory. Prentice-Hall, Englewood Cliffs, NJzbMATHGoogle Scholar
  12. van Doren JFM, Douma SG, van den Hof PMJ, Jansen JD, Bosgra OH (2009) Identifiability: From qualitative analysis to model structure approximation. In: Proceedings of the 15th IFAC Symposium on System Identification, Saint-Malo, FranceGoogle Scholar
  13. Durbin J (1960) Estimation of parameters in time-series regression models. J Roy Statistical Society B 22(1):139–153zbMATHMathSciNetGoogle Scholar
  14. Edward J, Fitelson M (1973) Notes on maximum-entropy processing (Corresp.). IEEE Trans Inf Theory 19(2):232–234zbMATHCrossRefGoogle Scholar
  15. Eykhoff P (1974) System identification: Parameter and state estimation. Wiley-Interscience, LondonGoogle Scholar
  16. Gauss KF (1809) Theory of the motion of the heavenly bodies moving about the sun in conic sections: Reprint 2004. Dover phoenix editions, Dover, Mineola, NYGoogle Scholar
  17. Genin Y (1968) A note on linear minimum variance estimation problems. IEEE Trans Autom Control 13(1):103–103CrossRefGoogle Scholar
  18. Gevers M (2005) Identification for control: From early achievements to the revival of experimental design. Eur J Cont 2005(11):1–18MathSciNetGoogle Scholar
  19. Goodwin GC, Sin KS (1984) Adaptive filtering, prediction and control. Prentice-Hall information and system sciences series, Prentice-Hall, Englewood Cliffs, NJzbMATHGoogle Scholar
  20. Heij C, Ran A, Schagen F (2007) Introduction to mathematical systems theory : linear systems, identification and control. Birkhäuser Verlag, BaselzbMATHGoogle Scholar
  21. Hoerl AE, Kennard RW (1970a) Ridge regression: Application to nonorthogonal problems. Technometrics 12(1):69–82zbMATHCrossRefMathSciNetGoogle Scholar
  22. Hoerl AE, Kennard RW (1970b) Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12(1):55–67zbMATHCrossRefMathSciNetGoogle Scholar
  23. Isermann R (1974) Prozessidentifikation: Identifikation und Parameterschätzung dynamischer Prozesse mit diskreten Signalen. Springer, HeidelbergGoogle Scholar
  24. Isermann R (1987) Digitale Regelsysteme Band 1 und 2. Springer, HeidelbergGoogle Scholar
  25. Isermann R (1991) Digital control systems, 2nd edn. Springer, BerlinGoogle Scholar
  26. Isermann R (2005) Mechatronic Systems: Fundamentals. Springer, LondonGoogle Scholar
  27. Isermann R (2006) Fault-diagnosis systems: An introduction from fault detection to fault tolerance. Springer, BerlinGoogle Scholar
  28. Isermann R, Baur U (1974) Two-step process identification with correlation analysis and least-squares parameter estimation. J Dyn Syst Meas Contr 96:426–432zbMATHCrossRefGoogle Scholar
  29. Johnston J, DiNardo J (1997) Econometric Methods: Economics Series, 4th edn. McGraw-Hill, New York, NYGoogle Scholar
  30. Kalman RE (1958) Design of a self-optimizing control system. Trans ASME 80:468–478Google Scholar
  31. Kendall MG, Stuart A (1977a) The advanced theory of statistics: Design and analysis, and time-series (vol. 3). Charles Griffin, LondonGoogle Scholar
  32. Kendall MG, Stuart A (1977b) The advanced theory of statistics: Inference and relationship (vol. 2). Charles Griffin, LondonGoogle Scholar
  33. Klinger A (1968) Prior information and bias in sequential estimation. IEEE Trans Autom Control 13(1):102–103CrossRefGoogle Scholar
  34. Koopmans TC (1937) Linear regression analysis of economic time series. Netherlands Economic Institute, HaarlemzbMATHGoogle Scholar
  35. Lee KI (1964) Optimal estimation, identification, and control, Massachusetts Institute of Technology research monographs, vol 28. MIT Press, Cambridge, MAGoogle Scholar
  36. Levin MJ (1960) Optimum estimation of impulse response in the presence of noise. IRE Trans Circuit Theory 7(1):50–56Google Scholar
  37. Ljung L (1999) System identification: Theory for the user, 2nd edn. Prentice Hall Information and System Sciences Series, Prentice Hall PTR, Upper Saddle River, NJGoogle Scholar
  38. Makhoul J (1975) Linear prediction: A tutorial review. Proc IEEE 63(4):561–580CrossRefGoogle Scholar
  39. Makhoul J (1976) Correction to “Linear prediction : A tutorial review”. Proc IEEE 64(2):285CrossRefGoogle Scholar
  40. Mann HB, Wald W (1943) On the statistical treatment of linear stochastic difference equations. Econometrica 11(3/4):173–220zbMATHCrossRefMathSciNetGoogle Scholar
  41. Mendel JM (1973) Discrete techniques of parameter estimation: The equation error formulation, Control Theory, vol 1. Marcel Dekker, New YorkGoogle Scholar
  42. Neumann D (1991) Fault diagnosis of machine-tools by estimation of signal spectra. In: Proceedings of the IFAC/IMACS Sympsoium on Fault Detection, Supervision, and Safety for Technical Processes SAFEPROCESS’91, Baden-Baden, GermanyGoogle Scholar
  43. Neumann D, Janik W (1990) Fehlerdiagnose an spanenden Werkzeugmaschinen mit parametrischen Signalmodellen von Schwingungen. In: VDI-Schwingungstagung Mannheim, VDI-Verlag, Düsseldorf, GermanyGoogle Scholar
  44. Ninness B, Goodwin GC (1995) Estimation of model quality. Automatica 31(12):32–74CrossRefMathSciNetGoogle Scholar
  45. Pandit SM, Wu SM (1983) Time series and system analysis with applications. Wiley, New YorkzbMATHGoogle Scholar
  46. Panuska V (1969) An adaptive recursive least squares identification algorithm. In: Proceedings of the IEEE Symposium in Adaptive Processes, Decision and ControlGoogle Scholar
  47. Pintelon R, Schoukens J (2001) System identification: A frequency domain approach. IEEE Press, Piscataway, NJCrossRefGoogle Scholar
  48. Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes: The art of scientific computing, 3rd edn. Cambridge University Press, Cambridge, UKzbMATHGoogle Scholar
  49. Sagara S, Wada K, Gotanda H (1979) On asymptotic bias of linear least squares estimator. In: Proceedings of the 5th IFAC Symposium on Identification and System Parameter Estimation Darmstadt, Pergamon Press, Darmstadt, GermanyGoogle Scholar
  50. Scheuer HG (1973) Ein für den Prozessrechnereinsatz geeignetes Identifikationsverfahren auf der Grundlage von Korrelationsfunktionen. Dissertation. Universität Trier, TrierGoogle Scholar
  51. Staley RM, Yue PC (1970) On system parameter identifability. Inf Sci 2(2):127–138zbMATHCrossRefMathSciNetGoogle Scholar
  52. Stuart TA, Ord JK, Kendall MG (1987) Kendalls advanced theory of statistics: Distribution theory (vol. 1). Charles: Griffin BookGoogle Scholar
  53. Tikhonov AN (1995) Numerical methods for the solution of ill-posed problems, Mathematics and its applications, vol 328. Kluwer Academic Publishers, DordrechtGoogle Scholar
  54. Tikhonov AN, Arsenin VY (1977) Solutions of ill-posed problems. Scripta series in mathematics, Winston, Washington, D.C.Google Scholar
  55. Tong H (1975) Autoregressive model fitting with noisy data by Akaike’s information criterion. IEEE Trans Inf Theory 21(4):476–480zbMATHCrossRefGoogle Scholar
  56. Tong H (1977) More on autoregressive model fitting with noisy data by Akaike’s information criterion. IEEE Trans Inf Theory 23(3):409–410zbMATHCrossRefGoogle Scholar
  57. Tse E, Anton J (1972) On the identifiability of parameters. IEEE Trans Autom Control 17(5):637–646zbMATHCrossRefMathSciNetGoogle Scholar
  58. Ulrych TJ, Bishop TN (1975) Maximum entropy spectral analysis and autoregressive decomposition. Rev Geophys 13(1):183–200CrossRefGoogle Scholar
  59. Vuerinckx R, Pintelon R, Schoukens J, Rolain Y (2001) Obtaining accurate confidence regions for the estimated zeros and poles in system identification problems. IEEE Trans Autom Control 46(4):656–659zbMATHCrossRefMathSciNetGoogle Scholar
  60. Weyer E, Campi MC (2002) Non-asymptotic confidence ellipsoids for the leastsquares estimate. Automatica 38(9):1539–1547zbMATHCrossRefMathSciNetGoogle Scholar
  61. Young P (1984) Recursive estimation and time-series analysis: An introduction. Communications and control engineering series, Springer, BerlinGoogle Scholar

Copyright information

© Springer Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.Technische Universität Darmstadt, Institut für AutomatisierungstechnikDarmstadtGermany

Personalised recommendations