Skip to main content
SpringerLink
Log in

Modifications of the Least Squares Parameter Estimation

  • Chapter
  • First Online:
Identification of Dynamic Systems

  • 5906 Accesses

Abstract

In order to obtain bias-free estimates of linear dynamic processes by the method of least squares, the error signal e(k) may not be correlated. This requirement is only satisfied if the disturbance n(k) that is acting on the system is a colored noise that is generated from a white noise v(k) filtered by a form filter with the transfer function 1/A(z -1). Since this prerequisite is hardly ever met in practice, the method of least squares typically works on a correlated error signal and hence yields biased estimates. The bias can be so high for larger noise levels that the results are unusable. To avoid this problem, in the following, methods are presented which yield bias-free estimates for larger classes of dynamic processes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Albert AE, Gardner LA Jr (1967) Stochastic approximation and non-linear regression. MIT Press, Cambridge, MA

    Google Scholar 

  • Baur U (1976) On-Line Parameterschätzverfahren zur Identifikation linearer, dynamischer Prozesse mit Prozeßrechnern: Entwicklung, Vergleich, Erprobung: KfK-PDV-Bericht Nr. 65. Kernforschungszentrum Karlsruhe, Karlsruhe

    Google Scholar 

  • Blum J (1954) Multidimensional stochastic approximation procedures. Ann Math Statist 25(4):737–744

    Article  MATH  Google Scholar 

  • Brown M, Harris C (1994) Neurofuzzy adaptive modelling and control. Prentice-Hall international series in systems and control engineering, Prentice Hall, New York

    Google Scholar 

  • Clarke DW (1967) Generalized least squares estimation of the parameters of a dynamic model. In: Preprints of the IFAC Symposium on Identification, Prag

    Google Scholar 

  • Durbin J (1954) Errors in variables. Revue de l’Institut International de Statistique / Review of the International Statistical Institute 22(1/3):23–32

    Article  MathSciNet  Google Scholar 

  • Dvoretzky A (1956) On stochastic approximation. In: Proceedings of the 3rd Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA

    Google Scholar 

  • Goedecke W (1987) Fehlererkennung an einem thermischen Prozeß mit Methoden der Parameterschätzung: Fortschr.-Ber. VDI Reihe 8 Nr. 130. VDI Verlag, Düsseldorf

    Google Scholar 

  • Golub GH, van Loan C (1980) An analysis of the total least squares problems. SIAM J Numer Anal 17(6):883–893

    Article  MathSciNet  MATH  Google Scholar 

  • Golub GH, Reinsch C (1970) Singular value decomposition and least squares solutions. Numer Math 14(5):403–420

    Article  MathSciNet  MATH  Google Scholar 

  • Goodwin GC, Sin KS (1984) Adaptive filtering, prediction and control. Prentice-Hall information and system sciences series, Prentice-Hall, Englewood Cliffs, NJ

    MATH  Google Scholar 

  • Hastings-James R, Sage MW (1969) Recursive generalized least-squares procedure for on-line identification of process parameters. Proc IEE 116(12):2057–2062

    Google Scholar 

  • Haykin S, Widrow B (2003) Least-mean-square adaptive filters. Wiley series in adaptive and learning systems for signal processing, communication and control, Wiley-Interscience, Hoboken, NJ

    Google Scholar 

  • van Huffel S, Vandewalle J (1991) The total least squares problem: Computational aspects and analysis, Frontiers in applied mathematics, vol 9. SIAM, Philadelphia, PA

    Google Scholar 

  • Isermann R (1974) Prozessidentifikation: Identifikation und Parameterschätzung dynamischer Prozesse mit diskreten Signalen. Springer, Heidelberg

    Google Scholar 

  • Isermann R, Baur U (1973) Results of testcase A. In: Proceedings of the 3rd IFAC Symposium on System Identification, North Holland Publ. Co., Amsterdam, Netherlands

    Google Scholar 

  • Jolliffe IT (2002) Principal component analysis, 2nd edn. Springer series in statistics, Springer, New York

    MATH  Google Scholar 

  • Joseph P, Lewis J, Tou J (1961) Plant identification in the presence of disturbances and application to digital control systems. Trans AIEE (Appl and Ind) 80:18–24

    Google Scholar 

  • Kendall MG, Stuart A (1961) The advanced theory of statistics. Volume 2. Griffin, London, UK

    Google Scholar 

  • Kiefer J, Wolfowitz J (1952) Statistical estimation of the maximum of a regression function. Ann Math Stat 23(3):462–466

    Article  MathSciNet  MATH  Google Scholar 

  • Kumar PR, Moore JB (1979) Towards bias elimination in least squares system identification via detection techniques. In: Proceedings of the 5th IFAC Symposium on Identification and System Parameter Estimation Darmstadt, Pergamon Press, Darmstadt, Germany

    Google Scholar 

  • Kushner HJ, Yin GG (2003) Stochastic approximation and recursive algorithms and applications, Applications of mathematics, vol 35, 2nd edn. Springer, New York, NY

    Google Scholar 

  • Ljung L (1999) System identification: Theory for the user, 2nd edn. Prentice Hall Information and System Sciences Series, Prentice Hall PTR, Upper Saddle River, NJ

    Google Scholar 

  • Markovsky I, van Huffel S (2007) Overview of total least-squares methods. Signal Proc 87(10):2283–2302

    Article  MATH  Google Scholar 

  • Markovsky I, Willems JC, van Huffel S, de Bart M, Pintelon R (2005) Application of structured total least squares for system identification and model reduction. IEEE Trans Autom Control 50(10):1490–1500

    Article  Google Scholar 

  • Panuska V (1969) An adaptive recursive least squares identification algorithm. In: Proceedings of the IEEE Symposium in Adaptive Processes, Decision and Control

    Google Scholar 

  • Pintelon R, Schoukens J (2001) System identification: A frequency domain approach. IEEE Press, Piscataway, NJ

    Book  Google Scholar 

  • Reiersøl O (1941) Confluence analysis by means of lag moments and other methods of confluence analysis. Econometrica 9(1):1–24

    Article  MathSciNet  MATH  Google Scholar 

  • Robbins H, Monro S (1951) A stochastic approximation method. Ann Math Statist 22(3):400–407

    Article  MathSciNet  MATH  Google Scholar 

  • Sage AP, Melsa JL (1971) System identification. Academic Press, New York

    MATH  Google Scholar 

  • Sakrison DJ (1966) Stochastic approximation. Adv Commun Syst 2:51–106

    Google Scholar 

  • Saridis GN, Stein G (1968) Stochastic approximation algorithms for linear discretetime system identification. IEEE Trans Autom Control 13(5):515–523

    Article  Google Scholar 

  • Söderström T (2007) Errors-in-variables methods in system identification. Automatica 43(6):939–958

    Article  MathSciNet  MATH  Google Scholar 

  • Söderström T, Stoica PG (1983) Instrumental variable methods for system identification, Lecture notes in control and information sciences, vol 57. Springer, Berlin

    Book  Google Scholar 

  • Steiglitz K, McBride LE (1965) A technique for the identification of linear systems. IEEE Trans Autom Control 10:461–464

    Article  Google Scholar 

  • Stoica P, Jansson M (2000) On the estimation of optimal weights for instrumental variable system identification methods. In: Proccedings of the 12th IFAC Symposium on System Identification, Santa Barbara, CA, USA

    Google Scholar 

  • Stoica PG, Söderström T (1977) A method for the identification of linear systems using the generalized least squares principle. IEEE Trans Autom Control 22(4):631–634

    Article  MATH  Google Scholar 

  • Stoica PG, Söderström T (1982) Bias correction in least squares identification. Int J Control 35(3):449–457

    Article  MATH  Google Scholar 

  • Wong K, Polak E (1967) Identification of linear discrete time systems using the instrumental variable method. IEEE Trans Autom Control 12(6):707–718

    Article  Google Scholar 

  • Young P (1968) The use of linear regression and related procedures for the identification of dynamic processes. In: Proceedings of the 7th IEEE Symposium in Adaptive Processes, Los Angeles, CA, USA

    Google Scholar 

  • Young P (1970) An instrumental variable method for real-time identification of a noisy process. Automatica 6(2):271–287

    Article  Google Scholar 

  • Zimmerschied R (2008) Identifikation nichtlinearer Prozesse mit dynamischen lokalaffinen Modellen : Maßnahmen zur Reduktion von Bias und Varianz. Fortschr.-Ber. VDI Reihe 8 Nr. 1150. VDI Verlag, Düsseldorf

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Technische Universität Darmstadt, Institut für Automatisierungstechnik, Landgraf-Georg-Straße 4, 64283, Darmstadt, Germany

    Prof. Dr. -Ing. Dr. h.c. Rolf Isermann & Dr. -Ing. Marco Münchhof M.S./SUNY

Authors
  1. Prof. Dr. -Ing. Dr. h.c. Rolf Isermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dr. -Ing. Marco Münchhof M.S./SUNY
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rolf Isermann .

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Berlin Heidelberg

About this chapter

Cite this chapter

Isermann, R., Münchhof, M. (2011). Modifications of the Least Squares Parameter Estimation. In: Identification of Dynamic Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78879-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-78879-9_10

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-78878-2

  • Online ISBN: 978-3-540-78879-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation