Recursive Estimation in Armax Models

  • Tze Leung Lai
Part of the The IMA Volumes in Mathematics and its Applications book series (IMA, volume 46)


Herein we first review some important algorithms and their statistical properties in the literature on recursive estimation of the parameters of an ARMAX model. We then describe some recent developments of efficient procedures for recursive estimation and their statistical theory. These developments not only extend important statistical properties such as consistency, asymptotic normality, asymptotic efficiency, that have been established for certain classes of offline estimators to their recursive counterparts, but are also applicable to on-line adaptive prediction and adaptive control of ARMAX systems.


Adaptive Control Stochastic Approximation Recursive Estimation Martingale Difference Sequence Stochastic Gradient Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    T. W. ANDERSON, Maximum likelihood estimation of parameters of autoregressive processes with moving average residuals and other covariance matrices with linear structure, Ann. Statist. 3(1975), pp. 1283–1304.MathSciNetMATHCrossRefGoogle Scholar
  2. [2]
    T. W. ANDERSON, Maximum likelihood estimation for vector autoregressive moving average models, in Directions in Time Series (D. R. Brillinger and G. C. Tiao, ed.), Institute of Mathematical Statistics, Hayward, 1980, pp. 49–59.Google Scholar
  3. [3]
    M. AOKI, Optimization of Stochastic Systems: Topics in Discrete-time Dynamics, Second edition, Academic Press, New York, 1989.Google Scholar
  4. [4]
    K. J. ÅSTRÖM AND P. EYKHOF, System identification — a survey, Automatica, 7(1971), pp. 123–167.MATHCrossRefGoogle Scholar
  5. [5]
    G. E. P. BOX AND G. M. JENKINS, Time Series Analysis, Forecasting and Control, Holden-Day, San Francisco, 1970.MATHGoogle Scholar
  6. [6]
    O. H. BUSTOS AND V. J. YOHAI, Robust estimates for ARMA models, J. Amer. Statist. Assoc, 81(1986), pp. 155–168.MathSciNetCrossRefGoogle Scholar
  7. [7]
    P. E. CAINES AND S. LAFORTUNE, Adaptive control with recursive identification for stochastic linear systems, IEEE Trans. Automat. Contr., AC-29(1984), pp. 312–321.MathSciNetCrossRefGoogle Scholar
  8. [8]
    N. H. CHAN AND C. Z. WEI, Limiting distributions of least squares estimates of unstable autoregressive processes, Ann. Statist., 16(1988), pp. 367–401.MathSciNetMATHCrossRefGoogle Scholar
  9. [9]
    H. F. CHEN AND L. GUO, Asymptotically optimal adaptive control with consistent parameter estimates, SIAM J. Contr. Optimiz., 25(1987), pp. 558–575.MathSciNetMATHCrossRefGoogle Scholar
  10. [10]
    A. P. DEMPSTER, N. M. LAIRD, AND D. B. RUBIN, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc. Ser. B(1977), pp. 1–38.Google Scholar
  11. [11]
    L. DENBY AND R. D. MARTIN, Robust estimation on the first order autoregressive parameter, J. Amer. Statist. Assoc, 74(1979), pp. 140–146.MATHCrossRefGoogle Scholar
  12. [12]
    P. J. DHRYMES, Distributed Lags: Problems of Estimation and Formulation, Holden-Day, San Francisco 1971.MATHGoogle Scholar
  13. [13]
    W. A. FULLER, Introduction to Statistical Time Series, Wiley, New York, 1976.MATHGoogle Scholar
  14. [14]
    W. A. FULLER, Nonstationary autoregressive time series, in Handbook of Statistics Vol. 5 (E. J. Hannan, P. R. Krishnaiah, M. M. Rao, ed.), North Holland, Amsterdam, 1985, pp. 1–23.Google Scholar
  15. [15]
    G. C. GOODWIN, P. J. RAMADGE AND P. E. CAINES, A globally convergent adaptive predictor, Automatica, 17(1981), pp. 135–140.MathSciNetMATHCrossRefGoogle Scholar
  16. [16]
    G. C. GOODWIN, P. J. RAMADGE AND P. E. CAINES, Discrete time stochastic adaptive control, SIAM J. Contr. Optimiz., 19(1981), pp. 829–853.MathSciNetMATHCrossRefGoogle Scholar
  17. [17]
    G. C. GOODWIN AND K. S. SIN, Adaptive Filtering, Prediction and Control, Prentice-Hall, Englewood Cliffs, 1984.MATHGoogle Scholar
  18. [18]
    I. A. IBRAGIMOV AND R. Z. HAS’MINSKII, Statistical Estimation — Asymptotic Theory, Springer-Verlag, New York, 1981.MATHGoogle Scholar
  19. [19]
    T. L. LAI, Asymptotically efficient adaptive control in stochastic regression models, Adv. Appl. Math., 7(1986), pp. 23–45.MATHCrossRefGoogle Scholar
  20. [20]
    T. L. LAI, Extended stochastic Lyapunov functions and recursive algorithms in stochastic linear systems, in Stochastic Differential Systems: Proceedings of the 4th Bad Honnef Conference (N. Christopeit et al., ed.), Springer-Verlag, New York, 1989, pp. 206–220.Google Scholar
  21. [21]
    T. L. LAI AND C. Z. WEI, Least squares estimates in stochastic regression models with applications to identification and control, Ann. Statist., 10(1982), pp. 154–166.MathSciNetCrossRefGoogle Scholar
  22. [22]
    T. L. LAI AND C. Z. WEI, Some asymptotic properties of general autoregressive models and strong consistency of least squares estimates of their parameters, J. Multivariate Anal., 13(1983), pp. 1–23.MathSciNetMATHCrossRefGoogle Scholar
  23. [23]
    T. L. LAI AND C. Z. WEI, Extended least squares and their applications to adaptive control and prediction in linear systems, IEEE Trans. Automat. Contr., AC-31(1986), pp. 898–906.MathSciNetGoogle Scholar
  24. [24]
    T. L. LAI AND C. Z. WEI, On the concept of excitation in least squares identification and adaptive control, Stochastics, 16(1986), pp. 227–254.MathSciNetMATHCrossRefGoogle Scholar
  25. [25]
    T. L. LAI AND C. Z. WEI, Asymptotically efficient self-tuning regulators, SIAM J. Contr. Optimiz., 25(1987), pp. 466–481.MathSciNetMATHCrossRefGoogle Scholar
  26. [26]
    T. L. LAI, C. Z. WEI AND Y. G. ZHANG, Convergence properties of some recursive identification schemes and adaptive predictors, Proc. 2nd Amer. Control Conference, 1982, pp. 176–180.Google Scholar
  27. [27]
    T. L. LAI AND Z. YING, Parallel recursive algorithms in asymptotically efficient adaptive control of linear stochastic systems, to appear in SIAM J. Contr. Optimiz.Google Scholar
  28. [28]
    T. L. LAI AND Z. YING, Recursive identification and adaptive prediction in linear stochastic systems, to appear in SIAM J. Contr. Optimiz.Google Scholar
  29. [29]
    T. L. LAI AND Z. YING, Recursive solutions of estimating equations and adaptive spectral factorization, to appear in IEEE Trans. Automat. Contr.Google Scholar
  30. [30]
    T. L. LAI AND Z. YING, Consistent and asymptotically efficient recursive estimators in time series and stochastic regression models with moving average errors, Technical Report, Department of Statistics, Stanford University, 1990.Google Scholar
  31. [31]
    L. LJUNG, On positive real transfer functions and the convergence of some recursive schemes, IEEE Trans. Automat. Contr., AC-22(1977), pp. 539–551.MathSciNetCrossRefGoogle Scholar
  32. [32]
    L. LJUNG, Analysis of recursive stochastic algorithms, IEEE Trans. Automat. Contr., AC-22(1977), pp. 551–575.MathSciNetCrossRefGoogle Scholar
  33. [33]
    L. LJUNG AND T. SÖDERSTÖm, Theory and Practice of Recursive Estimation, MIT Press, Cambridge, 1983.Google Scholar
  34. [34]
    R. D. MARTIN AND V. J. YOHAI, Robustness in time series and estimating ARMA models, in Handbook of Statistics Vol. 5 (E. J. Hannan, P. R. Krishnaiah, M. M. Rao, ed.), North Holland, Amsterdam, 1985, pp. 119–155.Google Scholar
  35. [35]
    J. B. MOORE AND G. LEDWICH, Multivariable adaptive parameter and state estimators with convergence analysis, J. Austr. Math. Soc. Ser. B, 21(1979), pp. 176–197.MathSciNetMATHCrossRefGoogle Scholar
  36. [36]
    M. NERLOVE, D. M. GRETHER AND J. L. CARVALHO, Analysis of Economic Time Series: A Synthesis, Academic Press, New York, 1979.MATHGoogle Scholar
  37. [37]
    M. B. NEVEL’SON AND R. Z. HAS’MINSKII, Stochastic Approximation and Recursive Estimation, Amer. Math. Soc. Transi., Providence, 1973.Google Scholar
  38. [38]
    H. ROBBINS AND S. MONRO, A stochastic approximation method, Ann. Math. Statist., 22(1951), pp. 400–407.MathSciNetMATHCrossRefGoogle Scholar
  39. [39]
    R. J. SCHILLER, Rational expectations and the dynamic structure of macroeconomic models, J. Monetary Economics, 4(1978), pp. 1–44.CrossRefGoogle Scholar
  40. [40]
    V. SOLO, On the convergence of AML, IEEE Trans. Automat. Contr., AC-24(1979), pp. 958–962.CrossRefGoogle Scholar
  41. [41]
    B. P. STIGUM, Asymptotic properties of dynamic stochastic parameter estimates, J. Multivariate Anal., 4(1974), pp. 351–381.MathSciNetMATHCrossRefGoogle Scholar
  42. [42]
    R. S. TSAY AND G. C. TIAO, Consistent estimates of autoregressive parameters and extended sample autocorrelation functions for stationary and nonstationary ARMA models, J. Amer. Statist. Assoc, 79(1984), pp. 84–96.MathSciNetMATHCrossRefGoogle Scholar
  43. [43]
    K. F. WALLIS, Econometric implications of the rational expectations hypothesis, Econometrica, 48(1980), pp. 49–73.MathSciNetMATHCrossRefGoogle Scholar
  44. [44]
    C. Z. WEI, Adaptive prediction by least squares in stochastic regression models with applications to time series, Ann. Statist., 15(1987), pp. 1667–1682.MathSciNetMATHCrossRefGoogle Scholar
  45. [45]
    G. WILSON, Factorization of the covariance generating function of a pure moving average process, SIAM J. Numer. Anal., 6(1979), pp. 1–7.CrossRefGoogle Scholar
  46. [46]
    P. C. YOUNG, Recursive Estimation and Time Series Analysis: An Introduction, Springer-Verlag, New York, 1984.MATHGoogle Scholar

Copyright information

© Springer-Verlag New York, Inc. 1993

Authors and Affiliations

  • Tze Leung Lai
    • 1
  1. 1.Department of StatisticsStanford UniversityStanfordUSA

Personalised recommendations