A New Look at the Statistical Model Identification

  • Hirotugu Akaike
Part of the Springer Series in Statistics book series (SSS)

Abstract

The history of the development of statistical hypothesis testing in time series analysis is reviewed briefly and it is pointed out that the hypothesis testing procedure is not adequately defined as the procedure for statistical model identification. The classical maximum likelihood estimation procedure is reviewed and a new estimate minimum information theoretical criterion (AIC) estimate (MAICE) which is designed for the purpose of statistical identification is introduced. When there are several competing models the MAICE is defined by the model and the maximum likelihood estimates of the parameters which give the minimum of AIC defined by AIC = (−2)log- (maximum likelihood) + 2(number of independently adjusted parameters within the model). MAICE provides a versatile procedure for statistical model identification which is free from the ambiguities inherent in the application of conventional hypothesis testing procedure. The practical utility of MAICE in time series analysis is demonstrated with some numerical examples.

Keywords

Entropy Covariance 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    H. Akaike, “Stochastic theory of minimal realization,” this issue, pp. 667–674.Google Scholar
  2. [2]
    E. L. Lehman, Testing Statistical Hypothesis. New York: Wiley, 1959.Google Scholar
  3. [3]
    H. Akaike, “Information theory and an extension of the maximum likelihood principle,” in Proc. 2nd Int. Symp. Information Theory, Supp. to Problems of Control and Information Theory, 1972, pp. 267–281.Google Scholar
  4. [4]
    M. H. Quenouille, “A large-sample test for the goodness of fit of autoregressive schemes,” J. Roy. Statist. Soc., vol. 110, pp. 123–129, 1947.MathSciNetCrossRefMATHGoogle Scholar
  5. [5]
    H. Weld, “A large-sample test for moving averages,” J. Roy. Statist. Soc., B, vol. 11, pp. 297–305, 1949.Google Scholar
  6. [6]
    M. S. Barlett and P. H. Diananda, “Extensions of Quenouille’s test for autoregressive scheme,” J. Roy. Statist. Soc., B, vol. 12, pp. 108–115, 1950.Google Scholar
  7. [7]
    M. S. Bartlett and D. V. Rajalakshman, “Goodness of fit test for simultaneous autoregressive series,” J. Roy. Statist. Soc., B, vol. 15, pp. 107–124, 1953.MathSciNetMATHGoogle Scholar
  8. [8]
    A. M. Walker, “Note on a generalization of the large sample goodness of fit test for linear autoregressive schemes,” J. Roy. Statist. Soc., B, vol. 12, pp. 102–107, 1950.MATHGoogle Scholar
  9. [9]
    A. M. Walker,, “The existence of Bartlett-Rajalakshman goodness of fit G-tests for multivariate autoregressive processes with finitely dependent residuals,” Proc. Cambridge Phil. Soc., vol. 54, pp. 225–232, 1957.CrossRefGoogle Scholar
  10. [10]
    P. Whittle, Hypothesis Testing in Time-Series Analysis. Uppsala, Sweden: Almqvist and Wiksell, 1951.Google Scholar
  11. [11]
    Some recent contributions to the theory of stationary processes,“ A Study in the Analysis of Stationary Time Series. Uppsala, Sweden: Almqvist and Wiksell, 1954, appendix 2.Google Scholar
  12. [12]
    G. E. P. Box and G. M. Jenkins, Time Series, Forecasting and Control. San Francisco, Calif.: Holden-Day, 1970.Google Scholar
  13. [13]
    I. Gustaysson, “Comparison of different methods for identifies-t197ion2. of industrial processes,” Automatics,vol. 8, pp. 127–142Google Scholar
  14. [14]
    R. K. Mehra, “On the identification of variances and adaptive Kalman filtering,” IEEE Trans. Automat. Contr., vol. AC-15, pp. 175–184, Apr. 1970.Google Scholar
  15. [15]
    R. K. Mehra, “On-line identification of linear dynamic systems with applications to Kalman filtering,” IEEE Trans. Automat. Contr., vol. AC-16, pp. 12–21, Feb. 1971.Google Scholar
  16. [16]
    E. J. Hannan, Time Series Analysis. London, England: Methuen, 1960.MATHGoogle Scholar
  17. [17]
    T. W. Anderson, “Determination of the order of dependence in normally distributed time series,” in Time Series Analysis, M. Rosenblatt, Ed. New York: Wiley, 1963, pp. 425–446.Google Scholar
  18. [18]
    C. L. Mallows, “Some comments on Cr,” Technometrics, vol. 15, pp. 661–675, 1973.MATHGoogle Scholar
  19. [19]
    L. D. Davisson, “The prediction error of stationary Gaussian time series of unknown covariance,” IEEE Trans. Inform. Theory, vol. IT-11, pp. 527–532, Oct. 1965.Google Scholar
  20. [20]
    A theory of adaptive filtering,“ IEEE Trans. Inform. Theory,vol. IT-12, pp. 97–102, Apr. 1966.Google Scholar
  21. [21]
    H. Akaike, “Fitting autoregressive models for prediction,” Ann. Inst. Statist. Math., vol. 21, pp. 243–247, 1969.MathSciNetCrossRefMATHGoogle Scholar
  22. [22]
    Statistical predictor identification,“ Ann. Inst. Statist.Math., vol. 22, pp. 203–217, 1970.Google Scholar
  23. [23]
    On a semiautomatic power spectrum estimation pro-cedure,“ in Proc. 3rd Hawaii Int. Conf. System Sciences,1970, pp. 974–977.Google Scholar
  24. [24]
    R. H. Jones, “Autoregressive spectrum estimation,” in 3rd Conf. Probability and Statistics In Atmospheric Sciences, Preprints, Boulder, Colo., June 19–22, 1973.Google Scholar
  25. [25]
    W. Gersch and D. R. Sharpe, “Estimation of power spectra with finite-order autoregressive models,” IEEE Trans. Automat. Contr., vol. AC-18, pp. 367–379, Aug. 1973.Google Scholar
  26. [26]
    R. J. Bhansali, “A Monte Carlo comparison of the regression method and the spectral methods of prediction,” J. Amer. Statist. Ass., vol. 68, pp. 621–625, 1973.CrossRefMATHGoogle Scholar
  27. [27]
    H. Akaike, “Autoregressive model fitting for control,” Ann. Inst. Statist. Math., vol. 23, pp. 163–180, 1971.MathSciNetCrossRefMATHGoogle Scholar
  28. [28]
    T. Otomo, T. Nakagawa, and H. Akaike, “Statistical approach to computer control of cement rotary kilns,” Automatica, vol. 8, pp. 3548, 1972.CrossRefGoogle Scholar
  29. [29]
    H. Cramer, Mathematical Methods of Statistics. Princeton, N. J.: Princeton Univ. Press, 1946.Google Scholar
  30. [30]
    S. Kullback, Information Theory and Statistics. New York: Wiley, 1959.MATHGoogle Scholar
  31. [31]
    M. S. Bartlett, “The statistical approach to the analysis of time series,” in Proc. Symp. Information Theory, London, England, Ministry of Supply, 1950, pp. 81–101.Google Scholar
  32. [32]
    P. J. Hither, “The behavior of maximum likelihood estimates under nonstandard conditions,” in Proc. 5th Berkeley Symp. Mathematical Statistics and Probability, vol. 1, pp. 221–233, 1967.Google Scholar
  33. [33]
    H. Akaike, “Markovian representation of stochastic processes and its application to the analysis of autoregressive moving average processes,” Ann. Inst. Statist. Math.,to be published.Google Scholar
  34. [34]
    P. Whittle, “Gaussian estimation in stationary time series, ” Bull. Int. Statist. Inst., vol. 39, pp. 105–129, 1962.MATHGoogle Scholar
  35. [35]
    H. Akaike, “Use of an information theoretic quantity for statistical model identification,” in Proc. 5th Hawaii Int. Conf. System Sciences, pp. 249–250, 1972.Google Scholar
  36. [36]
    H. Akaike, “Automatic data structure search by the maximum likelihood,” in Computer in Biomedicine Suppl. to Proc. 5th Hawaii Int. Conf. on System Sciences, pp. 99–101, 1972.Google Scholar
  37. [37]
    T. W. Anderson, The Statistical Analysis of Time Series. New York: Wiley, 1971.MATHGoogle Scholar
  38. [38]
    J. D. Sargon, “An approximate treatment of the properties of the cxrrelogram and periodgram,” J. Roy. Statist. Soc. B, vol. 15, pp. 140–152, 1953.Google Scholar
  39. [39]
    G. M. Jenkins and D. G. Watts, Spectral Analysis and its Applications. San Francisco, Calif.: Holden-Day, 1968.MATHGoogle Scholar
  40. [40]
    P. Whittle, “The statistical analysis of a seiche record,” J. Marine Res., vol. 13, pp. 76–100, 1954.MathSciNetGoogle Scholar
  41. [41]
    P. Whittle, Prediction and Regulation. London, England: English Univ. Press, 1963.Google Scholar
  42. [42]
    D. R. Cox, “Tests of separate families of hypotheses,” in Proc. 4th Berkeley Symp. Mathematical Statistics and Probability, vol. 1, 1961, pp. 105–123.Google Scholar
  43. [43]
    D. R. Cox, “Further results on tests of separate families of hypotheses,” J. Roy. Statist. Soc., B, vol. 24, pp. 406–425, 1962.MATHGoogle Scholar
  44. [44]
    A. M. Walker, “Some tests of separate families of hypotheses in time series analysis,” Biometrika, vol. 54 pp. 39–68, 1987.Google Scholar
  45. [45]
    W. J. Kennedy and T. A. Bancroft, “Model building for prediction in regression based upon repeated significance testa,” Ann. Math. Statist., vol. 42, pp. 1273–1284, 1971.MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1974

Authors and Affiliations

  • Hirotugu Akaike
    • 1
  1. 1.The Institute of Statistical MathematicsMinato-ku, TokyoJapan

Personalised recommendations