Circuits, Systems and Signal Processing

, Volume 1, Issue 3–4, pp 395–406 | Cite as

Estimation of structure by minimum description length

  • J. Rissanen


A theorem is proved which demonstrates that an earlier derived minimum description length estimation criterion is capable of distinguishing between structures in linear models for vector processes. A fairly simple algorithm is described for the estimation of the best model, including its structure and the number of its parameters.


Linear Model Simple Algorithm Minimum Description Length Length Estimation Vector Process 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    H. Akaike, “A New Look at the Statistical Model Identification,” IEEE Trans. AC-19, 1974, pp. 716–723.Google Scholar
  2. 2.
    H. Akaike, “Markovian Representation of Stochastic Processes by Canonical Variables,“ SIAM J. Control, 13, 1975, pp. 162–173.Google Scholar
  3. 3.
    J. M. C. Clark, “The Consistent Selection of Local Coordinates in Linear Systems Identification,” JACC, Purdue University, Lafayette, Indiana, pp. 576–580, July 1976.Google Scholar
  4. 4.
    M. Deistler and E. J. Hannan, ‘Some Properties of the Parameterization of Arma Systems with Unknown Order,” J. Multivariate Analysis, 11, 1981, pp. 474–484.Google Scholar
  5. 5.
    K. Glover and J. C. Willems, “Parameterizations of Linear Dynamical Systems: Canonical Forms and Identifiability,” IEEE Trans. AC-19, 1974, pp.640–646.Google Scholar
  6. 6.
    E. J. Hannan, “The Estimation of the Order of an ARMA Process,” Ann. Stat., 8, 1980, pp. 1071–1081.Google Scholar
  7. 7.
    E. J. Hannan and B. Quinn, “The Determination of the Order of an Autoregression,” J. Royal Statist. Soc. B, 1979, pp. 190–195.Google Scholar
  8. 8.
    E. J. Hannan and J. Rissanen (1981), “Recursive Estimation of ARMA Order,” Biometrika, 1982, 69, 1, pp. 81–94.Google Scholar
  9. 9.
    R. E. Kalman, “Algebraic Geometric Description of the Class of Linear Systems of Constant Dimension,” 8th Ann. Princeton Con. on Inf. Sciences and Systems, Princeton, New Jersey, March, 1974, pp. 189–191.Google Scholar
  10. 10.
    R. E. Kalman, P. L. Falb, M. A. Arbib, Topics in Mathematical System Theory, McGraw Hill, New York, 1969.Google Scholar
  11. 11.
    L. Ljung and J. Rissanen, “On Canonical Forms, Parameter Identifiability and the Concept of Complexity,“ IFAC Symp. on Identification, Tbilisi, USSR, 1976.Google Scholar
  12. 12.
    D. G. Luenberger, “Canonical Forms for Linear Multivariable Systems,“ IEEE Trans. AC-12, 1974, pp. 290–293.Google Scholar
  13. 13.
    A. J. M. Van Overbeek and L Ljung, “On Line Structure Selection for Multivariable State Space Models,” IFAC Symp. on Identification and Parameter Estimation, Darmstadt, 1979.Google Scholar
  14. 14.
    V. M. Popov, “Invariant Description of Linear, Time-Invariant Controllable Systems,” SIAM J. Control, 10, 1972, pp. 254–264.Google Scholar
  15. 15.
    J. Rissanen, “Basis of Invariants and Canonical Forms for Linear Dynamic Systems,” Automatica, 10, 1974, pp. 175–182.Google Scholar
  16. 16.
    J. Rissanen, “Modeling by Shortest Data Description,” Automatica, 14, 1978, pp. 465–471.Google Scholar
  17. 17.
    J. Rissanen, “Consistent Order Estimates of Autoregressive Processes by Shortest Description of Data,” Analysis and Optimisation of Stochastic Systems (ed. O. Jacobs, M. Davis, M. Dempster, C. Harris, P. Parks), Academic Press, New York, 1980.Google Scholar
  18. 18.
    J. Rissanen, “Universal Prior for Parameters and Estimation by Minimum Description Length,“ IBM Res. Report RJ3127, 1981.Google Scholar
  19. 19.
    J. Rissanen and L. Ljung, “Estimation of Optimum Structures and Parameters for Linear Systems,“ Proc. CNR. CISM Symp. on Algebraic System Theory, Udine, 1975; Math. System Theory 131, Springer-Verlag, pp. 76–91.Google Scholar
  20. 20.
    V. Wertz, M. Gevers, E. J. Hannan, “The Determination of Optimum Structures for the State Space Representation of Multivariable Stochastic Processes,” (personal communication).Google Scholar
  21. 21.
    A. H. Wallace, Differential Topology: First Step, Benjamin, Elmsford, New York, 1968.Google Scholar

Copyright information

© Birkhäuser Boston, Inc. 1982

Authors and Affiliations

  • J. Rissanen
    • 1
  1. 1.IBM CorporationSan JoseUSA

Personalised recommendations