Complexity Approximation Principle and Rissanen’s Approach to Real-Valued Parameters

  • Yuri Kalnishkan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1810)


In this paper an application of the Complexity Approximation Principle to the non-linear regression is suggested. We combine this principle with the approximation of the complexity of a real-valued vector parameter proposed by Rissanen and thus derive a method for the choice of parameters in the non-linear regression.


Dual Variable Ridge Regression Little Square Estimate Minimum Description Length Kolmogorov Complexity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Thomas H. Cormen, Charles E. Leiserson, and Ronald L. Rivest. Introduction to Algorithms. The MIT Press, 1990. 205Google Scholar
  2. 2.
    M. Li and P. Vitányi. An Introduction to Kolmogorov Complexity and Its Applications. Springer, New York, 2nd edition, 1997. 204, 206zbMATHGoogle Scholar
  3. 3.
    J Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):416–431, 1983. 203, 204, 205, 206zbMATHMathSciNetCrossRefGoogle Scholar
  4. 4.
    J Rissanen. Stochastic complexity. Journal of Royal Statistical Society, 49(3):223–239, 1987. 204zbMATHMathSciNetGoogle Scholar
  5. 5.
    J Rissanen. Stochastic complexity in learning. Journal of Computer and System Sciences, 55:89–95, 1997. 204zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    C. Saunders, A. Gammerman, and V. Vovk. Ridge regression learning algorithm in dual variables. In Proceedings of the 15th International Conference on Machine Learning, pages 515–521, 1998. 207, 208, 209Google Scholar
  7. 7.
    Mark J. Schervish. Theory of Statistics. Springer-Verlag, 1995. 207Google Scholar
  8. 8.
    Gábor J. Székely. Paradoxes in Probability Theory and Mathematical Statistics. Akadémiai Kiadó Budapest, 1986. 207Google Scholar
  9. 9.
    V. Vovk and A. Gammerman. Complexity approximation principle. The Computer Journal, 42(4):318–322, 1999. 203, 204zbMATHCrossRefGoogle Scholar
  10. 10.
    V. Vovk and C. J. H. C. Watkins. Universal portfolio selection. In Proceedings of the 11th Annual Conference on Computational Learning Theory, pages 12–23, 1998. 204Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Yuri Kalnishkan
    • 1
  1. 1.Department of Computer Science, Royal HollowayUniversity of LondonEgham, SurreyUK

Personalised recommendations