Skip to main content
SpringerLink
Log in

Predictive Complexity and Information

  • Conference paper
  • First Online:
Computational Learning Theory (COLT 2002)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2375))

Included in the following conference series:

Abstract

A new notion of predictive complexity and corresponding amount of information are considered. Predictive complexity is a generalization of Kolmogorov complexity which bounds the ability of any algorithm to predict elements of a sequence of outcomes. We consider predictive complexity for a wide class of bounded loss functions which are generalizations of square-loss function. Relations between unconditional KG(x) and conditional KG(xy) predictive complexities are studied. We define an algorithm which has some “expanding property”. It transforms with positive probability sequences of given predictive complexity into sequences of essentially bigger predictive complexity. A concept of amount of predictive information IG(y: x) is studied. We show that this information is non-commutative in a very strong sense and present asymptotic relations between values IG(y: x), IG(x: y), KG(x) and KG(y).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Haussler, D., Kivinen, J., Warmuth, M. K. (1994) Tight worst-case loss bounds for predicting with expert advice. Technical Report UCSC-CRL-94-36, University of California at Santa Cruz, revised December 1994. Short version in P. Vitányi, editor, Computational Learning Theory, Lecture Notes in Computer Science, volume 904, pages 69–83, Springer, Berlin, 1995.

    Google Scholar 

  2. Cesa-Bianchi, N., Freund, Y., Helmbold, D. P., Haussler, D., Schapire, R. E., Warmuth, M. K. (1997) How to use expert advice. Journal of the ACM, 44, 427–485

    Article  MATH  MathSciNet  Google Scholar 

  3. Cormen, H., Leiserson, E., Rivest, R (1990) Introduction to Algorithms. New York: McGraw Hill.

    Google Scholar 

  4. Kalnishkan, Y. (1999) General linear relations among different types of predictive complexity. In. Proc. 10th international Conference on Algorithmic Learning Theory-ALT’ 99, v. 1720 of Lecture Notes in Artificial Intelligence, pp. 323–334, Springer-Verlag.

    Google Scholar 

  5. Li, M., Vitányi, P. (1997) An Introduction to Kolmogorov Complexity and Its Applications. Springer, New York, 2nd edition.

    MATH  Google Scholar 

  6. Rogers, H. (1967) Theory of recursive functions and effective computability, New York: McGraw Hill.

    MATH  Google Scholar 

  7. Vovk, V. (1990) Aggregating strategies. In M. Fulk and J. Case, editors, Proceedings of the 3rd Annual Workshop on Computational Learning Theory, pages 371–383, San Mateo, CA, 1990. Morgan Kaufmann.

    Google Scholar 

  8. Vovk, V. (1998) A game of prediction with expert advice. J. Comput. Syst. Sci., 56:153–173.

    Article  MATH  MathSciNet  Google Scholar 

  9. Vovk, V., Gammerman, A. (1999) Complexity estimation principle, The Computer Journal, 42:4, 318–322.

    Article  MATH  Google Scholar 

  10. Vovk, V., Watkins, C. J. H. C. (1998) Universal portfolio selection, Proceedings of the 11th Annual Conference on Computational Learning Theory, 12–23.

    Google Scholar 

  11. V’yugin, V. V. (1999) Does snooping help? Technical report No CLRC-TR-99-06, December 1999, Computer Learning Research Centre, Royal Holloway, University of London.

    Google Scholar 

  12. Vyugin, M. V., V’yugin, V. V. (2001) Non-linear inequalities between Kolmogorov and predictive complexities, In Proc. Twelfth International Conference on Algorithmic Learning Theory—ALT’ 01, 190–204.

    Google Scholar 

  13. Zvonkin, A. K., Levin, L. A. (1970) The complexity of finite objects and the algorithmic concepts of information and randomness, Russ. Math. Surv. 25, 83–124.

    Article  MATH  MathSciNet  Google Scholar 

  14. Yamanishi, K. (1995) Randomized approximate aggregating strategies and their applications to prediction and discrimination, in Proceedings, 8th Annual ACM Conference on Computational Learning Theory, 83–90, Assoc. Comput. Mach., New York.

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Royal Holloway, University of London, Egham, Surrey, TW20 0EX, England

    Michael V. Vyugin

  2. Institute for Information Transmission Problems, Russian Academy of Sciences, Bol’shoi Karetnyi per. 19, Moscow GSP-4, 101447, Russia

    Vladimir V. V’yugin

  3. Computer Learning Research Centre, Royal Holloway, University of London, Egham, Surrey, TW20 0EX, England

    Vladimir V. V’yugin

Authors
  1. Michael V. Vyugin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vladimir V. V’yugin
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Research School of Information Sciences and Engineering, Australian National University, Canberra, ACT, 0200, Australia

    Jyrki Kivinen

  2. Computer Science Department, University of Illinois at Chicago, 851 S. Morgan St., Chicago, IL, 60607, USA

    Robert H. Sloan

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Vyugin, M.V., V’yugin, V.V. (2002). Predictive Complexity and Information. In: Kivinen, J., Sloan, R.H. (eds) Computational Learning Theory. COLT 2002. Lecture Notes in Computer Science(), vol 2375. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45435-7_7

Download citation

  • DOI: https://doi.org/10.1007/3-540-45435-7_7

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43836-6

  • Online ISBN: 978-3-540-45435-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation