Generalised Entropy and Asymptotic Complexities of Languages

  • Yuri Kalnishkan
  • Vladimir Vovk
  • Michael V. Vyugin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4539)

Abstract

In this paper the concept of asymptotic complexity of languages is introduced. This concept formalises the notion of learnability in a particular environment and generalises Lutz and Fortnow’s concepts of predictability and dimension. Then asymptotic complexities in different prediction environments are compared by describing the set of all pairs of asymptotic complexities w.r.t. different environments. A geometric characterisation in terms of generalised entropies is obtained and thus the results of Lutz and Fortnow are generalised.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)MATHGoogle Scholar
  2. Vovk, V., Watkins, C.J.H.C.: Universal portfolio selection. In: Proceedings of the 11th Annual Conference on Computational Learning Theory, pp. 12–23. ACM Press, New York (1998)Google Scholar
  3. Fortnow, L., Lutz, J.H.: Prediction and dimension. Journal of Computer and System Sciences 70(4), 570–589 (2005)MATHCrossRefMathSciNetGoogle Scholar
  4. Grünwald, P.D., Dawid, A.P.: Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory. The Annals of Statistics 32(4), 1367–1433 (2004)MATHCrossRefMathSciNetGoogle Scholar
  5. Kalnishkan, Y., Vovk, V., Vyugin, M.V.: Loss functions, complexities, and the Legendre transformation. Theoretical Computer Science 313(2), 195–207 (2004)MATHCrossRefMathSciNetGoogle Scholar
  6. Kalnishkan, Y., Vyugin, M.V.: The weak aggregating algorithm and weak mixability. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS (LNAI), vol. 3559, pp. 188–203. Springer, Heidelberg (2005)Google Scholar
  7. Eggleston, H.G.: Convexity. Cambridge University Press, Cambridge (1958)MATHGoogle Scholar
  8. Hoeffding, W.: Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58(301), 13–30 (1963)MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Yuri Kalnishkan
    • 1
  • Vladimir Vovk
    • 1
  • Michael V. Vyugin
    • 1
  1. 1.Department of Computer Science, Royal Holloway, University of London, Egham, Surrey, TW20 0EXUK

Personalised recommendations