Toward a Classification of Finite Partial-Monitoring Games

  • Gábor Bartók
  • Dávid Pál
  • Csaba Szepesvári
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6331)

Abstract

In a finite partial-monitoring game against Nature, the Learner repeatedly chooses one of finitely many actions, the Nature responds with one of finitely many outcomes, the Learner suffers a loss and receives feedback signal, both of which are fixed functions of the action and the outcome. The goal of the Learner is to minimize its total cumulative loss. We make progress towards classification of these games based on their minimax expected regret. Namely, we classify almost all games with two outcomes: We show that their minimax expected regret is either zero, \(\widetilde{\Theta}(\sqrt{T})\), Θ(T2/3), or Θ(T) and we give a simple and efficiently computable classification of these four classes of games. Our hope is that the result can serve as a stepping stone toward classifying all finite partial-monitoring games.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Lugosi, G., Cesa-Bianchi, N.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)MATHGoogle Scholar
  2. 2.
    Audibert, J.-Y., Bubeck, S.: Minimax policies for adversarial and stochastic bandits. In: Proceedings of the 22nd Annual Conference on Learning Theory (2009)Google Scholar
  3. 3.
    Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM Journal on Computing 32(1), 48–77 (2003)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Cesa-Bianchi, N., Lugosi, G., Stoltz, G.: Regret minimization under partial monitoring. Mathematics of Operations Research 31(3), 562–580 (2006)MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM 44(3), 427–485 (1997)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Piccolboni, A., Schindelhauer, C.: Discrete prediction games with arbitrary feedback and loss. In: Helmbold, D.P., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 208–223. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  7. 7.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Wiley, New York (2006)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Gábor Bartók
    • 1
  • Dávid Pál
    • 1
  • Csaba Szepesvári
    • 1
  1. 1.Department of Computing ScienceUniversity of AlbertaCanada

Personalised recommendations