Abstract
This work addresses a classic problem of online prediction with expert advice. We assume an adversarial opponent, and we consider both the finite horizon and random stopping versions of this zero-sum, two-person game. Focusing on an appropriate continuum limit and using methods from optimal control, we characterize the value of the game as the viscosity solution of a certain nonlinear partial differential equation. The analysis also reveals the predictor’s and the opponent’s minimax optimal strategies. Our work provides, in particular, a continuum perspective on recent work of Gravin et al. (in: Proceedings of the twenty-seventh annual ACM-SIAM symposium on discrete algorithms, SODA ’16, (Philadelphia, PA, USA), Society for Industrial and Applied Mathematics, 2016). Our techniques are similar to those of Kohn and Serfaty (Commun Pure Appl Math 63(10):1298–1350, 2010), where scaling limits of some two-person games led to elliptic or parabolic PDEs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abernethy, J., Warmuth, M.K., Yellin, J.: Optimal strategies from random walks. In: Proceedings of the 21st Annual Conference on Learning Theory, pp. 437–446. Citeseer (2008)
Antunovic, T., Peres, Y., Sheffield, S., Somersille, S.: Tug-of-war and infinity Laplace equation with vanishing Neumann boundary condition. Commun. Part. Differ. Equ. 37(10), 1839–1869 (2012)
Armstrong, S.N., Smart, C.K.: A finite difference approach to the infinity Laplace equation and tug-of-war games. Trans. Am. Math. Soc. 364(2), 595–636 (2012)
Barles, G., Souganidis, P.E.: Convergence of approximation schemes for fully nonlinear second order equations. Asymptot. Anal. 4(3), 271–283 (1991)
Bayraktar, E., Ekren, I., Zhang, Y.: On the asymptotic optimality of the comb strategy for prediction with expert advice (2019). arXiv e-prints, arXiv:1902.02368
Calder, J.: Lecture notes on viscosity solutions (2018). http://www-users.math.umn.edu/~jwcalder. Accessed 1 July 2019
Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, New York, NY (2006)
Cover, T.M.: Behavior of sequential predictors of binary sequences. Technical report, DTIC Document (1966)
Crandall, M.G., Ishii, H., Lions, P.-L.: User’s guide to viscosity solutions of second order partial differential equations. Bull. Am. Math. Soc. (N.S.) 27(1), 1–67 (1992)
Giga, Y., Goto, S., Ishii, H., Sato, M.-H.: Comparison principle and convexity preserving properties for singular degenerate parabolic equations on unbounded domains. Indiana Univ. Math. J. 40(2), 443–470 (1991)
Gravin, N., Peres, Y., Sivan, B.: Towards optimal algorithms for prediction with expert advice. In: Proceedings of the Twenty-Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’16, (Philadelphia, PA, USA), pp. 528–547. Society for Industrial and Applied Mathematics (2016)
Haussler, D., Kivinen, J., Warmuth, M.: Tight worst-case loss bounds for predicting with expert advice. Technical report, Santa Cruz, CA, USA (1994)
Kohn, R., Serfaty, S.: A deterministic-control-based approach motion by curvature. Commun. Pure Appl. Math. 59(3), 344–407 (2006)
Kohn, R.V., Serfaty, S.: A deterministic-control-based approach to fully nonlinear parabolic and elliptic equations. Commun. Pure Appl. Math. 63(10), 1298–1350 (2010)
Lewicka, M., Manfredi, J.J.: The obstacle problem for the p-laplacian via optimal stopping of tug-of-war games. In: Probability Theory and Related Fields, pp. 1–30 (2015)
Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Inf. Comput. 108, 212–261 (1994)
Luo, H., Schapire, R.E.: Towards minimax online learning with unknown time horizon. In: Proceedings of the 31st International Conference on International Conference on Machine Learning—Volume 32, ICML’14, pp. I–226–I–234. JMLR.org (2014)
Naor, A., Sheffield, S.: Absolutely minimal Lipschitz extension of tree-valued mappings. Math. Ann. 354(3), 1049–1078 (2012)
Oberman, A.M.: Convergent difference schemes for degenerate elliptic and parabolic equations: Hamilton–Jacobi equations and free boundary problems. SIAM J. Numer. Anal. 44(2), 879–895 (2006)
Peres, Y., Sheffield, S.: Tug-of-war with noise: a game-theoretic view of the \(p\)-Laplacian. Duke Math. J. 145, 91–120 (2008)
Peres, Y., Schramm, O., Sheffield, S., Wilson, D.B.: Tug-of-war and the infinity Laplacian. J. Am. Math. Soc. 22(1), 167–210 (2009)
Rakhlin, S., Shamir, O., Sridharan, K.: Relax and randomize: from value to algorithms. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 2141–2149. Curran Associates, Inc., Red Hook (2012)
Vovk, V.G.: Aggregating Strategies. In: Proceedings of the Third Annual Workshop on Computational Learning Theory, COLT ’90, (San Francisco, CA, USA), pp. 371–386. Morgan Kaufmann Publishers Inc (1990)
Zhu, K.: Two problems in applications of PDE. PhD thesis, New York University (2014). http://pqdtopen.proquest.com/pubnum/3635320.html. Accessed 25 Apr 2019
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Michael Ward.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This research was partially supported by NSF Grant DMS-1311833. This work is a refinement of the first author’s Ph.D. thesis, A PDE Approach to a Prediction Problem Involving Randomized Strategies, NYU, 2017.
Rights and permissions
About this article
Cite this article
Drenska, N., Kohn, R.V. Prediction with Expert Advice: A PDE Perspective. J Nonlinear Sci 30, 137–173 (2020). https://doi.org/10.1007/s00332-019-09570-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00332-019-09570-3
Keywords
- Prediction with expert advice
- Regret minimization
- Two-person games
- Dynamic programming
- Viscosity solutions