Abstract
In this paper we study the problem of tracking the mean of a piecewise stationary sequence of independent random variables. First we consider the case where the transition times are known and show that a direct running average performs the tracking in short time and with high accuracy. We then use a single valued weighted running average with a tunable parameter for the case when transition times are unknown and establish deviation bounds for the tracking accuracy. Our result has applications in choosing the optimal rewards for the multiarmed bandit scenario.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availibility
Data sharing not applicable to this article as no datasets were generated or analysed during the current study.
References
N. Alon and J. Spencer. (2008). The Probabilistic Method. Wiley Interscience.
P. Auer, N.C-Bianchi and P. Fischer. (2002). Finite Time Analysis of the Multiarmed Bandit Problem. Machine Learning, 47, 235–256.
P. Auer, P. Gajane and R. Ortner. (2018). Adaptively Tracking the Best Arm with an Unknown Number of Distribution Changes. European Workshop on Reinforcement Learning, 14, 1–8.
O. Besbes, Y. Gur and A. Zeevi. (2014). Stochastic multi-armed-bandit problem with nonstationary rewards. Advances in Neural Information Processing Systems, 27, 199–207.
A. N. Burnetas and M. N. Katehakis. (1996). Optimal Adaptive Policies for Sequential Allocation Problems. Advances in Applied Mathematics, 17, 122–142.
T. Lai and H. Robbins. (1985). Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6, 4–22.
T. Lattimore and C. Szepesvári. (2020). Bandit Algorithms. Cambridge University Press.
A. Slivkins. (2019). Introduction to Multi-armed Bandits. Foundations and Trends in Machine Learning.
Acknowledgements
I thank Professors Rahul Roy, Thomas Mountford, C. R. Subramanian and the referee for crucial comments that led to an improvement of the paper. I also thank IMSc and IISER Bhopal for my fellowships.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article. No funding was received to assist with the preparation of this manuscript.
Additional information
Communicated by Arvind Ayyer.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ganesan, G. Tracking the mean of a piecewise stationary sequence. Indian J Pure Appl Math (2024). https://doi.org/10.1007/s13226-024-00573-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s13226-024-00573-9