A note on strategies for bandit problems with infinitely many arms

Abstract.

A bandit problem consisting of a sequence of n choices (n→∞) from a number of infinitely many Bernoulli arms is considered. The parameters of Bernoulli arms are independent and identically distributed random variables from a common distribution F on the interval [0,1] and F is continuous with F(0)=0 and F(1)=1. The goal is to investigate the asymptotic expected failure rates of k-failure strategies, and obtain a lower bound for the expected failure proportion over all strategies presented in Berry et al. (1997). We show that the asymptotic expected failure rates of k-failure strategies when 0<b≤1 and a lower bound can be evaluated if the limit of the ratio F(1)−F(t) versus (1−t)^b exists as t→1⁻ for some b>0.