Extended Formulations for Online Action Selection on Big Action Sets

Abstract

There are big data applications where there is an abundance of latent structure in the data. The online action selection learning algorithms in the literature use an exponential weighting for action selection. However, such strategies are provably suboptimal or computationally inefficient or both. The complexity of addressing such action selection problem is attributed to the combinatorial structure of the action set $\mathscr {A} = n^d$, where n is the number of instances and d is the dimensionality of the problem. Here, we develop an online algorithm for structured big data by adapting striking techniques from discrete optimization and approximation algorithms called ‘extended formulations’. Such formulations appeal to the underlying geometry of the set with efficient exploration of feasible actions with a guaranteed logarithmic dependence on the dimensionality. An empirical evaluation over simulated and real dataset show our method outperforms the state-of-the-art online algorithms over combinatorial action sets.