On-Line Estimation with the Multivariate Gaussian Distribution
We consider on-line density estimation with the multivariate Gaussian distribution. In each of a sequence of trials, the learner must posit a mean μ and covariance Σ; the learner then receives an instance x and incurs loss equal to the negative log-likelihood of x under the Gaussian density parameterized by (μ,Σ). We prove bounds on the regret for the follow-the-leader strategy, which amounts to choosing the sample mean and covariance of the previously seen data.
KeywordsExponential Family Multivariate Gaussian Distribution Gaussian Density Minimax Regret Computational Learn Theory
Unable to display preview. Download preview PDF.
- Crammer, K.: Online tracking of linear subspaces. 19th Annual Conference on Learning Theory (2006)Google Scholar
- Freund, Y.: Predicting a binary sequence almost as well as the optimal biased coin. 9th Annual Conference on Computational Learning Theory (1996)Google Scholar
- Hannan, J.: Approximation to Bayes risk in repeated play. In: M. Dresher, A. Tucker, P. Wolfe (Eds.), Contributions to the Theory of Games, vol. III, pp. 97–139 (1957)Google Scholar
- Hazan, E., Kalai, A., Kale, S., Agarwal, A.: Logarithmic regret algorithms for online convex optimization. 19th Annual Conference on Learning Theory (2006)Google Scholar
- Kalai, A., Vempala, S.: Efficient algorithms for the online decision problem. 16th Annual Conference on Learning Theory (2005)Google Scholar
- Shalev-Shwartz, S., Singer, Y.: Convex repeated games and Fenchel duality. Advances in Neural Information Processing Systems 19 (2006)Google Scholar
- Takimoto, E., Warmuth, M.: The last-step minimax algorithm. 11th International Conference on Algorithmic Learning Theory (2000a)Google Scholar
- Takimoto, E., Warmuth, M.: The minimax strategy for Gaussian density estimation. 13th Annual Conference on Computational Learning Theory (2000b)Google Scholar
- Warmuth, M., Kuzmin, D.: Randomized PCA algorithms with regret bounds that are logarithmic in the dimension. Advances in Neural Information Processing Systems 19 (2006)Google Scholar
- Zinkevich, M.: Online convex programming and generalized infinitesimal gradient ascent. In: 20th International Conference on Machine Learning (2003)Google Scholar