Advertisement

On-Line Estimation with the Multivariate Gaussian Distribution

  • Sanjoy Dasgupta
  • Daniel Hsu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4539)

Abstract

We consider on-line density estimation with the multivariate Gaussian distribution. In each of a sequence of trials, the learner must posit a mean μ and covariance Σ; the learner then receives an instance x and incurs loss equal to the negative log-likelihood of x under the Gaussian density parameterized by (μ,Σ). We prove bounds on the regret for the follow-the-leader strategy, which amounts to choosing the sample mean and covariance of the previously seen data.

Keywords

Exponential Family Multivariate Gaussian Distribution Gaussian Density Minimax Regret Computational Learn Theory 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Azoury, K., Warmuth, M.: Relative loss bounds for on-line density estimation with the exponential family of distributions. Journal of Machine Learning 43(3), 211–246 (2001)zbMATHCrossRefGoogle Scholar
  2. Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)zbMATHGoogle Scholar
  3. Crammer, K.: Online tracking of linear subspaces. 19th Annual Conference on Learning Theory (2006)Google Scholar
  4. Freund, Y.: Predicting a binary sequence almost as well as the optimal biased coin. 9th Annual Conference on Computational Learning Theory (1996)Google Scholar
  5. Hannan, J.: Approximation to Bayes risk in repeated play. In: M. Dresher, A. Tucker, P. Wolfe (Eds.), Contributions to the Theory of Games, vol. III, pp. 97–139 (1957)Google Scholar
  6. Hazan, E., Kalai, A., Kale, S., Agarwal, A.: Logarithmic regret algorithms for online convex optimization. 19th Annual Conference on Learning Theory (2006)Google Scholar
  7. Kalai, A., Vempala, S.: Efficient algorithms for the online decision problem. 16th Annual Conference on Learning Theory (2005)Google Scholar
  8. Shalev-Shwartz, S., Singer, Y.: Convex repeated games and Fenchel duality. Advances in Neural Information Processing Systems 19 (2006)Google Scholar
  9. Takimoto, E., Warmuth, M.: The last-step minimax algorithm. 11th International Conference on Algorithmic Learning Theory (2000a)Google Scholar
  10. Takimoto, E., Warmuth, M.: The minimax strategy for Gaussian density estimation. 13th Annual Conference on Computational Learning Theory (2000b)Google Scholar
  11. Warmuth, M., Kuzmin, D.: Randomized PCA algorithms with regret bounds that are logarithmic in the dimension. Advances in Neural Information Processing Systems 19 (2006)Google Scholar
  12. Zinkevich, M.: Online convex programming and generalized infinitesimal gradient ascent. In: 20th International Conference on Machine Learning (2003)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Sanjoy Dasgupta
    • 1
  • Daniel Hsu
    • 1
  1. 1.University of California, San Diego 

Personalised recommendations