Abstract
Consider the problem of estimating the centers 
of a uniform mixture of unit-variance spherical Gaussians in 
,
from i.i.d. samples x
1,...,x
m
drawn from this distribution. This can be done by maximizing the (average log) likelihood 
. Maximizing the likelihood is guaranteed to recover the correct centers, in the large-sample limit, for any mixture model of the form (1). Unfortunately, maximizing the likelihood is hard in the worst case, and we usually revert to local search heuristics such as Expectation Maximization (EM) which can get trapped in the many local minima the likelihood function might have.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dasgupta, S.: Learning mixtures of gaussians. In: Proc. of the 40th Ann. Symp. on Foundations of Computer Science (1999)
Arora, S., Kannan, R.: Learning mixtures of arbitrary gaussians. In: Proceedings of the thirty-third annual ACM symposium on Theory of computing (2001)
Vempala, S., Wang, G.: A spectral algorithm for learning mixture models. J. Comput. Syst. Sci. 68, 841–860 (2004)
Dasgupta, S., Schulman, L.: A two-round variant of em for gaussian mixtures. In: Proc. of the 16th Ann. Conf. on Uncertainty in Artificial Intelligence (2000)
Srebro, N., Shakhnarovich, G., Roweis, S.: An investigation of computational and informational limits in gaussian mixture clustering. In: Proceedings of the 23rd International Conference on Machine learning (ICML) (2006)
Ostrovsky, R., Rabani, Y., Schulman, L.J., Swamy, C.: The effectiveness of lloyd-type methods for the k-means problem. In: Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS’06), pp. 165–176 ( 2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Srebro, N. (2007). Are There Local Maxima in the Infinite-Sample Likelihood of Gaussian Mixture Estimation?. In: Bshouty, N.H., Gentile, C. (eds) Learning Theory. COLT 2007. Lecture Notes in Computer Science(), vol 4539. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72927-3_47
Download citation
DOI: https://doi.org/10.1007/978-3-540-72927-3_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72925-9
Online ISBN: 978-3-540-72927-3
eBook Packages: Computer ScienceComputer Science (R0)