Abstract
We consider a problem intimately related to the creation of maxima under Gaussian blurring: the number of modes of a Gaussian mixture in D dimensions. To our knowledge, a general answer to this question is not known. We conjecture that if the components of the mixture have the same covariance matrix (or the same covariance matrix up to a scaling factor), then the number of modes cannot exceed the number of components. We demonstrate that the number of modes can exceed the number of components when the components are allowed to have arbitrary and different covariance matrices.
We will review related results from scale-space theory, statistics and machine learning, including a proof of the conjecture in 1D. We present a convergent, EM-like algorithm for mode finding and compare results of searching for all modes starting from the centers of the mixture components with a brute-force search. We also discuss applications to data reconstruction and clustering.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Carreira-Perpiñán, M.Á., Williams, C.K.I.: On the number of modes of a Gaussian mixture. Technical Report EDI-INF-RR-0159, School of Informatics, University of Edinburgh, UK (2003). Available online at http://www.informatics.ed.ac.uk/publications/report/0159.html.
Carreira-Perpiñán, M.Á.: Continuous Latent Variable Models for Dimensionality Reduction and Sequential Data Reconstruction. PhD thesis, Dept. of Computer Science, University of Sheffield, UK (2001)
Carreira-Perpiñán, M.Á.: Mode-finding for mixtures of Gaussian distributions. Technical Report CS-99-03, Dept. of Computer Science, University of Sheffield, UK (1999), revised August 4, 2000. Available online at http://www.dcs.shef.ac.uk/~miguel/papers/cs-99-03.html.
Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Computation 14 (2002) 1771–1800
Lindeberg, T.: Scale-Space Theory in Computer Vision. Kluwer Academic Publishers Group, Dordrecht, The Netherlands (1994)
Koenderink, J.J.: The structure of images. Biol. Cybern. 50 (1984) 363–370
Babaud, J., Witkin, A.P., Baudin, M., Duda, R.O.: Uniqueness of the Gaussian kernel for scale-space filtering. IEEE Trans. on Pattern Anal. and Machine Intel. 8 (1986) 26–33
Yuille, A.L., Poggio, T.A.: Scaling theorems for zero crossings. IEEE Trans. on Pattern Anal. and Machine Intel. 8 (1986) 15–25
Roberts, S.J.: Parametric and non-parametric unsupervised cluster analysis. Pattern Recognition 30 (1997) 261–272
Lifshitz, L.M., Pizer, S.M.: A multiresolution hierarchical approach to image segmentation based on intensity extrema. IEEE Trans. on Pattern Anal. and Machine Intel. 12 (1990) 529–540
Kuijper, A., Florack, L.M.J.: The application of catastrophe theory to image analysis. Technical Report UU-CS-2001-23, Dept. of Computer Science, Utrecht University (2001). Available online at ftp://ftp.cs.uu.nl/pub/RUU/CS/techreps/CS-2001/2001-23.pdf.
Kuijper, A., Florack, L.M.J.: The relevance of non-generic events in scale space models. In Heyden, A., Sparr, G., Nielsen, M., Johansen, P., eds.: Proc. 7th European Conf. Computer Vision (ECCV’02), Copenhagen, Denmark (2002)
Damon, J.: Local Morse theory for solutions to the heat equation and Gaussian blurring. J. Diff. Equations 115 (1995) 368–401
Silverman, B.W.: Using kernel density estimates to investigate multimodality. Journal of the Royal Statistical Society, B 43 (1981) 97–99
Carreira-Perpiñán, M.Á.: Mode-finding for mixtures of Gaussian distributions. IEEE Trans. on Pattern Anal. and Machine Intel. 22 (2000) 1318–1323
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, B 39 (1977) 1–38
McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extensions. Wiley Series in Probability and Mathematical Statistics. John Wiley & Sons (1997)
Redner, R.A., Walker, H.F.: Mixture densities, maximum likelihood and the EM algorithm. SIAM Review 26 (1984) 195–239
Rose, K.: Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proc. IEEE 86 (1998) 2210–2239
Schölkopf, B., Mika, S., Burges, C.J.C., Knirsch, P., Müller, K.R., Rätsch, G., Smola, A.: Input space vs. feature space in kernel-based methods. IEEE Trans. Neural Networks 10 (1999) 1000–1017
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Computation 3 (1991) 79–87
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, New York, Oxford (1995)
Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Number 26 in Monographs on Statistics and Applied Probability. Chapman & Hall, London, New York (1986)
Carreira-Perpiñán, M.Á.: Reconstruction of sequential data with probabilistic models and continuity constraints. In Solla, S.A., Leen, T.K., Müller, K.R., eds.: Advances in Neural Information Processing Systems. Volume 12, MIT Press, Cambridge, MA (2000) 414–420
Bishop, C.M., Svensén, M., Williams, C.K.I.: GTM: The generative topographic mapping. Neural Computation 10 (1998) 215–234
Fukunaga, K., Hostetler, L.D.: The estimation of the gradient of a density function, with application in pattern recognition. IEEE Trans. Inf. Theory IT-21 (1975) 32–40
Cheng, Y.: Mean shift, mode seeking, and clustering. IEEE Trans. on Pattern Anal. and Machine Intel. 17 (1995) 790–799
Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. IEEE Trans. on Pattern Anal. and Machine Intel. 24 (2002) 603–619
Wong, Y.: Clustering data by melting. Neural Computation 5 (1993) 89–104
Chakravarthy, S.V., Ghosh, J.: Scale-based clustering using the radial basis function network. IEEE Trans. Neural Networks 7 (1996) 1250–1261
Leung, Y., Zhang, J.S., Xu, Z.B.: Clustering by scale-space filtering. IEEE Trans. on Pattern Anal. and Machine Intel. 22 (2000) 1396–1410
Minnotte, M.C., Scott, D.W.: The mode tree: A tool for visualization of nonparametric density features. Journal of Computational and Graphical Statistics 2 (1993) 51–68
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Carreira-Perpiñán, M.Á., Williams, C.K.I. (2003). On the Number of Modes of a Gaussian Mixture. In: Griffin, L.D., Lillholm, M. (eds) Scale Space Methods in Computer Vision. Scale-Space 2003. Lecture Notes in Computer Science, vol 2695. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44935-3_44
Download citation
DOI: https://doi.org/10.1007/3-540-44935-3_44
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40368-5
Online ISBN: 978-3-540-44935-5
eBook Packages: Springer Book Archive