Density Estimation Using Mixtures of Mixtures of Gaussians

  • Wael Abd-Almageed
  • Larry S. Davis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3954)


In this paper we present a new density estimation algorithm using mixtures of mixtures of Gaussians. The new algorithm overcomes the limitations of the popular Expectation Maximization algorithm. The paper first introduces a new model selection criterion called the Penalty-less Information Criterion, which is based on the Jensen-Shannon divergence. Mean-shift is used to automatically initialize the means and covariances of the Expectation Maximization in order to obtain better structure inference. Finally, a locally linear search is performed using the Penalty-less Information Criterion in order to infer the underlying density of the data. The validity of the algorithm is verified using real color images.


Bayesian Information Criterion Expectation Maximization Segmentation Result Mixture Component Segmented Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dempster, A., Laird, N., Rubin, D.: Maximum likelihood form incomplete data via the em algorithm. Journal of Royal Satistical Society, Series B 39 (1977)Google Scholar
  2. 2.
    Belongie, S., Carson, C., Greenspan, H., Malik, J.: Color- and texture-based image segmentation using em and its applications to content-based image retreval. In: Sixth International Conference on Computer Vision, pp. 675–682 (1998)Google Scholar
  3. 3.
    Abu-Naser, A., Galatsanos, N., Wernick, M., Schonfeld, D.: Object recognition based on impulse restoration with use of the expectation-maximization algorithm. Journal of the Optical Society of America A (Optics, Image Science and Vision) 15, 2327–2340 (1998)CrossRefGoogle Scholar
  4. 4.
    Figueirdeo, M., Jain, A.: Unsupervised Learning of Finite Mixture Models. IEEE Trans. on Pattern Analysis and Machine Intelligence 24, 381–396 (2002)CrossRefGoogle Scholar
  5. 5.
    Dasgupta, A., Rafetry, A.: Detecting Features ni Spatial Point Patterns with Clutter Via Model-Based Clustering. J. of American Statistical Association, 294–302 (1998)Google Scholar
  6. 6.
    Campbell, J., Fraley, C., Murtagh, F., Raftery, A.: Linear Flaw Detection in Woven Textiles Using Model-Based clustering. Pattern Recognition Letters 18, 1539–1548 (1997)CrossRefGoogle Scholar
  7. 7.
    Neal, R.: Bayesian Mixture Modeling. In: Proc. of 11th Int’l Workshop on Maximum Entropy and Bayesian Methods of Statistical Analysis, pp. 197–211 (1992)Google Scholar
  8. 8.
    Bensmail, H., Celeus, G., Rafetry, A., Robert, C.: Inference in Model-Based Cluster Analysis. Statistics and Computing 7, 1–10 (1997)CrossRefGoogle Scholar
  9. 9.
    Comaniciu, D., Meer, P.: Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE Trans. Pattern Analysis and Machine Intelligence 24 (2002)Google Scholar
  10. 10.
    Fukunaga, K., Hostetler, L.D.: The estimation of a gradient of a density function, with applications in pattern recognition. IEEE Trans. on Information Theory 21, 32–40 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Yang, C., Duraiswami, R., Gumerov, N., Davis, L.: Improved Fast Gauss Transform and Efficient Kernel Density Estimation. In: Proc. IEEE International Conference on Computer Vision (2003)Google Scholar
  12. 12.
    Wand, M., Jones, M.: Kernel Smoothing. Chapman and Hall, Boca Raton (1995)CrossRefzbMATHGoogle Scholar
  13. 13.
    Lin, J.: Divergence measuers based on the shannon entropy. IEEE Trans. Information Theory 37, 145–151 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Han, H., Comaniciu, D., Zhu, Y., Davis, L.: Incremental density approximation and kernel-based baesian filtering for object tracking. In: IEEE International Conference on Computer Vision and Pattern Recognition (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Wael Abd-Almageed
    • 1
  • Larry S. Davis
    • 1
  1. 1.Institute for Advanced Computer StudiesUniversity of MarylandCollege ParkUSA

Personalised recommendations