Estimation of Single-Gaussian and Gaussian Mixture Models for Pattern Recognition

  • Jan Vaněk
  • Lukáš Machlica
  • Josef Psutka
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8258)


Single-Gaussian and Gaussian-Mixture Models are utilized in various pattern recognition tasks. The model parameters are estimated usually via Maximum Likelihood Estimation (MLE) with respect to available training data. However, if only small amount of training data is available, the resulting model will not generalize well. Loosely speaking, classification performance given an unseen test set may be poor. In this paper, we propose a novel estimation technique of the model variances. Once the variances were estimated using MLE, they are multiplied by a scaling factor, which reflects the amount of uncertainty present in the limited sample set. The optimal value of the scaling factor is based on the Kullback-Leibler criterion and on the assumption that the training and test sets are sampled from the same source distribution. In addition, in the case of GMM, the proper number of components can be determined.


Maximum Likelihood Estimation Gaussian Mixture Model Kullback-Leibler Divergence Variance Scaling 


  1. [1]
    Wu, X., Kumar, V., Quinlan, J.R., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., et al.: Top 10 Algorithms in Data Mining. In: Knowledge and Information Systems, pp. 1–37 (2007)Google Scholar
  2. [2]
    Kullback, S., Leibler, R.A.: On Information and Sufficiency. Annals of Mathematical Statistics 22, 79–86 (1951)MathSciNetCrossRefzbMATHGoogle Scholar
  3. [3]
    Bell, P.: Full Covariance Modelling for Speech Recognition. Ph.D. Thesis, The University of Edinburgh (2010)Google Scholar
  4. [4]
    Figueiredo, M., Leitão, J., Jain, A.: On Fitting Mixture Models. In: Hancock, E.R., Pelillo, M. (eds.) EMMCVPR 1999. LNCS, vol. 1654, pp. 54–69. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  5. [5]
    Paclík, P., Novovičová, J.: Number of Components and Initialization in Gaussian Mixture Model for Pattern Recognition. In: Proc. Artificial Neural Nets and Genetic Algorithms, pp. 406–409. Springer, Wien (2001)CrossRefGoogle Scholar
  6. [6]
    Taboga, M.: Lectures on Probability Theory and Mathematical Statistics. CreateSpace Independent Publishing Platform (2008) ISBN: 978-1480215238Google Scholar
  7. [7]
    Bishop, C.M.: Pattern Recognition and Machine Learning, 1st edn. Springer (2007) ISBN: 978-0387310732Google Scholar
  8. [8]
    Vanek, J., Machlica, L., Psutka, J.V., Psutka, J.: Covariance Matrix Enhancement Approach to Train Robust Gaussian Mixture Models of Speech Data. In: SPECOM (2013)Google Scholar
  9. [9]
    Machlica, L., Vanek, J., Zajic, Z.: Fast Estimation of Gaussian Mixture Model Parameters on GPU using CUDA. In: Proc. PDCAT, Gwangju, South Korea (2011)Google Scholar
  10. [10]
    Vanek, J., Trmal, J., Psutka, J.V., Psutka, J.: Optimized Acoustic Likelihoods Computation for NVIDIA and ATI/AMD Graphics Processors. IEEE Transactions on Audio, Speech and Language Processing 20(6), 1818–1828 (2012)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Jan Vaněk
    • 1
  • Lukáš Machlica
    • 1
  • Josef Psutka
    • 1
  1. 1.Faculty of Applied Sciences, Department of CyberneticsUniversity of West Bohemia in PilsenPilsenCzech Republic

Personalised recommendations