Advertisement

A new hybrid discriminative/generative model using the full-covariance multivariate generalized Gaussian mixture models

  • Fatma NajarEmail author
  • Sami Bourouis
  • Nizar Bouguila
  • Safya Belghith
Methodologies and Application
First Online:
  • 35 Downloads

Abstract

Discriminative models have been shown to be more advantageous for pattern recognition problem in machine learning. For this study, the main focus is developing a new hybrid model that combines the advantages of a discriminative technique namely the support vector machines (SVM) with the full efficiency offered through covariance multivariate generalized Gaussian mixture models (MGGMM). This new hybrid MGGMM applies the Fisher and Kullback–Leibler kernels derived from MGGMM to improve the kernel function of SVM. This approach is based on two different learning techniques explicitly: the Fisher scoring algorithm and the Bayes inference technique based on Markov Chain Monte Carlo and Metropolis–Hastings algorithm. These learning methods work with two model selection approaches (minimum message length and marginal likelihood) to determine the number of clusters. The effectiveness of the framework is demonstrated through extensive experiments including synthetic datasets, facial expression recognition and human activity recognition.

Keywords

Multivariate generalized Gaussian mixture Support vector machines kernels Fisher scoring algorithm Bayesian learning Facial expression recognition Human activity recognition 
This is a preview of subscription content, log in to check access.

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

References

  1. Adama DA, Lotfi A, Langensiepen C, Lee K, Trindade P (2018) Human activity learning for assistive robotics using a classifier ensemble. Soft Comput 22(21):7027–7039CrossRefGoogle Scholar
  2. Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19(6):716–723MathSciNetzbMATHCrossRefGoogle Scholar
  3. Bartlett MS, Littlewort G, Fasel I, Movellan JR (2003) Real time face detection and facial expression recognition: development and applications to human computer interaction. In: Conference on computer vision and pattern recognition workshop, 2003. CVPRW’03, vol. 5. IEEE, pp 53–53Google Scholar
  4. Baxter RA, Oliver JJ (2000) Finding overlapping components with mml. Stat Comput 10(1):5–16CrossRefGoogle Scholar
  5. Bouguila N (2011) Bayesian hybrid generative discriminative learning based on finite liouville mixture models. Pattern Recognit 44(6):1183–1200zbMATHCrossRefGoogle Scholar
  6. Bouguila N (2012) Hybrid generative/discriminative approaches for proportional data modeling and classification. IEEE Trans Knowl Data Eng 24(12):2184–2202CrossRefGoogle Scholar
  7. Bouguila N, Ziou D (2007) High-dimensional unsupervised selection and estimation of a finite generalized Dirichlet mixture model based on minimum message length. IEEE Trans Pattern Anal Mach Intell 29(10):1716–1731CrossRefGoogle Scholar
  8. Boukouvalas Z, Fu GS, Adalı T (2015) An efficient multivariate generalized gaussian distribution estimator: Application to iva. In: 49th Annual conference on information sciences and systems (CISS), 2015. IEEE, pp 1–4Google Scholar
  9. Bourouis S, Al-Osaimi FR, Bouguila N, Sallay H, Aldosari F, Al Mashrgy M (2019) Bayesian inference by reversible jump mcmc for clustering based on finite generalized inverted dirichlet mixtures. Soft Comput 23(14):5799–5813CrossRefGoogle Scholar
  10. Carlo M (1992) Comment: one long run with diagnostics: implementation strategies for Markov chain. Stat Sci 7(4):493–497CrossRefGoogle Scholar
  11. Chib S, Greenberg E (1995) Understanding the Metropolis–Hastings algorithm. Am Stat 49(4):327–335Google Scholar
  12. Cohen I, Sebe N, Cozman FG, Huang TS (2003) Semi-supervised learning for facial expression recognition. In: Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval. ACM, pp 17–22Google Scholar
  13. Conway JH, Sloane NJA (2013) Sphere packings, lattices and groups, vol 290. Springer, BerlinGoogle Scholar
  14. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the em algorithm. J R Stat Soc Ser B (Methodol) 39(1):1–22MathSciNetzbMATHGoogle Scholar
  15. Dollár P, Rabaud V, Cottrell G, Belongie S (2005) Behavior recognition via sparse spatio-temporal features. VS-PETS, BeijingCrossRefGoogle Scholar
  16. Elguebaly T, Bouguila N (2015) A hierarchical nonparametric Bayesian approach for medical images and gene expressions classification. Soft Comput 19(1):189–204CrossRefGoogle Scholar
  17. Fan W, Bouguila N (2013) Online facial expression recognition based on finite beta-liouville mixture models. In: 2013 International conference on computer and robot vision (CRV). IEEE, pp 37–44Google Scholar
  18. Fan W, Bouguila N (2014) Variational learning for Dirichlet process mixtures of Dirichlet distributions and applications. Multimed Tools Appl 70(3):1685–1702CrossRefGoogle Scholar
  19. Fan W, Sallay H, Bouguila N, Bourouis S (2016) Variational learning of hierarchical infinite generalized Dirichlet mixture models and applications. Soft Comput 20(3):979–990CrossRefGoogle Scholar
  20. Figueiredo MAT, Jain AK (2002) Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell 24(3):381–396CrossRefGoogle Scholar
  21. Gelman A, Stern HS, Carlin JB, Dunson DB, Vehtari A, Rubin DB (2013) Bayesian data analysis. Chapman and Hall/CRC, Boca RatonzbMATHGoogle Scholar
  22. Hershey JR, Olsen PA (2007) Approximating the kullback leibler divergence between gaussian mixture models. In: IEEE international conference on acoustics, speech and signal processing, 2007. ICASSP 2007. vol. 4. IEEE, pp IV–317Google Scholar
  23. Jaakkola T, Haussler D (1999) Exploiting generative models in discriminative classifiers. In: Advances in neural information processing systems, pp 487–493Google Scholar
  24. Kanade T, Tian Y, Cohn JF (2000) Comprehensive database for facial expression analysis. In: Proceedings fourth IEEE international conference on automatic face and gesture recognition. IEEE, p 46Google Scholar
  25. Kass RE, Raftery AE (1995) Bayes factors. J Am Stat Assoc 90(430):773–795 MathSciNetzbMATHCrossRefGoogle Scholar
  26. Kelker D (1970) Distribution theory of spherical distributions and a location-scale parameter generalization. Sankhyā Indian J Stat Ser A 32:419–430MathSciNetzbMATHGoogle Scholar
  27. Kotz S (1975) Multivariate distributions at a cross-road. Stat Distrib Sci Work 1:247–270Google Scholar
  28. Lajevardi SM, Hussain ZM (2009) Zernike moments for facial expression recognition. rn 2, 3Google Scholar
  29. Lindley DV and Rao CR (1953) Advanced statistical methods in biometric research. J R Stat Soc 116(1):86–87 MathSciNetCrossRefGoogle Scholar
  30. Marin JM, Robert C (2007) Bayesian core: a practical approach to computational Bayesian statistics. Springer, BerlinzbMATHGoogle Scholar
  31. Moreno PJ, Ho PP, Vasconcelos N (2004) A Kullback–Leibler divergence based kernel for svm classification in multimedia applications. In: Advances in neural information processing systems, pp 1385–1392Google Scholar
  32. Najar F, Bourouis S, Bouguila N, Belghith S (2018) A fixed-point estimation algorithm for learning the multivariate ggmm: application to human action recognition. In: 2018 IEEE Canadian conference on electrical & computer engineering (CCECE). IEEE, pp 1–4Google Scholar
  33. Najar F, Bourouis S, Bouguila N, Belghith S (2019) Unsupervised learning of finite full covariance multivariate generalized Gaussian mixture models for human activity recognition. Multimed Tools Appl 78:1–23CrossRefGoogle Scholar
  34. Neal RM (1992) Bayesian mixture modeling. In: Maximum entropy and Bayesian methods. Springer, pp. 197–211Google Scholar
  35. Niebles JC, Wang H, Fei-Fei L (2008) Unsupervised learning of human action categories using spatial-temporal words. Int J Comput Vis 79(3):299–318CrossRefGoogle Scholar
  36. Pascal F, Bombrun L, Tourneret JY, Berthoumieu Y (2013) Parameter estimation for multivariate generalized gaussian distributions. IEEE Trans Signal Process 61(23):5960–5971MathSciNetzbMATHCrossRefGoogle Scholar
  37. Rissanen J (1978) Modeling by shortest data description. Automatica 14(5):465–471zbMATHCrossRefGoogle Scholar
  38. Robert C (2007) The Bayesian choice: from decision-theoretic foundations to computational implementation. Springer, BerlinzbMATHGoogle Scholar
  39. Robert C, Casella G (2000) Monte carlo statistical methods. Springer Text in Statistics, Springer.  https://doi.org/10.1007/978-1-4757-4145-2 zbMATHCrossRefGoogle Scholar
  40. Roberts GO, Tweedie RL (1999) Bounds on regeneration times and convergence rates for Markov chains. Stoch Process Appl 80(2):211–229MathSciNetzbMATHCrossRefGoogle Scholar
  41. Roh SB, Oh SK, Yoon JH, Seo K (2018) Design of face recognition system based on fuzzy transform and radial basis function neural networks. Soft Comput 23:1–17Google Scholar
  42. Schuldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: Proceedings of the 17th international conference on pattern recognition, 2004. ICPR 2004. vol. 3. IEEE, pp 32–36Google Scholar
  43. Tsai HH, Chang YC (2018) Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Comput 22(13):4389–4405CrossRefGoogle Scholar
  44. Verdoolaege G, Rosseel Y, Lambrechts M, Scheunders P (2009) Wavelet-based colour texture retrieval using the Kullback–Leibler divergence between bivariate generalized Gaussian models. In: 2009 16th IEEE international conference on image processing (ICIP). IEEE, pp 265–268Google Scholar
  45. Verdoolaege G, Scheunders P (2012) On the geometry of multivariate generalized Gaussian models. J Math Imaging Vis 43(3):180–193MathSciNetzbMATHCrossRefGoogle Scholar
  46. Vrigkas M, Nikou C, Kakadiaris IA (2015) A review of human activity recognition methods. Front Robot AI 2:28CrossRefGoogle Scholar
  47. Wallace CS, Boulton DM (1968) An information measure for classification. Comput J 11(2):185–194zbMATHCrossRefGoogle Scholar
  48. Wong SF, Cipolla R (2007) Extracting spatiotemporal interest points using global information. In: 2007 IEEE 11th international conference on computer vision. Citeseer, pp 1–8Google Scholar
  49. Yeasin M, Bullot B, Sharma R (2004) From facial expression to level of interest: a spatio-temporal approach. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004. vol. 2. IEEE, pp II–IIGoogle Scholar
  50. Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 29(6):915–928CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.ENIT, Laboratoire RISC Robotique Informatique et Systèmes ComplexesUniversité de Tunis El ManarTunisTunisia
  2. 2.ENIT, LR-SITI Laboratoire Signal Image et Technologies de l’InformationUniversité de Tunis El ManarTunisTunisia
  3. 3.College of Computers and Information TechnologyTaif UniversityAţ Ţā’ifSaudi Arabia
  4. 4.The Concordia Institute for Information Systems Engineering (CIISE)Concordia UniversityMontrealCanada

Personalised recommendations

Cite article

Buy options