Advertisement

A nonparametric Bayesian learning model using accelerated variational inference and feature selection

  • Wentao Fan
  • Nizar Bouguila
  • Xin Liu
Theoretical advances
  • 21 Downloads

Abstract

Developing effective machine learning methods for multimedia data modeling continues to challenge computer vision scientists. The capability of providing effective learning models can have significant impact on various applications. In this work, we propose a nonparametric Bayesian approach to address simultaneously two fundamental problems, namely clustering and feature selection. The approach is based on infinite generalized Dirichlet (GD) mixture models constructed through the framework of Dirichlet process and learned using an accelerated variational algorithm that we have developed. Furthermore, we extend the proposed approach using another nonparametric Bayesian prior, namely Pitman–Yor process, to construct the infinite generalized Dirichlet mixture model. Our experiments, which were conducted through synthetic data sets, the clustering analysis of real-world data sets and a challenging application, namely automatic human action recognition, indicate that the proposed framework provides good modeling and generalization capabilities.

Keywords

Infinite mixtures Variational Bayes Generalized Dirichlet Feature selection Human action recognition 

Notes

Acknowledgements

Funding was provided by National Natural Science Foundation of China (Grant No. 61502183), The Scientific Research Funds of Huaqiao University (Grant No. 600005-Z15Y0016) and Promotion Program for Young and Middle-aged Teacher in Science and Technology Research of Huaqiao University (Grant No. ZQN-PY510).

References

  1. 1.
    Alfò M, Nieddu L, Vicari D (2008) A finite mixture model for image segmentation. Stat Comput 18(2):137–150MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bentley JL (1975) Multidimensional binary search trees used for associative searching. Commun ACM 18(9):509–517CrossRefzbMATHGoogle Scholar
  3. 3.
    Blei D, Jordan M (2005) Variational inference for Dirichlet process mixtures. Bayesian Anal 1:121–144MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Bouguila N (2007) Spatial color image databases summarization. In: Proc. of the IEEE international conference on acoustics, speech and signal processing (ICASSP 2007), vol 1, pp I-953–I-956Google Scholar
  5. 5.
    Bouguila N, Ziou D (2004a) Improving content based image retrieval systems using finite multinomial Dirichlet mixture. In: Proc. of the 14th IEEE signal processing society workshop on machine learning for signal processing, pp 23–32Google Scholar
  6. 6.
    Bouguila N, Ziou D (2004b) A powerful finite mixture model based on the generalized Dirichlet distribution: unsupervised learning and applications. In: Proc. of the 17th international conference on pattern recognition (ICPR 2004), vol 1, pp 280–283 Vol 1Google Scholar
  7. 7.
    Bouguila N, Ziou D (2010) A Dirichlet process mixture of generalized Dirichlet distributions for proportional data modeling. IEEE Trans Neural Netw 21(1):107–122CrossRefGoogle Scholar
  8. 8.
    Boutemedjet S, Bouguila N, Ziou D (2009) A hybrid feature extraction selection approach for high-dimensional non-Gaussian data clustering. IEEE Trans Pattern Anal Mach Intell 31(8):1429–1443CrossRefGoogle Scholar
  9. 9.
    Constantinopoulos C, Titsias M, Likas A (2006) Bayesian feature and model selection for Gaussian mixture models. IEEE Trans Pattern Anal Mach Intell 28(6):1013–1018CrossRefGoogle Scholar
  10. 10.
    Fan W, Bouguila N (2013) Variational learning of a Dirichlet process of generalized Dirichlet distributions for simultaneous clustering and feature selection. Pattern Recognit 46(10):2754–2769CrossRefzbMATHGoogle Scholar
  11. 11.
    Fan X, Cao L, Xu RYD (2015) Dynamic infinite mixed-membership stochastic blockmodel. IEEE Trans Neural Netw Learn Syst 26(9):2072–2085MathSciNetCrossRefGoogle Scholar
  12. 12.
    Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Mach Learn 42(1/2):177–196CrossRefzbMATHGoogle Scholar
  13. 13.
    Korwar RM, Hollander M (1973) Contributions to the theory of Dirichlet processes. Ann Probab 1:705–711MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Kuehne H, Jhuang H, Garrote E, Poggio T, Serre T (2011) HMDB: a large video database for human motion recognition. In: Proc. of the international conference on computer vision (ICCV), pp 2556–2563Google Scholar
  15. 15.
    Kurihara K, Welling M, Vlassis N (2006) Accelerated variational Dirichlet process mixtures. In: Proc. of advances in neural information processing systems (NIPS)Google Scholar
  16. 16.
    Laptev I (2005) On space–time interest points. Int J Comput Vis 64(2/3):107–123CrossRefGoogle Scholar
  17. 17.
    Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: Proc. of IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8Google Scholar
  18. 18.
    Law MHC, Figueiredo MAT, Jain AK (2004) Simultaneous feature selection and clustering using mixture models. IEEE Trans Pattern Anal Mach Intell 26(9):1154–1166CrossRefGoogle Scholar
  19. 19.
    McLachlan G, Peel D (2000) Finite mixture models. Wiley, New YorkCrossRefzbMATHGoogle Scholar
  20. 20.
    Neal RM (2000) Markov chain sampling methods for Dirichlet process mixture models. J Comput Graph Stat 9(2):249–265MathSciNetGoogle Scholar
  21. 21.
    Nguyen NT, Zheng G, Han Z, Zheng R (2011) Device fingerprinting to enhance wireless security using nonparametric Bayesian method. In: Proc. of the IEEE conference on INFOCOM, pp 1404–1412Google Scholar
  22. 22.
    Pitman J, Yor M (1997) The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann Probab 25(2):855–900MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Sethuraman J (1994) A constructive definition of Dirichlet priors. Stat Sin 4:639–650MathSciNetzbMATHGoogle Scholar
  24. 24.
    Shyr A, Darrell T, Jordan M, Urtasun R (2011) Supervised hierarchical Pitman–Yor process for natural scene segmentation. In: Proc. of the 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 2281–2288Google Scholar
  25. 25.
    Song Y, Tang S, Zheng YT, Chua TS, Zhang Y, Lin S (2012) Exploring probabilistic localized video representation for human action recognition. Multimedia Tools and Applications 58(3):663–685CrossRefGoogle Scholar
  26. 26.
    Sudderth EB, Jordan MI (2008) Shared segmentation of natural scenes using dependent Pitman-Yor processes. In: Proc. of Advances in Neural Information Processing Systems (NIPS), pp 1585–1592Google Scholar
  27. 27.
    Teh YW (2006) A hierarchical Bayesian language model based on Pitman-Yor processes. In: Proc. of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, ACL-44, pp 985–992Google Scholar
  28. 28.
    Walker SG (2007) Sampling the Dirichlet mixture model with slices. Communications in Statistics- Simulation and Computation 36:45–54MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Walker SG, Gutierrez-Pena E (2007) Bayesian parametric inference in a nonparametric framework. Test 16:188–197MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Wang T, Hammoud R, Zhu Z (2014) Ground-based activity recognition at distance and behind wall. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 231–236Google Scholar
  31. 31.
    Wei X, Li C (2012) The infinite student’s t-mixture for robust modeling. Signal Processing 92(1):224–234CrossRefGoogle Scholar
  32. 32.
    Wei X, Yang Z (2012) The infinite student’s t-factor mixture analyzer for robust clustering and classification. Pattern Recognition 45(12):4346–4357CrossRefzbMATHGoogle Scholar
  33. 33.
    Zhang T, Liu S, Xu C, Lu H (2011) Boosted multi-class semi-supervised learning for human action recognition. Pattern Recognition 44(10):2334–2342CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and TechnologyHuaqiao UniversityXiamenChina
  2. 2.The Concordia Institute for Information Systems Engineering (CIISE)Concordia UniversityMontrealCanada

Personalised recommendations