Abstract
We propose a nonparametric Bayesian approach, based on hierarchical Dirichlet processes and generalized Dirichlet distributions, for simultaneous clustering and feature selection. The resulting statistical model is learned within a variational framework that we have developed. The merits of the developed model are shown via extensive simulations and experiments when applied to the challenging problem of images categorization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Attias, H.: A variational Bayes framework for graphical models. In: Proc. of Advances in Neural Information Processing Systems (NIPS), pp. 209–215 (1999)
Berg, T., Forsyth, D.: Animals on the web. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1463–1470 (2006)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)
Blei, D.M., Jordan, M.I.: Variational inference for Dirichlet process mixtures. Bayesian Analysis 1, 121–144 (2005)
Bouguila, N., Ziou, D.: A hybrid SEM algorithm for high-dimensional unsupervised learning using a finite generalized Dirichlet mixture. IEEE Transactions on Image Processing 15(9), 2657–2668 (2006)
Bouguila, N., Ziou, D.: High-dimensional unsupervised selection and estimation of a finite generalized Dirichlet mixture model based on minimum message length. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(10), 1716–1731 (2007)
Boutemedjet, S., Bouguila, N., Ziou, D.: A hybrid feature extraction selection approach for high-dimensional non-Gaussian data clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(8), 1429–1443 (2009)
Fan, W., Bouguila, N.: Variational learning of a Dirichlet process of generalized Dirichlet distributions for simultaneous clustering and feature selection. Pattern Recognition 46(10), 2754–2769 (2013)
Fan, W., Bouguila, N., Ziou, D.: Unsupervised hybrid feature extraction selection for high-dimensional non-gaussian data clustering with variational inference. IEEE Transactions on Knowlege and Data Engineering 25(7), 1670–1685 (2013)
Ferguson, T.S.: Bayesian Density Estimation by Mixtures of Normal Distributions. Recent Advances in Statistics 24, 287–302 (1983)
Ishwaran, H., James, L.F.: Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association 96, 161–173 (2001)
Korwar, R.M., Hollander, M.: Contributions to the theory of Dirichlet processes. The Annals of Probability 1, 705–711 (1973)
Law, M.H.C., Figueiredo, M.A.T., Jain, A.K.: Simultaneous feature selection and clustering using mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(9), 1154–1166 (2004)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)
Parkhi, O.M., Vedaldi, A., Zisserman, A., Jawahar, C.V.: Cats and dogs. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3498–3505 (2012)
Ramanan, D., Forsyth, D.A.: Using temporal coherence to build models of animals. In: Proc. of the 9th IEEE International Conference on Computer Vision (ICCV), pp. 338–345. IEEE Computer Society (2003)
Sato, M.: Online model selection based on the variational Bayes. Neural Computation 13, 1649–1681 (2001)
Sethuraman, J.: A constructive definition of Dirichlet priors. Statistica Sinica 4, 639–650 (1994)
Teh, Y.W., Jordan, M.I.: Hierarchical Bayesian Nonparametric Models with Applications. In: Hjort, N., Holmes, C., Müller, P., Walker, S. (eds.) Bayesian Nonparametrics: Principles and Practice. Cambridge University Press (2010)
Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Hierarchical Dirichlet processes. Journal of the American Statistical Association 101(476), 1566–1581 (2006)
Wang, C., Paisley, J.W., Blei, D.M.: Online variational inference for the hierarchical Dirichlet process. Journal of Machine Learning Research - Proceedings Track 15, 752–760 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Fan, W., Sallay, H., Bouguila, N., Bourouis, S. (2014). A Hierarchical Infinite Generalized Dirichlet Mixture Model with Feature Selection. In: Bouchachia, A. (eds) Adaptive and Intelligent Systems. ICAIS 2014. Lecture Notes in Computer Science(), vol 8779. Springer, Cham. https://doi.org/10.1007/978-3-319-11298-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-11298-5_1
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11297-8
Online ISBN: 978-3-319-11298-5
eBook Packages: Computer ScienceComputer Science (R0)