Advertisement

Variational Learning of Finite Inverted Dirichlet Mixture Models and Applications

  • Parisa Tirdad
  • Nizar BouguilaEmail author
  • Djemel Ziou
Chapter
Part of the Studies in Computational Intelligence book series (SCI, volume 607)

Abstract

Statistical modeling provides a useful and well grounded framework to conduct inference from data. This has given rise to the development of varied rich suite of models and techniques. In particular, finite mixture models have received a lot of attention by offering a formal approach to unsupervised learning which allows to discover the latent structure expressed in observed data. In this chapter, we propose a mixture model based on the inverted Dirichlet mixture which provides a natural way of clustering positive data. An EM-style algorithm is developed based upen variational inference for learning the parameters of the mixture model. The proposed statistical framework is applied to the challenging tasks of natural scene categorization and human activity classification.

Keywords

Inverted dirichlet Mixture models Variational learning Scene categorization Human activity recognition 

Notes

Acknowledgments

The completion of this research was made possible thanks to the Natural Sciences and Engineering Research Council of Canada (NSERC). The authors would like to thank Dr. Wentao Fan for helpful discussions and suggestions.

References

  1. 1.
    Picard, R.W.: Light-years from lena: video and image libraries of the future. In: Proceeding of the IEEE International Conference on Image Processing (ICIP), vol. 1, pp. 310–313 (1995)Google Scholar
  2. 2.
    Ortega, M., Rui, Y., Chakrabarti, K., Porkaew, K., Mehrotra, S., Huang, T.S.: Supporting ranked boolean similarity queries in mars. IEEE Trans. Knowl. Data Eng. 10(6), 905–925 (1998)CrossRefGoogle Scholar
  3. 3.
    Guha, S., Rastogi, R., Shim, K.: Cure: an efficient clustering algorithm for large databases. Inf. Syst. 26(1), 35–58 (2001)CrossRefzbMATHGoogle Scholar
  4. 4.
    Pal, N.R., Biswas, J.: Cluster validation using graph theoretic concepts. Pattern Recognit. 30(6), 847–857 (1997)CrossRefGoogle Scholar
  5. 5.
    Comaniciu, D., Meer, P.: Distribution free decomposition of multivariate data. Pattern Anal. Appl. 2(1), 22–30 (1999)CrossRefGoogle Scholar
  6. 6.
    Basu, S., Banerjee, A., Mooney, R.J.: Semi-supervised clustering by seeding. In: Proceeding of the Nineteenth International Conference on Machine Learning (ICML), pp. 27–34 (2002)Google Scholar
  7. 7.
    Dougherty, E.R., Brun, M.: A probabilistic theory of clustering. Pattern Recognit. 37, 917–925 (2004)CrossRefzbMATHGoogle Scholar
  8. 8.
    Bagirov, A.M., Ugon, J., Webb, D.: Fast modified global k-means algorithm for incremental cluster construction. Pattern Recognit. 44(4), 866–876 (2011)CrossRefzbMATHGoogle Scholar
  9. 9.
    Law, M.H.C., Topchy, A.P., Jain, A.K.: Multiobjective data clustering. In: Proceeding of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. II-424–II-430 (2004)Google Scholar
  10. 10.
    Hastie, T., Tibshirani, R.: Discriminant analysis by gaussian mixtures. J. Roy. Stat. Soc. Ser. B 58(1), 155–176 (1996)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Garcia, V., Nielsen, F., Nock, R.: Levels of details for gaussian mixture models. In: Zha, H., Taniguchi, R., Maybank, S.J. (eds.) ACCV (2), volume 5995 of Lecture Notes in Computer Science, pp. 514–525. Springer (2009)Google Scholar
  12. 12.
    Dixit, M., Rasiwasia, N., Vasconcelos, N.: Adapted gaussian models for image classification. In: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 937–943 (2011)Google Scholar
  13. 13.
    Elgammal, A.M., Harwood, D., Davis, L.S.: Non-parametric model for background subtraction. In: Vernon, D. (ed.) Computer Vision - ECCV 2000, 6th European Conference on Computer Vision, Dublin, Ireland, 26 June– 1 July 2000. Proceedings, Part II, volume 1843 of Lecture Notes in Computer Science, pp. 751–767. Springer (2000)Google Scholar
  14. 14.
    Liu, L., Fan, G.: Combined key-frame extraction and object-based video segmentation. IEEE Trans. Circ. Syst. Video Technol. 15(7), 869–884 (2005)CrossRefGoogle Scholar
  15. 15.
    Song, X., Fan, G.: Joint key-frame extraction and object segmentation for content-based video analysis. IEEE Trans. Circ. Syst. Video Technol. 16(7), 904–914 (2006)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Allili, M.S., Bouguila, N., Ziou, D.: Finite generalized gaussian mixture modeling and applications to image and video foreground segmentation. In: Proceeding of the Fourth Canadian Conference on Computer and Robot Vision (CRV), pp. 183–190 (2007)Google Scholar
  17. 17.
    Bouguila, N.: Spatial color image databases summarization. In: Proceeding of the IEEE International Conference on Acoustics, Speech, and Signal Processing, (ICASSP), pp. 953–956 (2007)Google Scholar
  18. 18.
    Bouguila, N., Ziou, D.: Online clustering via finite mixtures of dirichlet and minimum message length. Eng. Appl. Artif. Intell. 19(4), 371–379 (2006)CrossRefGoogle Scholar
  19. 19.
    Bdiri, T., Bouguila, N.: Learning inverted dirichlet mixtures for positive data clustering. In: Kuznetsov, S.O., Slezak, D., Hepting, D.H., Mirkin, B. (eds.) Rough Sets, Fuzzy Sets, Data Mining and Granular Computing - 13th International Conference, RSFDGrC 2011, Moscow, Russia, 25–27 June 2011. Proceedings, volume 6743 of Lecture Notes in Computer Science, pp. 265–272. Springer (2011)Google Scholar
  20. 20.
    Bdiri, T., Bouguila, N.: Positive vectors clustering using inverted dirichlet finite mixture models. Expert Syst. Appl. 39(2), 1869–1882 (2012)CrossRefGoogle Scholar
  21. 21.
    Bdiri, T., Bouguila, N.: Bayesian learning of inverted dirichlet mixtures for SVM kernels generation. Neural Comput. Appl. 23(5), 1443–1458 (2013)CrossRefGoogle Scholar
  22. 22.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. J. Roy. Stat. Soc. Ser. B 39, 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Bouguila, N., Ziou, D., Hammoud, R.I.: On bayesian analysis of a finite generalized dirichlet mixture via a metropolis-within-gibbs sampling. Pattern Anal. Appl. 12(2), 151–166 (2009)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Ghahramani, Z., Beal, M.J.: Variational inference for bayesian mixtures of factor analysers. In: Advances in Neural Information Processing Systems (NIPS), pp. 449–455 (1999)Google Scholar
  25. 25.
    Archambeau, C., Opper, M., Shen, Y., Cornford, D., Shawe-Taylor, J.: Variational inference for diffusion processes. In: Advances in Neural Information Processing Systems (NIPS) (2007)Google Scholar
  26. 26.
    Opper, M., Sanguinetti, G.: Variational inference for markov jump processes. In: Advances in Neural Information Processing Systems (NIPS) (2007)Google Scholar
  27. 27.
    Tiao, G.G., Cuttman, I.: The inverted dirichlet distribution with applications. J. Am. Stat. Assoc. 60(311), 793–805 (1965)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Yassaee, H.: Inverted dirichlet distribution and multivariate logistic distribution. Can. J. Stat. 2(1–2), 99–105 (1974)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Ghorbel, M.: On the inverted dirichlet distribution. Commun. Stat. Theory Methods 39(1), 21–37 (2009)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Corduneanu, A., Bishop, C.M.: Variational Bayesian model selection for mixture distributions. In: Proceeding of the International Conference on Artificial Intelligence and Statistics (AISTAT), pp. 27–34 (2001)Google Scholar
  31. 31.
    Saul, L., Jordan, M.I.: Exploiting tractable substructures in intractable networks. In: Advances in Neural Information Processing Systems 8, pp. 486–492. MIT Press, Cambridge (1995)Google Scholar
  32. 32.
    Jaakkola, T.S., Jordan, M.I.: Computing upper and lower bounds on likelihoods in intractable networks. In: Proceeding of the Twelfth International Conference on Uncertainty in Artificial Intelligence, UAI’96, pp. 340–348, Morgan Kaufmann Publishers Inc, San Francisco, CA, USA (1996)Google Scholar
  33. 33.
    Attias, H.: A variational bayesian framework for graphical models. In: Advances in Neural Information Processing Systems 12, pp. 209–215. MIT Press, Cambridge (2000)Google Scholar
  34. 34.
    Fei-Fei, L., Perona, P.: A bayesian hierarchical model for learning natural scene categories. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005. vol. 2, pp. 524–531 (2005)Google Scholar
  35. 35.
    Bosch, A., Zisserman, A., Munoz, X.: Scene classification via plsa. In: Proceeding of the 9th European Conference on Computer Vision - Volume Part IV, ECCV’06, pp. 517–530, Springer, Berlin, (2006)Google Scholar
  36. 36.
    Yang, J., Jiang, Y.G., Hauptmann, A.G., Ngo, C.W.: Evaluating bag-of-visual-words representations in scene classification. In: Proceeding of the International Workshop on Workshop on Multimedia Information Retrieval, pp. 197–206, ACM, New York, USA, 2007Google Scholar
  37. 37.
    Nilsback, M.E., Zisserman, A.: A visual vocabulary for flower classification. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1447–1454 (2006)Google Scholar
  38. 38.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceedings of the International Conference on Computer Vision-Volume 2 - Volume 2, ICCV ’99, pp. 1150-1157, IEEE Computer Society, Washington, DC, USA (1999)Google Scholar
  39. 39.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vision 42(3), 145–175 (2001)CrossRefzbMATHGoogle Scholar
  40. 40.
    Aggarwal, J.K., Ryoo, M.S.: Human activity analysis: a review. ACM Comput. Surv. 43(3):16:1–16:43 (2011)Google Scholar
  41. 41.
    Luo, J., Wang, W., Qi, H.: Feature extraction and representation for distributed multi-view human action recognition. IEEE J. Emerg. Sel. Top. Circ. Syst. 3(2), 145–154 (2013)CrossRefzbMATHGoogle Scholar
  42. 42.
    Ballan, L., Bertini, M., Del Bimbo, A., Seidenari, L., Serra, G.: Effective codebooks for human action representation and classification in unconstrained videos. IEEE Trans. Multimedia 14(4), 1234–1245 (2012)CrossRefGoogle Scholar
  43. 43.
    Ballan, L., Bertini, M., Del Bimbo, A., Seidenari, L., Serra, G.: Effective codebooks for human action representation and classification in unconstrained videos. IEEE Trans. Multimedia 14(4), 1234–1245 (2012)CrossRefGoogle Scholar
  44. 44.
    Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  45. 45.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Technical report, Massachusetts Institute of Technology, Cambridge, MA, USA (1980)Google Scholar
  46. 46.
    Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)CrossRefGoogle Scholar
  47. 47.
    Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)CrossRefGoogle Scholar
  48. 48.
    Bouguila, N.: A model-based discriminative framework for sets of positive vectors classification: Application to object categorization. In: 2014 1st International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 277–282 (2014)Google Scholar
  49. 49.
    Ma, Z., Leijon, A.: Bayesian estimation of beta mixture models with variational inference. IEEE Trans. Pattern Anal. Mach. Intell. 33(11), 2160–2173 (2011)CrossRefGoogle Scholar
  50. 50.
    Woolrich, M.W., Behrens, T.E.: Variational bayes inference of spatial mixture models for segmentation. IEEE Trans. Med. Imaging 25(10), 1380–1391 (2006)CrossRefGoogle Scholar
  51. 51.
    Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Mach. Learn. 37(2), 183–233 (1999)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Concordia Institute for Information Systems EngineeringConcordia UniversityMontrealCanada
  2. 2.DI, Faculté des SciencesUniversité de SherbrookeSherbrookeCanada

Personalised recommendations