Advertisement

Finite Inverted Beta-Liouville Mixture Models with Variational Component Splitting

  • Kamal MaanicshahEmail author
  • Muhammad Azam
  • Hieu Nguyen
  • Nizar Bouguila
  • Wentao Fan
Chapter
Part of the Unsupervised and Semi-Supervised Learning book series (UNSESUL)

Abstract

Use of mixture models to statistically approximate data has been an interesting topic of research in unsupervised learning methods. Mixture models based on exponential family of distributions have gained popularity in recent years. In this chapter, we introduce a finite mixture model based on Inverted Beta-Liouville distribution which has a higher degree of freedom to provide a better fit for the data. We use a variational learning framework to estimate the parameters which decreases the computational complexity of the model. We handle the problem of model selection with a component splitting approach which is an added advantage as it is done within the variational framework. We evaluate our model against some challenging applications like image clustering, speech clustering, spam image detection, and software defect detection.

Keywords

Positive vectors Inverted Beta-Liouville Model selection Variational learning Component splitting 

References

  1. 1.
    Bakhtiari, A.S., Bouguila, N.: A latent Beta-Liouville allocation model. Expert Syst. Appl. 45, 260–272 (2016)CrossRefGoogle Scholar
  2. 2.
    Banfield, J.D., Raftery, A.E.: Model-based gaussian and non-gaussian clustering. Biometrics 49(3), 803–821 (1993)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008). Similarity Matching in Computer Vision and MultimediaCrossRefGoogle Scholar
  4. 4.
    Bdiri, T., Bouguila, N.: Positive vectors clustering using inverted Dirichlet finite mixture models. Expert Syst. Appl. 39(2), 1869–1882 (2012)CrossRefGoogle Scholar
  5. 5.
    Belloni, A., Chernozhukov, V.: On the computational complexity of MCMC-based estimators in large samples. Ann. Statist. 37(4), 2011–2055 (2009)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bouguila, N.: A variational component splitting approach for finite generalized Dirichlet mixture models. In: 2012 International Conference on Communications and Information Technology (ICCIT), pp. 53–57 (2012)Google Scholar
  7. 7.
    Bouguila, N., Ziou, D., Vaillancourt, J.: Novel mixtures based on the Dirichlet distribution: Application to data and image classification. In: Perner, P., Rosenfeld, A. (eds.) Machine Learning and Data Mining in Pattern Recognition, pp. 172–181. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  8. 8.
    Chen, Y., Wang, J.Z., Krovetz, R.: An unsupervised learning approach to content-based image retrieval. In: Proceeding Seventh International Symposium Signal Processing and its Applications, vol. 1, pp. 197–200 (2003).  https://doi.org/10.1109/ISSPA.2003.1224674 Google Scholar
  9. 9.
    Chen, Y., Wang, J.Z., Krovetz, R.: Clue: cluster-based retrieval of images by unsupervised learning. IEEE Trans. Image Process. 14(8), 1187–1201 (2005).  https://doi.org/10.1109/TIP.2005.849770 CrossRefGoogle Scholar
  10. 10.
    Constantinopoulos, C., Likas, A.: Unsupervised learning of gaussian mixtures based on variational component splitting. IEEE Trans. Neural Netw. 18(3), 745–755 (2007)CrossRefGoogle Scholar
  11. 11.
    Csurka, G., Dance, C.R., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV, pp. 1–22 (2004)Google Scholar
  12. 12.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceeding IEEE Computer Society Conference Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 886–893 (2005)Google Scholar
  13. 13.
    Dredze, M., Gevaryahu, R., Elias-Bachrach, A.: Learning fast classifiers for image spam. In: CEAS 2007 - The Fourth Conference on Email and Anti-Spam, 2–3 August 2007, Mountain View, California, USA (2007)Google Scholar
  14. 14.
    Fan, W., Bouguila, N.: Variational learning of finite Beta-Liouville mixture models using component splitting. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2013)Google Scholar
  15. 15.
    Fan, W., Bouguila, N.: Model-based clustering based on variational learning of hierarchical infinite beta-liouville mixture models. Neural. Process. Lett. 44(2), 431–449 (2016)CrossRefGoogle Scholar
  16. 16.
    Fan, W., Bouguila, N., Ziou, D.: Variational learning of finite Dirichlet mixture models using component splitting. Neurocomputing 129, 3–16 (2014)CrossRefGoogle Scholar
  17. 17.
    Felix, E.A., Lee, S.P.: Integrated approach to software defect prediction. IEEE Access 5, 21524–21547 (2017)CrossRefGoogle Scholar
  18. 18.
    Fumera, G., Pillai, I., Roli, F.: Spam filtering based on the analysis of text information embedded into images. J. Mach. Learn. Res. 7, 2699–2720 (2006)Google Scholar
  19. 19.
    Islam, R., Sakib, K.: A package based clustering for enhancing software defect prediction accuracy. In: 2014 17th International Conference on Computer and Information Technology (ICCIT). IEEE, Piscataway, pp. 81–86 (2014)Google Scholar
  20. 20.
    Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: A review. ACM Comput. Surv. 31(3), 264–323 (1999)CrossRefGoogle Scholar
  21. 21.
    Jing, X.Y., Zhang, Z.W., Ying, S., Wang, F., Zhu, Y.P.: Software defect prediction based on collaborative representation classification. In: Companion Proceedings of the 36th International Conference on Software Engineering, pp. 632–633. ACM, New York (2014). ICSE Companion 2014Google Scholar
  22. 22.
    Kabal, P.: TSP speech database. Tech. rep., Department of Electrical & Computer Engineering. McGill University, Montreal (2002)Google Scholar
  23. 23.
    Liu, D., Chen, T.: Unsupervised image categorization and object localization using topic models and correspondences between images. In: Proceeding IEEE 11th International Conference Computer Vision, pp. 1–7 (2007).  https://doi.org/10.1109/ICCV.2007.4408852
  24. 24.
    Loh, W.Y.: Symmetric multivariate and related distributions. Technometrics 34, 235–236 (2012)CrossRefGoogle Scholar
  25. 25.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91 (2004)CrossRefGoogle Scholar
  26. 26.
    Opper, M., Saad, D.: Tutorial on Variational Approximation Methods. MITP, Cambridge (2001)CrossRefGoogle Scholar
  27. 27.
    Ravinder, M., Venugopal, T.: Content-based cricket video shot classification using bag-of-visual-features. In: Artificial Intelligence and Evolutionary Computations in Engineering Systems (2016)Google Scholar
  28. 28.
    Sayyad Shirabad, J., Menzies, T.: The PROMISE Repository of Software Engineering Databases. School of Information Technology and Engineering. University of Ottawa, Canada (2005)Google Scholar
  29. 29.
    Tyagi, V., Wellekens, C.: On desensitizing the mel-cepstrum to spurious spectral components for robust speech recognition. In: Proceedings (ICASSP ’05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005., vol. 1, pp. I/529–I/532 (2005)Google Scholar
  30. 30.
    Xu, R., Wunsch, D.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005)CrossRefGoogle Scholar
  31. 31.
    Yu, J.: Fault detection using principal components-based gaussian mixture model for semiconductor manufacturing processes. IEEE Trans. Semicond. Manuf. 24(3), 432–444 (2011)CrossRefGoogle Scholar
  32. 32.
    Zakariya, S.M., Ali, R., Ahmad, N.: Combining visual features of an image at different precision value of unsupervised content based image retrieval. In: Proceeding IEEE International Conference Computational Intelligence and Computing Research, pp. 1–4 (2010)Google Scholar
  33. 33.
    Zheng, F., Zhang, G., Song, Z.: Comparison of different implementations of MFCC. J. Comput. Sci. Technol. 16(6), 582–589 (2001)CrossRefGoogle Scholar
  34. 34.
    Zhu, Q., Zhong, Y., Zhao, B., Xia, G., Zhang, L.: Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery. IEEE Geosci. Remote Sens. Lett. 13(6), 747–751 (2016)CrossRefGoogle Scholar
  35. 35.
    Zhu, J., Ge, Z., Song, Z.: Variational Bayesian gaussian mixture regression for soft sensing key variables in non-gaussian industrial processes. IEEE Trans. Control Syst. Technol. 25(3), 1092–1099 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Kamal Maanicshah
    • 1
    Email author
  • Muhammad Azam
    • 2
  • Hieu Nguyen
    • 1
  • Nizar Bouguila
    • 1
  • Wentao Fan
    • 3
  1. 1.Concordia Institute for Information Systems EngineeringConcordia UniversityMontrealCanada
  2. 2.Department of Electrical and Computer Engineering (ECE)Concordia UniversityMontrealCanada
  3. 3.Department of Computer Science and TechnologyHuaqiao UniversityXiamenChina

Personalised recommendations