Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Multivariate bounded support Laplace mixture model

  • 29 Accesses

Abstract

In this paper, bounded Laplace mixture model (BLMM) is proposed. The parameters of proposed model are estimated by maximum likelihood approach via expectation maximization and Newton–Raphson algorithm. The model is proposed for data modeling to perform clustering using synthetic data for univariate and multivariate examples and real datasets of different medical experiments. BLMM is validated through correctness of estimated parameters for synthetic data and clustering accuracy of medical datasets. A new modeling scheme is also introduced for wavelet coefficients which is based on BLMM. It is applied to image clustering and content-based image retrieval (CBIR) for feature extraction in wavelet domain. For feature extraction in this application, each image is decomposed into a set of wavelet subspaces and BLMM with two components is adopted to model the statistical characteristics of the wavelet coefficients for each wavelet subspace. The model parameters adapted from BLMM represent the image features in wavelet domain for each subspace and selected to formulate the feature space which is further used in clustering and CBIR. In the framework for clustering and image retrieval, features extracted in wavelet domain are further modeled through BLMM to categorize images into different groups and trained model is adopted for CBIR. In order to perform image retrieval with trained model via BLMM, city block distance, posterior probability and Kullback–Leibler divergence are introduced. We also propose a novel solution to compute Kullback–Leibler divergence which is very effective for image retrieval due to its low computational complexity and high retrieval rate. The effectiveness and viability of BLMM in texture image clustering and CBIR are demonstrated through UIUC, KTH-TIPS, DTD, STex and Kylberg databases. Different experiments are performed in the chosen applications, and from the results, BLMM has demonstrated its effectiveness in modeling synthetic data, real datasets from medical experiments, feature extraction in wavelet domain, image clustering and CBIR.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

References

  1. Allili MS, Bouguila N, Ziou D (2008) Finite general Gaussian mixture modeling and application to image and video foreground segmentation. J Electron Imaging 17(1):013005

  2. Amin T, Zeytinoglu M, Guan L (2007) Application of Laplacian mixture model to image and video retrieval. IEEE Trans Multimed 9(7):1416–1429

  3. Andrearczyk V, Whelan PF (2017) Deep learning in texture analysis and its application to tissue image classification. In: Depeursinge A, Al-Kadi OS, Mitchell JR (eds) Biomedical texture analysis. Elsevier, Amsterdam, pp 95–129

  4. Azam M, Bouguila N (2018) Bounded Laplace Mixture Model with Applications to Image Clustering and Content Based Image Retrieval. In: 2018 17th IEEE international conference on machine learning and applications (ICMLA), pp 558–563, https://doi.org/10.1109/ICMLA.2018.00090

  5. Azam M, Bouguila N (2019) Bounded generalized gaussian mixture model with ICA. Neural Process Lett 49(3):1299–1320. https://doi.org/10.1007/s11063-018-9868-7

  6. Bai S, Bai X, Tian Q, Latecki LJ (2018) Regularized diffusion process on bidirectional context for object retrieval. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2018.2828815

  7. Banerjee I, Kurtz C, Devorah AE, Do B, Rubin DL, Beaulieu CF (2018) Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: application to bone tumor radiographs. J Biomed Inform 84:123–135. https://doi.org/10.1016/j.jbi.2018.07.002

  8. Banfield JD, Raftery AE (1993) Model-based Gaussian and non-Gaussian clustering. Biometrics 49:803–821

  9. Bhowmick D, Davison A, Goldstein DR, Ruffieux Y (2006) A Laplace mixture model for identification of differential expression in microarray experiments. Biostatistics 7(4):630–641

  10. Bradley PS, Mangasarian OL, Street WN (1997) Clustering via concave minimization. In: Advances in neural information processing systems, pp 368–374

  11. Cheeseman PC, Stutz JC et al (1996) Bayesian classification (autoclass): theory and results. Adv Knowl Discov Data Min 180:153–180

  12. Chen Y, Wang JZ (2004) Image categorization by learning and reasoning with regions. J Mach Learn Res 5(Aug):913–939

  13. Chipman HA, Kolaczyk ED, McCulloch RE (1997) Adaptive Bayesian wavelet shrinkage. J Am Stat Assoc 92(440):1413–1421

  14. Choy S, Tong C (2010) Statistical wavelet subband characterization based on generalized gamma density and its application in texture retrieval. IEEE Trans Image Process 19(2):281–289. https://doi.org/10.1109/TIP.2009.2033400

  15. Cimpoi M, Maji S, Kokkinos I, Mohamed S, , Vedaldi A (2014) Describing textures in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  16. Cord A, Ambroise C, Cocquerez JP (2006) Feature selection in robust clustering based on Laplace mixture. Pattern Recognit Lett 27(6):627–635

  17. Dang VM (1998) Classification de données spatiales: modèles probabilistes et critères de partitionnement. PhD thesis, Compiègne

  18. Dua D, Graff C (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml

  19. Elguebaly T, Bouguila N (2014) Background subtraction using finite mixtures of asymmetric Gaussian distributions and shadow detection. Mach Vis Appl 25(5):1145–1162

  20. Elter M, Schulz-Wendtland R, Wittenberg T (2007) The prediction of breast cancer biopsy outcomes using two cad approaches that both emphasize an intelligible decision process. Med Phys 34(11):4164–4172

  21. Eltoft T, Kim T, Lee TW (2006) On the multivariate Laplace distribution. IEEE Signal Process Lett 13(5):300–303

  22. Ernst MD (1998) A multivariate generalized Laplace distribution. Comput Stat 13:227–232

  23. Espindola R, Ebecken N (2005) On extending f-measure and g-mean metrics to multi-class problems. WIT Transactions on Information and Communication Technologies 35

  24. Farag A, El-Baz A, Gimel’farb G (2006) Precise segmentation of multimodal images. IEEE Trans Image Process 15(4):952–968. https://doi.org/10.1109/TIP.2005.863949

  25. Figueiredo MA, Jain AK (2002) Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell 24(3):381–396

  26. Franczak BC, Browne RP, McNicholas PD (2014) Mixtures of shifted asymmetric Laplace distributions. IEEE Trans Pattern Anal Mach Intell 36(6):1149–1157. https://doi.org/10.1109/TPAMI.2013.216

  27. Fritz M, Hayman E, Caputo B, olof Eklundh J (2004) The kth-tips database

  28. Gai S, Zhang B, Yang C, Yu L (2018) Speckle noise reduction in medical ultrasound image using monogenic wavelet and Laplace mixture distribution. Digital Signal Process 72:192–207. https://doi.org/10.1016/j.dsp.2017.10.006

  29. Garg L, McClean S, Meenan B, El-Darzi E, Millard P (2009) Clustering patient length of stay using mixtures of Gaussian models and phase type distributions. In: 2009 22nd IEEE international symposium on computer-based medical systems, pp 1–7, https://doi.org/10.1109/CBMS.2009.5255245

  30. Gil D, Girela JL, De Juan J, Gomez-Torres MJ, Johnsson M (2012) Predicting seminal quality with artificial intelligence methods. Expert Syst Appl 39(16):12564–12573

  31. Gorodkin J (2004) Comparing two k-category assignments by a k-category correlation coefficient. Comput Biol Chem 28(5–6):367–374

  32. Hedelin P, Skoglund J (2000) Vector quantization based on gaussian mixture models. IEEE Trans Speech Audio Process 8(4):385–401. https://doi.org/10.1109/89.848220

  33. Hedelin P, Skoglund J, Samuelsson J (1999) Performance bounds for lpc spectrum quantization. In: 1999 IEEE international conference on acoustics, speech, and signal processing. proceedings. ICASSP99 (Cat. No.99CH36258), vol 2, pp 677–680, https://doi.org/10.1109/ICASSP.1999.759757

  34. Jurman G, Furlanello C (2010) A unifying view for performance measures in multi-class prediction. arXiv preprint arXiv:1008.2908

  35. Jurman G, Riccadonna S, Furlanello C (2012) A comparison of MCC and CEN error measures in multi-class prediction. PLoS ONE 7(8):e41882

  36. Khozeimeh F, Alizadehsani R, Roshanzamir M, Khosravi A, Layegh P, Nahavandi S (2017a) An expert system for selecting wart treatment method. Comput Biol Med 81:167–175

  37. Khozeimeh F, Jabbari Azad F, Mahboubi Oskouei Y, Jafari M, Tehranian S, Alizadehsani R, Layegh P (2017b) Intralesional immunotherapy compared to cryotherapy in the treatment of warts. Int J Dermatol 56(4):474–478

  38. Kurgan LA, Cios KJ, Tadeusiewicz R, Ogiela M, Goodenday LS (2001) Knowledge discovery approach to automated cardiac spect diagnosis. Artif Intell Med 23(2):149–169

  39. Kwitt R, Meerwald P (2019) Salzburg texture image database (STex). http://www.wavelab.at/sources/STex/. Accessed 15 Jan 2019

  40. Kylberg G (2011) The Kylberg texture dataset v. 1.0. External report (Blue series) 35, Centre for Image Analysis, Swedish University of Agricultural Sciences and Uppsala University, Uppsala, Sweden. http://www.cb.uu.se/~gustaf/texture/

  41. Law MHC, Figueiredo MAT, Jain AK (2004) Simultaneous feature selection and clustering using mixture models. IEEE Trans Pattern Anal Mach Intell 26(9):1154–1166. https://doi.org/10.1109/TPAMI.2004.71

  42. Lazebnik S, Schmid C, Ponce J (2005) A sparse texture representation using local affine regions. IEEE Trans Pattern Anal Mach Intell 27(8):1265–1278. https://doi.org/10.1109/TPAMI.2005.151

  43. Lindblom J, Samuelsson J (2003) Bounded support Gaussian mixture modeling of speech spectra. IEEE Trans Speech Audio Process 11(1):88–99. https://doi.org/10.1109/TSA.2002.805639

  44. Little MA, McSharry PE, Roberts SJ, Costello DA, Moroz IM (2007) Exploiting nonlinear recurrence and fractal scaling properties for voice disorder detection. Biomed Eng Online 6(1):23

  45. Malik F, Baharudin B (2013) Analysis of distance metrics in content-based image retrieval using statistical quantized histogram texture features in the DCT domain. J King Saud Univ Comput Inf Sci 25(2):207–218. https://doi.org/10.1016/j.jksuci.2012.11.004

  46. Mayhew MB, Petersen BK, Sales AP, Greene JD, Liu VX, Wasson TS (2018) Flexible, cluster-based analysis of the electronic medical record of sepsis with composite mixture models. J Biomed Inform 78:33–42. https://doi.org/10.1016/j.jbi.2017.11.015

  47. Mclachlan G, Basford K (1988) Mixture models: inference and applications to clustering. J R Stat Soc Ser C Appl Stat 38:384–385. https://doi.org/10.2307/2348072

  48. McLachlan G, Krishnan T (2007) The EM algorithm and extensions, vol 382. Wiley, Hoboken

  49. McLachlan G, Peel D (2004) Finite mixture models. Wiley, Hoboken

  50. Medasani S, Krishnapuram R (1998) Categorization of image databases for efficient retrieval using robust mixture decomposition. In: Proceedings. IEEE workshop on content-based access of image and video libraries (Cat. No.98EX173), pp 50–54, https://doi.org/10.1109/IVL.1998.694495

  51. Medasani S, Krishnapuram R (2001) Categorization of image databases for efficient retrieval using robust mixture decomposition. Comput Vis Image Underst 83(3):216–235. https://doi.org/10.1006/cviu.2001.0926

  52. Mitianoudis N, Stathaki T (2005) Overcomplete source separation using Laplacian mixture models. IEEE Signal Process Lett 12(4):277–280. https://doi.org/10.1109/LSP.2005.843759

  53. Najjar A, Gagne C, Reinharz D (2015) Two-step heterogeneous finite mixture model clustering for mining healthcare databases. In: 2015 IEEE international conference on data mining, pp 931–936, https://doi.org/10.1109/ICDM.2015.70

  54. Neal RM, Hinton GE (1998) A view of the Em algorithm that justifies incremental, sparse, and other variants. In: Jordan MI (ed) Learning in graphical models. Springer, Berlin, pp 355–368

  55. Nguyen H, Azam M, Bouguila N (2019a) Data clustering using variational learning of finite scaled dirichlet mixture models. In: 2019 IEEE 28th international symposium on industrial electronics (ISIE), pp 1391–1396, https://doi.org/10.1109/ISIE.2019.8781334

  56. Nguyen H, Kalra M, Azam M, Bouguila N (2019b) Data clustering using online variational learning of finite scaled dirichlet mixture models. In: 2019 IEEE 20th international conference on information reuse and integration for data science (IRI), pp 267–274, https://doi.org/10.1109/IRI.2019.00050

  57. Nguyen TM, Wu QJ, Zhang H (2014) Bounded generalized Gaussian mixture model. Pattern Recognit 47(9):3132–3142

  58. Patrício M, Pereira J, Crisóstomo J, Matafome P, Gomes M, Seiça R, Caramelo F (2018) Using resistin, glucose, age and bmi to predict the presence of breast cancer. BMC Cancer 18(1):29

  59. Peel D, McLachlan G (2000) Robust mixture modelling using the t distribution. Stat Comput 10(4):339–348

  60. Rish I (2001) An empirical study of the naive Bayes classifier. Technical report

  61. Romberg JK, Choi H, Baraniuk RG (2001) Bayesian tree-structured image modeling using wavelet-domain hidden Markov models. IEEE Trans Image Process 10(7):1056–1068. https://doi.org/10.1109/83.931100

  62. Rui Y, Huang TS, Ortega M, Mehrotra S (1998) Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans Circuits Syst Video Technol 8(5):644–655. https://doi.org/10.1109/76.718510

  63. Scricciolo C (2018) Bayes and maximum likelihood for L-1-Wasserstein deconvolution of Laplace mixtures. Stat Methods Appl 27(2):333–362

  64. Shi F, Selesnick IW (2006) Multivariate quasi-Laplacian mixture models forwavelet-based image denoising. In: 2006 International Conference on Image Processing, pp 2625–2628, https://doi.org/10.1109/ICIP.2006.313048

  65. Simoncelli EP (1997) Statistical models for images: Compression, restoration and synthesis. In: Conference record of the thirty-first Asilomar conference on signals, systems & computers, vol 1, pp 673–678. IEEE

  66. Song Y, Li Q, Feng D, Zou JJ, Cai W (2016) Texture image classification with discriminative neural networks. Comput Vis Media 2(4):367–377

  67. Subramaniam AD, Rao BD (2000) PDF optimized parametric vector quantization of speech line spectral frequencies. In: 2000 IEEE workshop on speech coding. proceedings. Meeting the challenges of the new millennium (Cat. No.00EX421), pp 87–89, https://doi.org/10.1109/SCFT.2000.878407

  68. Sun J, Zhou A, Keates S, Liao S (2018) Simultaneous bayesian clustering and feature selection through student’s \({t}\) mixtures model. IEEE Trans Neural Netw Learn Syst 29(4):1187–1199. https://doi.org/10.1109/TNNLS.2016.2619061

  69. Titterington D, Smith A, Makov U (1985) Statistical analysis of finite mixture distributions. Wiley, New York

  70. Usevitch BE (2001) A tutorial on modern lossy wavelet image compression: foundations of jpeg 2000. IEEE Signal Process Mag 18(5):22–35. https://doi.org/10.1109/79.952803

  71. Usevitch BE, Orchard MT (1995) Smooth wavelets, transform coding, and Markov-1 processes. IEEE Trans Signal Process 43(11):2561–2569. https://doi.org/10.1109/78.482107

  72. Varma M, Zisserman A (2005) A statistical approach to texture classification from single images. Int J Comput Vis 62(1–2):61–81

  73. Xu L, Jordan MI (1996) On convergence properties of the em algorithm for gaussian mixtures. Neural Comput 8(1):129–151

  74. Yeh IC, Yang KJ, Ting TM (2009) Knowledge discovery on RFM model using Bernoulli sequence. Expert Syst Appl 36(3):5866–5871

  75. Yuan H, Zhang XP (2004) Texture image retrieval based on a Gaussian mixture model and similarity measure using a Kullback divergence. In: 2004 IEEE international conference on multimedia and expo (ICME) (IEEE Cat. No.04TH8763), vol 3, pp 1867–1870, https://doi.org/10.1109/ICME.2004.1394622

  76. Yuan H, Zhang XP (2010) Statistical modeling in the wavelet domain for compact feature extraction and similarity measure of images. IEEE Trans Circuits Syst Video Technol 20(3):439–445. https://doi.org/10.1109/TCSVT.2009.2031396

  77. Yuan H, Zhang XP, Guan L (2003) Content-based image retrieval using a Gaussian mixture model in the wavelet domain. In: Proceedings of SPIE, vol 5150, pp 422–429, https://doi.org/10.1117/12.503262

  78. Zheng L, Yang Y, Tian Q (2018) Sift meets CNN: a decade survey of instance retrieval. IEEE Trans Pattern Anal Mach Intell 40(5):1224–1244. https://doi.org/10.1109/TPAMI.2017.2709749

Download references

Acknowledgements

The completion of this research was made possible thanks to the Natural Sciences and Engineering Research Council of Canada (NSERC).

Author information

Correspondence to Nizar Bouguila.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by V. Loia.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Azam, M., Bouguila, N. Multivariate bounded support Laplace mixture model. Soft Comput (2020). https://doi.org/10.1007/s00500-020-04737-7

Download citation

Keywords

  • Bounded Laplace mixture model (BLMM)
  • Expectation maximization (EM)
  • Feature extraction
  • Medical data clustering
  • Wavelet transforms
  • Image clustering
  • Content-based image retrieval (CBIR)