Multimedia Tools and Applications

, Volume 78, Issue 11, pp 15141–15168 | Cite as

Speech and music classification using spectrogram based statistical descriptors and extreme learning machine

  • Gajanan K. BirajdarEmail author
  • Mukesh D. Patil


This article proposes a novel feature extraction approach for speech/music classification based on generalized Gaussian distribution descriptors extracted from IIR-CQT spectrogram representation. IIR-CQT spectrogram visual representation provides superior temporal resolution at high frequencies and better spectral resolution for low frequencies compared to the conventional short-time Fourier transform analysis which provides uniform frequency resolution. Multi-level decomposition of the spectrogram image is then performed using the Nonsubsampled Contourlet Transform (NSCT) which a fully shift-invariant, multi-scale, and multi-direction expansion that can preserve the edges of the textural pattern of speech and music. The generalized Gaussian distribution (GGD) parameters are produced using maximum likelihood estimation (MLE) from the NSCT subbands to create the image feature descriptor. Chaos crow search algorithm is employed to chose the most relevant feature sub-set and to discard redundant features and finally the extreme learning machine classifier categorizes input audio segment into speech/music. The experimental results show that the proposed feature descriptor is effective and performs better compared to the existing approaches in the speech/music classification. In addition, mismatched training and testing results are also presented.


IIR-CQT spectrogram Nonsubsampled contourlet transform Generalized Gaussian distribution Chaos crow search algorithm ELM classifier 



The authors would like to thank Professor Dan Ellis for providing the Scheirer & Slaney database.


  1. 1.
    Alam J, Kenny P (2017) Spoofing detection employing infinite impulse response-constant q transform-based feature representations. In: 25th European Signal Processing Conference (EUSIPCO 2017), pp 111–115Google Scholar
  2. 2.
    Anandhi D, Valli S (2018) An algorithm for multi-sensor image fusion using maximum a posteriori and nonsubsampled contourlet transform. Comput Electr Eng 65:139–152. CrossRefGoogle Scholar
  3. 3.
    Askarzadeh A (2016) A novel metaheuristic method for solving constrained engineering optimization problems: Crow search algorithm. Comput Struct 169:1–12. CrossRefGoogle Scholar
  4. 4.
    Bartlett PL (1997) For valid generalization, the size of the weights is more important than the size. In: Jordan M, Kearns M, Solla S (eds) Neural Information Processing Systems 1997, pp 134–139Google Scholar
  5. 5.
    Cancela P, Rocamora M, Lopez E (2009) Feature selection for high-dimensional data: a fast correlation-based filter solution. In: 10th International Society for Music Information Retrieval Conference (ISMIR 2009), pp 309–314Google Scholar
  6. 6.
    Chacko BP, Vimal Krishnan VR, Raju G, Babu Anto P (2012) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern 3(2):149–161. CrossRefGoogle Scholar
  7. 7.
    Costa Y, Oliveira LS, Silla C (2017) An evaluation of convolutional neural networks for music classification using spectrograms. Appl Soft Comput 52 (Supplement C):28–38. CrossRefGoogle Scholar
  8. 8.
    Cunha L, Zhou J (2006) The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans Image Process 15(10):3089–3101CrossRefGoogle Scholar
  9. 9.
    Devanna H, Kumar GAES, Giri Prasad MN (2017) A spatio-frequency orientational energy based medical image fusion using non-sub sampled contourlet transform. Cluster Computing.
  10. 10.
    Didiot E, Illina I, Fohr D, Mella O (2010) A wavelet-based parameterization for speech/music discrimination. Comput Speech Lang 24(2):341–357. CrossRefGoogle Scholar
  11. 11.
    Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106CrossRefGoogle Scholar
  12. 12.
    Do MN, Vetterli M (2006) Wavelet-based texture retrieval using generalized gaussian density and kullback-leibler distance. IEEE Trans Image Process 11(2):146–158MathSciNetCrossRefGoogle Scholar
  13. 13.
    El-Maleh K, Klein M, Petrucci G, Kabal P (2000) Speech/music discrimination for multimedia applications. In: Proceedings of the IEEE International Conference on Acoustics, Speech, Signal Processing, ICASSP 2000. IEEE, pp 2445–2448Google Scholar
  14. 14.
    Evans M, Hastings N, Peacock B (2000) Statistical distributions, third edn. Wiley Series in Probability and Statistics. WileyGoogle Scholar
  15. 15.
    Fuchs G (2015) A robust speech/music discriminator for switched audio coding. In: 23rd European Signal Processing Conference (EUSIPCO). IEEE, pp 569–573.
  16. 16.
    Ghosal A, Chakraborty R, Chakraborty R, Haty S, Dhara BC, Saha SK (2009) Speech/music classification using occurrence pattern of zcr and ste. In: 3rd International Symposium on Intelligent Information Technology Application. IEEE, pp 435–438Google Scholar
  17. 17.
    Ghosal A, Dhara BC, Saha SK (2011) Speech/music classification using empirical mode decomposition. In: 2nd International Conference on Emerging Applications of Information Technology (EAIT). IEEE, pp 49–52.
  18. 18.
    Ghosal A, Dutta S (2017) Speech/music discrimination using perceptual feature. In: International Conference on Computational Science and Engineering. CRC Press, pp 71–76Google Scholar
  19. 19.
    Guo JM, Prasetyo H, Farfoura ME, Lee H (2015) Vehicle verification using features from curvelet transform and generalized gaussian distribution modeling. IEEE Trans Intell Transp Syst 16(4):1989–1998CrossRefGoogle Scholar
  20. 20.
    Hirvonen T (2014) Speech/music classification of short audio segments. In: IEEE International symposium on multimedia. IEEE, pp 135–138.
  21. 21.
  22. 22.
    Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: Theory and applications. Neurocomputing 70(1):489–501. CrossRefGoogle Scholar
  23. 23.
    Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B (Cybern) 42(2):513–529. CrossRefGoogle Scholar
  24. 24.
    Huang X (2017) Automatic video superimposed text detection based on nonsubsampled contourlet transform. Multimedia Tools and Applications.
  25. 25.
    Jensen R, Shen Q (2008) Computational intelligence and feature selection. Wiley, HobokenCrossRefGoogle Scholar
  26. 26.
    Kacprzak S, Ziółko M (2013) Speech/music discrimination via energy density analysis, Springer, BerlinGoogle Scholar
  27. 27.
    Kacprzak S, ej Chwiec ko B, Zioko B (2017) Speech/music discrimination for analysis of radio stations. In: International Conference on Systems, Signals And Image Processing (IWSSIP). IEEE, pp 1–4.
  28. 28.
    Karpagachelvi S, Arthanari M, Sivakumar M (2012) Classification of electrocardiogram signals with support vector machines and extreme learning machine. Neural Comput Appl 21(6):1331–1339. CrossRefGoogle Scholar
  29. 29.
    Khan MKS, Al-Khatib WG (2006) Machine-learning based classification of speech and music. Multimed Syst 12(1):55–67. CrossRefGoogle Scholar
  30. 30.
    Khonglah BK, Prasanna SM (2016) Speech / music classification using speech-specific features. Digit Signal Process 48(Supplement C):71–83. MathSciNetCrossRefGoogle Scholar
  31. 31.
    Kos M, Kačič Z, Vlaj D (2013) Acoustic classification and segmentation using modified spectral roll-off and variance-based features. Digit Signal Process 23(2):659–674. MathSciNetCrossRefGoogle Scholar
  32. 32.
    Krupinski R, Purczynski J (2006) Approximated fast estimator for the shape parameter of generalized gaussian distribution. Sinal Process 86(2):205–211zbMATHCrossRefGoogle Scholar
  33. 33.
    Lan Y, Hu Z, Soh YC, Huang GB (2013) An extreme learning machine approach for speaker recognition. Neural Comput Applic 22(3):417–425. CrossRefGoogle Scholar
  34. 34.
    Lavner Y, Ruinskiy D (2009) A decision-tree-based algorithm for speech/music classification and segmentation. EURASIP Journal on Audio, Speech and Music Processing 2009(1).
  35. 35.
    Lee CC, Shih CY, Lee SK, Hong WT (2012) Enhancement of blood vessels in retinal imaging using the nonsubsampled contourlet transform. Multidim Syst Signal Process 23(4):423–436MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Li Y, Li T, Liu H (2017) Recent advances in feature selection and its applications. Knowl Inf Syst 53(3):551–577. CrossRefGoogle Scholar
  37. 37.
    Lim C, Chang H (2012) Enhancing support vector machine-based speech/music classification using conditional maximum a posteriori criterion. IET Signal Process 6:335–340MathSciNetCrossRefGoogle Scholar
  38. 38.
    Lim C, Chang JH (2015) Efficient implementation techniques of an svm-based speech/music classifier in smv. Multimed Tools Appl 74(15):5375–5400. CrossRefGoogle Scholar
  39. 39.
    Liu Q, Yin J, Leung VCM, Zhai JH, Cai Z, Lin J (2016) Applying a new localized generalization error model to design neural networks trained with extreme learning machine. Neural Comput Applic 27(1):59–66. CrossRefGoogle Scholar
  40. 40.
    Luo F, Guo W, Yu Y, Chen G (2017) A multi-label classification algorithm based on kernel extreme learning machine. Neurocomputing 260:313–320. CrossRefGoogle Scholar
  41. 41.
    Miao J, Niu L (2016) A survey on feature selection. Proced Comput Sci 91 (Supplement C):919–926. CrossRefGoogle Scholar
  42. 42.
    Muñoz-Expósito J, García-Galán S, Ruiz-Reyes N, Vera-Candeas P (2007) Adaptive network-based fuzzy inference system vs. other classification algorithms for warped lpc-based speech/music discrimination. Eng Appl Artif Intell 20(6):783–793. CrossRefGoogle Scholar
  43. 43.
    Nanni L, Costa Y, Lumini A, Kim MY, Baek SR (2016) Combining visual and acoustic features for music genre classification. Expert Syst Appl 45:108–117. CrossRefGoogle Scholar
  44. 44.
    Nanni L, Costa Y, Lucio D, Silla C, Brahnam S (2017) Combining visual and acoustic features for audio classification tasks. Pattern Recogn Lett 88(Supplement C):49–56. CrossRefGoogle Scholar
  45. 45.
    Pikrakis A, Giannakopoulos T, Theodoridis S (2008) A speech/music discriminator of radio recordings based on dynamic programming and bayesian networks. IEEE Trans Multimed 10(5):846–67. 0.1109/TMM.2008.922870CrossRefGoogle Scholar
  46. 46.
    Po DDY, Do MN (2006) Directional multiscale modeling of images using the contourlet transform. IEEE Trans Image Process 15(6):1610–1620MathSciNetCrossRefGoogle Scholar
  47. 47.
    Qu H, Peng Y, Sun W (2007) Texture image retrieval based on contourlet coefficient modeling with generalized gaussian distribution. In: Kang L, Liu Y, Zeng S (eds) Advances in Computation and Intelligence. Springer Berlin Heidelberg, pp 493–502Google Scholar
  48. 48.
    Rashno A, Nazari B, Sadri S, Saraee M (2017) Effective pixel classification of mars images based on ant colony optimization feature selection and extreme learning machine. Neurocomputing 226:66–79. CrossRefGoogle Scholar
  49. 49.
    Reyes NR, Candeas PV, Galán SG, Muñoz J (2010) Two-stage cascaded classification approach based on genetic fuzzy learning for speech/music discrimination. Eng Appl Artif Intell 23(2):151–159. CrossRefGoogle Scholar
  50. 50.
    Ruiz-Reyes N, Vera-Candeas P, Muñoz JE, García-galán S, Cañadas FJ (2009) New speech/music discrimination approach based on fundamental frequency estimation. Multimed Tools Appl 41(2):253–286. CrossRefGoogle Scholar
  51. 51.
    Salaken SM, Khosravi A, Nguyen T, Nahavandi S (2017) Extreme learning machine based transfer learning algorithms: a survey. Neurocomputing 267:516–524. CrossRefGoogle Scholar
  52. 52.
    Saunders J (1996) Real-time discrimination of broadcast speech/music. In: Proceedings of ICASSP, ICASSP 1996. IEEE, vol 2, pp 993–996Google Scholar
  53. 53.
    Sayed GI, Hassanien AE, Azar AT (2017) Feature selection via a novel chaotic crow search algorithm. Neural Computing and Applications.
  54. 54.
    Scheirer E, Slaney M (1997) Construction and evaluation of a robust multifeature speech/music discriminator. In: Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’97), ICASSP ’97. IEEE Computer Society, vol 2, pp 1331–1335Google Scholar
  55. 55.
    Sell G, Clark P (2014) Music tonality features for speech/music discrimination. In: IEEE International conference on acoustic, speech and signal processing (ICASSP). IEEE, pp 2489–2493.
  56. 56.
    Sharan RV, Moir TJ (2015) Noise robust audio surveillance using reduced spectrogram image feature and one-against-all svm. Neurocomputing 158:90–99. CrossRefGoogle Scholar
  57. 57.
    Shensa M (1992) The discrete wavelet transform: wedding the trous and mallat algorithms. IEEE Trans Signal Process 40(10):2464–2482zbMATHCrossRefGoogle Scholar
  58. 58.
    Shirazi J, Ghaemmaghami S (2010) Improvement to speech-music discrimination using sinusoidal model based features. Multimed Tools Appl 50(2):415–435. CrossRefGoogle Scholar
  59. 59.
    Tsipas N, Vrysis L, Dimoulas C, Papanikolaou G (2017) Efficient audio-driven multimedia indexing through similarity-based speech / music discrimination. Multimed Tools Appl 76(24):25603–25621. CrossRefGoogle Scholar
  60. 60.
    Varanasi M, Aazhang B (1989) Parametric generalized gaussian density estimation. J Acoust Soc Amer 86(4):1404–1415. CrossRefGoogle Scholar
  61. 61.
    Wan C, Wu Y (2015) Image retrieval by using non-subsampled shearlet transform and krawtchouk moment invariants. In: Jawahar CV, Shan S (eds) Computer Vision - ACCV 2014 Workshops. Springer International Publishing, pp 218–232Google Scholar
  62. 62.
    Wang WQ, GO W, Ying DW (2003) A fast and robust speech music discrimination approach. In: Fourth International Conference on Information, Communications & Signal Processing, Fourth IEEE Pacific-Rim Conference on Multimedia, ICICS-PCM 2003. IEEE, pp 1325–1329Google Scholar
  63. 63.
    Wang M, Chen H, Yang B, Zhao X, Hu L, Cai Z, Huang H, Tong C (2017) Toward an optimal kernel extreme learning machine using a chaotic moth-flame optimization strategy with applications in medical diagnoses. Neurocomputing 267:69–84. CrossRefGoogle Scholar
  64. 64.
    Wu Q, Yan Q, Deng H, Wang J (2010) A combination of data mining method with decision trees building for speech/music discrimination. Comput Speech Lang 24(2):257–272. CrossRefGoogle Scholar
  65. 65.
    Yan CC, Zhang Y, Xu J, Dai F, Zhang J, Dai Q, Wu F (2014) Efficient parallel framework for hevc motion estimation on many-core processors. IEEE Trans Circ Syst Video Tech 24(12):2077–2089CrossRefGoogle Scholar
  66. 66.
    Yan C, Xie H, Chen J, Zha ZJ, Hao X, Zhang Y, Dai Q (2018) An effective uyghur text detector for complex background images. IEEE Transactions on Multimedia pp 1–1Google Scholar
  67. 67.
    Yan C, Xie H, Liu S, Yin J, Zhang Y, Dai Q (2018) Effective uyghur language text detection in complex background images for traffic prompt identification. IEEE Trans. Intell Trans Syst 19(1):220–229CrossRefGoogle Scholar
  68. 68.
    Yan C, Xie H, Yang D, Yin J, Zhang Y, Dai Q (2018) Supervised hash coding with deep neural network for environment perception of intelligent vehicles. IEEE Trans. Intell Transp Syst 19(1):284–295CrossRefGoogle Scholar
  69. 69.
    Yang G, Li M, Chen L, Yu J (2015) The nonsubsampled contourlet transform based statistical medical image fusion using generalized gaussian density. Comput Math Methods Med 2015(Article ID 262819):1–13. MathSciNetzbMATHGoogle Scholar
  70. 70.
    Yu L, Liu H (2003) Feature selection for high-dimensional data: a fast correlation-based filter solution. In: Fawcett T, Mishra N (eds) Proceedings, Twentieth International Conference on Machine Learning, vol 2, pp 856–863Google Scholar
  71. 71.
    Yu S, Zhang A, Li H (2012) A review of estimating the shape parameter of generalized gaussian distribution. J Comput Inf Syst 8(21):9055–9064Google Scholar
  72. 72.
    Zhang Q, Guo-long B (2009) Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process 89(7):1334–1346zbMATHCrossRefGoogle Scholar
  73. 73.
    Zhang H, Yang XK, Zhang WQ, Zhang WL, Liu J (2016) Application of i-vector in speech and music classification. In: IEEE International symposium on signal processing and information technology (ISSPIT). IEEE, pp 1–5.
  74. 74.
    Zhao J, Zhou Z, Cao F (2014) Human face recognition based on ensemble of polyharmonic extreme learning machine. Neural Comput Appl 24(6):1317–1326. CrossRefGoogle Scholar
  75. 75.
    Zhou H, Sadka A, Jiang RM (2008) Feature extraction for speech and music discrimination. In: International workshop on content-based multimedia indexing, CBMI 2008. IEEE, pp 170–173.

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electronics EngineeringRamrao Adik Institute of TechnologyNavi MumbaiIndia
  2. 2.Department of Electronics & Telecommunication EngineeringRamrao Adik Institute of TechnologyNavi MumbaiIndia

Personalised recommendations