International Journal of Speech Technology

, Volume 21, Issue 4, pp 837–849 | Cite as

Improvement in monaural speech separation using sparse non-negative tucker decomposition

  • Yash Vardhan VarshneyEmail author
  • Prashant Upadhyaya
  • Zia Ahmad Abbasi
  • Musiur Raza Abidi
  • Omar Farooq


A monaural speech separation/enhancement technique based on non-negative tucker decomposition (NTD) has been introduced in this paper. In the proposed work, the effect of sparsity regularization factor on the separation of mixed signal is included in the generalized cost function of NTD. By using the proposed algorithm, the vector components of both target and mixed signal can be exploited and used for the separation of any monaural mixture. Experiment was done on the monaural data generated by mixing the speech signals from two speakers and, by mixing noise and speech signals using TIMIT and noisex-92 dataset. The separation results are compared with the other existing algorithms in terms of correlation of separated signal with the original signal, signal to distortion ratio, perceptual evaluation of speech quality and short-time objective intelligibility. Further, to get more conclusive information about separation ability, speech recognition using Kaldi toolkit was also performed. The recognition results are compared in terms of word error rate (WER) using the MFCC based features. Results show the average improved WER using proposed algorithm over the nearest performing algorithm is up to 2.7% for mixed speech of two speakers and 1.52% for noisy speech input.


Non-negative matrix factorization Kaldi ASR toolkit Non-negative tucker decomposition Sparse NTD 


  1. Anastasakos, T., McDonough, J., & Makhoul, J. (1997). Speaker adaptive training: A maximum likelihood approach to speaker normalization. In IEEE international conference on acoustics, speech, and signal processing (pp. 1043–1046).Google Scholar
  2. Bavkar, S. (2013). PCA based single channel speech enhancement method for highly noisy environment. In Advances in computing, communications and informatics (ICACCI) (pp. 1103–1107).Google Scholar
  3. Bertin, N., Févotte, C., & Badeau, R. (2009). A tempering approach for Itakura-Saito non-negative matrix factorization. With application to music transcription. In Proceedings of ICASSP, IEEE international conference on acoustics, speech and signal processing (pp. 1545–1548).Google Scholar
  4. Bouguelia, M. R., Nowaczyk, S., Santosh, K. C., & Verikas, A. (2018). Agreeing to disagree: active learning with noisy labels without crowdsourcing. International Journal of Machine Learning and Cybernetics, 9, 1307–1319. Scholar
  5. Cooke, M., Hershey, J. R., & Rennie, S. J. (2010). Monaural speech separation and recognition challenge. Computer Speech & Language, 24, 1–15. Scholar
  6. Dey, N., & Ashour, A. S. (2018a). Applied examples and applications of localization and tracking problem of multiple speech sources. In Direction of arrival estimation and localization of multi-speech sources (pp. 35–48). Cham: Springer.CrossRefGoogle Scholar
  7. Dey, N., & Ashour, A. S. (2018b). Challanges and future perspectives in speech-sources direction of arrival estimation and localization. In Direction of arrival estimation and localization of multi-speech sources (pp. 49–52). Cham: Springer.CrossRefGoogle Scholar
  8. Févotte, C. (2011). Majorization-minization algorithm for smooth Itakuro-Saito non-negative matrix factorization. Compute 1980–1983.
  9. Févotte, C., Bertin, N., & Durrieu, J.-L. (2009). Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis. Neural Computation, 21, 793–830. Scholar
  10. Févotte, C., Gribonval, R., & Vincent, E. (2005). BSS EVAL Toolbox User Guide. Tech Rep 1706, IRISA.Google Scholar
  11. Gales, M. J. F. (1998). Maximum likelihood linear transformations for HMM-based speech recognition. Computer Speech and Language, 12, 75–98. Scholar
  12. Garofolo, J., Lamel, L., & Fisher, W., et al. (1988). Getting started with the DARPA TIMIT CD-ROM: An acoustic phonetic continuous speech database. National Institute of Standards and Technology (NIST), Gaithersburg, MD, USA.Google Scholar
  13. Guan, N., Lan, L., & Tao, D., et al. (2014). Transductive nonnegative matrix factorization for semi-supervised high-performance speech separation. In Proceedings of ICASSP, IEEE international conference on acoustics, speech and signal processing (pp 2534–2538).Google Scholar
  14. Hoyer, P. O. (2004). Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 5, 1457–1469. Scholar
  15. ITU. (2001). Perceptual evaluation of speech quality (PESQ), an objective method for end-to-end speech quality assessment of narrowband telephone networks and speech codecs. In ITU-T recommendation (pp. 1–32).Google Scholar
  16. Jolliffe, I. T. (2002). Principal component analysis (2nd ed.). Berlin: SpringerGoogle Scholar
  17. Khademian, M., & Mehdi, M. (2016). Monaural multi-talker speech recognition using factorial speech processing models. 1–28.Google Scholar
  18. Kim, Y.-D. & Choi, S. (2007). Nonnegative tucker decomposition. 1–8.
  19. Kolda, T. G. (2006) Multilinear operators for higher-order decompositions, SANDIA Report SAND2006-2081.Google Scholar
  20. Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401, 788–791. Scholar
  21. Lef, A., & Bach, F. (2011). Online algorithms for nonnegative matrix factorization with the Itakura-Saito divergence to cite this version: online algorithms for nonnegative matrix factorization with the Itakura-Saito divergence.Google Scholar
  22. Lin, C.-J. (2007). On the convergence of multiplicative update for nonnegative matrix factorization. IEEE Transactions on Neural Networks and Learning Systems, 18, 1589–1596.CrossRefGoogle Scholar
  23. Liu, J., Liu, J., Wonka, P., & Ye, J. (2012). Sparse non-negative tensor factorization using columnwise coordinate descent. Pattern Recognition, 45, 649–656.CrossRefzbMATHGoogle Scholar
  24. Mallat, S. (1998) A wavelet tour of signal processing: the sparse way (3rd ed.). Cambridge: Academic Press.Google Scholar
  25. Mirzal, A. (2017). NMF versus ICA for blind source separation. Advances in Data Analysis and Classification, 11, 25–48. Scholar
  26. Mørup, M., & Hansen, L. K. (2009) Tuning pruning in sparse non-negative matrix factorization. In European signal processing conference (pp. 1923–1927).Google Scholar
  27. Mukherjee, H., Obaidullah, S. M., & Santosh, K. C., et al. (2018). Line spectral frequency-based features and extreme learning machine for voice activity detection from audio signal. International Journal of Speech Technology. Scholar
  28. Park, H.-M., Jung, H.-Y., Lee, T.-W., & Lee, S.-Y. (1999). Subband-based blind signal separation for noisy speech recognition. Electronics Letters, 35, 982–984. Scholar
  29. Plátek, O. (2014). Automatic speech recognition using Kaldi. Charles University in Prague.Google Scholar
  30. Povey, D., Ghoshal, A., Boulianne, G., et al. (2011). The Kaldi speech recognition toolkit. In IEEE workshop on automatic speech recognition and understanding (pp. 1–4).
  31. Rioul, O., & Duhamel, P. (1992). Fast algorithms for discrete and continuous wavelet transforms. IEEE Transactions on Information Theory, 38, 569–586. Scholar
  32. Schmidt, M., Winther, O., & Hansen, L. K. (2009). Bayesian non-negative matrix factorization. In Independent component analysis and signal separation (pp. 540–547).Google Scholar
  33. Stern, R. M. (2003). Signal separation motivated by human auditory perception: Applications to automatic speech recognition. In NSF symposium on speech separation.Google Scholar
  34. Taal, C. H., Hendriks, R. C., Heusdens, R., & Jensen, J. (2011). An algorithm for intelligibility prediction of time—Frequency weighted noisy speech. IEEE Transactions on Audio, Speech, and Language Processing, 19, 2125–2136.CrossRefGoogle Scholar
  35. Upadhyaya, P., Mittal, S. K., Varshney, Y. V., et al. (2017) Speaker adaptive model for hindi speech using Kaldi speech recognition toolkit. In International conference on multimedia, signal processing and communication technologies (IMPACT) (pp. 222–226).Google Scholar
  36. Varga, A., & Steeneken, H. J. M. (1993). Assessment for automatic speech recognition:{II}. {NOISEX-92}: A database and an experiment to study the effct of additive noise on speech recognition systems. Speech Communication, 12, 247–251.CrossRefGoogle Scholar
  37. Varshney, Y. V., Abbasi, Z. A., Abidi, M. R., & Farooq, O. (2017a). Variable sparsity regularization factor based SNMF for monaural speech separation. In 2017 40th international conference on telecommunications and signal processing, TSP 2017.Google Scholar
  38. Varshney, Y. V., Abbasi, Z. A., Abidi, M. R., & Farooq, O. (2017b). Frequency selection based separation of speech signals with reduced computational time using sparse NMF. Archives of Acoustics, 42, 287–295. Scholar
  39. Vincent, E., Gribonval, R., & F´evotte, C. (2006). Performance measurement in blind audio source separation. IEEE Transactions on Audio, Speech, and Language Processing Institute of Electrical and Electronics Engineers, 14, 1462–1469.Google Scholar
  40. Virtanen, T., Cemgil, A. T., & Godsill, S. (2008). Bayesian extensions to non-negative matrix factorisation for audio signal modelling. In Proceedings of ICASSP, IEEE international conference on acoustics, speech, and signal processing (pp. 1825–1828).
  41. Young, S., Hain, T., & Woodland, P., et al. (2002). The HTK book (for version 3.2.1). Cambridge: Cambridge University Engineering Department.Google Scholar
  42. Yuan, Z., Yang, Z., & Oja, E. (2007) Projective nonnegative matrix factorization: Sparseness, orthogonality, and clustering. Helsinki University of Technology 1–14.Google Scholar
  43. Zhou, G., Cichocki, A., Zhao, Q., & Xie, S. (2015). Efficient nonnegative tucker decompositions: Algorithms and uniqueness. IEEE Transactions on Image Processing, 24, 4990–5003. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electronics EngineeringAligarh Muslim UniversityAligarhIndia

Personalised recommendations