A Non-negative Approach to Language Informed Speech Separation

  • Gautham J. Mysore
  • Paris Smaragdis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7191)


The use of high level information in source separation algorithms can greatly constrain the problem and lead to improved results by limiting the solution space to semantically plausible results. The automatic speech recognition community has shown that the use of high level information in the form of language models is crucial to obtaining high quality recognition results. In this paper, we apply language models in the context of speech separation. Specifically, we use language models to constrain the recently proposed non-negative factorial hidden Markov model. We compare the proposed method to non-negative spectrogram factorization using standard source separation metrics and show improved results in all metrics.


Hide Markov Model Speech Recognition Language Model Spectral Component Automatic Speech Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  2. 2.
    Smaragdis, P., Raj, B., Shashanka, M.: Supervised and Semi-Supervised Separation of Sounds from Single-Channel Mixtures. In: Davies, M.E., James, C.J., Abdallah, S.A., Plumbley, M.D. (eds.) ICA 2007. LNCS, vol. 4666, pp. 414–421. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Mysore, G.J., Smaragdis, P., Raj, B.: Non-Negative Hidden Markov Modeling of Audio with Application to Source Separation. In: Vigneron, V., Zarzoso, V., Moreau, E., Gribonval, R., Vincent, E. (eds.) LVA/ICA 2010. LNCS, vol. 6365, pp. 140–148. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  4. 4.
    Ozerov, A., Fevotte, C., Charbit, M.: Factorial scaled hidden Markov model for polyphonic audio representation and source separation. In: Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (October 2009)Google Scholar
  5. 5.
    Nakano, M., Le Roux, J., Kameoka, H., Kitano, Y., Ono, N., Sagayama, S.: Nonnegative Matrix Factorization with Markov-Chained Bases for Modeling Time-Varying Patterns in Music Spectrograms. In: Vigneron, V., Zarzoso, V., Moreau, E., Gribonval, R., Vincent, E. (eds.) LVA/ICA 2010. LNCS, vol. 6365, pp. 149–156. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Smaragdis, P., Raj, B.: The Markov selection model for concurrent speech recognition. In: Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (August 2010)Google Scholar
  7. 7.
    Hershey, J.R., Rennie, S.J., Olsen, P.A., Kristjansson, T.T.: Super-human multi-talker speech recognition: A graphical modeling approach. Computer Speech and Language 24(1), 45–46 (2010)CrossRefGoogle Scholar
  8. 8.
    Virtanen, T.: Speech recognition using factorial hidden Markov models for separation in the feature space. In: Proceedings of Interspeech, Pittsburgh, PA (September 2006)Google Scholar
  9. 9.
    Cooke, M., Hershey, J.R., Rennie, S.J.: Monaural speech separation and recognition challenge. Computer Speech and Language 24(1), 1–15 (2010)CrossRefGoogle Scholar
  10. 10.
    Vincent, E., Gribonval, R., Févotte, C.: Performance measurement in blind audio source separation. IEEE Transactions on Audio, Speech, and Language Processing 14(4), 1462–1469 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Gautham J. Mysore
    • 1
  • Paris Smaragdis
    • 1
    • 2
  1. 1.Advanced Technology LabsAdobe Systems Inc.USA
  2. 2.University of Illinois at Urbana-ChampaignUSA

Personalised recommendations