An Enhanced Speech Emotion Recognition System Based on Discourse Information

  • Chun Chen
  • Mingyu You
  • Mingli Song
  • Jiajun Bu
  • Jia Liu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3991)


There are certain correlation between two persons’ emotional states in communication, but none of previous work has focused on it. In this paper, a novel conversation database in Chinese was collected and an emotion interaction matrix was proposed to embody the discourse information in conversation. Based on discourse information, an enhanced speech emotion recognition system was presented to improve the recognition accuracy. Some modifications were performed on traditional KNN classification, which could reduce the interruption of noise. Experiment result shows that our system makes 3% – 5% relative improvement compared with the traditional method.


Emotional State Emotion Recognition Recognition Accuracy Acoustic Feature Recognition Result 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Mehrabian, A.: Communication without words. Psychology Today 2(4), 53–56 (1968)Google Scholar
  2. 2.
    Chuang, Z.J., Wu, C.H.: Emotion recognition from textual input using an emotional semantic network. In: Proceedings of ICSLP, Denver, Colorado, USA, pp. 2033–2036 (2002)Google Scholar
  3. 3.
    Schuller, B., Rigoll, G., Lang, M.: Speech emotion recognition combining acoustic features and linguistic information in a hybrid support vector machine-belief network architecture. In: Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, pp. 577–580 (May 2004)Google Scholar
  4. 4.
    Ververidis, D., Kotropoulos, C., Pitas, I.: Automatic emotional speech classification. In: Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, pp. 593–596 (May 2004)Google Scholar
  5. 5.
    Lee, C., Narayanan, S., Pieraccini, R.: Recognition of negative emotions from the speech signal. In: Proc. Automatic Speech Recognition Understanding (December 2001)Google Scholar
  6. 6.
    Yuan, J., Shen, L., Chen, F.: The acoustic realization of anger, fear, joy, and sadness in chinese. In: Proceedings of ICSLP, Denver, Colorado, USA, pp. 2025–2028 (2002)Google Scholar
  7. 7.
    Liscombe, J., Riccardi, G., Hakkani-Tür, D.: Using Context to Improve Emotion Detection in Spoken Dialog Systems. In: The Proceedings of EUROSPEECH 2005 (September 2005)Google Scholar
  8. 8.
    Ang, J., Dhillon, R., Krupski, A., Shriberg, E., Stolcke, A.: Prosody-based automatic detection of annoyance and frustration in human-computer dialog. In: Proceedings of ICSLP, Denver, Colorado, USA, pp. 2037–2039 (2002)Google Scholar
  9. 9.
    Lee, C.M., Narayanan, S.: Towards detecting emotions in spoken dialogs. IEEE Transactions on Speech and Audio Processing (2004) (in press)Google Scholar
  10. 10.
    Batliner, A., Fischer, K., Huber, R., Spilker, J., Nöth, E.: How to find trouble in communication. Speech Communication 40, 117–143 (2003)MATHCrossRefGoogle Scholar
  11. 11.
    Crystal, D.: Prosodic Systems and Intonation in English. Cambridge University Press, Cambridge (1969)Google Scholar
  12. 12.
    Crystal, D.: The English Tone of Voice. Edward Arnold (1975)Google Scholar
  13. 13.
    Crystal, D.: The Cambridge Encyclopaedia of the English Language. Cambridge University Press, Cambridge (1995)Google Scholar
  14. 14.
    You, M., Chen, C., Bu, J.: Chad: A Chinese Affective Database. In: Proc. Affective Computing and Intelligent Interaction, pp. 542–549 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Chun Chen
    • 1
  • Mingyu You
    • 1
  • Mingli Song
    • 1
  • Jiajun Bu
    • 1
  • Jia Liu
    • 1
  1. 1.College of Computer Science, YuQuan CampusZheJiang UniversityHangzhouP.R.China

Personalised recommendations