Advertisement

Language Resources and Evaluation

, Volume 47, Issue 4, pp 1089–1125 | Cite as

Spontaneous speech and opinion detection: mining call-centre transcripts

  • Chloé ClavelEmail author
  • Gilles Adda
  • Frederik Cailliau
  • Martine Garnier-Rizet
  • Ariane Cavet
  • Géraldine Chapuis
  • Sandrine Courcinous
  • Charlotte Danesi
  • Anne-Laure Daquo
  • Myrtille Deldossi
  • Sylvie Guillemin-Lanne
  • Marjorie Seizou
  • Philippe Suignard
Original Paper

Abstract

Opinion mining on conversational telephone speech tackles two challenges: the robustness of speech transcriptions and the relevance of opinion models. The two challenges are critical in an industrial context such as marketing. The paper addresses jointly these two issues by analyzing the influence of speech transcription errors on the detection of opinions and business concepts. We present both modules: the speech transcription system, which consists in a successful adaptation of a conversational speech transcription system to call-centre data and the information extraction module, which is based on a semantic modeling of business concepts, opinions and sentiments with complex linguistic rules. Three models of opinions are implemented based on the discourse theory, the appraisal theory and the marketers’ expertise, respectively. The influence of speech recognition errors on the information extraction module is evaluated by comparing its outputs on manual versus automatic transcripts. The F-scores obtained are 0.79 for business concepts detection, 0.74 for opinion detection and 0.67 for the extraction of relations between opinions and their target. This result and the in-depth analysis of the errors show the feasibility of opinion detection based on complex rules on call-centre transcripts.

Keywords

Call-centre data Automatic speech recognition system Opinion detection Business concept detection Disfluency 

Notes

Acknowledgments

This work was partly financed by CAP DIGITAL, the Business Cluster for digital content through the VoxFactory project.

References

  1. Adda-Decker, M., & Lamel, L. (2005). Do speech recognizers prefer female speakers? In Proceedings of the 9th international conference on speech communication and technology (Interspeech’05) (pp. 2205–2208).Google Scholar
  2. Adda-Decker, M., Habert, B., Barras, C., Adda, G., & Boula De Mareuil, P. (2003). A disfluency study for cleaning spontaneous speech automatic transcripts and improving speech language models. In Proceedings of disfluency in spontaneous speech workshop (pp. 67–70), Göteborg, Sweden.Google Scholar
  3. Allauzen, A. (2007). Error detection in confusion network. In Proceedings of the 8th annual conference of the international speech communication association (Interspeech’ 07) (pp. 1749–1752), Antwerp, Belgium.Google Scholar
  4. Appelt D., Hobbs J., Bear J., Israel D., Kameyama M., & Tyson, M. (1993). FASTUS: A finite state processor for information extraction from real-world text. In Proceedings of the international joint conference on artificial intelligence (pp. 1172–1178), Chambéry, France.Google Scholar
  5. Barras, C., Geoffrois, E., Wu, Z., & Liberman, M. (2000). Transcriber: Development and use of a tool for assisting speech corpora production. Speech Communication (special issue on speech annotation and corpus tools), 33(1–2).Google Scholar
  6. Barras, C., Zhu, X., Meignier, S., & Gauvain, J.-L. (2006). Multi-stage speaker diarization of broadcast news. IEEE Transactions on Audio, Speech and Language Processing, 14(5), 1505–1512.CrossRefGoogle Scholar
  7. Baumann, T., Buß, O., Atterer, M., & Schlangen, D. (2009). Evaluating the potential utility of ASR N-best lists for incremental spoken dialogue systems. In Proceedings of interspeech (pp. 1031–1034), Brighton.Google Scholar
  8. Benveniste, E. (1970). L’appareil formel de l’énonciation. In Problèmes de linguistique générale II (pp. 79–88). Gallimard, 1974.Google Scholar
  9. Blanche-Benveniste, C. (1990). Le français parlé : Etudes grammaticales. Paris: Didier-Erudition.Google Scholar
  10. Bloom, K., Garg, N., & Argamon, S. (2007). Extracting appraisal expressions. In Proceedings of HLT-NAACL (pp. 308–315).Google Scholar
  11. Bozzi, L., Suignard, P., & Waast-Richard, C. (2009). Segmentation et classification non supervisée de conversations téléphoniques automatiquement retranscrites. In Proceedings of TALNTraitement Automatique des Langues Naturelles, Juillet 2009.Google Scholar
  12. Brill, E. (1995). Transformation-based error-driven learning and natural language processing: A case study in part of speech tagging. Computational Linguistics, 21(4), 1–37.Google Scholar
  13. Cailliau, F., & Cavet, A. (2010). Analyse des sentiments et transcription automatique: modélisation du déroulement de conversations téléphoniques. Revue Traitement Automatique des Langues, 51(3), 131–154.Google Scholar
  14. Cailliau, F., & Giraudel, A. (2008). Enhanced search and navigation on conversational speech. In Proceedings of SIGIR workshop of searching spontaneous conversational speech, Singapour.Google Scholar
  15. Charaudeau, P. (1992). Grammaire du sens et de l’expression. Paris: Hachette Education.Google Scholar
  16. Clavel, C., & Richard, G. (2011). Recognition of acoustic emotion. In C. Pelachaud (Ed.), Emotional interaction system, John Wisley.Google Scholar
  17. Dubreil E., Vernier M., Monceaux L., & Daille, B. (2008). Annotating opinion—evaluation of blogs. In Proceedings of the LREC workshop on sentiment analysis: metaphor, ontology and terminology (EMOT-08), Marrakech, 2008.Google Scholar
  18. Danesi, C., & Clavel, C. (2010). Impact of spontaneous speech features on business concept detection: A study of call-centre data. In Proceedings of the ACM multimedia workshop on searching spontaneous conversational speech, Firenze, Italy.Google Scholar
  19. Dave, K., Lawrence, S., & Pennock, D. (2003). Mining the peanut gallery: opinion extraction and semantic classification of product reviews. In Proceedings of the twelfth international world wide web conference (pp. 519–528).Google Scholar
  20. Devillers, L., Vaudable, C., & Chastagnol, C. (2010). Real-life emotion-related states detection in call centers: A cross-corpora study. In Proceedings of interspeech (pp. 2350–2353), Makhuari, Japan.Google Scholar
  21. Galliano, S., Gravier, G., & Chaubard, L. (2009). The ESTER2 evaluation campaign for the rich transcription of French radio broadcasts. In Proceedings of interspeech (pp. 2583–2586), Brighton.Google Scholar
  22. Garnier-Rizet, M., Adda, G., Cailliau, F., Gauvain, J.-L., Guillemin-Lanne, S., & Lamel, L. (2008). CallSurf: Automatic transcription, indexing and structuration of call center conversational speech for knowledge extraction and query by content. In Proceedings of the sixth international language resources and evaluation (LREC) European Language Resources Association (ELRA) (pp. 2623–2628), Marrakech, Morocco.Google Scholar
  23. Garofolo, J., Auzanne, C., & Voorhees, E. (1999). The TREC spoken document retrieval track: A success story. In Proceedings of text retrieval conference (TREC) (Vol. 8, pp. 16–19).Google Scholar
  24. Gauvain, J.-L., Lamel, L., & Adda, G. (1998). Partitioning and transcription of broadcast news data. In International conference on speech and language processing (Vol. 4, pp. 1335–1338), Sydney, Australia.Google Scholar
  25. Gauvain, J. L., Lamel, L., Schwenk, H., Adda, G., Chen, L., & Lefevre, F. (2003). Conversational telephone speech recognition. In Proceedings of ICASSP (pp. 212–215), Hong Kong, 2003.Google Scholar
  26. Gillick, L., Ito, Y., & Young, J. (1997). A probabilistic approach to confidence estimation and evaluation. In Proceedings of the IEEE international conference on acoustics, speech, and signal processing (Vol. 2, pp. 879–882), Munich, Germany.Google Scholar
  27. Godfrey, J. J., Holliman, E. C. & Mcdaniel, J. (1992). Switchboard: Telephone speech corpus for research and development. In Proceedings of ICASSP (Vol. 1, pp. 517–520), San Francisco.Google Scholar
  28. Goldwater, S., Jurafsky, D., & Manning, C. D. (2010). Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates. Speech Communication, 52(3), 181–200.Google Scholar
  29. Hain, T., Woodland, P. C., Evermann, G., & Povey, D. (2000). The CU-HTK Hub5e Transcription System. College Park, MD: In Proceedings of NIST Speech Transcription Workshop.Google Scholar
  30. Halliday, M. (1994). Introduction to functional grammar (2nd ed.). London: Edward Arnold.Google Scholar
  31. Hardya, H., Bierman, A., Bryce Inouye, R., Mckenzie, A., Strzalkowski, T., Ursu, C., et al. (2006). The Amities system: Data-driven techniques for automated dialogue. Speech Communication (issue Spoken Language Understanding in Conversational Systems), 48(3–4), 354–373.Google Scholar
  32. Hazen, T., Burianek, T., Polifroni, J., & Seneff, S. (2000). Recognition confidence scoring for use in speech understanding systems. In Proceedings of the ISCA ASR2000 tutorial and research workshop (pp. 213–220). Paris.Google Scholar
  33. Kneser, R., & Ney, H. (1995). Improved backing-off for m-gram language modeling. In Proceedings of the IEEE international conference on acoustics, speech, and signal processing (Vol. 1, p. 181).Google Scholar
  34. Ljolje, A., Hindle, D., Riley, M., & Sproat, R. (2000). The AT&T LVCSR-2000 system. In Proceedings of NIST speech transcription workshop, College Park, MD.Google Scholar
  35. Martin, J. R., & White, P. R. R. (2005). The language of evaluation, appraisal in English. London, New York: Palgrave Macmillan.Google Scholar
  36. Matsoukas, S., Colthurst, T., Kimball, O., Solomonoff, A., Richardson, F., Quillen, C., et al. (2002). The 2001 Byblos English large vocabulary conversational speech recognition system. In Proceedings of ICASSP (Vol. 1, pp. 721–724).Google Scholar
  37. Olsson, J. S., Wintrode, J., & Lee, M. (2007). Fast unconstrained audio search in numerous human languages. In Proceedings of the IEEE international conference on acoustics, speech, and signal processing (Vol. 4, pp. 77–80), Honolulu, Hawaii.Google Scholar
  38. Ostendorf, M., (2009). Transcribing human-directed speech for spoken language processing, In Proceedings of interspeech (pp. 21–26), Brighton, United Kingdom.Google Scholar
  39. Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. In Foundation and trends in information retrieval (Vol. 2(1–2), pp. 1–135). Hanover, USA: Now Publishers Inc.Google Scholar
  40. Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the Acl-02 conference on empirical methods in natural language processing (Vol. 10, pp. 79–86), Morristown, NJ.Google Scholar
  41. Park, Y., Patwardhan, S., Visweswariah, K., & Gates, S. C. (2008). An empirical analysis of word error rate and keyword error rate. In Proceedings of Interspeech (pp. 2070–2073).Google Scholar
  42. Paumier S. (2002). Manuel d’utilisation d’Unitex, Université de Marne-la-Vallée. http://www-lipn.univ-paris13.fr/~rozenknop/Cours/MICR_REI/Seance2/ManuelUnitex1.2.pdf. Accessed 31 Jan 2012.
  43. Poibeau, T. (2002). Extraction d’information à base de connaissances hybrides, PhD Thesis, Université Paris-Nord, March 2002.Google Scholar
  44. Quirk, R., Greenbaum, S., Leech, G., & Svartvik, J. (1985). A comprehensive grammar of the English language. Harlow: Longman.Google Scholar
  45. Riloff, E., & Wiebe, J. (2003). Learning extraction patterns for subjective expressions. In Proceedings of the conference on empirical methods in natural language processingtheoretical issues in natural language processing. association for computational linguistics (Vol. 10, pp. 105–112), Morristown, NJ.Google Scholar
  46. Shriberg, E. (1994). Preliminaries to a theory of speech disfluencies. PhD Thesis, University of Berkeley, California.Google Scholar
  47. Silberztein, M. (1994). Dictionnaires électroniques et analyse automatique de textes: le système INTEX. Paris: Masson.Google Scholar
  48. Stolcke, A., Bratt, H., Butzberger, J., Franco, H., Rao Gadde, V. R., Plauche, M., et al. (2000). The SRI March 2000 Hub-5 Conversational speech transcription system. In Proceedings of NIST Speech Transcription Workshop, College Park, MD.Google Scholar
  49. Tang, H., Tan, S., & Cheng, X. (2009). A survey on sentiment detection of reviews. Expert Systems with Applications, 36(7), 10760–10773.CrossRefGoogle Scholar
  50. Ten Bosch, L., & Boves, L. (2004). Survey of spontaneous speech phenomena in a multimodal dialogue system and some implications for ASR. In Proceedings of Interspeech (pp. 1505–1508), Korea.Google Scholar
  51. Turney, P. D. (2002). Thumbs up or thumbs down?: Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 417–424), Morristown, NJ.Google Scholar
  52. Turney, P. D., & Littman, M. L. (2003). Measuring praise and criticism: Inference of semantic orientation from association. ACM Transactions on Information Systems, 21(4), 315–346.CrossRefGoogle Scholar
  53. Whitelaw, C., Garg, N., & Argamon, S. (2005). Using appraisal groups for sentiment analysis. In Proceedings of the 14th ACM international conference on information and knowledge management- CIKM (pp. 625–631), Bremen, Germany,.Google Scholar
  54. Wiebe, J. (2000). Learning subjective adjectives from corpora. In Proceedings of the seventeenth national conference on artificial intelligence and twelfth conference on innovative applications of artificial intelligence (pp. 735–740).Google Scholar
  55. Wiebe, J., & Riloff, E. (2005). Creating subjective and objective sentence classifiers from unannotated texts. In Proceedings of the 6th international conference on computational linguistics and intelligent text processing (CICLing-05), invited paper, springer LNC (p. 3406). Berlin: Springer.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Chloé Clavel
    • 1
    Email author
  • Gilles Adda
    • 2
  • Frederik Cailliau
    • 3
  • Martine Garnier-Rizet
    • 4
  • Ariane Cavet
    • 3
  • Géraldine Chapuis
    • 4
  • Sandrine Courcinous
    • 5
  • Charlotte Danesi
    • 1
  • Anne-Laure Daquo
    • 4
  • Myrtille Deldossi
    • 6
  • Sylvie Guillemin-Lanne
    • 6
  • Marjorie Seizou
    • 6
  • Philippe Suignard
    • 1
  1. 1.EDF R&DClamartFrance
  2. 2.LIMSIUniversité Paris XIOrsay CedexFrance
  3. 3.SinequaParisFrance
  4. 4.VecsysLes UlisFrance
  5. 5.Vocapia Research, 3 rue Jean RostandParc Orsay UniversitéOrsayFrance
  6. 6.TEMISParisFrance

Personalised recommendations