Semantic Role Labeling of Speech Transcripts Without Sentence Boundaries

  • Niraj ShresthaEmail author
  • Marie-Francine Moens
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11107)


Speech data is an extremely rich and important source of information. However, we lack suitable methods for the semantic annotation of speech data. For instance, semantic role labeling (SRL) of speech that has been transcribed by an automated speech recognition (ASR) system is still an unsolved problem. SRL of ASR data is difficult and complex due to the absence of sentence boundaries, punctuation, grammar errors, words that are wrongly transcribed, and word deletions and insertions. In this paper we propose a novel approach to SRL of ASR data based on the following idea: (1) train the SRL system on data segmented into frames, where each frame consists of a predicate and its semantic roles without considering sentence boundaries; (2) label it with the semantics of PropBank roles; and to assist the above (3) train a part-of-speech (POS) tagger to work on noisy and error prone ASR data. Experiments with the OntoNotes corpus show improvements compared to the state-of-the-art SRL applied on ASR data.


Frame semantics Speech 


  1. 1.
    Punyakanok, V., Roth, D., Yih, W.: The importance of syntactic parsing and inference in semantic role labeling. Comput. Linguist. 34(2), 257–287 (2008)CrossRefGoogle Scholar
  2. 2.
    Johansson, R., Nugues, P.: Dependency-based semantic role labeling of PropBank. In: Proceedings of the EMNLP, Stroudsburg, PA, USA. ACL, pp. 69–78 (2008)Google Scholar
  3. 3.
    Zhao, H., Chen, W., Kit, C., Zhou, G.: Multilingual dependency learning: a huge feature engineering method to semantic dependency parsing. In: Proceedings of the Thirteenth CoNLL 2009, Boulder, Colorado, USA, pp. 55–60 (2009)Google Scholar
  4. 4.
    Stenchikova, S., Hakkani-Tür, D., Tür, G.: QASR: question answering using semantic roles for speech interface. In: Proceeding of INTERSPEECH, ISCA (2006)Google Scholar
  5. 5.
    Kolomiyets, O., Moens, M.F.: A survey on question answering technology from an information retrieval perspective. Inf. Sci. 181(24), 5412–5434 (2011)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Hüwel, S., Wrede, B.: Situated speech understanding for robust multi-modal human-robot communication. In: Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, pp. 391–398. ACL (2006)Google Scholar
  7. 7.
    Huang, X., Baker, J., Reddy, R.: A historical perspective of speech recognition. Commun. ACM 57(1), 94–103 (2014)CrossRefGoogle Scholar
  8. 8.
    Favre, B., Bohnet, B., Hakkani-Tür, D.: Evaluation of semantic role labeling and dependency parsing of automatic speech recognition output. In: Proceedings of ICASSP 2010, pp. 5342–5345, March 2010Google Scholar
  9. 9.
    Hovy, E., Marcus, M., Palmer, M., Ramshaw, L., Weischedel, R.: OntoNotes: the 90% solution. In: Proceedings of NAACL HLT, Stroudsburg, PA, USA, pp. 57–60. ACL (2006)Google Scholar
  10. 10.
    Mohammad, S., Zhu, X., Martin, J.: Semantic role labeling of emotions in Tweets. In: Proceedings of the 5th Workshop on WASSA, Maryland, pp. 32–41. ACL June 2014Google Scholar
  11. 11.
    Stolcke, A.: SRILM - an extensible language modeling toolkit. In: Proceedings of the 7th ICSLP 2002, pp. 901–904 (2002)Google Scholar
  12. 12.
    Fonseca, E., Rosa, J.: A two-step convolutional neural network approach for semantic role labeling. In: Proceedings of IJCNN 2013, pp. 1–7, August 2013Google Scholar
  13. 13.
    Shrestha, N., Moens, M.F.: Semi-automatically alignment of predicates between speech and ontonotes data. In: Proceedings of the 10th edition of LREC 2016 (2016)Google Scholar
  14. 14.
    Manning, C.D.: Part-of-speech tagging from 97% to 100%: is it time for some linguistics? In: Gelbukh, A.F. (ed.) CICLing 2011. LNCS, vol. 6608, pp. 171–189. Springer, Heidelberg (2011). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceKU LeuvenLeuvenBelgium

Personalised recommendations