Advertisement

Multimodal Interactive Parsing

  • José-Miguel Benedí
  • Joan-Andreu Sánchez
  • Luis A. Leiva
  • Ricardo Sánchez-Sáez
  • Mauricio Maca
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7887)

Abstract

Probabilistic parsing is a fundamental problem in Computational Linguistics, whose goal is obtaining a syntactic structure associated to a sentence according to a probabilistic grammatical model. Recently, an interactive framework for probabilistic parsing has been introduced, in which the user and the system cooperate to generate error-free parse trees. In an early prototype developed according to this interactive parsing technology, user feedback was provided by means of mouse actions and keyboard strokes. Here we augment the interaction style with support for (non-deterministic) natural handwritten recognition, and provide confidence measures as a visual aid to ease the correction process. Handwriting input seems to be a modality specially suitable for parsing, since the vocabulary size involved in the recognition of syntactic labels is fairly limited and thus intuitively errors should be small. However, errors may increase as handwriting quality (i.e., calligraphy) degrades. To solve this problem, we introduce a late fusion approach that leverages both on-line and off-line information, corresponding to pen strokes and contextual information from the parse trees. We demonstrate that late fusion can effectively help to disambiguate user intention and improve system accuracy.

Keywords

syntactic parsing interactive pattern recognition multimodal interaction late fusion 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Afonso, S., Bick, E., Haber, R., Santos, D.: Floresta sintá(c)tica: a treebank for portuguese. In: Proc. LREC, pp. 1698–1703 (2002)Google Scholar
  2. 2.
    Brants, T., Plaehn, O.: Interactive corpus annotation. In: Proc. LREC (2000)Google Scholar
  3. 3.
    Guyon, I., Schomaker, L., Plamondon, R., Liberman, M., Janet, S.: UNIPEN project of on-line data exchange and recognizer benchmarks. In: Proc. ICPR, pp. 29–33 (1994)Google Scholar
  4. 4.
    Lease, M., Charniak, E., Johnson, M., McClosky, D.: A look at parsing and its applications. In: Proc. AAAI, pp. 1642–1645 (2006)Google Scholar
  5. 5.
    Marcus, M.P., Santorini, B., Marcinkiewicz, M.A.: Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics 19(2), 313–330 (1993)Google Scholar
  6. 6.
    Ortiz, D., Leiva, L.A., Alabau, V., Casacuberta, F.: Interactive machine translation using a web-based architecture. In: Proc. IUI, pp. 423–425 (2010)Google Scholar
  7. 7.
    Romero, V., Leiva, L.A., Toselli, A.H., Vidal, E.: Interactive multimodal transcription of text images using a web-based demo system. In: Proc. IUI, pp. 477–478 (2009)Google Scholar
  8. 8.
    Sánchez-Sáez, R., Leiva, L.A., Sánchez, J.A., Benedí, J.M.: Interactive predictive parsing using a web-based architecture. In: Proc. NAACL-HLT, pp. 37–40 (2010)Google Scholar
  9. 9.
    Sánchez-Sáez, R., Sánchez, J.A., Benedí, J.M.: Interactive predictive parsing. In: Proc. IWPT, pp. 222–225 (2009)Google Scholar
  10. 10.
    Sánchez-Sáez, R., Sánchez, J.A., Benedí, J.M.: Confidence measures for error discrimination in an interactive predictive parsing framework. In: Proc. COLING, pp. 1220–1228 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • José-Miguel Benedí
    • 1
  • Joan-Andreu Sánchez
    • 1
  • Luis A. Leiva
    • 1
  • Ricardo Sánchez-Sáez
    • 1
  • Mauricio Maca
    • 2
  1. 1.Instituto Tecnológico de InformáticaUniversitat Politècnica de ValènciaSpain
  2. 2.Departamento de MatemáticasUniversidad del CaucaColombia

Personalised recommendations