Skip to main content
SpringerLink
Log in
Book cover

Conference of the European Association for Computational Linguistics

European Workshop on Natural Language Generation

EACL 2009, ENLG 2009: Empirical Methods in Natural Language Generation pp 294–327Cite as

Generating Referring Expressions in Context: The GREC Task Evaluation Challenges

  • Chapter

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5790))

Abstract

Until recently, referring expression generation (reg) research focused on the task of selecting the semantic content of definite mentions of listener-familiar discourse entities. In the grec research programme we have been interested in a version of the reg problem definition that is (i) grounded within discourse context, (ii) embedded within an application context, and (iii) informed by naturally occurring data. This paper provides an overview of our aims and motivations in this research programme, the data resources we have built, and the first three shared-task challenges, grec-msr’08, grec-msr’09 and grec-neg’09, we have run based on the data.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bagga, A., Baldwin, B.: Algorithms for scoring coreference chains. In: Proceedings of the Linguistic Coreference Workshop at LREC 1998, pp. 563–566 (1998)

    Google Scholar 

  2. Belz, A., Reiter, E.: Comparing automatic and human evaluation of NLG systems. In: Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2006), pp. 313–320 (2006)

    Google Scholar 

  3. Belz, A., Varges, S.: Generation of repeated references to discourse entities. In: Proceedings of the Eleventh European Workshop on Natural Language Generation (ENLG 2007), pp. 9–16 (2007)

    Google Scholar 

  4. Belz, A.: That’s nice.. what can you do with it? Computational Linguistics 35(1), 111–118 (2009)

    Article  Google Scholar 

  5. Bohnet, B., Dale, R.: Viewing referring expression generation as search. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005), pp. 1004–1009 (2005)

    Google Scholar 

  6. Charniak, E., Knight, K., Yamada, K.: Syntax-based language models for machine translation. In: Proceedings of MT Summit IX (2003)

    Google Scholar 

  7. Cheng, H., Poesio, M., Henschel, R., Mellish, C.: Corpus-based np modifier generation. In: Proceedings of The Second Meeting of the North American Chapter of the Association for Computational Linguistics, NAACL 2001 (2001)

    Google Scholar 

  8. Choi, B., Pak, A.: A catalog of biases in questionnaires. Preventing Chronic Disease 2(1) (2005)

    Google Scholar 

  9. Dale, R.: Cooking up referring expressions. In: Proceedings of the 27th Annual Meeting of the Association for Computational Linguistics, ACL 1989 (1989)

    Google Scholar 

  10. Dale, R., Reiter, E.: Computational interpretations of the Gricean maxims in the generation of referring expressions. Cognitive Science 19(2), 233–263 (1995)

    Article  Google Scholar 

  11. van Deemter, K.: Generating referring expressions: Boolean extensions of the Incremental Algorithm. Computational Linguistics 28(1), 37–52 (2002)

    Article  MATH  Google Scholar 

  12. van Deemter, K., van der Sluis, I., Gatt, A.: Building a semantically transparent corpus for the generation of referring expressions. In: Proceedings of the 4th International Conference on Natural Language Generation (INLG 2006), Sydney, Australia, pp. 130–132 (2006)

    Google Scholar 

  13. Deneefe, S., Knight, K.: Synchronous tree adjoining machine translation. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009 (2009)

    Google Scholar 

  14. Forster, K.I., Forster, J.C.: DMDX: A windows display program with millisecond accuracy. Behavior Research Methods, Instruments, & Computers 35(1), 116–124 (2003)

    Article  MathSciNet  Google Scholar 

  15. Gatt, A., Belz, A.: Introducing Shared Tasks to NLG: The TUNA Shared Task Evaluation Challenges. In: Krahmer, E., Theune, M. (eds.) Empirical Methods in NLG. LNCS (LNAI), vol. 5790, pp. 264–293. Springer, Heidelberg (2010)

    Google Scholar 

  16. Gupta, S., Stent, A.: Automatic evaluation of referring expression generation using corpora. In: Proceedings of the 1st Workshop on Using Copora in Natural Language Generation, pp. 1–6 (2005)

    Google Scholar 

  17. Huddleston, R., Pullum, G.: The Cambridge Grammar of the English Language. Cambridge University Press, Cambridge (2002)

    Google Scholar 

  18. Jordan, P.W.: Contextual influences on attribute selection for repeated descriptions. In: van Deemter, K., Kibble, R. (eds.) Information Sharing: Reference and Presupposition in Language Generation and Interpretation. CSLI Publications, Stanford (2002)

    Google Scholar 

  19. Jordan, P.W., Walker, M.: Learning attribute selections for non-pronominal expressions. In: Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, ACL 2000 (2000)

    Google Scholar 

  20. Koller, A., Striegnitz, K., Byron, D., Cassell, J., Dale, R., Moore, J., Oberlander, J.: The first challenge on generating instructions in virtual environments. In: Krahmer, E., Theune, M. (eds.) Empirical Methods in NLG. LNCS (LNAI), vol. 5790, pp. 329–353. Springer, Heidelberg (2010)

    Google Scholar 

  21. Krahmer, E., Theune, M.: Efficient context-sensitive generation of referring expressions. In: van Deemter, K., Kibble, R. (eds.) Information Sharing: Reference and Presupposition in Language Generation and Interpretation, pp. 223–264. CSLI Publications, Stanford (2002)

    Google Scholar 

  22. Lin, C.Y., Och, F.J.: ORANGE: A method for evaluating automatic evaluation metrics for machine translation. In: Proceedings of the 20th International Conference on Computational Linguistics (COLING 2004), Geneva, pp. 501–507 (2004)

    Google Scholar 

  23. Luo, X.: On coreference resolution performance metrics. In: Proceedings of HLT-EMNLP, pp. 25–32 (2005)

    Google Scholar 

  24. Morton, T.: Using Semantic Relations to Improve Information Retrieval. Ph.D. thesis, University of Pensylvania (2005)

    Google Scholar 

  25. Nenkova, A.: Entity-driven rewrite for multi-document summarization. In: Proceedings of the Third International Joint Conference on Natural Language Processing, IJCNLP 2008 (2008)

    Google Scholar 

  26. Nenkova, A.: Understanding the process of multi-document summarization: content selection, rewrite and evaluation. Ph.D. thesis, Columbia University (2006)

    Google Scholar 

  27. Otterbacher, J., Radev, D., Luo, A.: Revisions that improve cohesion in multi-document summaries: a preliminary study. In: Proceedings of the ACL 2002 Workshop on Automatic Summarization, pp. 27–36 (2002)

    Google Scholar 

  28. Poesio, M.: Annotating a corpus to develop and evaluate discourse entity realization algorithms: issues and preliminary results. In: Proceedings of the Second International Conference on Language Resources and Evaluation, LREC 2000 (2000)

    Google Scholar 

  29. Poesio, M.: Discourse annotation and semantic annotation in the gnome corpus. In: Proceedings of the ACL 2004 Discourse Annotation Workshop (2004)

    Google Scholar 

  30. Qiu, L., Kan, M., Chua, T.S.: A public reference implementation of the rap anaphora resolution algorithm. In: Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004), pp. 291–294 (2004)

    Google Scholar 

  31. Reiter, E., Dale, R.: A fast algorithm for the generation of referring expressions. In: Proceedings of the 14th International Conference on Computational Linguistics (ACL 1992), Nantes, France, pp. 232–238 (1992)

    Google Scholar 

  32. Siddharthan, A., Copestake, A.: Generating referring expressions in open domains. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL 2004), Barcelona, Spain (2004)

    Google Scholar 

  33. Siegel, S.: Non-parametric statistics. The American Statistician 11(3), 13–19 (1957)

    MathSciNet  Google Scholar 

  34. Steinberger, J., Poesio, M., Kabadjov, M., Jezek, K.: Two uses of anaphora resolution in summarization. Information Processing and Management: Special issue on Summarization 43(6), 1663–1680 (2007)

    Article  Google Scholar 

  35. Trang Dang, H.: DUC 2005: Evaluation of question-focused summarization systems. In: Proceedings of the COLING-ACL 2006 Workshop on Task-Focused Summarization and Question Answering, Prague, pp. 48–55 (2006)

    Google Scholar 

  36. Turian, J., Shen, L., Melamed, I.D.: Evaluation of machine translation and its evaluation. In: Proceedings of MT Summmit IX, New Orleans, pp. 386–393 (2003)

    Google Scholar 

  37. Viethen, J., Dale, R.: Towards the evaluation of referring expression generation. In: Proceedings of the 4th Australasian Language Technology Workshop (ALTW 2006), pp. 115–122 (2006)

    Google Scholar 

  38. Vilain, M., Burger, J., Aberdeen, J., Connolly, D., Hirschman, L.: A model-theoretic coreference scoring scheme. In: Proceedings of the 6th Conference on Message Understanding (MUC-6), pp. 45–52 (1995)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. NLTG, School of Computing, Mathematical and Information Sciences, University of Brighton, Brighton, BN2 4GJ, UK

    Anja Belz & Eric Kow

  2. Macquarie University, Sydney, NSW 2109, Australia

    Jette Viethen

  3. Institute of Linguistics, Centre for Communication Technology, University of Malta, The Netherlands

    Albert Gatt

  4. Communication and Cognition, Faculty of Arts, Tilburg University, The Netherlands

    Albert Gatt

Authors
  1. Anja Belz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric Kow
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jette Viethen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Albert Gatt
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Humanities, Department of Communication and Information Sciences (DCI), Tilburg University, P.O.Box 90153, 5000 LE, Tilburg, The Netherlands

    Emiel Krahmer

  2. Human Media Interaction (HMI), Department of Electrical Engineering, Mathematics and Computer Science (EEMCS), University of Twente, P.O. Box 217, 7500 AE, Enschede, The Netherlands

    Mariët Theune

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Belz, A., Kow, E., Viethen, J., Gatt, A. (2010). Generating Referring Expressions in Context: The GREC Task Evaluation Challenges. In: Krahmer, E., Theune, M. (eds) Empirical Methods in Natural Language Generation. EACL ENLG 2009 2009. Lecture Notes in Computer Science(), vol 5790. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15573-4_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-15573-4_15

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-15572-7

  • Online ISBN: 978-3-642-15573-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation