Language Resources and Evaluation

, Volume 51, Issue 2, pp 439–462 | Cite as

Stars2: a corpus of object descriptions in a visual domain

  • Ivandré ParaboniEmail author
  • Michelle Reis Galindo
  • Douglas Iacovelli
Original Paper


This paper presents the Stars2 corpus of definite descriptions for referring expression generation (REG). The corpus was produced in collaborative communication involving speaker-hearer pairs, and includes situations of reference that are arguably under-represented in similar work. Stars2 is intended as an incremental contribution to the research in REG and related fields, and it may be used both as training/test data for algorithms of this kind, and also to gain further insights into reference phenomena in general, with a particular focus on the issue of attribute choice in referential overspecification.


Natural language generation Referring expressions  Content selection Corpora 



This work has been supported by FAPESP and the University of São Paulo. The authors are also grateful to all the participants in the data collection.


  1. Belke, E., & Meyer, A. (2002). Tracking the time course of multidimensional stimulus discrimination. European Journal of Cognitive Psychology, 14(2), 237–266.CrossRefGoogle Scholar
  2. Byron, D., Koller, A., Oberlander, J., Stoia, L., & Striegnitz, K. (2007). In Generating instructions in virtual environments (GIVE): A challenge and evaluation testbed for NLG. Workshop on shared tasks and comparative evaluation in natural language generation.Google Scholar
  3. Clarke, A. D. F., Elsner, M., & Rohde, H. (2013). Where’s Wally: The influence of visual salience on referring expression generation. Frontiers in Psychology, 4, 329. doi: 10.3389/fpsyg.2013.00329.Google Scholar
  4. Dale, R. (2002). Cooking up referring expressions. In Proceedings of the 27th Annual Meeting of the Association for Computational Linguistics, (pp. 68–75).Google Scholar
  5. Dale, R., & Viethen, J. (2009). Referring expression gene ration through attribute-based heuristics. In Proceedings of ENLG-2009, (pp. 58–65).Google Scholar
  6. Dale, R., & Haddock, N. J. (1991). Content determination in the generation of referring expressions. Computational Intelligence, 7(4), 252–265.CrossRefGoogle Scholar
  7. Dale, R., & Reiter, E. (1995). Computational interpretations of the Gricean maxims in the generation of referring expressions. Cognitive Science, 19(2), 233–263.CrossRefGoogle Scholar
  8. de Lucena, D. J., Paraboni, I., & Pereira, D. B. (2010). From semantic properties to surface text: The generation of domain object descriptions. Inteligencia Artificial. Revista Iberoamericana de. Inteligencia Artificial, 14(45), 48–58.Google Scholar
  9. Dice, L. R. (1945). Measures of the amount of ecologic association between species. Ecology, 26(3), 297–302.CrossRefGoogle Scholar
  10. dos Santos Silva, D., & Paraboni, I. (2015). Generating spatial referring expressions in interactive 3D worlds. Spatial Cognition & Computation, 15(03), 186–225. doi: 10.1080/13875868.2015.1039166.CrossRefGoogle Scholar
  11. Ferreira, T. C., & Paraboni, I. (2014a). Classification-based referring expression generation. Lecture Notes in Computer Science, 8403, 481–491.CrossRefGoogle Scholar
  12. Ferreira, T. C., & Paraboni, I. (2014b). Referring expression generation: Taking speakers’ preferences into account. Lecture Notes in Artificial Intelligence, 8655, 539–546.Google Scholar
  13. FitzGerald, N., Artzi, Y., & Zettlemoyer, L. (2013). Learning distributions over logical forms for referring expression generation. In Proceedings of the 2013 conference on empirical methods in natural language processing, (pp. 1914–1925). Association for Computational Linguistics.Google Scholar
  14. Gatt, A., Belz, A., & Kow, E. (2009). The TUNA challenge 2009: Overview and evaluation results. In Proceedings of the 12nd European workshop on natural language generation, (pp. 174–182).Google Scholar
  15. Gatt, A., Krahmer, E., van Gompel, R., & van Deemter, K. (2013). Production of referring expressions: Preference trumps discrimination. 35th meeting of the cognitive science society, (pp. 483–488).Google Scholar
  16. Gatt, A., van der Sluis, I., & van Deemter, K. (2007). Evaluating algorithms for the generation of referring expressions using a balanced corpus. Proceedings of ENLG-07.Google Scholar
  17. Gorniak, P., & Roy, D. (2004). Grounded semantic composition for visual scenes. Journal of Artificial Intelligence Research, 21, 429–470.Google Scholar
  18. Grice, H. P. (1975). Logic and conversation Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Syntax and semantics Syntax and semantics (Vol. 3). New York: Academic Press.Google Scholar
  19. Kazemzadeh, S., Ordonez, V., Matten, M., & Berg, T. (2014). ReferItGame: Referring to objects in photographs of natural scenes. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), (pp. 787–798). Association for Computational Linguistics.Google Scholar
  20. Kelleher, J. D., & Costello, F. J. (2009). Applying computational models of spatial prepositions to visually situated dialog. Computational Linguistics, 35(2), 271–306. doi: 10.1162/coli.06-78-prep14.CrossRefGoogle Scholar
  21. Krahmer, E., & van Deemter, K. (2012). Computational generation of referring expressions: A survey. Computational Linguistics, 38(1), 173–218.CrossRefGoogle Scholar
  22. Mitchell, M., van Deemter, K., & Reiter, E. (2010). Natural reference to objects in a visual domain. Proceedings of INLG-2010. The Association for Computer Linguistics.Google Scholar
  23. Paraboni, I. (2000). An algorithm for generating document-deictic references. In Proceedings of workshop coherence in generated multimedia, associated with first int. conf. on natural language generation (INLG-2000), Mitzpe Ramon, (pp. 27–31).Google Scholar
  24. Paraboni, I., & van Deemter, K. (2014). Reference and the facilitation of search in spatial domains. Language, Cognition and Neuroscience, 29(8), 1002–1017.CrossRefGoogle Scholar
  25. Passonneau, R. (2006). Measuring agreement on set-valued items (MASI) for semantic and pragmatic annotation. In Proceedings of the international conference on language resources and evaluation (LREC).Google Scholar
  26. Pechmann, T. (1989). Incremental speech production and referential overspecification. Linguistics, 27(1), 98–110.CrossRefGoogle Scholar
  27. Reiter, E., & Dale, R. (2000). Building natural language generation systems. New York, NY, USA: Cambridge University Press.CrossRefGoogle Scholar
  28. Teixeira, C. V. M., Paraboni, I., da Silva, A. S. R., & Yamasaki, A. K. (2014). Generating relational descriptions involving mutual disambiguation. Lecture Notes in Computer Science, 8403, 492–502.CrossRefGoogle Scholar
  29. van Deemter, K., Gatt, A., van der Sluis, I., & Power, R. (2012). Generation of referring expressions: Assessing the incremental algorithm. Cognitive Science, 36(5), 799–836.CrossRefGoogle Scholar
  30. van Gompel, R., Gatt, A., Krahmer, E., & Deemter, K. V. (2014). Testing computational models of reference generation as models of human language production: The case of size contrast. In Refnet workshop on psychological and computational models of reference comprehension and production. Edinburgh, Scotland.Google Scholar
  31. Viethen, J., & Dale, R. (2011). GRE3D7: A corpus of distinguishing descriptions for objects in visual scenes. In Proceedings of UCNLG+Eval-2011, (pp. 12–22).Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • Ivandré Paraboni
    • 1
    Email author
  • Michelle Reis Galindo
    • 1
  • Douglas Iacovelli
    • 1
  1. 1.School of Arts, Sciences and HumanitiesUniversity of São Paulo (USP/EACH)São PauloBrazil

Personalised recommendations