Skip to main content

Using Robustness to Learn to Order Semantic Properties in Referring Expression Generation

Part of the Lecture Notes in Computer Science book series (LNAI,volume 10022)


A sub-task of Natural Language Generation (NLG) is the generation of referring expressions (REG). REG algorithms aim to select attributes that unambiguously identify an entity with respect to a set of distractors. Previous work has defined a methodology to evaluate REG algorithms using real life examples with naturally occurring alterations in the properties of referring entities. It has been found that REG algorithms have key parameters tuned to exhibit a large degree of robustness. Using this insight, we present here experiments for learning the order of semantic properties used by a high performing REG algorithm. Presenting experiments on two types of entities (people and organizations) and using different versions of DBpedia (a freely available knowledge base containing information extracted from Wikipedia pages) we found that robustness of the tuned algorithm and its parameters do coincide but more work is needed to learn these parameters from data in a generalizable fashion.


  • Referential Expressions
  • Order Semantic Properties
  • DBpedia
  • Wikinews
  • Popularity Ordering

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-47955-2_14
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   64.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-47955-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   84.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.


  1. 1.

    Statistics taken from the DBpedia change log available at

  2. 2.

    These potential REG tasks, but not actual REG tasks. We use the news article to extract naturally co-occurring entities.

  3. 3.

    In DBpedia 2014, there was an average of 30.12 properties per person while in DBpedia 3.6, there was an average of 17.3.

  4. 4.

    Our publicly available implementation:


  1. Cahill, L., Carroll, J., Evans, R., Paiva, D., Power, R., Scott, D., van Deemter, K.: From rags to riches: exploiting the potential of a flexible generation architecture. In: Proceedings of the 39th Annual Meeting on Association for Computational Linguistics, pp. 106–113. Association for Computational Linguistics (2001)

    Google Scholar 

  2. Dale, R., Reiter, E.: Computational interpretations of the gricean maxims in the generation of referring expressions. Cogn. Sci. 19(2), 233–263 (1995)

    CrossRef  Google Scholar 

  3. van Deemter, K., Gatt, A., van der Sluis, I., Power, R.: Generation of referring expressions: assessing the incremental algorithm. Cogn. Sci. 36(5), 799–836 (2012)

    CrossRef  Google Scholar 

  4. Duboue, P., Domínguez, M., Estrella, P.: Evaluating robustness of referring expression generation algorithms. In: Proceedings of Mexican International Conference on Artificial Intelligence 2015. IEEE Computer Society (2015)

    Google Scholar 

  5. Gatt, A., Belz, A.: Empirical Methods in Natural Language Generation: Data-oriented Methods and Empirical Evaluation. Springer, Heidelberg (2010)

    Google Scholar 

  6. Gatt, A., van der Sluis, I., van Deemter, K.: Evaluating algorithms for the generation of referring expressions using a balanced corpus. In: Proceedings of the Eleventh European Workshop on Natural Language Generation, ENLG 2007, pp. 49–56. Association for Computational Linguistics, Stroudsburg, PA, USA (2007).

  7. Krahmer, E., Deemter, K.V.: Computational generation of referring expressions: a survey. Comput. Linguist. 38, 173–218 (2009)

    CrossRef  Google Scholar 

  8. Krahmer, E., Koolen, R., Theune, D.M.: Is it that difficult to find a good preference order for the incremental algorithm? Cogn. Sci. 36(5), 837–841 (2012)

    CrossRef  Google Scholar 

  9. Lassila, O., Swick, R.R., Wide, W., Consortium, W.: Resource description framework (rdf) model and syntax specification (1998)

    Google Scholar 

  10. Lebanon, G., Lafferty, J.: Combining rankings using conditional probability models on permutations. In: Sammut, C., Hoffmann, A. (eds.) Proceedings of the 19th International Conference on Machine Learning. Morgan Kaufmann Publishers, San Francisco, CA (2002)

    Google Scholar 

  11. Lehmann, J., Isele, R., Jakob, M., Jentzsch, A., Kontokostas, D., Mendes, P.N., Hellmann, S., Morsey, M., van Kleef, P., Auer, S., Bizer, C.: DBpedia - a large-scale, multilingual knowledge base extracted from wikipedia. Semant. Web J. 6(2), 167–195 (2015)

    Google Scholar 

  12. Pacheco, F., Duboue, P.A., Domínguez, M.A.: On the feasibility of open domain referring expression generation using large scale folksonomies. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2012, pp. 641–645. Association for Computational Linguistics, Stroudsburg, PA, USA (2012).

Download references


The authors would like to thank Annie Ying and the three anonymous reviewers for comments and suggestions.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Pablo Ariel Duboue .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Duboue, P.A., Domínguez, M.A. (2016). Using Robustness to Learn to Order Semantic Properties in Referring Expression Generation. In: Montes y Gómez, M., Escalante, H., Segura, A., Murillo, J. (eds) Advances in Artificial Intelligence - IBERAMIA 2016. IBERAMIA 2016. Lecture Notes in Computer Science(), vol 10022. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-47954-5

  • Online ISBN: 978-3-319-47955-2

  • eBook Packages: Computer ScienceComputer Science (R0)