Skip to main content
SpringerLink
Log in
Book cover

International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems

IPMU 2016: Information Processing and Management of Uncertainty in Knowledge-Based Systems pp 191–203Cite as

The Role of Graduality for Referring Expression Generation in Visual Scenes

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 610))

Abstract

Referring Expression Generation (reg) algorithms, a core component of systems that generate text from non-linguistic data, seek to identify domain objects using natural language descriptions. While reg has often been applied to visual domains, very few approaches deal with the problem of fuzziness and gradation. This paper discusses these problems and how they can be accommodated to achieve a more realistic view of the task of referring to objects in visual scenes.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Reiter, E., Dale, R.: Building Natural Language Generation Systems. Cambridge University Press, Cambridge (2000)

    Book  Google Scholar 

  2. Kacprzyk, J., Zadrozny, S.: Computing with words is an implementable paradigm: Fuzzy queries, linguistic data summaries, and natural-language generation. IEEE Trans. Fuzzy Syst. 18(3), 461–472 (2010)

    Article  MathSciNet  Google Scholar 

  3. Marín, N., Sánchez, D.: On generating linguistic descriptions of time series. Fuzzy Sets Syst. 285, 6–30 (2016)

    Article  MathSciNet  Google Scholar 

  4. Ramos-Soto, A., Bugarín, A., Barro, S.: On the role of linguistic descriptions of data in the building of natural language generation systems. Fuzzy Sets Syst. 285, 31–51 (2016)

    Article  MathSciNet  Google Scholar 

  5. Mitchell, M., Dodge, J., Goyal, A., Yamaguchi, K., Stratos, K., Han, X., Mensch, A., Berg, A., Han, X., Berg, T., Daume III., H.: Midge: Generating Image Descriptions From Computer Vision Detections. In: EACL 2012, Avignon, France, pp. 747–756. Association for Computational Linguistics (2012)

    Google Scholar 

  6. Kulkarni, G., Premraj, V., Ordonez, V., Dhar, S., Li, S., Choi, Y., Berg, A.C., Berg, T.L.: Baby talk: Understanding and generating simple image descriptions. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2891–2903 (2013)

    Article  Google Scholar 

  7. Yatskar, M., Galley, M., Vanderwende, L., Zettlemoyer, L.: See No Evil, Say No Evil : Description Generation from Densely Labeled Images. In: Proceedings of the Third Joint Conference on Lexical and Computation Semantics (*SEM) (2014)

    Google Scholar 

  8. Reiter, E., Dale, R.: A fast algorithm for the generation of referring expressions. In: COLING 1992, pp. 232–238 (1992)

    Google Scholar 

  9. Krahmer, E., van Deemter, K.: Computational generation of referring expressions: A survey. Comput. Linguist. 38(1), 173–218 (2012)

    Article  Google Scholar 

  10. Elsner, M., Rohde, H., Clarke, A.D.F.: Information structure prediction for visual-world referring expressions. In: EACL 2014, Gothenburg, Sweden, pp. 520–529. Association for Computational Linguistics (2014)

    Google Scholar 

  11. Kazemzadeh, S., Ordonez, V., Matten, M., Berg, T.L.: ReferItGame: referring to objects in photographs of natural scenes. In: EMNLP 2014, Doha, Qatar, pp. 787–798. Association for Computational Linguistics (2014)

    Google Scholar 

  12. van Deemter, K.: Generating referring expressions that involve gradable properties. Comput. Linguist. 32(2), 195–222 (2006)

    Article  MATH  Google Scholar 

  13. Turner, R., Sripada, S., Reiter, E., Davy, I.P.: Selecting the content of textual descriptions of geographically located events in spatio-temporal weather data. In: Ellis, R., Allen, T., Petridis, M. (eds.) Applications and Innovations in Intelligent Systems XV, pp. 75–88. Springer, London (2008)

    Chapter  Google Scholar 

  14. Reiter, E., Sripada, S., Hunter, J., Yu, J., Davy, I.: Choosing words in computer-generated weather forecasts. Artif. Intell. 167(1–2), 137–169 (2005)

    Article  Google Scholar 

  15. Cadenas, J.T., Marín, N., Vila, M.A.: Context-aware fuzzy databases. Appl. Soft Comput. 25, 215–233 (2014)

    Article  Google Scholar 

  16. Castillo-Ortega, R., Chamorro-Martínez, J., Marín, N., Sánchez, D., Soto-Hidalgo, J.M.: Describing images via linguistic features and hierarchical segmentation. In: FUZZ-IEEE 2010, pp. 1–8 (2010)

    Google Scholar 

  17. Soto-Hidalgo, J.M., Chamorro-Martínez, J., Sánchez, D.: A new approach for defining a fuzzy color space. In: FUZZ-IEEE 2010, pp. 1–6 (2010)

    Google Scholar 

  18. Chamorro-Martínez, J., Medina, J.M., Barranco, C.D., Galán-Perales, E., Soto-Hidalgo, J.M.: Retrieving images in fuzzy object-relational databases using dominant color descriptors. Fuzzy Sets Syst. 158(3), 312–324 (2007)

    Article  MathSciNet  Google Scholar 

  19. Chamorro-Martínez, J., Martínez-Jiménez, P.M., Soto-Hidalgo, J.M., León-Salas, A.: A fuzzy approach for modelling visual texture properties. Inf. Sci. 313, 1–21 (2015)

    Article  MathSciNet  Google Scholar 

  20. Chamorro-Martínez, J., Martínez-Jiménez, P.M., Soto-Hidalgo, J.M., Prados-Suárez, B.: Fuzzy sets on 2D spaces for fineness representation. Int. J. Approx. Reasoning 62, 46–60 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  21. Prados-Suárez, B., Chamorro-Martínez, J., Sánchez, D., Abad, J.: Region-based fit of color homogeneity measures for fuzzy image segmentation. Fuzzy Sets Syst. 158(3), 215–229 (2007)

    Article  MathSciNet  Google Scholar 

  22. Hudelot, C., Atif, J., Bloch, I.: Fuzzy spatial relation ontology for image interpretation. Fuzzy Sets Syst. 159(15), 1929–1951 (2008)

    Article  MathSciNet  Google Scholar 

  23. Buck, A.R., Keller, J.M., Skubic, M.: A memetic algorithm for matching spatial configurations with the histograms of forces. IEEE Trans. Evol. Comput. 17(4), 588–604 (2013)

    Article  Google Scholar 

  24. Dale, R.: Cooking up referring expressions. In: ACL 1989, Vancouver, BC, pp. 68–75. Association for Computational Linguistics (1989)

    Google Scholar 

  25. Grice, H.P.: Logic and conversation. In: Cole, P., Morgan, J.L. (eds.) Syntax and Semantics 3: Speech Acts, pp. 41–58. Elsevier, Amsterdam (1975)

    Google Scholar 

  26. Dale, R., Reiter, E.: Computational interpretations of the gricean maxims in the generation of referring expressions. Cognitive Sci. 19(2), 233–263 (1995)

    Article  Google Scholar 

  27. Gatt, A., Belz, A.: Introducing shared tasks to NLG: the TUNA shared task evaluation challenges. In: Krahmer, E., Theune, M. (eds.) Empirical Methods in NLG. LNCS, vol. 5790, pp. 264–293. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  28. Jordan, P.W., Walker, M.A.: Learning content selection rules for generating object descriptions in dialogue. J. Artif. Intell. Res. 24, 157–194 (2005)

    MATH  Google Scholar 

  29. Janarthanam, S., Lemon, O.: Adaptive generation in dialogue systems using dynamic user modeling. Computat. Linguist. 40(4), 883–920 (2014)

    Article  Google Scholar 

  30. Castillo-Ortega, R., Marín, N., Sánchez, D., Tettamanzi, A.G.B.: Quality assessment in linguistic summaries of data. In: Greco, S., Bouchon-Meunier, B., Coletti, G., Fedrizzi, M., Matarazzo, B., Yager, R.R. (eds.) IPMU 2012, Part II. CCIS, vol. 298, pp. 285–294. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  31. Bugarín, A., Marín, N., Sánchez, D., Triviño, G.: Aspects of quality evaluation in linguistic descriptions of data. In: FUZZ-IEEE 2015, pp. 1–8 (2015)

    Google Scholar 

  32. Gardent, C.: Generating minimal definite descriptions. In: ACL 2002, pp. 96–103, July 2002

    Google Scholar 

  33. Frank, M.C., Goodman, N.D.: Predicting pragmatic reasoning in language games. Science (New York, N.Y.) 336(6084), 998 (2012)

    Article  MathSciNet  Google Scholar 

  34. Kelleher, J.D., Kruijff, G.: Incremental generation of spatial referring expressions in situated dialog. In: COLING-ACL 2006, Sydney, Australia, pp. 1041–1048. Association for Computational Linguistics (2006)

    Google Scholar 

  35. Pechmann, T.: Incremental speech production and referential overspecification. Linguistics (1989)

    Google Scholar 

  36. Belke, E., Meyer, A.S.: Tracking the time course of multidimensional stimulus discrimination: Analyses of viewing patterns and processing times during “same”-“different” decisions. Eur. J. Cognit. Psychol. 14(2), 237–266 (2002)

    Article  Google Scholar 

  37. Viethen, J., Goudbeek, M., Krahmer, E.: The impact of colour difference and colour codability on reference production. In: CogSci 2012, Austin, TX, pp. 1084–1089. Cognitive Science Society (2012)

    Google Scholar 

  38. van Gompel, R.P., Gatt, A., Krahmer, E., van Deemter, K.: Overspecification in reference: Modelling size contrast effects. In: AMLAP 2014 (2014)

    Google Scholar 

  39. Itti, L., Koch, C.: Computational modelling of visual attention. Nat. Rev. Neurosci. 2(3), 194–203 (2001)

    Article  Google Scholar 

  40. Erdem, E., Erdem, A.: Visual saliency estimation by nonlinearly integrating features using region covariances. J. Vis. 13(4:11), 1–20 (2013)

    Google Scholar 

  41. Oliva, A., Torralba, A.: The role of context in object recognition. Trends Cogn. Sci. 11(12), 520–527 (2007)

    Article  Google Scholar 

  42. Torralba, A., Oliva, A., Castelhano, M.S., Henderson, J.M.: Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search. Psychol. Rev. 113(4), 766–786 (2006)

    Article  Google Scholar 

  43. Kanan, C., Tong, M.H., Zhang, L., Cottrell, F.W.: SUN: Top-down saliency using natural statistics. Vis. Cogn. 17(6–7), 979–1003 (2009)

    Article  Google Scholar 

  44. Sedivy, J.C.: Pragmatic versus form-based accounts of referential contrast: evidence for effects of informativity expectations. J. Psycholinguist. Res. 32(1), 3–23 (2003)

    Article  Google Scholar 

  45. Westerbeek, H., Koolen, R., Maes, A.: Stored object knowledge and the production of referring expressions: the case of color typicality. Frontiers Psychol. 6(July), 1–12 (2015)

    Google Scholar 

  46. Dale, R., Haddock, N.: Generating referring expressions involving relations. In: EACL 1991, Berlin, Germany, pp. 161–166 (1991)

    Google Scholar 

  47. Areces, C., Koller, A., Striegnitz, K.: Referring expressions as formulas of description logic. In: INLG 2008, pp. 42–49 (2008)

    Google Scholar 

  48. Gatt, A., Portet, F.: Multilingual generation of uncertain temporal expressions from data: A study of a possibilistic formalism and its consistency with human subjective evaluations. Fuzzy Sets Syst. 285, 73–93 (2016)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work has been partially supported by the Spanish Ministry of Economy and Competitiveness and the European Regional Development Fund (FEDER) under project TIN2014-58227-P.

Author information

Authors and Affiliations

  1. Institute of Linguistics, University of Malta, Msida, Malta

    Albert Gatt

  2. Department of Computer Science and A.I., University of Granada, 18071, Granada, Spain

    Nicolás Marín & Daniel Sánchez

  3. Laboratoire d’Informatique de Grenoble, Grenoble Institute of Technology, Grenoble, France

    François Portet

Authors
  1. Albert Gatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicolás Marín
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. François Portet
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicolás Marín .

Editor information

Editors and Affiliations

  1. INESC-ID,Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal

    Joao Paulo Carvalho

  2. LIP 6, Université Pierre et Marie Curie, Paris, France

    Marie-Jeanne Lesot

  3. School of Industrial Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands

    Uzay Kaymak

  4. IDMEC,Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal

    Susana Vieira

  5. LIP6, Université Pierre et Marie Curie, CNRS, Paris, France

    Bernadette Bouchon-Meunier

  6. Machine Intelligence Institute, Iona College, New Rochelle, New York, USA

    Ronald R. Yager

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Gatt, A., Marín, N., Portet, F., Sánchez, D. (2016). The Role of Graduality for Referring Expression Generation in Visual Scenes. In: Carvalho, J., Lesot, MJ., Kaymak, U., Vieira, S., Bouchon-Meunier, B., Yager, R. (eds) Information Processing and Management of Uncertainty in Knowledge-Based Systems. IPMU 2016. Communications in Computer and Information Science, vol 610. Springer, Cham. https://doi.org/10.1007/978-3-319-40596-4_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-40596-4_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-40595-7

  • Online ISBN: 978-3-319-40596-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation