Abstract
References to objects in our physical environment are common especially in language about wayfinding. Advanced wayfinding systems that interact with the pedestrian by means of (spoken) natural language therefore need to be able to resolve references given by pedestrians (i.e. understand what entity the pedestrian is referring to). The contribution of this paper is a probabilistic approach to reference resolution in a large-scale, real city environment, where the context changes constantly as the pedestrians are moving. The geographic situation, including information about objects’ location and type, is represented using OpenStreetMap data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
The Reciprocal Rank measure calculates the reciprocal of the rank. It is 1 if the correct object is ranked highest, 0.5 if the correct object is ranked second, etc. The Mean Reciprocal Rank (mrr) is the average across many such calculations.
- 3.
The data is split on the utterance level, where each utterance contains one or more referring expressions.
References
Ballatore A, Bertolotto M, Wilson DC (2013) Geographic knowledge extraction and semantic similarity in openstreetmap. Knowl Info Syst 37(1):61–81
Baltaretu A, Krahmer E, Maes A (2015) Improving route directions: the role of intersection type and visual clutter for spatial reference. Appl Cognitive Psychol 29(5):647–660
Blaylock N (2011) Semantic annotation of street-level geospatial entities. Proceedings of the IEEE ICSC workshop on semantic annotation for computational linguistic resources
Boye J, Fredriksson M, Götze J, Gustafson J, Königsmann J (2014) Walk this way: spatial grounding for city exploration. Natural interaction with robots, knowbots and smartphones, pp 59–67
Bruni E, Tran N-K, Baroni M (2014) Multimodal distributional semantics. J Artif Intell Res (JAIR) 49:1–47
Denis M (1997) The description of routes: a cognitive approach to the production of spatial discourse. Curr Psychol Cogn 16(4):409–458
Funakoshi K, Nakano M, Tokunaga T, Iida R (2012) A unified probabilistic approach to referring expressions. Proceedings of the 13th annual meeting of the special interest group on discourse and dialogue, pp 237–246
Garoufi K, Koller A (2011) The potsdam NLG systems at the GIVE-2.5 challenge. Proceedings of the 13th European workshop on natural language generation (ENLG), pp 307–311
Gorniak P, Roy D (2005) Probabilistic grounding of situated speech using plan recognition and reference resolution. Proceedings of the 7th international conference on multimodal interfaces, pp 138–143
Götze J, Boye J (2015) Resolving spatial references using crowdsourced geographical data. Proceedings of the 20th Nordic conference of computational linguistics, NODALIDA, pp 61–68
Götze J, Boye J (2016a) Learning landmark salience models from users’ route instructions. J Locat Based Serv 10(1):47–63
Götze J, Boye J (2016b) SpaceRef: a corpus of street-level geographic descriptions. Proceedings of LREC
Haklay M, Weber P (2008) Openstreetmap: user-generated street maps. Pervasive Comput, IEEE 7(4):12–18
Iida R, Yasuhara M, Tokunaga T (2011) Multi-modal reference resolution in situated dialogue by integrating linguistic and extra-linguistic clues. In: The 5th international joint conference on natural language processing, pp. 84–92
Kennington C, Schlangen D (2015) Simple learning and compositional application of perceptually grounded word meanings for incremental reference resolution. Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, pp. 292–301
Krishnamurthy J, Kollar T (2013) Jointly learning to parse and perceive: connecting natural language to the physical world. TACL 1:193–206
Kruijff G-J, Kelleher J, Hawes N (2006) Information fusion for visual reference resolution in dynamic situated dialogue. Percept Interact Technol 4021:117–128
MacMahon M, Stankiewicz B, Kuipers B (2006) Walk the talk: connecting language, knowledge, and action in route instructions. Proceedings of the 21st national conference on artificial intelligence, pp 1475–1482
Malinowski M, Fritz M (2014) A multi-world approach to question answering about real-world scenes based on uncertain input. NIPS, pp 1682–1690
Matuszek C, Bo L, Zettlemoyer L, Fox D (2014) Learning from unscripted deictic gesture and language for human-robot interactions. Proceedings of AAAI, pp 2556–2563
Matuszek C, FitzGerald N, Zettlemoyer LS, Bo L, Fox D (2012) A joint model of language and perception for grounded attribute learning. ICML
Misu T, Raux A, Gupta R, Lane I (2014) Situated language understanding at 25 miles per hour. Proceedings of the 15th SIGdial workshop on discourse and dialogue
Modsching M, Kramer R, ten Hagen K (2006) Field trial on gps accuracy in a medium size city: the influence of built-up. In: 3rd workshop on positioning, navigation and communication, pp 209–218
Mooney RJ (2008) Learning to connect language and perception. Proceedings of AAAI, pp 1598–1601
Paraboni I, van Deemter K (2014) Reference and the facilitation of search in spatial domains. Lang, Cogn Neurosci 29(8):1002–1017
Raubal M, Winter S (2002) Enriching wayfinding instructions with local landmarks. In: Geographic information science. Lecture notes in computer science, vol 2478, pp 243–259
Roy D (2005) Grounding words in perception and action: computational insights. Trends in cognitive sciences 9(8):389–396
Salmon-Alt S, Romary L (2009) Reference resolution within the framework of cognitive grammar. Int Colloquium Cognitive Sci, pp 284–299
Schütte N, Kelleher J, Mac Namee B (2010) Visual salience and reference resolution in situated dialogues: a corpus-based evaluation. AAAI symposium on dialog with robots, pp 109–114
Thórisson, K. R. (1994), Simulated perceptual grouping: an application to human computer interaction. Proceedings of the sixteenth annual conference of the cognitive science society CSS94, pp 876–881
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Götze, J., Boye, J. (2017). Reference Resolution for Pedestrian Wayfinding Systems. In: Bregt, A., Sarjakoski, T., van Lammeren, R., Rip, F. (eds) Societal Geo-innovation. AGILE 2017. Lecture Notes in Geoinformation and Cartography. Springer, Cham. https://doi.org/10.1007/978-3-319-56759-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-56759-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56758-7
Online ISBN: 978-3-319-56759-4
eBook Packages: Earth and Environmental ScienceEarth and Environmental Science (R0)