Abstract
It is known that the comprehension of spatial prepositions involves the deployment of visual attention. For example, consider the sentence “The salt is to the left of the stove”. Researchers [29, 30] have theorized that people must shift their attention from the stove (the reference object, RO) to the salt (the located object, LO) in order to comprehend the sentence. Such a shift was also implicitly assumed in the Attentional Vector Sum (AVS) model by [35], a cognitive model that computes an acceptability rating for a spatial preposition given a display that contains an RO and an LO. However, recent empirical findings showed that a shift from the RO to the LO is not necessary to understand a spatial preposition ([3], see also [15, 38]). In contrast, these findings suggest that people perform a shift in the reverse direction (i.e., from the LO to the RO). Thus, we propose the reversed AVS (rAVS) model, a modified version of the AVS model in which attention shifts from the LO to the RO. We assessed the AVS and the rAVS model on the data from [35] using three model simulation methods. Our simulations show that the rAVS model performs as well as the AVS model on these data while it also integrates the recent empirical findings. Moreover, the rAVS model achieves its good performance while being less flexible than the AVS model. (This article is an updated and extended version of the paper [23] presented at the 8th International Conference on Agents and Artificial Intelligence in Rome, Italy. The authors would like to thank Holger Schultheis for helpful discussions about the additional model simulation.)
Keywords
- Spatial language
- Spatial prepositions
- Cognitive modeling
- Model flexibility
- Visual attention
This is a preview of subscription content, access via your institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
In the case of other prepositions, the corresponding part of the RO is chosen for the location of the focus (e.g., the focus lies on the bottom of the RO for below).
- 3.
[35, p. 276]: “A central feature of this [angular] characterization of spatial term acceptability is that it is dependent only on the direction, not the length, of the vector connecting the landmark [RO] to the trajector [LO].”
- 4.
Therefore, the rAVS model does not need to compute a vector sum nor does it rely on an underlying attentional distribution and thus has a lower computational complexity. This lower computational complexity, however, originates from the simplification of the LO. Accordingly, these considerations are also only valid for simplified LOs.
- 5.
We thank Terry Regier and Laura Carlson for sharing these data.
- 6.
- 7.
The prediction error is also a measure of model generalizability, the property of a model to account for new empirical data, see [34].
- 8.
Note that it is difficult to call data “non-empirical”. You can tell what people do, but it is harder to tell what people do not do. Given the right study design, previously considered “non-empirical” data might become empirical. Nevertheless, the greater the range of model predictions, the higher the probability that some of these predictions are at least implausible (or conflict with other generated predictions).
- 9.
One could argue that we did not compute enough model predictions to define 10 cells on each dimension of the data space. However, if we want to follow the suggestion from [41] and use \(\root n \of {j^k}\) cells, we would need more model predictions due to the high-dimensional data space (\(n= 337\)), namely \(j^k = 10^{337}\), i.e. \(j=10^{\frac{337}{4}}\). Unfortunately, splitting each parameter range into \(j=100\) instead of \(j=50\) intervals already resulted in an unmanageable amount of data.
- 10.
Conceptually, the rAVS model also uses a vector sum on the LO. However, since the LO is simplified as a single point for the current model input, the rAVS model in fact does not compute a vector sum.
- 11.
A model that implements a shift from the RO to the LO without using a vector sum could be a modified rAVS model: One could change the direction of the vector and the reference direction. Computationally, this model computes the same output as the rAVS model.
- 12.
We thank an anonymous reviewer for suggesting this idea.
References
Burigo, M.: On the role of informativeness in spatial language comprehension. Ph.D. thesis, School of Psychology, University of Plymouth (2008)
Burigo, M., Coventry, K.R., Cangelosi, A., Lynott, D.: Spatial language and converseness. Q. J. Exp. Psychol. 69(12), 2319–2337 (2016). doi:10.1080/17470218.2015.11248942016
Burigo, M., Knoeferle, P.: Visual attention during spatial language comprehension. PLoS ONE 10(1), e0115758 (2015)
Burigo, M., Sacchi, S.: Object orientation affects spatial language comprehension. Cogn. Sci. 37(8), 1471–1492 (2013)
Cangelosi, A., Coventry, K.R., Rajapakse, R., Joyce, D., Bacon, A., Richards, L., Newstead, S.N.: Grounding language in perception: a connectionist model of spatial terms and vague quantifiers. Prog. Neural Process. 16, 47 (2005)
Canty, A., Ripley, B.: Boot: Bootstrap R (S-Plus) Functions (2015). R package version 1.3-15
Carlson, L.A., Logan, G.D.: Attention and spatial language. In: Itti, L., Rees, G., Tsotsos, J.K. (eds.) Neurobiology of Attention, pp. 330–336. Elsevier, Amsterdam (2005). Chap. 54
Carlson, L.A., Regier, T., Lopez, W., Corrigan, B.: Attention unites form and function in spatial language. Spat. Cogn. Comput. 6(4), 295–308 (2006)
Carlson-Radvansky, L.A., Covey, E.S., Lattanzi, K.M.: “What” effects on “where”: functional influences on spatial relations. Psychol. Sci. 10(6), 516–521 (1999)
Carrasco, M.: Visual attention: the past 25 years. Vis. Res. 51(13), 1484–1525 (2011)
CGAL: Computational geometry algorithms library. http://www.cgal.org
Coventry, K.R., Garrod, S.C.: Saying, Seeing, and Acting: The Psychological Semantics of Spatial Prepositions. Essays in Cognitive Psychology, Psychology Press, Taylor and Francis, Hove and New York (2004)
Coventry, K.R., Lynott, D., Cangelosi, A., Monrouxe, L., Joyce, D., Richardson, D.C.: Spatial language, visual attention, and perceptual simulation. Brain Lang. 112(3), 202–213 (2010)
Coventry, K.R., Prat Sala, M., Richards, L.: The interplay between geometry and function in the comprehension of over, under, above, and below. J. Mem. Lang. 44(3), 376–398 (2001)
Franconeri, S.L., Scimeca, J.M., Roth, J.C., Helseth, S.A., Kahn, L.E.: Flexible visual processing of spatial relationships. Cognition 122(2), 210–227 (2012)
Gapp, K.-P.: An empirically validated model for computing spatial relations. In: Wachsmuth, I., Rollinger, C.-R., Brauer, W. (eds.) KI 1995. LNCS, vol. 981, pp. 245–256. Springer, Heidelberg (1995). doi:10.1007/3-540-60343-3_41
Georgopoulos, A.P., Schwartz, A.B., Kettner, R.E.: Neuronal population coding of movement direction. Science 233, 1416–1419 (1986)
Hörberg, T.: Influences of form and function on the acceptability of projective prepositions in Swedish. Spat. Cogn. Comput. 8(3), 193–218 (2008)
Kelleher, J.D., Kruijff, G.J.M., Costello, F.J.: Proximity in context: an empirically grounded computational model of proximity for processing topological spatial expressions. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pp. 745–752. Association for Computational Linguistics (2006)
Kim, W., Navarro, D.J., Pitt, M.A., Myung, I.J.: An MCMC-based method of comparing connectionist models in cognitive science. Adv. Neural Inf. Process. Syst. 16, 937–944 (2004)
Kluth, T.: A C++ implementation of the reversed Attentional Vector Sum (rAVS) model. Bielefeld University (2016). doi:10.4119/unibi/2900103
Kluth, T., Burigo, M., Knoeferle, P.: Investigating the parameter space of cognitive models of spatial language comprehension. In: 5. Interdisziplinärer Workshop Kognitive Systeme, Bochum (2016)
Kluth, T., Burigo, M., Knoeferle, P.: Shifts of attention during spatial language comprehension: a computational investigation. In: Proceedings of the 8th International Conference on Agents and Artificial Intelligence, vol. 2, pp. 213–222. SCITEPRESS (2016). doi:10.5220/0005851202130222
Kluth, T., Burigo, M., Schultheis, H., Knoeferle, P.: Distinguishing cognitive models of spatial language understanding. In: Proceedings of the International Conference on Cognitive Modeling (2016). Poster presented at the ICCM 2016
Kluth, T., Burigo, M., Schultheis, H., Knoeferle, P.: Testing the predictions of cognitive models of spatial language comprehension (in preparation)
Kluth, T., Schultheis, H.: Attentional distribution and spatial language. In: Freksa, C., Nebel, B., Hegarty, M., Barkowsky, T. (eds.) Spatial Cognition 2014. LNCS (LNAI), vol. 8684, pp. 76–91. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11215-2_6
Lee, C., Rohrer, W.H., Sparks, D.L.: Population coding of saccadic eye movements by neurons in the superior colliculus. Nature 332, 357–360 (1988)
Logan, G.D.: Spatial attention and the apprehension of spatial relations. J. Exp. Psychol. Hum. Percept. Perform. 20(5), 1015 (1994)
Logan, G.D.: Linguistic and conceptual control of visual spatial attention. Cogn. Psychol. 28(2), 103–174 (1995)
Logan, G.D., Sadler, D.D.: A computational analysis of the apprehension of spatial relations. In: Bloom, P., Peterson, M.A., Nadel, L., Garrett, M.F. (eds.) Language and Space, pp. 493–530. The MIT Press, Cambridge (1996). Chap. 13
Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)
Navarro, D.J., Pitt, M.A., Myung, I.J.: Assessing the distinguishability of models and the informativeness of data. Cogn. Psychol. 49(1), 47–84 (2004)
Pitt, M.A., Kim, W., Navarro, D.J., Myung, J.I.: Global model analysis by parameter space partitioning. Psychol. Rev. 113(1), 57–83 (2006)
Pitt, M.A., Myung, I.J.: When a good fit can be bad. Trends Cogn. Sci. 6(10), 421–425 (2002)
Regier, T., Carlson, L.A.: Grounding spatial language in perception: an empirical and computational investigation. J. Exp. Psychol.: Gen. 130(2), 273–298 (2001)
Richter, M., Lins, J., Schneegans, S., Sandamirskaya, Y., Schöner, G.: Autonomous neural dynamics to test hypotheses in a model of spatial language. In: Bello, P., Guarini, M., Mc-Shane, M., Scassellati, B. (eds.) Proceedings of the 36th Annual Conference of the Cognitive Science Society, pp. 2847–2852. Cognitive Science Society, Austin (2014)
Roberts, S., Pashler, H.: How persuasive is a good fit? A comment on theory testing. Psychol. Rev. 107(2), 358–367 (2000)
Roth, J.C., Franconeri, S.L.: Asymmetric coding of categorical spatial relations in both language and vision. Front. Psychol. 3, Article No. 464 (2012)
Schultheis, H., Carlson, L.A.: Mechanisms of reference frame selection in spatial term use: computational and empirical studies. Cogn. Sci. (2015). doi:10.1111/cogs.12327
Schultheis, H., Singhaniya, A., Chaplot, D.S.: Comparing model comparison methods. In: Proceedings of the 35th Annual Conference of the Cognitive Science Society, pp. 1294–1299. Cognitive Science Society, Austin (2013)
Veksler, V.D., Myers, C.W., Gluck, K.A.: Model flexibility analysis. Psychol. Rev. 122(4), 755–769 (2015)
Wagenmakers, E.J., Ratcliff, R., Gomez, P., Iverson, G.J.: Assessing model mimicry using the parametric bootstrap. J. Math. Psychol. 48(1), 28–50 (2004)
Acknowledgments
This research was supported by the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG). The authors would also like to thank two anonymous reviewers for their useful comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Kluth, T., Burigo, M., Knoeferle, P. (2017). Modeling the Directionality of Attention During Spatial Language Comprehension. In: van den Herik, J., Filipe, J. (eds) Agents and Artificial Intelligence. ICAART 2016. Lecture Notes in Computer Science(), vol 10162. Springer, Cham. https://doi.org/10.1007/978-3-319-53354-4_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-53354-4_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-53353-7
Online ISBN: 978-3-319-53354-4
eBook Packages: Computer ScienceComputer Science (R0)