Skip to main content

Modeling the Directionality of Attention During Spatial Language Comprehension

Part of the Lecture Notes in Computer Science book series (LNAI,volume 10162)


It is known that the comprehension of spatial prepositions involves the deployment of visual attention. For example, consider the sentence “The salt is to the left of the stove”. Researchers [29, 30] have theorized that people must shift their attention from the stove (the reference object, RO) to the salt (the located object, LO) in order to comprehend the sentence. Such a shift was also implicitly assumed in the Attentional Vector Sum (AVS) model by [35], a cognitive model that computes an acceptability rating for a spatial preposition given a display that contains an RO and an LO. However, recent empirical findings showed that a shift from the RO to the LO is not necessary to understand a spatial preposition ([3], see also [15, 38]). In contrast, these findings suggest that people perform a shift in the reverse direction (i.e., from the LO to the RO). Thus, we propose the reversed AVS (rAVS) model, a modified version of the AVS model in which attention shifts from the LO to the RO. We assessed the AVS and the rAVS model on the data from [35] using three model simulation methods. Our simulations show that the rAVS model performs as well as the AVS model on these data while it also integrates the recent empirical findings. Moreover, the rAVS model achieves its good performance while being less flexible than the AVS model. (This article is an updated and extended version of the paper [23] presented at the 8th International Conference on Agents and Artificial Intelligence in Rome, Italy. The authors would like to thank Holger Schultheis for helpful discussions about the additional model simulation.)


  • Spatial language
  • Spatial prepositions
  • Cognitive modeling
  • Model flexibility
  • Visual attention

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. 1.

    Apart from the AVS model, a range of other computational models of spatial language processing were also proposed, e.g., [5, 16, 19, 36, 39].

  2. 2.

    In the case of other prepositions, the corresponding part of the RO is chosen for the location of the focus (e.g., the focus lies on the bottom of the RO for below).

  3. 3.

    [35, p. 276]: “A central feature of this [angular] characterization of spatial term acceptability is that it is dependent only on the direction, not the length, of the vector connecting the landmark [RO] to the trajector [LO].”

  4. 4.

    Therefore, the rAVS model does not need to compute a vector sum nor does it rely on an underlying attentional distribution and thus has a lower computational complexity. This lower computational complexity, however, originates from the simplification of the LO. Accordingly, these considerations are also only valid for simplified LOs.

  5. 5.

    We thank Terry Regier and Laura Carlson for sharing these data.

  6. 6.

    Following [41] we favor the term model flexibility over model complexity (used by, e.g., [34]). Both terms mean the same.

  7. 7.

    The prediction error is also a measure of model generalizability, the property of a model to account for new empirical data, see [34].

  8. 8.

    Note that it is difficult to call data “non-empirical”. You can tell what people do, but it is harder to tell what people do not do. Given the right study design, previously considered “non-empirical” data might become empirical. Nevertheless, the greater the range of model predictions, the higher the probability that some of these predictions are at least implausible (or conflict with other generated predictions).

  9. 9.

    One could argue that we did not compute enough model predictions to define 10 cells on each dimension of the data space. However, if we want to follow the suggestion from [41] and use \(\root n \of {j^k}\) cells, we would need more model predictions due to the high-dimensional data space (\(n= 337\)), namely \(j^k = 10^{337}\), i.e. \(j=10^{\frac{337}{4}}\). Unfortunately, splitting each parameter range into \(j=100\) instead of \(j=50\) intervals already resulted in an unmanageable amount of data.

  10. 10.

    Conceptually, the rAVS model also uses a vector sum on the LO. However, since the LO is simplified as a single point for the current model input, the rAVS model in fact does not compute a vector sum.

  11. 11.

    A model that implements a shift from the RO to the LO without using a vector sum could be a modified rAVS model: One could change the direction of the vector and the reference direction. Computationally, this model computes the same output as the rAVS model.

  12. 12.

    We thank an anonymous reviewer for suggesting this idea.


  1. Burigo, M.: On the role of informativeness in spatial language comprehension. Ph.D. thesis, School of Psychology, University of Plymouth (2008)

    Google Scholar 

  2. Burigo, M., Coventry, K.R., Cangelosi, A., Lynott, D.: Spatial language and converseness. Q. J. Exp. Psychol. 69(12), 2319–2337 (2016). doi:10.1080/17470218.2015.11248942016

    CrossRef  Google Scholar 

  3. Burigo, M., Knoeferle, P.: Visual attention during spatial language comprehension. PLoS ONE 10(1), e0115758 (2015)

    CrossRef  Google Scholar 

  4. Burigo, M., Sacchi, S.: Object orientation affects spatial language comprehension. Cogn. Sci. 37(8), 1471–1492 (2013)

    CrossRef  Google Scholar 

  5. Cangelosi, A., Coventry, K.R., Rajapakse, R., Joyce, D., Bacon, A., Richards, L., Newstead, S.N.: Grounding language in perception: a connectionist model of spatial terms and vague quantifiers. Prog. Neural Process. 16, 47 (2005)

    Google Scholar 

  6. Canty, A., Ripley, B.: Boot: Bootstrap R (S-Plus) Functions (2015). R package version 1.3-15

    Google Scholar 

  7. Carlson, L.A., Logan, G.D.: Attention and spatial language. In: Itti, L., Rees, G., Tsotsos, J.K. (eds.) Neurobiology of Attention, pp. 330–336. Elsevier, Amsterdam (2005). Chap. 54

    CrossRef  Google Scholar 

  8. Carlson, L.A., Regier, T., Lopez, W., Corrigan, B.: Attention unites form and function in spatial language. Spat. Cogn. Comput. 6(4), 295–308 (2006)

    Google Scholar 

  9. Carlson-Radvansky, L.A., Covey, E.S., Lattanzi, K.M.: “What” effects on “where”: functional influences on spatial relations. Psychol. Sci. 10(6), 516–521 (1999)

    CrossRef  Google Scholar 

  10. Carrasco, M.: Visual attention: the past 25 years. Vis. Res. 51(13), 1484–1525 (2011)

    CrossRef  Google Scholar 

  11. CGAL: Computational geometry algorithms library.

  12. Coventry, K.R., Garrod, S.C.: Saying, Seeing, and Acting: The Psychological Semantics of Spatial Prepositions. Essays in Cognitive Psychology, Psychology Press, Taylor and Francis, Hove and New York (2004)

    Google Scholar 

  13. Coventry, K.R., Lynott, D., Cangelosi, A., Monrouxe, L., Joyce, D., Richardson, D.C.: Spatial language, visual attention, and perceptual simulation. Brain Lang. 112(3), 202–213 (2010)

    CrossRef  Google Scholar 

  14. Coventry, K.R., Prat Sala, M., Richards, L.: The interplay between geometry and function in the comprehension of over, under, above, and below. J. Mem. Lang. 44(3), 376–398 (2001)

    CrossRef  Google Scholar 

  15. Franconeri, S.L., Scimeca, J.M., Roth, J.C., Helseth, S.A., Kahn, L.E.: Flexible visual processing of spatial relationships. Cognition 122(2), 210–227 (2012)

    CrossRef  Google Scholar 

  16. Gapp, K.-P.: An empirically validated model for computing spatial relations. In: Wachsmuth, I., Rollinger, C.-R., Brauer, W. (eds.) KI 1995. LNCS, vol. 981, pp. 245–256. Springer, Heidelberg (1995). doi:10.1007/3-540-60343-3_41

    CrossRef  Google Scholar 

  17. Georgopoulos, A.P., Schwartz, A.B., Kettner, R.E.: Neuronal population coding of movement direction. Science 233, 1416–1419 (1986)

    CrossRef  Google Scholar 

  18. Hörberg, T.: Influences of form and function on the acceptability of projective prepositions in Swedish. Spat. Cogn. Comput. 8(3), 193–218 (2008)

    Google Scholar 

  19. Kelleher, J.D., Kruijff, G.J.M., Costello, F.J.: Proximity in context: an empirically grounded computational model of proximity for processing topological spatial expressions. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pp. 745–752. Association for Computational Linguistics (2006)

    Google Scholar 

  20. Kim, W., Navarro, D.J., Pitt, M.A., Myung, I.J.: An MCMC-based method of comparing connectionist models in cognitive science. Adv. Neural Inf. Process. Syst. 16, 937–944 (2004)

    Google Scholar 

  21. Kluth, T.: A C++ implementation of the reversed Attentional Vector Sum (rAVS) model. Bielefeld University (2016). doi:10.4119/unibi/2900103

  22. Kluth, T., Burigo, M., Knoeferle, P.: Investigating the parameter space of cognitive models of spatial language comprehension. In: 5. Interdisziplinärer Workshop Kognitive Systeme, Bochum (2016)

    Google Scholar 

  23. Kluth, T., Burigo, M., Knoeferle, P.: Shifts of attention during spatial language comprehension: a computational investigation. In: Proceedings of the 8th International Conference on Agents and Artificial Intelligence, vol. 2, pp. 213–222. SCITEPRESS (2016). doi:10.5220/0005851202130222

  24. Kluth, T., Burigo, M., Schultheis, H., Knoeferle, P.: Distinguishing cognitive models of spatial language understanding. In: Proceedings of the International Conference on Cognitive Modeling (2016). Poster presented at the ICCM 2016

    Google Scholar 

  25. Kluth, T., Burigo, M., Schultheis, H., Knoeferle, P.: Testing the predictions of cognitive models of spatial language comprehension (in preparation)

    Google Scholar 

  26. Kluth, T., Schultheis, H.: Attentional distribution and spatial language. In: Freksa, C., Nebel, B., Hegarty, M., Barkowsky, T. (eds.) Spatial Cognition 2014. LNCS (LNAI), vol. 8684, pp. 76–91. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11215-2_6

    Google Scholar 

  27. Lee, C., Rohrer, W.H., Sparks, D.L.: Population coding of saccadic eye movements by neurons in the superior colliculus. Nature 332, 357–360 (1988)

    CrossRef  Google Scholar 

  28. Logan, G.D.: Spatial attention and the apprehension of spatial relations. J. Exp. Psychol. Hum. Percept. Perform. 20(5), 1015 (1994)

    CrossRef  Google Scholar 

  29. Logan, G.D.: Linguistic and conceptual control of visual spatial attention. Cogn. Psychol. 28(2), 103–174 (1995)

    CrossRef  Google Scholar 

  30. Logan, G.D., Sadler, D.D.: A computational analysis of the apprehension of spatial relations. In: Bloom, P., Peterson, M.A., Nadel, L., Garrett, M.F. (eds.) Language and Space, pp. 493–530. The MIT Press, Cambridge (1996). Chap. 13

    Google Scholar 

  31. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)

    CrossRef  Google Scholar 

  32. Navarro, D.J., Pitt, M.A., Myung, I.J.: Assessing the distinguishability of models and the informativeness of data. Cogn. Psychol. 49(1), 47–84 (2004)

    CrossRef  Google Scholar 

  33. Pitt, M.A., Kim, W., Navarro, D.J., Myung, J.I.: Global model analysis by parameter space partitioning. Psychol. Rev. 113(1), 57–83 (2006)

    CrossRef  Google Scholar 

  34. Pitt, M.A., Myung, I.J.: When a good fit can be bad. Trends Cogn. Sci. 6(10), 421–425 (2002)

    CrossRef  Google Scholar 

  35. Regier, T., Carlson, L.A.: Grounding spatial language in perception: an empirical and computational investigation. J. Exp. Psychol.: Gen. 130(2), 273–298 (2001)

    CrossRef  Google Scholar 

  36. Richter, M., Lins, J., Schneegans, S., Sandamirskaya, Y., Schöner, G.: Autonomous neural dynamics to test hypotheses in a model of spatial language. In: Bello, P., Guarini, M., Mc-Shane, M., Scassellati, B. (eds.) Proceedings of the 36th Annual Conference of the Cognitive Science Society, pp. 2847–2852. Cognitive Science Society, Austin (2014)

    Google Scholar 

  37. Roberts, S., Pashler, H.: How persuasive is a good fit? A comment on theory testing. Psychol. Rev. 107(2), 358–367 (2000)

    CrossRef  Google Scholar 

  38. Roth, J.C., Franconeri, S.L.: Asymmetric coding of categorical spatial relations in both language and vision. Front. Psychol. 3, Article No. 464 (2012)

    Google Scholar 

  39. Schultheis, H., Carlson, L.A.: Mechanisms of reference frame selection in spatial term use: computational and empirical studies. Cogn. Sci. (2015). doi:10.1111/cogs.12327

  40. Schultheis, H., Singhaniya, A., Chaplot, D.S.: Comparing model comparison methods. In: Proceedings of the 35th Annual Conference of the Cognitive Science Society, pp. 1294–1299. Cognitive Science Society, Austin (2013)

    Google Scholar 

  41. Veksler, V.D., Myers, C.W., Gluck, K.A.: Model flexibility analysis. Psychol. Rev. 122(4), 755–769 (2015)

    CrossRef  Google Scholar 

  42. Wagenmakers, E.J., Ratcliff, R., Gomez, P., Iverson, G.J.: Assessing model mimicry using the parametric bootstrap. J. Math. Psychol. 48(1), 28–50 (2004)

    CrossRef  MathSciNet  MATH  Google Scholar 

Download references


This research was supported by the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG). The authors would also like to thank two anonymous reviewers for their useful comments and suggestions.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Thomas Kluth .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Kluth, T., Burigo, M., Knoeferle, P. (2017). Modeling the Directionality of Attention During Spatial Language Comprehension. In: van den Herik, J., Filipe, J. (eds) Agents and Artificial Intelligence. ICAART 2016. Lecture Notes in Computer Science(), vol 10162. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-53353-7

  • Online ISBN: 978-3-319-53354-4

  • eBook Packages: Computer ScienceComputer Science (R0)