An Eye-Tracking Study of Exploitations of Spatial Constraints in Diagrammatic Reasoning

  • Atsushi Shimojima
  • Yasuhiro Katagiri
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5223)


The semantic studies of diagrammatic notations [1,2,3] have revealed that so-called “perceptual,” “non-deductive,” or “emergent” effects of diagrams [4,5,6,7] are all rooted in a common inferential process, namely, the exploitation of spatial constraints on graphical structures. Thus, theoretically, this process is a key factor in inference with diagrams, explaining the oft-observed unburdening of the inferential load. In the present study, we inspect the empirical basis of this theoretical suggestion. Eye-movements were recorded while the participants were engaged in three-term transitive inference problems. They were provided with simple positions diagrams, on which we can define positions that should be fixated if the hypothesized inferential process occurs. Our analysis has revealed that the participants could exploit spatial constraints on graphical structures even when (1) they were not in the position of actually manipulating diagrams, and (2) the semantic rule of the provided diagrams did not match their preference. These findings indicate that the hypothesized practice is in fact robust, with a potential to broadly account for the inferential advantage of diagrams.


Problem Category Spatial Constraint Transitive Relation Semantic Rule Constraint Exploitation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barwise, J., Etchemendy, J.: Visual information and valid reasoning. In: Allwein, G., Barwise, J. (eds.) Logical Reasoning with Diagrams, pp. 3–25. Oxford University Press, Oxford (1990)Google Scholar
  2. 2.
    Shimojima, A.: On the Efficacy of Representation. PhD thesis, Department of Philosophy, Indiana University (1996b)Google Scholar
  3. 3.
    Stenning, K., Lemon, O.: Aligning logical and psychological perspectives on diagrammatic reasoning. Artificial Intelligence Review 15(1-2), 29–62 (2001)MATHCrossRefGoogle Scholar
  4. 4.
    Larkin, J.H., Simon, H.A.: Why a diagram is (sometimes) worth ten thousand words. In: Glasgow, J., Narayanan, N.H., Chanrasekaran, B. (eds.) Diagrammatic Reasoning: Cognitive and Computational Perspectives, pp. 69–109. AAAI Press, Menlo Park (1987)Google Scholar
  5. 5.
    Lindsay, R.K.: Images and inference. In: Glasgow, J.I., Narayanan, N.H., Chandrasekaran, B. (eds.) Diagrammatic Reasoning: Cognitive and Computational Perspectives, pp. 111–135. The MIT Press and the AAAI Press, Cambridge, Menlo Park (1988)Google Scholar
  6. 6.
    Kulpa, Z.: From picture processing to interval diagrams. Technical report, Instytut Podstawowych Problemów Techniki Polskiej Akademii Nauk, Warszawa (2003)Google Scholar
  7. 7.
    Chandrasekaran, B., Kurup, U., Banerjee, B., Josephson, J.R., Winkler, R.: An architecture for problem solving in diagrams. In: Diagrammatic Representation and Inference: Third International Conference, Diagrams 2004, pp. 151–165 (2004)Google Scholar
  8. 8.
    Sloman, A.: Interactions between philosophy and ai: the role of intuition and non-logical reasoning in intelligence. Artificial Intelligence 2, 209–225 (1971)CrossRefGoogle Scholar
  9. 9.
    Narayanan, N.H., Suwa, M., Motoda, H.: Hypothesizing behaviors from device diagrams. In: Glasgow, J., Narayanan, N.H., Chanrasekaran, B. (eds.) Diagrammatic Reasoning: Cognitive and Computational Perspectives, pp. 501–534. AAAI Press, Menlo Park (1995)Google Scholar
  10. 10.
    Trafton, J.G., Trickett, S.B.: A new model of graph and visualization usage. In: Proceedings of the Twenty-Third Annual Conference of the Cognitive Science Society, pp. 1048–1053 (2001)Google Scholar
  11. 11.
    Shimojima, A., Fukaya, T.: Do we really reason about the picture as the referent? In: Proceedings of the Twenty-Fifth Annual Conference of the Cognitive Science Society, pp. 1076–1081 (2003)Google Scholar
  12. 12.
    Yoon, D., Narayanan, H.: Mental imagery in problem solving: An eye tracking study. In: Proceedings of the 2004 symposium on Eye tracking research & applications, pp. 77–84 (2004)Google Scholar
  13. 13.
    Kozhevnikov, M., Motes, M.A., Hegarty, M.: Spatial visualization in physics problem solving. Cognitive Science 31(4), 549–579 (2007)Google Scholar
  14. 14.
    Ullman, S.: Visual routines. Cognition 18, 97–159 (1984)CrossRefGoogle Scholar
  15. 15.
    Chapman, D.: Penguins can make cake. AI Magazine 10(4), 45–50 (1989)Google Scholar
  16. 16.
    Pylyshyn, Z.W.: The role of location indexes in spatial perception: A sketch of the finst spatial-index model. Cognition 32, 65–97 (1989)CrossRefGoogle Scholar
  17. 17.
    Ballard, D.H., Hayhoe, M.M., Pook, P.K., Rao, R.P.N.: Deictic codes for the embodiment of cognition. Behavioral and Brain Sciences 20(4), 723–767 (2001)Google Scholar
  18. 18.
    Pylyshyn, Z.W.: Seeing and Visualizing: It’s Not What You Think. The MIT Press, Cambridge (2003)Google Scholar
  19. 19.
    Spivey, M.J., Richardson, D.C., Fitneva, S.A.: Thinking outside the brain: Spatial indices to visual and linguistic information. In: Henderson, J.M., Ferreira, F. (eds.) The Interface of Language, Vision, and Action: Eye Movements and the Visual World, pp. 161–189. Psychology Press, New York (2004)Google Scholar
  20. 20.
    Knauff, M., Johnson-Laird, P.N.: Visual imagery can impede reasoning. Memory and Cognition 30(3), 363–371 (2002)Google Scholar
  21. 21.
    Schwartz, D.L.: Reasoning about the referent of a picture versus reasoning about the picture as the referent: an effect of visual realism. Memory and Cognition 23(6), 709–722 (1995)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Atsushi Shimojima
    • 1
  • Yasuhiro Katagiri
    • 2
  1. 1.Faculty of Culture and Information ScienceDoshisha UniversityJapan
  2. 2.Department of Media ArchitectureFuture UniversityHakodateJapan

Personalised recommendations