Experimental Brain Research

, Volume 234, Issue 8, pp 2415–2431 | Cite as

Anticipatory eye fixations reveal tool knowledge for tool interaction

  • Anna BelardinelliEmail author
  • Marissa Barabas
  • Marc Himmelbach
  • Martin V. Butz
Research Article


Action-oriented eye-tracking studies have shown that eye fixations reveal much about current behavioral intentions. The eyes typically fixate those positions of a tool or an object where the fingers will be placed next, or those positions in a scene, where obstacles need to be avoided to successfully reach or transport a tool or object. Here, we asked to what extent eye fixations can also reveal active cognitive inference processes, which are expected to integrate bottom-up visual information with internal knowledge for planning suitable object interactions task-dependently. In accordance to the available literature, we expected that task-relevant knowledge will include sensorimotor, semantic, and mechanical aspects. To investigate if and in which way this internal knowledge influences eye fixation behavior while planning an object interaction, we presented pictures of familiar and unfamiliar tools and instructed participants to either pantomime ‘lifting’ or ‘using’ the respective tool. When confronted with unfamiliar tools, participants fixated the tool’s effector part closer and longer in comparison with familiar tools. This difference was particularly prominent during ‘using’ trials when compared with ‘lifting’ trials. We suggest that this difference indicates that the brain actively extracts mechanical information about the unknown tool in order to infer its appropriate usage. Moreover, the successive fixations over a trial indicate that a dynamic, task-oriented, active cognitive process unfolds, which integrates available tool knowledge with visually gathered information to plan and determine the currently intended tool interaction.


Tool use Sensorimotor/mechanical knowledge Vision for action Anticipation Eye movements 



During this study A.B. was supported by the Institutional Strategy of the University of Tübingen (Deutsche Forschungsgemeinschaft, ZUK 63). M.B. and M.H. received funding from the DFG (HI 1372/2-1).


  1. Armbrüster C, Spijkers W (2006) Movement planning in prehension: do intended actions influence the initial reach and grasp movement? Mot Control 10:311–329Google Scholar
  2. Baumard J, Osiurak F, Lesourd M, Le Gall D (2014) Tool use disorders after left brain damage. Front Psychol 5:473CrossRefPubMedPubMedCentralGoogle Scholar
  3. Belardinelli A, Herbort O, Butz MV (2015) Goal-oriented gaze strategies afforded by object interaction. Vis Res 106:47–57CrossRefPubMedGoogle Scholar
  4. Belardinelli A, Stepper MY, Butz MV (2016) It’s in the eyes: planning precise manual actions before execution. J Vis 16(1):18. doi: 10.1167/16.1.18 CrossRefPubMedGoogle Scholar
  5. Brouwer AM, Franz VH, Gegenfurtner KR (2009) Differences in fixations between grasping and viewing objects. J Vis. doi: 10.1167/9.1.18 PubMedCentralGoogle Scholar
  6. Cavina-Pratesi C, Hesse C (2013) Why do the eyes prefer the index finger? Simultaneous recording of eye and hand movements during precision grasping. J Vis. doi: 10.1167/13.5.15 PubMedGoogle Scholar
  7. Cavina-Pratesi C, Kuhn G, Ietswaart M, Milner AD (2011) The magic grasp: motor expertise in deception. PLoS One. doi: 10.1371/journal.pone.0016568 PubMedPubMedCentralGoogle Scholar
  8. Foerster RM, Carbone E, Koesling H, Schneider WX (2011) Saccadic eye movements in a high-speed bimanual stacking task: changes of attentional control during learning and automatization. J Vis 11(7):1–16. doi: 10.1167/11.7.9 CrossRefGoogle Scholar
  9. Fukutake T (2003) Apraxia of tool use: an autopsy case of biparietal infarction. Eur Neurol 49:45–52CrossRefPubMedGoogle Scholar
  10. Goldenberg G, Hagmann S (1998) Tool use and mechanical problem solving in apraxia. Neuropsychologia 36:581–589CrossRefPubMedGoogle Scholar
  11. Goodale MA, Jakobson LS, Keillor JM (1994) Differences in the visual control of pantomimed and natural grasping movements. Neuropsychologia 32:1159–1178CrossRefPubMedGoogle Scholar
  12. Halsband U, Schmitt J, Weyers M, Binkofski F, Grützner G, Freund HJ (2001) Recognition and imitation of pantomimed motor acts after unilateral parietal and premotor lesions: a perspective on apraxia. Neuropsychologia 39:200–216CrossRefPubMedGoogle Scholar
  13. Hayhoe MM, Shrivastava A, Mruczek R, Pelz JB (2003) Visual memory and motor planning in a natural task. J Vis. doi: 10.1167/3.1.6 Google Scholar
  14. Herbort O, Butz MV (2011) Habitual and goal-directed factors in (everyday) object handling. Exp Brain Res 213:371–382CrossRefPubMedGoogle Scholar
  15. Herbort O, Butz MV (2012) The continuous end-state comfort effect: weighted integration of multiple biases. Psychol Res 76:345–363CrossRefPubMedGoogle Scholar
  16. Hermsdörfer J, Terlinden G, Muehlau M, Goldenberg G, Wohlschlaeger AM (2007) Neural representations of pantomimed and actual tool use: evidence from an event-related fMRI study. Neuroimage 36(Supplement 2):T109–T118CrossRefPubMedGoogle Scholar
  17. Hodges JR, Spatt J, Patterson K (1999) “What” and “how”: evidence for the dissociation of object knowledge and mechanical problem-solving skills in the human brain. Proc Natl Acad Sci USA 96:9444–9448CrossRefPubMedPubMedCentralGoogle Scholar
  18. Hodges JR, Bozeat S, Lambon Ralph MA, Patterson K, Spatt J (2000) The role of conceptual knowledge in object use evidence from semantic dementia. Brain 123:1913–1925CrossRefPubMedGoogle Scholar
  19. Johansson RS, Westling G, Backstrom A, Flanagan JR (2001) Eye–hand coordination in object manipulation. J Neurosci 21:6917–6932PubMedGoogle Scholar
  20. Króliczak G, Cavina-Pratesi C, Goodman DA, Culham JC (2007) What does the brain do when you fake it? An FMRI study of pantomimed and real grasping. J Neurophysiol 97:2410–2422CrossRefPubMedGoogle Scholar
  21. Kwok R, Braddick O (2003) When does the Titchener Circles illusion exert an effect on grasping? Two- and three-dimensional targets. Neuropsychologia 41:932–940CrossRefPubMedGoogle Scholar
  22. Land MF (2009) Vision, eye movements, and natural behavior. Vis Neurosci 26:51–62CrossRefPubMedGoogle Scholar
  23. Lewis JW (2006) Cortical networks related to human use of tools. Neuroscientist 12:211–231CrossRefPubMedGoogle Scholar
  24. Motomura N, Yamadori A (1994) A case of ideational apraxia with impairment of object use and preservation of object pantomime. Cortex 30:167–170CrossRefPubMedGoogle Scholar
  25. Myachykov A, Ellis R, Cangelosi A, Fischer MH (2013) Visual and linguistic cues to graspable objects. Exp Brain Res 229(4):545–599CrossRefPubMedGoogle Scholar
  26. Noppeney U (2008) The neural systems of tool and action semantics: a perspective from functional imaging. J Physiol Paris 102:40–49CrossRefPubMedGoogle Scholar
  27. Osiurak F, Jarry C, Allain P, Aubin G, Etcharry-Bouyx F, Richard I, Le Gall D (2009) Unusual use of objects after unilateral brain damage. The technical reasoning model. Cortex 45:769–783CrossRefPubMedGoogle Scholar
  28. Osiurak F, Jarry C, Le Gall D (2010) Grasping the affordances, understanding the reasoning: toward a dialectical theory of human tool use. Psychol Rev 117:517–540CrossRefPubMedGoogle Scholar
  29. Roberts KL, Humphreys GW (2011) Action- related objects influence the distribution of visuo-spatial attention. Q J Exp Psychol 64(4):669–688CrossRefGoogle Scholar
  30. Rosenbaum DA, Marchak F, Barnes HJ, Vaughan J, Slotta JD, Jorgensen MJ (1990) Constraints for action selection: overhand versus underhand grips. In: Jeannerod M (ed) Attention and performance, vol XIII. Lawrence Erlbaum Associates, London, pp 321–345Google Scholar
  31. Rosenbaum DA, Chapman KM, Weigelt M, Weiss DJ, van der Wel R (2012) Cognition, action, and object manipulation. Psychol Bull 138:924–946CrossRefPubMedPubMedCentralGoogle Scholar
  32. Salvucci D, Goldberg J (2000) Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 symposium on eye tracking research & applications, pp 71–78. doi:  10.1145/355017.355028
  33. Sartori L, Straulino E, Castiello U (2011) How objects are grasped: the interplay between affordances and end-goals. PLoS One. doi: 10.1371/journal.pone.0025203 Google Scholar
  34. van der Linden L, Mathôt S, Vitu F (2015) The role of object affordances and center of gravity in eye movements toward isolated daily-life objects. J Vis 15(5):8. doi: 10.1167/15.5.8 CrossRefPubMedGoogle Scholar
  35. Võ MLH, Henderson JM (2010) The time course of initial scene processing for eye movement guidance in natural scene search. J Vis 10(3):14. doi: 10.1167/10.3.14 CrossRefPubMedGoogle Scholar
  36. Westwood DA, Danckert J, Servos P, Goodale M (2002) Grasping two-dimensional images and three-dimensional objects in visual-form agnosia. Exp Brain Res 144:262–267CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Anna Belardinelli
    • 1
    Email author
  • Marissa Barabas
    • 2
  • Marc Himmelbach
    • 2
  • Martin V. Butz
    • 1
    • 3
  1. 1.Cognitive Modeling, Department of Computer Science, Faculty of ScienceEberhard Karls University of TübingenTübingenGermany
  2. 2.Division of Neuropsychology, Center for Neurology, Hertie-Institute for Clinical Brain ResearchEberhard Karls University of TübingenTübingenGermany
  3. 3.Department of Psychology, Faculty of ScienceEberhard Karls University of TübingenTübingenGermany

Personalised recommendations