Abstract
Attentional orienting towards others’ gaze direction or pointing has been well investigated in laboratory conditions. However, less is known about the operation of attentional mechanisms in online naturalistic social interaction scenarios. It is equally plausible that following social directional cues (gaze, pointing) occurs reflexively, and/or that it is influenced by top-down cognitive factors. In a mobile eye-tracking experiment, we show that under natural interaction conditions, overt attentional orienting is not necessarily reflexively triggered by pointing gestures or a combination of gaze shifts and pointing gestures. We found that participants conversing with an experimenter, who, during the interaction, would play out pointing gestures as well as directional gaze movements, continued to mostly focus their gaze on the face of the experimenter, demonstrating the significance of attending to the face of the interaction partner—in line with effective top-down control over reflexive orienting of attention in the direction of social cues.
Similar content being viewed by others
Data availability statement
The data are not publicly available due to them containing information that could compromise research participant privacy/consent, but upon reasonable cause a request may be made to the author [JK].
References
Awh, E., Belopolsky, A. V., & Theeuwes, J. (2012). Top-down versus bottom-up attentional control: a failed theoretical dichotomy. Trends in Cognitive Sciences, 16(8), 437–443.
Bacon, W. F., & Egeth, H. E. (1994). Overriding stimulus-driven attentional capture. Perception & Psychophysics, 55(5), 485–496.
Baron-Cohen, S. (1995). Mindblindness: An essay on autism and the theory of mind. Boston: MIT Press/Bradford Books.
Birmingham, E., Bischof, W. F., & Kingstone, A. (2009). Get real! Resolving the debate about equivalent social stimuli. Visual Cognition, 17(6–7), 904–924.
Broekens, J., Heerink, M., & Rosendal, H. (2009). Assistive social robots in elderly care: A review. Gerontechnology, 8(2), 94–103.
Cabibihan, J.-J., Javed, H., Ang, M., Jr., & Aljunied, S. (2013). Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism. International Journal of Social Robotics, 5(4), 593–618.
Chapman, C. D., Benedict, C., & Schiöth, H. B. (2018). Experimenter gender and replicability in science. Science Advances, 4(1), e1701427.
Cole, G. G., Smith, D. T., & Atkinson, M. A. (2015). Mental state attribution and the gaze cueing effect. Attention, Perception, & Psychophysics, 77(4), 1105–1115.
Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., & Baron-Cohen, S. (1999). Gaze perception triggers reflexive visuospatial orienting. Visual Cognition, 6(5), 509–540.
Ellsworth, P. C., & Ludwig, L. M. (1972). Visual Behavior in social interaction. Journal of Communication, 22(4), 375–403.
Foulsham, T., Walker, E., & Kingstone, A. (2011). The where, what and when of gaze allocation in the lab and the natural environment. Vision Research, 51(17), 1920–1931.
Freeth, M., Foulsham, T., & Kingstone, A. (2013). What affects social attention? Social presence, eye contact and autistic traits. PLoS One, 8(1), e53286.
Friesen, C. K., & Kingstone, A. (1998). The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin & Review, 5(3), 490–495.
Friesen, C. K., Moore, C., & Kingstone, A. (2005). Does gaze direction really trigger a reflexive shift of spatial attention? Brain and Cognition, 57(1), 66–69.
Gluckman, M., & Johnson, S. P. (2013). Attentional capture by social stimuli in young infants. Frontiers in Psychology, 4, 527.
Hamilton, S. J. (2017). The effects of pointing gestures on visual attention. University Honors Program Theses, 243.
Hayward, D. A., Voorhies, W., Morris, J. L., Capozzi, F., & Ristic, J. (2017). Staring reality in the face: A comparison of social attention across laboratory and real world measures suggests little common ground. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale, 71(3), 212.
Kajopoulos, J. (2019) Social cognition and robotics. (Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences). University library repository of the Ludwig-Maximilians-Universität München. https://edoc.ub.uni-muenchen.de/25034/.
Kajopoulos, J., Wong, A. H. Y., Yuen, A. W. C., Dung, T. A., Kee, T. Y., & Wykowska, A. (2015). Robot-assisted training of joint attention skills in children diagnosed with autism. Social Robotics: 7th International Conference, ICSR 2015 (pp. 296–305). Paris, France.
Kerzel, D., Zarian, L., & Souto, D. (2009). Involuntary cueing effects on accuracy measures: Stimulus and task dependence. Journal of Vision, 9(11), 1–16.
Kleinke, C. L. (1986). Gaze and eye contact: a research review. Psychological Bulletin, 100(1), 78.
Kompatsiari, K., Ciardo, F., Tikhanoff, V., Metta, A., & Wykowska, A. (2018a). On the role of eye contact in gaze cueing. Scientific Reports, 8, 17842.
Kompatsiari, K., Pérez-Osorio, J., De Tommaso, D., Metta, G., Wykowska, A. (2018b). Neuroscientifically-grounded research for improved human-robot interaction. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3403–3408). Madrid, Spain.
Kuhn, G., & Kingstone, A. (2009). Look away! Eyes and arrows engage oculomotor responses automatically. Attention, Perception, & Psychophysics, 71(2), 314–327.
Langton, S. R., & Bruce, V. (2000). You must see the point: automatic processing of cues to the direction of social attention. Journal of Experimental Psychology: Human Perception and Performance, 26(2), 747.
Macdonald, R. G., & Tatler, B. W. (2018). Gaze in a real-world social interaction: A dual eye-tracking study. Quarterly Journal of Experimental Psychology, 71(10), 2162–2173.
Moore, C., & Dunham, P. J. (2014). Joint attention: Its origins and role in development. New York: Psychology Press.
Müller, H. J., Geyer, T., Zehetleitner, M., & Krummenacher, J. (2009). Attentional capture by salient color singleton distractors is modulated by top-down dimensional set. Journal of Experimental Psychology: Human Perception and Performance, 35(1), 1–16.
Mundy, P. (2017). A review of joint attention and social-cognitive brain systems in typical development and autism spectrum disorder. European Journal of Neuroscience, 47(6), 497–514.
Perez-Osorio, J., Müller, H. J., Wiese, E., & Wykowska, A. (2015). Gaze following is modulated by expectations regarding others’ action goals. PLoS One, 10(11), e0143614.
Prinzmetal, W., McCool, C., & Park, S. (2005). Attention: reaction time and accuracy reveal different mechanisms. Journal of Experimental Psychology: General, 134(1), 73.
Ricciardelli, P., Bricolo, E., Aglioti, S. M., & Chelazzi, L. (2002). My eyes want to look where your eyes are looking: exploring the tendency to imitate another individual’s gaze. NeuroReport, 13(17), 2259–2264.
Ristic, J., & Kingstone, A. (2005). Taking control of reflexive social attention. Cognition International Journal of Cognitive Science, 94(3), B55–B65.
Schellen, E., & Wykowska, A. (2019). Intentional mindset toward robots—open questions and methodological challenges. Frontiers in Robotics and AI, 5, 139.
Schilbach, L., Timmermans, B., Reddy, V., Costall, A., Bente, G., Schlicht, T., & Vogeley, K. (2013). Toward a second-person neuroscience. Behavioral and Brain Sciences, 36(4), 393–414.
Skarratt, P. A., Cole, G. G., & Kingstone, A. (2010). Social inhibition of return. Acta Psychologica, 134(1), 48–54.
Teufel, C., Fletcher, P. C., & Davis, G. (2010). Seeing other minds: attributed mental states influence perception. Trends in Cognitive Sciences, 14(8), 376–382.
Theeuwes, J. (1994). Stimulus-driven capture and attentional set: Selective search for color and visual abrupt onsets. Journal of Experimental Psychology: Human Perception and Performance, 20(4), 799–806.
Theeuwes, J. (2004). Top-down search strategies cannot override attentional capture. Psychonomic Bulletin & Review, 11(1), 65–70.
Theeuwes, J. (2010). Top-down and bottom-up control of visual selection. Acta Psychologica, 135(2), 77–99.
Theeuwes, J., Atchley, P., & Kramer, A. F. (2000). On the time course of top-down and bottom-up control of visual attention. In S. M. J. Driver (Ed.), Attention and Performance, 18 (pp. 105–125). Cambridge: MIT Press.
Tomasello, M. (1995). Joint attention as social cognition. In C. Moore & P. J. Dunham (Eds.), Joint attention: Its origins and role in development (pp. 103–130). USA: Lawrence Erlbaum Associates Inc.
Wiese, E., Metta, G., & Wykowska, A. (2017). Robots as intentional agents: Using neuroscientific methods to make robots appear more social. Frontiers in Psychology, 8, 1663.
Wiese, E., Wykowska, A., & Müller, H. J. (2014). What we observe is biased by what other people tell us: Beliefs about the reliability of gaze behavior modulate attentional orienting to gaze cues. PLoS One, 9(4), e94529.
Wiese, E., Wykowska, A., Zwickel, J., & Müller, H. J. (2012). I see what you mean: how attentional selection is shaped by ascribing intentions to others. PLoS One, 7(9), e45391.
Wiese, E., Zwickel, J., & Müller, H. J. (2013). The importance of context information for the spatial specificity of gaze cueing. Attention, Perception, & Psychophysics, 75(5), 967–982.
Willemse, C., & Wykowska, A. (2019). In natural interaction with embodied robots we prefer it when they follow our gaze: a gaze-contingent mobile eyetracking study. Philosophical Transactions of the Royal Society B, 374, 20180036.
Wykowska, A., Chaminade, T., & Cheng, G. (2016). Embodied artificial agents for understanding human social cognition. Philosophical Transactions of the Royal Society London: B. Biological Sciences, 371, 20150375.
Wykowska, A., Kajopoulos, J., Obando-Leitón, M., Chauhan, S., Cabibihan, J.-J., & Cheng, G. (2015). Humans are well tuned to detecting agents among non-agents: Examining the sensitivity of human perception to behavioral characteristics of intentional systems. International Journal of Social Robotics, 7(5), 767–781.
Wykowska, A., & Schubö, A. (2010). On the temporal relation of top-down and bottom-up mechanisms during guidance of attention. Journal of Cognitive Neuroscience, 22(4), 640–654.
Wykowska, A., Wiese, E., Prosser, A., & Müller, H. J. (2014). Beliefs about the minds of others influence how we process sensory information. PLoS One, 9(4), e94339.
Zehetleitner, M., Hegenloh, M., & Müller, H. J. (2011). Visually guided pointing movements are driven by the salience map. Journal of Vision, 11(1):1–18.
Zehetleitner, M., Krummenacher, J., & Müller, H. J. (2009). The detection of feature singletons defined in two dimensions is based on salience summation, rather than on serial exhaustive or interactive race architectures. Attention, Perception, & Psychophysics, 71(8), 1739–1759.
Acknowledgements
We thank the Japan Society for the Promotion of Science (JSPS) for supporting Jasmin Kajopoulos as an International Research Fellow. We are grateful towards Ayano Okoso and several students from the OPU, who supported us with data collection and participant recruitment. The content of this article and the figures of this article have been extracted, reproduced and partly adapted (see figure captions in particular) with permission from the doctoral thesis (mainly from Chapter 1) of Kajopoulos, J. (2019), “Social cognition and robotics” [Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences] published at the university library repository of the Ludwig-Maximilians-Universität München, accessible at https://edoc.ub.uni-muenchen.de/25034/.
Funding
This research was supported by the Deutsche Forschungsgemeinschaft (WY-122/1-1, awarded to AW; MU 773/16-2, awarded to HJM ), the Japan Society for the Promotion of Science (JSPS Summer Program, awarded to JK), the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program: Grant ERC-2016-StG-715058 awarded to AW.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Kajopoulos, J., Cheng, G., Kise, K. et al. Focusing on the face or getting distracted by social signals? The effect of distracting gestures on attentional focus in natural interaction. Psychological Research 85, 491–502 (2021). https://doi.org/10.1007/s00426-020-01383-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00426-020-01383-4