Skip to main content
SpringerLink
Log in

Focusing on the face or getting distracted by social signals? The effect of distracting gestures on attentional focus in natural interaction

  • Original Article
  • Published:
Psychological Research Aims and scope Submit manuscript

Abstract

Attentional orienting towards others’ gaze direction or pointing has been well investigated in laboratory conditions. However, less is known about the operation of attentional mechanisms in online naturalistic social interaction scenarios. It is equally plausible that following social directional cues (gaze, pointing) occurs reflexively, and/or that it is influenced by top-down cognitive factors. In a mobile eye-tracking experiment, we show that under natural interaction conditions, overt attentional orienting is not necessarily reflexively triggered by pointing gestures or a combination of gaze shifts and pointing gestures. We found that participants conversing with an experimenter, who, during the interaction, would play out pointing gestures as well as directional gaze movements, continued to mostly focus their gaze on the face of the experimenter, demonstrating the significance of attending to the face of the interaction partner—in line with effective top-down control over reflexive orienting of attention in the direction of social cues.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Reproduced with permission from “Social cognition and robotics” [Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences], by J. Kajopoulos, 2019, University library repository of the Ludwig-Maximilians-Universität München. https://edoc.ub.uni-muenchen.de/25034/

Fig. 2

Reproduced with permission from “Social cognition and robotics” [Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences], by J. Kajopoulos, 2019, University library repository of the Ludwig-Maximilians-Universität München. https://edoc.ub.uni-muenchen.de/25034/

Fig. 3

Adapted with permission from “Social cognition and robotics” [Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences], by J. Kajopoulos, 2019, University library repository of the Ludwig-Maximilians-Universität München. https://edoc.ub.uni-muenchen.de/25034/

Fig. 4

Adapted with permission from “Social cognition and robotics” [Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences], by J. Kajopoulos, 2019, University library repository of the Ludwig-Maximilians-Universität München. https://edoc.ub.uni-muenchen.de/25034/

Fig. 5

Adapted with permission from “Social cognition and robotics” [Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences], by J. Kajopoulos, 2019, University library repository of the Ludwig-Maximilians-Universität München. https://edoc.ub.uni-muenchen.de/25034/

Fig. 6

Adapted with permission from “Social cognition and robotics” [Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences], by J. Kajopoulos, 2019, University library repository of the Ludwig-Maximilians-Universität München. https://edoc.ub.uni-muenchen.de/25034/

Fig. 7

Adapted with permission from “Social cognition and robotics” [Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences], by J. Kajopoulos, 2019, University library repository of the Ludwig-Maximilians-Universität München. https://edoc.ub.uni-muenchen.de/25034/

Similar content being viewed by others

Data availability statement

The data are not publicly available due to them containing information that could compromise research participant privacy/consent, but upon reasonable cause a request may be made to the author [JK].

References

  • Awh, E., Belopolsky, A. V., & Theeuwes, J. (2012). Top-down versus bottom-up attentional control: a failed theoretical dichotomy. Trends in Cognitive Sciences, 16(8), 437–443.

    PubMed  PubMed Central  Google Scholar 

  • Bacon, W. F., & Egeth, H. E. (1994). Overriding stimulus-driven attentional capture. Perception & Psychophysics, 55(5), 485–496.

    Google Scholar 

  • Baron-Cohen, S. (1995). Mindblindness: An essay on autism and the theory of mind. Boston: MIT Press/Bradford Books.

    Google Scholar 

  • Birmingham, E., Bischof, W. F., & Kingstone, A. (2009). Get real! Resolving the debate about equivalent social stimuli. Visual Cognition, 17(6–7), 904–924.

    Google Scholar 

  • Broekens, J., Heerink, M., & Rosendal, H. (2009). Assistive social robots in elderly care: A review. Gerontechnology, 8(2), 94–103.

    Google Scholar 

  • Cabibihan, J.-J., Javed, H., Ang, M., Jr., & Aljunied, S. (2013). Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism. International Journal of Social Robotics, 5(4), 593–618.

    Google Scholar 

  • Chapman, C. D., Benedict, C., & Schiöth, H. B. (2018). Experimenter gender and replicability in science. Science Advances, 4(1), e1701427.

    PubMed  PubMed Central  Google Scholar 

  • Cole, G. G., Smith, D. T., & Atkinson, M. A. (2015). Mental state attribution and the gaze cueing effect. Attention, Perception, & Psychophysics, 77(4), 1105–1115.

    Google Scholar 

  • Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., & Baron-Cohen, S. (1999). Gaze perception triggers reflexive visuospatial orienting. Visual Cognition, 6(5), 509–540.

    Google Scholar 

  • Ellsworth, P. C., & Ludwig, L. M. (1972). Visual Behavior in social interaction. Journal of Communication, 22(4), 375–403.

    Google Scholar 

  • Foulsham, T., Walker, E., & Kingstone, A. (2011). The where, what and when of gaze allocation in the lab and the natural environment. Vision Research, 51(17), 1920–1931.

    PubMed  Google Scholar 

  • Freeth, M., Foulsham, T., & Kingstone, A. (2013). What affects social attention? Social presence, eye contact and autistic traits. PLoS One, 8(1), e53286.

    PubMed  PubMed Central  Google Scholar 

  • Friesen, C. K., & Kingstone, A. (1998). The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin & Review, 5(3), 490–495.

    Google Scholar 

  • Friesen, C. K., Moore, C., & Kingstone, A. (2005). Does gaze direction really trigger a reflexive shift of spatial attention? Brain and Cognition, 57(1), 66–69.

    PubMed  Google Scholar 

  • Gluckman, M., & Johnson, S. P. (2013). Attentional capture by social stimuli in young infants. Frontiers in Psychology, 4, 527.

    PubMed  PubMed Central  Google Scholar 

  • Hamilton, S. J. (2017). The effects of pointing gestures on visual attention. University Honors Program Theses, 243.

  • Hayward, D. A., Voorhies, W., Morris, J. L., Capozzi, F., & Ristic, J. (2017). Staring reality in the face: A comparison of social attention across laboratory and real world measures suggests little common ground. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale, 71(3), 212.

    PubMed  Google Scholar 

  • Kajopoulos, J. (2019) Social cognition and robotics. (Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences). University library repository of the Ludwig-Maximilians-Universität München. https://edoc.ub.uni-muenchen.de/25034/.

  • Kajopoulos, J., Wong, A. H. Y., Yuen, A. W. C., Dung, T. A., Kee, T. Y., & Wykowska, A. (2015). Robot-assisted training of joint attention skills in children diagnosed with autism. Social Robotics: 7th International Conference, ICSR 2015 (pp. 296–305). Paris, France.

  • Kerzel, D., Zarian, L., & Souto, D. (2009). Involuntary cueing effects on accuracy measures: Stimulus and task dependence. Journal of Vision, 9(11), 1–16.

    PubMed  Google Scholar 

  • Kleinke, C. L. (1986). Gaze and eye contact: a research review. Psychological Bulletin, 100(1), 78.

    PubMed  Google Scholar 

  • Kompatsiari, K., Ciardo, F., Tikhanoff, V., Metta, A., & Wykowska, A. (2018a). On the role of eye contact in gaze cueing. Scientific Reports, 8, 17842.

    PubMed  PubMed Central  Google Scholar 

  • Kompatsiari, K., Pérez-Osorio, J., De Tommaso, D., Metta, G., Wykowska, A. (2018b). Neuroscientifically-grounded research for improved human-robot interaction. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3403–3408). Madrid, Spain.

  • Kuhn, G., & Kingstone, A. (2009). Look away! Eyes and arrows engage oculomotor responses automatically. Attention, Perception, & Psychophysics, 71(2), 314–327.

    Google Scholar 

  • Langton, S. R., & Bruce, V. (2000). You must see the point: automatic processing of cues to the direction of social attention. Journal of Experimental Psychology: Human Perception and Performance, 26(2), 747.

    PubMed  Google Scholar 

  • Macdonald, R. G., & Tatler, B. W. (2018). Gaze in a real-world social interaction: A dual eye-tracking study. Quarterly Journal of Experimental Psychology, 71(10), 2162–2173.

    Google Scholar 

  • Moore, C., & Dunham, P. J. (2014). Joint attention: Its origins and role in development. New York: Psychology Press.

    Google Scholar 

  • Müller, H. J., Geyer, T., Zehetleitner, M., & Krummenacher, J. (2009). Attentional capture by salient color singleton distractors is modulated by top-down dimensional set. Journal of Experimental Psychology: Human Perception and Performance, 35(1), 1–16.

    PubMed  Google Scholar 

  • Mundy, P. (2017). A review of joint attention and social-cognitive brain systems in typical development and autism spectrum disorder. European Journal of Neuroscience, 47(6), 497–514.

    Google Scholar 

  • Perez-Osorio, J., Müller, H. J., Wiese, E., & Wykowska, A. (2015). Gaze following is modulated by expectations regarding others’ action goals. PLoS One, 10(11), e0143614.

    PubMed  PubMed Central  Google Scholar 

  • Prinzmetal, W., McCool, C., & Park, S. (2005). Attention: reaction time and accuracy reveal different mechanisms. Journal of Experimental Psychology: General, 134(1), 73.

    Google Scholar 

  • Ricciardelli, P., Bricolo, E., Aglioti, S. M., & Chelazzi, L. (2002). My eyes want to look where your eyes are looking: exploring the tendency to imitate another individual’s gaze. NeuroReport, 13(17), 2259–2264.

    PubMed  Google Scholar 

  • Ristic, J., & Kingstone, A. (2005). Taking control of reflexive social attention. Cognition International Journal of Cognitive Science, 94(3), B55–B65.

    Google Scholar 

  • Schellen, E., & Wykowska, A. (2019). Intentional mindset toward robots—open questions and methodological challenges. Frontiers in Robotics and AI, 5, 139.

    PubMed  PubMed Central  Google Scholar 

  • Schilbach, L., Timmermans, B., Reddy, V., Costall, A., Bente, G., Schlicht, T., & Vogeley, K. (2013). Toward a second-person neuroscience. Behavioral and Brain Sciences, 36(4), 393–414.

    Google Scholar 

  • Skarratt, P. A., Cole, G. G., & Kingstone, A. (2010). Social inhibition of return. Acta Psychologica, 134(1), 48–54.

    PubMed  Google Scholar 

  • Teufel, C., Fletcher, P. C., & Davis, G. (2010). Seeing other minds: attributed mental states influence perception. Trends in Cognitive Sciences, 14(8), 376–382.

    PubMed  Google Scholar 

  • Theeuwes, J. (1994). Stimulus-driven capture and attentional set: Selective search for color and visual abrupt onsets. Journal of Experimental Psychology: Human Perception and Performance, 20(4), 799–806.

    PubMed  Google Scholar 

  • Theeuwes, J. (2004). Top-down search strategies cannot override attentional capture. Psychonomic Bulletin & Review, 11(1), 65–70.

    Google Scholar 

  • Theeuwes, J. (2010). Top-down and bottom-up control of visual selection. Acta Psychologica, 135(2), 77–99.

    PubMed  Google Scholar 

  • Theeuwes, J., Atchley, P., & Kramer, A. F. (2000). On the time course of top-down and bottom-up control of visual attention. In S. M. J. Driver (Ed.), Attention and Performance, 18 (pp. 105–125). Cambridge: MIT Press.

    Google Scholar 

  • Tomasello, M. (1995). Joint attention as social cognition. In C. Moore & P. J. Dunham (Eds.), Joint attention: Its origins and role in development (pp. 103–130). USA: Lawrence Erlbaum Associates Inc.

    Google Scholar 

  • Wiese, E., Metta, G., & Wykowska, A. (2017). Robots as intentional agents: Using neuroscientific methods to make robots appear more social. Frontiers in Psychology, 8, 1663.

    PubMed  PubMed Central  Google Scholar 

  • Wiese, E., Wykowska, A., & Müller, H. J. (2014). What we observe is biased by what other people tell us: Beliefs about the reliability of gaze behavior modulate attentional orienting to gaze cues. PLoS One, 9(4), e94529.

    PubMed  PubMed Central  Google Scholar 

  • Wiese, E., Wykowska, A., Zwickel, J., & Müller, H. J. (2012). I see what you mean: how attentional selection is shaped by ascribing intentions to others. PLoS One, 7(9), e45391.

    PubMed  PubMed Central  Google Scholar 

  • Wiese, E., Zwickel, J., & Müller, H. J. (2013). The importance of context information for the spatial specificity of gaze cueing. Attention, Perception, & Psychophysics, 75(5), 967–982.

    Google Scholar 

  • Willemse, C., & Wykowska, A. (2019). In natural interaction with embodied robots we prefer it when they follow our gaze: a gaze-contingent mobile eyetracking study. Philosophical Transactions of the Royal Society B, 374, 20180036.

    Google Scholar 

  • Wykowska, A., Chaminade, T., & Cheng, G. (2016). Embodied artificial agents for understanding human social cognition. Philosophical Transactions of the Royal Society London: B. Biological Sciences, 371, 20150375.

    Google Scholar 

  • Wykowska, A., Kajopoulos, J., Obando-Leitón, M., Chauhan, S., Cabibihan, J.-J., & Cheng, G. (2015). Humans are well tuned to detecting agents among non-agents: Examining the sensitivity of human perception to behavioral characteristics of intentional systems. International Journal of Social Robotics, 7(5), 767–781.

    Google Scholar 

  • Wykowska, A., & Schubö, A. (2010). On the temporal relation of top-down and bottom-up mechanisms during guidance of attention. Journal of Cognitive Neuroscience, 22(4), 640–654.

    PubMed  Google Scholar 

  • Wykowska, A., Wiese, E., Prosser, A., & Müller, H. J. (2014). Beliefs about the minds of others influence how we process sensory information. PLoS One, 9(4), e94339.

    PubMed  PubMed Central  Google Scholar 

  • Zehetleitner, M., Hegenloh, M., & Müller, H. J. (2011). Visually guided pointing movements are driven by the salience map. Journal of Vision, 11(1):1–18.

    Google Scholar 

  • Zehetleitner, M., Krummenacher, J., & Müller, H. J. (2009). The detection of feature singletons defined in two dimensions is based on salience summation, rather than on serial exhaustive or interactive race architectures. Attention, Perception, & Psychophysics, 71(8), 1739–1759.

    Google Scholar 

Download references

Acknowledgements

We thank the Japan Society for the Promotion of Science (JSPS) for supporting Jasmin Kajopoulos as an International Research Fellow. We are grateful towards Ayano Okoso and several students from the OPU, who supported us with data collection and participant recruitment. The content of this article and the figures of this article have been extracted, reproduced and partly adapted (see figure captions in particular) with permission from the doctoral thesis (mainly from Chapter 1) of Kajopoulos, J. (2019), “Social cognition and robotics” [Doctoral thesis, Ludwig-Maximilians-Universität München: Graduate School of Systemic Neurosciences] published at the university library repository of the Ludwig-Maximilians-Universität München, accessible at https://edoc.ub.uni-muenchen.de/25034/.

Funding

This research was supported by the Deutsche Forschungsgemeinschaft (WY-122/1-1, awarded to AW; MU 773/16-2, awarded to HJM ), the Japan Society for the Promotion of Science (JSPS Summer Program, awarded to JK), the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program: Grant ERC-2016-StG-715058 awarded to AW.

Author information

Authors and Affiliations

  1. Graduate School of Systemic Neurosciences, Ludwig-Maximilians-Universität München, Munich, Germany

    Jasmin Kajopoulos

  2. Japan Society for the Promotion of Science, Tokyo, Japan

    Jasmin Kajopoulos

  3. Institute for Cognitive Systems, Technical University of Munich, Karlstrasse 45, 80333, Munich, Germany

    Jasmin Kajopoulos & Gordon Cheng

  4. General and Experimental Psychology Unit, Dept. Psychologie, Ludwig-Maximilians-Universität München, Munich, Germany

    Jasmin Kajopoulos & Hermann J. Müller

  5. Department of Computer Science and Intelligent Systems, Graduate School of Engineering, Osaka Prefecture University, Sakai, Japan

    Koichi Kise

  6. School of Psychological Science, Birkbeck College, University of London, London, UK

    Hermann J. Müller

  7. Social Cognition in Human-Robot Interaction Unit, Istituto Italiano di Tecnologia, Genoa, Italy

    Agnieszka Wykowska

Authors
  1. Jasmin Kajopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gordon Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Koichi Kise
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hermann J. Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Agnieszka Wykowska
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jasmin Kajopoulos.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 23 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kajopoulos, J., Cheng, G., Kise, K. et al. Focusing on the face or getting distracted by social signals? The effect of distracting gestures on attentional focus in natural interaction. Psychological Research 85, 491–502 (2021). https://doi.org/10.1007/s00426-020-01383-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00426-020-01383-4

Search

Navigation