How ubiquitous is the direct-gaze advantage? Evidence for an averted-gaze advantage in a gaze-discrimination task

Riechelmann, Eva; Gamer, Matthias; Böckler, Anne; Huestegge, Lynn

doi:10.3758/s13414-020-02147-3

How ubiquitous is the direct-gaze advantage? Evidence for an averted-gaze advantage in a gaze-discrimination task

Open access
Published: 01 November 2020

Volume 83, pages 215–237, (2021)
Cite this article

Download PDF

You have full access to this open access article

Attention, Perception, & Psychophysics Aims and scope Submit manuscript

How ubiquitous is the direct-gaze advantage? Evidence for an averted-gaze advantage in a gaze-discrimination task

Download PDF

Eva Riechelmann¹,
Matthias Gamer¹,
Anne Böckler² &
…
Lynn Huestegge¹

2944 Accesses
10 Citations
1 Altmetric
Explore all metrics

Abstract

Human eye gaze conveys an enormous amount of socially relevant information, and the rapid assessment of gaze direction is of particular relevance in order to adapt behavior accordingly. Specifically, previous research demonstrated evidence for an advantage of processing direct (vs. averted) gaze. The present study examined discrimination performance for gaze direction (direct vs. averted) under controlled presentation conditions: Using a backward-masking gaze-discrimination task, photographs of faces with direct and averted gaze were briefly presented, followed by a mask stimulus. Additionally, effects of facial context on gaze discrimination were assessed by either presenting gaze direction in isolation (i.e., by only showing the eye region) or in the context of an upright or inverted face. Across three experiments, we consistently observed a facial context effect with highest discrimination performance for faces presented in upright position, lower performance for inverted faces, and lowest performance for eyes presented in isolation. Additionally, averted gaze was generally responded to faster and with higher accuracy than direct gaze, indicating an averted-gaze advantage. Overall, the results suggest that direct gaze is not generally associated with processing advantages, thereby highlighting the important role of presentation conditions and task demands in gaze perception.

From eye to arrow: Attention capture by direct gaze requires more than just the eyes

Article Open access 02 November 2021

Seeing eye-to-eye: Social gaze interactions influence gaze direction identification

Article Open access 15 July 2019

Gaze direction and face orientation modulate perceptual sensitivity to faces under interocular suppression

Article Open access 10 May 2022

Introduction

The direction of human eye gaze conveys a significant amount of relevant social information, especially with respect to attention and intention, and serves as a reliable source of non-verbal information (Argyle & Cook, 1976; Emery, 2000; Frischen, Bayliss, & Tipper, 2007; Kendon & Cook, 1969; Kleinke, 1986; Schilbach, 2015). As such, gaze direction of others can point the gaze recipient towards sources of potential interest, or alert the gaze recipient about potential dangers, like the occurrence of an enemy. Therefore, it is obvious that quickly determining other persons’ gaze directions can be of high relevance in social situations and comes with an adaptive advantage given the potentially disastrous consequences of non-detection. Many studies have examined face- and gaze-processing mechanisms, thereby touching upon the topic of gaze discrimination. However, fewer studies focused on the systematic investigation of humans’ ability to discriminate gaze direction. The present study reports a series of three experiments dedicated to fill this research gap. Given that gaze discrimination is of special importance in situations of danger or threat, where “every second counts” and viewing conditions might be suboptimal, we put emphasis on the investigation of gaze discrimination ability when the perception of gaze direction is limited to a very short time window, that is, under brief and masked presentation conditions. Such conditions might be considered paradigmatic for real-life instances where gaze perception can be difficult due to situational constraints (e.g., time pressure, spatial distance, low lighting, etc.). Further, the present experiments are dedicated to investigating effects of facial context information on gaze discrimination ability.

Social functions of gaze

In social interactions, gaze cues serve several purposes, for example providing information, guiding social interaction, fostering communication, expressing intimacy, and supporting goal-setting (Csibra & Gergely, 2009; Kleinke, 1986; Patterson, 1982; Richardson & Dale, 2005; Riechelmann, Raettig, Böckler, & Huestegge, 2019). Consequently, it comes as no surprise that humans are highly sensitive to gaze cues. Researchers have provided striking evidence that the perception of both direct and averted gaze direction affects social cognition. Perceiving averted gaze is assumed to trigger a quasi-reflexive shift of attention into the respective gaze direction (Driver et al., 1999; Friesen & Kingstone, 1998; Mansfield, Farroni, & Johnson, 2003; see Frischen et al., 2007, for a review on gaze cueing), whereas seeing a face with direct gaze can capture the attention of the observer. More specifically, it has been shown that presenting faces that are directly looking at the observer (vs. faces that are looking away) modulate cognitive processes, related to, for example, action control (Sato & Itakura, 2013; but see Riechelmann, Weller, Huestegge, Böckler, & Pfister, 2019) or to face perception in terms of faster recognition and better memorization, thereby establishing a direct-gaze advantage (Böckler, van der Wel, & Welsh, 2014; Boyer & Wang, 2018; Hood, Macrae, Cole-Davies, & Dias, 2003; Macrae, Hood, Milne, Rowe, & Mason, 2002; Senju & Hasegawa, 2005).

Despite the powerful function of averted gaze to automatically direct visual attention of the perceiver toward the direction of gaze, a strong line of research has emphasized the crucial role of (perceived) direct gaze as a socially relevant signal that indicates potential social interaction and therefore receives privileged visual attention (Chen & Yeh, 2012; von Grünau & Anston, 1995; Palanica & Itier, 2011, 2012; Senju, Hasegawa, & Tojo, 2005; Senju & Johnson, 2009; Stein, Senju, Peelen, & Sterzer, 2011). Findings from cognitive neuroscience that found modulations of brain activity by the perceived gaze direction in regions that are known to be involved in gaze processing (e.g., superior temporal sulcus or amygdala) support the claim that the perceptual system is specialized for the detection of and the attention to direct gaze (e.g., Hoffman & Haxby, 2000; Kawashima et al., 1999; but see Itier & Batty, 2009, for a review). von Grünau and Anston (1995) applied a visual search paradigm to empirically demonstrate this saliency of direct gaze – the “stare-in-the-crowd” effect. In their experiments, participants were presented with displays that showed eye stimuli in various gaze directions (straight vs. left- and rightward gaze). Participants had to indicate via keypress the presence or absence of a target (e.g., straight gaze) among distractors (e.g., left- and rightward gaze). When the task was to detect straight gaze among distractors of left- and rightward gaze, participants were faster compared to the detection of averted gaze targets among their distractors. A modified replication of the von Grünau and Anston (1995) study replicated the original findings (Senju et al., 2005). By including laterally averted faces as stimuli (instead of an isolated pairs of eyes) that looked either directly at the participant or away from the participant (to the left or right), the study of Senju et al. (2005) ruled out low-level perceptual characteristics as confounding factors for the original results, such as symmetry of straight gaze. On top of that, several studies provided evidence for an enhanced unconscious processing of direct (vs. averted) gaze on both behavioral (Chen & Yeh, 2012; Madipakkam, Rothkirch, Guggenmos, Heinz, & Sterzer, 2015; Stein et al., 2011) and neural (Yokoyama, Noguchi, & Kita, 2013) levels. Evidence for a privileged processing of direct (vs. averted) gaze was obtained for the presentation of both whole faces and isolated eye regions, irrespective of the level of processing (conscious vs. unconscious) (Chen & Yeh, 2012; Senju et al., 2005). This sensitivity to gaze direction and a privileged processing of direct gaze has been shown to arise very early in human development. Studies with infants observed that even neonates preferred to look at faces with direct gaze over faces with averted gaze or closed eyes (Batki, Baron-Cohen, Wheelwright, Connellan, & Ahluwalia, 2000; Farroni, Csibra, Simion, & Johnson, 2002; Vecera & Johnson, 1995). In sum, the abovementioned studies provide cumulative evidence for a general direct-gaze advantage and emphasize the social relevance for humans to be able to detect as quickly and accurately as possible if somebody is looking at them.

Cues to gaze discrimination

Based on the considerations and findings reviewed above, it appears vital to ask what kind of cues humans use to discriminate gaze directions. In response to this issue, it has been suggested that the visual system integrates several factors to derive gaze direction judgments, depending on specific task requirements, thus establishing a multiple-cue account to gaze perception (Anderson, Risko, & Kingstone, 2016; Gamer & Hecht, 2007; Olk, Symons, & Kingstone, 2008). These include cues from the eye region, for instance, geometrical configuration of the eyes (Anstis, Mayhew, & Morley, 1969), luminance distribution (Anderson et al., 2016; Ando, 2002, 2004; Olk et al., 2008), or motion signals (Anderson et al., 2016), as well as cues outside of the eye region, for instance, head orientation (Balsdon & Clifford, 2017; Gamer & Hecht, 2007; Langton, 2000; Langton, Watt, & Bruce, 2000; Ricciardelli & Driver, 2008) or torso orientation (Seyama & Nagayama, 2005). The use of geometrical cues to detect gaze direction implies that gaze direction is derived from the relative position of the iris within the surrounding sclera and eyelids (Anstis et al., 1969). The human eye is characterized by a particularly salient contrast between the dark iris and the bright sclera, inducing a specific luminance distribution across the eye (Emery, 2000). When changing this luminance distribution, gaze discrimination ability is severely disrupted (Ando, 2002, 2004; Ricciardelli, Bayliss, & Driver, 2000; Sinha, 2000). As a consequence, it has been suggested that the visual system follows the rule that the darker region of the eye is a key feature for gaze direction estimation.

The present study

Given that humans are highly sensitive to gaze cues, the present series of experiments was designed to examine the ability of humans to discriminate direct from averted gaze direction in varying face contexts under controlled presentation conditions. In our experimental setup, a gaze stimulus was presented for 35 ms, and masked afterwards in order to control the time available to process the stimulus (see Langton, Honeyman, & Tessler, 2004, for a similar gaze-discrimination task). As we were mainly interested in the processing of briefly presented face stimuli, the duration of the gaze stimulus was set such that gaze processing was at the lower boarder of conscious perception (cf. Kiss & Eimer, 2008; Pegna, Landis, & Khateb, 2008). Thus, the very short image duration used in the current study is likely to introduce a certain degree of uncertainty about the sensory input. Given that perception studies have shown the potential impact of facial context on gaze processing (see below), we decided to examine the effect of facial context on gaze discrimination accuracy by presenting gaze direction either in the context of an upright face, an inverted face, or in isolation (i.e., by only displaying the eye region). Presenting faces upside down is known to disrupt holistic face processing (Bruce & Langton, 1994; Yin, 1969) and to impair gaze discrimination accuracy (Jenkins & Langton, 2003; Vecera & Johnson, 1995). Participants were asked to indicate as quickly and accurately as possible the gaze direction of the presented stimulus via keypress. The present approach is methodically different from other studies on the detection of perceived direct (vs. averted) gaze where the target is not briefly, but continuously presented, for example among distractors during a visual search task (von Grünau & Anston, 1995; Senju et al., 2005). To the best of our knowledge, it is still an open issue how discrimination performance for categorizing gaze directions would differ depending on task type (visual search/gaze detection as investigated previously vs. gaze discrimination as studied in the present experiments).

We addressed our research questions by means of five dependent measures. First, we measured discrimination accuracy. We expected accuracy rates to be higher in the context of an upright face than in the context of an inverted face, as demonstrated by Vecera and Johnson (1995). Based on findings from Schwaninger, Lobmaier, and Fischer (2005), who reported comparable gaze location judgments when whole faces or the isolated eye regions were presented, one could probably expect accuracy rates to be similar for upright faces and eyes with the facial context removed. On the other hand, Langton et al. (2004) showed impaired gaze discrimination ability when the facial context information was removed as compared to eyes presented within the context of an upright face. In line with these latter results, one could thus also expect gaze discrimination ability to be reduced when eyes are presented without any facial context information.

Given that both looking at somebody and looking away from somebody are socially relevant signals that can indicate the intention of another person towards the perceiver, it is useful to be sensitive to both direct and averted gaze cues even in situations where the gaze stimulus is perceived only shortly. However, in the light of the great number of studies demonstrating enhanced representation and processing of direct gaze (Böckler et al., 2014; Boyer & Wang, 2018; Chen & Yeh, 2012; Farroni et al., 2002; Senju et al., 2005; Stein et al., 2011), we originally hypothesized that direct gaze processing is associated with higher accuracy rates compared to averted gaze in the present backward masking paradigm. This hypothesis is further affirmed by results suggesting a response bias toward direct gaze under conditions of uncertainty (Mareschal, Calder, Dadds, & Clifford, 2013; Mareschal, Otsuka, & Clifford, 2014)

Reviewing current findings on the processing of gaze did not allow us to come to a clear expectation about an interaction of context and gaze (especially with respect to the upright vs. inverted face context) under conditions of near-threshold stimulus presentation for two reasons. First, available evidence on the prioritized processing of direct (vs. averted) gaze in the context of upright (vs. inverted) face stimuli mostly relied on other experimental designs and dependent measures (e.g., response times rather than accuracy rates). Second, these response time (RT) studies reported the presence or absence of an interaction pattern depending on the paradigm used. For example, Senju et al. (2005) observed the absence of a direct-gaze advantage for inverted faces under conditions of conscious gaze processing. Similarly, Böckler, van der Wel, and Welsh (2015) reported the absence of attention capture by direct gaze when faces were presented upside-down. Others, however, showed that the preferential processing of direct gaze was independent of face inversion for unconscious processing of gaze (Chen & Yeh, 2012; Stein et al., 2011).

As a second behavioral measure, we analyzed RTs. Since participants were instructed to respond as quickly and accurately as possible, potential systematic differences in RTs across conditions, viewed in conjunction with accuracy rates, might provide insight into cognitive processes underlying the observed result patterns (e.g., with respect to a speed-accuracy tradeoff).

Data were also analyzed by means of signal detection theory (SDT), focusing on the sensitivity parameter d’ and the response criterion c (see Method section of Experiment 1 for a detailed explanation). Based on Vecera and Johnson (1995) we expected gaze discrimination to be more efficient in upright faces than in inverted faces or for eyes only stimuli. We had no prior hypothesis about differences in gaze discrimination between inverted face and eyes-only stimuli. Note that the SDT analysis only reflects a measure of participants’ ability to discriminate direct from averted gaze in a given facial context (upright vs. inverted vs. eyes only) without allowing for any direct conclusion about which gaze direction is associated with a processing advantage (see Langton et al., 2004, and Vecera & Johnson, 1995, for a similar procedure).

Finally, we included a confidence measure after each trial to explore whether participants’ confidence in their performance mirrors actual discrimination accuracy. Dissociations between the subjective confidence and the objective performance measure may be informative with respect to potential origins of observed effects.

To anticipate the most surprising results of Experiment 1, we observed a clear advantage of averted gaze (in terms of faster RTs and higher accuracy rates for averted vs. direct gaze direction) for upright-face and eyes-only stimuli. Based on this surprising finding, we decided to run follow-up experiments in order to substantiate the evidence for an averted-gaze advantage. Furthermore, these follow-up experiments were also designed to systematically investigate potential boundary conditions of the averted-gaze advantage as observed in Experiment 1. More specifically, we tested whether the results of Experiment 1 are reliable (Experiment 2) and whether low-level task-specific demands provide a potential explanation for the observed results (Experiment 3). To anticipate our results once more, the results of Experiment 1 were closely replicated across experiments independent of the parameters varied, suggesting that the averted-gaze advantage as well as the declining ability to discriminate direct from averted gaze in upright faces versus inverted faces and eyes-only stimuli as observed in Experiment 1 is robust.

Experiment 1

Experiment 1 investigated the ability of humans to discriminate direct from averted gaze direction in varying face contexts. To do so, targets with direct or averted gaze were briefly presented in a gaze-discrimination task and masked afterwards. Participants were asked to indicate the perceived gaze direction as quickly and accurately as possible via keypress.

The present setup implemented a backward-masking paradigm, a well-established way to study recognition of briefly presented target stimuli (see Breitmeyer & Ogmen, 2000, for a review). In backward-masking paradigms, the short presentation of a target stimulus is immediately followed by the presentation of a mask. This technique allows for the control of visual afterimages by accurately controlling the time available for sensory processing of the target stimulus. As one of the first, the study of Esteves and Öhmann (1993) used the backward masking procedure to investigate the recognition of emotional face expression as a function of different parameters. Their main result was that the stimulus-onset asynchrony (SOA) was the parameter that influenced recognition most. Further, backward masking can be applied to examine stimulus processing along different levels of visual consciousness by varying the presentation duration of the target stimulus. EEG backward-masking studies, for instance, investigated the time course of emotional face processing at subliminal and supraliminal processing stages (Eimer, Kiss, & Holmes, 2008; Kiss & Eimer, 2008; Vukusic, Ciorciari, & Crewther, 2017). Interestingly, several findings have shown that the modulation of affective responses in response to emotional stimuli and affective faces was similar for both masked stimuli and stimuli at longer presentation duration (e.g., Del Zotto & Pegna, 2015; Ruiz-Padial, Mata, Rodríguez, Fernández, & Vila, 2005; Ruiz-Padial & Vila, 2007). Relatedly, it has been shown that masked gaze cues are also capable of producing a gaze-cueing effect given that participant’s task set is tuned to the eye gaze information of the masked stimulus (Al-Janabi & Finkbeiner, 2012; Sato, Okada, & Toichi, 2007; but see Al-Janabi & Finkbeiner, 2014). Together, the results of these studies suggest that masked stimuli can produce qualitatively similar effects as have been reported for unmasked stimuli.

Method

Participants

Twenty-five participants (mean age: 20.3 years, SD = 2.4 years, age range: 18–30 years, one male) took part in the study in exchange for course credit or payment. All participants reported normal or corrected-to-normal vision and gave informed consent prior to the study. On a nine-point scale ranging from 1 (“not difficult at all”) to 9 (“extremely difficult”), participants rated the task as rather difficult with an average of 7.0 points (SD = 1.7). A power analysis with G*Power (Faul, Erdfelder, Lang, & Buchner, 2007) was conducted based on a pilot study which incorporated stimulus material that differed only slightly from the stimulus material used in the present study. The power analysis indicated a sample size of 13 participants as sufficient to observe effects of face inversion, and a sample size of eight participants as sufficient to observe effects of gaze direction on accuracy (power = .95, α = .05). However, to account for possible dropouts and to avoid an underpowered study, we decided to recruit a minimum of 24 participants per experiment.

Apparatus and stimuli

In a dimly lit room, participants were seated in front of a 20-in. cathode ray monitor (temporal resolution, 100 Hz, spatial resolution 1,024 × 768 pixels) at a viewing distance of 65 cm. The experiment was programmed using Experiment Builder (SR Research, Ontario, Canada). Color face photographs of two female and two male Caucasian models that were taken by our lab served as stimulus material. All faces were of neutral facial expression and available with direct and averted gaze (to left and right). Gaze directions were inserted into the same face picture using photo editing software to maximize control. The faces were cropped with an oval shape, removing all external features (5.7° × 8.4° of visual angle, width × height). Stimuli were available in three versions, that is, as upright stimuli, inverted stimuli (by turning them upside down), and eyes in isolation (by cutting the eye regions of the upright-face stimuli into two separate ellipses with a size of 2.2° × 1.3° each, see Fig. 1).

Masks were created with MATLAB (Version R2017b) by dividing the direct gaze face of each of the four models into boxes of 20 × 20 pixels and randomly rearranging each box within the outline of the face (box scrambling). Using box-scrambled images as masks provided the advantage of creating unrecognizable stimuli without altering any low-level stimulus features (e.g., power spectrum, contrast, and luminance). One mask was created for each model and context condition, resulting in twelve different masks. Finally, we ensured that the relevant eye region always appeared at the same screen location for all stimulus conditions. Since eyes are not exactly centered in the head this resulted in slightly different stimulus positions for upright and eye stimuli as compared to inverted stimuli. The same location adjustment was done for the respective masks.

Procedure

Before the experiment started, participants were instructed and familiarized with the different stimulus conditions. Each trial started with the presentation of a white central fixation cross (0.4° of visual angle, 700-ms presentation duration) on a black background, followed by a target stimulus with either direct or averted gaze direction for 35 ms. Shortly thereafter, the target was replaced by a mask, which was created based on the respective target stimulus, for 265 ms, resulting in a total presentation time of 300 ms (see Fig. 2). After that, the two response alternatives (arrow up vs. down) appeared on the screen together with the associated gaze direction (direct vs. averted). Participants indicated their response via keypress (up or down arrow key). For half of the participants, a direct gaze response was assigned to the “arrow-up” key, and an averted gaze response to the “arrow-down” key. For the other half, the response-key mapping was reversed. There was no time constraint with respect to the keypress, and the next display appeared only after the participant’s response. Participants received the instruction to respond as accurately and fast as possible, and to guess in case they were not able to detect any direction. Participants provided a confidence rating after each trial. To do so, they entered a number on the computer keyboard using a rating scale which ranged from one to five (with one meaning “definitely not detected” and five meaning “definitely detected”). The next trial started 500 ms after participants entered their confidence rating. Stimuli were presented in a fully randomized order in ten blocks of 48 trials each (resulting in 480 trials in total) with breaks in between the blocks. Each stimulus condition and gaze condition appeared equally often. The averted gaze direction (left vs. right) was counterbalanced across participants such that each observer only saw one averted gaze direction across all trials. This design decision was made as part of a strategy that applies to the entire series of presented experiments (for a strategy overview, see Table 1). Among other things, we varied between experiments whether averted gaze direction included only one averted gaze direction per participant, or both (i.e., averted gaze targets to the left and right direction). At the end of the experiment, participants judged the overall difficulty of the task on a scale ranging from one (“not difficult at all”) to nine (“extremely difficult”). This difficulty measure was included in order to obtain an initial assessment of the subjectively experienced difficulty of a newly designed task but served no further diagnostic function within the experimental process.

Table 1 Overview of the experimental manipulations across experiments

Full size table

Design and analyses

We measured accuracy rates (percentage of correct responses) and confidence ratings. Response time was defined as the interval from the onset of the response screen until the first key press. For the analysis of RTs, we eliminated all error trials (19.87% of the data). After removing error trials, we additionally excluded trials with anticipatory responses or extremely high RTs (RTs < 100 ms or RTs > 5,000 ms; corresponding to 0.28% of the remaining data). After that, an outlier correction was applied by removing trials with RTs that deviated more than 2.5 SDs from the corresponding cell mean (calculated separately per participant and condition), corresponding to 3.25% of the remaining data. Separate repeated-measures analyses of variance (ANOVAs, α = .05, throughout) with context (upright vs. inverted vs. eyes only) and gaze (direct vs. averted) as within-subjects factors were conducted to analyze accuracy rates, confidence ratings, and RTs. Significant interactions were further analyzed by using additional one-way repeated-measures ANOVAs. Two-tailed paired-samples t-tests were conducted for follow-up comparisons between conditions. For ANOVAs and paired-samples t-tests we report ƞ²_p and Cohen’s d_z (calculated as \( {d}_z=\frac{t}{\sqrt{n}} \)) as effect size estimates, respectively. Sphericity violations were assessed using the Mauchly’s sphericity test. In case of sphericity violations, we report Greenhouse-Geisser corrected p values along with original degrees of freedoms as well as corresponding ε values. To explore whether the averted-gaze advantage was merely driven by a response bias under high uncertainty, we additionally analyzed accuracy rates including high confidence trials only (i.e., the upper 50% of confidence ratings). Note that while we report the analysis of confidence ratings in the results section of each experiment, we will restrict the combined interpretation of these analyses to the General Discussion.

As an additional estimate of gaze discrimination performance, we computed the sensitivity parameter d’ (d’= z(hit rate) – z(false alarm rate)) and the response criterion c (c = -0.5 [z(hit rate) + z(false alarm rate)]) separately for each participant for each stimulus condition as specified by signal detection theory (SDT) and implemented by Macmillan and Creelman (2010). That is, the d’ score measured the sensitivity of subjects to distinguish between direct and averted gaze in the context of an upright face, an inverted face, and for the eyes-only condition. Direct gaze trials were assigned to represent signal trials, while averted gaze trials served as noise trials. To avoid infinite d’ values in case of perfect accuracy we adjusted proportions of 0 and 1 to 1/(2N) and 1-1/(2N), respectively, where N reflects the number of trials on which the proportion is based (Macmillan & Creelman, 2010). The response criterion c reflects the participants’ tendency toward one or the other response (Macmillan & Creelman, 2010) with more positive values indicating a higher tendency toward “averted” answers. We conducted repeated-measures one-way ANOVAs for d’ and c with context (upright vs. inverted vs. eyes) as within-subjects factor.^{Footnote 1} Two-tailed pairwise t-tests were conducted for follow-up comparisons between conditions. Note that none of the participants had to be excluded due to any d’ lower than zero (indicating that the false alarm rate was higher than the hit rate, see Macmillan and Creelman, 2010).

Results

Accuracy

The effect of context, F(2,48) = 41.17, p < .001, ƞ²_p = .63, indicated differences in accuracy rate across stimuli (see Fig. 3). Accuracy rate was highest for upright faces, lower for inverted faces, and lowest for the eyes-only condition (see Table 2). Paired t-tests revealed significant pairwise differences between all context conditions (ps < .05). Also, there was an effect of gaze, F(1,24) = 6.74, p = .016, ƞ²_p = .22. Interestingly, and contrary to our initial assumption, accuracy was higher for averted gaze (M = 83.93%, SE = 2.29%) compared to direct gaze (M = 76.33%, SE = 2.99%). The interaction of context and gaze was not significant, F(2,48) = 3.44, ε = .70, p = .059, ƞ²_p = .13. Given the medium effect size, we further analyzed this interaction revealing a nominal tendency towards higher accuracy rates for averted gaze (vs. direct gaze) for both upright and eyes only stimuli, t(24) = 2.68, p = .013, d = 0.54, and t(24) = 2.50, p = .020, d = 0.50, respectively, while there was no such difference for inverted stimuli, t(24) = 0.37, p = .716, d = 0.07. Analyzing only high-confidence trials showed a very similar pattern of results, as revealed by the significant effect of context, F(2,48) = 18.52, p < .001, ƞ²_p = .44, the significant effect of gaze, F(1,24) = 5.13, p = .033, ƞ²_p = .18, and the significant interaction, F(2,48) = 5.35, ε = .80, p = .013, ƞ²_p = .18.

Table 2 Mean accuracy rates and response times (RTs) across experiments

Full size table

Response times

Similar to the accuracy rate, we observed a significant main effect of context on RTs, F(2,48) = 14.78, ε = .76, p < .001, ƞ²_p = .38 (see Fig. 4). Participants responded faster in the upright condition compared to both the inverted condition and the eyes condition, both ps < .001. RTs did not differ between the inverted and the eyes-only condition, t(24) = 0.99, p = .332, d = 0.20 (see Table 2). The effect of gaze was not statistically significant, F(1,24) = 3.12, p = .090, ƞ²_p = .12, but indicated a nominal tendency towards faster RTs in the averted condition (M = 790 ms, SE = 46 ms) compared to the direct condition (M = 836 ms, SE = 49 ms). The interaction was not significant, F(2,48) = 1.57, p = .219, ƞ²_p = .06).

Signal-detection analyses

Participants differed in their discrimination sensitivity d’ dependent on context, F(2,48) = 54.27, p < .001, ƞ²_p = .69. Specifically, they showed increased gaze discrimination sensitivity for upright stimuli (M = 2.88, SE = 0.22) compared to both inverted stimuli (M = 1.90, SE = 0.23) and the eyes-only condition (M = 1.58, SE = 0.22), t(24) = 7.92, p < .001, d = 1.58, and t(24) = 9.83, p < .001, d = 1.97, respectively. Importantly, gaze-discrimination sensitivity was also higher for inverted stimuli as compared to the eyes-only condition, t(24) = 2.37, p = .026, d = 0.47. Participants showed no difference in the response criterion c across context conditions, F(2,48) = 2.48, ε = .76, p = .110, ƞ²_p = .09. The mean response criterion c amounted to 0.19 (SE = 0.06) for the upright condition, to 0.05 (SE = 0.07) for the inverted condition, and to 0.26 (SE = 0.11) for the eyes-only condition, indicating a small tendency toward responding with “averted gaze”.

Response confidence

Analogous to accuracy rates, an effect of context emerged, F(2,48) = 45.43, p < .001, ƞ²_p = .65, with highest confidence ratings for upright faces (M = 3.74, SE = 0.17), followed by inverted faces (M = 3.22, SE = 0.17) and the eyes-only condition, (M = 2.85, SE = 0.20). Paired t-tests showed significant differences between all context conditions (ps < .001). Also in line with accuracy findings, the significant effect of gaze, F(1,24) = 9.79, p = .005, ƞ²_p = .29, was driven by higher confidence ratings for averted gaze (M = 3.38, SE = 0.18) compared to direct gaze (M = 3.16, SE = 0.17). The interaction was not statistically significant, F(2,48) = 0.54, p = .584, ƞ²_p = .02.

Experiment 1: Discussion

Experiment 1 was designed to examine performance in a gaze-discrimination task for varying face contexts under near-threshold presentation durations using a backward-masking paradigm. We observed straightforward effects of context and gaze across measures. The effect of context revealed gradually decreasing accuracy rates for upright faces versus inverted faces versus eyes-only stimuli, in line with our predictions. Decreasing accuracy rates came along with gradually increasing RTs for upright face versus inverted face versus eyes-only stimuli (note that while the absolute values for RTs reflected this ordinal pattern, the difference in RTs between the inverted and eyes condition was actually not statistically significant). This ordinal pattern was also reflected in both the confidence rating and the sensitivity parameter d’ with decreasing sensitivity for upright face versus inverted face versus eyes-only stimuli (cf. Vecera & Johnson, 1995). Presumably, facial context information facilitates gaze discrimination, especially if humans are very familiar with the context provided, as in case of an upright presentation of faces. These effects of context cannot be explained by differences in response bias, as we did not find significant differences between response biases across conditions.

We observed an effect of gaze, but in the opposite direction as expected based on the abundance of evidence for a direct-gaze advantage in a variety of experimental research paradigms. Interestingly, we observed higher accuracy rates for averted versus direct gaze (for upright-face and eyes-only stimuli), and higher confidence as well as a tendency toward faster responses for averted versus direct gaze (across all context conditions). Note that accuracy rates were highest in conditions where participants responded fastest, thereby ruling out the presence of a speed-accuracy tradeoff.

Consequently, we asked ourselves about the reasons for the currently observed surprising averted-gaze advantage. Does it truly represent a processing advantage for averted (vs. direct) gaze when targets are presented near-threshold, that is, with a certain degree of uncertainty about the sensory input, or are there any confounds inherent to the task demands that modulate the gaze effect (see e.g., Burra, Mares, & Senju, 2019, for a review on the influence of task demands on gaze processing)? The former account contrasts with results of several previous studies demonstrating that the perceived gaze direction was biased toward direct as uncertainty about the visual input increased (e.g., by adding noise or removing facial context information; see Mareschal et al., 2013, 2014). The latter account could be motivated by the following facts: First, in Experiment 1 we opted for a two-alternative forced-choice task in order to be able to distinguish between perceptual sensitivity and response bias according to signal detection theory. Consequently, each participant had to decide whether targets displayed direct or averted gaze, but the direction of the averted gaze (to the left or right) was held constant within participants, while the specific pairing (direct with left gaze or direct with right gaze) was counterbalanced across participants. Second, direct gaze, which has been implemented in terms of straight gaze in the present study, is characterized by symmetry on low-level perception: Pupil and iris are centrally arranged within the eye region and surrounded by the white sclera. In contrast, pupil and iris shift to the left or right edge of the eye region for averted gaze, yielding a larger coherent area of eye white. We consider it plausible that the observed effects of gaze are due to the increased saliency of the eye white for averted (vs. direct) gaze targets. Under conditions of near-threshold presentation duration, the salient eye white of the averted gaze might pop out, while the eye white and the pupil become somewhat “blurred” for direct gaze, potentially making it harder to indicate gaze direction. We conducted the following series of experiments in order to test whether the results of Experiment 1 are reliable (Experiment 2) and to rule out low-level stimulus-specific demands as described above as a potential explanation of the present results (Experiment 3).

In these follow-up experiments, each participant was confronted with gaze stimuli of both left and right averted gaze direction. However, participants still had to classify gaze stimuli as direct versus averted gaze (two response options) in Experiment 2. Consequently, the target-key mapping for averted gaze targets was at a ratio of 2:1 (i.e., direct gaze was mapped onto one key and both averted-gaze conditions were mapped onto the other key) in Experiment 2. In Experiment 3, participants were required to distinguish between direct, left, and right gaze (three response options); therefore, the target-key mapping was again at equal ratio (i.e., direct gaze was mapped onto one key, left gaze onto another key, and right gaze onto a third key) in Experiment 3. Another experimental manipulation was motivated by the fact that each target (direct vs. averted) was presented at the same ratio, and therefore, both direct and averted gaze direction were presented in 50% of the trials in Experiment 1. With three gaze directions as stimuli (as in Experiments 2 and 3) it was not possible to maintain both an equal ratio of target presentation and an equal ratio of direct versus averted gaze targets (summed up for left and right gaze) at the same time. More precisely, as soon as each gaze direction (direct vs. left vs. right) will be presented in one third of the trials, the proportion of direct versus averted gaze targets is not equal anymore but will shift to 1:2 for direct versus averted gaze direction. To address this issue, we manipulated the relative target presentation frequency, that is, whether targets of each gaze condition (direct vs. left vs. right) were presented at equal proportion, or whether direct and averted gaze targets were presented in half of the trials with the averted-gaze condition (with an equal number of left and right gaze stimuli). Table 1 provides an overview of these experimental manipulations across experiments.

Experiment 2

Experiment 2 was a conceptual replication of Experiment 1 and was conducted to test whether the averted-gaze advantage as observed in Experiment 1 is reliable. We adapted our task in a way that each participant was not only presented with averted gaze targets of the same direction (left or right) but with averted gaze targets signaling both gaze directions (left and right). As in Experiment 1, there were only two response options indicating direct vs. averted gaze. Consequently, the target-key mapping for averted gaze changed from 1:1 in Experiment 1 to 2:1 in Experiment 2 (left and right averted gaze targets were mapped onto the same key). If, for example, the averted-gaze advantage was mainly due to the salient sclera in the averted gaze targets, as hypothesized above, we would expect to observe the same result pattern as in Experiment 1, since the task of Experiment 2 required the same classification of direct and averted gaze as in Experiment 1, but no differentiation between left and right gaze.

In Experiment 1, presenting both targets at equal ratio can be equated with presenting an equal proportion of direct and averted gaze targets. However, the design of Experiment 2 entailed that both types of equalities cannot be maintained at the same time: If all three targets (direct/left/right) were presented at equal ratio, the ratio of presenting direct and averted gaze targets would inevitably change from equal ratio to 1:2 (for direct/averted gaze targets), and vice versa. Therefore, comparability between Experiments 1 and 2 would be reduced. In order to rule out that the ratio of stimulus presentation modulates the effect, we therefore conducted two versions of this experiment (Experiment 2a and 2b). While each target (direct vs. left vs. right) was presented at equal ratio in Experiment 2a, the ratio of target presentation was altered in Experiment 2b in order to present direct versus averted gaze targets in half of the trials each (see Method section for details). Again, if the averted-gaze advantage was due to the salient sclera in the averted gaze targets, we would expect to observe the same result pattern as in Experiment 1 across both Experiment 2a and 2b, independent of the ratio of target presentation.