During social interactions, individuals process the movements of other people for a variety of purposes, including engaging in nonverbal communication, decoding intention, and coordinating action. One important cue used during social interactions is the direction of eye gaze. Indeed, gaze has been repeatedly shown to be a powerful biological orienting cue (e.g., Friesen & Kingstone, 1998; see Frischen, Bayliss, & Tipper, 2007a, for a review). Other cues, such as a pointed finger, might act similarly (e.g., Ariga & Watanabe, 2009). Even though different social cues are used for similar purposes, they are likely to be processed differently. Separate regions of the extrastriate and other visual cortices process body and face stimuli (see Peelen & Downing, 2007, for a review), and separate networks of motor areas process information for the generation of hand and eye movements (Cisek & Kalaska, 2010; Kandel, Schwartz, & Jessell, 2000). As such, pointing and gaze cues might also be decoded using separate networks. The present article reports a set of studies designed to further our understanding of the mechanisms involved in the processing of social gaze and pointing cues, by using an upper-limb reaching task. The spatial and temporal characteristics of reaching movements to target locations were used here because such analyses can provide additional information concerning the dynamics of the cognitive mechanisms involved in the cueing paradigm (see Song & Nakayama, 2009; Welsh & Weeks, 2010).

The time course of cueing effects

The present studies involved an adapted version of the conventional spatial-cueing paradigm (Posner, 1980; Posner & Cohen, 1984). In a common version of this paradigm, a sudden-onset cue is presented peripherally at one of two potential target locations. Even though the cue is nonpredictive of the target location, RTs are affected by the cue, with the direction of this influence being dependent on the timing of the cue and target onsets (the stimulus onset asynchrony, SOA). A facilitation effect of the cue (shorter RTs for targets at cued over uncued locations) emerges at short (~ 100 ms) SOAs. This facilitation is thought to emerge because the cue has rapidly drawn attention to its location. At SOAs longer than 300 ms, however, RTs are actually longer to cued than to uncued targets (Posner & Cohen, 1984). The latter pattern of longer RTs to cued than to uncued targets is termed inhibition of return (IOR). It is thought that, as a consequence of attention being redirected from the cued location back to the central fixation point, a residual inhibitory code is placed on the location of the cue and/or on the response to the cued location (Posner & Cohen, 1984; see Klein, 2000, for review). This inhibitory code hinders the reorientation of attention back to the cued location and/or the processing of the sensory information at the previously cued/attended location.

When the spatial cue is presented centrally, however, the time course and pattern of cueing effects are distinct from those that emerge when peripheral cues are used. Although peripheral cues typically elicit facilitation that peaks at very short SOAs (100 ms), maximal facilitation effects following central cues typically take longer to arise, occurring around 300 ms. Furthermore, IOR is typically not observed (Cheal & Lyon, 1991; Müller & Rabbitt, 1989). Consequently, peripheral and central cueing paradigms differ in the nature of their behavioral effects on attention orienting and, because of these differences, they are often thought to have separate loci of control. Therefore, it may be possible to distinguish the mechanisms involved in the processing of various cues on the basis of the patterns of RTs to cued and uncued targets that emerge following different types of cues (e.g., peripheral vs. central). Indeed, examination of the time course of the cueing effects following presentation of a central face with gaze directed toward one of two placeholders has been central to longstanding debate on whether or not gaze cues share the mechanisms involved in peripheral or central gaze cueing (or an entirely different mechanism; see Frischen, Bayliss, & Tipper, 2007a, for a review).

The processing of gaze and finger-point cues

The research on social gaze cues has revealed that these cues seem to share properties of both peripheral and central cues. Because gaze cues are presented at central fixation and do not involve a dynamic change in the periphery at a potential target location, gaze cues most resemble central cues. However, the processing advantages (as seen in RTs) from gaze cues can resemble those from peripheral cues, in that they tend to show facilitation effects at very short SOAs (100–300 ms) and these early facilitation effects are relatively immune to top-down influences such as the instruction to ignore the gaze cue (e.g., Driver et al., 1999; Friesen & Kingstone, 1998; see Frischen, Bayliss, & Tipper, 2007a, for a review). In contrast to the effects of peripheral cues (e.g., Posner & Cohen, 1984), however, the facilitation effects stemming from gaze cues are still present at longer SOAs (700–1,000 ms), though the facilitation effects do gradually diminish (Frischen, Bayliss, & Tipper, 2007a). Additionally, IOR is rarely observed in RTs following gaze cues, with the exception of at SOAs greater than 2,000 ms, and only when there is an event to disengage attention from the gazed-at location and draw it back to central fixation, such as a mask or an offset of the gaze cue (Frischen, Smilek, Eastwood, & Tipper, 2007b; Frischen & Tipper, 2004). In sum, it is not clear from the data whether the mechanisms involved in gaze cueing of attention are most similar to those used in peripheral cueing, central cueing, or something different altogether.

The processing of finger-pointing cues has received comparatively little study. In one of the few studies to examine the time course of cueing effects following pointing cues, Ariga and Watanabe (2009) found a facilitation effect at the pointed-to location at a short (107-ms) SOA, but no difference between the RTs to targets at the pointed-to and non-pointed-to locations at a longer (1,000-ms) SOA. The pattern of cueing effects (short-term facilitation with no IOR) is similar to that observed following gaze cues. Interestingly, IOR-like effects have been observed when participants were required to reach and touch a target location more than 1,200 ms after a co-actor pointed to one of two targets (Exp. 3; Atkinson, Simpson, Skarratt, & Cole, 2014). Specifically, RTs were longer for targets presented at locations that the co-actor had previously pointed to than for targets at the other location. These data suggest that the mechanisms of IOR may be activated following a finger-pointing stimulus. Overall, relatively little research has been conducted with the goal of understanding the processing of finger cues, but these few studies do suggest that these stimuli can generate shifts of attention.

An action-centered approach

The extant work in this area of social cueing has typically used discrete button-pressing tasks to record RTs and assess the orienting of attention. However, it is of interest to investigate social cues from an action-oriented approach, because human behavior requires interaction with objects in the environment via reaching, grasping, and manipulation actions. Furthermore, the recording and analysis of reaching movements provides additional variables and measures that may generate deeper insights into cognitive processes. Deviations in an individual’s upper limb trajectory toward or away from nontarget stimuli while reaching toward a target can give an index of the locus of attention and the dynamics of the continuous processing of information (Gallivan & Chapman, 2014; Song & Nakayama, 2009) and, in particular, the coupling between action and attention (see Welsh & Weeks, 2010, for review). Thus, it has become evident that investigating changes in the spatial and temporal characteristics of motor responses can provide new and nuanced insight into the dynamics of the cognitive mechanisms that enable complex behavior. Furthermore, examining the trajectories of reaching movements can provide information on ongoing cognitive mechanisms (see also Chapman et al., 2010; Gallivan & Chapman, 2014; Moher & Song, 2013; Welsh & Elliott, 2004).

Though attention and action can be tightly coupled (see Welsh, 2011), there may be differences in how the relevant orienting and prioritization mechanisms temporally propagate throughout the attentional and motor networks. In work concerning the processing of nonpredictive peripheral cues, Neyedli and Welsh (2012) mapped the time course of facilitation and inhibition following the onset of an attention-capturing peripheral cue in a reaching task. Participants executed aiming movements to targets presented at one of three potential locations 100, 350, 850, or 1,100 ms after the onset of a nonpredictive peripheral cue (a 50-ms “flash”) at one of the potential target locations. The time courses of RTs to cued and uncued targets were similar to the previously described time course in traditional peripheral cueing tasks, wherein IOR emerged: Participants had significantly shorter RTs to uncued than to cued targets at SOAs of 350, 850, and 1,100 ms. No facilitation effect at 100 ms, however, was observed for RTs. The pattern of trajectory deviations differed slightly from this RT pattern. Specifically, even though no facilitation was observed and an inhibitory influence on RTs was observed at 350 ms and later, movement trajectories deviated toward the cued locations at SOAs lower than 350 ms (revealing a facilitation effect associated with the response to the cue), and then deviated away from the cued locations at the larger SOAs of 850 and 1,000 ms (revealing an inhibition effect associated with the response to the cue). These data indicate that the mechanisms that generate “attentional” facilitation and inhibition are represented in the motor system and influence the execution of action, but that motoric facilitation (i.e., presence of a competing response) lasted longer and that motoric inhibition (inhibition of that competing response) was relatively delayed when compared to manifestations of facilitation and inhibition in RTs. This pattern of RTs and trajectory effects was replicated in a further study with smaller increments between SOAs (Welsh, Neyedli, & Tremblay, 2013). Consequently, it is possible that different behavioral expressions of facilitation and inhibition are seen in RTs and in measures of trajectory deviation. As such, kinematic analyses of reaching movements can provide new insights into the sensorimotor systems involved in the processing of different types of stimuli. For this reason, investigating attention-orienting cues with upper-limb reaching movements can clearly provide additional insight into the associated neural mechanisms.

The present experiments

The purpose of the present experiments was to examine the mechanisms underlying the processing of social cues (gaze and pointing) by examining the influences of these cues on the spatial and temporal characteristics of upper-limb reaching movements. Three experiments are reported herein. Experiments 1 and 2 concerned gaze cues, and Experiment 3 concerned finger-pointing cues. We predicted several patterns of data, based on the premise that social cues engage attentional mechanisms and that these attentional mechanisms are linked to and exert an influence on the motor system (as in, e.g., Welsh, 2011). If social cues engage mechanisms that are similar to those engaged with the bottom-up processing of peripheral cues, then patterns of RTs and movement trajectory deviations, and dissociations between the two behavioral effects, similar to those observed in Neyedli and Welsh (2012), might be observed here. Specifically, we predicted that RTs might be shorter to cued than to uncued targets at short SOAs, and that trajectories to uncued targets would deviate toward the cued location at short SOAs. Predictions regarding deviations toward or away from the cued location at longer SOAs were unclear, given the previously observed differences in the emergence of the spatial and temporal effects at longer SOAs (Neyedli & Welsh, 2012). Indeed, it was even possible that inhibitory mechanisms might work in the motor system while excitatory mechanisms operate in attention. On the other hand, if attentional shifts following social cues do not drive response-producing processes to interact with the object at the cued location, then even though there still might be cueing effects in RTs, the trajectories of the movements to uncued and cued target locations would not differ.

Experiment 1

The participants in Experiment 1 completed aiming movements to targets that appeared randomly at a left or right target location. A gaze cue preceded the onset of the target by SOAs varying from 100 to 2,400 ms. On the basis of previous research, we predicted that RTs would be shorter to cued targets than uncued targets at most SOAs. It was possible, though unlikely, that RTs might be longer to cued targets than to uncued targets at the longest SOA, because the gaze cue in this experiment remained fixed on the potential target location throughout a given trial (see Frischen & Tipper, 2004). The pattern of trajectory deviations of the hand movements to the target would depend on the relationship between the attention and action systems and the mechanisms that generated the changes in RTs following gaze cues. Although it is unlikely for IOR to emerge in RTs, it is possible that such effects of inhibition could be detected in the trajectory deviations, as a dissociation between these behavioral effects has previously been shown (e.g., Neyedli & Welsh, 2012; Welsh et al., 2013).

Method

Participants

Twenty participants (15 women, five men) 19–37 years of age (mean age = 26 years) were recruited from the University of Toronto community. All participants were right-hand dominant and had normal or corrected-to-normal vision. Participants provided full and informed consent, and they were monetarily compensated for their time. All procedures were approved and were consistent with the standards put forth by the University of Toronto Research Ethics Board.

Apparatus

Participants sat comfortably in front of a 24-in. widescreen monitor (Acer GD235HZ) with a resolution of 1,920 (w) × 1,080 (h) pixels. The monitor was angled approximately 20 deg from the surface of the table. The initial display included a home position (a blue circle 1.5 cm in diameter) located 1 cm above the bottom of the screen, and two open blue square target placeholders (2 cm) that were located approximately 28 cm horizontally from one another and 25 cm diagonally from the home position. The cue stimulus was the image of a young adult male face, whose gaze (face and eyes) was initially directed centrally toward the participant. The face measured approximately 6 cm (w) and 8 cm (h) and was centered between the two target placeholders. The eye gaze cue consisted of an image of the same male with the iris and pupils displaced to the left or right. All images were presented on a light gray background. Throughout the experiment, participants were in view of an optoelectric motion-tracking system (Optotrack, Northern Digital Inc., Waterloo, Ontario, Canada). An infrared-emitting diode (IRED) was attached to the participants’ right index fingers and the motion of this IRED was recorded at a rate of 250 Hz for 1,500 ms, starting at the onset of the target.

Task and procedure

In a given trial, a participant would begin with his or her right index finger on the home position. After 1,000 ms, the eyes of the face presented in the visual display would shift toward the left or right target placeholder, providing a nonpredictive gaze cue (see Fig. 1 for a schematic depiction of the time course of a given trial). After a variable SOA (100, 250, 400, 700, 1,000, 1,700, or 2,400 ms), one of the target placeholders would become solid, signaling the participant to move their index finger from the home position to the given target placeholder as soon as possible. We used a relatively long range of SOAs in order to assess the time course of facilitation and potential inhibition (as it was at these longer SOAs that IOR for gaze cues was observed by Frischen & Tipper, 2004).

Fig. 1
figure 1

Schematic of the timeline for a given trial in Experiment 1. The gaze cue could occur toward the right or the left target. The response was to reach out and touch the dark blue target (square)

Full size image

The participant’s movements were recorded for 1,500 ms after which the display would reset to the initial display and the participant could initiate another trial by placing his or her finger in the home position. Participants were instructed to move toward and touch the target location as soon as they saw one of the target placeholders become solid. They were informed that the preceding gaze cue was entirely nonpredictive (i.e., that the target would appear on both the left and right side equally and randomly, and the cue and SOA were presented randomly with respect to the target). The target could be presented at either the cued (i.e., in the location specified by the cue) or the uncued (i.e., in the direction opposite the cue) location. Fifteen cued and uncued trials apiece were performed for both the left and right target placeholders at each of the seven SOAs, for a total 420 experimental trials. Trial types were randomly distributed throughout the experiment and were broken up into five blocks of 84 trials, with self-paced breaks between blocks. Prior to the experimental trials, participants executed 14 practice trials. The total time during testing was approximately 1 h.

Data reduction and analysis

The data were stored for offline analysis using a custom analysis program to calculate kinematic data using Matlab (The MathWorks Inc.). The IRED position data were filtered using a second-order dual-pass Butterworth filter with a low-pass cutoff frequency of 10 Hz. The position data were then differentiated in order to obtain the instantaneous velocity of the movement. The start and the end of the movement were identified as the first samples in which the instantaneous velocity in the z-axis (vertical axis) surpassed and fell below 50 mm/s, respectively, for three consecutive samples. Each trial was inspected visually. If these criteria did not isolate the start and end of the movement properly (e.g., due to the participant making unnecessary movement at the home or end position), the start and end of movement were determined visually. The reaction time (RT) was defined as the time from target onset to the calculated movement start. The movement time (MT) was defined as the time from the calculated movement start to the calculated movement end.

Calculation of initial movement angle

The initial movement angle for a given trial was calculated in order to provide an index of the initial curvature or direction of a participant’s movement. This measure was calculated by finding the absolute angle in degrees between the y-axis and the line created from the (x, y) coordinates of their starting position and the (x, y) coordinates of the participant’s position at 20% of the movement trajectory. This measure was chosen because it represented a point in the movement trajectory that best represented the initial representations of action in the motor system, because it was early enough that the trajectory was not likely to be subject to online corrective processes based on visual information (Elliott et al., 2010).Footnote 1

Outlier analysis

Thirty-five trials across all participants (0.4% of all data) were removed prior to analysis for obvious recording and/or experimental errors. Trials in which a participant’s RT was less than 100 ms or greater than 1,000 ms, as well as trials in which MT was greater than 1,000 ms, were removed from the analysis (0.1% of all data). Following these initial screenings, trials in which RT fell outside 2.5 standard deviations for each participant and condition were also removed from the analysis (2% of all data). Overall, 2.6% of the entire data set was excluded from the analysis.

Statistical analysis

A 2 (target: cued, uncued) × 7 (SOA: 100, 250, 400, 700, 1,000, 1,700, 2,400 ms) repeated measures analysis of variance (ANOVA) was conducted on each of the three dependent measures: RT, MT, and IMA. For each of these analyses, when Mauchly’s test of sphericity indicated that the assumption of sphericity was violated, Hyun–Feldt-corrected values are reported (indicated by degrees of freedom with decimals). Where a significant interaction was found, planned comparisons were conducted to determine differences between the cued and uncued trials at each of the seven SOAs.

Results

Reaction time

Significant main effects of target, F(1, 19) = 42.97, p < .001, ηp2 = .69, and SOA, F(3.19, 60.63) = 10.32, p < .001, ηp2 = .35, were found (Fig. 2, top left). The main effect of target revealed that the RTs on cued-target trials (M = 299 ms, SD = 46.98) were shorter than those on uncued-target trials (M = 310 ms, SD = 48.46). For the main effect of SOA, examination of the RT data presented in Fig. 2 suggests that RTs were longer at shorter SOAs and generally decreased as SOA increased—consistent with this observation, we found a significant linear trend, F(1, 19) = 17.07, p < .01, ηp2 = .47. There was no significant Target × SOA interaction, F(6, 114) = 1.78, p = .11, ηp2 = .09. This result indicates that no statistical differences in the cueing effect emerged across the seven SOAs, and in particular, there was no evidence of IOR at the long SOAs.

Fig. 2
figure 2

Schematic of the timeline for a given trial for Experiment 2. The gaze cue could occur towards the right or left target. The response was to reach out and touch the dark blue target (square)

Full size image

Movement time

We observed no significant main effect of target, F(1, 19) = 1.42, p = .25, ηp2 = .07, or SOA, F(4.75, 90.26) = 1.02, p = .41, ηp2 = .05, and no significant interaction between target and SOA, F(6, 114) = 1.35, p = .24, ηp2 = .07. Consequently, it is clear that neither the cue nor the SOA had any significant effect on MTs.

Initial movement angle

There was no significant main effect of target, F(1, 19) = .001, p = .98, ηp2 < .001, or SOA, F(4.46, 5.90) = 2.26, p = .06, ηp2 = .10, and no significant interaction between the two factors, F(4.55, 86.35) = 1.81, p = .12, ηp2 = .09 (Fig. 2, top right). Therefore, neither cue nor SOA had a significant effect on IMAs in this experiment.

Discussion

The present experiment used an upper-limb aiming task to investigate the motoric and attentional components of facilitation and inhibition in a gaze-cueing task. Participants reached toward a left or right target placeholder in the presence of a gaze cue that was either in the direction of or opposite to the target. Two key findings were revealed.

First, RTs to cued targets were shorter than RTs to uncued targets. This facilitation effect associated with the centrally presented social cue is consistent with results in the previous literature (Friesen & Kingstone, 1998; Frischen, Smilek, & Tipper, 2007b; Frischen & Tipper, 2004). However, the absence of an interaction between RT and SOA suggests that there were no statistically significant changes in the cueing effect as the time between the cue and the target increased. Thus, the eye gaze cue and reaching task used in this experiment did not lead to the patterns of RTs to cued and uncued targets from previous gaze-cueing experiments (i.e., short-term facilitation and later diminishing of cueing effects and possibly of IOR; see also Frischen et al., 2007a, b; Frischen & Tipper, 2004).

The second, and perhaps more novel, finding was that the social gaze cue did not affect either the temporal (MT) or spatial (IMA) characteristics of the movement. The latter finding of an absent cueing effect in IMAs was not expected, given the history of previous work demonstrating coupling between the attentional effects observed in RTs and IMAs (e.g., Lee, 1999; Neyedli & Welsh, 2012; Welsh, 2011; Welsh et al., 2013). This unexpected distinction might suggest that, although gaze cues are capable of influencing information processing and response initiation processes, these processes may be less tightly coupled with the manual motor system.

Experiment 2

Experiment 2 was designed and conducted to further address the processing of gaze cues, because the results of Experiment 1 were not entirely consistent with the findings of previous work on cueing effects, in two main ways. First and foremost, although facilitatory cueing effects were present in RTs, there were no differences in the trajectories of the executed aiming movements. This distinction between the presence of cueing effects in RTs and the absence of cueing effects in trajectories is not consistent with a series of studies that revealed cueing effects in both RTs and trajectories when peripheral cues were used (e.g., Lee, 1999; Neyedli & Welsh, 2012; Welsh, 2011; Welsh et al., 2013). Thus, the data from Experiment 1 suggest that gaze cues might not activate responses in the same way that peripheral cues do. Given this unexpected result, Experiment 2 was conducted to provide an additional test of the prediction that response codes would be activated following the shift of attention generated by gaze cues.

Second, the gaze cues in Experiment 1 led to a somewhat different pattern of RTs from those revealed in previous studies of gaze cues. Specifically, an overall facilitation effect was associated with the cue in RTs, and this effect was not significantly different across SOAs. The latter RT effect is not entirely consistent with, for example, the results of the study by Frischen and Tipper (2004), who demonstrated that no cueing effects were present at the 1,200-ms SOA. It is possible that the chosen methodology might account for the discrepancy in results. In Experiment 1 reported herein, the central face remained gazing toward the placeholder throughout the longest SOA period. In the Frischen and Tipper study, the greatest changes across SOA were found when the methodology increased the potential that attention would be removed from the cued peripheral target. For instance, IOR emerged at the long SOA when the central face with the gaze cue was masked with a fixation cross before the presentation of the target, because it is likely that the mask and disappearance of the gaze cue disengages attention from the cued location (and presumably reengages attention centrally). Thus, it is likely that the sustained cueing effects in Experiment 1 were due to the continued orientation of the central gaze cue to one of the target locations. It is possible that the chosen methodology also failed to engage response-producing processes in the motor system, which in turn led to the noneffects of cue on movement trajectories (see the previous paragraph). Therefore, the gaze of the face in Experiment 2 returned to a neutral position (i.e., direct gaze at the participant) 150 ms after the initial presentation of the gaze cue toward one of the cued locations. This return of gaze to the central location might have the effect of disengaging attention from the peripheral location and drawing it back to central fixation (Böckler, van der Wel, & Welsh, 2014, 2015). All other aspects of the experiment remained identical to those of Experiment 1.

Method

Participants

Twenty participants (15 women, five men) 18–30 years of age (mean age = 23.5 years) were recruited from the University of Toronto community. All participants were right-handed and had normal or corrected-to-normal vision. Participants provided full and informed consent and were monetarily compensated for their time. All procedures were approved and were consistent with the standards put forth by the University of Toronto Research Ethics Board. Prior to the analysis, four participants were removed due to technical difficulties that resulted in improper recording of the data.

Apparatus

The apparatus and experimental setup were identical to those aspects of Experiment 1. All display and movement recording properties were consistent across the two experiments.

Task and procedure

The task, timing of the stimuli, and procedure were identical to Experiment 1 except that the gaze of the model in central fixation would shift back to the center neutral position 150 ms after it shifted to the periphery (see Fig. 3 for a schematic depiction of the time course of an individual trial).

Fig. 3
figure 3

Schematic of the timeline for a given trial for Experiment 3. The pointing cue could occur towards the right or left target. The response was to reach out and touch the dark blue target (square)

Full size image

Data reduction and analysis

The data were processed and analyzed in a manner similar to that in Experiment 1. A total of 62 trials across all participants (1% of the data) were removed prior to analysis for obvious recording and/or experimental errors. As in Experiment 1, trials in which a participant’s RT was less than 100 ms or greater than 1,000 ms, as well as trials in which the MT was greater than 1,000 ms, were also removed (0.2% of all data). Following this initial screening, trials in which the RT fell outside 2.5 standard deviations for each condition within a participant’s responses were removed from the analysis (2.3% of all data). All together, 3.5% of the data set was excluded from the statistical analysis. One participant’s mean MTs were longer than 2.5 SDs of the mean MT for the group as a whole. For this reason, this participant was removed for moving too slowly and not following instructions. Thus, the final sample size was 15 for Experiment 2. The statistical analyses followed those outlined in Experiment 1.

Results

Reaction time

Significant main effects of target, F(1, 14) = 9.50, p < .01, ηp2 = .40, and SOA, F(4.85, 67.99) = 21.78, p < .001, ηp2 = .61, were found. The main effect of target revealed that RTs to cued targets (M = 264 ms, SD = 33.44) were shorter than those to uncued targets (M = 271 ms, SD = 30.69). For the main effect of SOA, there was a significant linear trend, F(1, 14) = 35.29, p < .001, ηp2 = .72, revealing that RTs generally increased as a function of SOA. We also observed a significant Target × SOA interaction, F(6, 84) = 3.18, p < .01 , ηp2 = .19 (Fig. 2, center left). Planned comparisons revealed that RTs were significantly shorter in reaches to cued targets than in reaches to uncued targets at the 250-ms, t(14) = 5.0, p < .001, 95% CI [– 26.07, – 7.20], dz = 1.29; 400-ms, t(14) = 2.47, p < .05, 95% CI [– 15.01, – 1.07], dz = 0.64; 700-ms, t(14) = 2.13, p < .05, 95% CI [– 17.47, – 0.68], dz = 0.59; and 1,000-ms SOAs, t(14) = 2.74, p < .05, 95% CI [– 12.18, – 1.9], dz = 0.71. There were no other statistically significant differences (p > .05) between cued and uncued targets at the remaining SOAs (i.e., 100, 1,700, and 2,400 ms). Therefore, it appears that at shorter SOAs there were facilitation effects of cue (with the exception of the 100-ms SOA) that dissipated at SOAs longer than 1,000 ms. No evidence of IOR was observed.

Movement time

No effect was detected of target, F(1, 14) = 0.84, p = .38, ηp2 = .06, nor of SOA, F(6, 84) = 1.35, p = .09, ηp2 = .09. We also found no significant interaction between target and SOA, F(6, 84) = 0.35, p = .91, ηp2 = .02. Therefore, neither the cue nor the SOA had a significant effect on MTs in this experiment.

Initial movement angle

No significant main effect of target, F(1, 14) = 0.46, p = .51, ηp2 = .03, or Target × SOA interaction, F(6, 84) = 0.30, p = .94, ηp2 = .02, was found. There was, however, a significant main effect of SOA, F(6, 84) = 2.64, p < .05, ηp2 = .16 (Fig. 2, center right). Accordingly, we observed a significant linear trend for SOA, F(1, 14) = 13.32, p < .01, ηp2 = .49, revealing that IMAs generally increased as SOA increased. Overall, and consistent with Experiment 1, there was no difference in IMAs between reaches to cued and uncued targets.

Discussion

Experiment 2 differed from Experiment 1 in that the gaze cue did not remain fixed on a given target. Following 150 ms, the direction of the gaze cue returned from one of the placeholders to the center neutral position and was oriented toward the participant. A different pattern of RTs emerged in Experiment 2 than in Experiment 1. Of particular interest is the significant Target × SOA interaction for RTs. Planned comparisons demonstrated that this interaction was driven by a facilitation associated with the cue at SOAs of 250–1,000 ms that diminished at longer SOAs (i.e., 1,700 ms). This interaction more closely matches the results of previous experiments, demonstrating a loss of facilitation effects at longer SOAs (Friesen & Kingstone, 1998; Friesen, Ristic, & Kingstone, 2004; Frischen, Bayliss, & Tipper, 2007a; Frischen & Tipper, 2004). Despite the diminished cueing effect, however, IOR still did not emerge. This finding is likely due to the persistence of the gaze cue, as previous findings only demonstrated IOR when the gaze cue was completely masked (i.e., removed) before the presentation of the target onset (Frischen & Tipper, 2004).

The more theoretically relevant finding from Experiments 1 and 2, however, is that there were no IMA effects, despite the RT findings noted here. Although the change in methodology between Experiments 1 and 2 caused a shift in the pattern of RT effects across SOAs, no such change was found for movement trajectories. This repeated null finding in IMAs provides some support for the notion that the results of Experiment 1 were not likely to be a Type II error. Thus, collectively, these findings suggest that gaze cueing may not alter activity in the motor system and exert an influence on reaching movements in the way that peripheral cues have been shown to do (e.g., Neyedli & Welsh, 2012).

Experiment 3

Although gaze cues may cause an orienting of attention (as evidenced by the RT effects in Exps. 1 and 2), it appears that these gaze cues may not be sufficiently or strongly linked to representations of pointing or reaching movements in a meaningful way. The disconnect between RT and trajectory effects is in contrast to the previous work showing cueing effects using peripheral cues (e.g., Welsh, 2011; Welsh & Elliott, 2004; Welsh et al., 2013). This difference between peripheral cues and social gaze cues may be an informative difference, especially considering eye–hand coupling and how eye gaze tends to precede the hand to the object with which the individual will interact (see Flanagan & Johansson, 2003).

Experiment 3 was conducted to investigate whether or not a social cue that might be more tightly linked to the manual motor system (e.g., a pointing finger) affects the spatiotemporal characteristics of goal-directed movements. Although finger pointing may be social and communicative in a manner similar to eye gaze, the observation of a pointing finger might engage the neural action codes associated with pointing in the observer—for instance, the action observation (putative mirror neuron system; see Gallese, Fadiga, Fogassi, & Rizzolatti, 1996; Rizzolatti & Craighero, 2004). This alternative social cue might then be more closely linked to the reaching or pointing movements employed in the present tasks, leading to a stronger propensity to generate competing response codes that would lead to larger deviations in the trajectory of the participants’ hand movements. Indeed, trajectory deviations in reaching movements executed in a Simon effect task have previously been reported when a hand with a pointing finger was used as the stimulus (Welsh, Pacione, Neyedli, Ray, & Ou, 2015). If all social cues affect response initiation processes, but not response planning and execution, then the same pattern of cueing effects in RTs but not IMAs that was observed in Experiments 1 and 2 would be observed in Experiment 3. If, on the other hand, a finger pointing toward a target is more closely coupled with the reaching movement being performed, then perhaps RT effects and trajectory deviations would be observed in Experiment 3. We predicted, therefore, that trajectory deviations would differ more between cued and uncued targets in this experiment.

Method

Participants

Twenty participants (11 women, eight men, and the age and gender data for one participant are not reported) 18–33 years of age (mean age = 22.6 years) were recruited from the University of Toronto community. All participants were right-handed and had normal or corrected-to-normal vision. Participants provided full and informed consent and were financially compensated for their time. All procedures were approved by and were consistent with the standards put forth by the University of Toronto Research Ethics Board. The data set from one participant was removed prior to analysis due to a technical difficulty that resulted in improper recording of the data.

Apparatus

The apparatus and experimental setup were identical to those of Experiments 1 and 2. Recording properties were consistent across the three experiments. The display of the home position and target placeholders was identical to that aspect of Experiments 1 and 2. The centrally presented images consisted of the hand of a Caucasian individual pointing at the participant or at one of the two target locations. The central pointing image measured approximately 7 cm (w) × 6.5 cm (h) and was centered between the two target placeholders.

Task and procedure

The trial procedures for Experiment 3 were identical to those of Experiment 1, except for the stimulus that was used. The left- and right-pointing finger images measured approximately 12 cm (w) × 6.5 cm (h). As in Experiment 1, the pointed-finger cue remained fixed toward the given target placeholder throughout the trial (see Fig. 4 for a schematic depiction of the time course of a given trial). After a variable SOA, one of the target placeholders became solid, signaling the participant to place their index finger in the given target placeholder. The same seven SOAs were used as in the previous experiments: 100, 250, 400, 700, 1,000, 1,700, and 2,400 ms. Fifteen cued and uncued trials were performed for both the left and right target placeholders at each of the seven SOAs (total trials = 420).

Fig. 4
figure 4

Reaction Time (left side panel) and initial movement angle (IMA, right side panel) for reach of Experiments 1, 2, and 3. Asterisks (*) denote significant differences between cued and uncued trials

Full size image

Data reduction and analysis

Altogether, 225 trials were removed across all participants (2.8% of the data) prior to the analysis, for obvious recording and/or experimental errors. The start and end of the movement were determined in the same manner as in the previous experiments. As in the previous experiments, trials in which a participant’s RT was less than 100 ms or greater than 1,000 ms, as well as trials in which the MT was greater than 1,000 ms, were removed from the analysis (0.14% of all trials). Following this initial screening, trials in which the RT fell outside 2.5 standard deviations for each participant and condition were removed from the analysis (2.32% of all trials). The statistical analysis was the same as we outlined in Experiment 1.

Results

Reaction time

We found a significant effect of target, F(1, 18) = 6.29, p < .05, ηp2 = .26, and a significant Target × SOA interaction, F(6, 108) = 10.43, p < .001, ηp2 = .37 (Fig. 2, bottom left). The main effect of target revealed that RTs to cued targets (M = 344 ms, SD = 48.91) were shorter than those to uncued targets (M = 354 ms, SD = 52.95). There was no significant main effect of SOA, F(6, 108) = 1.83, p = .10, ηp2 = 0.09. Planned comparisons revealed that RTs were significantly shorter in reaches to cued targets than in reaches to uncued targets at SOAs of 100 ms, t(18) = 4.17, p < .01, 95% CI [– 41.29, – 13.61], dz = 0.96, and 250 ms, t(18) = 5.04, p < .001, 95% CI [– 47.43, – 19.42], dz = 1.16. No other statistically significant cued-versus-uncued target RT differences emerged across the other SOAs (ps > .05).

Movement time

We observed no significant effects of target, F(1, 18) = .023, p = .88, ηp2 = .001, or SOA, F(6, 108) = 1.07, p = .29, ηp2 = .06, and no significant interaction between the two factors, F(6, 108) = .044, p = .85, ηp2 = .02. Therefore, there was no effect of either cue or SOA on MTs.

Initial movement angle

Significant main effects of target, F(1, 18) = 5.37, p < .05, ηp2 = .23, and SOA, F(6, 108) = 2.91, p < .05, ηp2 = .14, were found. There was, however, no Target × SOA interaction, F(4.46, 80.36) = 0.47, p = .79, ηp2 = .03 (Fig. 2, bottom right). The main effect of target indicated that IMAs in reaches to uncued targets were smaller (i.e., more central, M = 26.22, SD = 5.18) than IMAs in reaches to cued targets overall (M = 27.02, SD = 5.69). That is, the movements to locations that were not cued (i.e., not pointed at) deviated toward the cued (pointed-at) location.

Between-experiment analysis

To assess any potential significant differences in trajectory deviations to cued and uncued targets between gaze and pointing cues, the IMA data from the two experiments with the most similar designs were analyzed using a 2 (target: cued, uncued) × 6 (SOA: 100, 250, 400, 700, 1,000, 1,700, 2,400) × 2 (experiment: 1, 3) mixed ANOVA with experiment as a between-group factor and SOA and target as within-subjects factors. There was no significant main effect of SOA, F(6, 222) = 1.29, p = .26, ηp2 = .03, and no significant interactions between target and SOA, F(6, 222) = 0.20, p = .98, ηp2 = .005, or target, SOA, and experiment, F(6, 222) = 1.56, p = .16, ηp2 = .04. There was, however, a significant SOA × Experiment interaction, F(6, 222), p < .001, ηp2 = .10, suggesting that the pattern of IMAs across SOAs varied differently between the two experiments. The main effect of target approached but did not reach conventional levels of statistical significance, F(1, 37) = 3.67, p = .06, ηp2 = .09. Likewise, the critical interaction between Target × Experiment interaction approached, but did not surpass, conventional levels of statistical significance, F = 3.79, p = .06, ηp2 = .09.

Discussion

In Experiment 3 we explored the mechanisms of social cueing activated when a cue that more closely matched the response was presented. The finger-pointing cue used in this experiment generated a short-lived facilitatory cueing effect in RTs. Unlike in Experiments 1 and 2, however, significant effects of cue were now also detected in the initial angle of the movement trajectory. The between-experiment analysis suggested that the difference in IMAs between cued and uncued targets tended to be larger in Experiment 3 than in Experiment 1, although this difference was not statistically significant. Overall, the results of Experiment 3 suggest that when the social cue presented (i.e., pointing, in this case) more closely matched the response being generated, reach trajectories were more likely to be affected.

General discussion

The aim of the three present experiments was to investigate the mechanisms of social cueing by assessing the time course of facilitation and potential inhibition in social gaze and finger cueing using an upper-limb reaching task. These experiments were grounded in action-centered theories of attention (Song & Nakayama, 2009; Tipper, Lortie, & Baylis, 1992; Welsh & Weeks, 2010) and previous work showing that the trajectories of limb movements to targets are affected by the attentional mechanisms activated by a preceding cue (Lee, 1999; Welsh, 2011; Welsh et al., 2013). As such, the temporal and kinematic characteristics of upper-limb reaching movements were analyzed to determine the linked attentional and motoric components of facilitation and inhibition in each of the cueing paradigms. Although the analyses of the RTs in each of the experiments were consistent with previous findings related to gaze cueing and generally revealed facilitatory effects of the social cues, the critical findings were that no differences in initial movement trajectory were found between reaches to cued and uncued targets following gaze cues (Exps. 1 and 2). Conversely, we observed a significant effect of the cue on the reach trajectories of participants when a pointed finger was used as the cue (Exp. 3). Consequently, the data seem to indicate that task specificity and perception–action coupling (i.e., a direct match in effector, in this case) might play a role in the processing of these cues and, as a result, in their subsequent effects on action initiation and execution.

To address the main purpose of the experiments, the present data are consistent with the idea that the mechanisms underlying gaze cues are different from those that lead to stimulus-driven attentional shifts following peripheral cues. This conclusion is based on a comparison between the results of Experiments 1 and 2 and a series of previous studies in which participants completed aiming movements to targets following dynamic peripheral cues (e.g., Lee, 1999; Neyedli & Welsh, 2012; Welsh, 2011; Welsh et al., 2013). In Experiments 1 and 2, clear facilitatory effects associated with the social gaze cue were observed in response initiation times (RTs), but there was no evidence of any manifestation of facilitation (or inhibition) in the movement trajectories. This pair of findings stands in contrast to the results of numerous studies that have provided evidence of excitatory and inhibitory mechanisms expressed in both RTs and movement trajectories in studies employing peripheral cues to evoke stimulus-driven captures of attention (e.g., Welsh, 2011; Welsh et al., 2013). It has been previously suggested that, because of the tight coupling of attention and action processes, the dedication of attention to a specific location or object activates processes to produce a response to interact with that location (Tipper et al., 1992; Welsh & Weeks, 2010). Thus, the absence of such a tight coupling between attention and action production following gaze cues suggests that the shifts of attention following gaze cues and dynamic peripheral cues are likely to be generated by different mechanisms. Similar conclusions regarding the potential for independent processing channels for social (gaze) and non-social (motion) cues have been previously made (Böckler et al., 2014, 2015).

Conclusions regarding the (dis) similarity of the mechanisms underlying gaze and central symbolic cues are more difficult to draw, because no study that we are aware of has directly examined the pattern of RTs and trajectory deviations that emerges following nonpredictive centrally presented symbolic cues such as arrows. Nonetheless, there is evidence from studies involving keypress tasks that social gaze cues are processed differently from centrally presented arrow cues (Marotta, Lupiáñez, Martella, & Casagrande, 2012; Marotta, Román-Caballero, & Lupiáñez, 2018). Furthermore, there is a contrast between the results of Experiments 1 and 2, in which no trajectory deviations were observed following gaze cues, and Experiment 3, in which a centrally presented pointing finger generated facilitatory effects in RTs and trajectory deviations. In this sense, not all social cues influence the attention and action system in the same way. As such, social cues (and indeed, nonsocial centrally presented cues) should not be taken as one and the same and nuanced explorations of cueing effects are required (see Atkinson, Simpson, & Cole, 2018, for a review and discussion).

As we already suggested, the trajectory deviations in the pointing-cue task might have emerged because of a match between the cue and the effector that the participant used for the task. Previous work has demonstrated that the relationship between the characteristics of the target and nontarget stimuli and the type of response that will be executed can have an influence on the interference caused by distractors. For instance, Welsh and Pratt (2008; see also Welsh & Zbinden, 2009) demonstrated that offset distractors caused interference in responding to an onset target in a key-press task, but not in an upper-limb aiming task. The authors suggested that this response-related difference in the impact of the offset distractor on an onset target was due to the differences in action affordances of the stimuli between the two tasks. Because the visuomotor system needs a stable source of information about the target to ensure that accurate termination of an aiming movement, an offset stimulus is not salient to the attention/action system because it does not provide that stable source of visual information (there is no longer any stimulus information at the location of an offset). Hence, the offset distractor does not capture attention and cause interference when an aiming movement is executed, because it is not salient to the system when an aiming movement is required. In contrast, the attention/action system does not need a stable source of endpoint information to ensure accuracy when discrete button responses are being executed. Hence, any dynamic change in the environment is salient to the visuomotor system and, in this case, offset stimuli can capture attention and cause interference. Similarly, Bekkering and Neggers (2002) found that visual processing of distractor items was different when participants prepared a reach-to-grasp action than when they prepared a reach-to-point action. Specifically, participants made more saccadic eye movements to a nontarget object with the wrong orientation when they reached to grasp the target object than when they reached to point to the object. When the distractor item was of a different color, there were no differences between the tasks. Thus, orientation was a salient feature that caused enhanced distraction when performing grasping actions, in which orientation is a key feature. Overall, these studies demonstrate an interaction between response type (i.e., action) and the stimulus characteristics that capture attention.

In the context of the present work, it is possible that the nature of the relationship between the characteristics of the upcoming action and those of the stimuli shaped processing of the cue stimuli in the present study and influenced the motoric effects of spatial attention. Specifically, it is likely that neural codes for visual perception of the gaze cue and hand cue are coupled with different neural codes for action. There is a great deal of overlap between the neuronal networks active in action observation and those active during action execution (Grèzes & Decety, 2001). This overlap might be rooted in a mirror-neuron-like system (MNS), wherein neurons are active for both action perception and execution (Rizzolatti & Craighero, 2004). Consequently, when an action is observed, (e.g., the shift of gaze or the turning of a hand), the perception of this action can in turn activate the neural codes associated with actual performance and control of that action. Perhaps because of the greater overlap in the coding of hand cues and hand actions, the pointed finger led to the activation of a compatible response, whereas the eye gaze cue did not.

Furthermore, it should be noted that neurophysiological work has demonstrated that eyes (faces) and hands (bodies) are represented and processed in distinct areas of the human cerebral cortex. For instance, areas of the visual cortex have been found to be particularly selective to the perception of faces (e.g., the occipital face area and fusiform face area), whereas other distinct areas are selective for the perception of bodies, and hands in particular (e.g., the extrastriate body area and the fusiform body area) (Peelen & Downing, 2007). Additionally, in parietal cortex, the lateral intraparietal area is connected with other areas associated with gaze control (e.g., the frontal eye field and the superior colliculus), whereas the medial intraparietal area is more active in reaching movements and is connected with frontal regions that are also associated with reaching (Cisek & Kalaska, 2010). Therefore, it is possible that the different apparent motion cues (i.e., eye gaze shift and hand rotation) used in this experiment were perceived with distinct areas and were then linked with distinct actions and, consequently, control systems.

In summary, the present experiments have demonstrated that although gaze shifts may facilitate temporal aspects of the initiation of reaching movements such as RT, there is little effect of gaze cueing on the spatial parameters of reaching movements. These data indicate that it is unlikely that a response-producing process is activated when a shift of attention has been made to a location following a gaze cue. When the gaze cue was replaced with a finger-pointing cue, however, spatial effects on the movement trajectory were seen, indicating that a response-producing process was activated following the finger-pointing cue. Overall, this work highlights the important interactions between action systems, cognition, and attention by demonstrating how the effector of a social cue (i.e., eye or hand) can influence the spatial characteristics of a reaching response.