Eye direction aftereffect
- First Online:
- Cite this article as:
- Seyama, J. & Nagayama, R.S. Psychological Research (2006) 70: 59. doi:10.1007/s00426-004-0188-3
- 183 Views
Three experiments using computer-generated human figures showed that after a prolonged observation of eyes looking to the left (or right), eyes looking directly toward the viewer appeared directed to the right (or left). Observation of an arrow pointing left or right did not induce this aftereffect on the perceived eye direction. Happy faces produced the aftereffect more effectively than surprised faces, even though the image features of the eyes were identical for both the happy and the surprised faces. These results suggest that the eye direction aftereffect may reflect the adaptation of relatively higher-level mechanisms analyzing the other’s eye direction.
Viewing abstract visual stimuli such as tilted lines, curves, gratings, and simple shapes is widely known to induce various types of aftereffects (e.g., Coltheart, 1971; Frisby, 1979; Köhler & Emery, 1947). More natural stimuli, such as facial images, also can induce aftereffects (O’Leary & McMahon, 1991; Watson & Clifford, 2003; Webster & MacLin, 1999; Zhao & Chubb, 2001). For example, Webster and MacLin (1999) demonstrated that after observation of an abnormally expanded (or contracted) face, a normal face appeared contracted (or expanded). This face distortion aftereffect (FDAE) can be viewed as a negative aftereffect, since prolonged observation of an abnormally distorted face (adaptation face) induced an aftereffect where a normal face appeared distorted in the opposite manner to the adaptation face.
Facial images that vary within normal ranges also can induce aftereffects for facial information. Leopold and colleagues reported that observation of a face with a certain identity biased the identification of the face observed afterward (Leopold, O’Toole, Vetter, & Blanz, 2001). Viewers did not perceive any specific identity for an average face before adaptation, but they reported after adaptation that the average face had an identity that contrasted to the adaptation face’s identity. Webster and colleagues reported that aftereffects occurred for facial expression, gender, and ethnicity (Webster, Kaping, Mizokami, & Duhamel, 2004). The viewers in their experiment judged characteristics of faces that varied along a male–female axis, a Japanese–Caucasian axis, a happy–angry axis, a disgust–surprise axis, or a fear–contempt axis. After adaptation to a face with a characteristic of one end of the axis, an average face (a face at the midpoint of the axis) appeared to have the characteristic of the opposite end of the axis. For example, after adaptation to a male face, an average face (i.e., an androgynous face) appeared as a female face.
Rhodes and colleagues reported that the aftereffect for facial attractiveness occurred when the FDAE was induced by adaptation to an abnormally distorted face (Rhodes, Jeffery, Watson, Clifford, & Nakayama, 2003). After adapting to a distorted face, the face that was slightly distorted toward the adapted face was rated as more attractive than the original face, which had been judged as highly attractive.
Observation of a facial image should activate the detectors for the low-level image features that comprise the facial image. Adaptation by the low-level feature detectors may induce various types of classic aftereffects, which contribute to the aftereffects for facial information. However, it has been argued that the aftereffects for facial information reflect at least partly the adaptation of higher-level mechanisms (Leopold et al., 2001; Rhodes et al., 2003; Watson & Clifford, 2003; Webster & MacLin, 1999; Zhao & Chubb, 2001). The aftereffects for facial information were observed even when adaptation and test stimuli had different sizes (Zhao & Chubb, 2001), orientations (Rhodes et al., 2003; Watson & Clifford, 2003), and retinal location (Leopold et al., 2001), but the aftereffects for low-level image features are less likely to show such tolerances.
Aftereffects for facial information are thought to reflect that the face space, a higher-level representation of faces, was renormalized due to the prolonged observation of faces (Leopold et al., 2001; Rhodes et al., 2003; Webster et al., 2004). The face space is a multidimensional space, whose axes represent various facial characteristics, and faces can be represented as points in the face space (Valentine, 1991). The origin of the face space represents the prototype face. Before adaptation, an average face is mapped to the origin. The aftereffects for facial information are consistent with the idea that the adaptation to a face shifts the origin of the face space toward the point of the adaptation face in order to renormalize the face space. After the origin of the face space has been shifted, the average face does not imply a prototype face anymore, but is interpreted as a point in the space on the opposite side of the origin relative to the adaptation face. Thus, the average face may be interpreted to have the opposite characteristics to the adaptation face.
In this study, we investigated whether prolonged observation of facial images induces an aftereffect of eye direction. Some researchers have argued that the perception of another individual’s eye direction is mediated by a higher-level mechanism specialized for the analysis of eye direction (Baron-Cohen, 1995b; Downing, Dodds, & Bray, 2004; Emery, 2000; Perrett & Emery, 1994; Ricciardelli, Baylis, & Driver, 2000; Zorzi, Mapelli, Rusconi, & Umiltà, 2003). Such a specialized mechanism would play an important role in communication, since eye direction serves as a clue to infer the other’s attention or intention (Baron-Cohen, 1995b; Baron-Cohen & Cross, 1992; Emery, 2000; Langton, Watt, & Bruce, 2000). If an adaptation occurs for the specialized mechanism, the prolonged observation of another individual’s eyes looking in a certain direction would induce the eye direction aftereffect (EDAE). If the EDAE were induced as a negative aftereffect similar to the FDAE, eyes that are directed toward the viewer would appear directed in the opposite direction to the adaptation stimuli’s eyes.
Sixteen students at the University of Tokyo (9 men and 7 women, ranging from 22 to 36 years of age, mean age 26.6 years) were participants in this experiment. They were not informed of the purpose of the experiment.
The experiment was conducted using a personal computer (PowerMac G4, Apple) and a display (FlexScan L565, Eizo) located in a dimly lit room. Participants were seated about 50 cm from the display and responded by using the computer’s keyboard. The participants were instructed to maintain the viewing distance, although they were free to move their head.
The stimuli were human figures generated by computer graphics software (Poser 4, Curious Labs Inc). There were images of four characters (two women and two men). The head and the torso of the stimuli were frontally oriented toward the viewer. The length from the waist to the top of the head was 9.4° of visual angle. The distance between the pupils was about 0.9° of visual angle. Each human figure was rendered against a uniform gray background.
The test stimuli created from each character presented one of five eye directions: −4°, −2°, 0°, +2°, or +4°. Negative values indicate that the eyes were rotated to the left from the viewer’s direction, and positive values indicate rotations to the right.
The experiment consisted of adaptation and test phases. In an adaptation phase, the adaptation stimuli of one type (direct or averted gaze) were presented at the center of the computer display. The stimuli of the four characters were presented in random order, with each stimulus lasting 1.25 s. Thus, one adaptation phase lasted 5 s (1.25 s × 4). Participants were instructed to observe the eyes of the figures during the adaptation phase. At the end of the adaptation phase, a beep sounded for 370 ms, and then a test phase was started.
At the beginning of each trial in the test phase, the screen was blank for 1 s. Then, one of the test stimuli was randomly chosen and presented at the center of the display for 200 ms. Participants pressed the J-key or L-key on a QWERTY-type keyboard depending on their judgment whether the eyes of the test stimulus were directed to the left or to the right (J-key for the left and L-key for the right). After the participants’ key press, the next trial was started. After five trials were completed, another adaptation phase was started with the presentation of a beep sound. The participants completed 200 trials in a session.
Each participant completed two sessions. In the first session, the participants adapted to the direct gaze stimuli. In the second, half of the participants adapted to the leftward gaze stimuli, and the other participants adapted to the rightward gaze stimuli. It was thought that the adaptation in the first session would not affect the participants’ judgment of eye direction, since the direct gaze would not induce the EDAE. Thus, the second session was started immediately after the first session. The mean interval between the two sessions was 1.3 min.
Figure 1 shows the psychometric functions obtained for the direct gaze condition, the leftward gaze condition, and the rightward gaze condition. Each psychometric function represents the percentages of responses where the participants judged that the eyes of the test stimuli were directed to the right as a function of the actual eye direction of the test stimuli. Each data point indicates the response percentage averaged across the participants. In comparison to the direct gaze condition, which may indicate the baseline response, the participants in the rightward gaze condition judged less frequently that the eyes were directed to the right, resulting in the psychometric function’s rightward shift in Fig. 1. In the leftward gaze condition, the participants judged that the eyes were directed to the right more frequently than in the direct gaze condition, and the psychometric function shifted to the left of the baseline function.
Cumulative normal functions were fitted to the psychometric functions of each participant using the psignifit toolbox version 2.5.41 for Matlab (see http://www.bootstrap-software.com/psignifit/) which implements the maximum-likelihood method described by Wichmann and Hill (2001), and the 50% points of the fitted functions were obtained. The eye directions of the test stimuli corresponding to the 50% points are interpreted to give the impression that the eyes present a direct gaze.
The magnitude of the EDAE is indicated by the 50% point for the averted gaze condition minus the 50% point for the direct gaze condition. The mean magnitude of the EDAE for the rightward-gaze condition yielded a positive value (1.3°), and that for the leftward-gaze condition yielded a negative value (−.9°). These EDAE magnitudes were significantly different from each other, t(14) = 6.36, p < .001. Each EDAE magnitude was significantly different from 0°; t(7) = 6.84, p < .001 for the rightward-gaze condition, and t(7) = 3.02, p < .05 for the leftward-gaze condition. The 50% point for the direct gaze condition (.1°) was not significantly different from 0°, t(15) = .91, p > .05.
Participants may have gazed at the center of the iris of one eye during the experiment, because they were instructed to observe the eyes instead of a fixation point. For the test stimuli, the deviation of the iris from the center of an eye was largest when the eye was rotated by ±4°. This deviation was much smaller than that for the iris of the adaptation stimuli, which was rotated by ±35°. This implies that the first trial of each test phase for the averted gaze conditions always started with the participants shifting their point of gaze in the direction opposite to the eye direction of the adaptation stimuli (i.e., toward the center of the eye). The consistent shift in the point of gaze in the first trial of each test phase may have played a role in producing the EDAE. However, this turned out not to be true, because a positive EDAE magnitude (1.3°) for the rightward-gaze condition and a negative EDAE magnitude (−.7°) for the leftward-gaze condition were obtained even when the data from the first trials were excluded. These EDAE magnitudes were significantly different from 0°; t(7) = 7.4, p < .001 for the rightward-gaze condition, and t(7) = 2.62, p < .05 for the leftward-gaze condition.
The results of this experiment demonstrated that the judgments of the test stimuli’s eye direction were influenced by the eye direction of the adaptation stimuli. The results summarized in Fig. 1 imply that the eyes actually directed toward the viewer appeared to gaze in the opposite direction to the eyes of the adaptation stimuli. In this sense, the EDAE is a negative aftereffect.
One interpretation of the results of Experiment 1 is that the EDAE reflects the adaptation of a higher-level mechanism specialized for the perception of eye direction. Let us assume that the higher-level mechanism represents the horizontal component of an eye direction by a point along a left-right axis whose origin corresponds to the perceived direct gaze. As shown in Fig. 1, the 50% point of the psychometric function, which corresponds to the origin of the axis, was close to 0° before adaptation (the direct gaze condition). After the EDAE had been induced, the 50% point (i.e., the origin) was shifted toward the eye direction of the adaptation stimuli (the averted gaze conditions). This implies that the adapted eye direction and the eye direction of 0° are mapped to the points on opposite sides of the origin of the left–right axis. Thus, the eye direction of 0° is perceived as the opposite direction to the adapted eye direction.
However, an alternative interpretation is also possible. Adaptation to the localized image features of the eyes may have induced classic aftereffects that are not specific to the perception of eye direction, and such aftereffects modulated the appearance of the test stimuli’s eyes. This possibility was tested in Experiment 2.
If the EDAE reflects only the aftereffects of the localized image features of the eyes, the three conditions should produce the same result independently of the facial expressions. One prediction is that all three conditions would produce no EDAE, since the effects of adaptation to the opposing eye directions would cancel each other out. Even if the cancellation was incomplete, the three conditions should produce the EDAE with the same small magnitude because the image features of the eyes gazing in each direction were identical among the conditions.
Past studies suggested that facial expressions can influence the mechanisms processing the eye direction (Hooker et al., 2003; Mathews, Fox, Yiend, & Calder, 2003; Tomonaga, Kumada, & Yoshikawa, 2002), although the influences are not so robust (see, Hietanen & Leppänen, 2003). If the EDAE reflects the adaptation of a higher-level mechanism for eye direction, there is a possibility that the EDAE would be influenced by the facial expressions. If the happy and the surprised faces produce the EDAE with different magnitudes, the effects of adaptation to the opposing eye directions would not be completely cancelled out. The EDAE would be produced according to the eye direction of the more effective facial expression (except in the neutral/neutral condition).
Forty students at the University of Tokyo (23 males and 17 females, between 20 and 41 years of age, mean age = 24.7 years) were participants in this experiment. They did not know the purpose of the experiment.
The apparatus was the same as in Experiment 1.
The adaptation stimuli were identical to the averted gaze stimuli used in Experiment 1 except that they presented neutral, happy or surprised expressions. To produce the happy and the surprised expressions, the eyebrows and the mouth of the stimuli used in Experiment 1 were manipulated (Fig. 2). However, the eyes were untouched. For each facial expression, the eyes were rotated to the left or right by 35°. Combinations of the eye directions and the facial expressions defined the happy/surprised condition, the surprised/happy condition, and the neutral/neutral condition.
The test stimuli were identical to those used in Experiment 1.
There were two sessions for each participant. In one session, the happy/surprised condition was conducted for half of the participants, and the surprised/happy condition was conducted for the other participants. In the other session, the neutral/neutral session was conducted for all participants. The two sessions for each participant were conducted on different days, and the order of the sessions was counterbalanced among the participants.
In each session, the adaptation and the test phases were alternated in a manner similar to that of Experiment 1. In each adaptation phase, eight adaptation stimuli were presented (2 eye directions × 4 characters). Each adaptation stimuli lasted for 1.25 s, and the next one was presented after a blank period of .1 s. Thus, one adaptation phase lasted 10.8 s. To increase the level of adaptation, each adaptation stimuli was presented six times during the first adaptation phase, which lasted 64.8 s. The blank period of .1 s was introduced to eliminate any apparent motion of the eyes between two consecutively presented adaptation stimuli with different eye directions.
The adaptation stimuli used in the happy/surprised and the surprised/happy conditions differed in their mouths and eyebrows, but the image features of the eyes gazing in each direction were identical between the conditions (Fig. 2). Nevertheless, the EDAEs produced by the happy/surprised and the surprised/happy conditions were significantly different (Fig. 3). Therefore, the results cannot be explained by the aftereffects induced by adaptation to the localized image features of the eyes. The results indicate that the EDAE was influenced by the facial expressions derived from the image features of the whole face. This suggests that the EDAE may reflect the adaptation of a relatively higher-level mechanism that can receive the information of the facial expressions.
The EDAE magnitudes obtained in Experiment 2 were much smaller than those obtained in Experiment 1. This is because the EDAE magnitude in Experiment 2 represents the difference between the opposing effects of adaptation to the rightward and leftward eye directions.
The positive EDAE magnitude in the surprised/happy condition and the negative EDAE magnitude in the happy/surprised condition imply that the happy faces had a more powerful effect than the surprised faces in producing the EDAE in both conditions. Given that the EDAE is a negative aftereffect, the positive EDAE magnitude in the surprised/happy condition indicates that the EDAE was induced by the rightward eye direction. In the surprised/happy condition, the happy faces presented the rightward eye direction (Fig. 2). In contrast, the negative EDAE magnitude in the happy/surprised condition indicates that the EDAE was induced by the leftward eye direction. In the happy/surprised condition, the happy faces presented the leftward eye direction (Fig. 2).
The interpretation noted above (i.e., the dominance of the happy expression over the surprised expression) assumes that the happy and the surprised faces induce the EDAE as a negative aftereffect similar to that induced by neutral faces in Experiment 1. If the happy and the surprised faces induced the EDAE as a positive aftereffect, the results of Experiment 2 imply a dominance of the surprised expression over the happy expression. In a preliminary experiment, we confirmed that the happy and the surprised faces induced the EDAE as a negative aftereffect. There was only one trial for each participant (67 volunteers with a mean age of 20.2 years) in the preliminary experiment. Each participant observed one of four types of adaptation stimuli (happy or surprised, and leftward or rightward gaze) for 30 s, and then decided whether the eyes of a test stimulus with an actual eye direction of 0° were directed to the left or right. The percentages of participants who reported that the eye direction of the test stimulus was opposite to that of the adaptation stimulus were significantly higher than 50% (binomial test, p < .05): 67.6% (23 out of 34) for the happy faces and 72.7% (24 out of 33) for the surprised faces.
One hypothetical explanation for the dominance of the happy expression over the surprised expression is that the visual system may have an intrinsic bias to produce a stronger signal of eye direction for the happy faces than for the surprised faces. However, the results do not necessarily support this hypothesis, because the expressiveness of the happy and the surprised expressions were not strictly equated.
Another possible explanation for the dominance of the happy expression over the surprised expression is that the happy faces presented the eye direction more saliently than the surprised faces, due to an illusory effect. Participants in the study by Seyama and Nagayama (2002) reported that the perceived eye size was larger on happy faces than on surprised faces even when the actual eye size was identical in the happy and the surprised faces. In our Experiment 2, this illusory effect may have made the eye direction of the happy faces more salient than the eyes of the surprised faces, and the more salient eyes of the happy faces may have induced the EDAE more effectively than the eyes of the surprised faces.
One possible interpretation for the dominance of the happy expression over the surprised expression is that the participants had paid more attention to the happy faces than to the surprised faces. If the attentional factor had played a role in producing the results of Experiment 2, the results do not necessarily support the idea that the EDAE reflects the adaptation of a relatively higher-level mechanism, because attentional factors are known to influence lower-level mechanisms in several cases (e.g., Hikosaka, Miyauchi, & Shimojo, 1993; Kitazaki & Sato, 2003; Spivey & Spirn, 2000). However, the attentional factor fails to explain the results of Experiment 2. Past studies suggest that observers pay more attention to faces with a negative expression than to faces with a positive expression (Eastwood, Smilek, & Merikle, 2001; Öhman, Lundqvist, & Esteves, 2001; Vuilleumier, 2002). Thus, the participants in the present experiment may have paid more attention to the surprised faces than to the happy faces, and the attentional factor should have produced the dominance of the surprised expression over the happy expression, which did not occur in the actual results.
The results of Experiment 2 suggest the involvement of a relatively higher-level mechanism in the EDAE. One candidate for such a higher-level mechanism is analyzing another individual’s eye direction. However, it is possible that any directional stimulus other than the eyes might have induced the EDAE. To test this possibility, the image of an arrow pointing left or right was used as the adaptation stimulus in Experiment 3.
Fourteen students at the University of Tokyo (11 men and 3 women, ranging from 20 to 36 years of age, with a mean age of 26.9 years) participated in the experiment.
The apparatus was the same as in Experiments 1 and 2.
The method for Experiment 3 was almost identical to Experiment 1 except for the adaptation stimuli. The adaptation stimuli used in this experiment were schematically depicted arrows pointing left or right. The arrow consisted of an arrowhead (a triangle) and a shaft (a rectangle), each of which was filled in black. The horizontal size of the arrow was 1.9° of visual angle (.55° for the arrowhead and 1.35° for the shaft). The vertical sizes of the arrowhead and the shaft were .55° and .2° respectively.
The participants observed the arrow pointing left or right during the adaptation phase. Half of the participants observed the right-pointing arrow, and the others observed the left-pointing arrow. The first adaptation phase lasted for 60 s to increase the level of adaptation, and the second and later adaptation phase lasted for 5 s. The details of the test phase were identical to those for Experiment 1.
The results of Experiment 3 do not support the hypothesis that any directional stimulus can induce the EDAE. The results suggest that the gazing eyes are appropriate stimuli to induce the EDAE but an arrow is not. However, the results do not necessarily imply that the eyes are the unique stimuli that can induce the EDAE. There remains a possibility that directional stimuli other than the eyes can induce the EDAE.
Recent studies on the perception of other individuals’ eye direction have measured participants’ performance using various experimental paradigms. For example, the effect of reflexive orienting of attention triggered by another’s eye direction (Driver et al., 1999; Friesen & Kingstone, 1998; Hietanen, 1999, 2002; Langton & Bruce, 1999), the speed and the accuracy of simple discrimination of eye direction (Jenkins & Langton, 2003; Langton, 2000; Ricciardelli et al., 2000; Seyama & Nagayama, in press), visual search performance for eye direction (von Grünau & Anston, 1995), and visual illusions for perceived eye direction (Ando, 2002; Sinha, 2000; Troje & Siebeck, 1998) have all provided clues to understanding the characteristics of the mechanisms for analyzing the others’ eye direction.
Aftereffects, called psychologists’ microelectrodes (Frisby, 1979), are valuable phenomena for investigating the mechanisms of visual perception. The EDAE can serve as a new tool to study the perception of eye direction. Experiments 2 and 3 in the present study can be viewed as examples of the application of the EDAE. In Experiment 2, the EDAE revealed that facial expression could influence the perceptual processing of eye direction. The results of Experiment 3, together with those of Experiment 1, showed that the eyes are effective inducers for the EDAE but the arrows are not. This may suggest that the directions of the eyes and arrows are processed separately at the stage where the EDAE is induced. Past studies also suggested that the eye direction and the arrow direction are processed separately in the brain (Hooker et al., 2003; Ristic, Friesen, & Kingstone, 2002).
The results of Experiment 2 suggest that the EDAE may at least partly reflect the adaptation of a relatively higher-level mechanism, since the classic visual aftereffects induced by the adaptation to the localized image features of the eyes do not explain the difference in the EDAE magnitude between the happy/surprised and the surprised/happy conditions. One candidate for the higher-level mechanism underlying the EDAE may be one that is specialized for the analysis of the eye direction. For example, the adaptation of the eye direction detector (EDD) proposed by Baron-Cohen (1995a; 1995b) may be able to induce the EDAE. However, the findings in the present study do not necessarily lead to a conclusion that the EDAE reflects the adaptation of a specialized mechanism. Past studies suggest that the mechanisms for the analyses of directional social signals accept several types of stimuli. For example, it was reported that the directions of the head and the torso influenced the reaction time for judgment of eye direction (Langton, 2000; Seyama & Nagayama, in press) and the induction of the reflexive orienting (Hietanen, 1999, 2002). These findings suggest that the directions of the body parts are integrated at a certain processing stage. Perrett and his colleagues found that a group of cells in the superior temporal sulcus of the macaque brain responded to the directions of the eyes, the head, and the torso, and proposed that such cells may serve as the detector for the other individual’s direction of attention (Perrett & Emery, 1994; Perrett, Hietanen, Oram, & Benson, 1992). Thus, although the results of Experiment 3 showed the ineffectiveness of arrows in inducing the EDAE, it may be possible that the mechanism underlying the EDAE receives several types of directional stimuli in addition to the direction of the eyes. Therefore, the range of stimulus types that can induce the EDAE should be investigated in a future study.
This work was supported by MEXT.KAKENHI (14710039). The authors thank two anonymous reviewers for their helpful comments and suggestions.