The role of perceptual load in action affordance by ignored objects

Abstract

The extent to which an irrelevant distractor is processed during selective attention depends critically on the level of perceptual load on the relevant task. Here we show that perceptual load also affects the tendency of graspable objects to afford associated actions. Participants carried out a letter-search task and identified a target letter with the right or left hand while ignoring a graspable object with a handle oriented on the left or the right side of the object. The target letter was presented either on its own (low perceptual load) or alongside five nontarget letters (high load). Responses were faster when the action afforded by the ignored object was compatible (vs. incompatible) with the current target response, but only when the perceptual load of the letter search task was low. This finding is the first to demonstrate the role of perceptual load in action affordances by ignored objects.

Much evidence has indicated that the extent to which irrelevant information is processed depends on the perceptual processing demands of the task. Perceptual load theory (Lavie, 1995; Lavie & Tsal, 1994) suggests that perception proceeds automatically until a capacity limit is reached. This means that distracting information is only processed when the perceptual load of the relevant task does not exceed the capacity limit. Evidence from behavioral and neuroimaging studies supports this claim (see Lavie, 2005, for a review), and increasing the level of perceptual load (e.g., by presenting a target among nontarget letters, rather than on its own) significantly reduces response competition from an irrelevant flanker.

The effect of perceptual load has been reported with a wide range of distractor types (see Lavie, 2005, for a review; but see Lavie, Ro, & Russell, 2003, for evidence that human faces produce the same interference effects regardless of the level of load, although this may not be the case for unfamiliar faces: He & Chen, 2010). Many studies have used distractors whose identity is associated with a response in the task at hand (such as letters from the target set as distractors in a letter-search task; e.g., Beck & Lavie, 2005; Lavie, 1995). As a distractor occurs on every trial in these studies, its presence is perfectly predictable, yet observers are unable to prevent processing it under low perceptual load, presumably because of the relevance of the distractor to the task. Perceptual load effects have also been shown with entirely irrelevant distractor objects that are not associated with any task response (e.g., Cartwright-Finch & Lavie, 2007; Forster & Lavie, 2008). In these cases, however, the distractor objects only occur on a minority of the trials, and their unpredictable presence presumably makes them harder to ignore, at least under low perceptual load. Other evidence has suggested that distractor effects are indeed reduced (yet not entirely eliminated) when distractors are always present during an experiment (Theeuwes, 1992). The first aim of the present study was to investigate whether perceptual load can modulate distractibility effects from distractors that are not directly associated with the relevant task and whose presence is perfectly predictable. The distractors were photographs of common objects that had a graspable component, such as a cup with a handle. Compatibility between the targets and the distractors was manipulated by presenting the object with the graspable handle either on the same side as the hand required to identify the current target (compatible conditions), or on the opposite side (incompatible conditions).

Previous work has shown that perceiving an object indeed initiates the actions it affords (e.g., Ellis & Tucker, 2000; Pavese & Buxbaum, 2002; Tucker & Ellis, 1998, 2001). When observers respond with either the right or the left hand to a target that has a graspable handle on either the right or the left side, responses are faster and more accurate when the handle and the response hand are on the same side, as compared to when they are on different sides (Tucker & Ellis, 1998). For example, a cup can be presented either upright or inverted, and with the graspable handle pointing either to the left or to the right. When participants are asked to make a right- or left-hand response to indicate whether the object is upright or inverted, responses are either facilitated or impaired depending on whether or not the response hand is spatially compatible with the orientation of the handle. These affordance effects suggest that the object-appropriate action was activated and occurred even when the orientation of the handle was irrelevant to the task and the stimuli were photographs that could not practically be acted on, and that activating the object-appropriate action leads to a performance cost on half of the trials, when the affordance is incompatible with the current response. Further, recent electrophysiological evidence has suggested that compatibility between the affordance of an object and the required response affects brain activity early in sensory pathways (Goslin, Dixon, Fischer, Cangelosi, & Ellis, 2012), suggesting that affordance-related information becomes available early during visual processing.

Given the apparent robustness of the ability of attended objects to potentiate actions, the question arises whether an object that remains outside the focus of attention can also produce compatibility effects. Previous evidence suggested that it can, but that the direction of the compatibility effect may be reversed for ignored objects. Ellis, Tucker, Symes, and Vainio (2007) presented two abstract objects, each either compatible or incompatible with a power or precision grip, one of which was a relevant target, and the other a to-be-ignored distractor. When the response grip was compatible with the size of the target, responses were faster. However, responses were impaired when the grip was compatible with the distractor, which may suggest that the action properties of the distractor were inhibited in order to respond accurately to the target. Pavese and Buxbaum (2002) also showed that an ignored object can afford action, but they found reversed compatibility only when the required task response was identical to that afforded by the ignored object (such as a right-hand reaching response toward a cup with a right handle). Conversely, no such reversal was seen when the task response was dissimilar to the action associated with the object (a right-hand buttonpress toward a cup with a right handle). Thus, it seems that the direction of the compatibility effect depends on how close the response is to the appropriate action that the object affords.

The second aim of the present study was to investigate whether perceptual load affects the proneness of an object to activate the associated action that it affords. Although previous studies have investigated the effects of attention in action affordance by unattended objects (e.g., Phillips & Ward, 2002; Symes, Ellis, & Tucker, 2005; Vainio, Ellis, & Tucker, 2007), this study is the first to specifically explore the role of perceptual load in such occurrences. We presented irrelevant objects with a graspable component to either the right or the left alongside a relevant letter identification task requiring either a right- or left-hand response, and with either low or high perceptual load. According to perceptual load theory, processing of the irrelevant distractor object should occur under low perceptual load. Consequently, the associated action it affords on the basis of its graspable component would be activated, and a compatibility effect with the responding hand should be evident. Under high perceptual load, this compatibility effect would not be seen, as all available processing capacity would be allocated to the relevant task. However, a study on extinction patients found a greater ability to detect an object in the contralesional field whenever the handle afforded a left-hand grasp (di Pellegrino, Rafal, & Tipper, 2005). The increase in detection was not seen when the actual handle was replaced with a patch, which suggests that the affordance associated with the cup handle directed attention toward the part of the object that subsequently resolved the attentional bias toward the ipsilesional side when two objects are presented concurrently. Given that the associated action pattern of an object can influence attention, it is possible that the affordance related to the distractor object could override the effect of load, and as a result compatibility effects would be seen regardless of load.

Method

Participants

A group of 15 undergraduate psychology students (four males [two right-handed] and 11 females [ten right-handed]) took part in the experiment in exchange for course credits. Data were included only from participants whose accuracy was within 2 standard deviations (SDs) of the group mean for each perceptual load condition. This led to the exclusion of data from one participant (male, right-handed), whose accuracy under high load was more than 2 SDs below the group mean. The average age of the remaining 14 participants was 22.14 years (SD = 4.42), ranging from 18 to 30 years. All participants had normal or corrected-to-normal vision, and all were naïve as to the purpose of the study.

Materials

The experiment was designed and run using E-Prime software (Schneider, Eschman, & Zuccolotto, 2002). Stimuli were presented on a 20-in. monitor screen at a viewing distance of 60 cm, and they consisted of either a single target letter (low perceptual load) or a horizontal row of six letters (high perceptual load). Each letter subtended a visual angle of 0.86 ° horizontally and 0.95 ° vertically and was presented in black Arial font on a white background. The six letter positions were equally spaced and centered at fixation, and the row subtended 8.06 ° from edge to edge. Each display contained a target letter (X or N) that was equally likely to appear in each of the six letter positions. Under low perceptual load, the target appeared alone and the remaining five positions were empty. Under high perceptual load, the target was presented alongside five nontarget letters (selected from H, K, M, V, W, and Z). Each nontarget identity was equally likely to occur at each position. Each display also contained an additional distractor object. The distractors were color photographs (5.52 ° × 5.52 °) of five common graspable objects (see Fig. 1). Two versions were made of each photograph, one with the graspable handle pointing to the left, and a mirror image with the handle pointing to the right. On half of the trials, the direction of the handle was the same as the hand used for the correct response for that trial (compatible trials), whereas on the other half of the trials, the direction of the handle was opposite to the hand used for the correct response for that trial (incompatible trials).

Fig. 1

(Top) Example stimulus displays (images not to scale). Left: High perceptual load, compatible (for subjects who responded to X with the left hand). Right: Low perceptual load, incompatible. (Bottom) The five distractor objects used in the experiment, with left handles. In half of the trials, the images were mirror reversed to have right handles

Distractor objects were equally likely to appear above or below the row of letters, centered at a distance of 1.05 ° (edge to edge) from the letter display. For both low and high perceptual load, all combinations of target identity (X or N), target position (six positions), distractor identity (coffeepot, mug, jug, kettle, or saucepan), distractor position (top or bottom), and orientation of handle (left or right) were used to create 240 unique trials. These were split into two blocks of 120 trials, resulting in a total of four experimental blocks and 480 trials.

Procedure

Each trial started with a central fixation cross, displayed for 1,000 ms, followed by the stimulus, which was displayed for 200 ms. Participants were instructed to make a speeded keypress to identify the target letter, by using the “A” key with their left index finger for the target letter X and the “4” key on the numerical keyboard with their right index finger for N. This response mapping was counterbalanced between participants. The participants were also told to ignore any other objects appearing on the screen. A new trial commenced after the response or after 2,000 ms if no response was recorded. Feedback in form of a tone was given for inaccurate or missed responses. Participants first completed two practice blocks, one under high and one under low perceptual load, consisting of 24 randomly selected trials from each load condition, followed by the experimental blocks. All of the factors were manipulated within blocks, apart from perceptual load, which was varied between blocks. High- and low-perceptual load blocks alternated, and the order of blocks was counterbalanced between participants. At the end of each block participants were allowed a short break.

Results

Reaction times

Responses exceeding 2,000 ms were excluded from the analysis, as were incorrect responses. The data were analyzed in a 2 × 2 factorial design, with the factors Perceptual Load (high, low) and Compatibility Between Hand of Response and Orientation of Handle (compatible, incompatible). Mean reaction times (RTs) and standard deviations were calculated for each participant as a function of load and compatibility (see Table 1) and entered into a 2 (perceptual load: low, high) × 2 (compatibility: compatible, incompatible) within-subjects analysis of variance (ANOVA), with Participants as the random factor. The ANOVA revealed a significant main effect of load, F(1, 13) = 185.15, MSE = 6,116.37, p < .001, η ²_p = .934. RTs were slower during high perceptual load (M = 814 ms) than during low perceptual load (M = 530 ms), confirming that our manipulation of perceptual load was successful. There was no significant main effect of compatibility (F < 1). Importantly, however, we did find a significant interaction between load and compatibility, F(1, 13) = 6.16, MSE = 240.73, p < .05, η ²_p = .322. There was a reliable compatibility effect under low perceptual load, with faster responses for compatible (M = 524 ms) than for incompatible (M = 537 ms) trials, t(13) = 3.224, p < .01 (Bonferroni corrected), but not under high perceptual load (compatible, M = 818 ms; incompatible, M = 811 ms; t < 1). The magnitude of the compatibility effect under low load was comparable to those in previous reports (e.g., Ellis & Tucker, 2000; Tucker & Ellis, 1998, 2001).

Table 1 Mean correct reaction times (RTs, in milliseconds) and accuracy rates (percentages correct) as a function of perceptual load and compatibility between the orientation of the handle and the hand of response

Responses to high-load trials were slower than those to low-load trials, which could have resulted in a failure to demonstrate a reliable compatibility effect, as the impact of the irrelevant handle could have been abolished by the time that a response was made. This issue has previously been demonstrated in a classic flanker task when the relevant set size was large (Miller, 1991). To rule out this possibility, we used a median split to separate each experimental condition into fast and slow RTs. We then tested the effects of interest in the faster high-load responses and the slower low-load responses. A 2 (perceptual load) × 2 (compatibility) ANOVA revealed no significant main effect of load (F < 1), which confirmed that there was no difference in response latencies between the two conditions (high load, M = 438 ms; low load, M = 427 ms). The main effect of compatibility was marginally significant, F(1, 13) = 4.55, MSE = 448.82, p = .053, η ²_p = .259, with faster RTs in the compatible condition (M = 426 ms) than in the incompatible condition (M = 438 ms). Importantly, we again found a significant interaction between load and compatibility, F(1, 13) = 4.99, MSE = 508.58, p < .05, η ²_p = .277. As in the main analysis, there was a reliable compatibility effect under low load, with faster responses for compatible (M = 414 ms) than for incompatible (M = 440 ms) trials, t(13) = 2.93, p < .025 (Bonferroni corrected), but not under high load (compatible, M = 439 ms; incompatible, M = 437 ms; t < 1). This confirms that the effect of load on compatibility effects was not due to differences in the absolute latencies between load conditions.

As targets were presented at different horizontal positions, there was an additional compatibility effect involving target position and response hand. Although target position was fully counterbalanced with respect to both response hand and the position of the handle, we wanted to establish that there were no systematic differences in the key effects as a function of the compatibility between target position and response hand. To do so, we took trials on which the target appeared in the extreme left or right position and ran a 2 (perceptual load) × 2 (compatibility between response hand and handle orientation) × 2 (compatibility between response hand and target position) ANOVA. There was a significant effect of compatibility between response hand and target position, F(1, 13) = 5.24, MSE = 6,717.95, p < .05, η ²_p = .287: Responses were faster when the target letter was positioned on the same side as the hand of response (M = 484 ms), as compared to the other side (M = 520 ms). However, no other effects involved compatibility between response hand and target position (all ps > .05), confirming that the observed affordance effects were not systematically affected by this factor.

Finally, as most participants were right-handed, it was important to rule out the possibility that they were more perceptually sensitive to distractors with a handle presented to the right. Therefore, we ran an additional ANOVA, this time adding the factor of Handle Orientation (left, right). This ANOVA revealed no main effect of handle orientation, and neither were there any interactions involving handle orientation (all ps > .05), confirming that compatibility effects did not vary as a function of the handle orientation of the distractor object.

Accuracy rates

Mean accuracy rates and standard deviations were calculated for each participant as a function of load and compatibility (see Table 1) and entered into a 2 (perceptual load: low, high) × 2 (compatibility: compatible, incompatible) within-subjects ANOVA, with Participant as the random factor. The ANOVA revealed a significant main effect of load, F(1, 13) = 36.68, MSE = .005, p < .001, η ²_p = .738. Accuracy was lower during high perceptual load (M = 83.1 %) than during low load (M = 94.9 %), again confirming that perceptual load was successfully manipulated. No other significant effects were revealed in the accuracy ANOVA (all ps > .05). The pattern of accuracy rates thus indicated that no speed–accuracy trade-offs were present in the data. As the key effect of perceptual load on affordance compatibility was not significant in the accuracy analysis, we did not run the additional analyses to evaluate the effects of target position and handedness on the effects of interest.

Discussion

Here we have shown that an irrelevant object can potentiate an action, but only when it receives sufficient attention. When the perceptual load of a concurrent letter identification task was low, a significant positive compatibility effect between the affordance of the ignored object and the response to be executed was observed. Under high perceptual load, the difference in performance between conditions when the object was compatible versus incompatible with the response was not reliable. The results clearly support the first hypothesis under test, namely that the level of attention that is attributed to an irrelevant object modulates the extent to which it can afford an action. Previous findings suggested that the action an object affords can strongly influence attention (di Pellegrino et al., 2005). Our results demonstrated that this does not seem to hold when one is engaged in a relevant task of high perceptual demands.

The present study extends previous evidence demonstrating that a relevant object with an intrinsically associated action pattern can affect performance in an ongoing task, even when the graspable component is irrelevant to the task (e.g., Phillips & Ward, 2002; Symes et al., 2005; Tucker & Ellis, 1998; Vainio et al., 2007). The present study further confirmed the assumption that object recognition comprises both visual and motor codes, as these actions associated with the object were shown to influence performance, even though no intention to act was present.

This is a new demonstration of the effect of perceptual load on distractor processing. Previous evidence had shown that (1) interference by distractors is modulated by the level of perceptual load, even when their presence is highly predictable, as long as the distractors are part of the target set (i.e., a possible target letter presented as a distractor), and (2) processing of distractors that are not part of the target set can also depend on the level of perceptual load, as long as their presence is unpredictable. Crucially, we demonstrated that processing of entirely predictable distractors that are nonetheless not part of the target set can still be modulated by perceptual load, if the distractors potentiate an action. This makes the present result a particularly strong demonstration of the effect of perceptual load.

These findings are in line with recent evidence that object recognition can be affected by perceptual load. Lavie, Lin, Zokaei, and Thoma (2009) found that the level of recognition of distractor objects is reduced by high perceptual load, as measured by the extent of priming produced by the distractors. Our findings support this conclusion: Only under low perceptual load did the compatibility between the response hand and the hand that was afforded by an irrelevant distractor affect performance, suggesting that under high perceptual load, recognition of the distractor object was reduced. The present results add to these findings by showing that the extraction of affordances can also be affected by perceptual load. The evidence reported by Goslin et al. (2012) suggests that the processing of affordances occurs at an early stage of visual processing. Our findings show that performance was only affected by the compatibility between the response hand and the hand that was afforded by an irrelevant distractor under low perceptual load, suggesting that perceptual load affects early stages of visual processing. This is in line with previous evidence showing a reduction in activity associated with the processing of motion distractors during a high- (as compared to a low-) perceptual load task (Rees, Frith, & Lavie, 1997).

Our findings help to reconcile apparently contrasting previous findings, namely that ignored objects with a graspable component can produce a compatibility effect either in the same direction as that from attended objects (Pavese & Buxbaum, 2002) or in the opposite direction (Ellis et al., 2007). It seems that a negative compatibility effect is likely when the relevant task response involves actions that show a degree of overlap with the action afforded by the ignored object (e.g., a right-hand grasping action in the presence of a distractor that affords a right-hand grasp, as in the grasping condition in Pavese & Buxbaum, 2002, and similar to Ellis et al., 2007). By contrast, when the relevant task response involves actions that show relatively little overlap with the action afforded by the ignored object (e.g., a right-hand keypress in the presence of a distractor that affords a right-hand grasp, as in the present study, and in the buttonpress condition in Pavese & Buxbaum, 2002), a positive compatibility effect is observed. Together, these findings suggest that the nature of the relevant response is important in the direction of the compatibility effect. It may be that once an object has afforded an action, the necessity to inhibit the associated response is more pressing when the task responses are very similar in nature to the afforded actions (as in Ellis et al., 2007, and Pavese & Buxbaum, 2002). In our study, and in that of Pavese and Buxbaum, the overlap between the action afforded by the ignored object and the responses used in the task was merely in terms of the hand being used, rather than in the precise action involved.

It remains possible that the compatibility effect observed here was not due to actions potentiated by the ignored objects, but instead reflected an attentional shift toward the most visually salient part of the distractor stimulus (Anderson, Yamagishi, & Karavia, 2002). We argue that this is an unlikely explanation for the present data, as in our displays the distractor object was centered relative to the center of the photograph, rather than the center of the main body of the object. As a result, the object protruded to each side of fixation, making it less likely that attention was simply shifted toward the side of the object that contained the most visual information. Other evidence has shown that targets are not detected faster when they occur closer to the graspable component of an object (Vainio et al., 2007), again suggesting that these findings cannot be explained entirely in terms of the visual salience of the graspable components. Instead, we conclude that ignored and irrelevant objects can afford associated actions and that the level of affordance depends critically on the perceptual load on the relevant processing.