Intentional control of visual processing benefits from referential objects

Abstract

Arguments have been made that enhanced visual processing occurs in the area of the palms of the hands due to greater density of bimodal neurons. An alternative is that the hands serve as reference objects relative to which attentional resources are allocated. Two experiments were conducted to determine whether the palms are unique in speeding responses in an Eriksen flanker-type task compared with other parts of the hands and objects used as barriers. In Experiment 1, the hands were crossed and positioned so that the palms faced outward toward letters located in the outer positions. Trial blocks differed in whether the centrally located letter or outer letters were designated as the target for responding. Results yielded reductions in flanker interference much as obtained when the palms face inward. This reduction occurred regardless of whether the center or outer positions of the letters were designated as the target. Experiment 2 replicated these results using as reference objects wooden blocks that mimicked the hands' physical contours, positioned with a curve-edge facing outwards. The results lend support to the referential coding account of the reduction of flanker interference.

Recent research has provided evidence suggesting that stimuli appearing in the proximity of the hands are processed differently than ones that are further away (Brockmole, Davoli, Abrams, & Witt, 2013). One prominent view is that visual stimuli near the hands receive priority for attentional resources, because the hands are crucial for manipulating objects (Reed, Grubb, & Steele, 2006). This attentional prioritization often is postulated to have its basis in bimodal neurons (Reed et al., 2006), for which the density in the palm region is high (Graziano & Gross, 1994). According to the prioritization view, stimuli proximal to the palms should automatically be attended to more to more than remote stimuli and, consequently, receive enhanced processing.

Davoli and Brockmole (2012) tested an implication of the attentional prioritization view using a variation of Eriksen and Eriksen’s (1974) flanker task, for which participants identify a centered target letter flanked by distractor letters. In Davoli and Brockmole’s study, the visual display was flat on a table, and participants responded to the target H or S with a left or right button-press. The distractors were two instances of the same letter (compatible), the alternative possible target letter (incompatible), or a letter unassigned to either response (neutral). The flanker interference effect [lengthening of reaction time (RT) for incompatible trials] was measured under conditions in which the hands were placed between the target and distractors and conditions in which they were not. The interference effect was smaller when the target was separated from the flankers by the hands than when the hands were located below the letters or away from the screen entirely. A condition with artificial barriers separating the target and flankers yielded only a nonsignificant reduction in interference, leading Davoli and Brockmole to conclude that the reduction of interference was likely due to attentional prioritization: “Thus, processing enhancements within hand space perhaps arise from greater bimodal neuronal representation of that space, whereas the diminished processing of objects outside of hand space might be a consequence of that space receiving lesser representation from bimodal neurons” (p. 1389).

However, as noted by Weidler and Abrams (2014), results obtained with the flanker task and related tasks are open to an alternative interpretation: “The changes caused by hand proximity could all be explained by changes in the spatial allocation of attention” (p. 465). Murchison and Proctor (2015) obtained evidence consistent with this possibility using a similar method to that of Davoli and Brockmole (2012), except for the responses being foot-presses dissociated from the hand-placement manipulation. Participants responded to the inner letter in some trial blocks and the outer letters in others, treating the inner letter as the irrelevant flanker. For both conditions, interference was reduced when the hands were placed around the inner stimulus rather than below the screen. This result implies that attention could be directed centrally or peripherally with greater efficiency in the presence of the hands, even though the palms were always facing the inner letter. Rather than bimodal neurons of the palm providing attentional prioritization, the hands seemingly serve as reference objects relative to which regions of the visual scene are prioritized. Similar results were obtained with curved wooden blocks as barriers, rather than the hands, suggesting that the reduction of interference from incompatible distractors is a consequence of the presence of salient reference objects that organize visual space so that visual processing can be allocated effectively.

This referential coding explanation is a variant of the referential coding account for spatial stimulus–response compatibility effects, for which there is considerable evidence (Hommel, 1993, 2011). As applied to compatibility effects, the fundamental idea is that the locations of stimuli are coded with respect to multiple reference frames. As applied to the reduction of flanker interference when the hands or similar objects are placed between the center and outer letters, the explanation is that the letters’ locations are coded relative to the hands/objects, allowing more efficient allocation of visual processing to the target location, whether it is inner or outer.

In Murchison and Proctor’s (2015) study, the hands were placed so that the palms faced inward, toward the inner letter and away from the outer letters. As a consequence, the direction in which the palms faced was confounded with letter position. The hypothesis that the hands act as reference objects, rather than the palm region being unique, predicts that the direction in which the palms face is not crucial to the reduction of interference. Therefore, in the present Experiment 1 participants crossed their hands (as has been done in studies of spatial stimulus–response compatibility; e.g., Wallace, 1971) so that the palms faced outward. Attentional prioritization to the palm side predicts a larger reduction when the outer letters are the target than when the inner letter is. In Experiment 2, the hands as barriers were replaced with wooden blocks for which the curvatures matched those of the outward-facing hands. Similar reduction of flanker interference with the blocks would replicate Murchison and Proctor’s finding, obtained when the blocks’ curvature matched that of inward-facing hands, and provide evidence consistent with the referential coding account.

Experiment 1: hands crossed, palms facing outward

Participants performed the flanker task used by Davoli and Brockmole (2012) and Murchison and Proctor (2015), but with the hands crossed in the hands-above condition such that the palms faced the outer letters. Participants responded to the inner or outer letters as target. Similar reduced flanker interference for these hands-above conditions are predicted by the referential coding account but not the palm-specific, attentional prioritization account.

Method

Participants

Sixty students (28 males; 95 % right-handed) participated for credit toward an Introductory Psychology class requirement. All had normal or corrected-to-normal vision.

Apparatus and stimuli

The experiment was programmed in E-Prime 2.0 software (Psychology Software Tools, Sharpsburg, PA) on a personal computer. Stimuli were the letters H, S, or X presented in three positions of a 16- × 13-in. monitor laid horizontally (placed on a table, 15 cm from the participant and viewed from 40 cm). The letter displayed at the inner position was 2.2° wide × 3.7° high, whereas letters at the outer positions were 5.0° × 9.0° (to compensate for decreased acuity), spaced about 15° to the left and right of the inner letter. Responses were presses with the corresponding foot of a left or right foot pedal (14-cm apart), connected to the computer through a serial-response box.

Design and procedure

The study used a 2 (target location: inner-outer letters) × 2 (hand location: around-below) × 3 (compatibility: incompatible-compatible-neutral) within-subjects design. Inner and outer target conditions were conducted in separate halves of the experiment, with distinct hand-location blocks conducted within each half. Both variables were counterbalanced for order across participants. Each block contained 72 trials (24 each of compatible, neutral, and incompatible), with order randomized, preceded by 15 practice trials. Stimuli remained visible until a response was registered. Feedback (correct or incorrect) was given for 300 ms following each trial, followed by a 200-ms delay before the next trial.

Before beginning, the experimenter provided instructions and demonstrated the appropriate hand placements for the hands-around and hands-below placements (Fig. 1). Participants were told which stimuli were assigned to the respective foot-press responses, and this assignment remained constant throughout the experiment. Participants were told to respond as quickly and accurately as possible. Preceding each block, instructions indicating where the participant was to place his/her hands and the letter-response mapping were displayed on the monitor. The experimenter remained in the room throughout the session, seated in a corner behind the participant, ensuring that the participant’s hand positions were appropriate.

Fig. 1

Participants crossed their hands, with the palms facing outward, and placed them around the centrally located letter (hands around condition) or at a location 18-cm vertically below the target letters (at the bottom of the screen, around the “Dell” icon). Thus, in the hands around condition the palms faced the outer letters and their backs facing the centrally located letter. Participants were also told not to use their hands to physically block the outer letters

Results and discussion

Outliers were determined by calculating the mean and standard deviation (SD) for each hand placement and target location for each participant, and trials with RT longer than three SDs of the mean (approximately 2 %) were excluded. Table 1 includes the mean correct RT and the PE data. A 2 (target location: inner-outer) × 2 (hand placement: hands-around or hands-below) × 3 (flanker compatibility: compatible-neutral-incompatible) within-subjects analysis of variance (ANOVA) was performed separately for RT and PE. As in Murchison and Proctor (2015), the compatibility variable was partitioned into orthogonal components: The interference component compared incompatible trials (which should reflect interference) to the average of the neutral and compatible trials, whereas the facilitation component compared the neutral and compatible trials.

Table 1 Mean correct reaction time (in ms) and percent error (PE) for each experiment as a function of flanker compatibility, target location, and barrier placement (Hands in Experiment 1 and Wooden Blocks in Experiment 2)

Reaction time

There was a main effect of target location: RT was shorter for the inner-target condition (M = 597 ms) than for the outer-target condition (M = 681 ms), F(1,59) = 18.96, p < 0.001, η_p ² = 0.243. There also was flanker-compatibility main effect, F(2,118) = 13.51, p < 0.001, η_p ² = 0.186: This reflects the interference component, F(1,59) = 22.38, p < 0.001, η_p ² = 0.275, with RT longer for incompatible trials (M = 654 ms) than for compatible (M = 632 ms) and neutral (M = 630 ms) trials, for which F < 1.0.

The only interaction involving flanker compatibility was that with hand placement, F(2,118) = 7.03, p = 0.001, η_p ² = 0.101. This interaction was due to a smaller compatibility effect for the hands-around placement (M = 12 ms) than the hands-below placement (M = 33 ms), also reflecting mainly the interference component, F(1,59) = 9.23, p = 0.004, η_p ² = 0.056 [facilitation component: F(1,59) = 3.49, p = 0.067, η_p ² = 0.031]. Of importance, the three-way interaction of those variables with target location was not significant, F < 1.0, indicating that the reduction in flanker interference when the hands were placed facing outward was of similar size regardless of whether the inner or outer location was specified as the target location. This result is as predicted by the referential coding account.

Percentage error

The PE was 1.5 %. The main effect of compatibility was significant, F(2,118) = 3.22, p = 0.044, η_p ² = 0.052, with neither the interference component, F(1,59) = 3.12, p = 0.083, η_p ² = 0.050, nor the facilitation component F(1,59) = 3.33, p = 0.073, η_p ² = 0.053, attaining the 0.05 level. In addition, there were fewer errors for the inner-target condition compared with the outer-target condition, F(1,59) = 8.02, p = 0.006, η_p ² = 0.120. No other effects were significant.

Experiment 2: artificial barriers, facing outward

Murchison and Proctor’s (2015) Experiment 4 showed a reduction in the flanker compatibility effect when curve-shaped wooden blocks, approximating the global contours of the hands, were placed as barriers between the center and outer letters. In that experiment, the blocks were positioned so that the inner surface of the curvature was inward. The present Experiment 2 was analogous to the prior Experiment 4 in placing the blocks as barriers, but with the inner part of the curvature positioned outward rather than inward. The expectation was that this placement would reduce the flanker interference much like the other positioning of the blocks did in the prior study.

Method

Sixty new students (24 males; 97 % right-handed) participated for course credit. All had normal or corrected-to-normal vision. Methodology was the same as in Experiment 1, with wooden barriers placed in analogous positions (Fig. 2) to the hands. The hands were placed on the participants’ knees, palms facing down, throughout the experiment.

Fig. 2

Wooden block placement such that blocks faced outward (wooden blocks were placed such that the curved area of the block was facing outward and the concave surface was toward closest to the position of the centrally located letter). All methodology was held consistent with the exception of using wooden blocks, designed to match the contours of an adult male’s hand, rather than the hands as referential objects

Results and discussion

Outliers were determined applying the same criteria as in Experiment 1, resulting in 2.7 % being excluded. Separate ANOVAs were performed separately for mean-correct RT and PE data (Table 1) with target location, barrier location, and compatibility as factors.

Reaction time

There was a main effect of flanker compatibility, F(2,118) = 14.65, p < 0.001, η_p ² = 0.191. Responses were faster for compatible trials (M = 606 ms) than neutral trials (M = 620 ms) and incompatible trials (M = 627 ms). The interference component was significant, F(1,59) = 19.78, p < 0.001, η_p ² = 0.242, as in Experiment 1, but there also was a facilitation component (neutral minus compatible), F(1,59) = 10.95, p = 0.002, η_p ² = 0.150. The 21-ms difference between the incompatible and compatible trials was comparable to the 22-ms difference found previously.

Two interactions involving compatibility were significant. One was the target location × compatibility interaction, F(2,118) = 3.59, p = 0.031, η_p ² = 0.242. The compatibility effect was larger for the outer target (incompatible – compatible = 25 ms) than for the inner target (incompatible – compatible = 17 ms), with only the interference component being significant, F(1,59) = 5.66, p = 0.020, η_p ² = 0.084 [facilitation component, F(1,59) = 2.00, p = 0.162]. The other interaction was that of barrier condition × compatibility, F(2,118) = 6.50, p = 0.002, η_p ² = 0.095, which was due to a smaller compatibility effect for the barrier-around placement (M = 16 ms) than the barrier-below placement (M = 27 ms), also reflecting primarily the interference component, F(1,59) = 9.38 p = 0.003, η_p ² = 0.131 [facilitation component: F(1,59) = 3.51, p = 0.066, η_p ² = 0.131]. The three-way interaction of target location × barrier condition × compatibility approached significance, F(2,118) = 2.81, p = 0.064, but this was mainly due to the atypically high RT for neutral trials with the inside target and the barrier around it. Thus, the primary effect of the wooden blocks was to reduce the flanker interference for both the inner and outer target conditions.

Percentage error

The PE was 4.4 %. The main effect of compatibility was significant, F(2,118) = 7.45, p < 0.001, η_p ² = 0.112, reflecting the interference component, F(1,59) = 13.08, p < 0.001, η_p ² = 0.181 [facilitation component: F < 1.0. In addition, there were fewer errors for the inner-target conditions compared to the outer-target conditions (1 % to 3 %, respectively), F(1,59) = 20.55, p < 0.001, η_p ² = 0.258.

The barrier condition × compatibility interaction followed the same pattern as the RT data, F(2,118) = 5.88, p = 0.004, η_p ² = 0.091, due to the interference component, F(1,59) = 13.28, p < 0.001, η_p ² = 0.184 [facilitation component: F(1,59) = 1.77, p = 0.189]. Finally, the three-way interaction was significant due to the reduced interference being greater for outside compared with inside targets, F(1,59) = 12.95, p < 0.001, η_p ² = 0.180, reflecting mainly the interference component, F(1,59) = 28.38, p < 0.001, η_p ² = 0.325 [facilitation component: F(1,59) = 0.87, p = 0.354].

Comparison to Murchison and Proctor (2015)

Because this study was similar to that of Murchison and Proctor (2015), but with outward-facing rather than inward-facing hands and wooden blocks, ANOVAs were conducted comparing the RT results of Experiments 1 and 2 to the analogous conditions of Murchison and Proctor’s Experiments 3 and 4, respectively. Because hand placement was a between-subjects factor for that study, it was treated as such in these analyses.

When comparing Experiment 1 to Experiment 3, RT was longer overall for outer targets than inner targets, F(1,59) = 25.20, p < 0.001, η_p ² = 0.176, and there was a compatibility main effect, F(2,118) = 20.05, p < 0.001, η_p ² = 0.148. The hand placement × compatibility interaction revealed the reduction of the compatibility effect when the hands were placed around the inner letter compared to when they were not, F(2,118) = 11.94, p < 0.001, η_p ² = 0.092, which was mostly due to the interference component, F(1,59) = 21.34, p = 0.001, η_p ² = 0.153 (facilitation component: F(1,59) = 3.42, p = 0.067). The experiment × target location × compatibility interaction, which did not involve hand placement, was not significant, F(2,118) = 2.63, p = 0.074. Most important, the experiment × hand placement × compatibility interaction and the four-way interaction of those variables with target location were not significant, Fs < 1.0.

Results comparing Experiment 2 to Experiment 4 revealed longer RT overall for outer than inner targets, F(1,59) = 17.58, p < 0.001, η_p ² = 0.130. RT was longer in the barrier condition compared with the condition in which there was no barrier, F(1,59) = 8.98, p = 0.003, η_p ² = 0.071, and there also was a compatibility main effect, F(2,118) = 24.99, p < 0.001, η_p ² = 0.175. The experiment × target location interaction was significant, F(1,59) = 6.59, p = 0.012, η_p ² = 0.053, as was the experiment × target location × barrier interaction, F(1,59) = 4.05, p = 0.046, η_p ² = 0.033, which reflects that in Murchison and Proctor’s (2015) experiment, there was a target × barrier interaction that was not significant in this experiment. However, neither of these terms involves flanker compatibility. The target location × compatibility interaction was significant, F(2,118) = 12.66, p < 0.001, η_p ² = 0.097. Importantly, the barrier condition × compatibility interaction revealed the reduction of the flanker compatibility effect when the barrier was placed around the inner letter compared to when it was not, F(2,118) = 5.47, p = 0.005, η_p ² = 0.044, which was due to the interference, F(1,59) = 7.22, p = 0.008, η_p ² = 0.058, and some facilitation, F(1,59) = 4.28, p = 0.041, η_p ² = 0.035). The critical experiment × target location × barrier condition × compatibility interaction was not significant, F < 1.0. Thus, the results obtained with the palms and blocks facing outward were similar to those obtained with them facing inward.

General discussion

The outcomes of this study suggest that intentional control of visual processing is facilitated by referential objects. Experiment 1 showed similar reduction in flanker interference for outer and inner distractors with the palms facing outward. This complements the results obtained by Murchison and Proctor (2015) with the palms facing inward. Similarly, Experiment 2 showed similar reduction in flanker interference with outward-facing wooden blocks, again complementing Murchison and Proctor’s results with inward-facing blocks. Hence, results are in agreement with the referential coding account advocated by Murchison and Proctor, according to which, enhancement of visual processing reflects an ability to attend better to the instructed locations, relative to referential objects.

Although selection of the target location is the only process discussed, perception more broadly may be impacted by the presence of nearby objects (e.g., located near a display; Brockmole et al., 2013). Thus, alternative interpretations of attentional and other effects have been proposed. One alternative account is that of modulated visual-pathways, which distinguishes the magnocellular and parvocellular pathways. The account suggests an enhancement of the magnocellular pathway (Taylor, Gozli, Chan, Huffman, & Pratt, 2015) due to a disparity in the spatial frequency sensitivities, which could affect perceptual-grouping processes (Huffman, Gozli, Welsh, & Pratt, 2015). According to Goodhew, Edwards, Ferber, and Pratt (2015), these effects implicate “integrating dynamic information about the location of the hands in space (body-centred) with visual information about the external world” (p. 227).

Notably, the assimilation of the hands into the information about the external world suggests that the hands are referential to that world, and thus this account is not counter to referential coding. That said, the intricacies of the modulated visual-pathways account are more complicated, and this current research does not speak to the full breadth of related studies. More would have to be done to compare this account directly with referential coding. Regardless, whereas several incidents of changes to visual attention due to hand location have been reported (Abrams, Davoli, Du, Knapp, & Paull, 2008; Reed et al., 2006), spatial properties of visual processing being affected by hand proximity is an equally reasonable alternative explanation (Weidler & Abrams, 2014). The latter authors reported that responding with the hands on the sides of a monitor reduced flanker interference, which is not predicted by the referential coding account. However, this result was not evident in RT, F < 1.0, which is typically the primary performance measure, but only PE.

Our study also is not directly relevant to near-tool effects when a tool is being manipulated. According to a review by Brown and Goodale (2013), analogous results for near-hand and near-tool effects have been found with adequate practice or prior experience with the tool. The proposed mechanism is that the recruitment of bimodal neurons is extended to include the tip of the tool for which practice on the motor control has established its functional meaning. The referential coding account also would predict that the end of the tool relevant for the distal goals of the task should serve as a referential object, so this pattern of results is not problematic for the account. It is worthwhile to discern whether the effects with tools are spatially relevant, which according to Umiltà et al. (2008) may be the case. That is, when training has been provided, the brain activation in monkey’s cortical motor neurons follows the tool and not the hand movements. Thus, there appears to be enough evidence within the near-tool effect literature that supports the hypothesis that the tools can serve as a referential object, a hypothesis worth exploring.

This set of experiments is relevant to those studies evaluating effector location and position as a referential object. Cho and Proctor (2002) had participants respond using toggle switches with their hands in a prone or supine hand position located to the left, center, or right of the body midline. In analyzing compatibility effects, the results support the hand-referent hypothesis, which is “Response location is coded with respect to multiple frames of reference and that the body of the hand provides one such frame.… The hand posture effect itself was shown to be due, at least in part, to location coding relative to the body of the hand” (p. 1314). Notably, that the hands provide one of multiple reference frames is in agreement with the account proposed by Hommel (2011), which we are endorsing here. Thus, in the hands-around conditions of our experiments, a reduction in interference arises from the position-fixed stimuli being coded as left/right relative to the hands. Our results extend the implications of the hand-referent hypothesis by dissociating the response from the hand location and position; thus the effect is not tied into the motor component of responding but is one of more distinct spatial coding of the stimuli.

In summary, this study tested implications of two accounts of visual processing, those of attentional prioritization of palm space and referential coding. These studies are specific to situations in which the visual information is segmented by the referential objects: hands or wooden blocks. Prior explanations of compatibility consequently extend findings of hand-unique effects, because hands can provide strong and salient referential objects coded in terms of the task, context, and instructions: These together define the specific distal goals. Thus, in this flanker task, when the hands segment the display, the array is separated into meaningful areas to benefit responding. Accordingly, intentional control of visual processing may be most economically explained by referential codes accessed when instructions are given to prioritize or separate target from non-target space. The hands and artificial barriers are the strongest referential frame, which is the reason they govern the spatial relationships in the task.