Attention, Perception, & Psychophysics

, Volume 78, Issue 1, pp 133–142 | Cite as

Spatial action-effect binding

Article

Abstract

The temporal interval between an action and its ensuing effect is perceptually compressed. Specifically, the perceived onset of actions is shifted towards their effects in time and, vice versa, the perceived onset of effects is shifted towards their causing actions. In four experiments, we report evidence showing that action-effect binding also occurs in the spatial domain. Participants controlled the location of a visual stimulus by performing stylus movements before they judged either the position of the stylus or the position of the visual stimulus. The results yielded spatial binding between the perceived stylus position and the perceived stimulus position when the stimulus was under full control of the hand movement compared to control conditions without direct control.

Keywords

Embodied perception Perception and action Multisensory processing Intentional binding 

Introduction

The perceived time of a voluntary action shifts towards the time of an ensuing external effect of that action whereas the perceived time of the action effect shifts towards the time of the action (Haggard, Clark, & Kalogeras, 2002). This perceptual attraction between actions and their effects in time has been labeled intentional binding (see Moore & Obhi, 2012 for a review). Here we ask whether an analogous phenomenon exists in the spatial domain – i.e., a subjective compression of space between a moving effector and a visual consequence of that movement.

The possibility of such a spatial action-effect binding is suggested by several observations. For example, wearing prism glasses which displace the optical position of the hand causes the hand to feel like it was located where it looks to be (Hay, Plek, & Ikeda, 1965; cf. also Redding, Rossetti, & Wallace, 2005). A similar proprioceptive drift is apparent in the so-called “rubber hand illusion.” Participants often indicate that their hand is closer to a dummy hand after seeing the dummy hand being stroked by a paintbrush at the same time as they felt stroking of their actual hand (e.g., Botvinick & Cohen, 1998). Comparable effects are observed when the position of the hand is represented by a visual marker (such as a dot) moving in the same plane as the hand: after visuomotor adaptation to misaligned visual feedback, the felt position of the hand drifts towards the seen marker position (e.g., Izawa, Criscimagna-Hemminger, & Shadmehr, 2012; Salomonczyk, Henriques, & Cressman 2012). Similar effects were reported recently even without extensive visuomotor adaptation when hand movements were translated into cursor movements in a different plane (Rand, Wang, Müsseler, & Heuer, 2013). Thus, the proprioceptive perception of the hand seems to be attracted by its visual position even if that position is indicated by an extrinsic object.

According to basic principles of multisensory integration, visual and proprioceptive information are optimally combined in the estimation of hand position (van Beers, Sittig, & Denier van der Gon, 1999; van Beers, Baraduc, & Wolpert, 2002; cf. also Ernst & Bülthoff, 2004). In light of these principles, however, not only proprioceptive perception of the hand should be affected by the vision of an object that visually represents hand position, but also the visual perception of that object could simultaneously be biased by proprioception. In other words, it is reasonable to assume that not only the effector’s perception is affected by its visual effect but also the perception of the effect is affected by its actual cause.

Such mutual biases were in fact recently observed in a study on tool use (Rand & Heuer, 2013). Participants performed arm movements on a digitizing tablet (horizontal plane) while visual feedback was displayed on a monitor (vertical plane). The felt hand direction shifted towards the seen cursor direction (see also Rand et al., 2013) and, vice versa, the judged cursor direction shifted (albeit much less) toward the felt hand direction after the visual movement direction was misaligned in respect to the actual movement direction. These findings might be taken as first evidence for spatial action-effect binding. In order to substantiate this preliminary interpretation, however, at least one additional precondition needs to be met: A mutual spatial attraction of a visual object and the hand should crucially depend on a causal link between hand movement and object movement. Specifically, such attraction should be reduced or vanished when the visual object does not represent an effect of the movement (i.e., when it does not serve as a cursor). This critical condition, however, has not to our knowledge been tested so far. Thus, to substantiate evidence for the spatial attraction between an effector producing an action and visual consequences of that action, one has to show that multisensory integration is reduced or disappears when the visual object does not relate to the effector movement.

Four experiments were conducted to meet this criterion. We implemented varying relations of effector movements to a visual target in conditions where effector movements and the visual target both occurred on a horizontal plane (note that this approach is even more conservative than the abovementioned studies of Rand & Heuer, 2013, and Rand et al., 2013). In Experiment 1, participants moved their right hand horizontally while holding a stylus. These stylus movements made a visual cursor move on the screen (called target1 hereafter). The target moved under full control of the hand but, crucially, it could be displaced by a certain distance relative to the hand. After a certain stylus position was reached participants estimated either the perceived position of the stylus or of the target in separate blocks of trials. In Experiment 2, judgments of either the stylus or the target were requested in a trial-by-trial rather than a blocked manner, to ensure that the main results of Experiment 1 generalize to the conditions used in previous studies (Rand & Heuer, 2013; Rand et al., 2013). In Experiment 3, hand and target had the same displacements during estimation as in Experiment 1. However, we used a stationary target, i.e., its position was no longer controlled by the hand. If spatial binding between the real and the virtual hand position exists then the estimates of the stylus position should be biased in the direction of the target position in Experiment 1 as compared to Experiment 3. Simultaneously, the estimates of the target position should be attracted by the current stylus position in Experiment 1 as compared to Experiment 3. Results were in line with these predictions. Finally, Experiment 4 provided additional baseline conditions to further validate the observed effects.

Experiment 1

With Experiment 1 we aimed to show mutual attraction effects between the perceived position of the acting hand that was operating a stylus, and the perceived position of a continuously controlled target stimulus. To this end, participants first moved the target stimulus with their stylus while target and stylus positions were spatially displaced in one of five mapping conditions (target far/close to the left of the stylus, target and stylus at the same position, target close/far to the right of the stylus). After each movement, participants were either asked to report the perceived stylus position or the perceived target position in different blocks of trials. We expected the perceived stylus positions to be shifted towards the target position, and, vice versa, we expected the perceived target position to be shifted towards the stylus.

Methods

Participants

We recruited a group of 24 right-handed participants (Mage = 29 years, SD = 11; 17 female, seven male). The sample size was determined based on prior research and ensured a power of 0.80 for effect sizes of d = 0.60 (as estimated from Haggard et al., 2002, Fig. 1). Participants gave their informed consent for the procedures and received monetary compensation for their participation.
Fig. 1

Main trial events in Experiment 1. Participants operated a stylus on a graphics tablet and their stylus movements controlled a cursor (the target) on the screen. The main experimental manipulation concerned the displacement of the target relative to the stylus position and participants had to judge either of the two positions at the end of each trial. Note, the hand holding the stylus was not visible to the participants. The lower right corner shows a schematic illustration of the main apparatus of the present study

Apparatus

The main apparatus consisted of a graphics tablet (Intuos 4 XL, Wacom, Kazo, Saitama, Japan), a digitizing stylus, a monitor, and a semi-silvered mirror (see Fig. 1). The monitor was mounted above a table at a distance of approximately 47 cm. The tablet was mounted on the table; it was covered by a semi-silvered mirror that was positioned in-between the monitor and the tablet. This apparatus ensured that stimuli presented on the monitor appeared to the participant to be on the plane of the tablet. The mirror prevented the vision of the participant’s hand in the dimmed lab. One pixel (px) of the monitor was approximately 0.382 mm in size.

The monitor and the mirror were supported by a metal frame. The participants were asked to lean their forehead on the upper part of this frame to reduce a possible impact of head movements (see Fig. 1). Stylus movements were performed with the right hand. Perceptual adjustments, in contrast, were made with the left hand using a keyboard placed to the left of the main apparatus.

Procedure and design

Participants were asked to judge the position of a visual stimulus (target) in one type of block and the position of the stylus in another type of block. For these judgments, a gray vertical line (about 15 mm in length and 0.5 mm in width) was presented about 141 mm above either the current target or the current stylus position. The initial horizontal position randomly deviated from the current position of the stylus or of the target by 9.5 mm to the left or to the right. The task was to align the line with the position of the stylus or of the target by pressing the left and right Arrow keys on the keyboard. The estimate had to be confirmed by the Enter key. When the Enter key was pressed without changing the initial line position an error feedback was presented and the trial was repeated. The respective judgment instruction was presented before each block. We also presented a judgment cue in each trial to encourage participants to keep the instruction in mind throughout a block of trials (German words for cursor or stylus presented in white on a gray background).

The target stimulus was fully controlled by the movement of the stylus. That is, the stimulus served as a cursor of the stylus. After participants were informed about which estimate was to be made and pressed the Space bar on the keyboard, a gray horizontal line (about 382 mm in length and 0.5 mm in width) and the target (green dot, about 2 mm in size) appeared (see Fig. 1). The task here was to move the stylus along the line until the target changed its color from green to red and then to press a stylus button. When the target did not touch the line during the button-press an error feedback appeared and the trial was repeated. The color change could occur at five different positions: when the target was either in the middle of the display, or about 17 or 34 mm left or right of the middle (the allowed deviation was about 2 mm). A comparison line appeared after the stylus button was pressed (see above) and the participant judged the position of the target or of the stylus. In response to the first press of one of the Arrow keys the stimulus changed its color back from red to green and it was slightly adjusted at the middle of the horizontal line when necessary to ensure identical visual conditions during the judgment procedure across the different conditions. Participants had to maintain the current hand position during the judgment. Otherwise they received error feedback and the trial was repeated.

The critical experimental manipulation was related to the spatial mapping between the horizontal position of the stylus and the horizontal position of the target (target-stylus mapping hereafter). The target could be at the position of the stylus or could deviate from it by 38 or 76 mm to the left or to the right. This manipulation was implemented in a trial-wise manner.

The experiment was divided into four blocks with 50 trials each. Two blocks included target judgments, two other blocks included stylus judgments. The succession of blocks was counterbalanced across the participants. In each block, each combination of the position of the target (five levels) and of the mapping between the stylus and the target (five levels) was presented twice in a randomized order. At the beginning of the session, participants performed 20 practice trials (ten for each judgment condition), which were not included in the analyses.

Data preprocessing

A difference score was computed between the actual and the estimated position for each judgment mode and each trial (constant error hereafter). Then, median constant error values were calculated for each participant and each experimental condition. One participant seemed either to have misunderstood the task instructions or to have a not reported visual impairment. Her estimates of target position deviated considerably from those of other participants as well as from the actual target positions (see Fig. S1). This participant was removed prior to analysis.

Results and discussion

Mean constant error values of target and stylus estimates are shown in Fig. 2. When the stylus was on the right side of the target, participants tended to judge the position of the stylus to be more left and they tended to judge the position of the target to be more right than when the stylus was on the left side of the target. An analysis of variance (ANOVA) including target position and target-stylus mapping as within-participants factors and judgments of target position as a dependent measure revealed significant main effects of mapping, F(4, 88) = 3.48, p = .011, ηp2 = .137, and of target position, F(4, 88) = 73.28, p < .001, ηp2 = .769 (p = . 274 for the interaction). An analogous analysis on the judgments of stylus position as dependent variable revealed a significant main effect of mapping, F(4, 88) = 28.27, p < .001, ηp2 = .562, and a significant interaction between target position and the target-stylus mapping, F(16, 352) = 2.15, p = .006, ηp2 = .089 (p = .927 for the main effect of target position). Moreover, linear contrasts associated with the factor target-stylus mapping were significant in both analyses (F(1, 22) = 9.92, p = .005, ηp2 = .311, F(1, 22) = 31.38, p < .001, ηp2 = .588).
Fig. 2

Constant judgment errors for the target judgments and for the stylus judgments in Experiment 1. Mean values of all experimental conditions are shown in the left and middle panels. Judgment errors as a function of the spatial relation between the target and the stylus only are in the right panel (black stands for target judgments, gray stands for stylus judgments). Negative error values reflect biases to the left of the actual target / stylus position, positive error values reflect rightward biases. Negative/positive values in the target-stylus mapping indicate that the stylus was to the left/right of the target. Error bars indicate between-participant standard errors in the left and middle panels, and 95 % within-participant confidence intervals computed according to Cousineau (2005) in the right panel

These results indicate that the judgment of target position is biased towards the current stylus position, whereas the judgment of stylus position is biased towards the current target position, i.e., spatial action-effect binding. With Experiment 2 we aimed to conceptually replicate this basic finding. In particular, we asked whether a random succession of the different types of judgments would reproduce the results of Experiment 1. This would indicate that possible conclusions can be generalized to the designs used in previous studies (Rand & Heuer, 2013; Rand et al., 2013).

Experiment 2

Experiment 2 aimed at replicating the spatial attraction effects between the perceived position of the operating stylus and the perceived position of a controlled target as observed in Experiment 1. Instead of blocked presentations of target judgments and stylus judgments, however, judgment modes now varied from trial to trial. Despite this procedural change, we still expected perceived stylus positions to be attracted towards the controlled target, and vice versa.

Methods

Participants

A new group of 24 right-handed participants was recruited (Mage = 26 years, SD = 4; 20 female, four male). Participants gave their informed consent for the procedures and received monetary compensation for their participation.

Apparatus, design, and procedure

Apparatus, design, and procedure were as in Experiment 1 with the following modifications. Most importantly, participants were now required to judge both the target position and the stylus position in each block of trials. In each block, each combination of the position of the target, of the mapping between the stylus and the target, and each judgment mode was presented once in a randomized order.

Data preprocessing

One participant seemed to have misunderstood the task instructions (see Fig. S1). This participant was removed prior to analysis.

Results and discussion

The mean judgment errors observed in Experiment 2 are shown in Fig. 3. As in Experiment 1, an impact of the mapping between the stylus and the target is evident for the target as well as for the stylus judgments of Experiment 2. An ANOVA on the judgments of the target position revealed a significant main effect of mapping, F(4, 88) = 6.09, p < .001, ηp2 = .217, and a significant main effect of target position, F(4, 88) = 29.70, p < .001, ηp2 = .574 (p = .513 for the interaction). The corresponding analysis of stylus judgments yielded a significant main effect of mapping only, F(4, 88) = 59.38, p < .001, ηp2 = .730 (p = .314 for the main effect of target position, and p = .299 for the interaction). Linear contrasts associated with the target-stylus mapping were significant in both analyses (F(1, 22) = 18.25, p < .001, ηp2 = .453, F(1, 22) = 67.04, p < .001, ηp2 = .753). Thus, the main finding of Experiment 1 was replicated in Experiment 2.
Fig. 3

Constant judgment errors for the target judgments and for the stylus judgments in Experiment 2. Mean values of all experimental conditions are shown in the left and middle panels. Judgment errors as a function of the spatial relation between the target and the stylus only are in the right panel (black stands for target judgments, gray stands for stylus judgments). Negative error values reflect biases to the left of the actual target/stylus position, positive error values reflect rightward biases. Negative/positive values in the target-stylus mapping indicate that the stylus was to the left/right of the target. Error bars indicate between-participant standard errors in the left and middle panels, and 95 % within-participant confidence intervals computed according to Cousineau (2005) in the right panel

Experiments 3 and 4 aimed to test for boundary conditions of the observed effects in order to draw a full analogy to previous studies on intentional binding (e.g., Haggard et al., 2002). In particular, we asked whether the mutual biases between the stylus position and the target position would vanish when the target does not represent stylus movements. In these experiments stylus and target judgments were made in separate blocks of trials as in Experiment 1.

Experiment 3

As a first control condition with a less direct action-effect relation between the stylus movements and the target, the spatial location of the target was no longer coupled to the stylus. Rather, the target remained stationary but changed its color to indicate that the participant had successfully completed the movement. We therefore expected the spatial attraction effect to be reduced and possibly even absent.

Methods

Participants

We recruited a new group of 24 right-handed participants (Mage = 22 years, SD = 8; 21 female, three male). Participants gave their informed consent for the procedures and received monetary compensation or course credit for their participation.

Apparatus, design, and procedure

Apparatus, design, and procedure were as in Experiment 1 with the following modifications. Most importantly, the visual target stimulus no longer served as a cursor of stylus movements. After the judgment cue disappeared, the target was immediately presented on the horizontal line at one of five horizontal positions (see Exp. 1). As in Experiment 1, participants were asked to move the stylus along the horizontal line until the target changed its color, but the target now remained stationary (it was termed a dot in the instructions). Two small horizontal markers (about 4 mm in length and 1 mm in width) presented at the left and right edges of the display now served as indicators of the y-coordinate of the stylus. That is, any deviations of the stylus position from the position of the horizontal line were indicated by a spatial deviation of the markers from the position of the horizontal line. The target changed its color when a “virtual” cursor (that was not visible) reached the current target position (i.e., when the stylus position corresponded to that of Exp. 1 in the respective condition; see also below). Thus, except for the color change, the movement of the stylus was completely unrelated to the target stimulus in Experiment 3, whereas the spatial relation between the stylus and the target was preserved.

Data preprocessing

Two participants seemed to have misunderstood the task instructions (see Fig. S1). These participants were removed prior to analysis.

Results and discussion

As shown in Fig. 4, the manipulation of the target-stylus mapping also had an impact in Experiment 3, but only on the judgments of stylus position. An ANOVA on the target judgments as dependent variable only revealed a significant main effect of target position, F(4, 84) = 40.15, p < .001, ηp2 = .657 (p = .973 for the main effect of mapping, and p = .476 for the interaction). An ANOVA performed on stylus judgments, in contrast, revealed a significant main effect of mapping, F(4, 84) = 39.47, p < .001, ηp2 = .653, in addition to a significant main effect of target position, F(4, 84) = 2.52, p = .047, ηp2 = .107 (p = .393 for the interaction). Thus, when the target stimulus no longer served as a movement cursor, an impact of the introduced spatial deviation between the stylus and the target on target estimates was no longer observed. Stylus judgments were still affected by the target-stylus mapping. This effect, however, seemed to be less pronounced in Experiment 3 than in Experiments 1 and 2 (see also regression analyses below).
Fig. 4

Constant judgment errors for the target judgments and for the stylus judgments in Experiment 3. Mean values of all experimental conditions are shown in the left and middle panels. Judgment errors as a function of the spatial relation between the target and the stylus only are in the right panel (black stands for target judgments, gray stands for stylus judgments). Negative error values reflect biases to the left of the actual target/stylus position, positive error values reflect rightward biases. Negative/positive values in the target-stylus mapping indicate that the stylus was to the left/right of the target. Error bars indicate between-participant standard errors in the left and middle panels, and 95 % within-participant confidence intervals computed according to Cousineau (2005) in the right panel

As in Experiments 1 and 2, participants tended to judge the target to be more right the more right the target was. The effect of target-stylus mapping on stylus-judgments seemed (as in Exp. 1) slightly to increase the more right the target was. This trend was not significant, however. Instead, the stylus position was generally judged to be more right the more left the target was.

A comparison of the results of Experiments 1 and 3 reveals the expected pattern with reduced attraction effects in Experiment 3 compared to Experiment 1. However, the continued effect of target-stylus mapping on the stylus estimates was not predicted beforehand. Two reasons may underlie this outcome. First, participants might still have experienced some sense of control over the target in Experiment 3 (due to the color change of the target after achievement of a certain stylus position). Second, a type of central tendency effect often observed in perceptual tasks may also explain this result. That is, independent of the relation between the stylus position and the target, participants’ estimates may shift in the direction of the mean of all stylus positions (which can be termed as “adaptation level”; see e.g., Helson, 1964). In a similar vein, the observed variation of target estimates depending on the absolute target position might be related to some categorization or optical factors which are not directly related to the movement of the stylus. Experiment 4, in which hand movements were absent before target judgments and the color change was omitted before stylus judgments, aimed to explore these possibilities.

Experiment 4

The rationale behind Experiment 4 was twofold. First, we aimed to further reduce a possible sense of control over the target that may still have been present in Experiment 3 due to the fact that the target changed its color in response to certain movement characteristics (i.e., when a certain stylus position was achieved). Therefore, Experiment 4 provided an even more controlled baseline condition to Experiment 1. Second, we also aimed to access possible biases in judgments of the target which are unrelated to stylus movements. Hand movements were therefore absent in blocks of target judgments. For stylus judgments, a stationary visual target was still present in Experiment 4 as in Experiment 3. However, an acoustic signal indicated the end of the movement instead of the change in target color used in Experiment 3. Because these procedural changes eliminated any type of action-effect relation between stylus movements and the target, Experiment 4 provides the critical baseline to assess to what extent the previously observed effects do indeed resemble spatial action-effect binding.

Methods

Participants

We recruited a sample of 25 participants, four of whom had also participated in Experiment 3 (Mage = 20 years, SD = 2; 21 female, four male). Participants gave their informed consent for the procedures and received course credit for their participation.

Apparatus, design, and procedure

The apparatus was as in Experiments 1 and 3, except that headphones were used for the presentation of acoustic signals. Furthermore, in Experiment 4, participants were asked to put the stylus aside during the blocks of trials in which target judgments were made. That is, hand movements were absent in these blocks. The target (that was termed a dot in the instructions) and a comparison line appeared immediately after the judgment cue disappeared. Stylus judgments, in contrast, were preceded by stylus movements as in Experiments 1 and 3. The visual target was present but did not change its color, however. Instead, an acoustic signal (a sequence of short beep tones) signaled the reaching of the respective stylus position. Each block of target judgments now simply featured ten repetitions of each target position because there was no stylus movement during these blocks. Due to a slight mistake of the experimenter, the succession of the blocks was not fully counterbalanced across the participants.

Results and discussion

An ANOVA performed on target judgments3 only revealed a significant main effect of target position, F(4, 96) = 12.24, p < .001, ηp2 = .338 (p = .462 for the main effect of mapping, and p = .382 for the interaction). An impact of the target-stylus mapping on stylus judgments, in contrast, was again evident as indicated by the results of the corresponding ANOVA, including stylus judgments as a dependent variable (see Fig. 5). This last analysis revealed a significant main effect of mapping, F(4, 96) = 11.92, p < .001, ηp2 = .332, and a significant interaction between target-stylus mapping and target position, F(16, 384) = 3.00, p < .001, ηp2 = .111 (p = .477 for the main effect of target position). When the stylus was on the right side of the target, participants tended to judge the position of the stylus more left than when the stylus was on the left side of the target. This effect was also evident in Experiments 1 and 3. However, it was less pronounced in Experiment 4 than in Experiments 1 and 3 (see Figs. 2, 4 and 5, and the regression coefficient analyses below). This suggests that at least one procedural effect contributed to the results of all experiments (at least of Exps. 1, 3, and 4). This effect could be related to a tendency towards a mean stylus position (i.e., effect of central tendency, see above). Alternatively, the visual stimulus (i.e., target) might serve as a type of anchor (or reference) during stylus judgments which attracts the estimates to some extent. Moreover, because the effect of stylus-target mapping on stylus judgments was reduced in Experiment 4 as compared with Experiment 3, the sense of control over the target seems to have had an impact on the results of Experiment 3.
Fig. 5

Constant judgment errors for the target judgments and for the stylus judgments in Experiment 4. Mean values of all experimental conditions are shown in the left and middle panels. Judgment errors as a function of the spatial relation between the target and the stylus only are in the right panel (black stands for target judgments, gray stands for stylus judgments). Negative error values reflect biases to the left of the actual target/stylus position, positive error values reflect rightward biases. Negative/positive values in the target-stylus mapping indicate that the stylus was to the left/right of the target. Error bars indicate between-participant standard errors in the left and middle panels, and 95 % within-participant confidence intervals computed according to Cousineau (2005) in the right panel

Similar to the results of Experiment 1, the effect of target-stylus mapping on stylus-judgments tended to increase the more rightward the target was. This indicates a rather unspecific origin of this slight interaction. As in Experiments 1, 2, and 3, participants also judged the target to be more right the more right the target was. This effect can thus be considered as another side effect of the given task setting rather than be directly related to the critical manipulation of the target-stylus mapping.

Between-experiment analyses

In order to quantify critical differences across the experiments, we performed a regression coefficient analysis to compare the slope coefficients of the different experiments when regressing the judgment data on the five target-stylus mappings (for general remarks about regression coefficient analysis, see Lorch & Myers, 1990; Pfister, Schwarz, Carson, & Janczyk, 2013). The spatial relation between the target and the stylus positions (i.e., the target-stylus mapping) was used as a predictor variable and median constant error values as the dependent variable in these analyses. Figure 6 shows mean unstandardized regression coefficients (B) indicating the mean slopes of the individual regression lines for each judgment mode and each experiment. In Experiments 1 and 2, the corresponding slopes were significantly different from zero for judgments of the target, t(22) = 3.15, p = .005, t(22) = 4.27, p < .001, as well as for judgments of the stylus position, t(22) = 5.60, p < .001, t(22) = 8.19, p < .001. In Experiments 3 and 4, the target-stylus mapping significantly predicted the constant error in the case of the stylus judgments, t(21) = 6.63, p < .001, t(24) = 3.80, p = .001, but not in the case of the target judgments, t(21) = .30, p = .769, t(24) = .19, p = .850.
Fig. 6

Mean regression slopes (B) indicating increasing (positive values) and decreasing (negative values) trends in the constant error depending on the target-stylus mapping. Judgments of the target position are plotted in black whereas judgments of the stylus position are plotted in gray. Error bars indicate 95 % confidence intervals for each individual value

To ensure comparability, only experiments in which stylus and target judgments were made in separate block of trials were compared with each other (i.e., Exps. 1, 3, and 4). Mean coefficients associated with the judgments of the stylus position were maximal in Experiment 1 and were lower in Experiment 3, t(43) = 2.21, p = .032, and Experiment 4, t(46) = 3.89, p < .001. Also, the mean regression coefficient in Experiment 4 was lower than that of Experiment 3, t(45) = 2.58, p = .013. In the judgments of the target position, the mean slope was more positive in Experiment 1 than in Experiment 3, t(43) = 1.85, p = .071, and Experiment 4, t(46) = 2.44, p = .018.

These results substantiate the differences across the individual experiments observed in the main analyses of variance. In particular, when a visual stimulus is used as a cursor of stylus movements, the estimates of that stimulus are attracted by the current stylus position and simultaneously, the estimates of the stylus position are attracted by the current stimulus position. These effects are reduced or even disappear when the visual stimulus is unrelated to the stylus movement.

Discussion

In the present study we report an extension of intentional binding to the spatial domain. Participants either exerted control over the movement of a visual stimulus by their own hand movement or were prevented from doing so before they judged either the position of that stimulus or the position of their hand. We observed a mutual bias between the perceived positions of the hand and the visual stimulus when that stimulus was used as a movement cursor, compared to conditions without control over the stimulus location. The present results thus extend the previous observations (Rand & Heuer, 2013), by revealing that visuomotor distortions are accompanied by changes in the perception of a visual stimulus as well as of the effector as long as the latter can be considered as an effect of the former.

Considered from a multisensory perspective, the present finding might reflect the results of sensory integration processes which combine and weight proprioceptive and visual signals depending on their reliability in order to provide a robust percept of the current hand position (see e.g., Ernst & Bülthoff, 2004; van Beers et al., 2002). This would imply that the visual target position was treated, at least in part, as equivalent to the felt hand position. In other words, the results can be considered as indicative for the incorporation of a visual stimulus into a body representation. This conclusion corresponds well with the assumed constraints of multisensory integration: different sensory signals are combined as long as these signals are likely to be produced by the same object or event (Bedford, 1995; Ernst, 2006; see also Bresciani et al., 2005; Gepshtein, Burge, Ernst, & Banks, 2005). For example, an increase in spatial separation between visual and haptic signals decreases their integration (Gepshtein et al. 2005). Against this theoretical background, one might wonder why integration occurred in the mentioned studies of Rand and colleagues (Rand & Heuer, 2013; Rand et al., 2013) in which proprioception and vision obviously indicated different positions of clearly different objects. This can possibly be understood in terms of the changed body representation when using a tool (cf. e.g., Canzonieri et al., 2013; Cardinali et al., 2009; Sposito, Bolognini, Vallar, & Maravita, 2012). That is, mutual biases between vision and proprioception may inform here that the internal representation of the body was changed by the acquisition of the kinematic transformation of the tool. As a result, the hand and the cursor were treated, at least to some degree, as one and the same object.4

It has been suggested that the temporal binding is due to a cognitive module linking voluntary actions to their effects (e.g., Haggard et al., 2002; Walsh & Haggard, 2013). Specifically, the perceptual attraction (at least of the sensory effect toward the action) is assumed to result from matching between the predicted and observed action consequences (Waszak, Cardoso-Leite, & Hughes, 2012). In the essence, a sensory action effect is assumed to be perceived earlier (than the same sensory event being not a consequence of own action) due to a preactivation of the representation associated with that effect during action selection (see also Pfister, Obhi, Rieger, & Wenke, 2014). A related mechanism may also be involved in spatial binding reported here. As demonstrated by Buehner and Humphreys (2010), however, similar phenomena can also be observed without intentional actions when a causal connection between events exists. Thus, the critical mechanism involved in action-effect binding might be more abstract and primarily be related to causality (see also Eagleman & Holcombe, 2002).

Explanations based on multisensory mechanisms and on processes akin to temporal binding might be not mutually exclusive. For example, following the ideomotor approach actions are initiated by the anticipation of sensory effects associated with those actions (e.g., Hommel, Müsseler, Aschersleben, & Prinz, 2001; Kunde, 2001). In the given task setting, such a preactivation of body-related action consequences and those in the environment could be a basic prerequisite for multisensory integration to occur. If, for example, an external object does not represent a predicted effect (as in the control experiments of the present study) then there would be no reason to integrate its visual position with proprioceptive information derived from the acting effector. In other words, spatial binding could basically result from concurrently anticipated body-related and environment-related effects of an action which then mutually attract each other. The relation between these types for action effects is currently not completely understood and seems to depend on task conditions (e.g., Sutter, Sülzenbrück, Rieger, & Müsseler, 2013). Accordingly, the suggested explanations are tentative and more research is needed before definite conclusions about the involved mechanisms will be possible.

In summary, the results of the present study indicate that the proprioceptive perception of an effector producing an action and the perception of visual consequences of that action are spatially attracted by each other. This appears to be a particular instance of sensory coupling following the rules that also apply to action-unrelated sensory signals. This finding may have important implications for the research of perceptual-motor interactions and may enable deeper insights into the origins of related phenomena. In particular, the present results indicate that the observed sensory coupling is specific to action-effect relations. Thus, in task situations where body-related signals are accompanied by visual stimulation being a likely consequence of those signals sensory coupling (or spatial binding) should be observed. Research on the rubber hand illusion, for example, showed that the perception of the felt hand position is biased by the position of a dummy hand. However, whether the visual perception of the dummy hand is affected by the position of the real hand has not been examined so far to our knowledge. The results of the present study suggest this possibility and point to a potential explanation of such an outcome.

Footnotes

  1. 1.

    Note, the term “target” is used here to designate a visual object independently of the relation between this object and a hand movement. Thus, it does not imply other meanings generally used in the context of hand movements (such as of objects to be reached by a movement).

  2. 2.

    Note, this rounded value was used in Experiments 1, 3, and 4 to convert pixels into mm. The more exact size of one pixel was about 0.3853 mm. Accordingly, the reported mm values could slightly underestimate the real measures.

  3. 3.

    Note that in Exp. 4 there were no stylus movements before the judgment of the target. The used software, however, randomly classified trials according to the target-stylus conditions used previously. Thus, any residual effects of target-stylus mapping on target estimates should be due to random noise.

  4. 4.

    The fact that participants did not completely fuse visual and proprioceptive signals into a unified percept in those studies does not speak against this conclusion. A rather low strength of coupling might enable maintaining access to the individual signals and thus enable learning new mappings (Ernst, 2006; Hillis, Ernst, Banks, & Landy, 2002). The mere presence of integration (albeit low), however, indicates that different signals are spatially related to each other (otherwise they should be kept separate, cf. Ernst, 2006). Alternatively, and as noted by Rand and Heuer (2013), however, multisensory integration could merely improve the precision of estimates which refer to different objects.

Notes

Acknowledgments

This research was supported by grant KI 1620/1-2 awarded to W. Kirsch by the German Research Council (DFG).

Supplementary material

13414_2015_997_MOESM1_ESM.docx (288 kb)
Figure S1(DOCX 287 kb)

References

  1. Bedford, F. L. (1995). Constrains on perceptual learning: objects and dimensions. Cognition, 54, 253–297.PubMedCrossRefGoogle Scholar
  2. Botvinick, M., & Cohen, J. (1998). Rubber hands feel touches that eyes see. Nature, 391, 756.PubMedCrossRefGoogle Scholar
  3. Bresciani, J., Ernst, M. O., Drewing, K., Bouyer, G., Maury, V., & Kheddar, A. (2005). Feeling what you hear: auditory signals can modulate tactile tap perception. Experimental Brain Research, 162, 172–180.PubMedCrossRefGoogle Scholar
  4. Buehner, M. J., & Humphreys, G. R. (2010). Causal contraction: spatial binding in the perception of collision events. Psychological Science, 21(1), 44–48.PubMedCrossRefGoogle Scholar
  5. Canzonieri, E., Ubalid, S., Rastelli, V., Finisguerra, A., Bassolino, M., & Serino, A. (2013). Tool-use reshapes the boundaries of body and peripersonal space representations. Experimental Brain Research, 228, 25–42.CrossRefGoogle Scholar
  6. Cardinali, L., Frassinetti, F., Brozzoli, C., Urquizar, C., Roy, A. C., & Farnè, A. (2009). Tool-use induces morphological updating of the body schema. Current Biology, 19, R478–R479.PubMedCrossRefGoogle Scholar
  7. Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson's method. Tutorials in Quantitative Methods for Psychology, 1, 75–78.Google Scholar
  8. Eagleman, D. M., & Holcombe, A. O. (2002). Causality and the perception of time. Trends in Cognitive Sciences, 6(8), 323–325.PubMedCrossRefGoogle Scholar
  9. Ernst, M. O. (2006). A Bayesian view on multimodal cue integration. In G. Knoblich, I. M. Thornton, M. Grosjean, & M. Shiffrar (Eds.), Human body perception from the inside out (pp. 105–131). New York: Oxford University Press.Google Scholar
  10. Ernst, M. O., & Bülthoff, H. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Science, 8(4), 162–169.CrossRefGoogle Scholar
  11. Gepshtein, S., Burge, J., Ernst, M. O., & Banks, M. S. (2005). The combination of vision and touch depends on spatial proximity. Journal of Vision, 5, 1013–1023.PubMedPubMedCentralCrossRefGoogle Scholar
  12. Haggard, P., Clark, S., & Kalogeras, J. (2002). Voluntary action and conscious awareness. Nature Neuroscience, 5, 382–385.PubMedCrossRefGoogle Scholar
  13. Hay, J. C., Plek, H. L., & Ikeda, K. (1965). Visual capture produced by prism spectacles. Psychonomic Science, 2, 215–216.CrossRefGoogle Scholar
  14. Helson, H. (1964). Adaptation-level theory. New York, NY: Harper & Row.Google Scholar
  15. Hillis, J. M., Ernst, M. O., Banks, M. S., & Landy, M. S. (2002). Combining sensory information: Mandatory fusion within, but not between senses. Science, 298, 1627–1630.PubMedCrossRefGoogle Scholar
  16. Hommel, B., Müsseler, J., Aschersleben, G., & Prinz, W. (2001). The theory of event coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences, 24, 849–937.PubMedCrossRefGoogle Scholar
  17. Izawa, J., Criscimagna-Hemminger, S. E., & Shadmehr, R. (2012). Cerebellar contributions to reach adaptation and learning sensory consequences of action. The Journal of Neuroscience, 32(12), 4230–4239.PubMedPubMedCentralCrossRefGoogle Scholar
  18. Kunde, W. (2001). Response-effect compatibility in manual choice reaction tasks. Journal of Experimental Psychology: Human Perception and Performance, 27, 387–394.PubMedGoogle Scholar
  19. Lorch, R. F., & Myers, J. L. (1990). Regression analyses of repeated measures data in cognitive research. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(1), 149–157.PubMedGoogle Scholar
  20. Moore, J. W., & Obhi, S. S. (2012). Intentional binding and the sense of agency: A review. Consciousness and Cognition, 21, 546–561.PubMedCrossRefGoogle Scholar
  21. Pfister, R., Obhi, S., Rieger, M., & Wenke, D. (2014). Action and perception in social contexts: Intentional binding for social action effects. Frontiers in Human Neuroscience, 8, 667.PubMedPubMedCentralCrossRefGoogle Scholar
  22. Pfister, R., Schwarz, K. A., Carson, R., & Janczyk, M. (2013). Easy methods for extracting individual regression slopes: Comparing SPSS, R, and Excel. Tutorials in Quantitative Methods for Psychology, 9(2), 72–78.Google Scholar
  23. Rand, M. K., & Heuer, H. (2013). Implicit and explicit representations of hand position in tool use. PLoS ONE, 8(7), e68471. doi:10.1371/journal.pone.0068471 PubMedPubMedCentralCrossRefGoogle Scholar
  24. Rand, M. K., Wang, L., Müsseler, J., & Heuer, H. (2013). Vision and proprioception in action monitoring by young and older adults. Neurobiology of Aging, 34, 1864–1872.PubMedCrossRefGoogle Scholar
  25. Redding, G. M., Rossetti, Y., & Wallace, B. (2005). Applications of prism adaptation: A tutorial in theory and method. Neuroscience and Biobehavioral Reviews, 29, 431–444.PubMedCrossRefGoogle Scholar
  26. Salomonczyk, D., Henriques, D. Y. P., & Cressman, E. K. (2012). Proprioceptive recalibration in the right and left hands following abrupt visuomotor adaptation. Experimental Brain Research, 217, 187–196.PubMedCrossRefGoogle Scholar
  27. Sposito, A., Bolognini, N., Vallar, G., & Maravita, A. (2012). Extension of perceived arm length following tool-use: Clues to plasticity of body metrics. Neuropsychologia, 50, 2187–2194.PubMedCrossRefGoogle Scholar
  28. Sutter, C., Sülzenbrück, S., Rieger, M., & Müsseler, J. (2013). Limitations of distal effect anticipation when using tools. New Ideas in Psychology, 31, 247–257.CrossRefGoogle Scholar
  29. Van Beers, R. J., Baraduc, P., & Wolpert, D. M. (2002). Role of uncertainty in sensorimotor control. Philosophical Transactions of the Royal Society of London B, 357, 1137–1145.CrossRefGoogle Scholar
  30. Van Beers, R. J., Sittig, A. C., & Denier van der Gon, J. J. (1999). Integration of proprioceptive and visual position-information: an experimentally supported model. Journal of Neurophysiology, 81, 1355–1364.PubMedGoogle Scholar
  31. Walsh, E., & Haggard, P. (2013). Action, prediction, and temporal awareness. Acta Psychologica, 142, 220–229.PubMedCrossRefGoogle Scholar
  32. Waszak, F., Cardoso-Leite, P., & Hughes, G. (2012). Action effect anticipation: Neurophysiological basis and functional consequences. Neuroscience and Biobehavioral Reviews, 36(2), 943–959.PubMedCrossRefGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2015

Authors and Affiliations

  • Wladimir Kirsch
    • 1
  • Roland Pfister
    • 1
  • Wilfried Kunde
    • 1
  1. 1.Department of PsychologyUniversity of WürzburgWürzburgGermany

Personalised recommendations