1 Sensorimotor adaptation in VR

Through learning processes, movement and perception are associated in such a way that people can precisely manipulate their environment. Disturbances of these learned associations can be caused by changes in our body or in the environment and are called perturbations. Sensorimotor adaptation is the gradual adjustment of our motor commands to compensate for such perturbations (Della-Maggiore et al. 2015). In the past, researchers have investigated sensorimotor adaptation primarily by using lenses, prisms or mirrors that alter the visual field by displacing, inverting, tilting or otherwise distorting it (Welch and Warren 1980).

The first study to systematically investigate sensorimotor adaptation in virtual reality (VR) was published by Biocca and Rolland (1998) and Rolland et al. (1995). The rationale for the research was the practical problem that with video-based head-mounted displays (HMDs) the virtual eye position did not correspond to the actual eye position because the camera could not be placed where the person's eyes were already located. The video-based HMD was compared to a dummy HMD that had the same weight, centre of mass and field of view, but no video transmission, to see if this technical problem influenced task performance. Participants in the video-based HMD condition showed movement errors at the beginning (= direct effect) but adapted to this visual displacement over time. When the participants took off the HMD, they also showed a change in their hand–eye coordination. These aftereffects appeared, for example, in the form of less accurate pointing movements compared to the baseline measurement (Biocca and Rolland 1998).

The study by Biocca and colleagues shows just one of many possibilities in which perturbations can arise in VR. First, reality can be misrepresented due to technical limitations, e.g. calibration errors. In addition, technological developments do not have the goal of VR as a perfect simulation of reality, because this will probably never be achieved. Instead, they have the goal of "perfecting" the user interface (Doerner et al. 2022). Therefore, some erroneous representations of reality are also intentional, such as in redirected walking (Nilsson et al. 2018). Here, the walking movement corresponds to a different walking movement in VR due to spatial restrictions. Another example of visuomotor illusions is reach redirection, which allows targets to be reached that are beyond actual reach while maintaining some semblance of body ownership (Cohn et al. 2020; Gonzalez et al. 2022). Furthermore, discrepancies between perception and movement are intentionally built in as part of the VR experience. These include, for example, body illusions in which body proportions are changed (Kilteni et al. 2012b).

Both body illusions and visuomotor illusions occur when there is still a sense of embodiment despite the discrepancies. A sense of embodiment emerges when the properties of the virtual body are processed as if they were the properties of one’s own biological body. The sense of embodiment consists of three subcomponents: the sense of self-location (= the feeling of being located in a determined volume in space), the sense of agency (= the sense of having global motor control), and the sense of body ownership (= one’s self-attribution of a body, Kilteni et al. 2012a). Factors that enhance these subcomponents, and thus the sense of embodiment, include the first-person perspective, sensory and visuomotor correlations within critical boundaries, and morphological similarity between the real and virtual body, among others (Kilteni et al. 2012a).

Once a sense of embodiment has been achieved, sensory impressions and movements are aligned with the virtual body representation. One example is the self-avatar follower effect: users tend to unconsciously follow their avatar when the virtual body does not overlay with their physical body (Gonzalez-Franco et al. 2020). Burin et al. (2019) found that deviations between intended and actual virtual movement led to stronger adjustment in favour of the virtual movement when the avatar was shown from a first-person perspective instead of a third-person perspective. A stronger sense of embodiment in VR thus seems to lead to stronger movement adjustment.

The sense of embodiment could also be important in the discussion of whether and to what extent sensorimotor adaptation differs between VR and reality. Studies show that visual displacement in VR leads to a similar adaptation (Wilf et al. 2021, 2023; Cho et al. 2022) and is influenced by similar factors (Gammeri et al. 2020) as in prism adaptation. Wähnert and Gerhards (2022), for example, found empirical evidence that the positive relationship between the number of interactions during perturbation and the aftereffect, which is already known from prism experiments (Dewar 1970; Welch 1971; Fernández-Ruiz and Díaz 1999), can be generalised to VR. However, comparative studies between VR and reality have also found differences. In VR, sensorimotor adaptation in a visuomotor task is associated with less implicit learning (Anglin et al. 2017) and more cognitive load (Juliano et al. 2022) than in a real environment. These differences may be due to how the body is perceived in VR and thus how movement errors are attributed.

2 Error attribution

The nervous system interprets sensory feedback with regard to its causes (Wei and Körding 2009). Hereafter, in the context of movement errors, the causal inference of sensory feedback is referred to as error attribution. In Welch's (1972) prism experiment, error attribution was manipulated using instructions. In contrast to the informed group, the misinformed group was given the misinformation that when they performed a pointing movement in the dark, they saw their own luminous finger, which was visually displaced by the prism glasses. In fact, the finger belonged to the experimenter and no actual visual displacement via prism took place. This misinformation was intended to enhance adaptation by suggesting that the displaced luminous dot represented the participant’s hand, and led to an aftereffect about twice as large as the true information given to the informed group that the experimenter moved the luminous dot. Following this study, Welch and Warren (1980) formulated the unity assumption. According to this assumption, an important precondition for the occurrence of a discrepancy between different sensory feedback is that the discrepant information comes from a single object, e.g. one's own hand. If the objects are different, there is no intersensory discrepancy and thus no reason for adaptation (Welch and Warren 1980). Bedford later incorporated this assumption and generalised it within the framework of his theory of object identity (Bedford 2001). Object identity is thus not only required for sensorimotor adaptation, but in general when information from different times, places, modalities, and eyes is to be integrated into a coherent view of the world.

Unity assumption and object identity can be influenced by various factors. Bedford (2001) refers to the distinction made by Radeau and Bertelson (1977), according to which the interpretation of single events is influenced by structural and cognitive factors. Structural factors are abstract properties of the sensory input, such as synchrony. Cognitive factors are “features of the situation which communicate knowledge about the distal situation from which the data originate, and so indicate whether a unitary interpretation is correct or not” (Radeau and Bertelson 1977, p. 137). Such features can consist of verbal instruction or a realistic context. Realistic context refers to “a context more or less evocative of situations known to produce correlated sense data in the two modalities” (Radeau and Bertelson 1977, p. 137). The aim of the present experiment was to investigate whether the two cognitive factors verbal instruction and realistic context influence sensorimotor adaptation in VR, and whether this relationship is mediated by error attribution.

Welch (1972) has already shown that instructions can have an influence on the aftereffect in prism adaptation. In real environments, it is usually assumed from previous experience that the felt hand and the seen hand belong to the same object. In VR, however, the user must be convinced, e.g. through visual-tactile or visual-motor synchrony, that the virtual body parts correspond with the real body parts in order to evoke a sense of embodiment (Slater et al. 2009). Therefore, in VR it should be possible to manipulate the perception of object identity by instructions without limiting visual information to a minimum, as it was necessary in Welch's experiment. Besides that, Welch used a misinformation about a correspondence between a luminous dot and one's own hand to increase adaptation and thus the aftereffect. We wanted to investigate whether misinformation about an erroneous correspondence between the virtual hand and the real hand would reduce the aftereffect. Thus, the following hypotheses were formulated:

H1: In VR, misinformation about an erroneous correspondence between a virtual and a real hand leads to a) a smaller and b) a less persistent aftereffect than no misinformation.

According to Radeau and Bertelson (1977), the second cognitive factor that influences the interpretation of single events is the realistic context. As already described in the last section, body representation seems to play an important role in the alignment with the virtual body. Therefore, we have limited the realistic context to the realistic body representation. Several prism studies have already been able to show that a more abstract representation of the body, e.g. a cursor, leads to a lower aftereffect (Clower and Boussaoud 2000; Norris et al. 2001; Veilleux and Proteau 2015; Aziz et al. 2020). One possible explanation for these results would be that more abstract representations increase the uncertainty that the abstract representation of the hand and the felt real hand belong to the same object, so that a discrepancy between them gives less cause for adaptation (Norris et al. 2001). In VR, body representations can be changed very easily (Kilteni et al. 2012b; Banakou et al. 2013). The present study investigated whether a more abstract body representation in VR affects sensorimotor adaptation. For the control group, the hand model from the experiment by Wähnert and Gerhards (2022) was adopted, with which large and persistent aftereffects were found. The abstract representation was intended to be achieved by displaying a 3D arrow instead of the virtual hand. The following hypothesis was formulated:

H2: In VR, the 3D arrow representation leads to a) a smaller and b) a less persistent aftereffect than the virtual hand representation.

If the perception of object identity is manipulated by instructions and body representation, then this should influence error attribution. More precisely formulated: If the real hand and the virtual representation are regarded as a unit or as one object, then errors related to this virtual representation should be assigned a different relevance. The relevance estimation model (Wei and Körding 2009) states that the nervous system estimates whether discrepancies between vision and proprioception are due to intrinsic factors that lie within the sensorimotor system or to external factors that are unrelated to the sensorimotor system, e.g. calibration errors. If visual cues are inferred to be irrelevant, they will be disregarded for movement production and thus adaptation (Wei and Körding 2009). True adaptation is characterised by generating new mappings of internal representations of one's own body. These are not restricted to the particular context in which perturbations were produced (Fleury et al. 2019). Applied to misinformation and abstract body representation, this would mean that both would lead to a lower aftereffect because the visual perturbation is seen as irrelevant to adaptation, i.e. attributed to a faulty system rather than to the self. The perception of error relevance was assessed by self-report. The following two hypotheses were formulated:

H3: Effect of misinformation on a) magnitude and b) persistence is mediated by error relevance.

H4: Effect of 3D arrow representation on a) magnitude and b) persistence is mediated by error relevance.

3 Methods

3.1 Participants

A total sample of 30 individuals were recruited to participate in this laboratory study. Two participants were excluded due to simulation sickness and task execution problems and were subsequently replaced, resulting in an analysis sample of 30 participants (aged 22–29 years; 12 men, 18 women). We adopted the sample size from the experiment by Wähnert and Gerhards (2022), who also analysed 10 individuals per group and found differences in terms of the aftereffect. Furthermore, according to our G*Power a priori power analysis for one-sided t-tests in a linear model regression, which we used to test the main hypotheses (see data analysis), a total sample size of 30 individuals is sufficient to detect effect sizes of \({f}^{2}=0.31\) or partial \({R}^{2}=0.23\) with \(1- \beta =0.9\). Eligibility criteria included healthy, right-handed individuals with normal or corrected-to-normal vision, native German language proficiency, no previous negative VR experience, a minimum body height of 1.55m, no physical impairment of the right arm, and no neurological disease affecting the upper body. All participants gave their informed consent prior to data collection and received course credits or money as compensation. The present study was approved by the Research Ethics Committee of the Department of Psychology and Ergonomics at the Technische Universität Berlin.

3.2 Apparatus

As shown in Fig. 1a, the participants sat at a table during the experiment with their heads resting on a chin rest. A cardboard marker was placed on the table to indicate the distance from the virtual wall. It served as an orientation to not put the arms too far forward after the pointing movement. The virtual environment (see Fig. 1b) was presented via an HMD with built-in off-ear headphones (Valve Corporation 2023). Acoustic signals were given via the headphones to indicate the start and end of the pointing movement. An HTC Vive tracker was attached to the participant's right hand to track the position of the index finger. A finger orthosis was also attached to stabilise the middle and index fingers. In the virtual environment, the participants sat at a table with a wooden board on which the targets appeared. The setup of the virtual environment was the same as in the experiment by Wähnert and Gerhards (2022).

Fig. 1
figure 1

Methods a Experimental setup in real environment. b Virtual environment. c Virtual hand vs. 3D arrow

Full size image

3.3 Procedure and experimental design

After reading the participant information and agreeing to the informed consent form, participants were asked to complete a demographic questionnaire. Afterwards, the participants were informed about the VR task and procedure and had the opportunity to ask questions. After adjusting the HMD to the participant's pupil distance, attaching and calibrating the Vive tracker, placing the participant's head on the chin rest, and putting on the HMD, the VR experiment began.

Before the pointing task started, participants were instructed to move their right hand along the front edge of the table from the centre to the right corner. The correspondence between visual and haptic feedback was intended to reinforce the perception of object identity between the real hand and its virtual representation.

The pointing task was the same as in Wähnert and Gerhards (2022). The position of the target could deviate both vertically and horizontally by 3 cm from the centre position, resulting in nine possible target positions. The sequence of the target positions was predetermined. The position and sequence of the targets can be found in the description of the data set under the link https://doi.org/10.14279/depositonce-17735. The participants were instructed to carry out the pointing movements as accurate as possible. Movement speed was not instructed but measured and tested for group differences. The experimental procedure was based on the original prism adaptation paradigm (Kornheiser 1976), and the terminology for the different paradigm phases was adopted from Prablanc et al. (2020). The familiarisation phase consisted of 30 trials: 10 with concurrent feedback of the pointing movement and 20 with only terminal feedback of the pointing movement, as in all subsequent phases. After recording the baseline performance in the 10 trials of the baseline phase, the virtual environment was visually displaced to the right by 11.31 degrees (or 20 dioptres) in the exposure phase, which consisted of 35 trials. It is known from prism research that both the size of the deviation and the number of trials in the exposure phase have an influence on the aftereffect (Dewar 1970; Welch 1971; Fernández-Ruiz and Díaz 1999). The parameters set (20 dioptres and 35 trials) are based on the experiment by Wähnert and Gerhards (2022), who were able to demonstrate a strong and persistent aftereffect in a virtual environment. In the de-exposure phase, which consisted of another 30 trials, the visual displacement was removed. Participants were informed about both the visual displacement and its removal by instructions in the virtual environment before the corresponding phase. Performance in the de-exposure phase was used to assess the aftereffect. To measure its magnitude, a target position was presented in the first trial of the de-exposure phase that had not been used in the previous trials (see data analysis). During the entire VR experiment, a ten-second rest period was inserted after a maximum of 15 trials. After the VR experiment, the participants answered questions on error relevance and realism perception. Finally, the participants were debriefed, and remaining questions were answered. The whole experiment was constructed to last 45 to 60 min. The VR part of the experiment lasted no longer than 20 min.

Participants were randomly assigned to one of three groups: the control group, the misinformation group, and the arrow group. The control group was equivalent to the group from the experiment by Wähnert and Gerhards (2022) with thirty-five pointing movements. The misinformation group differed from the control group with respect to the misinformation. Participants in the misinformation group were told that, during the exposure phase, a random error component was built into the movement in addition to the visual displacement of the environment. The difference between the random error component and the visual displacement was made clear in the instructions and when asked by the participants. The arrow group differed from the control group with respect to the representation of the hand. Instead of a virtual hand, a 3D arrow was displayed (see Fig. 1c). The tip of the arrow corresponded to the tip of the middle finger. The size and movement features were identical to those of the virtual hand.

3.4 Data analysis

Statistical analyses were conducted using R (version 4.2.2, R Core Team 2023). The data of the experiment can be found in a data repository under the link https://doi.org/10.14279/depositonce-17735. The hypotheses were tested by analysing the magnitude of the aftereffect, the persistence of the aftereffect and the error relevance. The post hoc power analysis was conducted with \(1-\beta\) = 0.9.

According to Prablanc et al. (2020), there are several ways to measure and calculate the magnitude and persistence of aftereffects. In this paper, we based our measurement methods on Fernández-Ruiz and Díaz (1999) and Wähnert and Gerhards (2022). First, we baseline-corrected the performance in the de-exposure phase for each participant. Magnitude was defined as the horizontal deviation in the first trial of the baseline-corrected de-exposure phase. There were two reasons why only the first trial was selected. First, in the first trial of the de-exposure phase, a target position was displayed that had not been used in the previous trials. The rationale for changing the target position for measuring the magnitude is that if there is a sensorimotor realignment beyond strategic or cognitive effects, this should generalise beyond the task conditions in which the visual displacement was induced (O’Shea et al. 2014). Generalisation to new target positions has already been shown for prisms (Bedford 1989) and virtual visual feedback (Vetter et al. 1999). Second, only the first trial was performed without feedback experience from the de-exposure phase. The remaining trials were used to measure persistence, which is the robustness of the horizontal deviation despite feedback. Persistence was defined as the mean of the absolute horizontal deviation in the remaining 29 trials of the baseline-corrected de-exposure phase. In addition to this approach of predetermining the number of trials and looking at the average performance within these trials, another possibility would be to set a certain performance level and analyse how many trials are required to reach and maintain this level. We decided against this option because we see a methodological problem here: participants who have reached the performance level in a trial do not remain constant at this level, but show fluctuations. It is difficult to determine what proportion of these fluctuations is already present in the baseline phase and can be neglected, and what proportion is due to persistence of the aftereffect. Even if a baseline range can be defined, it is still difficult to determine the persistence, e.g. in the case of single outliers after staying within the baseline range for several trials.

To analyse the group differences under hypotheses H1 and H2, we applied a linear model regression, dummy-coding the three groups with the control group as reference. The hypotheses about the differences of the misinformation and the arrow group compared to the control group were formulated as statistical hypotheses about the regression coefficients (Fox 2016).

Error relevance was measured at the end of the VR experience by asking the participants to estimate the reason for their movement error on two continuous 10 cm scales. One scale assessed the error attribution to oneself, the second scale assessed the error attribution to the system. Error relevance was defined as the difference between the two responses, with larger values indicating stronger self-attribution and thus greater error relevance.

In order to test the two mediator hypotheses H3 and H4, the conditions according to Baron and Kenny (1986) were analysed. Applied to hypotheses H3 (magnitude of aftereffect) and H4 (persistence of aftereffect), these can be formulated as follows: (a) there is an effect of the group on error relevance, (b) there is an effect of error relevance on the aftereffect, and finally (c) the effect of the group on the aftereffect becomes smaller when error relevance is added to the linear model (see Fig. 2).

Fig. 2
figure 2

Illustration of the mediator hypotheses H3 and H4. Description for (a) (b) and (c) see text

Full size image

Prior to hypothesis testing, we conducted a manipulation check. To test whether the arrow group perceived the hand representation more abstractly than the control group, we asked participants to rate the degree of realism of the pointer (realism perception: “How realistic would you rate the representation of the pointer in VR compared to the rest of the VR environment?”).

4 Results

4.1 Demographics and manipulation check

There were no significant differences between the groups for age (F(2,27) = 0.62, p = 0.54, \({\eta }^{2}\)< 0.32), gender (χ2(2) = 0.83, psim = 0.89, V < 0.65), VR experience (F(2,27) = 1.52, p = 0.24, \({\eta }^{2}\)< 0.32), tiredness (F(2,27) = 1.08, p = 0.35, \({\eta }^{2}\)< 0.32) and baseline performance (F(2,27) = 0.17, p = 0.85, \({\eta }^{2}\)< 0.32). No significant group differences were found for the time from start to end of the pointing movement in the baseline (F(2,27) = 0.39, p = 0.68, \({\eta }^{2}\)< 0.32), exposure (F(2,27) = 0.26, p = 0.78, \({\eta }^{2}\)< 0.32) and de-exposure phase (F(2,27) = 1.29, p = 0.29, \({\eta }^{2}\)< 0.32). As expected, the realism of the arrow in the arrow group was perceived significantly worse than that of the virtual hand in the control group (t(18) = 1.75, p = 0.049, \(\widehat{d}\) = 0.78).

4.2 Magnitude and persistence

Figure 3 shows the horizontal deviation depending on group and phase. Both the direct effect in the exposure phase and the aftereffect in the de-exposure phase are present in all groups.

Fig. 3
figure 3

Horizontal deviation as a function of the group and phase (without baseline correction). Positive values correspond to a deviation to the left, and negative values to a deviation to the right. The points represent group means, and the error bars represent standard deviations

Full size image

The magnitude of the aftereffect, i.e. the baseline-corrected horizontal deviation in the first trial of the de-exposure phase, for the three groups is shown in Fig. 4 on the left side. The one-sided t-tests on the model coefficients revealed that both the misinformation group (t(27) = 1.38, p = 0.91, \({d}_{\psi }\)< 1.35Footnote 1) and the arrow group (t(27) = 0.17, p = 0.57, \({d}_{\psi }\)< 1.35) showed no significant differences from the control group. Thus, the magnitude of the aftereffect does not seem to be influenced either by the misinformation about an erroneous correspondence between virtual and real hand (H1a) or by the 3D arrow representation of one’s own hand (H2a).

Fig. 4
figure 4

Magnitude (left) and persistence (right) as a function of the group. The points represent individual values, the bars represent group means, and the error bars represent standard errors. The significance asterisks indicate only results of hypothesis-based tests, e.g. differences between the misinformation group and the arrow group were not tested

Full size image

The statistical model used to analyse the magnitude was fitted for the variable persistence, which is defined as the mean absolute baseline-corrected horizontal deviation in the de-exposure phase without the first trial. Figure 4 on the right side shows the results. There was a significant difference between the misinformation group and the control group (t(27) = -2.46, p = 0.01, \({\widehat{d}}_{\psi }\) = 1.10). This confirms the hypothesis that the misinformation about an erroneous correspondence between a virtual and a real hand leads to a less persistent aftereffect (H1b).

In contrast, the arrow group showed no significant differences from the control group (t(27) = 1.99, p = 0.97, \({d}_{\psi }\)< 1.35). The hypothesis that the 3D arrow representation leads to a less persistent aftereffect (H2b) can be rejected (a one-sided t test in the other direction would even have reached significance).

4.3 Error relevance

A linear model with error relevance as criterion and the groups as predictors was fitted. The proportion of variance explained was not significantly different from zero (F(3,26) = 0.75, p = 0.53, R2 = 0.08). Error relevance showed no significant correlation with magnitude (r = 0.05, p = 0.81) or persistence of the aftereffect (r = -0.12, p = 0.53). Adding the error relevance to the linear model for analysing the group effects did not result in any changes in the group differences. As the conditions for a mediator effect were not met (Baron and Kenny 1986), it cannot be confirmed that error relevance mediates the effect of misinformation (H3) or 3D arrow representation (H4) on magnitude and persistence.

4.4 Explorative data analysis

The groups were tested for differences in pointing movement during exposure. As in the de-exposure phase, a distinction was made between baseline-corrected performance in the first trial and mean absolute performance in the remaining trials. A one-way ANOVA was performed in each case. No significant group differences were found (F(2,27) = 1.49, p = 0.24, \({\eta }^{2}\)< 0.32 and F(2,27) = 0.88, p = 0.43, \({\eta }^{2}\)< 0.32).

The standard deviation of the realism perception was almost twice as large in the arrow group (SDArrow = 2.75) as in the other groups (SDControl = 1.81, SDMisinf. = 1.52), indicating large individual differences within the arrow group. Therefore, realism perception was added as a metric predictor with interaction in the linear model. Within the arrow group, the slope of the regression line was significant (t(24) = 1.71, p = 0.04, \({\widehat{d}}_{\psi }\) = 0.23). In other words: within the arrow group, a higher realism perception led to a higher magnitude (see Fig. 5). Thus, the hypothesis about the effect of the 3D arrow representation on magnitude (H2a) cannot be confirmed at the group level, but there is some evidence that the realism perception of the arrow had an influence on magnitude.

Fig. 5
figure 5

Magnitude as a function of the group and the realism perception. The points represent individual values, and the lines represent regression lines

Full size image

Figure 3 gives an indication that the horizontal deviations across the groups showed certain patterns. This is particularly evident in the baseline phase: across all groups, trial 1 and trial 9, for example, showed a high horizontal deviation to the left. These are the only trials in the baseline phase with a left target position (as the horizontal deviation is the dependent variable in the experiment, vertical differences in the target positions are not considered here). To investigate the effect of target position on horizontal deviation, a 3 (within; horizontal target position: left vs. centre vs. right) × 3 (within; phase: baseline vs. exposure vs. de-exposure) × 3 (between; group: control vs. misinformation vs. arrow) mixed ANOVA was conducted. The first trial in each of the exposure and de-exposure phases was removed from the analysis to prevent confounding effects from distorting the results. For these trials, confounding effects existed because they had the largest horizontal deviation and were assigned a specific target position at the same time. Figure 6 shows the descriptive results. The two main effects horizontal target position (F(2,54) = 56.44, p < 0.001, \({\widehat{\eta }}_{p}\)= 0.68) and phase (F(2,54) = 25.07, p < 0.001, \({\widehat{\eta }}_{p}\)= 0.48) reached significance. In addition, the interaction between horizontal target position and phase was significant (F(4,108) = 7.69, p < 0.001, \({\widehat{\eta }}_{p}\)= 0.22). There were no other significant effects.

Fig. 6
figure 6

Horizontal deviation as a function of the group, the target position, and the phase (first trial not included). The bars represent group means, and the error bars represent standard errors

Full size image

5 Discussion

In the present experiment, the effect of error attribution through instruction and body representation was investigated. The hypotheses regarding instruction could only be partially confirmed: the misinformation group, which was instructed that a random error was built in in addition to the visual displacement, did not show a significantly different magnitude, but a significantly smaller persistence than the control group.

In terms of magnitude, Welch's (1972) results could not be replicated. Although the same underlying mechanisms were assumed, there are many differences between these two experiments in terms of the manipulation. First, Welch's misinformation created the illusion of object identity between the luminous finger and the participant's own finger, and to achieve this, the visual environment was reduced to a minimum (target and feedback were shown in darkness). In the present experiment, misinformation was used instead to break this illusion within an immersive VR environment. Perhaps the immersion of the VR was too high to break the illusion to such an extent that an effect on magnitude could be shown here. Second, Welch's misinformation manipulated the perceived reason for the displacement of the feedback. The informed group correctly believed that the luminous dot belonged to another person who was simply providing feedback offset from their own finger. The misinformed group falsely believed that the displacement was caused by prism glasses. In the present experiment, all participants were instructed that the virtual environment was visually displaced. The manipulation only affected the perceived accuracy of the feedback. Thus, the global displacement was not questioned, only the local feedback within the displacement. Although this may have reduced the effect of the instructions, it should also have led to a lower unity assumption, i.e. that the virtual hand is seen less as the same object as one's own hand. Finally, it should be noted that magnitude was only measured using one trial. To increase reliability, several pointing movements without visual feedback experience should be included next time.

In terms of persistence, the results were in accordance with the hypothesis. Besides the interpretation with respect to the unity assumption, further explanations are possible. For example, the false information of a random perturbation might have led to a higher perception of feedback variability. Studies have already shown that the actual additional trial-to-trial variability of a perturbation reduces the extent of adaptation during exposure (Havermann and Lappe 2010; Fernandes et al. 2012). According to Bayesian models, manipulating variability changes how much participants rely on new sensory information (Fernandes et al. 2012). Although the actual variability of the feedback was not manipulated, the illusion of higher variability may have led to greater use of cognitive strategies. When the contribution of cognitive strategies is increased, the aftereffects are reduced. Michel et al. (2007), for example, could show that a constant displacement, compared to a gradual displacement, led to greater awareness of the perturbation and was thus associated with a smaller aftereffect. Accordingly, the instruction that drew attention to a possible perturbation may also have increased awareness and thus led to less persistence of the aftereffect.

The hypotheses regarding body representation could not be confirmed, neither in terms of magnitude nor persistence. One possible explanation could be that the difference between hand and arrow in terms of how much it affects object identity was too small. All studies that were able to show an effect of an abstract body representation on the aftereffect compared different media: direct view of the hand vs. a video of the hand (Norris et al. 2001) vs. computer-generated image of a cursor or a line (Clower and Boussaoud 2000; Norris et al. 2001; Veilleux and Proteau 2015; Aziz et al. 2020). In the present experiment, abstraction was varied within the same medium, namely within VR. Comparisons between media include other differences that can influence aftereffects, such as resolution. The effects within a medium may be much smaller, as can be seen in our experiment, where only the representation is changed.

To reveal possible effects of representation within a medium, the difference between abstract and realistic representation should be further increased in terms of object identity. To further reduce object identity in an abstract representation, other forms of representation than an arrow could be investigated, which are perceived as less realistic. Realism perception was significantly associated with magnitude within the arrow group. Those who considered the arrow to be a realistic representation of their hand showed a higher magnitude of the aftereffect. In the research of body ownership, it has been shown that the viewed object should match a visual representation of the body part to elicit a sense of body ownership (see overview in Tsakiris 2010). For example, hand-shaped objects induce stronger body ownership illusions than non-hand-shaped objects (Haans et al. 2008). In the current experiment, the subjective realism rating of the arrow could to some extent reflect an identification with a computer-generated arrow as a valid representation of one's own hand, due to the habit of working on the computer. If this hypothesis is true, then the realism assessments should be related to the experience with computer-generated arrows in everyday life, which, however, was not measured in the context of this experiment. In future studies, in order to increase the group differences, other forms of representation can be investigated that are not already used in computer-influenced everyday life, but those that are not yet associated with hand and pointing movements.

In order to increase the object identity in the realistic representation, a better virtual hand model should be used, one that corresponds as closely as possible to the users' hands. To meet the individuality of the users, customisations can be made, for example to their sex and the colour and size of their hands. Another aspect concerns the realism of the movement. In the present experiment, the index and the middle finger were fixated to enable finger tracking. This fixation suits an arrow with fixed components better than a hand with movable fingers. This could be the reason for the surprising opposite effect that the arrow group showed a more persistent aftereffect than the control group. In future studies, a data glove, for example, could help to reduce this incompatibility and to transfer the mobility of the fingers into virtuality.

The hypotheses regarding error relevance could not be confirmed. Neither the misinformation group nor the arrow group showed an effect on perceived error relevance. Previous research has already revealed the influence of error relevance at the behavioural level (Wilke et al. 2013). The idea in this experiment was to measure the perception of error relevance through self-report. However, it is possible that the measurement of error relevance was not sensitive enough. The formulation referred to a general movement error and was thus too unspecific. Individual interpretations of what the movement error refers to probably increased the variance, so that no effect could be found. In addition to the error relevance, the indirect measurement of the sense of embodiment by questions about its subcomponents self-location, agency, and body ownership would have been helpful for the interpretation of the results (see an overview in Kilteni et al. 2012a).

In addition to the hypothesis testing, a further exploratory data analysis revealed an interesting effect: the target position led to a systematic movement error, namely to a stronger outward deviation, i.e. right positions led to a stronger deviation to the right and left positions to a stronger deviation to the left. This target position effect can probably be attributed to an underestimation of the egocentric distances in VR. Since no concurrent feedback about the movement was given, the movement planning had to be based on the target distance estimation. It is already known that distances are underestimated in VR (Renner et al. 2013). As can be seen in Fig. 7, the biased distance estimation may have led to biased movements as well. Due to the lack of concurrent feedback about the movement, this bias was not corrected during the movement and was reflected in a systematic movement error. This is relevant for the present study insofar as the magnitude was measured over a single target position, namely “down-right”. Since the aftereffect is defined as a deviation to the left in this experiment (because of the visual deviation to the right), the magnitude of the aftereffect was presumably reduced by the target position effect for all groups, which should be considered in the general evaluation of its size.

Fig. 7
figure 7

Pointing directions when distance to the wall is correctly estimated (left) and pointing directions when the wall is perceived as closer in VR (right)

Full size image

To our knowledge, there are no studies that focus on outward deviations in all phases without concurrent feedback. Redding and Wallace (2006) investigated the differentiation between recalibration and realignment, as well as untrained and trained target positions. They found that recalibration aftereffects were stronger when de-exposure and exposure conditions were similar regarding limb posture and training positions. Realignment aftereffects were not dependent on these conditions and generalised nonassociatively. It was also depicted that for terminal feedback in the de-exposure phase, targets on the left side had a stronger realignment aftereffect than targets positioned straight-ahead, and these again showed a stronger aftereffect than targets positioned on the right. The authors claim that proprioceptive targets to the right of the de-exposure straight-ahead position are trained, targets to the left are untrained (Redding and Wallace 2006). In our study, the left and right outward deviation was also found in the baseline phase with only ten trials, without concurrent feedback, and with targets positioned left, right, and straight-ahead. These results are related to, but not the same as, the outward biases, as we found a stronger outward deviation during all phases, i.e. baseline, exposure, and de-exposure phase.

Thomas (2012) investigated movement exercises crossing and not crossing the body midline via a sensorimotor adaptation task. The task differed from a prism adaptation task, as the movement was not explicitly hidden, i.e. the used hand was held besides the body and controlled a joystick, the targets were positioned equidistantly on the circumference of a not depicted circle on a desktop display and were rotated 60 degrees clockwise during the exposure phase. One group pointed at targets crossing their body midline, the other group pointed at targets not crossing their body midline. The results suggest that the retention of adaptive behaviour is weakened when the pointing movements cross the body midline. Barral and Debû (2004) did not focus on adaptation but on pointing accuracy, sex, and the preferred and non-preferred hand of right-handed participants. They found a leftward bias for the left hand and the opposite effect for the right hand (Barral and Debû 2004). In our case, this would only explain the rightward bias found for targets on the right side, e.g. during the baseline phase, but contradicts the leftward bias found for targets positioned on the left side. These two studies could partly explain that pointing accuracy could differ for targets positioned on the left or right due to midline crossing or handiness. However, both theories do not explain a horizontal outward deviation in both directions for all phases without concurrent movement feedback.

6 Conclusion and outlook

The present experiment investigated the influence of instruction and body representation on magnitude and persistence of the aftereffect following sensorimotor adaptation in VR. The instruction was manipulated via false information about a random error component and influenced the persistence. This demonstrates that cognitive factors should be considered, for example, in VR training. Further research could investigate, for instance, whether the instruction can influence effects of actual variability of visual error on the aftereffect. In this experiment, only negative expectations of the VR system were investigated. There were two reasons for this. First, with VR, it is generally more difficult to convince the user of a perfect application than of a faulty application. Second, the sample consisted of students who are generally very interested in technology, some of whom already have experience with VR and know that a flawless application is an exaggeration. Future studies could investigate the impact of positive expectations, e.g. grandiose capabilities of the device, on adaptation and aftereffects in VR.

No effects of body representation were found, but the results indicate that individual perception of realism may play a role. Since VR is becoming more and more realistic and at the same time can offer a novel experience in relation to one's own body, future studies could consider realism and unfamiliarity as possible factors influencing sensorimotor adaptation. Finally, a systematic movement error, the target position effect, was exploratively found, which also offers starting points for further research, e.g. whether this effect depends on the type of pointing movement (hand vs. cursor).