Effects of auditory feedback on movements with two-segment sequence and eye–hand coordination

Abstract

The present study investigated the effect of auditory feedback on planning and control of two-segment reaching movements and eye–hand coordination. In particular, it was examined whether additional auditory information indicating the progression of the initial reach (i.e., passing the midway and contacting the target) affects the performance of that reach and gaze shift to the second target at the transition between two segments. Young adults performed a rapid two-segment reaching task, in which both the first and second segments had two target sizes. One out of three auditory feedback conditions included the reach-progression information: a continuous tone was delivered at a consistent timing during the initial reach from the midway to the target contact. Conversely, the other two were control conditions: a continuous tone was delivered at a random timing in one condition or not delivered in the other. The results showed that the initial reach became more accurate with the auditory reach-progression cue compared to without any auditory cue. When that cue was available, movement time of the initial reach was decreased, which was accompanied by an increased peak velocity and a decreased time to peak velocity. These findings suggest that the auditory reach-progression feedback enhanced the preplanned control of the initial reach. Deceleration time of that reach was also decreased with auditory feedback, but it was observed regardless of whether the sound contained the reach-progression information. At the transition between the two segments, the onset latencies of both the gaze shift and reach to the second target became shorter with the auditory reach-progression cue, the effect of which was pronounced when the initial reach had a higher terminal accuracy constraint. This suggests that the reach-progression cue enhanced verification of the termination of initial reach, thereby facilitating the initiation of eye and hand movements to the second target. Taken together, the additional auditory information of reach-progression enhances the planning and control of multi-segment reaches and eye–hand coordination at the segment transition.

Appendices

Appendix 1: An assessment of a practice effect due to two different orders of auditory conditions

In the present study, half of the participants used the order A (Fix–Ran–NoS–Ran–Fix–NoS) of auditory conditions, while the other half used the order B (NoS–Fix–Ran–NoS–Ran–Fix). Thus, it is important to ensure that differences among the auditory conditions observed in the present study were not just artifacts of practice that could stem from the different orders of auditory conditions (i.e., order A vs B). Thus, we tested this aspect for all significant results of the sound-type effect and its interaction with other factors. Namely, we applied a 2 (group: order A, order B) × 2 (T1 size: large, small) × 2 (T2 size: large, small) × 3 (sound type: Fix, Ran, NoS) ANOVA with repeated measures, where group was a between-subject factor, in order to see whether the group factor had an interaction with the sound-type factor. The results were as follows.

Regarding the parameters that had a significant sound-type effect (reported in the main text), a significant interaction between group and sound type was found for movement time (F(2,36) = 4.51, p = 0.018) and deceleration time (F(2,36) = 4.14, p = 0.024). Conversely, there was no such an interaction for constant error in the X-position (F(2,36) = 0.14, p = 0.870), peak velocity (F(2,36) = 1.72, ɛ = 0.74, p = 0.203), time to peak velocity (F(2,36) = 2.30, p = 0.115), inter-segment interval (F(2,36) = 0.17, p = 0.846), and hand_offset-to-eye_onset dwell time (F(2,36) = 0.92, p = 0.407). Regarding the parameters that had a significant interaction between sound type and T1 size (reported in the main text), there was no significant 3-way interaction among group, sound type, and T1 size in all parameters (hand reaction time: F(2,36) = 0.57, p = 0.566; time to peak velocity: F(2,36) = 1.01, p = 0.374; inter-segment interval: F(2,36) = 1.62, p = 0.211; and hand_offset-to-eye_onset dwell time: F(2,36) = 0.26, p = 0.772). These results indicate that the movement time and deceleration time had some practice effect stemming from the different orders (i.e., order A vs B) that affected the results of sound-type effect. In contrast, no other parameters showed any such effect, meaning that differences among auditory conditions were unaffected by the practice effect.

We further assessed the practice effect on the movement time and deceleration time. In the present study, these two parameters showed a similar sound-type main effect (reported in the main text). Namely, the NoS condition had significantly longer duration than the Fix and Ran conditions, while the latter two did not differ from each other (Fig. 3a, d). Thus, the key question here is whether the lengthening of the NoS condition was caused by a pure practice effect or a sound-type effect that remained after taking into account the practice effect. Figure 5 shows mean movement time (plot a) and mean deceleration time (plot d) for three auditory conditions in each order group. Both parameters showed a similar pattern: Namely, both groups increased the durations for the NoS condition compared with the other two conditions. However, the increase was pronounced in the order-B group compared to the order-A group, which led to the above significant group by sound-type interaction.

Supposedly, if there were only a pure practice effect, both parameters for the NoS condition in the order-B group would have increased compared with the other two conditions because the NoS condition preceded the other two, whereas both parameters for the NoS condition in the order-A group would have been decreased because the NoS condition followed the other two. Consequently, the relation between the NoS and the other two conditions would have been reversed between the two groups. However, the results showed that no such reversal occurred in both parameters (Fig. 5a, d) regardless of whether the NoS condition preceded (Group B) or followed (Group A) the other conditions. Therefore, the difference between sound conditions clearly remained despite the practice effect.

Furthermore, when mean movement time and mean deceleration time are plotted against the consecutive order of the 6 sub-conditions for the order-A group (Fig. 5b, e) and order-B group (Fig. 5c, f), it becomes clearer that both parameters for the NoS condition increased compared to the other two conditions (Fig. 5) regardless of whether the NoS condition preceded or followed the other conditions. For example, in the case of the order-A group (Fig. 5b, e), both parameters increased in the NoS condition following the Fix and Ran conditions. Similarly, in the case of the order-B group (Fig. 5c, f), both parameters also increased in the second NoS sub-condition following the first pair of Fix and Ran sub-conditions. If there were only a pure practice effect, the NoS condition would have decreased in both cases. But in reality, both parameters increased in this condition. Therefore, the sound-type effect of the longer NoS condition remained after taking into account the practice effect.

Fig. 5

Movement time (a–c) and deceleration time (d–f) of the initial reach are plotted for the order-A group (filled circles) and the order-B group (open circles). Mean values across all participants of each group are plotted for the fixed-timing (Fix), randomized-timing (Ran), and no-sound (NoS) conditions. The order-A group used the consecutive order (Fix–Ran–NoS–Ran–Fix–NoS), whereas the order-B group used another order (NoS–Fix–Ran–NoS–Ran–Fix). In plots a, d, mean values are calculated by pooling the first and second sub-condition within each auditory condition. In plots b, c, e, f, mean values are plotted across the consecutive order for each group. The error bars represent the SE

Appendix 2: An assessment of progressive changes over trials

Since the participants performed each auditory condition as blocks of trials in the present study, there is a possibility that experience from the previous trials about the auditory feedback was stored in memory and used in the planning of subsequent trials. If that were the case, hand movements might improve over trials within the 10-trial blocks. Thus, we tested this aspect for three parameters (hand reaction time, peak velocity, and time to peak velocity) that occurred prior to the delivery of the fixed-timing auditory cue and that showed a significant sound-type effect and/or a significant interaction between sound type and T1 size (reported in the main text).

For each auditory condition of each parameter, we calculated an overall mean of each trial across all participants, two T2 size conditions, and two (first and second) sub-conditions. Next, a change in the means over 10 trials within each auditory condition and each T1 size condition was analyzed by using a linear regression analysis. Peak velocity for the large T1 condition showed a progressive increase over trials in the Fix and Ran conditions (Fig. 6a, b), revealing a significant positive trend across trials (for Fix, correlation coefficient: r = 0.81, p = 0.005, slope = 4.91; for Ran, r = 0.93, p < 0.001, slope = 4.68). Conversely, there was no significant change over trials in the NoS condition (Fig. 6c, r = 0.53, p = 0.112, slope = 2.34). On the other hand, peak velocity for the small T1 condition showed different patterns. There was no progressive change of peak velocity over trials for the Ran and Fix conditions (Fig. 6d, e; Fix: r = 0.40, p = 0.248, slope = -0.82; Ran: r = 0.37, p = 0.296, slope = 1.00), whereas peak velocity significantly decreased over trials for the NoS condition (Fig. 6f, r = 0.73, p = 0.016, slope = − 3.25). Taken together, these results indicate that when auditory feedback is provided, regardless of whether it contains the reach-progress cue, the auditory input contributes to increasing the preplanned speed of initial reach when the reaching accuracy demand is low, but that input contributes to maintaining the speed across trials when the reaching accuracy demand is high.

Contrary to peak velocity, time to peak velocity did not show any significant change over trials in any of the auditory conditions combined with the large T1 (Fix: r = 0.08, p = 0.828, slope = 0.08; Ran: r = 0.20, p = 0.576, slope = − 0.37; NoS: r = 0.20, p = 0.572, slope = − 0.30) or combined with the small T1 (Fix: r = 0.16, p = 0.659, slope = 0.36; Ran: r = 0.38, p = 0.279, slope = 0.55; NoS: r = 0.37, p = 0.297, slope = 0.56). Similarly, hand reaction time did not show significant changes in any of the auditory conditions combined with the large T1 (Fix: r = 0.43, p = 0.219, slope = − 1.37; Ran: r = 0.11, p = 0.763, slope = 0.37; NoS: r = 0.51, p = 0.136, slope = − 1.30) or combined with the small T1 (Fix: r = 0.09, p = 0.800, slope = 0.239; Ran: r = 0.00, p = 0.994, slope = 0.01; NoS: r = 0.01, p = 0.974, slope = − 0.02).

Fig. 6

The changes of peak velocity across 10 trials for the fixed-timing (Fix: a, d), randomized-timing (Ran: b, e), and no-sound (NoS: c, f) conditions. Mean values across all participants, two T2 size conditions, and two (first and second) sub-conditions of each auditory condition are plotted against the trial number for the large T1 (a–c) and small T1 (d–f) conditions. Solid lines indicate the linear-equation line