Introduction

The factors that influence grasping movements are multifarious in nature. It is well recognized that both extrinsic (position, orientation) and intrinsic (size, shape, mass, material) properties of the object affect how grasping movements are performed. Importantly, it has been also shown that perturbations of these properties can update the movements in-flight. Specific movement characteristics change when the object is displaced to a different position upon movement initiation (Paulignan et al. 1991b; Gentilucci et al. 1992; Carnahan et al. 1993; Gréa et al. 2000), when the object size is suddenly increased or decreased (Paulignan et al. 1991a; Castiello et al. 1993, 1998; Bock and Jüngling 1999; Dubrowski et al. 2002; Glover et al. 2005; Hesse and Franz 2009; van der Kamp et al. 2009), when the object is unexpectedly reoriented (Desmurget et al. 1996; Desmurget and Prablanc 1997; Tunik et al. 2005; Fan et al. 2006; Voudouris et al. 2013), or when the object shape is changed (Ansuini et al. 2007; Eloka and Franz 2011; Chen and Saunders 2015). Surprisingly, a less concerted attention has been directed toward the role played by the visual feedback of the hand during grasping movements. In the most commonly used procedure, visual feedback of the hand was either provided or simply eliminated by blocking the view of the moving limb (Jeannerod 1984; Jakobson and Goodale 1991; Gentilucci et al. 1994; Connolly and Goodale 1999; Churchill et al. 2000; Schettino et al. 2003; Watt and Bradshaw 2003; Winges et al. 2003; Fukui and Inui 2006; Rand et al. 2007).

The absence of the visual feedback of the hand usually causes an increase in the safety margin used to control the grip aperture. Unfortunately, merely withholding the visual feedback of the hand does not answer the question about the role of the visual feedback of the hand in the fine-tuning of grasping movements. Here, we investigated if and how direct vision of the hand supports this fine-tuning. Subjects were required to reach and grasp objects located at different distances, and the visual feedback of the grip aperture was artificially altered in a systematic fashion on a trial-by-trial basis.

An aspect that might play a relevant role in the on-line control of grasping movements and that has received scant attention in the literature, is the type and quality of information that can be visually monitored and that directly depends on the viewing geometry, that is, the relation between the viewing direction and the final grasp axis orientation. To illustrate this point, let us consider the case in which the final grasp axis is aligned with the plane perpendicular to the viewing direction. In this case, (a) both the thumb and the index finger remain clearly visible until contact with the object is made; and, (b) both grasp points on the object are constantly in view. Therefore, the span between the two digits of the approaching hand can be compared to the object’s relevant dimension and both digits can be monitored separately or simultaneously until they land on the object’s surface. By contrast, the situation is quite different in the case in which the final grasp axis is perfectly aligned with the viewing direction. In this case, (a) only one digit (usually, the thumb) remains clearly visible until contact with the object is made, whereas the other digit disappears behind the object during the final phase of the movement; and, (b) only one grasp point is constantly visible, the other one being occluded by the object itself. As a result, the span between the two digits cannot be directly matched to the object’s relevant dimension and only one digit can be monitored until the very end of the movement. In addition, stereoscopic judgments of depth extent are less precise than judgments of lateral extent (McKee et al. 1990; Gepshtein and Banks 2003), making both the depth extent of the object as well as the span between the digits, when they are in a radial direction relative to the observer, harder to estimate.

Following this line of thinking, we ran two experiments in which we varied the viewing geometry. In the first experiment, the final grasp axis orientation was along an oblique axis and it was thus almost perpendicular to the viewing direction. On the contrary, in the second experiment, the final grasp axis orientation was along the depth axis and was thus aligned with the viewing direction.

The main manipulation in this study was the modification of the visual feedback about the grip aperture, i.e., the Euclidean 3D distance between the two virtual digits. To this end, we altered the mapping between the actual and the visual grip aperture. In principle, this mapping can be changed in several ways. One way would be to increase (or decrease) the visual grip aperture by a constant offset as it was done in the studies by Marino et al. (2010) and Bernardi et al. (2013). Another way would be to gradually increase (or decrease) the visual grip aperture by attaining the maximal perturbation intensity at the end of the movement as it was done in the study by Karok and Newport (2010). However, both these alternatives lead to unwanted consequences. At the end of the movement, the visual size of the object, the visual size of the grip aperture and the proprioceptive information about the grip aperture never coincide resulting in an incongruence among the senses.

Fig. 1
figure 1

Perturbation of the visual grip aperture. a Relation between the visual and the actual grip aperture depending on the gain. b The visual feedback representing the tips of the digits was displaced along the grasp axis depending on the gain (dark blue—increase, medium blue—veridical, light blue—decrease). c Changes in the visual grip aperture with respect to an actual grip aperture from start to end of the movement (color figure online)

Full size image

Instead, we used a new dynamic perturbation method to alter the position of the visual feedback relative to the actual position of the thumb and index fingertips to virtually increase/decrease the visual grip aperture. Importantly, with this method the visual grip aperture is smoothly altered only when the actual grip aperture is larger than the size of the object to be grasped. In this way, the perturbation is present during most of the movement, but vanishes the moment the digits contact the object, in our case, a sphere. When the actual grip aperture is smaller than the diameter of the sphere, the visual grip aperture (v) is kept identical to the actual grip aperture. Instead, when the actual grip aperture is larger than the diameter of the sphere, the visual grip aperture is defined through the following equation:

$$\begin{aligned} v(a) = k a - 2 r (k-1), \end{aligned}$$
(1)

where a is the actual grip aperture, r is the radius of the sphere, and k is the gain that changes the mapping between the actual and the visual grip aperture. In practice, each visual digit is displaced in opposite directions along the grasp axis by an amount equal to \({(v-a)/2}\). Figure 1a represents the three mappings used in the present study obtained by setting the gain to 0.25, 1 or 1.75. When the gain was set to 1, the visual feedback of the digits was veridical (i.e., coincident with the actual position of the tips of the two digits). When the gain was set to 0.25, the visual grip aperture was smaller than the actual grip aperture. And, when the gain was set to 1.75, the visual grip aperture was larger than the actual grip aperture (see Fig. 1b). Figure 1c shows how the different gains alter the visual grip aperture with respect to an actual grip aperture profile from start to end of the movement. Note again that the perturbation appears when the actual grip aperture is larger than the object size and it disappears the instant the digits enclose the object.

Fig. 2
figure 2

Potential effects of gain at a specific point of the trajectory. If the actual grip aperture is adjusted to be wider (c), when the visual grip aperture is narrower (gain: 0.25), and the actual grip aperture is adjusted to be narrower (d), when the visual grip aperture is wider (gain: 1.75), then the slope of the actual grip aperture as a function of gain should be negative (a). The adjustments of each digit position in terms of distance (along the grasp axis) with respect to the final grasp point should also display a similar pattern with only negative slopes. However, if only one digit (e.g., the thumb) is under visual control and the other digit follows the first, the position adjustment of the digit under visual control should show a negative slope (b, solid lines), whereas the position adjustment of the other digit should show a slope of opposite sign (b, dashed lines). This would reflect the fact that the digit under visual control is farther away from its final grasp point, and the other digit is closer to its final grasp point (e), when the gain is 0.25, and the digit under visual control is closer to its final grasp point and the other digit is farther away from its final grasp point (f), when the gain is 1.75. Different shades of green represent different slope values, i.e., the strength of the effect of gain (color figure online)

Full size image

If grasping movements are entirely based on feedforward movement control, then no on-line adjustments are expected to occur. The extent by which the changes in the visual grip aperture affect the grasping movements is instead indicative of the role played by the visual feedback mechanisms (on-line visual control). However, whereas in reaching movements the hand trajectory can be amended by simply controlling the hand position or its motion (Paillard 1996; Saunders and Knill 2003, 2004; Gaveau et al. 2014), it is less clear which aspects are under on-line visual control in grasping movements (Glover 2004). One possibility is that the grip aperture is under visual control in the attempt to conform it to the object’s dimensions. Another possibility is that the positions of the digits are under visual control to guide them toward their respective grasp points. Clearly, the above aspects might be under simultaneous control and the degree of control might also depend on the task constraints such as the different grasp types needed to enclose the object (Volcic and Domini 2014).

In order to examine the influence of on-line visual control, we evaluated if and how the actual grip aperture and/or the digit positions are affected by the different gains. If the actual grip aperture is under visual control, we would expect a specific pattern of compensatory adjustments. The larger the compensation at a specific point of the trajectory, the steeper the negative slope of the actual grip aperture as a function of gain (see Fig. 2a). This would reflect the facts that the actual grip aperture is adjusted to be wider (Fig. 2c), when the visual grip aperture is narrower (gain: 0.25) and that the actual grip aperture is adjusted to be narrower (Fig. 2d), when the visual grip aperture is wider (gain: 1.75). The adjustments of each digit position in terms of distance (along the final grasp axis) with respect to the final grasp point should also display a similar pattern with negative slopes. The slopes are less steep in this case, because the perturbation affecting each individual digit is half the perturbation of the grip aperture. However, suppose that only one digit (for instance, the thumb) is under direct visual control and the other digit more or less rigidly follows the first. When the hand is relatively close to the object, we would expect the following scenarios. When the gain is small and thus the thumb visual feedback is displaced inwards, the digit under visual control should be farther away from its final grasp point and the other digit should be closer to its final grasp point (Fig. 2e). Conversely, when the gain is large and thus the thumb visual feedback is displaced outwards, the digit under visual control should be closer to its final grasp point and the other digit should be farther away from its final grasp point (Fig. 2f). Therefore, the position adjustment of the digit under visual control should show a negative slope (see Fig. 2b, solid lines), whereas the position adjustment of the other digit should show a slope of opposite sign (see Fig. 2b, dashed lines).

An important question is not only whether the perturbation of the visual grip aperture has an influence on grasping movements, but also how this influence evolves during the movement. To this aim, we computed the slopes for the actual grip aperture and for the positions of the digits at evenly spaced points along the three-dimensional trajectory to obtain the slope profiles that characterize the nature and extent of the on-line visual control along the path that the hand covers from movement start to the instant it encloses the object. Studies that examined eye movements in grasping have established that the first saccades (even before hand movement onset) are directed on the target object, are locked there, and very seldom follow the moving hand (de Grave et al. 2008; Brouwer et al. 2009; Desanghere and Marotta 2011; Cavina-Pratesi and Hesse 2013; Grant 2015; Voudouris et al. 2016). We therefore expect the visual feedback of the hand to be effectively used near the end of the movement trajectory when the hand is in the close proximity of the object.

Methods

Participants

Twenty undergraduate students (nine females) participated in this study. All had normal or corrected-to-normal vision. All of the subjects were naïve to the purpose of the experiments and were paid for their effort. Half of them participated in Experiment 1, the other half in Experiment 2. Experiments were undertaken with the understanding and written consent of each subject, with the approval of the Comitato Etico per la Sperimentazione con l’Essere Vivente of the University of Trento, and in compliance with national legislation and the Code of Ethical Principles for Medical Research Involving Human Subjects of the World Medical Association (Declaration of Helsinki).

Fig. 3
figure 3

Experimental setup. a Picture of the setup showing the arrangement of the motion capture system (one of the two position sensors), the mirror in front of the monitor, the motors for moving the monitor, and the motors for positioning the real object. b In Experiment 1, subjects were asked to grasp the sphere along an oblique axis. c In Experiment 2, subjects were asked to grasp the sphere along the depth axis. Subjects never had full vision of the hand, only visual feedback about the fingertips was provided

Full size image

Apparatus

Subjects were seated in a dark room in front of a high-quality, front-silvered \(400\times 300\, \hbox {mm}\) mirror (see Fig. 3a). The mirror was slanted at 45\(^\circ\) relative to the subjects’ sagittal body mid-line and reflected the image displayed on a ViewSonic 9613, 19” CRT monitor placed directly to the left of the mirror, producing the illusion that the rendered 3D objects were displayed in the space in front of the subjects. For consistent vergence and accommodative information, the position of the monitor, attached to a linear positioning stage (Velmex Inc., Bloomfield, NY, USA), was adjusted on a trial-by-trial basis to equal the distance from the subjects’ eyes to the virtual object. To present visual stimuli in 3D, we used a frame interlacing technique in conjunction with liquid crystal FE-1 goggles (Cambridge Research Systems, Cambridge, UK) synchronized to the frame rate of the monitor. A custom C++ program was used for stimulus presentation and response recording. The estimated system lag was about \(27.9 \pm 1.3\) ms (Fantoni et al. 2014).

Head, wrist, index and thumb movements were acquired on-line at 100 Hz with sub-millimeter resolution by using an Optotrak Certus motion capture system with two position sensors (Northern Digital Inc., Waterloo, Ontario, Canada). Head movements updated the subjects’ viewpoint to present the correct geometrical projection of the stimulus in real time. Subjects’ head position and orientation was tracked with three infrared-emitting diodes worn on the back of the head. The position of the tip of each digit was calculated during the system calibration phase with respect to three infrared-emitting diodes attached on each distal phalanx. Additional details about the experimental setup are available in Nicolini et al. (2014).

A high-contrast random-dot visual stimulus was rendered in stereo simulating a sphere (radius: 30 mm). The stimulus was simulated at four different distances (420, 450, 480 and 510 mm) along the line of sight. A real sphere (radius: 30 mm) made of styrofoam was mounted on a linear positioning stage that was used to position the real sphere in the exact same location in space in which the visual stimulus was simulated. In this way, consistent visual and haptic feedback about the object position and its dimension was provided on each trial.

The setup allowed subjects to comfortably reach behind the mirror to perform reaching and grasping movements with their right hand. The hand starting position (a pole) was shifted relative to the body of the observer by about 270 mm to the right from the sagittal plane, 150 mm from the frontal coronal plane and 300 mm lower than the subjects’ line of sight. Thus, to perform the grasping movement, subjects needed to move their hand in leftward, forward and upward direction. The stimuli were at about 486, 503, 521, and 541 mm from the hand starting position. During the experiments, visual feedback of both the index finger and the thumb was provided by means of two small virtual spheres representing the tips of the two digits (see Fig. 1b).

Procedure

Each subject was tested in a dark room with his/her head positioned on a chin rest. Before starting the experiment, subjects were tested for stereo vision with a custom stereo-test in which they were required to report whether a frontoparallel disparity-defined surface with different degrees of curvature (base-to-peak depth differences between 5 and 30 mm) was bended toward or away from them. None of our subjects failed this test. The subjects were subsequently presented with a set of practice trials to get accustomed to the task. Subjects started each trial of the experiment with their thumb and index fingertips in contact and resting on the top of a pole. After the visual stimulus appeared the subjects had to reach toward and grasp with a precision grip the visual sphere that was superimposed on the real object. In two experiments, we manipulated the viewing geometry (i.e., the relation between the viewing direction and the final grasp axis orientation). In Experiment 1, subjects were asked to grasp the sphere along an oblique axis (see Fig. 3b), whereas in Experiment 2 they had to grasp the sphere along the depth axis by positioning the thumb and index finger on the front and back sides, respectively (see Fig. 3c). A thin line passing through the center of the sphere (30 mm longer than its diameter) and aligned with the requested final grasp orientation was shown to facilitate the correct positioning of the digits. The visual feedback of the digits was provided as soon as the digits entered in the subject’s visual field (i.e., the first instant at which the visual feedback of both digits was simultaneously rendered on the screen in front of the participant) and remained visible for the whole duration of the trial. The trial ended when both digits stayed for at least 500 ms on the object’s surface. At the end of each trial the monitor turned black and the subject returned with his/her hand to the starting position. Then, the monitor and the real object moved to the new position ready for the start of the next trial.

In both experiments, we used three values of gain (0.25, 1, 1.75) and four object distances (420, 450, 480 and 510 mm). Each of these combinations was repeated ten times. These trials were in pseudo-randomized order with the constraint that the same combination could not be presented on two consecutive trials. We used a randomized design to avoid adaptation effects and to focus principally on the on-line control of grasping movements.

Data analysis

The raw positional data were processed and analyzed offline using custom software. The raw data were smoothed and differentiated with a second-order Savitzky–Golay filter with a window size of 41 points. These filtered data were then used to compute velocities and accelerations in three-dimensional space for each fingertip and the wrist. The moment of the lowest, non-repeating wrist velocity value prior to the continuously increasing wrist velocity values was the point used to define the start of the movement. The end of the grasping movement was defined on the basis of the Multiple Sources of Information method proposed by Schot et al. (2010). We used the criteria that the grip aperture is close to the diameter of the sphere, that the grip aperture is decreasing, that the second derivative of the grip aperture is positive, and that the velocities of the wrist, thumb and index finger are low. Moreover, the probability of a moment being the end of the movement decreased over time to capture the first instance in which the above criteria were met. Trials in which the end of the movement was not captured correctly or in which the missing marker samples could not be reconstructed using interpolation were discarded from further analysis. The total number of trails was thus 1107 and 1155 in Experiment 1 and 2, respectively.

All statistical analyses were performed by fitting linear mixed-effects models which included as fixed effects the variables gain (0.25, 1, 1.75) and distance (420, 450, 480 and 510 mm). The optimal structure of the random component was determined using likelihood ratio testing by comparing nested models fitted with restricted estimate maximum likelihood (REML). The most parsimonious models in both Experiment 1 and Experiment 2 included independent random intercept and random slope terms for subjects. A comparison of models with nested fixed effects showed that the interaction between gain and distance had no effect on our variables of interest. Therefore, the final fixed structure did not contain this interaction term. In all these analyses, the denominator degrees of freedom were adjusted for the F-tests by using the Kenward-Roger method (Kenward and Roger 1997).

Results

Temporal characteristics of the grasping movements

In both experiments, the duration of the grasping movement was influenced by the visual feedback about the grip aperture (Experiment 1: \(F(1,8.9)=12.78\), \(p=0.005\), Experiment 2: \(F(1,8.9)=5.53\), \(p=0.043\)). The smaller the gain, the longer it took to execute the movement. On average, the smaller gain increased the duration (mean \(\pm\) SD) from \(2055 \pm 179\) to \(2263 \pm 184\) ms and from \(1825 \pm 95\) to \(1883 \pm 108\) ms with respect to the larger gain in Experiment 1 and Experiment 2, respectively. Intermediate durations were observed when gain was set to one and thus did not alter the visual feedback about the grip aperture (Experiment 1: \(2133 \pm 167\) ms; Experiment 2: \(1834 \pm 96\) ms). Obviously, movement duration increased also as a function of the distance at which the spheres were placed (Experiment 1: \(F(1,8.9)=28.26\), \(p<0.001\), Experiment 2: \(F(1,8.9)=50.27\), \(p<0.001\)).

In the present experiments, the randomization of the gain prevented subjects from anticipating the modulation of the visual grip aperture on a trial-to-trial basis. The magnitude of the visual grip aperture became potentially available only from the moment the hand entered in the subjects’ visual field. In the two experiments, this event occurred on average 883 and 742 ms after movement onset, just before the time of peak wrist deceleration (936 and 803 ms) and long before the actual grip aperture reached its peak (1378 and 1190 ms). We thus expected that the differences in movement duration elicited by the different gains would be noticeable after peak wrist deceleration only. In fact, the gain did not influence the duration of the segments between movement onset and peak wrist acceleration (Experiment 1: \(F(1,8.9)=0.95\), \(p=0.355\), Experiment 2: \(F(1,8.9)=0.24\), \(p=0.639\)), between peak wrist acceleration and peak wrist velocity (Experiment 1: \(F(1,8.9)=0.02\), \(p=0.892\), Experiment 2: \(F(1,8.9)=0.23\), \(p=0.642\)), and between peak wrist velocity and peak wrist deceleration (Experiment 1: \(F(1,8.9)=0.01\), \(p=0.989\), Experiment 2: \(F(1,8.9)=1.19\), \(p=0.302\)). The differences in movement duration did only arise in the last segment between peak wrist deceleration and the end of the movement (Experiment 1: \(F(1,8.9)=15.15\), \(p=0.004\), Experiment 2: \(F(1,8.9)=5.77\), \(p=0.039\)).

A common procedure in many kinematics studies involves time-normalizing each movement to study how the grasping trajectories evolve from movement onset to movement end. However, since we found systematic differences in movement duration as a function of gain, this procedure might introduce artifacts in the outcome of the results. Instead, we opted to normalize the grasping trajectories as a function of space (Cuijpers et al. 2004, 2006), because the length of the trajectories was not modulated by gain in either of the two experiments.

Space-normalized grasping trajectories

Each grasping movement was spatially normalized using the total length of the movement, i.e., the length of the trajectory that the midpoint between the fingertip positions covered from the starting position to the end of the movement. Because the number of samples inevitably varies across trials, each wrist, thumb and index trajectory was resampled in 100 points evenly spaced along the three-dimensional trajectory in the range from 0 (movement onset) to 1 (movement end) in 0.01 steps using cubic spline interpolation. Figure 4 represents the relationship between the time from movement onset and the space-normalized trajectory. Apart from the differences in the total duration of the movements, the curves of the two experiments look very similar. Whenever a result will be presented in terms of the space-normalized trajectory, this figure can be used as a conversion graph to transition from the spatial to the temporal analysis domain.

Fig. 4
figure 4

Relation between the time from movement onset and the space-normalized trajectory. The two curves indicate the means of the two experiments and can be used to move from the spatial to the temporal domain and back. The two disks mark where and when the visual feedback about the hand started to be available. The two diamonds mark where and when the visual grip aperture perturbation was maximal

Full size image

The top and the frontal view of the average thumb and index paths along the last 20 % of the trajectory are shown in Fig. 5. Differences in the trajectories depending on the available visual feedback are already qualitatively apparent through visual inspection. With increasing gain, the final parts of the approach trajectories are less curved and more direct, especially for the thumb. However, to thoroughly investigate the effect of the altered visual feedback of the grip aperture, we proceeded with a series of analyses that track its impact along the whole unfolding of the grasping movement.

Fig. 5
figure 5

Average thumb and index finger paths (last 20 % of the space-normalized trajectory). a Top view of the trajectories in Experiment 1 and Experiment 2. b A zoomed top view of the trajectories, frontal part of the object. c A zoomed top view of the trajectories, back part of the object. d Frontal view of the trajectories in Experiment 1 and Experiment 2. e A zoomed frontal view of the trajectories, lower part of the object. f A zoomed frontal view of the trajectories, upper part of the object. Different shades of blue represent the different gains (dark blue: 1.75, medium blue: 1, light blue: 0.25) (color figure online)

Full size image

For each trial and for each point of the space-normalized trajectory, we computed the Euclidean distance between the virtual fingertips (visual grip aperture), the Euclidean distance between the fingertips of the thumb and the index finger (actual grip aperture), and the distance of each digit with respect to the position of its final grasp point (thumb and index position). The digit position was calculated along the axis that is aligned with the final grasp axis (oblique and in depth in Experiment 1 and Experiment 2, respectively; see Fig. 6).

Fig. 6
figure 6

Variables of interest. Visual grip aperture: the Euclidean distance between the virtual fingertips (black squares). Actual grip aperture: the Euclidean distance between the fingertips (black disks). Thumb and index position: the distance between each digit and its final grasp point measured along the final grasp axis (black segments connecting the empty disks with the surface of the object). This figure represents the case in which the gain is larger than one

Full size image

For all these variables of interest, we fitted separate linear mixed-effects models to extract the slope parameter (gain) for each point along the space-normalized trajectory to construct the slope profiles. These slope profiles thus represent the progress of the gain effect along the trajectory.

Effect of gain on actual grip aperture

Figures 7a and 8a represent the gain slope profiles for Experiment 1 and Experiment 2, respectively. The visual grip aperture perturbation (dashed slope profiles) took effect already early in the movement, reached its maximum at around 0.9 of the space-normalized trajectory (that coincides with the point at which the peak grip aperture occurred) and dropped at movement end. The perturbation was thus effective for most of the movement. Compensatory adjustments of the actual grip aperture should thus result in slope profiles with negative values (i.e., larger/smaller actual grip apertures to counteract smaller/larger visual grip apertures). As is evident from Figs. 7a and 8a (continuous lines), the compensatory adjustments occurred, if at all, only toward the very end of the space-normalized trajectory, even though the visual feedback about the hand was available already at around 0.65 of the trajectory. When the grasp was performed along the oblique axis of the object, subjects actively used the visual information about the span between their digits to control the final phase of the grasping movement. On the contrary, we found only weak evidence of on-line grip aperture adjustments, when the grasp was performed along the depth axis. Figures 7a and 8a can be used in conjunction with Figure 4 to obtain additional information about the temporal aspects of the perturbation and of the compensatory adjustments of the actual grip aperture.

Fig. 7
figure 7

Effect of gain in Experiment 1 (grasping along the oblique axis). a Slope profiles representing the extent of the visual grip aperture perturbation (dashed line) and the compensatory adjustment of the actual grip aperture along the space-normalized trajectory (continuous line). b Slope profile representing the extent of the thumb position adjustment along the space-normalized trajectory. c Slope profile representing the extent of the index finger position adjustment along the space-normalized trajectory. Light gray bands represent the standard error of the slope parameters. Shaded horizontal bars in each panel provide a measure of where the slopes of the actual grip aperture (a) and digit position (b, c) were statistically different from zero. On the right, mean values of each variable as a function of gain at the point at which the slope profile reached the maximum negative value indicated by the arrow. Error bars represent the standard error of the mean (color figure online)

Full size image
Fig. 8
figure 8

Effect of gain in Experiment 2 (grasping along the depth axis). a Slope profiles representing the extent of the visual grip aperture perturbation (dashed line) and the compensatory adjustment of the actual grip aperture along the space-normalized trajectory (continuous line). b Slope profile representing the extent of the thumb position adjustment along the space-normalized trajectory. c Slope profile representing the extent of the index finger position adjustment along the space-normalized trajectory. Light gray bands represent the standard error of the slope parameters. Shaded horizontal bars in each panel provide a measure of where the slopes of the actual grip aperture (a) and digit position (b, c) were statistically different from zero. On the right, mean values of each variable as a function of gain at the point at which the slope profile reached the maximum negative (a, b) or positive (c) value indicated by the arrow. Error bars represent the standard error of the mean (color figure online)

Full size image

Effect of gain on digit position

We can further explore the on-line control of grasping movements by focusing on the separate approach trajectories of the thumb and index finger. As mentioned earlier, the position of each digit was determined along the final grasp axis. The larger the value of the digit position, the farther away (along the grasp axis) was the digit from its own grasp point. When the grasp was performed along the oblique axis, we detected a profile with negative slope values toward the end of the trajectory for both the thumb position (Fig. 7b) and the index finger position (Fig. 7c). The fact that both digits exhibited negative slope profiles is in agreement with the changes of the grip aperture, i.e., both digits were further away from the grasp point when the gain was small and, conversely, both digits were closer to their respective grasp points when the gain was large. However, gain had a more pronounced effect on the thumb than on the index finger position, even though it is usually the index finger that contributes mostly to grip formation (Wing and Fraser 1983; Paulignan et al. 1991a; Haggard and Wing 1997). This aspect provides evidence that the compensatory adjustments of the grip aperture and the maneuvering of the thumb were under simultaneous visual control. Contrariwise, when the grasp was performed along the depth axis, the slope values for the thumb and index finger positions were of opposite sign toward the end of the trajectory (Figs. 8b, c). Since only the changes in thumb position were consistent with the changes in visual feedback, it is clear that, in this case, the visual control was mainly concerned with the guidance of the thumb toward its final grasp point. Figures 7b, c, 8b, c can be used in conjunction with Fig.  4 to obtain additional information about the temporal aspects of the perturbation and of the compensatory adjustments of the digits.

Effect of distance on actual grip aperture

Ultimately, we turned our attention to a further aspect, that is, how grasping movements are influenced by the object’s position. It is known that the distance at which objects are presented affects the perceived object depth which becomes smaller at larger object distances (Wallach and Zuckerman 1963; Johnston 1991; Norman et al. 1996; Richards 2009; Volcic et al. 2013). Intriguingly, a similar systematic lack of depth constancy has been found also in grasping actions in which smaller maximum grip apertures were found at larger object distances (Bozzacchi et al. 2014; Bozzacchi and Domini 2015; Bozzacchi et al. 2016). In the present case, we thus expect that the larger the lack of depth constancy, the steeper the negative slope of the actual grip aperture as a function of the object’s distance. The interesting aspect here is that we do not restrict our focus on the maximum grip aperture only, but we can monitor the progress of this potentially systematic distortion along the whole movement trajectory. Figure 9a, b represent the distance slope profiles for Experiment 1 and Experiment 2, respectively. When the grasp was performed along the oblique axis, the grip aperture was unaffected by the different distances at which the objects were grasped, except when the hand was in the close proximity of the object. Presumably, the depth extent of the object became relevant only when the thumb needed to circumvent the frontal part of the object to avoid collision and to reach the appropriate contact location (Verheij et al. 2014). If the relative depth of the object was misjudged as a function of the distance, closer objects could have induced more pronounced avoidance trajectories than objects at farther distances. On the contrary, when the grasp was performed along the depth axis, the grip aperture was modulated from very early in the movement trajectory through to almost the end. The negative slope values indicate that the farther away was the object positioned, the smaller the grip aperture.

Fig. 9
figure 9

Effect of distance in Experiment 1 (grasping along the oblique axis, a) and in Experiment 2 (grasping along the depth axis, b). The slope profiles represent the effect of distance on the actual grip aperture along the space-normalized trajectory. Light gray bands represent the standard error of the slope parameters. Shaded horizontal bars in each panel provide a measure of where the slopes of the actual grip aperture were statistically different from zero. On the right, mean values of the grip aperture as a function of distance at the point at which the slope profile reached the maximum negative value indicated by the arrow. Error bars represent the standard error of the mean (color figure online)

Full size image

Discussion

The main purpose of the present study was to determine whether, and to what extent, a dynamic perturbation of the visual feedback about the grip aperture affects grasping movements. Several salient results in the present study speak of the role of this visual feedback. First, we found systematic variations of the temporal characteristics of the grasping movements and these variations were restricted to the latest phase of the movement (after peak wrist deceleration). The smaller the visual grip aperture was in comparison with the actual grip aperture, the longer it took to complete the grasping movement. Presumably, these changes reflect different precision requirements that depend directly on the relation between the size of the object and the magnitude of the visual grip aperture (MacKenzie et al. 1987). Second, grasping movements were smoothly and automatically adjusted to the current visual feedback, even though it was in most cases suboptimal (i.e., it did not match the actual grip aperture) and it thus might have been more advantageous to totally disregard it and to rely solely on haptic feedback. Third, and most important, we found that the changes in visual grip aperture modulated specific aspects of the grasping behavior. However, these modifications depended on the viewing geometry.

The conflict between the visually perturbed and the actual grip aperture could potentially affect the unfolding of the movement by impacting the actual grip aperture and/or the position of the digits. For the problem in question, we focused our analysis on two features of the grasping behavior: the actual grip aperture and the distance of each digit (thumb and index finger) with respect to the position of their final grasp point, both expressed in terms of slopes as a function of the altered visual feedback (gain) along the space-normalized trajectory. We found that the effects of the altered visual feedback arose preeminently in the latter part of the movement, even though the hand was visible substantially earlier. These adjustments occurred just after the grip aperture (and thus also the perturbation) reached its maximum and are compatible with the response latencies to visual perturbations that are due to sensory and motor delays (Scott et al. 2015). On the basis of this result, the visual feedback of the moving hand is thus feasibly used only when the hand is in parafoveal vision, it is in the close proximity of the object and it is not moving too fast.

However, what is of prime importance here is to determine which compensatory corrections ensued to counteract the visually perturbed grip aperture. In Experiment 1, in which objects were grasped along the oblique axis, we found concurrent effects on the actual grip aperture and on the digit positioning. The information about the altered visual grip aperture was actively used to update the actual grip aperture. Smaller visual grip apertures induced an increase in the actual grip aperture and larger visual grip apertures induced a decrease in the actual grip aperture. At the same time, the altered visual information affected also the positioning of the hand. In particular, the visual control was more concerned with the guidance of the thumb than of the index finger toward the final point of contact. In Experiment 2, in which objects were grasped along the depth axis, a different outcome emerged. The actual grip aperture was barely adjusted to the visually perturbed information that, instead, mainly influenced the approach trajectory of the thumb.

How is it that the same visual perturbation elicits different compensatory adjustments? The main reason is most likely related to the fact that the type and quality of visual information used for on-line control differed in the two experiments; the visual information about the span between the digits with respect to the object’s relevant dimension is directly determined by the viewing geometry. When grasping objects along the depth axis, the contact point of the index finger is always occluded by the object itself. As a result, no direct match between the visual grip aperture and the object extent is possible. On the contrary, when grasping the object along the oblique axis, the visual grip aperture can be more easily matched to the object’s relevant dimension. Moreover, because stereoscopic judgments of depth extent are harder than judgments of lateral extent, the identical perturbations might have been more difficult to notice when they occurred along the depth axis. Thus, the present findings suggest that the visual control of the actual grip aperture is only possible when direct vision of both digits is available almost until the end of the movement concertedly with the most favorable viewing geometry. In absence of this information, the visual control is mainly concerned with the guidance of the digit toward the visible contact point.

The relevance of direct vision of the digits might also help to explain the contrasting pattern we found when looking at the effect of object’s distance on the actual grip aperture. As we mentioned earlier, the distance at which objects are presented affects perceived object depth (Wallach and Zuckerman 1963; Johnston 1991; Norman et al. 1996; Richards 2009; Volcic et al. 2013) and similar systematic distortions are observed also in grasping actions (Bozzacchi et al. 2014; Bozzacchi and Domini 2015; Bozzacchi et al. 2016). The present results extend this observation. We found that the actual grip aperture was appreciably modulated by the object’s distance along most of the trajectory when the object was grasped along the depth axis, despite the fact that consistent haptic feedback was always provided and the fact that the object physical dimension was always the same. This finding thus strengthens the idea that the depth extent of objects is systematically misjudged not only in perceptual tasks but also in tasks that are based on grasping actions, and, that the visuomotor system can never achieve perfect calibration. Interestingly, the actual grip aperture was only partially affected by the object’s distance when the object was grasped along the oblique axis and the object’s relevant dimension was readily present in the retinal image plane. This distinction raises the intriguing possibility that certain grasping actions are not influenced by perceptual distortions only because the grip aperture can be constantly adjusted through visual control based on low-level visual properties such as the relative disparity between the digits and the grasp points, and the distances in the retinal image plane (Morgan 1989; Bradshaw et al. 2004; Melmoth and Grant 2006; Melmoth et al. 2007; Domini and Caudek 2013).

Collectively, the present findings highlight the role of the visual feedback of the hand in grasping movements. Furthermore, this study shows that the role of the visual feedback is, in fact, multifaceted, serving at least two functions. First, the on-line visual feedback assists the scaling of the grip aperture to properly conform it to the object’s dimensions. Second, the on-line visual feedback aids the transport of the hand to appropriately position the digits on the object’s surface. However, even though these functions can operate in tandem, specific constraints due to viewing geometry can limit their full emergence.