Keywords

1 Introduction

Since the early of 1990s, virtual reality technologies have been gradually developed and widely applied. These technologies utilize computer graphics to create a realistic-looking world in which users interact the virtual world via various tools and get real-time feedback. The advantage of rapid prototyping of high-cost or imaginary environments makes these technologies widely and successfully applied to the areas of medicine, education, game, rehabilitation, commercial, etc.

Although the techniques of virtual reality seem matured and beneficial, perception of depth has been a critical issue while interacting with a virtual environment. Studies show that the poor perception of depth would degrade hand control movement performance in a virtual environment. To generate depth information, several depth cues, such as texture gradient, shadows, and drop line, were developed and used.

To evaluate these depth cues, although Fitts’ law is a widely accepted method, the application of Fitts’ law has a limitation that only allows practitioners obtain the performance information confounded with the two movement properties: speed and accuracy. To overcome the limitation, Lin and his colleagues [1, 2] proposed the ballistic movement method. Hence, this study aimed at testing the application of the ballistic movement method for assessing the effects of depth cues on hand-control movement in a virtual environment.

1.1 Applications of Various Depth Cues in Virtual Environments

In the real world, people use a variety of information, such as the changes of luminance, contrast, and size, to determine the distance from an object. Hence, it is critical to generate same or similar depth information via two dimensional devices. Helpful aids commonly used include background grid, object shadow, drop line, size change and inter-reflections.

By using the method of distance estimation, several studies investigated depth cues. Hu, Gooch, Creem-Regehr and Thompson [3] showed that shadows and inter-reflections could improve depth perception. Waller [4] found that the application of background grid greatly help the performance. BĂĽlthoff and Mallot [5] compared the integration of the depth cues of stereo and shading. Kenyon, Sandin, Smith, Pawlicki and Defanti [6] showed that the size change is a helpful depth cue. Witmer and Kline [7] reported that floor pattern and object size were helpful for distance estimation.

To directly assess the effects of these cues on hand-control movement in virtual environments, Fitts’ law has been the most common method applied. Teather and Stuerzlinger [8] applied pointing tasks to show that texturing and support cylinders did not significantly influence performance. Liao and Johnson [9] found that droplines helped participants navigate straighter paths and benefited range dimension acquisition.

1.2 Fitts’ Law and Its Limitation

As shown in Eqs. 1 and 2, Fitts’ law [10] predicts that the movement time (MT) required to execute a Fitts-type aiming movement is linearly related to the index of difficulty (ID) of that movement, defined as the dyadic logarithm of the quotient of amplitude of the movement and target width (Eq. 2).

$$ MT = a + b \times ID $$
(1)
$$ ID = log_{2} \frac{2A}{W} $$
(2)

where A is movement amplitude, W is target width, a and b are experimentally determined constants. Because Fitts-type aiming movements are easily tested and the measured data can be well predicted by Fitts’ law, Fitts’ law thus become one of the most popular evaluation methods in the domains of Human Factors and Human Computer Interaction.

Although Fitts’ law is easy to apply, as mentioned by Lin and colleagues [1, 2], Fitts’ law has the limitation. The application of Fitts’ law only allows practitioners obtain the performance information that is confounded with the two motor properties: speed and accuracy. A Fitts-type aiming movement that takes a longer movement time could result from lower motor speed, lower motor accuracy, or a combination of both. However, Fitts’ law has difficulty discriminating the extent to which the two motor properties contribute to the overall movement time.

1.3 Ballistic Movement Method

In recent year, the general model proposed by Lin and colleagues [2, 11, 12] indicates that a Fitts-type aiming movement is composed of ballistic movements, which are basic movement unit. The movement time and the endpoint variability of a ballistic movement are two essential factors that directly affect the speed and accuracy of a Fitts-type aiming movement. Lin and Drury [13] tested two ballistic movement models for describing how these two properties are associated to ballistic movement distance.

Ballistic movement time represents the required time for performing a ballistic movement. In an experiment in which participants performed hand control movements on a drawing tablet, Lin and Drury [13] verified that Eq. 3 (the ballistic movement time model), proposed by Gan and Hoffmann [14], can effectively describe and predict the relationship between ballistic movement time (t ballistic ) and the squared root of ballistic movement distance (\( \sqrt {d_{u} } \)).

$$ t_{ballistic} = i + j \times \sqrt {d_{u} } $$
(3)

where i and j are experimentally constants.

Ballistic movement variability describes the endpoint variability of a ballistic movement. No matter endpoint errors are measured in the movement direction or perpendicular to the movement direction, Lin and Drury [13] found that the probability of endpoint location formed a normal distribution around the aimed point. In order to predict two directions of endpoint variability, Lin and Drury [13] verified the application of Eq. 4 (the ballistic movement variability model).

$$ \sigma = e + f \times d_{ballistic} $$
(4)

where e and f are experimentally constants. As shown in the equation, the endpoint variability is linearly related to the square of movement distance (\( d_{ballistic} \)).

The two ballistic movement models (i.e., Eqs. 3 and 4) have been tested in several conditions by Lin and his colleagues [2, 13]. Lin and Drury [13] originally verified the two models by asking participations to perform ballistic movements using a drawing tablet. Further, the models were tested by Lin and Tsai [2] to show that the two models could be used as an additional method to assess performance of computer mice.

According to Lin and Tsai [2], a self-paced aiming movement is composed of one or more than one ballistic movement. By measuring ballistic movements, practitioners not only can predict the performance of self-paced aiming movements, more importantly, practitioners can obtain separated performance of motor speed and motor accuracy.

1.4 Research Objectives

This study aims at accessing the effects of depth-cue combination on hand-control movement. More specifically, the ballistic movement method was used to obtain independent measurements of ballistic movement time, constant errors, and variable errors so that the effects of depth-cue combination could be independently accessed.

2 Method

2.1 Participants and Equipment

Three female and three male graduate students, aged from 19 to 20 years, participated in this study. They were all right-handed with normal or corrected-to-normal vision.

Experimental apparatus included a personal computer (PC), a 54.6” LED (Light-Emitting Diode) television (TV), a motion capture system (Flex 3, OptiTrackTM), and a self-developed program. The PC ran Unity3D and the motion capture system using the self-developed program that both displayed the experimental tasks on the TV and measured task performance.

2.2 Experimental Procedures

After informed consent procedures, as shown in Fig. 1, the six participants used a stylus to perform ballistic movements in conditions with different depth-cue combination. To reduce training effect, the participants had sufficient time to practice all the experimental combinations. To reduce the fatigue effect, the participants finished only a measurement (totally 4 measurements) in a half-day. A measurement took approximately 30–40 min.

Fig. 1.
figure 1

The execution of ballistic movement in a virtual environment

Full size image

To perform ballistic movements, as shown in Fig. 2, the participants quickly moved from a starting point to the center of the green cross target with a certain distance. The movements were performed at manipulated distances, but with the same midpoint set approximately 20 cm in front of body at elbow high. The tasks started by moving the stylus tip, attached with a 6.4 mm reflection marker, to control the cursor within the starting point. Once the cursor was at the starting point, the program generated a signal sound and showed the green cross target. The participants then moved the cursor toward the center of the target as quickly as possible. Once the cursor was moved away from the starting point, the cursor and the cross target disappeared and the movement time started to record. When the movement stopped, the cross target and the endpoint of that movement were immediately displayed on the screen. The experimenter clicked the space key to continue on the next trial.

Fig. 2.
figure 2

Relationships between three types of constant errors and ballistic movement distance for six movement directions

Full size image

2.3 Experimental Variables

The independent variables were scene (Scene), movement direction (Direction), movement distance (Distance). As shown in Table 1, three depth cues, comprising background gradient, shadows, and drop line, were used to generate six scenes with different combinations. Movements were performed in six directions, comprising anterior, posterior, right, left, superior, and inferior. For each direction, six movement distances, comprising 50, 100, 150, 200, 250, and 300 mm, were tested. Each experimental combination was replicated eight times, resulting in a total of 1,728 trials.

Table 1. Six virtual reality scenes with six combinations of depth cues
Full size table

The dependent variables were movement time (Time) and the endpoint errors measured at X (left-right), Y (up-down), and Z (back-front) coordinates. The errors consisted of constant error and variable error. To analyze whether the independent variables had significant effects on these two types of errors, eight replications of each experimental combination were utilized to calculate three types of constant errors (measured by mean) and three types of variable errors (measured by variance) according to X, Y, and Z coordinates.

3 Results

3.1 Analysis of Variance

Analysis of variance was first performed to test the effects of independent variables on Time. As shown in Table 2, the main effect of Distance (F5, 25, p < 0.001) and the interaction effect of Direction and Distance (F25, 125, p < 0.05) were significant on Time. These effects were analyzed with the ballistic movement time model (Eq. 3) and are shown in the next section.

Table 2. Effects of independent variables on ballistic movement
Full size table

Analysis of variance was then performed to test the effects of independent variables on six types of endpoint errors, comprising Constant-X error, Constant-Y error, Constant-Z error, Variable-X error, Variable-Y error, and Variable-Z error. Regarding constant errors, as shown in Table 2, the main effect of Direction (F5, 25, p < 0.001) and the interaction effect of Direction and Distance (F25, 125, p < 0.001) were significant on three types of constant errors. As shown in Fig. 2, three types of constant errors differed among six movement directions. These errors were greater when movements were performed in anterior, left, and right directions. There was a trend that constant errors increased with increased movement distance, especially for Constant-X error. This explains the significant main effect of Distance on Constant-X error (F5, 25, p < 0.05). However, the rate of increase of constant errors differed among different movement directions. Furthermore, there was an interaction effect of Scene and Direction (F25, 125, p < 0.05) on Constant-Y error. As shown in Fig. 3, Constant-Y error varied according to different movement direction and these rates of difference were different in different scenes with depth-cue combinations. Regarding variable errors, the main effects of Direction (F5, 25, p < 0.001) and Distance (F5, 25, p < 0.001) and the interaction effect of Direction and Distance (F25, 125, p < 0.001) were significant on three types of variable errors. These effects were analyzed with the ballistic movement variability model (Eq. 4) and are shown in the next section.

Fig. 3.
figure 3

The interaction effect of movement direction and scene on Constant-Y error

Full size image

Since there were the main effect of Distance and certain related interaction effects of Distance on Time and three types of variable errors, in the next section, the two ballistic movement models were applied to show these effects.

3.2 Results Obtained by Applying the Two Ballistic Movement Models

The means of movement time of six movement direction were regressed on to the square root of movement distance to give the slopes, intercepts, and \( R^{2} \) values. The model fitted the data very well. It accounted for 97.8 % variance on average and at least 97.2 % variance of movement time. The regression lines of six movement directions are shown in Fig. 4, which also shows good model fittings. As shown in the figure, the movement time increased with the increased squared root of movement distance. The times required to perform the movement in the directions of posterior and superior were relatively shorter than that in the direction of the left. However, these rates of increase of movement time were different in different movement directions.

Fig. 4.
figure 4

Fittings of the ballistic movement time model for six movement directions

Full size image

Three types of variable errors measured in three directions were regressed on the distance to give the slopes, intercepts, and \( R^{2} \) values for each direction of movement. For Variance-X error, the ballistic movement model accounted for 98.17 % variance on average and at least 94.5 % of the data. For Variance-Y error, the ballistic movement model accounted for 98.27 % variance on average and at least 96.7 % of the data. For Variance-Z error, the ballistic movement model accounted for 99.17 % variance on average and at least 97.7 % of the data. The regression lines of three types of variable error are shown in Fig. 5, which also shows good model fittings. As shown in the figure, three types of variable errors increased with increased movement distance. However, these rates of increase of variable errors were different in different movement directions. The ranking of three types of variable errors differed among movement directions. However, Variable-X error was relatively less than Variable-Y and Variable Z errors. Furthermore, variable errors were relatively less when movements performed in left-right directions.

Fig. 5.
figure 5

Fittings of the ballistic movement variability model of three types of endpoint variability for six movement directions

Full size image

4 Discussion

This study attempted to use the ballistic movement method to assess combinations of depth cues on hand-control movements in a virtual environment. Surprisingly, there was no significant effect of six depth-cue combinations on ballistic movement properties, comprising movement time, constant errors, and variable errors (Table 2). It is expected that depth cues should affect hand-control movements. Hence, we hypothesize that the effect of depth cues might be on the corrective reaction time [15]. The corrective reaction time represents the time required to process visual information for executing a sub-movement to modify an ongoing hand control movement. An evidence was found by Mather and Smith [16] who found that the effect of depth cues was on reaction time.

This study showed that the ballistic movement method is helpful for assessing the effect of movement directions. By applying the ballistic movement method that proposed by Lin and Tsai [2], we were be able to assess how movement direction affect motor properties of ballistic movement time, movement endpoint constant errors and variable errors. The ballistic movement time model and the ballistic movement variability model predicted well the data (Figs. 4 and 5), which allow practitioners to measure and apply the human capabilities and limitations while executing hand control movements in a virtual environment.

This pilot study showed some interesting findings, but with certain limitations, suggesting modifications of experimental designs for future research. First, the ballistic movement method has potentials for assessing the effects of movement direction on hand-control movements. To provide sound references of the human capabilities and limitations, to increase the number of participants is necessary. Moreover, in this study, constant errors and variable errors were measured in the global coordinate system, which allows us to study how these errors differ among the six movement direction. However, because these errors differ if they are measured in the movement direction or measured perpendicular to the movement [2, 13], it is necessary to analyze these errors in the local coordinate system. Second, only three depth cues, comprising background gradient, shadows, and drop line, were tested in this study. More depth cues should be tested in future research. Finally, one surprising finding arises a research question for future research. That is, the effect of depth cue should be on the corrective reaction time, but not on ballistic movement time, constant errors, and variable errors. This hypothesis needs further investigation with appropriate measurement of the time before executing ballistic movements.

5 Conclusions

To test the application of the ballistic movement method for assessing the effects of depth cues on hand-control movements in a virtual environment, this study recruited six participants to perform ballistic movements in six scene settings manipulated with six depth-cue combinations of background gradient, shadows, and drop line. Surprisingly, there was no significant effect of six combinations of depth cues on ballistic movement properties, comprising movement time, constant errors, and variable errors. However, the results showed that the ballistic movement method is helpful for assessing the effect of movement directions. We suggested that the effects of depth cues should be on the corrective reaction time, but may not on ballistic movement time and ballistic movement endpoint variability. This hypothesis needs further investigation by measuring the time before executing ballistic movements.