In this section, first we report the analysis of our quantitative data and then report the participant comments in the post-experiment interview about the interfaces as a qualitative approach. To analyze our quantitative data, we performed 1-way within-subject two-tailed ANOVA on both our behavioral and introspective measures. As for the behavioral measures, we analyzed the participant’s position data to calculate their accuracy in terms of both distance and angular errors, and as for the introspective data, we analyzed their answers to 15 questions about each interface, which were a continuous rate between 0 ~ 100%.
4.1 Behavioral Results
As for the behavioral measures, considering both distance and angular error data were skewed negatively (toward zero), we converted them to logarithmic scales, and Shapiro Wilkes test showed the normality of the scaled data.
Accuracy of Each Interface.
Mauchly’s test showed the violation of sphericity assumption for both overall distance and angular error, so we performed the Greenhouse-Geisser correction of ANOVA, which showed no significant difference in terms of angular error. However the Greenhouse-Geisser correction of ANOVA showed that the interface has a significant effect on distance error (F(1.55, 20.2) = 5.53, p = .017, ƞ
2
p
= .298). Tukey HSD post hoc showed that NaviChair (M = 2.26 m, SD = 1.59 m) had higher distance error than both Joystick (M = 1.76 m, SD = 2.01 m) and Real-Rotation (M = 1.43 m, SD = 1.39 m) (all ps < .010) (Fig. 4).
Accuracy of Each Trial (Learning Effect).
Beside interfaces, which affect the average distance error of participants, we also compared the overall distance error of participants for each trial. Considering the Mauchly’s test showed violation of the sphericity assumption, we performed the Greenhouse-Geisser correction of ANOVA, which showed a significant effect of trials over distance error (F(1.88, 24.4) = 3.73, p = .040, ƞ
2
p
= .223). Tukey HSD post hoc showed that the average distance error of first trial (M = 2.14 m, SD = 2.17 m) had higher distance error than the third trial (M = 1.66 m, SD = 1.61 m) (p = .030), which shows a learning effect. But there were no significant effect for second trial (M = 1.89 m, SD = 1.85 m) (Fig. 5).
4.2 Introspective Results
Shapiro Wilkes test showed that the introspective data were normal or had small violation of normality, which ANOVA is robust enough for the small normality violations. The Mauchly’s test showed the violation of the sphericity assumption for all the measured variables, so we performed the Greenhouse-Geisser correction of ANOVA for all the variables, and all the significant effects have been mentioned at below sections (p < .05). All other results were not significant including immersion, enjoyment, learnability, and overall preference.
Motion Sickness.
The results showed no significant effect of interfaces on motion sickness (F(1.34, 17.7) = .170, p = .750). However, swivel-chair had the lowest average of motion sickness (M = 16.1%, SD = 18.5%), which is followed by Real-Rotation (M = 19.3%, SD = 20.8%) and then NaviChair (M = 21.4%, SD = 19.3%), and joystick had the highest average of motion sickness (M = 24.4%, SD = 32.1%) (Fig. 6-left).
Vection Intensity.
The results showed no significant effect of interfaces on the vection intensity (F(2.36, 30.6) = 3.09, p = .052). However, NaviChair had the highest average of vection intensity (M = 77.9%, SD = 18.2%), which if followed by Swivel-Chair (M = 73.4%, SD = 22.4%), and then Joystick (M = 64.6%, SD = 23.6%), and Real-Rotation had the lowest average of vection intensity (M = 61.8%, SD = 23.3%) (Fig. 6-right).
Spatial Presence.
The results showed no significant effect of interfaces on the spatial presence (being there physically) (F(2.12, 29.6) = 2.15, p = .132).However, NaviChair had the highest average of spatial presence (M = 69.1%, SD = 21.7%), which is followed by Swivel-Chair (M = 65.6%, SD = 20.5%), and then Real-Rotation (M = 57.9%, SD = 17.5%), and joystick had the lowest average of presence (M = 54.6%, SD = 21%) (Fig. 7-left).
Intuitiveness.
The results showed no significant effect of interfaces on the intuitiveness (F(2.12, 27.6) = .292, p = .761). However, there were slight differences in terms of intuitiveness average. Swivel-Chair had highest average of intuitiveness (M = 65.4%, SD = 24.3%), which is followed by Real-Rotation (M = 60.8%, SD = 26.1%), and then NaviChair (M = 59.9%, SD = 26.8%), and Joystick had the lowest average of intuitiveness (M = 58.6%, SD = 27.3%) (Fig. 7-right).
Precise Control.
The results showed that the interface had a significant effect on the precise control (F(2.55, 33.2) = 9.01, p = .0003,ƞ
2
p
= .409). Tukey HSD post-hoc showed that joystick (M = 75, SD = 23.9) had significantly higher precise control than both Swivel-Chair (M = 49.8%, SD = 29.9%) and NaviChair (M = 32.3%, SD = 18.7%) (all ps < .021). Moreover, Real-Rotation (M = 54.8%, SD = 20.7) had significant higher precise control than NaviChair (p = .046) (Fig. 8-left).
Ease of Use.
The results showed no significant effect of interfaces on the ease of use (F(2.15, 28.0) = 2.09, p = .139). However, Joystick had the highest ease of use mean (M = 66.6%, SD = 36.7%), which is followed by Real-Rotation (M = 58.1%, SD = 27.4%), and then Swivel-Chair (M = 49.9%, SD = 27.0%), and NaviChair had the lowest ease of use (M = 42.2%, SD = 28.0%) (Fig. 8-right).
Comfort.
The results showed that the interface has a significant effect on the comfort (F(1.37, 17.9) = 4.08, p = .048, ƞ
2
p
= .239). Tukey HSD post-hoc showed that joystick (M = 71.4%, SD = 32.3%) had higher comfort than NaviChair (M = 43.3%, SD = 28.9%) (p = .010). There were no other significant effect on comfort for other interfaces including Real-Rotation (M = 65.1%, SD = 28.0%) and Swivel-Chair (M = 58.9%, SD = 23.1%) (Fig. 9-left).
Long Time Use (i.e., Longevity).
The results showed that the interface had a significant effect on the longevity (F(1.74, 22.6) = 5.05, p = .019, ƞ
2
p
= .280). Tukey HSD post-hoc showed that participants could imagine using joystick (M = 62.8%, SD = 29.6%) for longer periods of time than NaviChair (M = 28.6%, SD = 22.6%) (p = .002). There were no other significant effect on longevity for other interfaces including Real-Rotation (M = 45.6%, SD = 25.2%) and Swivel-Chair (M = 42.9%, SD = 19.4%) (Fig. 9-right).
Problems Using the Interface.
The results showed that the interface has a significant effect on the reported problems (F(2.31, 30.1) = 5.00, p = .010, ƞ
2
p
= .278). Tukey HSD post-hoc showed that participants had significantly less problems using joystick (M = 21.4%, SD = 19.1%) rather than NaviChair (M = 54.6%, SD = 23.6%) (p = .002). There were no other significant effect on problems using interfaces for other interfaces including Real-Rotation (M = 38.6%, SD = 27.7%) and Swivel-Chair (M = 42.3%, SD = 28%) (Fig. 10-left).
Overall Usability.
The results showed that the interface has a significant effect on the overall usability (F(2.29, 29.8) = 4.11, p = .020, ƞ
2
p
= .240). Tukey HSD post-hoc showed that joystick (M = 76.3%, SD = 15.9%) had higher overall usability than NaviChair (M = 52.6%, SD = 23.1%) (p = .006). There were no other significant effect on overall usability for other interfaces including Real-Rotation (M = 63.7%, SD = 21.5%) and Swivel-Chair (M = 65.1%, SD = 15.8%) (Fig. 10-right).
4.3 Qualitative Results
After finishing the experiment, we interviewed each participant and ask them which interfaces they enjoyed and why? We also asked about the problems they had with any of the interfaces. These qualitative data will be reported in this section, to shed a light on strengths and weaknesses of each interface. This feedbacks also can be useful for future improvement of the interfaces.
Joystick Feedbacks.
Two (out of 14) participants enjoyed joystick over the other interfaces, because they were used to it or it was comfortable for them. However, as for the problems of joystick, some participants mentioned that joystick locomotion was not natural, and two of them mentioned that they also rotated their neck instead of rotating joystick. One participant suggested to use game controllers instead of joystick, because he were more familiar with the game controllers.
Real-Rotation Feedbacks.
Five (out of 14) participants enjoyed Real-Rotation over the other interfaces, because physical rotation helped them to be immersed in the VR, but also they could precisely control their speed with joystick. However, as for the problems of Real-Rotation, four participants mentioned that it was not easy to control two interfaces (i.e., chair for rotation and joystick for translation) simultaneously, and they were disconnected from the game.
Swivel-Chair Feedbacks.
Five (out of 14) participants enjoyed Swivel-Chair over the other interfaces, because it was a comfortable chair and intuitive. Four participants also mentioned that it was highly intuitive and natural, which made them highly engaged into the game. However, as for the problems of Swivel-Chair, five participants mentioned that it was not easy for them to control the speed by leaning back, so they could not control their speed precisely.
NaviChair Feedbacks.
Two (out of 14) participants enjoyed NaviChair over the other interfaces, because it was comfortable for them, so engaged and immersed them into the game. However, as for the problems of NaviChair, eight participants mentioned that it was hard for them to control it accurately. One of them mentioned that it was hard to focus on the bottom of their body to navigate, another participant mentioned that due to her back problem. One participant mentioned that it was hard to stop NaviChair, and another one mentioned that it was too sensitive. Two participants mentioned that it was too loose and jumpy. Because NaviChair was over the Wii Balance Board, even its lowest height was too high for some participants, so three participants mentioned that the height of NaviChair was too high for them. One participant mentioned that NaviChair is an interesting interface for free exploration of VE but not for accurate maneuvering purposes.