Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Manipulation of three dimensional (3D) data includes rotation, translation and scaling (RTS) tasks. Positioning of a virtual object can be subdivided into an initial selection, a coarse, large movement and a final, precise movement [19]. The efficiency and adequacy of a manipulation technique depends strongly on the task [39]. It is influenced by factors like distance from user to object, size of the object, needed amount of RTS and density of objects. Furthermore the input and output devices influence the choice of a possible interaction technique. Therefore this paper aims to give an overview of manipulation techniques grouped by the according input modality. In Sect. 2 some general information for 3D manipulation is established. Section 3 introduces several interaction techniques. Available evaluation results are presented in Sect. 4 and further discussed in Sect. 5. Finally a conclusion is drawn in Sect. 6.

2 Related Work

When interacting with an object it is not advantageous to allow the user to manipulate all DOF of the object at the same time. Users prefer a constrained interaction namely a 2D translation on a plane, a 1D rotation around an axis and an uniform 3D scaling [8]. Rotation is preferred as a 1D task because users cannot deconstruct an orientation into distinct rotations around several axes [59]. Rotation is also a more complex task than translation, as Ware [52] shows it takes about 50% more time. Martinet et al. [31] show that a separated translation and rotation increase the efficiency of users.

The introduction of constraints in a 3D interaction can increase precision and speed if they fit the task [47]. Besides this, a 3D or stereo display helps position and orient objects in a virtual environment (VE) and users perform better than with a classical 2D representation [52].

3 Manipulation Techniques

This section covers different techniques for RTS interaction grouped by the input modality. It covers approaches using a classical 2DOF mouse, an extended mouse with more than 2DOF, a full 6DOF device, a gesture and touch interaction.

3.1 2DOF Mouse Interaction

A possibility to control an object’s placement is the use of different sliders which control the dimensions of the movement [8]. However, a mouse interaction with virtual objects often utilizes 3D widgets [9, 45]. Typically a partition in different modes for RTS and constraints for 1D or 2D manipulations are presented to the user. Virtual Sphere Methods [8, 21, 46] let the user rotate an object by clicking & dragging a fictitious sphere that encapsulates the object. Schmidt et al. [45] do not switch between different interaction modes with buttons or shortcuts but rather use sketching gestures to derive manipulation modes accordingly. Techniques like Snapping [9], the Triad Mouse [35] or Snap-Dragging [5, 6] place reference points or cursors on objects in the VE which allow the user to perform several transformations on the object. Tail-Dragging [50] links the object to an virtual rope which can be used to drag an object around the VE while simultaneously performing translation and rotation.

3.2 2+DOF Mouse Interaction

Several works exist that increase the DOF of the classical mouse by one or two dimensions in order to perform more complex tasks. Zeleznik et al. [56] take two mice which are controlled by both hands of the user. They split the functionality according to the theory of the dominant and non-dominant hand from Guiard [13]. A bi-manual asymmetric interaction results in an increased performance [3, 23]. Another approach is to increase the dimensions of a single mouse as with the Two-Ball Mouse [30], the Rockin’Mouse [4], the Yawing Mouse [2] or the Turntable [12]. These let the user rotate or tilt the mouse for additional input.

3.3 6DOF Mouse Interaction

6DOF controllers consist of isotonic, free moving devices and isometric or elastic, (almost) fixed devices [59]. An isotonic mouse like the VideoMouse [24], the ToolStone [42] or the Bat [53] uses a tracking method that allows the detection of the position and orientation of the device in 6D. Examples for isometric and elastic devices are the SpaceMouse [1] or the Elastic General-purpose Grip (EGG) [60] respectively. The controller for these type of devices moves only a small amount. It measures input based on exerted forces or deviation from a zero position. Isotonic devices benefit from controlling the position of a virtual object directly (zero order control), whereas isometric and elastic devices should control the velocity of the object (first order control) [57]. An isotonic device with first order control and a isometric or elastic device with zero order control result in a significantly slower object manipulation. There is no significant difference between a isometric or elastic device control, except for a slight advantage for elastic controllers in the first 20 min of usage [58]. 6DOF devices may also introduce force feedback to the user with the use of a mechanical arm (e.g. Geomagic Touch X [11] or Haption Virtuose 6D [20]).

3.4 Gesture Interaction

Gesture interaction can utilize a hand and finger tracking or use a physical controller like an isotonic 6DOF mouse in combination with physical buttons on the controller. Using gestures rather than mapping the movement of the hand directly to the virtual object as in Sect. 3.3 allows for different interactions, but also introduces some problems. In the Grab and Twirl technique [10] and a technique from Schlattmann and Klein [43] the user frames a virtual object with both of his hands. The object is hereby grabbed and can be moved in the VE. In addition to that, the Grab and Scale [10] method lets the user set an axis and the amount of rotation with the dominant hand. Jerky Release [44] lets the user grab an object with an implicit grab gesture. The object is manipulated if the user moves his hand steady and controlled. The object is released if he makes some fast or jerky movements. Modeless RTS including constraints can be achieved using two isotonic controllers [16] or the users hands [48].

Focusing on Virtual Reality (VR), new object manipulation techniques need to be introduced because classical mouse or keyboard inputs may not be available or suitable. An object can be manipulated relative to the origin of an object’s coordinate system or the users hands [33]. Moving an object relative to its own center results in a direct manipulation. This is intuitive, fast, precise and utilizes proprioception [7, 19, 34]. Relative movement with respect to the users hand feels more like manipulating the object on the end of a rod and is more prone to tracking noise or a shaking hand. To remove the restriction of a limited interaction range for a direct manipulation, the techniques Go-Go [40], Fast Go-Go and Stretch Go-Go [19] can be used. With this techniques the arm extension of the user is mapped to an disproportionately high movement of the virtual hand. Scaled-World Grab [34] scales the users size in the world to allow him to reach distant objects, whereas Extender Grab moves the object without positioning it in the users hands, but scales the objects translation depending on the initial distance to the user. Other manipulation techniques are Worlds in Miniature (WIM) [33, 49], Hand-Held Widgets [34], Voodoo Dolls [37], a hybrid technique by de Haan et al. [15] which combines direct and ray-casting interaction or several projection techniques by Pierce et al. [38].

3.5 Touch Interaction

There are several techniques for a touch screen interaction like Sticky Tools [18], a fluid-based Manipulation by Kruger et al. [27], Screen-Space [41], Depth-Separated Screen-Space (DS3) [31], a Two-Finger interaction by Liu et al. [29], Pie Rotate and Turn & Roll [22], Z-technique and multi-touch viewport technique [32] and rizzo [51]. Hancock et al. [17] show that with one, two or three finger input in a 5DOF manipulation task it is easier and faster to use more fingers for interacting. Mobile devices contain a touch screen and also a gyroscope which can be combined to form device orientation dependent interaction as in [14, 55].

4 Evaluation

Chen et al. [8] compare slider-based rotation with the virtual sphere method and find that virtual sphere is faster and better rated by users (also [26, 36]). They compare the virtual sphere with a isometric trackball and find no significant differences in neither time nor accuracy. Several works compare a classical mouse with a 2+DOF mouse interaction and find that the mouse with more DOF is about 10–30% faster [2, 4]. Still, the given tasks seem to fit the DOF of the input devices exactly and it is questionable if these devices perform as well given a 6DOF task. Comparing isotonic 6DOF mice interaction with a virtual sphere interaction shows that the 6DOF devices are significantly faster \((\tilde{3}5\%)\) in a rotation task, with equal or slightly better precision and a higher user rating [25]. Ware and Rose [54] show that when using an isotonic device, an object rotation should be done with the object in the same position as the hand of the user. A comparison of an isotonic 6DOF mouse and an isotonic tracked hand with and without an explicit grabbing posture shows that the explicit grabbing is significantly slower and more imprecise [44]. Still, all techniques are accurate. A further analysis gives a similar result, but also shows that a bi-manual method can also be fast at precise movements [43]. High standard deviations for all methods show that the experience of the user with the devices is an important factor for the interaction. Above all an isometric device needs more time to get used to. Zhai et al. [61] compare an isotonic input from a tracked device and a glove. The main difference of the two techniques is the limited movement range of the hand. Since the tracked device could be manipulated with the fingers it did not have that limitation and as a result is about 20% faster. In another experiment Zhai and Milgram [60] compare an isotonic and an elastic control. The isotonic interaction is significantly faster, but the elastic interaction allows for a more controlled object manipulation (discrepancy from the shortest manipulation path). Both devices show a strong learning effect. As of [57, 58], there is no significant difference between an isometric and an elastic device concerning speed or user preference.

Comparisons of the touch screen input techniques Sticky Tools, Screen-Space and DS3 show that users have significantly more problems completing a task with Screen-Space while using twice as many touch events [31]. DS3 is about 35% faster in the given experiment. Liu et al. [29] compare these three techniques to their own Two-Finger method, but also consider different screen sizes in their experiment (5 and 11 in.). The Two-Finger method and Sticky Tools are the fastest techniques on all display sizes. Screen-Space performs better on a smaller screen, but is still slow at complex tasks. The technique DS3 is the slowest at executing a simple task and using a 5 in. screen.

5 Discussion

Using a classical mouse with a virtual Sphere technique is common, but inferior to a 6DOF device [59]. 2+DOF devices can be faster than a mouse, but suffer from the fact that other muscle groups are used to compare the other dimensions. On one hand, an isotonic 6DOF interaction is intuitive, but has a limited workspace and therefore needs a clutching method. Also fatigue can be a problem, but does not need to be, because the user might rest his arm on a table and perform movements with his wrist [53]. On the other hand, an isometric or elastic device needs little effort to control and allows for more coordination. However these devices need more time to learn. Zhai and Milgrim [60] conclude that a more direct controller, like an isotonic device or a touch screen, should be used for short tasks and less direct tools, like an isometric/elastic device, should be used for long interactions. Table 1 gives an overview of all mentioned manipulation techniques and their rating, based on the evaluation results.

Table 1. Overview: interaction techniques classified by domain and rated based on evaluation results. Rating from \({-}{-}\) (worst) to o (neutral) to \(++\) (best). Abbreviations: dm = discrete modes, ml = modeless, uni-m. = uni-manual, bi-m. = bi-manual, R = rotation, T = translation, S = scaling, Beginner = beginner-friendliness, Mental = mental demand, Physical = physical demand
Full size table

6 Conclusion

In a 3D object manipulation task the best technique depends on the input and output devices and the application domain. Firstly, desktop environments can combine a mouse interaction with a widget based manipulation like the virtual sphere. This works well and since a mouse is always available it is easy to integrate this type of control. Many different solutions try to extend the 2D mouse with one or two more axes, but these additional dimensions only benefit in specific systems that utilize exactly these axes. A 6DOF manipulation, that is translation and rotation, favors from the introduction of a 6DOF mouse. Scaling might be achieved with a mode switch.

Secondly, in a mobile context or on a larger touch screen techniques like Sticky Fingers or the Two-Finger approach by Liu et al. allow efficient and fast interactions. The orientation sensors of a smart phone can also be used to achieve a different interaction style like in [28].

Thirdly, VR and AR may not or do not want to use traditional input methods, but can rely on hand or controller input. This enables the user to interact directly and naturally with objects in the virtual environment (VE) by using a zero order control [19]. Techniques like Go-Go, Scaled-World Grab and Extender Grab or Worlds in Miniature extend on this direct metaphor.