Comparing Robot Grasping Teleoperation Across Desktop and Virtual Reality with ROS Reality
Teleoperation allows a human to remotely operate a robot to perform complex and potentially dangerous tasks such as defusing a bomb, repairing a nuclear reactor, or maintaining the exterior of a space station. Existing teleoperation approaches generally rely on computer monitors to display sensor data and joysticks or keyboards to actuate the robot. These approaches use 2D interfaces to view and interact with a 3D world, which can make using them difficult for complex or delicate tasks. To address this problem, we introduce a virtual reality interface that allows users to remotely teleoperate a physical robot in real-time. Our interface allows users to control their point of view in the scene using virtual reality, increasing situational awareness (especially of object contact), and to directly move the robot’s end effector by moving a hand controller in 3D space, enabling fine-grained dexterous control. We evaluated our interface on a cup-stacking manipulation task with 18 participants, comparing the relative effectiveness of a keyboard and mouse interface, virtual reality camera control, and positional hand tracking. Our system reduces task completion time from 153 s (\(\pm 44\)) to 53 s (\(\pm 37\)), a reduction of 66%, while improving subjective assessments of system usability and workload. Additionally, we have shown the effectiveness of our system over long distances, successfully completing a cup stacking task from over 40 miles away. Our paper contributes a quantitative assessment of robot grasping teleoperation across desktop and virtual reality interfaces.
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Grants No. W911NF-15-1-0503, YFA: D15AP00104, YFA: GR5245014, and D15AP00102, as well as NASA under Grants No. GR5227035.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of DARPA.
- 1.Interactive Manipulation IM Description. http://projects.csail.mit.edu/pr2/wiki/index.php?title=Interactive_Manipulation
- 2.IVRE - An Immersive Virtual Robotics Environment. https://cirl.lcsr.jhu.edu/research/human-machine-collaborative-systems/ivre/
- 3.Mind Meld. https://www.youtube.com/watch?v=kZlg0QvKkQQ
- 4.PR2 Surrogate. http://wiki.ros.org/pr2_surrogate
- 6.Brooke, J., et al.: Sus-a quick and dirty usability scale. Usability Evaluation in Industry, vol. 189(194), pp. 4–7 (1996)Google Scholar
- 7.Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., Dollar, A.M.: The ycb object and model set: towards common benchmarks for manipulation research. In: 2015 International Conference on Advanced Robotics (ICAR), pp. 510–517. IEEE (2015)Google Scholar
- 9.N. H. P. R. Group et al. Task load index (nasa-tlx) v1. 0 computerised version. NASA Ames Research Centre (1987)Google Scholar
- 11.Koenig, N., Howard, A.: Design and use paradigms for gazebo, an open-source multi-robot simulator. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004. (IROS 2004). Proceedings, vol. 3, pp. 2149–2154. IEEE (2004)Google Scholar
- 13.Moroney, W.F., Biers, D.W., Eggemeier, F.T., Mitchell, J.A.: A comparison of two scoring procedures with the nasa task load index in a simulated flight task. In: Proceedings of the IEEE 1992 National Aerospace and Electronics Conference, 1992. NAECON 1992, pp. 734–740. IEEE (1992)Google Scholar
- 15.Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3, p. 5. Kobe (2009)Google Scholar
- 16.Serrano, D.: Introduction to ros–robot operating systemGoogle Scholar