Comparing Robot Grasping Teleoperation Across Desktop and Virtual Reality with ROS Reality

Abstract

Teleoperation allows a human to remotely operate a robot to perform complex and potentially dangerous tasks such as defusing a bomb, repairing a nuclear reactor, or maintaining the exterior of a space station. Existing teleoperation approaches generally rely on computer monitors to display sensor data and joysticks or keyboards to actuate the robot. These approaches use 2D interfaces to view and interact with a 3D world, which can make using them difficult for complex or delicate tasks. To address this problem, we introduce a virtual reality interface that allows users to remotely teleoperate a physical robot in real-time. Our interface allows users to control their point of view in the scene using virtual reality, increasing situational awareness (especially of object contact), and to directly move the robot’s end effector by moving a hand controller in 3D space, enabling fine-grained dexterous control. We evaluated our interface on a cup-stacking manipulation task with 18 participants, comparing the relative effectiveness of a keyboard and mouse interface, virtual reality camera control, and positional hand tracking. Our system reduces task completion time from 153 s ($\pm 44$) to 53 s ($\pm 37$), a reduction of 66%, while improving subjective assessments of system usability and workload. Additionally, we have shown the effectiveness of our system over long distances, successfully completing a cup stacking task from over 40 miles away. Our paper contributes a quantitative assessment of robot grasping teleoperation across desktop and virtual reality interfaces.