Human-embodied drone interface for aerial manipulation: advantages and challenges

Abstract

Drones have performed various tasks, such as surveillance, photography, agriculture, and package delivery. However, these tasks typically involve drones simply observing or capturing information from their surroundings without physically interacting with them. Aerial manipulation shifts this paradigm and implements drones with robotic arms that allow interaction with the environment rather than simply touching it. For example, in construction, aerial manipulation in conjunction with human interaction could allow operators to perform several tasks, such as hosing decks, drilling into surfaces, and sealing cracks via a drone. For over a decade, researchers have been working on aerial manipulation for industrial applications. These works are valuable to aerial manipulation but have not been widespread in the public domain yet. This is because most of the works are conducted in controlled indoor environments (e.g., motion capture systems), and the knowledge gap exists between researchers and the wider public who are interested in deploying aerial manipulation for practical tasks. To fill this gap, our recent work integrated the worker’s experience into aerial manipulation using haptic technology. The net effect is that such a human-in-the-loop system could enable workers to leverage their experience to complete manipulation tasks while remotely controlling a mobile manipulating drone on the task site. The system increased the feasibility and adaptiveness of aerial manipulation. The remaining challenges are completing tasks beyond the operator’s line-of-sight and lack of dexterity. To address the challenges, we present a human-embodied drone interface in this article. The interface consists of immersive virtual/augmented reality and haptic technologies. Such an interface allows the drones to embody and transport the operator’s senses, actions, and presence to a remote location in real-time. Therefore, the operator can both physically interact with the environment and socially interact with actual workers on the worksite. Two different human-embodied interfaces are developed and tested with several tasks suggested by the United States Department-of-Transportation: pick-and-place, drilling, peg-in-hole, and key insert/rotation. The conclusion describes the advantages and challenges of the interface with future works.