Human Avatar Robotic Puppeteering (HARP)

  • Christopher Martinez
  • Cameron MacDonaldEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 528)


The Human Avatar Robotic Puppeteering (HARP) project is focused on studying whether homologous puppeteering (controlling via mimicry) is an effective control principle for robots, given minimal user training. This project aims to develop a practical implementation at low-cost. The HARP project is a three-joint robotic crane that is capable of grasping objects. In order to control via puppeteering, the system tracks the user’s hand moving in free space in real time as an avatar. This implementation relies on a Microsoft Kinect ($150) and a Leap Motion sensor ($100). This low-cost prototype is a proof-of-concept for a natural interface between user and robot, allowing gestures to be the method of communication rather than the traditional button-and-switch method. The system uses the Xbox Kinect to track the hand in reference to a known point on a table. This position is mapped using the imaging camera sensor in the Kinect, and an inverse kinematic algorithm is used to translate that position to the joint-angles for the crane. The grasping of the user’s hand is sensed with the Leap Motion sensor. A key contribution to the research field is the blending of two different gesture based sensor systems to form a robust control interface.


Robotic control Avatar control User interface 


  1. 1.
    Gallo, L., Placitelli, A.P., Ciampi, M.: Controller-free exploration of medical image data: Experiencing the Kinect. In: CBMS, 2011 Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems, pp. 1–6. doi: 10.1109/CBMS.2011.5999138
  2. 2.
    Gentile, V., Sorce, S., Gentile, A.: Continuous hand openness detection using a Kinect-like device. In: 2014 Eighth International Conference on Complex, Intelligent and Software Intensive Systems (CISIS), pp. 553–557 (2014). doi: 10.1109/CISIS.2014.80
  3. 3.
    Raheja, J., Chaudhary, A., Singal, K.: Tracking of fingertips and centres of palm using KINECT. In: 2011 Third International Conference on Computational Intelligence Modeling & Simulation, pp. 248–252 (2011)Google Scholar
  4. 4.

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.University of New HavenWest HavenUSA

Personalised recommendations