Evaluation of an Inverse-Kinematics Depth-Sensing Controller for Operation of a Simulated Robotic Arm

  • Akhilesh Kumar Mishra
  • Lourdes Peña-Castillo
  • Oscar Meruvia-PastorEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9748)


Interaction using depth-sensing cameras has many applications in computer vision and spatial manipulation tasks. We present a user study that compares a short-range depth-sensing camera-based controller with an inverse-kinematics keyboard controller and a forward-kinematics joystick controller for two placement tasks. The study investigated ease of use, user performance and user preferences. Task completion times were recorded and insights on the measured and perceived advantages and disadvantages of these three alternative controllers from the perspective of user efficiency and satisfaction were obtained. The results indicate that users performed equally well using the depth-sensing camera and the keyboard controllers. User performance was significantly better with these two approaches than with the joystick controller, the reference method used in comparable commercial simulators. Most participants found that the depth-sensing camera controller was easy to use and intuitive, but some expressed discomfort stemming from the pose required for interaction with the controller.


Gesture-based controllers for robot arm manipulation Depth-sensing cameras and short-range RGBD sensors Inverse and forward kinematics User studies 


  1. 1.
    Lun, R., Zhao, W.: A Survey of Applications and Human Motion Recognition with Microsoft Kinect. Int. J. Pattern Recogn. Artif. Intell. (2015) Google Scholar
  2. 2.
    Corradini, A., Gross, H.: Camera-based gesture recognition for robot control. In: 2000 Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000 (2000). doi: 10.1109/IJCNN.2000.860762
  3. 3.
    A. Anonymous: Real-time 3D pointing gesture with kinect for object-based navigation by the visually impaired. Presented at Biosignals and Biorobotics Conference (BRC), 2013 ISSNIP (2013). doi: 10.1109/BRC.2013.6487535
  4. 4.
    Suay, H.B., Chernova, S.: Humanoid robot control using depth camera. In: 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (2011)Google Scholar
  5. 5.
  6. 6.
    Leap Motion Controller, 10 June 2015.
  7. 7.
    Microsoft Kinect for Windows V2, 25 June 2015.
  8. 8.
    Intel Real Sense SDK, 16 June 2015.
  9. 9.
    Project Tango, ATAP, 25 June 2015.
  10. 10.
    Whitney, D.E.: Quasi-static assembly of compliantly supported rigid parts. J. Dyn. Syst. Measur. Control 104(1), 65–77 (1982)CrossRefzbMATHGoogle Scholar
  11. 11.
    Sian, N.E., Yokoi, K., Kajita, S., Kanehiro, F., Tanie, K.: Whole body teleoperation of a humanoid robot - development of a simple master device using joysticks. In: 2002 IEEE/RSJ International Conference on Intelligent Robots and Systems (2002). doi: 10.1109/IRDS.2002.1041657
  12. 12.
    Manikandan, R., Arulmozhiyal, R.: Position control of DC servo drive using fuzzy logic controller. In: 2014 International Conference on Advances in Electrical Engineering (ICAEE) (2014). doi: 10.1109/ICAEE.2014.6838474
  13. 13.
    Jan, V., Marek, S., Pavol, M., Vladimir, V., Stephen, D.J., Roy, P.: Near-time-optimal position control of an actuator with PMSM. In: 2005 European Conference on Power Electronics and Applications (2005). doi: 10.1109/EPE.2005.219516
  14. 14.
    Lu, Y., Huang, Q., Li, M., Jiang, X., Keerio, M.: A friendly and human-based teleoperation system for humanoid robot using joystick. In: 2008 7th World Congress on Intelligent Control and Automation, WCICA 2008 (2008). doi: 10.1109/WCICA.2008.4593278
  15. 15.
    Ng-Thow-Hing, V., Luo, P., Okita, S.: Synchronized gesture and speech production for humanoid robots. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2010)Google Scholar
  16. 16.
    House, B., Malkin, J., Bilmes, J.: The VoiceBot: a voice controlled robot arm. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2009). doi: 10.1145/1518701.1518731
  17. 17.
    Oniga, S., Tisan, A., Mic, D., Buchman, A., Vida-Ratiu, A.: Hand postures recognition system using artificial neural networks implemented in FPGA. In: 30th International Spring Seminar on Electronics Technology (2007). doi: 10.1109/ISSE.2007.4432909
  18. 18.
    Chen, C.X., Trivedi, M.M., Bidlack, C.R.: Simulation and animation of sensor-driven robots. IEEE Trans. Robot. Autom. 10(5), 684–704 (1994). doi: 10.1109/70.326572 CrossRefGoogle Scholar
  19. 19.
    (18/06/2015). VROV Manipulator Trainer [VROV Manipulator Trainer].
  20. 20.
    Parga, C., Li, X., Yu, W.: Tele-manipulation of robot arm with smartphone. In: 2013 6th International Symposium on Resilient Control Systems (ISRCS) (2013). doi: 10.1109/ISRCS.2013.6623751
  21. 21.
    Mishra, A.K., Meruvia-Pastor, O.: Robot arm manipulation using depth-sensing cameras and inverse kinematics. Oceans - St. John’s (2014). doi: 10.1109/OCEANS.2014.7003029
  22. 22.
    Kenwright, B.: Inverse kinematics - cyclic coordinate descent (CCD). J. Graphics, GPU & Game Tools, 177–217 (2012)Google Scholar
  23. 23.
    Hincapié-Ramos, J.D., Guo, X., Moghadasian, P., Irani, P.: Consumed endurance: a metric to quantify arm fatigue of mid-air interactions. Presented at Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2014, Available: doi: 10.1145/2556288.2557130
  24. 24.
    Kim, K., Kim, J., Choi, J., Kim, J., Lee, S.: Depth camera-based 3D hand gesture controls with immersive tactile feedback for natural mid-air gesture interactions. Sensors (Basel) 15(1), 1022–1046 (2015). doi: 10.3390/s150101022 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Akhilesh Kumar Mishra
    • 1
  • Lourdes Peña-Castillo
    • 1
    • 2
  • Oscar Meruvia-Pastor
    • 1
    Email author
  1. 1.Department of Computer ScienceMemorial University of NewfoundlandSt John’sCanada
  2. 2.Department of BiologyMemorial University of NewfoundlandSt John’sCanada

Personalised recommendations