Robot Control and Online Programming by Human Gestures Using a Kinect Motion Sensor

Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 275)


Monitoring and controlling of robots are key competences for robot producers and many research and development projects are carried out to increase robot performance and add new functionality. The paper presents a concept and implementation of online programming, controlling and monitoring an industrial robot using a Kinect sensor as a HMI part. Presented innovative solution is based on human gestures. A communication interface for gestures analysis performed by the robot operator was created using LabVIEW applications. Communication between the kinetic motion sensor and the robot was carried out through PLC Siemens S7-300 inside a flexible manufacturing system. The connection between the logic controller and the robot controller R30iA was realized in the ProfibusDP network. The connection between the controller and LabVIEW application has been established using the MPI protocol. A hardware communication protocol used by PLC was converted into the OPC protocol. NUI was used to communicate the Kinect with LabVIEW.


industrial robot control online programming HMI (Human-Machine Interface) PLC (Programmable Logic Controller) MPI Profibus OPC Kinect NUI (Natural User Interface) LabVIEW 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Brogardh, T.: Present and future robot control development - An industrial perspective. Annual Reviews in Control 31, 69–79 (2007)CrossRefGoogle Scholar
  2. 2.
    Chen, L., Wei, H., Ferryman, J.: A survey of human motion analysis using depth imagery. Pattern Recognition Letters 34, 1995–2006 (2013)CrossRefGoogle Scholar
  3. 3.
    Dutta, T.: Evaluation of the Kinect sensor for 3-D kinematic measurement in the workplace. Applied Ergonomics 43, 645–649 (2012)CrossRefGoogle Scholar
  4. 4.
    Gonzalez-Jorge, H., Riveiro, B., Vazquez-Fernandez, E., Martnez-Snchez, J., Arias, P.: Metrological evaluation of Microsoft Kinect and Asus Xtion sensors. Measurement 46, 1800–1806 (2013)CrossRefGoogle Scholar
  5. 5.
    Grabowik, C., Kalinowski, K.: Object-Oriented Models in an Integration of CAD/CAPP/CAP Systems. In: Corchado, E., Kurzyński, M., Woźniak, M. (eds.) HAIS 2011, Part II. LNCS, vol. 6679, pp. 405–412. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    Kahn, S., Bockholt, U., Kuijper, A., Fellner, D.W.: Towards precise real-time 3D difference detection for industrial applications. Computers in Industry 64, 1115–1128 (2013)CrossRefGoogle Scholar
  7. 7.
    Kolb, A., Barth, E., Koch, R., Larsen, R.: Time-of-flight sensors in computer graphics. In: Proc. Eurographics (State-of-the-Art Report), pp. 119–134 (2009)Google Scholar
  8. 8.
    Lau, B., Sprunk, C., Burgard, W.: Efficient grid-based spatial representations for robot navigation in dynamic environments. Robotics and Autonomous Systems 61, 1116–1130 (2013)CrossRefGoogle Scholar
  9. 9.
    Mentiplay, B., Clark, R., Mullins, A., Bryant, A., Bartold, S., Paterson, K.: Evaluation of foot posture using the Microsoft Kinect. Journal of Science and Medicine in Sport 16S, e2–e38 (2013)Google Scholar
  10. 10.
  11. 11.
    Oggier, T., Lustenberger, F., Blanc, N.: Miniature 3D TOF camera for real-time imaging. In: Perception and Interactive Technologies, pp. 212–216 (2006)Google Scholar
  12. 12.
    Pan, Z., Polden, J., Larkin, N., VanDuin, S., Norrish, J.: Recent progress on programming methods for industrial robots. Robotics and Computer-Integrated Manufacturing 28, 87–94 (2012)CrossRefGoogle Scholar
  13. 13.
    Pioskowik, D.: The integration of communication protocols with the control system of an industrial robot FANUC ARC Mate 100iC. Master thesis promoted by Zolkiewski S. report. Silesian University of Technology, Gliwice (2013)Google Scholar
  14. 14.
    Sgorbissa, A., Verda, D.: Structure-based object representation and classification in mobile robotics through a Microsoft Kinect. Robotics and Autonomous Systems 61, 1665–1679 (2013)CrossRefGoogle Scholar
  15. 15.
    Smith, C., Karayiannidis, Y., Nalpantidis, L., Gratal, X., Qi, P., Dimarogonas, D.V., Kragic, D.: Dual arm manipulation - A survey. Robotics and Autonomous Systems 60, 1340–1353 (2012)CrossRefGoogle Scholar
  16. 16.
    Stoyanov, T., Mojtahedzadeh, R., Andreasson, H., Lilienthal, A.J.: Comparative evaluation of range sensor accuracy for indoor mobile robotics and automated logistics applications. Robotics and Autonomous Systems 61, 1094–1105 (2013)CrossRefGoogle Scholar
  17. 17.
    Zalevsky, Z., Shpunt, A., Maizels, A., Garcia, J.: Method and System for Object Reconstruction (2007)Google Scholar
  18. 18.
    Tsai, Z.-R.: Robust Kinect-based guidance and positioning of a multidirectional robot by Log-ab recognition. Expert Systems with Applications 41, 1271–1282 (2014)CrossRefGoogle Scholar
  19. 19.
    Zolkiewski, S.: Dynamic Flexibility of Complex Damped Systems Vibrating Transversally in Transportation. Solid State Phenomena 164, 339–342 (2010)CrossRefGoogle Scholar
  20. 20.
    Zolkiewski, S.: Numerical Application for Dynamic Analysis of Rod and Beam Systems in Transportation. Solid State Phenomena 164, 343–348 (2010)CrossRefGoogle Scholar
  21. 21.
    Zolkiewski, S.: Attenuation-frequency Characteristics of Beam Systems in Spatial Motion. Solid State Phenomena 164, 349–354 (2010)CrossRefGoogle Scholar
  22. 22.
    Zolkiewski, S.: Damped Vibrations Problem of Beams Fixed on the Rotational Disk. International Journal of Bifurcation and Chaos 21(10), 3033–3041 (2011)CrossRefMATHGoogle Scholar
  23. 23.
    Zolkiewski, S.: Dynamic flexibility of the supported-clamped beam in transportation. Journal of Vibroengineering 13(4), 810–816 (2011)Google Scholar
  24. 24.
    Zolkiewski, S.: Vibrations of beams with a variable cross-section fixed on rotational rigid disks. Latin American Journal of Solids and Structures 10, 39–57 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Institute of Engineering Processes Automation and Integrated Manufacturing SystemsSilesian University of TechnologyGliwicePoland

Personalised recommendations