Off-line programming of an industrial robot in a virtual reality environment

  • Evgenia Manou
  • George-Christopher VosniakosEmail author
  • Elias Matsas
Original paper


Industrial robots need to be programmed quickly in order to be practically deployable in the production of small batches. In programming by teaching or demonstration, when most of the program’s content involves handling tasks, gesture recognition or other multimodal interfaces may be exploited. However, when the main task concerns manufacturing processing, typically tracing an edge in seam welding, deburring or cutting, then positioning and orienting the tool to considerable accuracy are required. This can only be achieved, if suitable tracking sensors are used. The current work employs a 6 degree-of-freedom magnetic sensor, but any other equivalent sensor could be used, too. The sensor is attached to a suitable hand-held teaching tool that is constructed in accordance with the real end-effector tool, enabling continuous tracking of its position and orientation interactively. A virtual reality platform records this stream of data in real time, making it possible to exploit it primarily in off-line programming of the robot. In this mode both the robot and the manufacturing cell are virtual, inverse kinematics allowing for calculation of joint coordinates from end-effector coordinates. Collision and clearance checks are also straightforwardly implemented. An edge-tracing application in 3D space was programmed following this paradigm. The resulting curves of the tool tip in the virtual and the real environment were close enough when compared by using photogrammetry. If required, the VR environment also allows for remote on-line programming, without any major modifications.


Ιndustrial robot Virtual reality Robot programming Trajectory teaching 



Mr Nikolaos Melissas, Chief Technician in NTUA Manufacturing Technology Laboratory, is gratefully acknowledged for his contribution to tool constructions. Miss Margeaux Beaubet, of ENISE, France is gratefully acknowledged for her contribution in photogrammetry measurements.


  1. 1.
    Pedersen, M.R., Nalpantidis, L., Andersen, R.S., et al.: Robot skills for manufacturing: from concept to industrial deployment. Robot. Comput. Integr. Manuf. 37, 282–291 (2016). CrossRefGoogle Scholar
  2. 2.
    Villani, V., Pini, F., Leali, F., Secchi, C.: Survey on human–robot collaboration in industrial settings: safety, intuitive interfaces and applications. Mechatronics (2018). CrossRefGoogle Scholar
  3. 3.
    Vosniakos, G.-C., Chronopoulos, A.: Industrial robot path planning in a constraint-based computer-aided design and kinematic analysis environment. Proc. Inst. Mech. Eng. Part B J. Eng. Manuf. 223, 523–533 (2009). CrossRefGoogle Scholar
  4. 4.
    Pan, Z., Polden, J., Larkin, N., et al.: Recent progress on programming methods for industrial robots. Robot. Comput. Integr. Manuf. 28, 87–94 (2012). CrossRefGoogle Scholar
  5. 5.
    Makris, S., Tsarouchi, P., Surdilovic, D., Krüger, J.: Intuitive dual arm robot programming for assembly operations. CIRP Ann. 63, 13–16 (2014). CrossRefGoogle Scholar
  6. 6.
    Pellegrinelli, S., Pedrocchi, N., Molinari Tosatti, L., et al.: Multi-robot spot-welding cells: an integrated approach to cell design and motion planning. CIRP Ann. 63, 17–20 (2014). CrossRefGoogle Scholar
  7. 7.
    Alami, R., Gharbi, M., Vadant, B., et al.: On human-aware task and motion planning abilities for a teammate robot. In: Human–Robot Collaboration for Industrial Manufacturing Workshop, RSS 2014 (2014)Google Scholar
  8. 8.
    Massa, D., Callegari, M., Cristalli, C.: Manual guidance for industrial robot programming. Ind. Robot. Int. J. 42, 457–465 (2015). CrossRefGoogle Scholar
  9. 9.
    Nathanael, D., Mosialos, S., Vosniakos, G.-C.: Development and evaluation of a virtual training environment for on-line robot programming. Int. J. Ind. Ergon. 53, 274–283 (2016). CrossRefGoogle Scholar
  10. 10.
    Jara, C.A., Candelas, F.A., Gil, P., et al.: EJS + EjsRL: an interactive tool for industrial robots simulation, Computer Vision and remote operation. Robot. Auton. Syst. 59, 389–401 (2011). CrossRefGoogle Scholar
  11. 11.
    Aleotti, J., Caselli, S.: Physics-based virtual reality for task learning and intelligent disassembly planning. Virtual Real. 15, 41–54 (2011). CrossRefGoogle Scholar
  12. 12.
    Chen, H., Sheng, W.: Transformative CAD based industrial robot program generation. Robot. Comput. Integr. Manuf. 27, 942–948 (2011). CrossRefGoogle Scholar
  13. 13.
    Angelidis, A., Vosniakos, G.-C.: Prediction and compensation of relative position error along industrial robot end-effector paths. Int. J. Precis. Eng. Manuf. 15, 63–73 (2014). CrossRefGoogle Scholar
  14. 14.
    Neto, P., Mendes, N., Araújo, R., et al.: High-level robot programming based on CAD: dealing with unpredictable environments. Ind. Robot. Int. J. 39, 294–303 (2012). CrossRefGoogle Scholar
  15. 15.
    Michas, S., Matsas, E., Vosniakos, G.-C.: Interactive programming of industrial robots for edge tracing using a virtual reality gaming environment. Int. J. Mechatron. Manuf. Syst. 10, 237–259 (2017). CrossRefGoogle Scholar
  16. 16.
    Pai, Y.S., Yap, H.J., Singh, R.: Augmented reality-based programming, planning and simulation of a robotic work cell. Proc. Inst. Mech. Eng. Part B J. Eng. Manuf. 229, 1029–1045 (2015). CrossRefGoogle Scholar
  17. 17.
    Fang, H.C., Ong, S.K., Nee, A.Y.C.: A novel augmented reality-based interface for robot path planning. Int. J. Interact. Des. Manuf. 8, 33–42 (2014). CrossRefGoogle Scholar
  18. 18.
    Sucan, I.A., Moll, M., Kavraki, L.E.: The open motion planning library. IEEE Robot. Autom. Mag. 19, 72–82 (2012). CrossRefGoogle Scholar
  19. 19.
    Kaltsoukalas, K., Makris, S., Chryssolouris, G.: On generating the motion of industrial robot manipulators. Robot. Comput. Integr. Manuf. 32, 65–71 (2015). CrossRefGoogle Scholar
  20. 20.
    Berenson, D., Abbeel, P., Goldberg, K.: A robot path planning framework that learns from experience. In: 2012 IEEE International Conference on Robotics and Automation, pp. 3671–3678. IEEE (2012)Google Scholar
  21. 21.
    Billard, A., Calinon, S., Dillmann, R., Schaal, S.: Robot programming by demonstration. In: Siciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, pp. 1371–1394. Springer, Berlin (2008)CrossRefGoogle Scholar
  22. 22.
    Zolkiewski, S., Pioskowik, D.: Robot Control and Online Programming by Human Gestures Using a Kinect Motion Sensor, pp. 593–604. Springer, Cham (2014)Google Scholar
  23. 23.
    Neto, P., Norberto Pires, J., Paulo Moreira, A.: High-level programming and control for industrial robotics: using a hand-held accelerometer-based input device for gesture and posture recognition. Ind. Robot. Int. J. 37, 137–147 (2010). CrossRefGoogle Scholar
  24. 24.
    Pedersen, M.R., Krüger, V.: Gesture-based extraction of robot skill parameters for intuitive robot programming. J. Intell. Robot. Syst. 80, 149–163 (2015). CrossRefGoogle Scholar
  25. 25.
    Tsarouchi, P., Athanasatos, A., Makris, S., et al.: High level robot programming using body and hand gestures. Procedia CIRP 55, 1–5 (2016). CrossRefGoogle Scholar
  26. 26.
    Ni, D., Yew, A.W.W., Ong, S.K., Nee, A.Y.C.: Haptic and visual augmented reality interface for programming welding robots. Adv. Manuf. 5, 191–198 (2017). CrossRefGoogle Scholar
  27. 27.
    Liu, H., Wang, L.: Gesture recognition for human-robot collaboration: a review. Int. J. Ind. Ergon. 1, 1–12 (2017). CrossRefGoogle Scholar
  28. 28.
    Qi, L., Zhang, D., Zhang, J., Li, J.: A lead-through robot programming approach using a 6-DOF wire-based motion tracking device. In: 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1773–1777. IEEE (2009)Google Scholar
  29. 29.
    Adept Technology Inc. V+ Language User's Guide, Version 12.1, Part # 00962-01130, Rev. A, September 1997Google Scholar
  30. 30.
    McCarthy, C., Callele, D.: Virtools User Guide (2006)Google Scholar
  31. 31.
    3DVIA Virtools: Behavior Libraries—VR Library/VR Publisher 2.6 (Virtools—5.0) User Guide (2009)Google Scholar
  32. 32.
    LaScalza, S., Arico, J., Hughes, R.: Effect of metal and sampling rate on accuracy of Flock of Birds electromagnetic tracking system. J. Biomech. 36, 141–144 (2003). CrossRefGoogle Scholar
  33. 33.
    Milne, A.D., Chess, D.G., Johnson, J.A., King, G.J.W.: Accuracy of an electromagnetic tracking device: a study of the optimal operating range and metal interference. J. Biomech. 29, 791–793 (1996)CrossRefGoogle Scholar
  34. 34.
    Mcquade, K.J., Finley, M.A., Harris-love, M., Mccombe-waller, S.: Dynamic error analysis of ascension’ s flock of birds tm electromagnetic tracking device using a pendulum model. J. Appl. Biomech. 18, 171–179 (2002)CrossRefGoogle Scholar
  35. 35.
    Ascension Technology Corporation: The Flock of Birds ® Installation and operation guide (2004)Google Scholar
  36. 36.
    Vosniakos, G.-C., Levedianos, E., Gogouvitis, X.V.: Streamlining virtual manufacturing cell modelling by behaviour modules. Int. J. Manuf. Res. 10, 17–43 (2015). CrossRefGoogle Scholar
  37. 37.
    Luhmann, T.: Close range photogrammetry for industrial applications. ISPRS J. Photogramm. Remote Sens. 65, 558–569 (2010). CrossRefGoogle Scholar
  38. 38.
    Mourelatos, A., Nathanael, D., Gkikas, K., Psarakis, L.: Development and evaluation of a wearable motion tracking system for sensorimotor tasks in VR environments. In: Bagnara, S., Tartaglia, R., Albolino, S., Alexander, Th., Fujita, Y. (eds.) Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018). Volume V: Human Simulation and Virtual Environments, Work With Computing Systems (WWCS), Process Control, pp 181–188. Springer, Berlin (2019)Google Scholar

Copyright information

© Springer-Verlag France SAS, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Mechanical Engineering, Manufacturing Technology SectionNational Technical University of AthensAthensGreece

Personalised recommendations