An Augmented Interface to Display Industrial Robot Faults

  • Francesco De PaceEmail author
  • Federico Manuri
  • Andrea Sanna
  • Davide Zappia
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10851)


Technology advancement is changing the way industrial factories have to face an increasingly complex and competitive market. The fourth industrial revolution (known as industry 4.0) is also changing how human workers have to carry out tasks and actions. In fact, it is no longer impossible to think of a scenario in which human operators and industrial robots work side-by-side, sharing the same environment and tools. To realize a safe work environment, workers should trust robots as well as they trust human operators. Such goal is indeed complex to achieve, especially when workers are under stress conditions, such as when a fault occurs and the human operators are no longer able to understand what is happening in the industrial manipulator. Indeed, Augmented Reality (AR) can help workers to visualize in real-time robots’ faults. This paper proposes an augmented system that assists human workers to recognize and visualize errors, improving their awareness of the system. The system has been tested using both an AR see-through device and a smartphone.


Industry 4.0 Industrial robots Human-machines interfaces Augmented reality 



This work is co-funded by the regional project HuManS (Human Centered Manufacturing Systems).


  1. 1.
    Bauer, A., Wollherr, D., Buss, M.: Human-robot collaboration: a survey. Int. J. Humanoid Robot. 5(01), 47–66 (2008)CrossRefGoogle Scholar
  2. 2.
    Morato, C., Kaipa, K.N., Zhao, B., Gupta, S.K.: Toward safe human robot collaboration by using multiple kinects based real-time human tracking. J. Comput. Inf. Sci. Eng. 14(1), 011006 (2014)CrossRefGoogle Scholar
  3. 3.
    Coovert, M.D., Lee, T., Shindev, I., Sun, Y.: Spatial augmented reality as a method for a mobile robot to communicate intended movement. Comput. Hum. Behav. 34, 241–248 (2014)CrossRefGoogle Scholar
  4. 4.
    Chen, J., Patton, R.J.: Robust Model-based Fault Diagnosis for Dynamic Systems, vol. 3. Springer, Boston (2012). Scholar
  5. 5.
    Miseikis, J., Knobelreiter, P., Brijacak, I., Yahyanejad, S., Glette, K., Elle, O.J., Torresen, J.: Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks (2018). arXiv preprint arXiv:1801.02025
  6. 6.
    Nikolaidis, S., Lasota, P., Rossano, G., Martinez, C., Fuhlbrigge, T., Shah, J.: Human-robot collaboration in manufacturing: quantitative evaluation of predictable, convergent joint action. In: 2013 44th International Symposium on Robotics (ISR), pp. 1–6. IEEE (2013)Google Scholar
  7. 7.
    Chadalavada, R.T., Andreasson, H., Krug, R., Lilienthal, A.J.: That’s on my mind! robot to human intention communication through on-board projection on shared floor space. In: 2015 European Conference on Mobile Robots (ECMR), pp. 1–6. IEEE (2015)Google Scholar
  8. 8.
    Matsumaru, T.: Mobile robot with preliminary-announcement and display function of forthcoming motion using projection equipment. In: The 15th IEEE International Symposium on Robot and Human Interactive Communication, 2006, ROMAN 2006, pp. 443–450. IEEE (2006)Google Scholar
  9. 9.
    Akan, B., Çürüklü, B.: Augmented reality meets industry: Interactive robot programming. In: Proceedings of SIGRAD 2010: Content Aggregation and Visualization, 25–26 November 2010, Västerås, Sweden, no. 052, pp. 55–58. Linköping University Electronic Press (2010)Google Scholar
  10. 10.
    Michalos, G., Karagiannis, P., Makris, S., Tokçalar, Ö., Chryssolouris, G.: Augmented reality (AR) applications for supporting human-robot interactive cooperation. Procedia CIRP 41, 370–375 (2016)CrossRefGoogle Scholar
  11. 11.
    Mateo, C., Brunete, A., Gambao, E., Hernando, M.: Hammer: an android based application for end-user industrial robot programming. In: 2014 IEEE/ASME 10th International Conference on Mechatronic and Embedded Systems and Applications (MESA), pp. 1–6. IEEE (2014)Google Scholar
  12. 12.
    Fantuzzi, C., Secchi, C., Visioli, A.: On the fault detection and isolation of industrial robot manipulators. IFAC Proc. Vol. 36(17), 399–404 (2003)CrossRefGoogle Scholar
  13. 13.
    Singh, V. D., Banga, V. K.: Overloading failures in robot manipulators. In: International Conference on Trends in Electrical, Electronics and Power Engineering (ICTEEP’2012)/Planetary Scientific Research Centre, pp. 15–16 (2012)Google Scholar
  14. 14.
    Michieletto, S., Chessa, N., Menegatti, E.: Learning how to approach industrial robot tasks from natural demonstrations. In: 2013 IEEE Workshop on Advanced Robotics and Its Social Impacts (ARSO), pp. 255–260. IEEE (2013)Google Scholar
  15. 15.
    Meta 2 ar headset. Accessed 2018Google Scholar
  16. 16.
  17. 17.
    Eski, I., Erkaya, S., Savas, S., Yildirim, S.: Fault detection on robot manipulators using artificial neural networks. Robot. Comput.-Integr. Manuf. 27(1), 115–123 (2011)CrossRefGoogle Scholar
  18. 18.
    Vemuri, A.T., Polycarpou, M.M.: Neural-network-based robust fault diagnosis in robotic systems. IEEE Trans. Neural Netw. 8(6), 1410–1420 (1997)CrossRefGoogle Scholar
  19. 19.
    Ghrieb, A.O., Kourd, Y., Guersi, N.: Supervision of industrial manipulators using ANFIS system. In: 2017 17th International Conference on Control, Automation and Systems (ICCAS), pp. 161–166. IEEE, October 2017Google Scholar
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
    Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. 77, 1321–1329 (1994)Google Scholar
  25. 25.
  26. 26.
  27. 27.
    Hulin, T., Hertkorn, K., Preusche, C.: Interactive features for robot viewers. In: Su, C.-Y., Rakheja, S., Liu, H. (eds.) ICIRA 2012. LNCS (LNAI), vol. 7508, pp. 181–193. Springer, Heidelberg (2012). Scholar
  28. 28.
    Sutherland, I.E.: A head-mounted three dimensional display. In: Proceedings of the AFIPS, pp. 757–764. ACM, San Francisco (1968)Google Scholar
  29. 29.
    De Pace, F., Manuri, F., Sanna, A.: Augmented reality in industry 4.0. Am. J. Comput. Sci. Inf. Technol. 6(1), 1–7 (2018)Google Scholar
  30. 30.
    Sanna, A., Manuri, F.: A survey on applications of augmented reality. Adv. Comput. Sci. Int. J. 5(1), 18–27 (2016)Google Scholar
  31. 31.

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Francesco De Pace
    • 1
    Email author
  • Federico Manuri
    • 1
  • Andrea Sanna
    • 1
  • Davide Zappia
    • 1
  1. 1.Dipartimento di Automatica e InformaticaPolitecnico di TorinoTorinoItaly

Personalised recommendations