Third Point of View Augmented Reality for Robot Intentions Visualization

  • Emanuele RuffaldiEmail author
  • Filippo Brizzi
  • Franco Tecchia
  • Sandro Bacinelli
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9768)


Lightweight, head-up displays integrated in industrial helmets allow to provide contextual information for industrial scenarios such as in maintenance. Moving from single display and single camera solutions to stereo perception and display opens new interaction possibilities. In particular this paper addresses the case of information sharing by a Baxter robot displayed to the user overlooking at the real scene. System design and interaction ideas are being presented.


Augmented Reality Mixed Reality Reprojection Error Stereo Display Safety Helmet 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work has been carried out within the framework of the Tuscany Regional Project TAUM. We also acknowledge Erika Di Stefano for having provided the Baxter setup and the computer vision algorithm, and Alessandro di Fava for the robot control.


  1. 1.
    Guizzo, E., Ackerman, E.: How rethink robotics built its new baxter robot worker. IEEE Spectrum (2012)Google Scholar
  2. 2.
    Heyer, C.: Human-robot interaction and future industrial robotics applications. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4749–4754. IEEE (2010)Google Scholar
  3. 3.
    Mavridis, N.: A review of verbal and non-verbal human-robot interactive communication. Robot. Auton. Syst. 63, 22–35 (2015)CrossRefGoogle Scholar
  4. 4.
    Chen, H., Lee, A.S., Swift, M., Tang, J.C.: 3d collaboration method over hololens and skype end points. In: Proceedings of the 3rd International Workshop on Immersive Media Experiences, pp. 27–30. ACM (2015)Google Scholar
  5. 5.
    Regenbrecht, H., Baratoff, G., Wilke, W.: Augmented reality projects in the automotive and aerospace industries. IEEE Comput. Graph. Appl. 25(6), 48–56 (2005)CrossRefGoogle Scholar
  6. 6.
    Chadalavada, R.T., Andreasson, H., Krug, R., Lilienthal, A.J.: Thats on my mind! robot to human intention communication through on-board projection on shared floor space. In: European Conference on Mobile Robots (ECMR) (2015)Google Scholar
  7. 7.
    Peppoloni, L., Brizzi, F., Ruffaldi, E., Avizzano, C.A.: Augmented reality-aided tele-presence system for robot manipulation in industrial manufacturing. In: Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology, pp. 237–240. ACM (2015)Google Scholar
  8. 8.
    Huang, W., Alem, L., Tecchia, F.: HandsIn3D: supporting remote guidance with immersive virtual environments. In: Winckler, M. (ed.) INTERACT 2013, Part I. LNCS, vol. 8117, pp. 70–77. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  9. 9.
    Ruffaldi, E., Brizzi, F., Filippeschi, A., Avizzano, C.A.: Co-located haptic interaction for virtual USG exploration. In: Proceedings of IEEE EMBC, pp. 1548–1551 (2015)Google Scholar
  10. 10.
    Ruffaldi, E., Brizzi, F.: CoCo - a framework for multicore visuo-haptic in mixed reality. In: De Paolis, L.T., Mongelli, A. (eds.) AVR 2016, Part I. LNCS, vol. 9768, pp. 339–357. Springer, Switzerland (2016)Google Scholar
  11. 11.
    Hinterstoisser, S., Holzer, S., Cagniart, C., Ilic, S., Konolige, K., Navab, N., Lepetit, V.: Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 858–865. IEEE (2011)Google Scholar
  12. 12.
    Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-Jiménez, M.J.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47(6), 2280–2292 (2014)CrossRefGoogle Scholar
  13. 13.
    Zaeh, M.F., Vogl, W.: Interactive laser-projection for programming industrial robots. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, ISMAR 2006, pp. 125–128, October 2006Google Scholar
  14. 14.
    Olwal, A., Gustafsson, J., Lindfors, C.: Spatial augmented reality on industrial CNC-machines. In: Electronic Imaging 2008. International Society for Optics and Photonics, p. 680409 (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Emanuele Ruffaldi
    • 1
    Email author
  • Filippo Brizzi
    • 1
  • Franco Tecchia
    • 1
  • Sandro Bacinelli
    • 1
  1. 1.TeCiP, Scuola Superiore Sant’AnnaPisaItaly

Personalised recommendations