CoCo - A Framework for Multicore Visuo-Haptics in Mixed Reality

  • Emanuele RuffaldiEmail author
  • Filippo Brizzi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9768)


Mixed Reality applications involve the integration of RGB-D streams with virtual entities potentially extended with force feedback. Increasing complexity of the applications pushes the limits of traditional computing structures, not keeping up with the increased computing power of multicore platform. This paper presents the CoCo framework, a component based, multicore system designed for tackling the challenges of visuo-haptics in mixed reality environment, with structural reconfiguration. Special care has been also given to the management of transformation between reference frames for easing registration, calibration and integration of robotic systems. The framework is described together with a description of two relevant case studies.


Point Cloud Transformation Graph Haptic Feedback Haptic Device Implicit Surface 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work has been carried out within the framework of the European Project REMEDI, grant number 610902, and the Tuscany Regional Project TAUM.


  1. 1.
    Ruffaldi, E., Brizzi, F., Filippeschi, A., Avizzano, C.A.: Co-located haptic interaction for virtual usg exploration. In: Proceedings of IEEE EMBC, pp. 1548–1551 (2015)Google Scholar
  2. 2.
    Filippeschi, A., Brizzi, F., Ruffaldi, E., Jacinto, J.M., Avizzano, C.A.: Encountered-type haptic interface for virtual interaction with real objects based on implicit surface haptic rendering for remote palpation. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5904–5909. IEEE (2015)Google Scholar
  3. 3.
    Peppoloni, L., Brizzi, F., Ruffaldi, E., Avizzano, C.A.: Augmented reality-aided tele-presence system for robot manipulation in industrial manufacturing. In: Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology, pp. 237–240. ACM (2015)Google Scholar
  4. 4.
    Burns, D., Osfield, R.: Tutorial: open scene graph a: introduction tutorial: open scene graph b: examples and applications. In: Proceedings of the IEEE Virtual Reality, p. 265 (2004)Google Scholar
  5. 5.
    Carrozzino, M., Tecchia, F., Bacinelli, S., Cappelletti, C., Bergamasco, M.: Lowering the development time of multimodal interactive application: the real-life experience of the XVR project. In: Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, pp. 270–273. ACM (2005)Google Scholar
  6. 6.
    Behr, J., Bockholt, U., Fellner, D.: Instantreality – a framework for industrial augmented and virtual reality applications. In: Ma, D., Fan, X., Gausemeier, J., Grafe, M. (eds.) Virtual Reality & Augmented Reality in Industry, pp. 91–99. Springer, Heidelberg (2011)Google Scholar
  7. 7.
    Figueroa, P., Bischof, W., Boulanger, P., Hoover, H., Taylor, R.: InTml: a dataflow oriented development system for virtual reality applications. Presence 17(5), 492–511 (2008)CrossRefGoogle Scholar
  8. 8.
    Allard, J., Gouranton, V., Lecointre, L., Limet, S., Melin, E., Raffin, B., Robert, S.: FlowVR: a middleware for large scale virtual reality applications. In: Danelutto, M., Vanneschi, M., Laforenza, D. (eds.) Euro-Par 2004. LNCS, vol. 3149, pp. 497–505. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Conti, F., Morris, D., Barbagli, F., Sewell, C.: Chai 3d (2006).
  10. 10.
    Ruffaldi, E., Frisoli, A., Gottlieb, C., Tecchia, F., Bergamasco, M.: A haptic toolkit for the development of immersive and web enabled games. In: ACM Symposium on Virtual Reality Software and Technology (VRST), pp. 320–323. ACM (2006)Google Scholar
  11. 11.
    Eck, U., Sandor, C.: Harp: a framework for visuo-haptic augmented reality. In: 2013 IEEE Virtual Reality (VR), pp. 145–146, March 2013Google Scholar
  12. 12.
    Bruyninckx, H.: Open robot control software: the orocos project. In: Proceedings of the 2001 IEEE International Conference on Robotics and Automation, ICRA 2001, vol. 3, pp. 2523–2528. IEEE (2001)Google Scholar
  13. 13.
    Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3(3.2), p. 5 (2009)Google Scholar
  14. 14.
    Seichter, H., Looser, J., Billinghurst, M.: Composar: an intuitive tool for authoring AR applications. In: Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 177–178. IEEE Computer Society (2008)Google Scholar
  15. 15.
    Salisbury, K., Conti, F., Barbagli, F.: Haptic rendering: introductory concepts. IEEE Comput. Graph. Appl. 24(2), 24–32 (2004)CrossRefGoogle Scholar
  16. 16.
    Ruffaldi, E., Morris, D., Barbagli, F., Salisbury, K., Bergamasco, M.: Voxel-based haptic rendering using implicit sphere trees. In: Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Haptics 2008, pp. 319–325, March 2008Google Scholar
  17. 17.
    Ruspini, D.C., Kolarov, K., Khatib, O.: The haptic display of complex graphical environments. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 345–352. ACM Press/Addison-Wesley Publishing Co. (1997)Google Scholar
  18. 18.
    Salisbury, K., Tarr, C.: Haptic rendering of surfaces defined by implicit functions. ASME Dyn. Syst. Control Div. 61, 61–67 (1997)Google Scholar
  19. 19.
    Cruz-Neira, C., Sandin, D.J., DeFanti, T.A.: Surround-screen projection-based virtual reality: the design and implementation of the cave. In: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, pp. 135–142. ACM (1993)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.TeCiP, Scuola Superiore Sant’AnnaPisaItaly

Personalised recommendations