Design review of CAD assemblies using bimanual natural interface

  • Michele FiorentinoEmail author
  • Rafael Radkowski
  • Christian Stritzke
  • Antonio E. Uva
  • Giuseppe Monno
Original Paper


The interaction metaphor, based on mouse, monitor and keyboard, presents evident limits in the engineering design review activities, when real and virtual models must be explored and compared, and also in “outside-the-office” environments, where the desk is not available. The presented research aims to explore a new generation of gesture-based interfaces, called “natural interfaces”, which promise an intuitive control using free hands and without the desk support. We present a novel natural design review workspace which acquires user motion using a combination of video and depth cameras and visualizes the CAD models using monitor-based augmented reality. We implemented a bimanual egocentric pointer paradigm by a virtual active surface in front of the user. We used a XML configurable approach to explore bimanual gesture commands to browse, select, dis/assembly and explode 3D complex models imported in standard STEP format. Our experiments demonstrated that the virtual active surface is able to effectively trigger a set of CAD specific commands and to improve technical navigation in non-desktop environments: e.g. shop floor maintenance, on site quality control, etc. We evaluated the feasibility and robustness of the interface and reported a high degree of acceptance from the users who preferred the presented interface to a unconstrained 3D manipulation.


Natural interfaces Augmented reality Human computer interfaces CAD Depth camera  3D manipulation 


  1. 1.
    Weiser, M., Brown, J.S.: The coming age of calm technology. (1996). Accessed May 2012
  2. 2.
    Kumar, R., Chatterjee, R.: Shaping ubiquity for the developing world. International Telecommunications Union (ITU) Workshop on Ubiquitous Network Societies, Geneva, Switzerland (2005)Google Scholar
  3. 3.
    Blake, J.: Natural user interfaces. In: NET. Manning Publication Co., Shelter Island (2010)Google Scholar
  4. 4.
    Bowman, D.A., Kruijff, E., LaViola, J.J., Poupyrev, I.: 3D User Interfaces: Theory and Practice. Addison Wesley Longman Publishing Co, Inc, Redwood City (2004) Google Scholar
  5. 5.
    Pausch, R., Burnette, T., Brockway, D., Weiblen, M.E.: Navigation and locomotion in virtual worlds via flight into hand-held miniatures. In: Mair, S.G., Cook, R. (eds.) Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’95), pp. 399–400. ACM, New York (1995)Google Scholar
  6. 6.
    Fiorentino, M., Monno, G., Uva, A.E.: Tangible digital master for product lifecycle management in augmented reality. J. Interact. Des. Manuf. 3(2), 121–129 (2009). doi: 10.1007/s12008-009-0062-z
  7. 7.
    Ullmer, B., Ishii, H.: Emerging frameworks for tangible user interfaces. IBM Syst. J. 393(3), 915–931 (2000)CrossRefGoogle Scholar
  8. 8.
    Buchmann, V., Violich, S., Billinghurst, M., Cockburn, A.: FingARtips, gesture based direct manipulation in Augmented Reality. In: Proceedings of the 2nd International Conference on Computer graphics and interactive techniques, GRAPHITE ’04 (2004)Google Scholar
  9. 9.
    Reifinger, S., Wallhoff, F., Ablassmeier, M., Poitschke, T., Rigoll, G.: Static and dynamic hand-gesture recognition for augmented reality applications. In: HCI Intelligent Multimodal Interaction Environments. Springer, Berlin (2007)Google Scholar
  10. 10.
    Microsoft Kinect website. Accessed May (2012)
  11. 11.
    Jaimes, A., Sebe, N.: Multimodal human-computer interaction: a survey. Comput. Vis. Image Underst. 108(1–2), 116–134 (2007)CrossRefGoogle Scholar
  12. 12.
    Zhu, Y., Xu, G., Kriegman, D.J.: A real-time approach to the spotting, representation, and recognition of hand gestures for human–computer interaction. Comput. Vis. Image Underst. 85(3), 189–208 (2002)CrossRefzbMATHGoogle Scholar
  13. 13.
    Liu, X., Fujimura, K.: Hand gesture recognition using depth data. In: Proceedings of Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 529–534 (2004) doi: 10.1109/AFGR.2004.1301587
  14. 14.
    Valentini, P.P.: Enhancing user role in augmented reality interactive simulations. In: Human Factors in Augmented Reality Environments, Springer, New York, pp. 233–256 (2013). doi: 10.1007/978-1-4614-4205-9_10
  15. 15.
    Dani, T.H., Gadh, R.: Creation of concept shape designs via a virtual reality interface. Comput. Aided Des. 29(8), 555–563 (1997). doi: 10.1016/S0010-4485(96)00091-7
  16. 16.
    Dellisanti, M., Fiorentino, M., Monno, G., Uva, A.E.: Enhanced 3D object snap for CAD modelling on large stereo displays. Int. J. Comput. Appl. Technol. 33(1), 54–62 (2008). doi: 10.1504/IJCAT.2008.021885 CrossRefGoogle Scholar
  17. 17.
    Valentini, P.P.: Natural interface in augmented reality interactive simulations. Virtual Phys. Prototyp. 7(2), 137–151 (2012)CrossRefGoogle Scholar
  18. 18.
    Fiorentino, M., Uva, A.E., Monno, G., Radkowski, R.: Augmented technical drawings: a novel technique for natural interactive visualization of computer-aided design models. J. Comput. Inf. Sci. Eng. 12(2) (2012) doi: 10.1115/1.4006431
  19. 19.
    STEP Application Handbook: ISO 10303 (Version 3rd edn), North Charleston, SC: SCRA (2006)Google Scholar
  20. 20.
    Dünser, A., Grasset, R., Seichter, H., Billinghurst, M.: Applying HCI principles to AR systems design, pp. 1–5. HIT Lab New Zeeland Technical Report (2007)Google Scholar
  21. 21.
    Radkowski, R., Stritzke, C.: Interactive hand gesture-based assembly for augmented reality applications. In: Proceedings of ARIA, pp. 303–308 (2012)Google Scholar
  22. 22.
    Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: Proceedings of the 2nd International Workshop on Augmented Reality (IWAR 99), San Francisco, USA, pp. 85–94 (1999)Google Scholar

Copyright information

© Springer-Verlag France 2012

Authors and Affiliations

  • Michele Fiorentino
    • 1
    Email author
  • Rafael Radkowski
    • 2
  • Christian Stritzke
    • 2
  • Antonio E. Uva
    • 1
  • Giuseppe Monno
    • 1
  1. 1.Department of Mechanics, Mathematics and ManagementPolitecnico di BariBariItaly
  2. 2.Heinz Nixdorf InstituteUniversity of PaderbornPaderbornGermany

Personalised recommendations