Skip to main content
SpringerLink
Log in

Reliable behaviour simulation of product interface in mixed reality

  • Original Article
  • Published:
Engineering with Computers Aims and scope Submit manuscript

Abstract

The validation of a product interface is often a critical issue in the design process. Virtual reality and mixed reality (MR) are able to enhance the interactive simulation of the product human–machine interface (HMI), as these technologies allow engineers to directly involve end users in the usability assessment. This paper describes a MR environment specifically addressed to the usability evaluation of a product interface, which allows the simulation of the HMI behaviour using the same models and the same software employed by engineers during the design phase. Our approach is based on the run-time connection between the visualisation software and the simulators used for product design and analysis. In particular, we use Matlab/Simulink to model and simulate the product behaviour, and Virtools to create the interactive MR environment in which the end user can test the product. Thanks to this architecture, any modification done on the behaviour models is immediately testable in MR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Jimeno A, Puerta A (2007) State of the art of the virtual reality applied to design and manufacturing processes. Int J Adv Manuf Tech 33–9(10):866–874

    Article  Google Scholar 

  2. Bruno F, Muzzupappa M (2010) Product interface design: a participatory approach based on virtual reality. Int J Hum Comput Stud 68–5:254–269

    Article  Google Scholar 

  3. Barbieri L, Bruno F, Caruso F, Muzzupappa M (2008) Innovative integration techniques between Virtual Reality systems and CAx tools. Int J Adv Manuf Tech 38–11(12):1085–1097

    Article  Google Scholar 

  4. Aoyama H, Kimishima Y (2009) Mixed reality system for evaluating designability and operability of information appliances. Int J Interact Des Manuf 3–3:157–164

    Article  Google Scholar 

  5. Bordegoni M, Cugini U, Caruso G, Polistena S (2009) Mixed prototyping for product assessment: a reference framework. Int J Interact Des Manuf 3–3:177–187

    Article  Google Scholar 

  6. Park H, Son JS, Lee KH (2008) Design evaluation of digital consumer products using virtual reality-based functional behaviour simulation. J Eng Des 19–4:359–375

    Article  Google Scholar 

  7. Park H, Moon HC, Lee JY (2009) Tangible augmented prototyping of digital handheld products. Comput Ind 60–2:114–125

    Article  Google Scholar 

  8. Bruno F, Caruso F, Muzzupappa M, Stork A (2007) An experimental environment for the runtime communication among different solvers and visualisation modules. Proceedings of 19th European Modeling and Simulation Symposium, Genova, Italy

  9. Bruno F, Caruso F, Li K, Milite A, Muzzupappa M (2009) Dynamic simulation of virtual prototypes in immersive environment. Int J Adv Manuf Tech 43–5(6):620–630

    Article  Google Scholar 

  10. Fiorentino M, de Amicis R, Monno G, Stork A (2002) Spacedesign: A mixed reality workspace for aesthetic industrial design. Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR02), Darmstadt, Germany

  11. Verlinden J, Horvath I (2006) Framework for testing and validating interactive augmented prototyping as a design means in industrial practice. Proceedings of Virtual Concept, Playa Del Carmen, Mexico

    Google Scholar 

  12. Verlinden J, Horvath I (2007) A critical systems position on augmented prototyping systems for industrial design. Proceedings of the ASME-CIE 2007, DETC2007-35642, International Design Engineering Technical Conferences, Las Vegas, USA

  13. Verlinden J, Horvath I (2009) Analyzing opportunities for using interactive augmented prototyping in design practice. Artif Intell Eng Des Anal Manuf 23:289–303

    Article  Google Scholar 

  14. Kanai S, Higuchi T (2008) Applying user interface-operable 3D digital prototypes to human-centered design of information appliances. Proceedings of the ASME 2008 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference IDETC/CIE 2008, New York, USA

  15. Harel D (1987) Statecharts: a visual formalism for complex system. Sci Comput Program 8:231–274

    Article  MathSciNet  MATH  Google Scholar 

  16. Kanai S, Higuchi T, Kikuta Y (2009) 3D digital prototyping and usability enhancement of information appliances based on UsiXML. Int J Interact Des Manuf 3–3:201–222

    Article  Google Scholar 

  17. Bruno F, Angilica A, Cosco F, Muzzupappa M (2010) Functional behaviour simulation of industrial products in virtual reality. Proceedings of the 8th International Symposium on Tools and Methods of Competitive Engineering, TMCE 2010, 2, pp 763–774

  18. Bruno F, Angilica A, Cosco F I, Muzzupappa M (2010) Product behaviour simulation in mixed reality. Proceedings of ASME World Conference on Innovative Virtual Reality (WINVR2010), Ames, Iowa, USA

  19. Bruno F, Cosco F, Angilica A, Muzzupappa M (2010) Mixed prototyping for products usability evaluation. Proceedings of the ASME 2010 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC/CIE 2010), Montreal, Quebec, Canada

  20. Vezhnevets V, Sazonov V, Andreeva A (2003) A survey on pixel-based skin color detection techniques. Proceedings of Graphicon 2003, Moscow, Russia

  21. Kakumanu P, Makrogiannis S, Bourbakis N (2007) A survey of skin-color modeling and detection methods. Pattern Recogn 40–3:1106–1122

    Article  Google Scholar 

  22. Phung SL, Bouzerdoum A, Chai D (2005) Skin segmentation using color pixel classification: analysis and comparison. IEEE Trans Pattern Anal Mach Intell 27–1:148–154

    Article  Google Scholar 

  23. Bordegoni M, Caruso G, Ferrise F (2008) Mixed-Reality Environment based on Haptic Control System for a tractor cabin design review. Proceedings of CIRP Design Conference, Enschede, April 7–9

  24. Hartmann B, Klemmer SR, Bernstein M, Abdulla L, Burr B, Robinson-Mosher A, Gee J (2006) Reflective physical prototyping through integrated design, test, and analysis. Proceedings of the Annual ACM Symposium on User Interface Software and Technology-UIST’06, Montreux, Switzerland

  25. Landauer TK, Nielsen J (1993) A mathematical model of the finding of usability problems. In: Proceedings of ACM INTERCHI’93 Conference, Amsterdam, 24–29 April, 206–213

  26. Virzi RA (1992) Refining the test phase of usability evaluation: how many subjects is enough? Hum Factors 34:457–468

    Google Scholar 

  27. Nielsen J (2000) Why you only need to test with 5 users: Alertbox. http://www.useit.com/alertbox/20000319.html

  28. ANSI (2001) Common industry format for usability test reports (ANSI-NCITS 354-2001). American National Standards Institute, Washington, DC

  29. ISO 9241 (1998) Ergonomic requirements for office work with visual display terminals (VDTs)–Part 11: guidance on usability (ISO 9241-11:1998(E)). Geneva, Switzerland

  30. Lewis JR (2006) Usability testing. In: Salvendy G (ed) Handbook of human factors and ergonomics. John Wiley, New York, NY, pp 1275–1316

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, University of Calabria, Rende, Italy

    Fabio Bruno, Agostino Angilica, Francesco Cosco & Maurizio Muzzupappa

Authors
  1. Fabio Bruno
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Agostino Angilica
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Francesco Cosco
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maurizio Muzzupappa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabio Bruno.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bruno, F., Angilica, A., Cosco, F. et al. Reliable behaviour simulation of product interface in mixed reality. Engineering with Computers 29, 375–387 (2013). https://doi.org/10.1007/s00366-012-0293-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00366-012-0293-7

Keywords

Search

Navigation