Personal and Ubiquitous Computing

, Volume 17, Issue 1, pp 69–79 | Cite as

How natural is a natural interface? An evaluation procedure based on action breakdowns

  • Luciano Gamberini
  • Anna SpagnolliEmail author
  • Lisa Prontu
  • Sarah Furlan
  • Francesco Martino
  • Beatriz Rey Solaz
  • Mariano Alcañiz
  • Josè Antonio Lozano
Original Article


This paper describes an issue-based method to evaluate the naturalness of an interface. The method consists of the execution of a series of tasks on that interface, which is subsequently systematically analyzed to identify breakdowns in the users’ actions. The systematic analysis of breakdowns is allowed by the support of video-coding software (The Observer by Noldus). This method is described on its theoretical bases and then applied to the evaluation of a natural interface, a walk-in-place locomotion system for virtual spaces called Superfeet. The procedure is comparative, since Superfeet is compared to two locomotion devices, Superfeet enhanced with headtracker and a more traditional Joypad. The test involves 36 participants (mean age = 23.68, SD = 3.14). The outcomes of the breakdown analysis are illustrated at a progressively finer level of granularity from the amount and length of breakdowns, to the circumstances of the breakdowns, to the type of actions involved in the breakdowns. The potential of this procedure for usability studies is finally synthesized.


Usability Naturalness Action breakdown Systematic video analysis 



The authors would like to thank Daniel Kerrigan for his help in the data collection and quantitative analysis; they also would like to thank Merche Calvet for her work in the development of the SuperFeet system. Beatriz Rey Solaz, Mariano Alcañiz, and Josè Antonio Lozano developed Superfeet and the virtual environments for the test, while Luciano Gamberini, Anna Spagnolli, Lisa Prontu, Sarah Furlan, and Francesco Martino carried out the breakdown analysis.


  1. 1.
    Bailey BP, Konstan JA, Carlis JV (2000) Measuring the effects of interruptions on task performance in the user interface. In: Proceedings of IEEE international conference on systems, man, and cybernetics, vol 2, Nashville, TN, pp 757–762, 8–11 NovemberGoogle Scholar
  2. 2.
    Barrera S, Takahashi H, Nakajima M (2004) Hands-free navigation methods for moving through a virtual landscape walking interface virtual reality input devices. In: Proceedings of computer graphics international, IEEE, Los Alamitos, pp 388–394Google Scholar
  3. 3.
    Beckhaus S, Blom KJ, Haringer M (2005) Intuitive, hands-free travel interfaces for virtual environments. In: Bowman D, Fröhlich B, Kitamura Y, Stürzlinger W (eds) Proceedings of IEEE VR2005 workshop on new directions in 3D user interfaces. Shaker Verlag, Aachen, pp 57s–60sGoogle Scholar
  4. 4.
    Bodker S (1991) Through the interface: a human activity approach to user interface design. Lawrence Erlbaum Associates, HillsdaleGoogle Scholar
  5. 5.
    Bouguila L, Ishii M, Sato M (2002) Realizing a new step-in-place locomotion interface for virtual environment with large display system. In: Proceedings of workshop on virtual environments 2002, Aire-la-Ville, Switzerland, pp 197–207Google Scholar
  6. 6.
    Bowman DA, Gabbard JL (2002) A survey of usability evaluation in virtual environments: classification and comparison of methods. Presence 11:404–424CrossRefGoogle Scholar
  7. 7.
    Chung S, Hahn JK (1999) Animation of human walking in virtual environments. In: Proceedings of computer animation, Geneva, Switzerland, 26–29 May 1999Google Scholar
  8. 8.
    Darken RP, Cockaybe WR, Carmein D (1997) The omni-directional treadmill: A locomotion device for virtual worlds. In: Proceedings of UIST, ACM Press, New York, pp 213–221Google Scholar
  9. 9.
    Hartswood M, Procter R (2000) Design guidelines for dealing with breakdowns and repairs in collaborative work settings. J Hum Comput St 53:91–120CrossRefGoogle Scholar
  10. 10.
    Heath C, Hindmarsh J (2002) Analysing Interaction: video, ethnography and situated conduct. In: May T (ed) Qualitative research in action. Sage, London, pp 99–121Google Scholar
  11. 11.
    Iqbal ST, Horvitz E (2007) Disruption and recovery of computing tasks: field study, analysis, and directions. In: Proceedings of SIGCHI conference on human factors in computing systems (CHI’07), ACM Press, New York, pp 677–686Google Scholar
  12. 12.
    Iwata H (1999) Walking about on an infinite floor. In: Proceedings of virtual reality, IEEE Computer Society Press, Houston, pp 286–293Google Scholar
  13. 13.
    Kapitsa M, Blinnikova I (2003) Task performance under the influence of interruptions. In: Hockey GRJ, Gaillard AWK, Burov O (eds) Operator functional state: the assessment and prediction of human performance degradation in complex tasks. IOS Press, Amsterdam, pp 323–329Google Scholar
  14. 14.
    Keppel G, Wickens TD (2004) Design and analysis: a researcher’s handbook. Pearson/Prentice Hall, Upper Saddle RiverGoogle Scholar
  15. 15.
    Mackinlay JD, Card SK, Robertson GG (1990) Rapid controlled movement through a virtual 3cI workspace. In: Proceedings of 17th annual conference on computer graphics and interactive techniques, ACM Press, New York, pp 171–176Google Scholar
  16. 16.
    McDaniel MA, Einstein GO, Graham T, Rall E (2004) Delaying execution of intentions: overcoming the costs of interruptions. App Cognitive Psych 18:533–547CrossRefGoogle Scholar
  17. 17.
    Norman DA, Nielsen J (2010) Gestural interfaces: a step backward in usability. Interactions 17:46–49CrossRefGoogle Scholar
  18. 18.
    Norman DA (2010) Natural user interfaces are not natural. Interactions 17:6–10CrossRefGoogle Scholar
  19. 19.
    Paelke V, Reimann C, Stichling D (2004) Foot-based mobile interaction with games. In: Proceedings of the 2004 ACM SIGCHI international conference on advances in computer entertainment technology, ACM Press, New York, pp 321–324Google Scholar
  20. 20.
    Pakkanen T, Raisamo R (2004) Appropriateness of foot interaction for non-accurate spatial tasks. In: Proceedings of CHI 2004, ACM Press, New York, pp 1123–1126Google Scholar
  21. 21.
    Rey Solaz B, Lozano JA, Alcañiz RM, Gamberini L, Calvet M, Kerrigan D, Martino F (2007) Super-feet: a wireless hand-free navigation system for virtual environments. In: Shumaker R (ed) Virtual reality. Springer, Heidelberg, pp 348–357Google Scholar
  22. 22.
    van Rhijn A, Mulder JD (2006) Spatial input device structure and bimanual object manipulation in virtual environments. In: Proceedings of the ACM symposium on virtual reality software and technology, ACM Press, New York, pp 51–60Google Scholar
  23. 23.
    Scrivener SAR, Urquijo SP, Palmen HK (1993) The use of breakdown analysis in synchronous CSCW system design. Int J Man Mach St 31:517–534Google Scholar
  24. 24.
    Spagnolli A, Gamberini L (2005) Action repair in mediated environments: presence displayed through bodily orientation. Paper presented at the second congress of the international society for gesture studies’ Lyon/France, 15–18 June 2005Google Scholar
  25. 25.
    Spagnolli A, Gamberini L, Gasparini D (2002) Situated breakdown analysis for the evaluation of a virtual environment. Psychnology 1:5–17Google Scholar
  26. 26.
    Suchman L (1987) Plans and situated actions: the problem of human-machine communication. Cambridge University Press, New YorkGoogle Scholar
  27. 27.
    Tullis T, Albert B (2008) Measuring the user experience. Morgan Kaufman, BurlingtonGoogle Scholar
  28. 28.
    Usoh M, Arthur K, Whitton MC, Bastos R, Steed A, Slater M, Brooks FP (1999) Walking > walking-in-place > flying. In: Proceedings of SIGGRAPH 99, ACM Press, New York, pp 359–364Google Scholar
  29. 29.
    Ward M, Azuma R, Bennett R, Gottschalk S, Fuchs H (1992) A demonstrated optical tracker with scalable work area for head-mounted display systems. In: Proceedings of the 1992 symposium on interactive 3D graphics, ACM Press, New York, pp 43–52Google Scholar
  30. 30.
    Welch G, Bishop G, Vicci L, Brumback S, Keller K, Colucci D (1999) The HiBall tracker: high-performance wide-area tracking for virtual and augmented environments. In: Proceedings of the ACM symposium on virtual reality software and technology, ACM, New York, pp 1–10Google Scholar
  31. 31.
    Wells M, Peterson B, Aten J (1996) The virtual motion controller: a sufficient-motion walking simulator. In: Proceedings of the IEEE virtual reality annual international symposium, pp 1–8Google Scholar
  32. 32.
    Winograd T, Flores F (1986) Understanding computers and cognition. Ablex Publishing, NorwoodzbMATHGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Luciano Gamberini
    • 1
  • Anna Spagnolli
    • 1
    Email author
  • Lisa Prontu
    • 1
  • Sarah Furlan
    • 1
  • Francesco Martino
    • 1
  • Beatriz Rey Solaz
    • 2
  • Mariano Alcañiz
    • 2
  • Josè Antonio Lozano
    • 2
  1. 1.Dipartimento di Psicologia GeneraleUniversità di PadovaPadovaItaly
  2. 2.Instituto de Investigación e Innovación en BioingenieríaUniversidad Politécnica de ValenciaValenciaSpain

Personalised recommendations