Heuristic Evaluation and Mobile Usability: Bridging the Realism Gap

  • Shirlina Po
  • Steve Howard
  • Frank Vetere
  • Mikael B. Skov
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3160)


Heuristic evaluation (HE) is problematic when applied to mobile technologies, in that contextual influences over use are poorly represented. Here we propose two lightweight variants of HE: the Heuristic Walkthrough (HW) combines HE with scenarios of use, and the Contextual Walkthrough (CW) involves conducting the HW in the field. 11 usability experts were asked to use one of these three approaches to evaluate a mobile device and the usability flaws discovered were compared across technique. HW discovered more critical usability flaws than HE. CW revealed some unique problems relating to I/O and ambient lighting not encountered in the other two approaches. Though contextualizing heuristic evaluation improves the assessment of mobile devices, it appears that it is possible to introduce contextual detail, i.e. to bridge the ’realism gap’, with scenarios rather than expensive in-situ testing.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abowd, G., Mynatt, E.: Charting past, present and future research in ubiquitous computing. ACM ToCHI 7(1), 29–58 (2000)CrossRefGoogle Scholar
  2. 2.
    Kjeldskov, J., Stage, J.: New Techniques for Usability Evaluation of Mobile Systems. International Journal of Human-Computer Studies (IJHCS) Elsevier (forthcoming 2004)Google Scholar
  3. 3.
    Dunlop, M., Brewster, S.: The Challenge of Mobile Devices for Human Computer Interaction. Personal and Ubiquitous Computing 6(4), 235–236 (2000)CrossRefGoogle Scholar
  4. 4.
    Greenberg, S., Fitzpatrick, G., Gutwin, C., Kaplan, S.: Adapting the locales framework for heuristic evaluation of groupware. In: Proc OZCHI 2000 (1999)Google Scholar
  5. 5.
    Johnson, P.: Usability and mobility; interaction on the move. In: Proc First Workshop on Human-Computer Interaction with Mobile Devices (1998)Google Scholar
  6. 6.
    Kjeldskov, J., Graham, C.: A review of MobileHCI research methods. In: Proc MobileHCI 2003 (2003)Google Scholar
  7. 7.
    Kjeldskov, J., Skov, M.B.: Creating a Realistic Laboratory Setting: A Comparative Study of Three Think-Aloud Usability Evaluations of a Mobile System. In: Proceedings of the 9th IFIP TC13 International Conference on Human Computer Interaction, Interact 2003, Zürich, Switzerland, IOS Press, Amsterdam (2003)Google Scholar
  8. 8.
    Law, L., Hvannberg, E.: Complementarity and convergence of heuristic evaluation and usability test: a case study of UNIVERSAL brokerage platform. In: Proc NordicCHI 2000 (2002)Google Scholar
  9. 9.
    Muller, M., McClard, A., Bell, B., Dooley, S., Meiskey, L., Meskill, J., Sparks, R., Tellam, D.: Validating an extension to participatory heuristic evaluation: quality of work and quality of work life. In: Proc CHI 1995 (1995)Google Scholar
  10. 10.
    Nielsen, C.: Testing in the field. In: Proc APCHI 1998, pp. 285–290 (1998)Google Scholar
  11. 11.
    Nielsen, J.: Heuristic Evaluation, in Nielsen and Mack Usability Inspection Methods. Wiley, Chichester (1994)Google Scholar
  12. 12.
    Po, S.: Mobile Usability Testing and Evaluation. BIS(Hons) Thesis. Department of Information System. The University of Melbourne (2003)Google Scholar
  13. 13.
    Vetere, F., Howard, S., Pedell, S., Balbo, S.: Walking through mobile use: novel heuristics and their application. In: Proc OZCHI 2003 (2003)Google Scholar
  14. 14.
    Nielsen, J.G.: HCI: Using Discount Usability Engineering to Penetrate the Intimidation Barrier, Available at (reviewed 25/02/2004 1:13:30 PM)
  15. 15.
    Millen, D.R., Dray, S.: Information sharing in an online community of journalists. In: Proceedings of Ethnographic studies in Real and Virtual Environments: Inhabited Information Spaces and Connected Communities, Edinburgh, Scotland (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Shirlina Po
    • 1
  • Steve Howard
    • 1
  • Frank Vetere
    • 1
  • Mikael B. Skov
    • 2
  1. 1.Interaction Design Group, Department of Information SystemsThe University of MelbourneAustralia
  2. 2.Department of Computer ScienceAalborg UniversityDenmark

Personalised recommendations