Skip to main content

Is It Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field

  • Conference paper
Mobile Human-Computer Interaction - MobileHCI 2004 (Mobile HCI 2004)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 3160))

Included in the following conference series:

Abstract

Evaluating the usability of mobile systems raises new concerns and questions, challenging methods for both lab and field evaluations. A recent literature study showed that most mobile HCI research projects apply lab-based evaluations. Nevertheless, several researchers argue in favour of field evaluations as mobile systems are highly context-dependent. However, field-based usability studies are difficult to conduct, time consuming and the added value is unknown. Contributing to this discussion, this paper compares the results produced by a laboratory- and a field-based evaluation of the same context-aware mobile system on their ability to identify usability problems. Six test subjects used the mobile system in a laboratory while another six used the system in the field. The results show that the added value of conducting usability evaluations in the field is very little and that recreating central aspects of the use context in a laboratory setting enables the identification of the same usability problem list.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Abowd, G., Mynatt, E.: Charting past, present and future research in ubiquitous computing. ACM Transactions on Computer-Human Interaction 7(1), 29–58 (2000)

    Article  Google Scholar 

  2. Bardram, J., Kjær, T.A.K., Nielsen, C.: Supporting Local Mobility in Healthcare by Application Roaming among Heterogeneous Devices. In: Chittaro, L. (ed.) Mobile HCI 2003. LNCS, vol. 2795, Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  3. Barkhuus, L., Dey, A.: Is Context-Aware Computing Taking Control away from the User? Three Levels of Interactivity Examined. In: Dey, A.K., Schmidt, A., McCarthy, J.F. (eds.) UbiComp 2003. LNCS, vol. 2864, pp. 149–156. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  4. Bohnenberger, T., Jameson, A., Krüger, A., Butz, A.: Location-Aware Shopping Assistance: Evaluation of a Decision-Theoretic Approach. In: Paternó, F. (ed.) Mobile HCI 2002. LNCS, vol. 2411, p. 155. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  5. Brewster, S.: Overcoming the Lack of Screen Space on Mobile Computers. Personal and Ubiquitous Computing 6, 188–205 (2002)

    Article  Google Scholar 

  6. Esbjörnsson, M., Juhlin, O., Östergren, M.: Motorcyclists Using Hocman – Field Trials on Mobile Interaction. In: Chittaro, L. (ed.) Mobile HCI 2003. LNCS, vol. 2795, Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  7. Graham, R., Carter, C.: Comparison of Speech Input and Manual Control of In-Car Devices while on-the-move. In: Proceedings of the Second Workshop on Human Computer Interaction with Mobile Devices, Mobile HCI 1999, Edinburgh, Scotland (1999)

    Google Scholar 

  8. Johnson, P.: Usability and Mobility; Interactions on the move. In: Proceedings of the First Workshop on Human-Computer Interaction with Mobile Devices, Glasgow, Scotland, GIST Technical Report G98-1 (1998)

    Google Scholar 

  9. Kjeldskov, J., Stage, J.: New Techniques for Usability Evaluation of Mobile Systems. International Journal of Human-Computer Studies (IJHCS) 60, 599–620 (2004)

    Article  Google Scholar 

  10. Kjeldskov, J., Graham, C.: A Review of Mobile HCI Research Methods. In: Chittaro, L. (ed.) Mobile HCI 2003. LNCS, vol. 2795, Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  11. Kjeldskov, J., Skov, M.B.: Creating a Realistic Laboratory Setting: A Comparative Study of Three Think-Aloud Usability Evaluations of a Mobile System. In: Proceedings of the 9th IFIP TC13 International Conference on Human Computer Interaction, Interact 2003, Zürich, Switzerland, IOS Press, Amsterdam (2003)

    Google Scholar 

  12. Lai, J., Cheng, K., Green, P., Tsimhoni, O.: On the Road and on the Web? Comprehension of synthetic speech while driving. In: Proceedings of CHI 2001, pp. 206–212. ACM, New York (2001)

    Google Scholar 

  13. Molich, R.: Usable Web Design (In Danish). Ingeniøren | bøger (2000)

    Google Scholar 

  14. Nielsen, C.: Testing in the Field. In: Proceedings of the third Asia Pacific Computer Human Interaction Conference, IEEE Computer Society, Los Alamitos (1998)

    Google Scholar 

  15. Nielsen, J.: Usability Engineering. Morgan Kaufmann, San Francisco (1993)

    MATH  Google Scholar 

  16. Pascoe, J., Ryan, N., Morse, D.: Using While Moving: HCI Issues in Fieldwork Environments. Transactions on Computer-Human Interaction 7(3), 417–437 (2000)

    Article  Google Scholar 

  17. Pirhonen, A., Brewster, S.A., Holguin, C.: Gestural and audio metaphors as a means of control for mobile devices. In: Proceedings of CHI 2002, ACM, New York (2002)

    Google Scholar 

  18. Rantanen, J., Impio, J., Karinsalo, T., Reho, A., Tasanen, M., Vanhala, J.: Smart Clothing Prototype for the Artic Environment. Personal and Ubiquitous Computing 6, 3–16 (2002)

    Article  Google Scholar 

  19. Rubin, J.: Handbook of Usability Testing. Wiley, Chichester (1994)

    Google Scholar 

  20. Salvucci, D.D.: Predicting the Effects of In-Car Interfaces on Driver Behaviour using a Cognitive Architecture. In: Proceedings of CHI 2001, New York, pp. 120–127 (2001)

    Google Scholar 

  21. Schilit, B.N., Theimer, M.M.: Disseminating Active Map information to Mobile Hosts. IEEE Network 8(5), 22–32 (1994)

    Article  Google Scholar 

  22. Skov, M.B., Høegh, R.T.: Supporting Cooperation and Coordination in a Hospital Ward by a Context-Aware Mobile Electronic Patient Record. Submitted to the Journal of Computer-Supported Cooperative Work, Elsevier (2004)

    Google Scholar 

  23. Urban, M., Kunath, M.: Design, Deployment and Evaluation of a Clinical Information System which uses Mobile Computers and Workstations. In: Proceedings of the 2nd conference on Mobile Computing in Medicine, Heidelberg (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kjeldskov, J., Skov, M.B., Als, B.S., Høegh, R.T. (2004). Is It Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field. In: Brewster, S., Dunlop, M. (eds) Mobile Human-Computer Interaction - MobileHCI 2004. Mobile HCI 2004. Lecture Notes in Computer Science, vol 3160. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28637-0_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-28637-0_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-23086-1

  • Online ISBN: 978-3-540-28637-0

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics