Abstract
There is a long-running debate about how to best evaluate mobile location-based services with users: in the lab or in the field? In this paper, we investigate how to combine benefits of both methods using an Immersive Video Environment (IVE), providing a convincing audio-visual simulation of real-world settings. We contrast three methods to evaluate mobile navigation systems: one in the field, one in the lab and one “hybrid” solution (IVE). We found that using the IVE allowed us to identify nearly the same number of major usability problems as the field test. We also observed similarities between the field study and the IVE study in terms of participants’ performance, which provides initial evidence that in some settings, an IVE study may yield results comparable to a field study.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Betiol A, Cybis AW (2005) Usability testing of mobile devices: a comparison of three approaches. In: Proceedings of interact 2005. Springer, Berlin, pp 470–481
Chervest K, Mitchell K, Davies N (2002) The role of adaptive hypermedia in a context-aware tourist guide. Commun ACM-The Adapt Web 45(5):47–51
Delikostidis I, van Elzakker CPJM (2009) Geo-identification and pedestrian navigation with geo-mobile applications: how do users proceed? In: Proceedings of the 5th international conference on location based services and telecartography. Springer, Berlin, pp 185–206
Duh HBL, Tan CBG, Chen VHH (2006) Usability evaluation for mobile device: a comparison of laboratory and field tests. In: Proceedings of mobile HCI ’06. ACM, New York, pp 181–186
Goodman J, Brewster S, Gray P (2004) Using field experiments to evaluate mobile guides. In: Proceedings of HCI in mobile guides, workshop at mobile HCI 2004. Springer, Glaskow, pp 38–48
Kaikkoken A, Kekäläinen A, Cankar M, Kallio T, Kankainen A (2008) Will laboratory test results be valid in mobile contexts? In: Lumsden J (ed) Handbook of research on user interface design and evaluation for mobile technology. Information Science Reference, Hersey, pp 897–909
Kimber J, Georgievski M, Sharda N (2005) Developing usability testing systems and procedures for mobile tourism services. In: Proceedings of the annual conference on information technology in the hospitality industry, HITA 2005. Los Angeles, USA, pp 79–96
Kjeldskov J, Paay J (2012) A longitudinal review of mobile HCI research methods. In: Proceedings of Mobile HCI ’12. ACM Press, San Francisco, pp 69–78
Kjeldskov J, Skov MB, Als BS, Høegh RT (2004) Is it worth the hassle? Exploring the added value of evaluating the usability of context-aware mobile systems in the field. In: Proceedings of mobile HCI ‘04. Springer, Berlin, pp 61–73
Loomis JM, Blascovich JJ, Beall AC (1999) Immersive virtual environment technology as a basic research tool in psychology. Behav Res Methods Instrum Comput 31(4):557–564
Jensen KL, Larsen LB (2008) The challenges of evaluating the mobile and ubiquitous user experience. In: Proceedings of 2nd international workshop on improved mobile user experience, pp 198–207
Lumsden J (2008) Handbook of research on user interface design and evaluation for mobile technology. Information Science Reference, Hershey
Lund AM (2008) Measuring usability with the use questionnaire. http://www.stcsig.org/usability/newsletter/0110_measuring_with_use.html. Accessed 17 May 2014
Nielsen CM, Overgaard M, Pedersen MB, Stage J, Stenild S (2006) It’s worth the hassle!: the added value of evaluating the usability of mobile systems in the field. NordiCHI ’06. ACM, New York, pp 272–280
Nielsen J (1994) Estimating the number of subjects needed for a think aloud test. Int J Hum Comput Stud 41(3):385–397
Ostkamp M, Kray C (2014) Supporting design, prototyping, and evaluation of public display systems. EICS’2014. ACM, New York, pp 263–272
Palen L, Salzman M, Youngs E (2000) Going wireless: behavior and practice of new mobile phone users. In: Proceedings of CSCW ’00. ACM, Philadelphia, pp 201–210
Rogers Y (2011) Interaction design gone wild: striving for wild theory. Interactions 18(4):58–62
Roto V, Oulasvirta A, Haikarainen T, Kuorelahti J, Lehmuskallio H, Nyyssönen T (2004) Examining mobile phone use in the wild with Quasi-experimentation. Helsinki Institute for Information Technology, Finland
Schellenbach M, Lövdén M, Verrel J, Krüger A, Lindenberger U (2009) Adult age differences in familiarization to treadmill walking within virtual environments. Gait Posture 31(3):295–299
Singh P, Ha NH, Kuang Z, Olivier P, Kray C, Blythe P, James P (2006) Immersive video as a rapid prototyping and evaluation tool for mobile and ambient applications. In: Proceedings of mobile HCI ’06. ACM, New York, p 264
Snowdon C, Kray C (2009) Exploring the use of landmarks for mobile navigation support in natural environments. In: Proceedings of mobile HCI ’09. ACM, New York, pp 1–10
van Elzakker CPJM, Delikostidis I, van Oosterom PPJM (2008) Field-based usability evaluation methodology for mobile geo-applications. The Cartographic J 45(2):139–149
van Veen HAHC, Distler HK, Braun SJ, Bülthoff HH (1998) Navigating through a virtual city: using virtual reality technology to study human action and perception. Future Gener Comput Syst 14(3–4):231–242
Winters JM, Story MF, Campbell S, Lemke M, Danturthi D, Barr A, Rempel DM (2001) Mobile usability lab: a tool for studying medical device accessibility for users with diverse abilities. In: Proceedings of the 27th RESNA conference
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Delikostidis, I., Fritze, H., Fechner, T., Kray, C. (2015). Bridging the Gap Between Field- and Lab-Based User Studies for Location-Based Services. In: Gartner, G., Huang, H. (eds) Progress in Location-Based Services 2014. Lecture Notes in Geoinformation and Cartography. Springer, Cham. https://doi.org/10.1007/978-3-319-11879-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-11879-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11878-9
Online ISBN: 978-3-319-11879-6
eBook Packages: Earth and Environmental ScienceEarth and Environmental Science (R0)