Abstract
How should Ubicomp technologies be evaluated? While lab studies are good at sensing aspects of human behavior and revealing usability problems, they are poor at capturing context of use. In-situ studies are good at demonstrating how people appropriate technologies in their intended setting, but are expensive and difficult to conduct. Here, we show how they can be used more productively in the design process. A mobile learning device was developed to support teams of students carrying out scientific inquiry in the field. An initial in-situ study showed it was not used in the way envisioned. A contextualized analysis led to a comprehensive understanding of the user experience, usability and context of use, leading to a substantial redesign. A second in-situ study showed a big improvement in device usability and collaborative learning. We discuss the findings and conclude how in-situ studies can play an important role in the design and evaluation of Ubicomp applications and user experiences.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bell, G.: No More SMS from Jesus: Ubicomp, Religion and Techno-spiritual Practices. In: Dourish, P., Friday, A. (eds.) UbiComp 2006. LNCS, vol. 4206, pp. 141–158. Springer, Heidelberg (2006)
Bell, M., Chalmers, M., Barkhuus, L., Hall, M., Sherwood, S., Tennent, P., Brown, B., Rowland, D., Benford, S., Hampshire, A., Captra, M.: Interweaving mobile games with everyday life. In: Proc. of CHI, pp. 417–426 (2006)
Bellotti, V., Back, M., Edwards, K., Grinter, R., Henderson, A., Lopes, C.: Making sense of sensing systems: five questions for designers and researchers. In: Proc. of CHI, pp. 415–422 (2002)
Benford, S., Seager, W., Flintham, M., Anastasi, R., Rowland, D., Humble, J., Stanton, S., Bowers, J., Tandavanitj, N., Adams, M., Farr, J.R., Oldroyd, A., Sutton, J.: The error of our ways: the experience of self-reported position in a location-based game. In: Davies, N., Mynatt, E.D., Siio, I. (eds.) UbiComp 2004. LNCS, vol. 3205, pp. 721–730. Springer, Heidelberg (2004)
Benford, S., Schanädelbach, H., Koleva, B., Anastasi, R., Greenhalgh, C., Rodden, T., Green, J., Ghali, A., Pridmore, T., Gaver, B., Boucher, A., Walker, B., Pennington, S., Schmidt, A., Gellersen, H., Steed, A.: Expected, sensed, and desired: A framework for designing sensing-based interaction. Proc. of TOCHI, 12(1), 3–30 (2005)
Boucher, A., Gaver, W.: Developing the drift table. Interactions 13(1), 24–27 (2006)
Brignull, H., Izadi, S., Fitzpatrick, G., Rogers, Y., Rodden, T.: The introduction of a shared interactive surface into a communal space. In: Proc. of CSCW, pp. 49–58 (2004)
Consolvo, S., Walker, M.: Using the experience sampling method to evaluate Ubicomp applications. IEEE Pervasive Computing Mobile and Ubiquitous Systems 2(2), 24–31 (2003)
Consolvo, S., Everitt, K., Smith, I., Landay, J.: Design requirements for technologies that motivate physical activity. In: Proc. of CHI, pp. 457–466 (2006)
Crabtree, A., Rodden, T.: Domestic routines and design for the home. Computer Supported Cooperative Work: The Journal of Collaborative Computing 13(2), 191–220 (2004)
Dumas, J.S., Redish, J.C.: A Practical Guide to Usability Testing. Ablex, Norwood, NJ (1994)
Gaver, W., Dunne, T., Pacenti, E.: Cultural probes and the value of uncertainty. Interactions 11(5), 53–56 (2004)
Hughes, J.A., Randall, D., Shapiro, D.: Faltering from ethnography to design. In: Proc. of CSCW, pp. 115–122 (1992)
Hull, R., Reid, J., Geelhoed, E.: Creating experiences with wearable computing. IEEE Pervasive Computing 1(4), 56–61 (2002)
Hutchins, E., Klausen, T.: Distributed Cognition in an Airline Cockpit. In: Middleton, Engeström, Y. (eds.) Communication and Cognition at Work, pp. 15–34. Cambridge University Press, D. Cambridg (1996)
Intille, S., Larson, K., Tapia, E., Beaudin, J., Kaushik, P., Nawyn, J., Rockinson, R.: Using a live-in laboratory for ubiquitous computing research. In: Fishkin, K.P., Schiele, B., Nixon, P., Quigley, A. (eds.) PERVASIVE 2006. LNCS, vol. 3968, pp. 349–365. Springer, Heidelberg (2006)
Kidd, C., Orr, R., Abowd, G., Atkeson, C., Essa, I., MacIntyre, B., Mynatt, E., Starner, T.: The Aware Home: A Living Laboratory for Ubiquitous Computing Research. In: Streitz, N.A., Hartkopf, V. (eds.) CoBuild 1999. LNCS, vol. 1670, pp. 191–198. Springer, Heidelberg (1999)
Kjeldskov, J., Skov, M., Als, B., Høegh, R.: Is it worth the hassle? Exploring the added value of evaluating the usability of context-aware mobile systems in the field. In: Brewster, S., Dunlop, M.D. (eds.) Mobile Human-Computer Interaction – MobileHCI 2004. LNCS, vol. 3160, pp. 61–73. Springer, Heidelberg (2004)
Larkin, J., Simon, H.: Why a diagram is (sometimes) worth ten thousand words. Cognitive Science 11, 65–99 (1987)
Lin, J., Mamykina, L., Lindtner, S., Delajoux, G., Strub, H.: Fish’n’Steps: Encouraging Activity with an Interactive Computer Game. In: Dourish, P., Friday, A. (eds.) UbiComp 2006. LNCS, vol. 4206, pp. 261–278. Springer, Heidelberg (2006)
Mankoff, J., Dey, A., Hsieh, G., Kientz, J., Lederer, J., Ames, M.: Heuristic evaluation of ambient displays. In: Proc. of CHI, pp. 169–176 (2003)
Nielsen, J.: Usability engineering at a discount. In: Salvendy, G., Smith, M.J. (eds.) Human-Computer interaction on Designing and Using Human-Computer Interfaces and Knowledge Based Systems, pp. 394–401 (1989)
Olsen, A., Rogers, Y., Sharp, H.: The Snap Method. In: Workshop Proceedings, Designing Methods for New Users, Technologies, and Design Processes, CHI (2007)
Rogers, Y., Connelly, K., Tedesco, L., Hazlewood, W.R.: Mobile technologies for integrated scientific inquiry. Journal of Learning Sciences (submitted)
Rogers, Y., Price, S., Randell, C., Stanton-Fraser, D., Weal, M., Fitzpatrick., G.: Ubi-learning: Integrating outdoor and indoor learning experiences. Comm. of ACM 48(1), 55–59 (2005)
Rowan, G., Mynatt, E.: Digital Family Portrait Field Trial: Support for Aging in Place. In: Proc. of CHI, pp. 521–530 (2005)
Salovaara, A., Jacucci, G., Oulasvirta, A., Saari, T., Kanerva, P., Kurvinen, E., Tiitta, S.: Collective creation and sense-making of mobile media. In: Proc. of CHI, pp. 1211–1220 (2006)
Scholtz, J., Consolvo, S.: Toward a Framework for Evaluating Ubiquitous Computing Applications. Pervasive Computing 3(2), 82–88 (2004)
Sharp, H., Rogers, Y., Preece, J.: Interaction Design, 2nd edn. Wiley, Chichester
Taylor, A.S., Swan, L.: Artful systems in the home. In: Proc. CHI, pp. 641–650 (2005)
Toscos, T., Faber, A., An, S., Gandhi, M.: Chick Clique: persuasive technology to motivate teenage girls to exercise. In: Proc. of CHI, pp. 1873–1878 (2006)
Winograd, T.: Bringing Design to Software. Addison Wesley, Reading (1996)
Yeh, R., Liao, C., Klemmer, S., Guimbretière, F., Lee, B., Kakaradov, B., Stamberger, J., Paepcke, A.: ButterflyNet: a mobile capture and access system for field biology research. In: Proc. of CHI, pp. 571–580 (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rogers, Y. et al. (2007). Why It’s Worth the Hassle: The Value of In-Situ Studies When Designing Ubicomp. In: Krumm, J., Abowd, G.D., Seneviratne, A., Strang, T. (eds) UbiComp 2007: Ubiquitous Computing. UbiComp 2007. Lecture Notes in Computer Science, vol 4717. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74853-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-540-74853-3_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74852-6
Online ISBN: 978-3-540-74853-3
eBook Packages: Computer ScienceComputer Science (R0)