Investigation of Indirect Oral Operation Method for Think Aloud Usability Testing
Usability testing with prototypes is typically conducted with a concurrent think-aloud protocol. Due to the simultaneous process of prototype operation and verbalization, participants of the think-aloud testing sometimes say very little and are likely to become silent when they are required to think abstractly or complete complex tasks. In this paper, we propose a method of user operation with oral instruction, which facilitates thinking aloud because oral operation would help participants to keep a continuous flow of verbalization. To investigate the quantity and quality of utterances made during think aloud protocols, we conducted a comparative study between oral and conventional manual operation methods. The study was carried out with two test objects: an interactive prototype of a touch-screen digital camera and photo album software with standard mouse/keyboard user interface. Our results demonstrated that the oral operation method was more effective in drawing more utterances for explanation and observation that would be an important source of discovering usability problems although the effect was dependent on the user interface of test objects.
KeywordsConcurrent think-aloud protocols keep talking usability testing prototype evaluation
Unable to display preview. Download preview PDF.
- 1.Vredenburg, K., Mao, J.-Y., Simith, P.W., Cary, T.: A survey of user-centered design practice. In: Proceedings of CHI 2002, Minneapolis, Minnesota, pp. 471–478 (2002)Google Scholar
- 2.Arnowitz, J., Arent, M., Berger, N.: Effective Prototyping for Software Makers. Morgan Kaufmann, San Francisco (2007)Google Scholar
- 3.Berger, N., Arent, M., Arnowitz, J., Sampson, F.: Effective Prototyping with Excel: A practical Handbook for Developers and Designers. Morgan Kaufmann, San Francisco (2009)Google Scholar
- 4.Hennipman, E.-J., Oppelaar, E.-J.R.G., van der Veer, G.C., Bongers, B.: Rapid and rich prototyping: Proof of concepts for experience. In: Proceedings of the 15th European Conference on Cognitive Ergonomics, Madeira, Portugal (2008)Google Scholar
- 5.Lim, Y.-K., Pangam, A., Periyasami, S., Aneja, S.: Comparative analysis of high- and low-fidelity prototypes for more valid usability evaluations of mobile devices. In: Proceedings of NordiCHI 2006, pp. 291–300 (2006)Google Scholar
- 6.Snyder, C.: Paper Prototyping: The Fast and Easy Way to Define and Refine User Interfaces. Morgan Kaufmann Publishers, San Francisco (2003)Google Scholar
- 7.Virzi, R.A., Sokolov, J.L., Karis, D.: Usability Problem Identification Using Both Low- and High-Fidelity Prototypes. In: Proceedings of CHI 1996, Vancouver, BC Canada, pp. 236–243 (1996)Google Scholar
- 10.Nielsen, J., Clemmensen, T., Yssing, C.: Getting access to what goes on in people’s heads? Reflections on the think-aloud technique. In: Proceedings of NordiCHI, pp. 101–110 (2002)Google Scholar
- 13.Ericcson, K.A., Simon, H.A.: Protocol Analysis: Verbal Reports as Data, revised ed. MIT Press, Cambridge (1993)Google Scholar