Investigation of Indirect Oral Operation Method for Think Aloud Usability Testing

  • Masahiro Hori
  • Yasunori Kihara
  • Takashi Kato
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6776)


Usability testing with prototypes is typically conducted with a concurrent think-aloud protocol. Due to the simultaneous process of prototype operation and verbalization, participants of the think-aloud testing sometimes say very little and are likely to become silent when they are required to think abstractly or complete complex tasks. In this paper, we propose a method of user operation with oral instruction, which facilitates thinking aloud because oral operation would help participants to keep a continuous flow of verbalization. To investigate the quantity and quality of utterances made during think aloud protocols, we conducted a comparative study between oral and conventional manual operation methods. The study was carried out with two test objects: an interactive prototype of a touch-screen digital camera and photo album software with standard mouse/keyboard user interface. Our results demonstrated that the oral operation method was more effective in drawing more utterances for explanation and observation that would be an important source of discovering usability problems although the effect was dependent on the user interface of test objects.


Concurrent think-aloud protocols keep talking usability testing prototype evaluation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vredenburg, K., Mao, J.-Y., Simith, P.W., Cary, T.: A survey of user-centered design practice. In: Proceedings of CHI 2002, Minneapolis, Minnesota, pp. 471–478 (2002)Google Scholar
  2. 2.
    Arnowitz, J., Arent, M., Berger, N.: Effective Prototyping for Software Makers. Morgan Kaufmann, San Francisco (2007)Google Scholar
  3. 3.
    Berger, N., Arent, M., Arnowitz, J., Sampson, F.: Effective Prototyping with Excel: A practical Handbook for Developers and Designers. Morgan Kaufmann, San Francisco (2009)Google Scholar
  4. 4.
    Hennipman, E.-J., Oppelaar, E.-J.R.G., van der Veer, G.C., Bongers, B.: Rapid and rich prototyping: Proof of concepts for experience. In: Proceedings of the 15th European Conference on Cognitive Ergonomics, Madeira, Portugal (2008)Google Scholar
  5. 5.
    Lim, Y.-K., Pangam, A., Periyasami, S., Aneja, S.: Comparative analysis of high- and low-fidelity prototypes for more valid usability evaluations of mobile devices. In: Proceedings of NordiCHI 2006, pp. 291–300 (2006)Google Scholar
  6. 6.
    Snyder, C.: Paper Prototyping: The Fast and Easy Way to Define and Refine User Interfaces. Morgan Kaufmann Publishers, San Francisco (2003)Google Scholar
  7. 7.
    Virzi, R.A., Sokolov, J.L., Karis, D.: Usability Problem Identification Using Both Low- and High-Fidelity Prototypes. In: Proceedings of CHI 1996, Vancouver, BC Canada, pp. 236–243 (1996)Google Scholar
  8. 8.
    Boren, M.T., Ramey, J.: Thinking aloud: Reconciling theory and practice. IEEE Trans. on Professional Communication 43(3), 261–278 (2000)CrossRefGoogle Scholar
  9. 9.
    Van den Haak, M.J., de Jong, M.D.T., Schellens, P.J.: Retrospective versus concurrent think-aloud protocols: Testing the usability of an online library catalogue. Behaviour & Information Technology 22(5), 339–351 (2003)CrossRefGoogle Scholar
  10. 10.
    Nielsen, J., Clemmensen, T., Yssing, C.: Getting access to what goes on in people’s heads? Reflections on the think-aloud technique. In: Proceedings of NordiCHI, pp. 101–110 (2002)Google Scholar
  11. 11.
    Krahmer, E., Ummelen, N.: Thinking about thinking aloud: A comparison of two verbal protocols for usability testing. IEEE Trans. on Professional Communication 47(2), 105–117 (2004)CrossRefGoogle Scholar
  12. 12.
    Cooke, L.: Assessing concurrent think-aloud protocol as a usability test method: A technical communication approach. IEEE Trans. on Professional Communication 53(3), 202–215 (2010)CrossRefGoogle Scholar
  13. 13.
    Ericcson, K.A., Simon, H.A.: Protocol Analysis: Verbal Reports as Data, revised ed. MIT Press, Cambridge (1993)Google Scholar
  14. 14.
    Buur, J., Bagger, K.: Replacing usability testing with user dialogue - How a Danish manufacturing company enhanced its product design process by supporting user participation. Communications of the ACM 42(5), 63–66 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Masahiro Hori
    • 1
  • Yasunori Kihara
    • 1
  • Takashi Kato
    • 1
  1. 1.Graduate School of InformaticsKansai UniversityOsakaJapan

Personalised recommendations