A Comparative Analysis of Usability Evaluation Methods on Their Versatility in the Face of Diversified User Input Methods
Every command consists of an action and an object, suggesting that a usability problem can occur whenever the user is unable to identify an appropriate action and/or the object associated with his/her current goal. The recent shift from mouse-based to touch-based interaction demands that any usability evaluation method be sensitive to not only object-related but also action-related usability problems. This study involved a total of 32 participants, four kinds of tasks differing in the difficulty of identifying objects and executing actions, and four qualitative methods of usability evaluation. Analyses of sets of observation data with concurrent and retrospective protocol by the same participant and interpretive protocol by a new participant indicate that while the oral instruction method seems least appropriate, the newly-devised narration method seems to have better prospects than the observation and the think aloud method for the usability evaluation of touch-based interaction.
KeywordsUsability evaluation method Qualitative method Touch-based interaction Mouse-based interaction
This research was supported by JSPS KAKENHI Grant Number 25510018. Daiju Ishikawa is currently at Marubeni Information Systems Co., Ltd.
- 1.Carroll, J.M., Mack, R.L.: Learning to use a word processor: by doing, by thinking, and by knowing. In: Thomas, J.C., Schneider, M.L. (eds.) Human Factors in Computer Systems, pp. 13–51. Ablex, Norwood (1984)Google Scholar
- 2.Elling, S., Lentz, L., de Jong, M.: Retrospective think-aloud method: using eye movements as an extra cue for participants’ verbalizations. In: CHI 2011 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1161–1170. ACM, New York (2011)Google Scholar
- 7.Olmsted-Hawala, E.L., Murphy, E.D., Hawala, S., Ashenfelter, K.T.: Think-aloud protocols: a comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability. In: CHI 2010 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2381–2390. ACM, New York (2010)Google Scholar