Are Ten Participants Enough for Evaluating Information Scent of Web Page Hyperlinks?
Information scent of hyperlinks, that is the user’s assessment of semantic relevance of navigation options in a webpage, has been identified as a critical factor in Web navigation. An important question in this context is to identify the minimum number of participants required to measure reliably information scent. A two phase study was conducted in an attempt to provide an answer to this question. In the first phase, involving 101 participants, ratings produced by different size subsets of participants were compared to those of the whole set. In the second phase, the ratings of these different size subsets of participants where compared with measures of behavior of 54 participants, who performed the same information navigation tasks using a typical web browser. Results indicate that representative estimates of information scent can be obtained from 10 participants in both cases. This finding has important implications for future scent-related studies.
KeywordsInformation scent Web usability study cost-benefit analysis
- 1.Fu, W., Pirolli, P.: SNIF-ACT: a cognitive model of user navigation on the world wide web. Human-Computer Interaction 22(4), 355–412 (2007)Google Scholar
- 2.Spool, J., Perfetti, C., Brittan, D.: Design for the Scent of Information. UIE Fundamentals, User Interface Engineering (2004)Google Scholar
- 3.Kaur, I., Hornof, A.J.: A comparison of LSA, wordNet and PMI-IR for predicting user click behavior. In: Proceedings of CHI 2005, USA, pp. 51–60. ACM Press, New York (2005)Google Scholar
- 4.Blackmon, M., Kitajima, M., Polson, P.: Tool for accurately predicting website navigation problems, non-problems, problem severity, and effectiveness of repairs. In: Proceedings of CHI 2005, USA, pp. 31–40. ACM Press, New York (2005)Google Scholar
- 5.Katsanos, C., Tselios, N., Avouris, N.: InfoScent evaluator: a semi-automated tool to evaluate semantic appropriateness of hyperlinks in a web site. In: Proceedings of OZCHI, Sydney, Australia, pp. 373–376. ACM Press, New York (2006)Google Scholar
- 9.Tullis, T., Albert, W.: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Morgan Kaufmann, San Francisco (2008)Google Scholar
- 10.Poole, A., Ball, L.J.: Eye Tracking in Human-Computer Interaction and Usability Research: Current Status and Future Prospects. In: Ghaoui, C. (ed.) Encyclopedia of Human Computer Interaction, pp. 211–219. Idea Group Reference (2006)Google Scholar