Are Ten Participants Enough for Evaluating Information Scent of Web Page Hyperlinks?

  • Christos Katsanos
  • Nikolaos Tselios
  • Nikolaos Avouris
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5726)


Information scent of hyperlinks, that is the user’s assessment of semantic relevance of navigation options in a webpage, has been identified as a critical factor in Web navigation. An important question in this context is to identify the minimum number of participants required to measure reliably information scent. A two phase study was conducted in an attempt to provide an answer to this question. In the first phase, involving 101 participants, ratings produced by different size subsets of participants were compared to those of the whole set. In the second phase, the ratings of these different size subsets of participants where compared with measures of behavior of 54 participants, who performed the same information navigation tasks using a typical web browser. Results indicate that representative estimates of information scent can be obtained from 10 participants in both cases. This finding has important implications for future scent-related studies.


Information scent Web usability study cost-benefit analysis 


  1. 1.
    Fu, W., Pirolli, P.: SNIF-ACT: a cognitive model of user navigation on the world wide web. Human-Computer Interaction 22(4), 355–412 (2007)Google Scholar
  2. 2.
    Spool, J., Perfetti, C., Brittan, D.: Design for the Scent of Information. UIE Fundamentals, User Interface Engineering (2004)Google Scholar
  3. 3.
    Kaur, I., Hornof, A.J.: A comparison of LSA, wordNet and PMI-IR for predicting user click behavior. In: Proceedings of CHI 2005, USA, pp. 51–60. ACM Press, New York (2005)Google Scholar
  4. 4.
    Blackmon, M., Kitajima, M., Polson, P.: Tool for accurately predicting website navigation problems, non-problems, problem severity, and effectiveness of repairs. In: Proceedings of CHI 2005, USA, pp. 31–40. ACM Press, New York (2005)Google Scholar
  5. 5.
    Katsanos, C., Tselios, N., Avouris, N.: InfoScent evaluator: a semi-automated tool to evaluate semantic appropriateness of hyperlinks in a web site. In: Proceedings of OZCHI, Sydney, Australia, pp. 373–376. ACM Press, New York (2006)Google Scholar
  6. 6.
    Miller, C.S., Remington, R.W.: Modeling information navigation: implications for information architecture. Human-Computer Interaction 19(3), 225–271 (2004)CrossRefGoogle Scholar
  7. 7.
    Brumby, D., Howes, A.: Strategies for Guiding Interactive Search: An Empirical Investigation into the Consequences of Label Relevance for Assessment and Selection. Human-Computer Interaction 23(1), 1–46 (2008)CrossRefGoogle Scholar
  8. 8.
    Caulton, D.A.: Relaxing the homogeneity assumption in usability testing. Behaviour & Information Technology 20(1), 1–7 (2001)CrossRefGoogle Scholar
  9. 9.
    Tullis, T., Albert, W.: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Morgan Kaufmann, San Francisco (2008)Google Scholar
  10. 10.
    Poole, A., Ball, L.J.: Eye Tracking in Human-Computer Interaction and Usability Research: Current Status and Future Prospects. In: Ghaoui, C. (ed.) Encyclopedia of Human Computer Interaction, pp. 211–219. Idea Group Reference (2006)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2009

Authors and Affiliations

  • Christos Katsanos
    • 1
  • Nikolaos Tselios
    • 1
  • Nikolaos Avouris
    • 1
  1. 1.Human-Computer Interaction Group, Electrical and Computer Engineering Dept.University of PatrasRio PatrasGreece

Personalised recommendations