Crowdsourced Web Site Evaluation with CrowdStudy

  • Michael Nebeling
  • Maximilian Speicher
  • Michael Grossniklaus
  • Moira C. Norrie
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7387)


Many different automatic usability evaluation tools have been specifically developed for web sites and web-based services, but they usually cannot replace user testing. At the same time, traditional usability evaluation methods can be both expensive and time consuming. We will demonstrate CrowdStudy, a toolkit for crowdsourced testing of web interfaces that allows, not only to efficiently recruit larger amounts of test users, but also to evaluate web sites under many different conditions.


Web usability web site evaluation crowdsourcing 


  1. 1.
    Matera, M., Rizzo, F., Carughi, G.: Web Usability: Principles and Evaluation Methods. Web Engineering (2006)Google Scholar
  2. 2.
    Insfran, E., Fernandez, A.: A Systematic Review of Usability Evaluation in Web Development. In: Hartmann, S., Zhou, X., Kirchberg, M. (eds.) WISE 2008. LNCS, vol. 5176, pp. 81–91. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Atterer, R., Wnuk, M., Schmidt, A.: Knowing the User’s Every Move – User Activity Tracking for Website Usability Evaluation and Implicit Interaction. In: Proc. WWW (2006)Google Scholar
  4. 4.
    Hong, J.I., Heer, J., Waterson, S., Landay, J.A.: WebQuilt: A Proxy-based Approach to Remote Web Usability Testing. TOIS 19(3) (2001)Google Scholar
  5. 5.
    Kittur, A., Chi, E.H., Suh, B.: Crowdsourcing User Studies With Mechanical Turk. In: Proc. CHI (2008)Google Scholar
  6. 6.
    Nebeling, M., Norrie, M.C.: Tools and Architectural Support for Crowdsourced Adaptation of Web Interfaces. In: Auer, S., Díaz, O., Papadopoulos, G.A. (eds.) ICWE 2011. LNCS, vol. 6757, pp. 243–257. Springer, Heidelberg (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Michael Nebeling
    • 1
  • Maximilian Speicher
    • 1
  • Michael Grossniklaus
    • 2
  • Moira C. Norrie
    • 1
  1. 1.Institute of Information SystemsETH ZurichZurichSwitzerland
  2. 2.Computer Science DepartmentPortland State UniversityPortlandUnited States of America

Personalised recommendations