Towards a Generic Framework for Automatic Measurements of Web Usability Using Affective Computing Techniques

  • Payam Aghaei Pour
  • Rafael A. Calvo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6974)


We propose a generic framework for the automatic usability evaluation of web sites by combining traditional automatic usability methods with affective computing techniques. To evaluate a framework a pilot study was carried out where users (n=4) reported their affective states using dimensional and categorical models. Binary task completion, time, mouse clicks, and error rates as an indicator of web usability were automatically captured for each page. Results suggested that frustration experienced when error rates and time for the task were higher. Delight on the other hand was at the other side of the spectrum. In the case that usability measurements had almost same values (e.g. confusing or engaging pages), affective states may be a way to show the difference.


affective computing web usability framework 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cato, J.: User-centered web design. Pearson Education, London (2001)Google Scholar
  2. 2.
    Nielsen, J.: Usability engineering. Morgan Kaufmann, San Francisco (1993)zbMATHGoogle Scholar
  3. 3.
    Paternò, F., Santoro, C.: Remote Usability Evaluation: Discussion of a General Framework and Experiences from Research with a Specific Tool. Maturing Usability, 197–221 (2008)Google Scholar
  4. 4.
    Ivory, M.Y., Hearst, M.A.: The state of the art in automating usability evaluation of user interfaces. ACM Computing Surveys (CSUR) 33, 470–516 (2001)CrossRefGoogle Scholar
  5. 5.
    Paternò, F., Piruzza, A., Santoro, C.: Remote Web usability evaluation exploiting multimodal information on user behavior. In: Computer-Aided Design of User Interfaces V, pp. 287–298 (2007)Google Scholar
  6. 6.
    Picard, R.: Affective computing. The MIT Press, Cambridge (1997)CrossRefGoogle Scholar
  7. 7.
    Dolan, R.J.: Emotion, cognition, and behavior. Science 298, 1191 (2002)CrossRefGoogle Scholar
  8. 8.
    Aghaei Pour, P., Hussain, M., AlZoubi, O., D’Mello, S., Calvo, R.: The Impact of System Feedback on Learners Affective and Physiological States, pp. 264–273. Springer, Heidelberg (2010)Google Scholar
  9. 9.
    Klein, J., Moon, Y., Picard, R.: This computer responds to user frustration: Theory, design, and results. Interacting with Computers 14, 119–140 (2002)CrossRefGoogle Scholar
  10. 10.
    D’Mello, S., Craig, S., Gholson, B., Franklin, S., Picard, R., Graesser, A.: Integrating affect sensors in an intelligent tutoring system, pp. 7–13 (2005) Google Scholar
  11. 11.
    Bianchi-Berthouze, N., Lisetti, C.L.: Modeling multimodal expression of user’s affective subjective experience. User Modeling and User-Adapted Interaction 12, 49–84 (2002)CrossRefzbMATHGoogle Scholar
  12. 12.
    Calvo, R.A., D’Mello, S.: Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 18–37 (2010) Google Scholar
  13. 13.
    Iso, I.: 9241-11. Ergonomic requirements for office work with visual display terminals (VDT’s). Part 11 (1997) Google Scholar
  14. 14.
    Hornbæk, K.: Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies 64, 79–102 (2006)CrossRefGoogle Scholar
  15. 15.
    Petty, R.E., Fabrigar, L.R., Wegener, D.T.: Emotional factors in attitudes and persuasion (2003) Google Scholar
  16. 16.
    Sauro, J., Kindlund, E.: A method to standardize usability metrics into a single score, pp. 401–409. ACM, New York (2005)Google Scholar
  17. 17.
    Edwardson, M.: Measuring consumer emotions in service encounters: an exploratory analysis. Australasian Journal of Market Research 6, 34–48 (1998)Google Scholar
  18. 18.
    Partala, T., Kangaskorte, R.: The Combined Walkthrough: Measuring Behavioral, Affective, and Cognitive Information in Usability Testing. Journal of Usability Studies 5, 21–33 (2009)Google Scholar
  19. 19.
    Frøkjær, E., Hertzum, M., Hornbæk, K.: Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?, pp. 345–352. ACM, New York (2000)Google Scholar
  20. 20.
    D’Mello, S., Graesser, A., Picard, R.: Toward an affect-sensitive AutoTutor. IEEE Intelligent Systems 22, 53–61 (2007)CrossRefGoogle Scholar
  21. 21.
    Arroyo, I., Cooper, D., Burleson, W., Woolf, B., Muldner, K., Christopherson, R.: Emotion Sensors go to School (2009)Google Scholar
  22. 22.
    Calvo, R.A., D’Mello, S.K.: Affect Detection: An Interdisciplinary Review of Models, Methods, and their Applications. IEEE Transactions on Affective Computing 1, 18–37 (2010)CrossRefGoogle Scholar
  23. 23.
    Vargas, A., Weffers, H., da Rocha, H.V.: A method for remote and semi-automatic usability evaluation of web-based applications through users behavior analysis, p. 19. ACM, New York (2010)Google Scholar
  24. 24.
    Calvo, R.A., O’Rourke, S.T., Jones, J., Yacef, K., Reimann, P.: Collaborative Writing Support Tools on the Cloud. IEEE Transactions on Learning Technologies 4, 88–97 (2011)CrossRefGoogle Scholar
  25. 25.
    Desmet, P.: Emotion through expression; designing mobile telephones with an emotional fit. Report of Modeling the Evaluation Structure of KANSEI 3, 103–110 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Payam Aghaei Pour
    • 1
  • Rafael A. Calvo
    • 1
  1. 1.School of Electrical and Information EngineeringUniversity of SydneyAustralia

Personalised recommendations