Evaluating User’s Emotional Experience in HCI: The PhysiOBS Approach

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8511)


As computing is changing parameters, apart from effectiveness and efficiency in human-computer interaction, such as emotion have become more relevant than before. In this paper, a new tool-based evaluation approach of user’s emotional experience during human-computer interaction is presented. The proposed approach combines user’s physiological signals, observation data and self-reported data in an innovative tool (PhysiOBS) that allows continuous and multiple emotional states analysis. To the best of our knowledge, such an approach that effectively combines all these user-generated data in the context of user’s emotional experience evaluation does not exist. Results from a preliminary evaluation study of the tool were rather encouraging revealing that the proposed approach can provide valuable insights to user experience practitioners.


User Emotional Experience Human Computer Interaction Evaluation Physiological Signals Emotions 


  1. 1.
    Lazar, J., Jones, A., Shneiderman, B.: Workplace user frustration with computers: An exploratory investigation of the causes and severity. Behaviour & Information Technology 25(2007), 239–251 (2007)Google Scholar
  2. 2.
    Igbaria, M., Chakrabarti, A.: Computer anxiety and attitudes towards microcomputer use. Behaviour & Information Technology 9, 229–241 (1990)CrossRefGoogle Scholar
  3. 3.
    Lazar, J., Feng, J.H., Hochheise, H.: Research Methods in Human-Computer-Interaction. John Wiley & Sons Ltd. (2010)Google Scholar
  4. 4.
    Sanderson, P.M., Fisher, C.: Exploratory Sequential Data Analysis: Foundations. Human-Computer Interaction 9, 251–317 (1994)CrossRefGoogle Scholar
  5. 5.
    Carter, K., Anderson, B.: Can video research escape the technology? Some reflections on the problems and possibilities of A.V. research. In: SIGCHI Bulletin, vol. 27, pp. 112–114 (1989)Google Scholar
  6. 6.
    Ambady, N., Rosenthal, R.: Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. Psychological Bulletin 111, 256–274 (1992)CrossRefGoogle Scholar
  7. 7.
    Bartneck, C., Lyons, M.J.: HCI and the face: Towards an art of the soluble. In: Jacko, J.A. (ed.) Human-Computer Interaction, Part I, HCII 2007. LNCS, vol. 4550, pp. 20–29. Springer, Heidelberg (2007)Google Scholar
  8. 8.
    Kim, J.-H., André, E.: Emotion recognition using physiological and speech signal in short-term observation. In: André, E., Dybkjær, L., Minker, W., Neumann, H., Weber, M. (eds.) PIT 2006. LNCS (LNAI), vol. 4021, pp. 53–64. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Epp, C., Lippold, M., Mandryk, R.L.: Identifying emotional states using keystroke dynamics. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2011), pp. 715–724. ACM, New York (2011)Google Scholar
  10. 10.
    Wagner, J., Jonghwa, K., Andre, E.: From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification. In: IEEE International Conference on Multimedia and Expo, pp. 940–943. IEEE press (2005)Google Scholar
  11. 11.
  12. 12.
  13. 13.
  14. 14.
    Lin, T., Hu, W.: Do Physiological Data Relate To Traditional Usability Indexes? In: 17th Australia conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future, pp. 1–10 (2005)Google Scholar
  15. 15.
    Ward, R.D., Marsden, P.M.: Physiological responses to different WEB page designs. International Journal of Human Computer Studies 59, 199–212 (2003)CrossRefGoogle Scholar
  16. 16.
    Wilson, G., Sasse, M.A.: Do users always know what’s good for them utilising physiological responses to assess media quality. In: McDonald, S., Waern, Y., Cockton, G. (eds.) People and Computers XIV-Usability or Else, pp. 327–339. Springer, London Limited (2000)CrossRefGoogle Scholar
  17. 17.
    Scheirer, J., Fernandez, R., Klein, J., Picard, R.W.: Frustrating the user on purpose: A step toward building an affective computer. Interacting with Computers 14, 93–118 (2002)CrossRefGoogle Scholar
  18. 18.
    Mandryk, R.L., Atkins, M.S.: A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Human-Computer Studies 65, 329–347 (2007)CrossRefGoogle Scholar
  19. 19.
    Kivikangas, J.M., Ekman, I., Chanel, G., Järvelä, S., Salminen, M., Cowley, B., Henttonen, P., Ravaja, N.: Review on Psychophysiological Methods in Game Research. In: Proceedings of 1st Nordic DiGRA. DiGRA (2010)Google Scholar
  20. 20.
    Kramer, A.F.: Physiological Metrics of Mental Workload: A Review of Recent Progress. In: Damos, D.L. (ed.) Multiple-Task Performance, pp. 279–328. Taylor and Francis, London (1991)Google Scholar
  21. 21.
    Ikehara, C.S., Crosby, M.: Physiological Measures Used for Identification of Cognitive States and Continuous Authentication, Atlanta, USA (2010)Google Scholar
  22. 22.
    Bellur, S., Sundar, S.S.: Psychophysiological responses to media interfaces. In: ACM SIGCHI, Atlanta, GA,USA (2010)Google Scholar
  23. 23.
    Calhoun, B.H., Lach, J., Stankovic, J., Wentzloff, D.D., Whitehouse, K., Barth, A.T., Brown, J.K., Li, Q., Oh, S., Roberts, N.E., Zhang, Y.: Body sensor networks: A holistic approach from silicon to users. In: Proceedings of the IEEE, pp. 91–106 (2012)Google Scholar
  24. 24.
    James, W.: Discussion: The physical basis of emotion. Psychological Review 1, 516–529 (1894)CrossRefGoogle Scholar
  25. 25.
    Cacioppo, J.T., Tassinary, L.G.: Inferring psychological significance from physiological signals. American Psychologist 45, 16–28 (1990)CrossRefGoogle Scholar
  26. 26.
    Cacioppo, J.T., Tassinary, L.G., Berntson, G.G.: Psychophysiological science. In: Handbook of Psychophysiology. Cambridge University Press, Cambridge (2000)Google Scholar
  27. 27.
    Schachter, S.: The Interaction of Cognitive and Physiological Determinants of Emotional State. Advances in Experimental Psychology 1, 49–80 (1964)CrossRefGoogle Scholar
  28. 28.
    Ekman, P.: An argument for basic emotions. Cognition and Emotion 6, 169–200 (1993)CrossRefGoogle Scholar
  29. 29.
    Posner, J., Russell, J.A., Peterson, B.S.: The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathololy 17, 715–734 (2005)Google Scholar
  30. 30.
    Branco, P., Firth, P., Encarnac¸ao, L.M., Bonato, P.: Faces of emotion in human–computer interaction. In: CHI 2005 Extended Abstracts on Human Factors in Computing Systems, pp. 1236–1239. ACM Press, New York (2005)Google Scholar
  31. 31.
    Foglia, P., Prete, C.A., Zanda, M.: Relating GSR Signals to traditional Usability Metrics: A Case Study with an anthropomorphic Web Assistant. In: IEEE International Instrumentation & Measurement Technology Conference, Victoria, Canada, pp. 1814–1819 (2008)Google Scholar
  32. 32.
    Picard, R.W., Vyzas, E., Healey, J.: Toward Machine Emotional Intelligence: Analysis of Affective Physiological State. IEEE Transactions on Pattern Analysis And Machine Intelligence 23, 1175–1191 (2001)CrossRefGoogle Scholar
  33. 33.
    Vyzas, E., Picard, R.W.: Affective Pattern Classification, In: Symposium, Emotional and Intelligent: The Tangled Knot of Cognition (October 1998)Google Scholar
  34. 34.
    Gunes, H., Piccardi, M.: Bi-modal emotion recognition from expressive face and body gestures. Journal of Network and Computer Applications 30, 1334–1345 (2007)CrossRefGoogle Scholar
  35. 35.
    Fridlund, A.J., Izard, C.E.: Electromyographic Studies of Facial Expressions of Emotions and Patterns of Emotions. In: Cacioppo, J.T., Petty, R.E. (eds.) Social Psychophysiology: A Sourcebook. Guilford Press, New York (1983)Google Scholar
  36. 36.
    Bartneck, C., Lyons, M.J.: HCI and the face: Towards an art of the soluble. In: Jacko, J.A. (ed.) Human-Computer Interaction, Part I, HCII 2007. LNCS, vol. 4550, pp. 20–29. Springer, Heidelberg (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.School of Science and TechnologyHellenic Open UniversityPatraGreece

Personalised recommendations