Impact of Performance and Subjective Appraisal of Performance on the Assessment of Technical Systems

  • Matthias HaaseEmail author
  • Martin Krippl
  • Mathias Wahl
  • Swantje Ferchow
  • Jörg Frommer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9731)


Technical systems of the future are companion systems. These systems should be individualized, adaptive and accommodating. In order to create this technology, detailed evidence about users’ behavior is needed. In this study, user-specific factors (user performance, subjective appraisal of user performance and user characteristics) are examined in terms of their impact on user satisfaction. In the WOZ experiment “last minute”, 130 subjects interacted with a simulated speech-controlled technical system and had to complete a specific task. Over the course of the experiment, the subjects had to cope with different challenging situations. Using bivariate and point-biserial correlations, significant correlations for age and NEO-FFI personality dimension extraversion on a user’s assessment of the simulated system were determined. Consequently, the postulated model could not be empirically proven, but provides important information for future studies .


Companion system User characteristics Personality traits Wizard of oz experiment User experience User satisfaction 



The presented study is performed in the framework of the Transregional Collaborative Research Centre SFB/TRR 62\A Companion-Technology for Cognitive Technical Systems” funded by the German Research Foundation (DFG). The responsibility for the content of this paper remains with the authors.


  1. 1.
    Wendemuth, A., Biundo, S.: A companion technology for cognitive technical systems. In: Esposito, A., Esposito, A.M., Vinciarelli, A., Hoffmann, R., Müller, V.C. (eds.) COST 2102. LNCS, vol. 7403, pp. 89–103. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  2. 2.
    Rösner, D., Haase, M., Bauer, T., Günther, S., Krüger, J., Frommer, J.: Desiderata for the design of companion systems. KI - Künstliche Intelligenz 30(1), 53–61 (2016)CrossRefGoogle Scholar
  3. 3.
    Krüger, J., Wahl, M., Frommer, J.: Making the system a relational partner: users’ ascriptions in individualization-focused interactions with companion-systems. In: Berntzen, L.: Böhm, S. (eds.) Proceedings of the 8th International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services (CENTRIC 2015), pp. 48–54. IARIA XPS Press (2015)Google Scholar
  4. 4.
    Haase, M., Krippl, M., Ferchow, S., Otto, M., Frommer, J.: Influence of user characteristics on coping with stress. In: Human-Computer Interaction. LNCS. Springer, Berlin (2016). Submission ID: 611Google Scholar
  5. 5.
    Bailey, J.E., Pearson, S.W.: Development of a tool for measuring and analyzing computer user satisfaction. Manage. Sci. 29(5), 530–545 (1983)CrossRefGoogle Scholar
  6. 6.
    Hassenzahl, M.: The thing and I: understanding the relationship between user and product. In: Blythe, M., Overbeeke, C., Monk, A.F., Wright, P.C. (eds.) Funology: From Usability to Enjoyment, pp. 31–42. Kluwer Academic Publishers, Dordrecht (2003)CrossRefGoogle Scholar
  7. 7.
    Lin, C., He, Y., Pedrinaci, C., Domingue, J.: Feature LDA: a supervised topic model for automatic detection of web API documentations from the web. In: Cudré-Mauroux, P., Heflin, J., Sirin, E., Tudorache, T., Euzenat, J., Hauswirth, M., Parreira, J.X., Hendler, J., Schreiber, G., Bernstein, A., Blomqvist, E. (eds.) ISWC 2012, Part I. LNCS, vol. 7649, pp. 328–343. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  8. 8.
    Szalma, J.L., Hancock, P.: Task loading and stress in human-computer interaction: theoretical frameworks and mitigation strategies. In: Sears, A., Jacko, J.A. (eds.) Human Factors and Ergonomics. The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications, pp. 115–132. Lawrence Erlbaum Associates, New York (2007)Google Scholar
  9. 9.
    DeLone, W.H., McLean, E.R.: Information system success: the quest for the dependent variable. Inf. Syst. Res. 3(1), 60–95 (1992)CrossRefGoogle Scholar
  10. 10.
    Tullis, T., Albert, B.: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Elsevier, Amsterdam (2013)Google Scholar
  11. 11.
    DeLone, W.H., McLean, E.R.: The DeLone and McLean model of information systems success: a ten-year update. J. Manag. Inf. Syst. 19(4), 9–30 (2003)Google Scholar
  12. 12.
    Hassenzahl, M.: User experience (UX): toward an experiential perspective on product quality. In: Brangier, É., Michel, G., Bastien, J.M.C., Carbonell, N. (eds.) IHM 2008 Proceedings of the 20th International Conference of the Association Francophone d’Interaction Homme-Machine, pp. 11–15. ACM, New York (2008)Google Scholar
  13. 13.
    Hassenzahl, M., Burmester, M., Koller, F.: AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In: Szwillus, G., Ziegler, J. (eds.) Berichte des German Chapter of the ACM. Mensch & Computer 2003, pp. 187–196. Vieweg + Teubner Verlag, Wiesbaden (2003)Google Scholar
  14. 14.
    Winter, D., Schrepp, M., Thomaschewski, J.: Faktoren der User Experience: Systematische Übersicht über produktrelevante UX-Qualitätsaspekte. In: Fischer, H., Endmann, A., Krökel, M. (eds.) Mensch und Computer 2015 - Usability Professionals, pp. 33–41. De Gruyter, Berlin (2015)Google Scholar
  15. 15.
    McNamara, N., Kirakowski, J.: Functionality, usability, and user experience. Interactions, 13(6), 26–28 (2006)Google Scholar
  16. 16.
    Desmet, P., Hekkert, P.: Framework of product experience. Int. J. Des. 1(1), 57–66 (2007)Google Scholar
  17. 17.
    LePine, J.A., van Dyne, L.: Voice and cooperative behavior as contrasting forms of contextual performance: evidence of differential relationships with big five personality characteristics and cognitive ability. J. Appl. Psychol. 86(2), 326–336 (2001)CrossRefGoogle Scholar
  18. 18.
    Borman, W.C., Motowidlo, S.J.: Task performance and contextual performance: the meaning for personnel selection research. Hum. Perform. 10(2), 99–109 (1997)CrossRefGoogle Scholar
  19. 19.
    Stiensmeier-Pelster, J., Schöne, C.: Fähigkeitsselbstkonzept. In:. Handbuch der Psychologie, Handbuch der Pädagogischen Psychologie, vol. 10, pp. 62–73. Hogrefe, Göttingen (2008)Google Scholar
  20. 20.
    Motowidlo, S.J., van Scotter, J.R.: Evidence that task performance should be distinguished from contextual performance. J. Appl. Psychol. 79(4), 475–480 (1994)CrossRefGoogle Scholar
  21. 21.
    Frommer, J., Rösner, D., Haase, M., Lange, J., Friesen, R., Otto, M.: Project A3 - Detection and Avoidance of Failures in Dialogues. Pabst Science Publisher, Lengerich (2012)Google Scholar
  22. 22.
    Grawe, K.: Grundriß einer allgemeinen psychotherapie. Psychotherapeut 40, 130–145 (1995)Google Scholar
  23. 23.
    Funke, J.: Complex problem solving: a case for complex cognition? Cogn. Process. 11(2), 133–142 (2010)CrossRefGoogle Scholar
  24. 24.
    McCrae, R.R., Costa, P.T.: A contemplated revision of the NEO five-factor inventory. Pers. Individ. Differ. 36(3), 587–596 (2004)CrossRefGoogle Scholar
  25. 25.
    Rösner, D., Frommer, J., Friesen, R., Haase, M., Lange, J., Otto, M.: LAST MINUTE: a multimodal corpus of speech-based user-companion interactions. In: Workshop Abstracts, LREC 2012, Istanbul, Turkey, pp. 2559–2566. ELRA, Istanbul (2012)Google Scholar
  26. 26.
    Dickhäuser, O., Stiensmeier-Pelster, J.: Erlernte hilflosigkeit am computer? geschlechtsunterschiede in computerspezifischen attributionen. Psychologie in Erziehung und Unterricht 27, 486–496 (2002)Google Scholar
  27. 27.
    Ferchow, S., Haase, M., Krüger, J., Vogel, M., Wahl, M., Frommer, J.: Speech matters – psychological aspects of artificial versus anthropomorphic system voices in user-companion interaction. In: Human-Computer Interaction. LNCS. Springer, Berlin (2016). Submission ID: 1110Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Matthias Haase
    • 1
    Email author
  • Martin Krippl
    • 2
  • Mathias Wahl
    • 1
  • Swantje Ferchow
    • 1
  • Jörg Frommer
    • 1
  1. 1.Department of Psychosomatic Medicine and Psychotherapy, Medical FacultyOtto-von-Guericke University MagdeburgMagdeburgGermany
  2. 2.Institute for Psychology, Department Methodology, Psychodiagnostics and Evaluation ResearchOtto-von-Guericke University MagdeburgMagdeburgGermany

Personalised recommendations