From Objective to Subjective Difficulty Evaluation in Video Games

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10514)


This paper describes our research investigating the perception of difficulty in video games, defined as players’ estimation of their chances of failure. We discuss our approach as it relates to psychophysical studies of subjective difficulty and to cognitive psychology research into the overconfidence effect. The starting point for our study was the assumption that the strong motivational pull of video games may lead players to become overconfident, and thereby underestimate their chances of failure. We design and implement a method for an experiment using three games, each representing a different type of difficulty, wherein players bet on their capacity to succeed. Our results confirm the existence of a gap between players’ actual and self-evaluated chances of failure. Specifically, players seem to underestimate high levels of difficulty. The results do not show any influence on difficulty underestimation from the players gender, feelings of self-efficacy, risk aversion or gaming habits.


User modelling Affective HCI Emotion Motivational aspects Tools for design Modelling Evaluation Fun/Aesthetic design 



Authors would like to thank Daniel Andler, Jean Baratgin, Lauren Quiniou, and Laurence Battais & Hélène Malcuit from Carrefour Numérique.


  1. 1.
    Juul, J.: The game, the player, the world: looking for a heart of gameness. In: Raessens, J. (ed.) Level Up: Digital Games Research Conference Proceedings, vol. 1, pp. 30–45 (2003)Google Scholar
  2. 2.
    Malone, T.W.: Heuristics for designing enjoyable user interfaces: lessons from computer games. In: Proceedings of the 1982 Conference on Human Factors in Computing Systems, pp. 63–68 (1982)Google Scholar
  3. 3.
    Lazzaro, N.: Why we play games: four keys to more emotion without story. In: Game Developers Conference, March 2004Google Scholar
  4. 4.
    Sweetser, P., Wyeth, P.: Gameflow: a model for evaluating player enjoyment in games. Computers in Entertainment (CIE) 3(3), 3 (2005)CrossRefGoogle Scholar
  5. 5.
    Nakamura, J., Csikszentmihalyi, M.: The Concept of Flow. In: Nakamura, J., Csikszentmihalyi, M. (eds.) Flow and the Foundations of Positive Psychology, pp. 239–263. Springer, Dordrecht (2014). doi: 10.1007/978-94-017-9088-8_16 Google Scholar
  6. 6.
    Allart, T., Levieux, G., Pierfitte, M., Guilloux, A., Natkin, S.: Difficulty influence on motivation over time in video games using survival analysis. In: Proceedings of Foundation of Digital Games, Cap Cod, MA, USA (2017)Google Scholar
  7. 7.
    Ryan, R.M., Rigby, C.S., Przybylski, A.: The motivational pull of video games: a self-determination theory approach. Motiv. Emot. 30(4), 344–360 (2006)CrossRefGoogle Scholar
  8. 8.
    Juul, J.: A Casual Revolution: Reinventing Video Games and Their Players. Mit Press, Cambridge (2009)Google Scholar
  9. 9.
    Juul, J.: The Art of Failure, 1st edn. The MIT Press, Cambridge (2013)Google Scholar
  10. 10.
    Hunicke, R.: The case for dynamic difficulty adjustment in games. In: Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, pp. 429–433. ACM (2005)Google Scholar
  11. 11.
    Andrade, G., Ramalho, G., Santana, H., Corruble, V.: Extending reinforcement learning to provide dynamic game balancing. In: Proceedings of the Workshop on Reasoning, Representation, and Learning in Computer Games, 19th International Joint Conference on Artificial Intelligence (IJCAI), pp. 7–12 (2005)Google Scholar
  12. 12.
    Vicencio-Moreira, R., Mandryk, R.L., Gutwin, C.: Now you can compete with anyone: Balancing players of different skill levels in a first-person shooter game. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 2255–2264. ACM (2015)Google Scholar
  13. 13.
    Rani, P., Sarkar, N., Liu, C.: Maintaining optimal challenge in computer games through real-time physiological feedback. In: Proceedings of the 11th International Conference on Human Computer Interaction, vol. 58 (2005)Google Scholar
  14. 14.
    Afergan, D., Peck, E.M., Solovey, E.T., Jenkins, A., Hincks, S.W., Brown, E.T., Chang, R., Jacob, R.J.K.: Dynamic difficulty using brain metrics of workload. In: Jones, M., Palanque, P. (eds.) CHI 2014 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, Ontario, Canada, pp. 3797–3806. ACM, New York (2014)Google Scholar
  15. 15.
    Aponte, M.V., Levieux, G., Natkin, S.: Difficulty in videogames: an experimental validation of a formal definition. In: Romão, T. (ed.) Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology, ACE 2011, Lisbon, Portugal, pp. 1–18. ACM, New York (2011)Google Scholar
  16. 16.
    Passyn, K., Sujan, M.: Skill-based versus effort-based task difficulty: a task-analysis approach to the role of specific emotions in motivating difficult actions. J. Consum. Psychol. 22(3), 461–468 (2012)CrossRefGoogle Scholar
  17. 17.
    Levieux, G.: Mesure de la difficulté dans les jeux vidéo. Thèse, Conservatoire National des Arts et Métiers CNAM Paris (2011)Google Scholar
  18. 18.
    Hunicke, R., LeBlanc, M., Zubeck, R.: MDA: a formal approach to game design and game research. In: Proceedings of the AAAI Workshop on Challenges in Game AI, San Jose, CA, USA. AAAI Press (2004)Google Scholar
  19. 19.
    Delignières, D., Famose, J.: Perception de la difficulté et nature de la tâche. Science et motricité 23, 39–47 (1994)Google Scholar
  20. 20.
    Borg, G., Bratfisch, O., Dorni’c, S.: On the problems of perceived difficulty. Scand. J. Psychol. 12(1), 249–260 (1971)CrossRefGoogle Scholar
  21. 21.
    Slifkin, A.B., Grilli, S.M.: Aiming for the future: prospective action difficulty, prescribed difficulty, and fitts law. Exp. Brain Res. 174(4), 746–753 (2006)CrossRefGoogle Scholar
  22. 22.
    Delignières, D., Famose, J.P.: Perception de la difficulté, entropie et performance. Sci. Sports 7(4), 245–252 (1992)CrossRefGoogle Scholar
  23. 23.
    Delignières, D., Famose, J.P., Genty, J.: Validation d’une échelle de catégories pour la perception de la difficulté. Revue STAPS 34, 77–88 (1994)Google Scholar
  24. 24.
    Delignières, D., Famose, J.P., Thépaut-Mathieu, C., Fleurance, P., et al.: A psychophysical study of difficulty rating in rock climbing. Int. J. Sport Psychol. 24, 404 (1993)Google Scholar
  25. 25.
    Delignières, D., Brisswalter, J., Legros, P.: Influence of physical exercise on choice reaction time in sports experts: the mediating role of resource allocation. J. Hum. Mov. Stud. 27(4), 173–188 (1994)Google Scholar
  26. 26.
    Fitts, P.M.: The information capacity of the human motor system in controlling the amplitude of movement. J. Exp. Psychol. 47(6), 381 (1954)CrossRefGoogle Scholar
  27. 27.
    Kahneman, D., Frederick, S.: A model of heuristic judgment. In: Holyoak, K.J., Morrison, R.G. (eds.) The Cambridge Handbook of Thinking and Reasoning, 1st edn, pp. 267–293. Cambridge University Press, Cambridge (2005)Google Scholar
  28. 28.
    Shah, A.K., Oppenheimer, D.M.: Heuristics made easy: an effort-reduction framework. Psychol. Bull. 134(2), 207–222 (2008)CrossRefGoogle Scholar
  29. 29.
    Kahneman, D., Tversky, A.: Judgment under uncertainty: heuristics and biases. Science (New York, N.Y.) 185(4157), 1124–1131 (1974)CrossRefGoogle Scholar
  30. 30.
    Russo, J.E., Schoemaker, P.J.H.: Managing overconfidence. Sloan Manag. Rev. 33(2), 7–17 (1992)Google Scholar
  31. 31.
    Bessière, V.: Excès de confiance des dirigeants et décisions financières: une synthèse. Finance Contrôle Stratégie 10, 39–66 (2007)Google Scholar
  32. 32.
    Moore, D.A., Healy, P.J.: The trouble with overconfidence. Psychol. Rev. 115(2), 502–517 (2008)CrossRefGoogle Scholar
  33. 33.
    Griffin, D., Tversky, A.: The weighing of evidence and the determinants of confidence. Cogn. Psychol. 411435, 411–435 (1992)CrossRefGoogle Scholar
  34. 34.
    Johnson, D.D.P., Fowler, J.H.: The evolution of overconfidence. Nature 477(7364), 317–320 (2011)CrossRefGoogle Scholar
  35. 35.
    Bandura, A.: Self-efficacy: toward a unifying theory of behavioral change. Psychol. Rev. 84(2), 191–215 (1977)CrossRefGoogle Scholar
  36. 36.
    Keren, G.: Facing uncertainty in the game of bridge: a calibration study. Organ. Behav. Hum. Decis. Process. 39(1), 98–114 (1987)CrossRefGoogle Scholar
  37. 37.
    Linnet, J., Gebauer, L., Shaffer, H., Mouridsen, K., Møller, A.: Experienced poker players differ from inexperienced poker players in estimation bias and decision bias. J. Gambl. Issues 24, 86–100 (2010)CrossRefGoogle Scholar
  38. 38.
    Park, Y.J., Santos-Pinto, L.: Overconfidence in tournaments: evidence from the field. Theor. Decis. 69(1), 143–166 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Sundali, J., Croson, R.: Biases in casino betting: the hot hand and the gambler’s fallacy. Judgm. Decis. Mak. 1(1), 1–12 (2006)Google Scholar
  40. 40.
    Parke, J., Griffiths, M.: The psychology of the fruit machine: the role of structural characteristics (revisited). Int. J. Ment. Health Addict. 4(2), 151–179 (2006)CrossRefGoogle Scholar
  41. 41.
    Lichtenstein, S., Fischhoff, B.: Do those who know more also know more about how much they know? Organ. Behav. Hum. Perform. 20, 159–183 (1977)CrossRefGoogle Scholar
  42. 42.
    Klayman, J., Soll, J.B.: Overconfidence: it depends on how, what, and whom you ask. Organ. Behav. Hum. Decis. Process. 79(3), 216–247 (1999)CrossRefGoogle Scholar
  43. 43.
    Kahneman, D., Tversky, A.: Subjective probability: a judgment of representativeness. Cogn. Psychol. 3(3), 430–454 (1972)CrossRefGoogle Scholar
  44. 44.
    Croson, R., Sundali, J.: The gambler’s fallacy and the hot hand: empirical data from casinos. J. Risk Uncertain. 30(3), 195–209 (2005)CrossRefzbMATHGoogle Scholar
  45. 45.
    Gilovich, T., Vallone, R., Tversky, A.: The hot hand in basketball: on the misperception of random sequences. Cogn. Psychol. 17(3), 295–314 (1985)CrossRefGoogle Scholar
  46. 46.
    Langer, E.J.: The illusion of control. J. Pers. Soc. Psychol. 32(2), 311–328 (1975)CrossRefGoogle Scholar
  47. 47.
    Goodie, A.S.: The role of perceived control and overconfidence in pathological gambling. J. Gambl. Stud. 21(4), 481–502 (2005)CrossRefGoogle Scholar
  48. 48.
    Pulford, B.D., Colman, A.M.: Overconfidence: feedback and item difficulty effects. Pers. Individ. Differ. 23(1), 125–133 (1997)CrossRefGoogle Scholar
  49. 49.
    Costikyan, G.: Uncertainty in Games, 1st edn. MIT Press, Cambridge (2013)Google Scholar
  50. 50.
    Lankoski, P., Björk, S.: Game Research Methods: An Overview, 1st edn. ETC Press, Halifax (2015)Google Scholar
  51. 51.
    Bates, D., Mächler, M., Bolker, B.M., Walker, S.C.: Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67(1), 1–48 (2015)CrossRefGoogle Scholar
  52. 52.
    Chen, G., Gully, S.M., Eden, D.: Validation of a new general self-efficacy scale. Organ. Res. Methods 4(1), 62–83 (2001)CrossRefGoogle Scholar
  53. 53.
    Bandura, A.: Guide for constructing self-efficacy scales. In: Urdan, T., Pajares, F. (eds.) Self-efficacy Beliefs of Adolescents, 1st edn, pp. 307–337. Information Age Publishing, Charlotte (2006)Google Scholar
  54. 54.
    Holt, C.A., Laury, S.K.: Risk aversion and incentive effects. Am. Econ. Rev. 92(5), 1644–1655 (2002)CrossRefGoogle Scholar
  55. 55.
    Nakagawa, S., Schielzeth, H.: A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods Ecol. Evol. 4(2), 133–142 (2013)CrossRefGoogle Scholar
  56. 56.
    Keren, G.: On the calibration of probability judgments: some critical comments and alternative perspectives. J. Behav. Decis. Mak. 10(3), 269–278 (1997)CrossRefGoogle Scholar
  57. 57.
    Barber, B.M., Odean, T.: Boys will be boys: gender, overconfidence, and common stock investment. Quart. J. Econ. 116(1), 261–292 (2001)CrossRefzbMATHGoogle Scholar
  58. 58.
    Stone, D.N.: Overconfidence in initial self-efficacy judgments: effects on decision processes and performance. Organ. Behav. Hum. Decis. Process. 59(3), 452–474 (1994)CrossRefGoogle Scholar
  59. 59.
    Caillois, R.: Les jeux et les hommes : le masque et le vertige, 2nd edn. Gallimard, Paris (1958)Google Scholar
  60. 60.
    Arkes, H.R., Christensen, C., Lai, C., Blumer, C.: Two methods of reducing overconfidence. Organ. Behav. Hum. Decis. Process. 39, 133–144 (1987)CrossRefGoogle Scholar
  61. 61.
    Goldberg, J.H., Kotval, X.: Computer interface evaluation using eye movements: methods and constructs computer interface evaluation using eye movements: methods and constructs. Int. J. Ind. Ergon. 24(November 2015), 631–645(1999)Google Scholar
  62. 62.
    Klingner, J., Tversky, B., Hanrahan, P.: Effects of visual and verbal presentation on cognitive load in vigiolance, memory, and arithmetic tasks. Psychophysiology 48, 323–332 (2011)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2017

Authors and Affiliations

  1. 1.Conservatoire National des Arts et Métiers, CNAM-CédricParis Cedex 03France

Personalised recommendations