Artificial Intelligence Review

, Volume 52, Issue 3, pp 1997–2017 | Cite as

A systematic review of data-driven approaches in player modeling of educational games

  • Danial Hooshyar
  • Moslem Yousefi
  • Heuiseok LimEmail author


Recent years have seen growing interest in open-ended interactive educational tools such as games. One of the most crucial aspects of developing games lies in modeling and predicting individual behavior, the study of computational models of players in games. Although model-based approaches have been considered standard for this purpose, their application is often extremely difficult due to the huge space of actions that can be created by educational games. For this reason, data-driven approaches have shown promise, in part because they are not completely reliant on expert knowledge. This study seeks to systematically review the existing research on the use of data-driven approaches in player modeling of educational games. The primary objectives of this study are to identify, classify, and bring together the relevant approaches. We have carefully surveyed a 10-year sample (2008–2017) of research studies conducted on data-driven approaches in player modeling of educational games, and thereby found 67 significant research works. However, our criteria for inclusion reduced the sample to 21 studies that addressed four primary research questions, and so we analyzed and classified the questions, methods, and findings of these published works, which we evaluated and from which we drew conclusions based on non-statistical methods. We found that there are three primary avenues along which data-driven approaches have been studied in educational games research: first, the objective of data-driven approaches in player modeling of educational games, namely behavior modeling, goal recognition, and procedural content generation; second, approaches employed in such modeling; finally, current challenges of using data-driven approaches in player modeling of educational games, namely game data, temporal forecasting in player models, statistical techniques, algorithmic efficiency, knowledge engineering, problem of generalizability, and data sparsity problem. In conclusion we addressed four critical future challenges in the area, namely, the lack of proper and rich data publicly available to the researchers, the lack of a data-driven method to identify conceptual features from log data, hybrid player modeling approaches, and data mining techniques for individual prediction.


Player modeling Educational games Data-driven approach User modeling Systematic literature review (SLR) 



Special thanks to the anonymous reviewers for their insightful comments which helped us make this paper better. This work was supported by Ministry of Culture, Sport and Tourism (MCST) and Korea Creative Content Agency (KOCCA) in the Culture Technology (CT) Research & Development Program 2017 (No. R2016030031).


  1. Albrecht DW, Zukerman I, Nicholson AE (1998) Bayesian models for keyhole plan recognition in an adventure game. User Model User Adapt Interact 8:5–47CrossRefGoogle Scholar
  2. Andersen E (2012) Optimizing adaptivity in educational games. Paper presented at the proceedings of the international conference on the foundations of digital games, Raleigh, North CarolinaGoogle Scholar
  3. Andersen E, Liu Y-E, Snider R, Szeto R, Cooper S, Popovi Z (2011) On the harmfulness of secondary game objectives. Paper presented at the proceedings of the 6th international conference on foundations of digital games, Bordeaux, FranceGoogle Scholar
  4. Andersen E, Gulwani S, Popović Z (2013) A trace-based framework for analyzing and synthesizing educational progressions. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 773–782Google Scholar
  5. Baker, Yacef K (2009) The state of educational data mining in 2009: a review and future visions. J Educ Data Min 1:3–17Google Scholar
  6. Baker, Habgood MJ, Ainsworth SE (2007) Corbett AT Modeling the acquisition of fluent skill in educational action games. In: International conference on user modeling. Springer, pp 17–26Google Scholar
  7. Baker CL, Saxe R, Tenenbaum JB (2009) Action understanding as inverse planning. Cognition 113:329–349CrossRefGoogle Scholar
  8. Bellotti F, Berta R, De Gloria A, Primavera L (2009) Adaptive experience engine for serious games. IEEE Trans Comput Intell AI Games 1:264–280CrossRefGoogle Scholar
  9. Ben M (2011) Using recommendation systems to adapt gameplay. In: Richard EF (ed) Discoveries in gaming and computer-mediated simulations: new interdisciplinary applications. IGI Global, Hershey, pp 64–77. Google Scholar
  10. Brooks C, Epp CD, Logan G, Greer J (2011) The who, what, when, and why of lecture capture. Paper presented at the Proceedings of the 1st international conference on learning analytics and knowledge, Banff, Alberta, CanadaGoogle Scholar
  11. Butler E, Andersen E, Smith AM, Gulwani S, Popović Z (2015) Automatic game progression design through analysis of solution features. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, pp 2407–2416Google Scholar
  12. Charles D, Black M (2004) Dynamic player modelling: a framework for player-centered digital games. In: Proceedings of the international conference on computer games: artificial intelligence, design and education. pp 29–35Google Scholar
  13. Charles D et al. (2005) Player-centred game design: player modelling and adaptive digital games. In: Proceedings of the digital games research conference, 2005, p 00100Google Scholar
  14. Conati C (2002) Probabilistic assessment of user’s emotions in educational games. Appl Artif Intell 16:555–575CrossRefGoogle Scholar
  15. Duong T, Phung D, Bui H, Venkatesh S (2009) Efficient duration and hierarchical modeling for human activity recognition. Artif intell 173:830–856CrossRefGoogle Scholar
  16. Falakmasir MH, Jos, Gonzalez-Brenes P, Gordon GJ, DiCerbo KE (2016) A data-driven approach for inferring student proficiency from game activity logs. Paper presented at the proceedings of the third (2016) ACM conference on learning @ Scale, Edinburgh, Scotland, UKGoogle Scholar
  17. Feldman LA (1995) Valence focus and arousal focus: individual differences in the structure of affective experience. J Pers Soc Psychol 69:153CrossRefGoogle Scholar
  18. Frias-martinez E, Chen SY, Liu X (2006) Survey of data mining approaches to user modeling for adaptive hypermedia. Trans Syst Man Cyber Part C 36:734–749. CrossRefGoogle Scholar
  19. Frijda NH (1986) The emotions: studies in emotion and social interaction. Maison de Sciences de l’Homme, ParisGoogle Scholar
  20. Frome J (2007) Eight ways videogames generate emotion. In: DiGRA conferenceGoogle Scholar
  21. Galway L, Charles D, Black M (2008) Machine learning in digital games: a survey. Artif Intell Rev 29:123–161. CrossRefGoogle Scholar
  22. Geib CW, Goldman RP (2009) A probabilistic plan recognition algorithm based on plan tree grammars. Artif Intell 173:1101–1132MathSciNetCrossRefGoogle Scholar
  23. Grappiolo C, Cheong YG, Togelius J, Khaled R, Yannakakis GN (2011) Towards player adaptivity in a serious game for conflict resolution. In: 2011 third international conference on games and virtual worlds for serious applications (VS-GAMES), 4–6 May 2011, pp 192–198.
  24. Ha EY, Rowe JP, Mott BW, Lester JC (2012) Goal recognition with Markov logic networks for player-adaptive games. Paper presented at the Proceedings of the twenty-sixth AAAI conference on artificial intelligence, Toronto, ON, CanadaGoogle Scholar
  25. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update SIGKDD. Explor Newsl 11:10–18. CrossRefGoogle Scholar
  26. Harpstead E, MacLellan CJ, Aleven V, Myers BA (2014) Using extracted features to inform alignment-driven design ideas in an educational game. Paper presented at the Proceedings of the SIGCHI conference on human factors in computing systems, Toronto, ON, CanadaGoogle Scholar
  27. Hendrikx M, Meijer S, Van Der Velden J, Iosup A (2013) Procedural content generation for games: a survey. ACM Trans Multimed Comput Commun Appl (TOMM) 9:1CrossRefGoogle Scholar
  28. Hickey DT, Ingram-Goble AA, Jameson EM (2009) Designing assessments and assessing designs in virtual educational environments. J Sci Educ Technol 18:187–208. CrossRefGoogle Scholar
  29. Hooshyar D, Ahmad RB, Yousefi M, Yusop FD, Horng SJ (2015) A flowchart-based intelligent tutoring system for improving problem-solving skills of novice programmers. J Comput Assist Learn 31:345–361CrossRefGoogle Scholar
  30. Hooshyar D, Ahmad RB, Yousefi M, Fathi M, Horng S-J, Lim H (2016) Applying an online game-based formative assessment in a flowchart-based intelligent tutoring system for improving problem-solving skills. Comput Educ 94:18–36. CrossRefGoogle Scholar
  31. Hooshyar D, Yousefi M, Lim H (2017) A procedural content generation-based framework for educational games: toward a tailored data-driven game for developing early English reading skills. J Educ Comput Res. Google Scholar
  32. Isbister K, Schaffer N (2008) Game usability: advancing the player experience. CRC Press, Boca RatonCrossRefGoogle Scholar
  33. Jong T (2006) Technological advances in inquiry learning. Science 312:532–533. CrossRefGoogle Scholar
  34. Kardan AA, Aziz M, Shahpasand M (2015) Adaptive systems: a content analysis on technical side for e-learning environments. Artif Intell Rev 44:365–391CrossRefGoogle Scholar
  35. Ketelhut DJ, Nelson BC, Clarke J, Dede C (2010) A multi-user virtual environment for building and assessing higher order inquiry skills in science. Br J Educ Technol 41:56–68. CrossRefGoogle Scholar
  36. Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering. Keele University and Durham University Joint ReportGoogle Scholar
  37. Kodaganallur V, Weitz RR, Rosenthal D (2005) A comparison of model-tracing and constraint-based intelligent tutoring paradigms. Int J Artif Intell Ed 15:117–144Google Scholar
  38. Koedinger KR, Brunskill E, Baker RS, McLaughlin EA, Stamper J (2013) New potentials for data-driven intelligent tutoring system development and optimization. AI Mag 34:27–41CrossRefGoogle Scholar
  39. Lee SJ, Liu YE, Popović Z (2014a) Learning individual behavior in an educational game: a data-driven approach. In: the 7th international conference on educational data mining (EDM), London, UK, pp 114–121Google Scholar
  40. Lee SJ, Liu Y, Popović Z (2014b) Learning individual behavior in an educational game: a data-driven approach. Paper presented at the Educational Data MiningGoogle Scholar
  41. Lee SY, Rowe JP, Mott BW, Lester JC (2014c) A supervised learning framework for modeling director agent strategies in educational interactive narrative. IEEE Trans Comput Intell AI Games 6:203–215CrossRefGoogle Scholar
  42. Liu Y, Mandel T, Butler E, Andersen E, O’Rourke E, Brunskill E, Popović Z (2013) Predicting player moves in an educational game: a hybrid approach. Paper presented at the Educational Data Mining 2013Google Scholar
  43. Liu B, Lester JC, Min W, Mott BW, Rowe JP (2016) Player goal recognition in open-world digital games with long short-term memory networks. Paper presented at the The 25th international joint conference on artificial intelligence (IJCAI)Google Scholar
  44. Machado MC, Fantini EPC, Chaimowicz L (2011) Player modeling: Towards a common taxonomy. In: 2011 16th international conference on computer games (CGAMES), 27–30 July 2011, pp 50–57.
  45. Mandel T, Liu Y, Levine S, Brunskill E, Popovic Z (2014) Offline policy evaluation across representations with applications to educational games. Paper presented at the proceedings of the 2014 international conference on autonomous agents and multi-agent systems, Paris, FranceGoogle Scholar
  46. Min W, Ha EY, Rowe J, Mott B, Lester J (2014) Deep learning-based goal recognition in open-ended digital games. AIIDE 14:3–7Google Scholar
  47. Min W, Baikadi A, Mott B, Rowe J, Liu B, Ha EY, Lester J (2016a) A generalized multidimensional evaluation framework for player goal recognition. In: the twelfth annual AAAI conference on artificial intelligence and interactive digital entertainmentGoogle Scholar
  48. Min W, Mott B, Rowe J, Liu B, Lester J (2016b) Player goal recognition in open-world digital games with long short-term memory networks. In: The 25th international joint conference on artificial intelligence, pp 2590-2596Google Scholar
  49. Min W, Mott B, Rowe J, Lester J (2017) Deep LSTM-based goal recognition models for open-world digital gamesGoogle Scholar
  50. Mott B, Lee S, Lester J (2006) Probabilistic goal recognition in interactive narrative environments. In Proceedings of the twenty-first national conference on artificial intelligence. AAAI Press, Menlo Park, pp 187–192Google Scholar
  51. Ortony A, Clore GL, Collins A (1990) The cognitive structure of emotions. Cambridge University Press, CambridgeGoogle Scholar
  52. Peirce N, Conlan O, Wade V (2008) Adaptive educational games: providing non-invasive personalised learning experiences. Paper presented at the proceedings of the 2008 second IEEE international conference on digital game and intelligent toy enhanced learningGoogle Scholar
  53. Prensky M (2003) Digital game-based learning. Comput Entertain 1:21Google Scholar
  54. Rowe JP, Shores LR, Mott BW, Lester JC (2010) integrating learning and engagement in narrative-centered learning environments. In: Aleven V, Kay J, Mostow J (eds) Intelligent tutoring systems: 10th international conference, ITS 2010, Pittsburgh, PA, USA, June 14–18, 2010, Proceedings, Part II. Springer, Berlin, pp 166–177.
  55. Rus V, Lintean M, Azevedo R (2009) Automatic detection of student mental models during prior knowledge activation in MetaTutor. In: International working group on educational data miningGoogle Scholar
  56. Russell JA (2003) Core affect and the psychological construction of emotion. Psychol Rev 110:145CrossRefGoogle Scholar
  57. Shaffer D (2007) How computer games help children learn. Palgrave Macmillan, New YorkGoogle Scholar
  58. Sukthankar G, Geib C, Bui HH, Pynadath D, Goldman RP (eds) (2014) Plan, activity, and intent recognition: theory and practice. NewnesGoogle Scholar
  59. Valls-Vargas J, Ontanón S, Zhu J (2015) Exploring player trace segmentation for dynamic play style prediction. In: Proceedings of the eleventh AAAI conference on artificial intelligence and interactive digital entertainment, pp 93–99Google Scholar
  60. Yannakakis, Spronck P, Loiacono D, Andre E (2013) Player modeling. Paper presented at the Dagstuhl seminar on game artificial and computational intelligence, GermanyGoogle Scholar
  61. Yannakakis G (2012) Game AI revisited. Paper presented at the proceedings of the 9th conference on computing frontiers, Cagliari, ItalyGoogle Scholar
  62. Yannakakis GN, Togelius J (2011) Experience-driven procedural content generation. IEEE Trans Affect Comput 2:147–161CrossRefGoogle Scholar
  63. Yannakakis GN, Togelius J (2017) Artificial intelligence and games (First Public Draft)Google Scholar
  64. Yee N (2006) Motivations for play in online games. Cyber Psychol Behav 9:772–775CrossRefGoogle Scholar
  65. Yin H, Luo L, Cai W, Ong YS, Zhong J (2015) A data-driven approach for online adaptation of game difficulty. In: 2015 IEEE conference on computational intelligence and games (CIG), August 31 2015–September 2 2015, pp 146–153.
  66. Zook A, Lee-Urban S, Drinkwater MR, Riedl MO (2012) Skill-based mission generation: a data-driven temporal player modeling approach. Paper presented at the proceedings of the the third workshop on procedural content generation in games, Raleigh, NC, USAGoogle Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2017

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringKorea UniversitySeoulRepublic of Korea
  2. 2.School of Civil, Environmental and Architectural EngineeringKorea UniversitySeoulRepublic of Korea

Personalised recommendations