Advertisement

Reinforcing Stealth Assessment in Serious Games

  • Konstantinos GeorgiadisEmail author
  • Giel van Lankveld
  • Kiavash Bahreini
  • Wim Westera
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11899)

Abstract

Stealth assessment is a principled assessment methodology proposed for serious games that uses statistical models and machine learning technology to infer players’ mastery levels from logged gameplay data. Although stealth assessment has been proven to be valid and reliable, its application is complex, laborious, and time-consuming. A generic stealth assessment tool (GSAT), proven for its robustness with simulation data, has been proposed to resolve these issues. In this study, GSAT’s robustness is further investigated by using real-world data collected from a serious game on personality traits and validated with an associated personality questionnaire (NEO PI-R). To achieve this, (a) a stepwise regression approach was followed for generating statistical models from logged data for the big five personality traits (OCEAN model), (b) the statistical models are then used with GSAT to produce inferences regarding learners’ mastery level on these personality traits, and (c) the validity of GSAT’s outcomes are examined through a correlation analysis using the results of the NEO PI-R questionnaire. Despite the small dataset GSAT was capable of making inferences on players’ personality traits. This study has demonstrated the practicable feasibility of the SA methodology with GSAT and provides a showcase for its wider application in serious games.

Keywords

Stealth assessment Serious games Generic tool Statistical model Machine learning Stepwise regression Personality traits 
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Larson, L.C., Miller, T.N.: 21st century skills: prepare students for the future. Kappa Delta Pi Rec. 47(3), 121–123 (2011)CrossRefGoogle Scholar
  2. 2.
    Shute, V.J.: Stealth assessment in computer-based games to support learning. Comput. Games Instr. 55(2), 503–524 (2011)Google Scholar
  3. 3.
    Mislevy, R.J.: Evidence-centered design for simulation-based assessment. CRESST Report 800. National Center for Research on Evaluation, Standards, and Student Testing (CRESST) (2011)Google Scholar
  4. 4.
    Shute, V.J., Ventura, M., Kim, Y.J.: Assessment and learning of qualitative physics in newton’s playground. J. Educ. Res. 106(6), 423–430 (2013)CrossRefGoogle Scholar
  5. 5.
    Ventura, M., Shute, V., Small, M.: Assessing persistence in educational games. Design Recomm. Adapt. Intell. Tutor. Syst.: Learn. Model. 2, 93–101 (2014)Google Scholar
  6. 6.
    Shute, V.J., Wang, L., Greiff, S., Zhao, W., Moore, G.: Measuring problem solving skills via stealth assessment in an engaging video game. Comput. Hum. Behav. 63, 106–117 (2016)CrossRefGoogle Scholar
  7. 7.
    Moore, G.R., Shute, V.J.: Improving learning through stealth assessment of conscientiousness. In: Marcus-Quinn, A., Hourigan, T. (eds.) Handbook on Digital Learning for K-12 Schools, pp. 355–368. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-33808-8_21CrossRefGoogle Scholar
  8. 8.
    Georgiadis, K., Van Lankveld, G., Bahreini, K., Westera, W.: Accommodating stealth assessment in serious games: towards developing a generic tool. In 2018 10th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), pp. 1–4. IEEE (2018)Google Scholar
  9. 9.
    Georgiadis, K., Van Lankveld, G., Bahreini, K., Westera, W.: Learning analytics should analyse the learning: proposing a generic stealth assessment tool. Accepted at the IEEE Conference on Games (CoG) (2019)Google Scholar
  10. 10.
    Georgiadis, K., Van Lankveld, G., Bahreini, K., Westera, W.: On the robustness of steath assessment. IEEE Trans. Games (2019, submitted)Google Scholar
  11. 11.
    Van Lankveld, G., Spronck, P., Van den Herik, J., Arntz, A.: Games as personality profiling tools. In: 2011 IEEE Conference on Computational Intelligence and Games (CIG 2011), pp. 197–202. IEEE (2011)Google Scholar
  12. 12.
    McCrae, R.R., Costa Jr., P.T.: Personality trait structure as a human universal. Am. Psychol. 52(5), 509 (1997)CrossRefGoogle Scholar
  13. 13.
    Costa, P.T., McCrae, R.R.: The revised neo personality inventory (neo-pi-r). SAGE Handb. Pers. Theory Assess. 2(2), 179–198 (2008)Google Scholar
  14. 14.
    Sabourin, J.L.: Stealth assessment of self-regulated learning in game-based learning environments (2013)Google Scholar
  15. 15.
    Min, W., et al.: DeepStealth: leveraging deep learning models for stealth assessment in game-based learning environments. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M.F. (eds.) AIED 2015. LNCS (LNAI), vol. 9112, pp. 277–286. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-19773-9_28CrossRefGoogle Scholar
  16. 16.
    Wiggins, J.S. (ed.): The Five-Factor Model of Personality: Theoretical Perspectives. Guilford Press, New York (1996)Google Scholar
  17. 17.
    Domingos, P.M.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012)CrossRefGoogle Scholar
  18. 18.
    Field, A.: Discovering Statistics Using SPSS. Sage Publications, London (2009)zbMATHGoogle Scholar
  19. 19.
    Green, S.B.: How many subjects does it take to do a regression analysis. Multivar. Behav. Res. 26(3), 499–510 (1991)CrossRefGoogle Scholar
  20. 20.
    Maxwell, S.E.: Sample size and multiple regression analysis. Psychol. Methods 5(4), 434 (2000)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Open University of the NetherlandsHeerlenThe Netherlands
  2. 2.Fontys Applied University of EindhovenEindhovenThe Netherlands

Personalised recommendations

Cite paper

Buy options