Challenges and Opportunities in Evaluating Learning in Serious Games: A Look at Behavioural Aspects

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7528)


In this paper, we describe an approach to modelling competences as learning resources in a serious game environment, where competences are described in detail to identify observable behavioural indicators. This enables the evaluation and assessment of learning, where specific behaviours indicate if a player does or does not have a competence. We have used the OKEI Competence Modelling Framework to describe the competences, where the application of a competence in a specific situation or within a context can be modelled. The main focus of this paper is to analyse and discuss the opportunities and challenges that we have experienced during this work. While the approach is resources intensive to describe the competences in sufficient level of detail, it provides a reusable set of Behavioural Indicators that can be used both in designing and evaluating other Technology Enhanced Learning applications. Most importantly, the work provided important input for the design of the game scenarios in describing situations and relevant contextual information as well as input for improving the believability of the avatars in the game.


Evaluation Serious Games Behaviour Competence Assessment Game Design Character Design 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andersen, B., Fradinho, M., Lefrere, P., Niitamo, V.P.: The Coming Revolution in Competence Development: Using Serious Games to Improve Cross-Cultural Skills. In: Ozok, A.A., Zaphiris, P. (eds.) OCSC 2009. LNCS, vol. 5621, pp. 413–422. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Kirkpatrick, D.L., Kirkpatrick, J.D.: Evaluating Training Programs, The Four Levels. Berrett-Koehler Publishers, San Francisco (2006)Google Scholar
  3. 3.
  4. 4.
    Cheetham, G., Chivers, G.: Professional Competence: Harmonizing Reflective Practitioner and Competence-based Approaches. In: O’Reilly, D., Cuningham, L., Lester, S. (eds.) The Capable Practitioner: Professional Capability Through Higher Education, pp. 215–226. Kogan Page Limited, London (1999)Google Scholar
  5. 5.
    Engeström, Y.: Learning by Expanding: An Activity-Theoretical Approach to Developmental Research. Orienta-KonsultitOy, Helsinki (1987)Google Scholar
  6. 6.
    Luckin, R.: Re-designing Learning Contexts. Routledge Taylor & Francis Group, London (2010)Google Scholar
  7. 7.
    Petersen, S.A., Heikura, T.: Modelling Project Management and Innovation Competences for Technology Enhanced Learning. In: Proceedings of the eChallenges 2010, Warsaw, Poland, pp. 1–9 (2010)Google Scholar
  8. 8.
    Csikszentmihalyi, M.: Flow: The psychology of optimal experience. Harper & Collins, New York (1990)Google Scholar
  9. 9.
    Bedek, M.A., Petersen, S.A., Heikura, T.: From Behavioral Indicators to Con-textualized Competence Assessment. In: Proceedings of the 11th IEEE International Conference on Advanced Learning Technologies, Athens, United States, pp. 277–281 (2011)Google Scholar
  10. 10.
    Hall, T.: The hidden dimension. Doubleday Anchor, Garden City (1969)Google Scholar
  11. 11.
    Hayduk, L.A.: Personal space: An evaluative and orienting overview. Psychological Bulletin 55, 117–134 (1978)CrossRefGoogle Scholar
  12. 12.
    Hall, E.T.: Beyond Culture. Doubleday Anchor, New York (1977)Google Scholar
  13. 13.
    Bull, S.: Supporting learning with open learner models. In: Proceedings of the 4th Hellenic Conference on Information and Communication Technologies in Education, Athens, Greece, pp. 47–61 (2004)Google Scholar
  14. 14.
    Mitrović, A., Martin, B.: Evaluating the Effects of Open Student Models on Learning. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) AH 2002. LNCS, vol. 2347, pp. 296–305. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  15. 15.
    Cowley, B., Bedek, M.A., Ribeiro, C.S., Heikura, T., Petersen, S.A.: The QUARTIC process model to support serious games development for contextu-alized competence-based learning and assessment. In: Cruz-Cunha, M.M. (ed.) Handbook of Research on Serious Games as Educational, Business, and Research Tools: Development and Design. IGI Global Publishers, Hershey (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.SINTEF Technology and SocietyTrondheimNorway
  2. 2.Knowledge Management InstituteGraz University of TechnologyGrazAustria

Personalised recommendations