Challenges and Opportunities in Evaluating Learning in Serious Games: A Look at Behavioural Aspects
In this paper, we describe an approach to modelling competences as learning resources in a serious game environment, where competences are described in detail to identify observable behavioural indicators. This enables the evaluation and assessment of learning, where specific behaviours indicate if a player does or does not have a competence. We have used the OKEI Competence Modelling Framework to describe the competences, where the application of a competence in a specific situation or within a context can be modelled. The main focus of this paper is to analyse and discuss the opportunities and challenges that we have experienced during this work. While the approach is resources intensive to describe the competences in sufficient level of detail, it provides a reusable set of Behavioural Indicators that can be used both in designing and evaluating other Technology Enhanced Learning applications. Most importantly, the work provided important input for the design of the game scenarios in describing situations and relevant contextual information as well as input for improving the believability of the avatars in the game.
KeywordsEvaluation Serious Games Behaviour Competence Assessment Game Design Character Design
Unable to display preview. Download preview PDF.
- 2.Kirkpatrick, D.L., Kirkpatrick, J.D.: Evaluating Training Programs, The Four Levels. Berrett-Koehler Publishers, San Francisco (2006)Google Scholar
- 3.TARGET: Deliverable D4.1 - Competence Portfolios (2011), http://www.reachyourtarget.org/index.php?option=com_content&view=article&id=156:d41-competence-portfolios&catid=15:public-deliverables&Itemid=43
- 4.Cheetham, G., Chivers, G.: Professional Competence: Harmonizing Reflective Practitioner and Competence-based Approaches. In: O’Reilly, D., Cuningham, L., Lester, S. (eds.) The Capable Practitioner: Professional Capability Through Higher Education, pp. 215–226. Kogan Page Limited, London (1999)Google Scholar
- 5.Engeström, Y.: Learning by Expanding: An Activity-Theoretical Approach to Developmental Research. Orienta-KonsultitOy, Helsinki (1987)Google Scholar
- 6.Luckin, R.: Re-designing Learning Contexts. Routledge Taylor & Francis Group, London (2010)Google Scholar
- 7.Petersen, S.A., Heikura, T.: Modelling Project Management and Innovation Competences for Technology Enhanced Learning. In: Proceedings of the eChallenges 2010, Warsaw, Poland, pp. 1–9 (2010)Google Scholar
- 8.Csikszentmihalyi, M.: Flow: The psychology of optimal experience. Harper & Collins, New York (1990)Google Scholar
- 9.Bedek, M.A., Petersen, S.A., Heikura, T.: From Behavioral Indicators to Con-textualized Competence Assessment. In: Proceedings of the 11th IEEE International Conference on Advanced Learning Technologies, Athens, United States, pp. 277–281 (2011)Google Scholar
- 10.Hall, T.: The hidden dimension. Doubleday Anchor, Garden City (1969)Google Scholar
- 12.Hall, E.T.: Beyond Culture. Doubleday Anchor, New York (1977)Google Scholar
- 13.Bull, S.: Supporting learning with open learner models. In: Proceedings of the 4th Hellenic Conference on Information and Communication Technologies in Education, Athens, Greece, pp. 47–61 (2004)Google Scholar
- 15.Cowley, B., Bedek, M.A., Ribeiro, C.S., Heikura, T., Petersen, S.A.: The QUARTIC process model to support serious games development for contextu-alized competence-based learning and assessment. In: Cruz-Cunha, M.M. (ed.) Handbook of Research on Serious Games as Educational, Business, and Research Tools: Development and Design. IGI Global Publishers, Hershey (2012)Google Scholar