Abstract
Evaluating a serious game is a time-consuming task. However, good evaluations are necessary to improve the effectiveness of serious games, and to prove this effectiveness to stakeholders. Computer support of evaluations requires addressing several problems, including security, privacy protection, data collection from both questionnaires and in-game activities, data analysis, and management of the experimental workflow. We describe improvements to the Simva architecture to add scalability and a bridge to exploratory data science to our one-stop serious games evaluation platform. Simva supports evaluations ranging from small-scale pilots to full-fledged validations with complex experimental designs. The improvements described in this paper greatly increase ease of deployment, interoperation with existing authentication infrastructure, and scalability of Simva, and can be readily applied to tools with similar goals.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Boyle, E.A., Hainey, T., Connolly, T.M., Gray, G., Earp, J., Ott, M., et al.: An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Comput. Educ. 94, 178–192 (2016). https://doi.org/10.1016/j.compedu.2015.11.003
Calvo-Morata, A., Alonso-Fernández, C., Freire, M., Martínez-Ortiz, I., Fernández-Manjón, B.: Serious games to prevent and detect bullying and cyberbullying: A systematic serious games and literature review. Comput. Educ. (accepted), (2020)
Advanced Distributed Learning Initiative: Experience API. https://github.com/adlnet/xAPI-Spec/releases/tag/xAPI-1.0.3. Accessed 10 July 2020
IMS Global Learning Consortium Inc.: Caliper Analytics Specification, version 1.1. https://www.imsglobal.org/sites/default/files/caliper/v1p1/caliper-spec-v1p1/caliper-spec-v1p1.html. Accessed 10 July 2020
Serrano-Laguna, Á., Martínez-Ortiz, I., Haag, J., Regan, D., Johnson, A., Fernández-Manjón, B.: Applying standards to systematize learning analytics in serious games. Comput. Stand. Interfaces 50, 116–123 (2017). https://doi.org/10.1016/j.csi.2016.09.014
Freire, M., Serrano-Laguna, Á., Iglesias, B.M., Martínez-Ortiz, I., Moreno-Ger, P., Fernández-Manjón, B.: Game learning analytics: Learning analytics for serious games. In: Spector, M., Lockee, B., Childress, M. (eds.) Learning, Design, and Technology, pp. 1–29. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-17727-4_21-1
José, P.I., Manuel, P.V., Iván, M., Manuel, F.: Simplifiying serious games authoring and validation with uAdventure and SIMVA, pp. 106–108 (2020). https://doi.org/10.1109/ICALT49669.2020.00039
RAGE Consortium: RAGE realizing an applied gaming eco-system. http://rageproject.eu/. Accessed 10 July 2020
Calvo-Morata, A., Rotaru, D.C., Alonso-Fernandez, C., Freire-Moran, M., Martinez-Ortiz, I., Fernandez-Manjon, B.: Validation of a cyberbullying serious game using game analytics. IEEE Trans. Learn. Technol. 13, 186–197 (2020). https://doi.org/10.1109/TLT.2018.2879354
Alonso-Fernandez, C., Perez-Colado, I., Calvo-Morata, A., Freire, M., Martinez-Ortiz, I., Fernandez-Manjon, B.: Applications of Simva to simplify serious games validation and deployment. IEEE Rev. Iberoam. Tecnol. del Aprendiz, p. 1 (2020). https://doi.org/10.1109/RITA.2020.3008117
Alonso-Fernández, C., Perez-Colado, I.J., Calvo-Morata, A., Freire, M., Martinez-Ortiz, I., Fernández-Manjón, B.: Using Simva to evaluate serious games and collect game learning analytics data. In: LASI Spain 2019: Learning Analytics in Higher Education (2019)
Perez-Colado, I., Alonso-Fernandez, C., Freire, M., Martinez-Ortiz, I., Fernandez-Manjon, B.: Game learning analytics is not informagic! In: 2018 IEEE Global Engineering Education Conference (EDUCON), pp. 1729–1737. IEEE (2018). https://doi.org/10.1109/EDUCON.2018.8363443
Alonso-Fernández, C., Caballero Roldán, R., Freire, M., Martinez-Ortiz, I., Fernández-Manjón, B.: Predicting students’ knowledge after playing a serious game based on learning analytics data: A case study (in press). J. Comput. Assist. Learn. (2019). https://doi.org/10.1111/jcal.12405
Alonso-Fernández, C., Calvo-Morata, A., Freire, M., Martínez-Ortiz, I., Fernández-Manjón, B.: Applications of data science to game learning analytics data: A systematic literature review. Comput. Educ. 141, 103612 (2019). https://doi.org/10.1016/j.compedu.2019.103612
IMS Global Learning Consortium, I.: Learning tools interoperability core specification, version 1.3. https://www.imsglobal.org/spec/lti/v1p3/. Accessed 10 July 2020
Acknowledgements
This work has been partially funded by Regional Government of Madrid (eMadrid P2018/TCS4307), by the Ministry of Education (TIN2017-89238-R), by the European Commission (Erasmus+IMPRESS 2017-1-NL01-KA203-035259) and by the Telefonica-Complutense Chair on Digital Education and Serious Games.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pérez-Colado, I.J., Pérez-Colado, V.M., Martínez-Ortiz, I., Freire, M., Fernández-Manjón, B. (2020). A Scalable Architecture for One-Stop Evaluation of Serious Games. In: Marfisi-Schottman, I., Bellotti, F., Hamon, L., Klemke, R. (eds) Games and Learning Alliance. GALA 2020. Lecture Notes in Computer Science(), vol 12517. Springer, Cham. https://doi.org/10.1007/978-3-030-63464-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-63464-3_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63463-6
Online ISBN: 978-3-030-63464-3
eBook Packages: Computer ScienceComputer Science (R0)