Abstract
This paper presents the evaluation methods and results of a pilot tourism MOOC (Massive Open Online Course) called eTourism: Communication Perspectives, based on the Kirkpatrick model. It assigned twelve indicators to the model’s four levels of evaluation (reaction, learning, behaviour, results). Indicators include: self-efficacy and motivation, satisfaction, relevance, course performance, collaborative learning, higher-order learning, reflective and integrative learning, skills development, post-course practices, corporate social responsibility, public relations, and marketing. With various measurement tools such as pre-, in- and post-course surveys, post-course interviews, and analytics data by the host platform, the paper explains the available data with the twelve indicators and provides meaningful performance assessment for the MOOC. Results show that the MOOC was successful in all four levels according to the twelve indicators. The limitations and the future directions are also discussed at the end of the study.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado-Kloos, C., & Munoz-Organero, M. (2014). Delving into participants’ profiles and use of social tools in MOOCs. IEEE Transactions on Learning Technologies, 3(7), 260–266.
Bates, R. (2004). A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficence. Evaluation and Program Planning, 27(3), 341–347.
Bradley, K., & Connors, E. (2007). Training evaluation model: Evaluating and improving criminal justice training. Retrieved August 18, 2016, from http://bit.ly/2dkiU4j.
Class Central. (2015). By the numbers: MOOCS in 2015. Retrieved August 22, 2016, from http://bit.ly/2e1XmIF.
Creelman, A., Ehlers, U., & Ossiannilsson, E. (2014). Perspectives on MOOC quality-an account of the EFQUEL MOOC quality project. INNOQUAL-International Journal for Innovation and Quality in Learning, 2(3), 78–87.
Cross, S. (2013). Evaluation of the OLDS MOOC curriculum design course: Participant perspectives, expectations and experiences. Retrieved September 4, 2016, from http://bit.ly/2eiihJL.
Douglas, K. A., Mihalec-Adkins, B. P., Hicks, N. M., & Diefes-Dux, H. A. (2016). Learners in advanced nanotechnology MOOCs: Understanding their intention and motivation. Retrieved October 18, 2016 from http://bit.ly/2eoIIIm.
Downes, S. (2013). The quality of massive open online courses. Retrieved August 21, 2016, from http://bit.ly/2eij9Ou.
Gamage, D., Fernando, S., & Perera, I. (2015). Quality of MOOCs: A review of literature on effectiveness and quality aspects. Education, 121, 2.
Jordan, K. (2014). Initial trends in enrolment and completion of massive open online courses. The International Review of Research in Open and Distributed Learning, 15(1).
Khalil, M., Brunner, H., & Ebner, M. (2015). Evaluation grid for xMOOCs. International Journal of Emerging Technologies in Learning (iJET), 10(4).
Khalil, H., & Ebner, M. (2014). MOOCs completion rates and possible methods to improve retention-a literature review. In EdMedia: World Conference on Educational Media and Technology (Vol. 2014, No. 1, pp. 1305–1313).
Kirkpatrick, D. (1975). Evaluating training programs. Tata McGraw-Hill Education.
Kirkpatrick Partners. (2016). The Kirkpatrick model. Retrieved August 18, 2016, from http://bit.ly/2edF4U8.
Lesjak, B., & Florjancic, V. (2014). Evaluation of MOOC: Hands-On project or creative use of ict in teaching. In Human Capital without Borders: Knowledge and Learning for Quality of Life; Proceedings of the Management, Knowledge and Learning International Conference 2014 (pp. 1147–1155). ToKnowPress.
Lin, J., Cantoni, L., & Kalbaska, N. (2016). How to develop and evaluate an eTourism MOOC: An experience in progress. e-Review of Tourism Research (eRTR), 7, 1–5.
Lin, J., Kalbaska, N., Cantoni, L., & Murphy, J. (2016). A new framework to describe and analyse MOOC design: Multiple case study of hospitality and tourism MOOCs. Revised and resubmitted for publication.
Murphy, J., Tracey, J. B., & Horton-Tognazzini, L. (2016). MOOC camp: A flipped classroom and blended learning model. In Information and Communication Technologies in Tourism 2016 (pp. 653–665). Springer International Publishing.
Onah, D. F., Sinclair, J., & Boyatt, R. (2014). Exploring the use of MOOC discussion forums. In Proceedings of London International Conference on Education (pp. 1–4). LICE.
Parra, J. (2016). Moving beyond MOOC mania: Lessons from a faculty-designed MOOC. Current Issues in Emerging eLearning, 3(1), 10.
Poce, A. (2015). Developing critical perspectives on technology in education: A tool for MOOC evaluation. European Journal of Open, Distance and E-learning, 18(1).
Rodrigo, C., Read, T., Santamaría, M., & Sánchez-Elvira, A. (2014). OpenupEdLabel for MOOC quality assurance: UNED COMA initial self-evaluation. In Actas del V Congreso Internacional sobre Calidad y Accesibilidad en la Formación Virtual (CAFVIR 2014) (pp. 551–555).
Tracey, J. B., Murphy, J., & Horton-Tognazzini, L. (2016). A framework for evaluating MOOCs in applied hospitality and tourism settings. In Information and Communication Technologies in Tourism 2016 (pp. 667–679). Springer International Publishing.
Wintrup, J., Wakefield, K., & Davis, H. C. (2015). Engaged learning in MOOCs: A study using the UK engagement survey.
Yousef, A. M. F., Chatti, M. A., Ahmad, I., Schroeder, U., & Wosnitza, M. (2015). An evaluation of learning analytics in a blended MOOC environment. In Proceedings of the Third European MOOCs Stakeholders Summit EMOOCs (pp. 122–130).
Yousef, A. M. F., Chatti, M. A., Schroeder, U., & Wosnitza, M. (2015). A usability evaluation of a blended MOOC environment: An experimental case study. The International Review of Research in Open and Distributed Learning, 16(2).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Lin, J., Cantoni, L. (2017). Assessing the Performance of a Tourism MOOC Using the Kirkpatrick Model: A Supplier’s Point of View. In: Schegg, R., Stangl, B. (eds) Information and Communication Technologies in Tourism 2017. Springer, Cham. https://doi.org/10.1007/978-3-319-51168-9_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-51168-9_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-51167-2
Online ISBN: 978-3-319-51168-9
eBook Packages: Business and ManagementBusiness and Management (R0)