Assessing the Performance of a Tourism MOOC Using the Kirkpatrick Model: A Supplier’s Point of View
- 3.6k Downloads
This paper presents the evaluation methods and results of a pilot tourism MOOC (Massive Open Online Course) called eTourism: Communication Perspectives, based on the Kirkpatrick model. It assigned twelve indicators to the model’s four levels of evaluation (reaction, learning, behaviour, results). Indicators include: self-efficacy and motivation, satisfaction, relevance, course performance, collaborative learning, higher-order learning, reflective and integrative learning, skills development, post-course practices, corporate social responsibility, public relations, and marketing. With various measurement tools such as pre-, in- and post-course surveys, post-course interviews, and analytics data by the host platform, the paper explains the available data with the twelve indicators and provides meaningful performance assessment for the MOOC. Results show that the MOOC was successful in all four levels according to the twelve indicators. The limitations and the future directions are also discussed at the end of the study.
KeywordsMOOCs Massive Open Online Course Kirkpatrick model Tourism
- Bradley, K., & Connors, E. (2007). Training evaluation model: Evaluating and improving criminal justice training. Retrieved August 18, 2016, from http://bit.ly/2dkiU4j.
- Class Central. (2015). By the numbers: MOOCS in 2015. Retrieved August 22, 2016, from http://bit.ly/2e1XmIF.
- Creelman, A., Ehlers, U., & Ossiannilsson, E. (2014). Perspectives on MOOC quality-an account of the EFQUEL MOOC quality project. INNOQUAL-International Journal for Innovation and Quality in Learning, 2(3), 78–87.Google Scholar
- Cross, S. (2013). Evaluation of the OLDS MOOC curriculum design course: Participant perspectives, expectations and experiences. Retrieved September 4, 2016, from http://bit.ly/2eiihJL.
- Douglas, K. A., Mihalec-Adkins, B. P., Hicks, N. M., & Diefes-Dux, H. A. (2016). Learners in advanced nanotechnology MOOCs: Understanding their intention and motivation. Retrieved October 18, 2016 from http://bit.ly/2eoIIIm.
- Downes, S. (2013). The quality of massive open online courses. Retrieved August 21, 2016, from http://bit.ly/2eij9Ou.
- Gamage, D., Fernando, S., & Perera, I. (2015). Quality of MOOCs: A review of literature on effectiveness and quality aspects. Education, 121, 2.Google Scholar
- Jordan, K. (2014). Initial trends in enrolment and completion of massive open online courses. The International Review of Research in Open and Distributed Learning, 15(1).Google Scholar
- Khalil, M., Brunner, H., & Ebner, M. (2015). Evaluation grid for xMOOCs. International Journal of Emerging Technologies in Learning (iJET), 10(4).Google Scholar
- Khalil, H., & Ebner, M. (2014). MOOCs completion rates and possible methods to improve retention-a literature review. In EdMedia: World Conference on Educational Media and Technology (Vol. 2014, No. 1, pp. 1305–1313).Google Scholar
- Kirkpatrick, D. (1975). Evaluating training programs. Tata McGraw-Hill Education.Google Scholar
- Kirkpatrick Partners. (2016). The Kirkpatrick model. Retrieved August 18, 2016, from http://bit.ly/2edF4U8.
- Lesjak, B., & Florjancic, V. (2014). Evaluation of MOOC: Hands-On project or creative use of ict in teaching. In Human Capital without Borders: Knowledge and Learning for Quality of Life; Proceedings of the Management, Knowledge and Learning International Conference 2014 (pp. 1147–1155). ToKnowPress.Google Scholar
- Lin, J., Cantoni, L., & Kalbaska, N. (2016). How to develop and evaluate an eTourism MOOC: An experience in progress. e-Review of Tourism Research (eRTR), 7, 1–5.Google Scholar
- Lin, J., Kalbaska, N., Cantoni, L., & Murphy, J. (2016). A new framework to describe and analyse MOOC design: Multiple case study of hospitality and tourism MOOCs. Revised and resubmitted for publication.Google Scholar
- Murphy, J., Tracey, J. B., & Horton-Tognazzini, L. (2016). MOOC camp: A flipped classroom and blended learning model. In Information and Communication Technologies in Tourism 2016 (pp. 653–665). Springer International Publishing.Google Scholar
- Onah, D. F., Sinclair, J., & Boyatt, R. (2014). Exploring the use of MOOC discussion forums. In Proceedings of London International Conference on Education (pp. 1–4). LICE.Google Scholar
- Parra, J. (2016). Moving beyond MOOC mania: Lessons from a faculty-designed MOOC. Current Issues in Emerging eLearning, 3(1), 10.Google Scholar
- Poce, A. (2015). Developing critical perspectives on technology in education: A tool for MOOC evaluation. European Journal of Open, Distance and E-learning, 18(1).Google Scholar
- Rodrigo, C., Read, T., Santamaría, M., & Sánchez-Elvira, A. (2014). OpenupEdLabel for MOOC quality assurance: UNED COMA initial self-evaluation. In Actas del V Congreso Internacional sobre Calidad y Accesibilidad en la Formación Virtual (CAFVIR 2014) (pp. 551–555).Google Scholar
- Tracey, J. B., Murphy, J., & Horton-Tognazzini, L. (2016). A framework for evaluating MOOCs in applied hospitality and tourism settings. In Information and Communication Technologies in Tourism 2016 (pp. 667–679). Springer International Publishing.Google Scholar
- Wintrup, J., Wakefield, K., & Davis, H. C. (2015). Engaged learning in MOOCs: A study using the UK engagement survey.Google Scholar
- Yousef, A. M. F., Chatti, M. A., Ahmad, I., Schroeder, U., & Wosnitza, M. (2015). An evaluation of learning analytics in a blended MOOC environment. In Proceedings of the Third European MOOCs Stakeholders Summit EMOOCs (pp. 122–130).Google Scholar
- Yousef, A. M. F., Chatti, M. A., Schroeder, U., & Wosnitza, M. (2015). A usability evaluation of a blended MOOC environment: An experimental case study. The International Review of Research in Open and Distributed Learning, 16(2).Google Scholar