The Quality of MOOCs: How to Improve the Design of Open Education and Online Courses for Learners?

Open Access
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10295)

Abstract

This paper presents the current status of Open Education and MOOCs and discusses their quality following the main question: How can we introduce new design and evaluation methods and personalization strategies to improve the learning quality of Open Education? First, the dimensions of Open Education are differentiated. Then the dimensions of holistic quality development are transferred to Open Education and discussed for the design of MOOCs leading to recommendations for personalization. A new quality indicator for evaluating the quality of MOOCs is introduced: It is proposed not to measure the traditional drop-out rates but the completion of individual goals and intentions by the MOOC learner. It is concluded that Open Education and MOOCs have got the potential for the next revolution in learning experiences.

Keywords

Open Education MOOCs Learning quality Design Personalization Evaluation Learners Designers Intention Personal goal Quality indicator 

1 Introduction

The world is changing and traditional formal education is challenged: New ways of communication and collaboration are asking for innovative learning experiences and environments and for lifelong learning. Opening up education is a holistic approach to facilitate such new learning experiences. This article analyses how we can improve the design of online courses and in particular of MOOCs to increase the learning quality and experience for learners.

2 Open Education: What Is the Current Situation?

The concepts “open” and “openness” are becoming more and more in vogue even though their concepts and descriptions are vague [1]. In many cases it is only referred to “open” and “openness” as general characteristics without any precise definition.

To avoid such confusion just mentioned above, we define Open Education as follows:

“Open Education covers and addresses all dimensions related to operational, legal and visionary aspects throughout the analysis, design, realization and evaluation of learning experiences to facilitate high quality education meeting the given situation, needs and objectives.”

This definition reduces Open Education not only on the open access but includes further legal dimensions such as open licensing and open availability as well as operational dimensions such as open resources, open technologies and open standards as well as visionary dimensions such as open methodologies, open recognition and open innovations. The Fig. 1 presents the overview of the Open Education dimensions:
Fig. 1.

Dimensions of open education [3]

However, Open Education is not a fad but an increasing requirement due to dramatic changes in societies [2]. The rise of the globalization and the establishment of the worldwide internet have strongly changed communication, business, work and leisure. People as well as organisations, communities and even whole societies are getting under pressure to adopt these changes and to face the arising challenges in business, individual and social life. That leads to new demands for alternative and innovative approaches and concepts in education. Orientation on key (or also so-called “21st century”) competences, practical relevance and lifelong learning are three core requests on Therefore, open education is garnering interest as well as spurring adaptations, implementations, and success.

While these Open Education developments were taking root, another phenomenon suddenly appeared and changed the public discussion on open courses: Massive Open Online Courses (MOOCs). Currently the quality of MOOCs is questioned based on the high drop-out and non-completion rates. One main reason is that the concept of quality development is (still) not introduced in the design of Open Education and MOOCs [3] what is ignoring the long-term research and analysis of its relevance for learning processes in general [5].

3 Quality in Open Education

We could conclude in former articles [4, 6] that (learning) quality is most important for learning, education and training. The debates on holistic quality management and on learning quality are very old [7, 8, 9], but discussions and theories on quality development in learning and education only began a few years ago [5].

The concept and philosophy of holistic quality development with a continuous improvement cycle was first introduced in Japan and quickly gained recognition, acceptance, and inspire implementations worldwide [10, 11]: A long-term debate has focused on quality development in general regarding the different quality issues, aspects and approaches [5]. In its broadest sense, quality development can be defined as covering ‘every kind of strategy, analysis, design, realization, evaluation, and continuous improvement of the quality within given systems’ [6]. Thus, quality development is described formally by the chosen paradigm and adapted approach: Quality is not a fixed characteristic belonging to subjects or systems but rather depends on adapting to specific situations.

Holistic (also called “Total”) quality management is divided into three generic quality dimensions: potential, processes and results [12]. Quality assurance is the pure focus on the results (e.g., screws), whereas quality management includes also the processes and their optimization (e.g., the production of screws). The third dimension potential that is addressed by holistic quality management is often not addressed as it requires continuous formative and summative evaluation, needs analysis and improvements based on their results (e.g., new business models and types of screws).

Due to the dramatic changes in societies, openness and open education are becoming not only more and more in vogue, but also vital: It is not a fashion but an increasing requirement [2]. To address and meet the societal challenges, we have transferred and applied the three generic quality dimensions (potential, processes and results) to Open Education by relating them to objectives (from designers as well as from learners), realization (including both, design and learning processes) and achievements (related to both objectives) as illustrated in Fig. 2 [3].
Fig. 2.

Quality dimensions in open education [3]

Open Education and learning, education and training in general constitute a special field: Not tangible products are sold but learning opportunities are provided as intangible offers and services that are targeting on learning results as achievements to be built by the learners themselves and not by the learning providers. In consequence, one learning opportunity is not fitting to all potential learners and the quality can only be assessed and measured individually. And in particular in MOOCs with their mass audience there are plenty of different individual motivations, intentions and goals that should be considered already during the design processes before and as well as during the learning processes and experiences themselves [26].

4 Drop-Out Rates and Quality Indicators in MOOCs

In Open Education, the new term MOOC has immediately attracted the masses, despite the fact that it is just another label for a diversity of different online learning scenarios and methodologies that were already developed and implemented many years before [13]. On the one hand, MOOCs can be seen as a product and in this sense they are a special type of Open Educational Resources (OER). On the other hand, MOOCs can also be considered and defined as a learning process or experience and in this sense they are a special type of technology-enhanced learning, also called e-learning, piquing interest anew and offering opportunities to once again reach learners that are attracted toe-learning solutions for many reasons [14]. Discussions of Open Education and e-Learning revealed their historical development and interdependences [3, 4].

Thus, MOOCs can be the facilitators for a renaissance of e-learning and for a broad implementation of online learning even though their completion rates are very low and their general quality is questionable and currently under lively debate [15]. Nowadays, different types of MOOCs (mainly cMOOCs and xMOOCs plus many others) are discussed, but the focus is still on the masses, on technology and promised innovations that are not easy to discover: Most MOOCs lack continuous tutoring and support for all learners who are expected to teach themselves [2, 13]. Having high drop-out rates raised the question of quality regarding MOOCs that currently is discussed heavily [16].

We believe that high drop-out rates are the wrong measure for the success of MOOCs and are only demonstrating the diversity of motivations and personal goals that MOOC learners are bringing with them: In common understanding the drop-out rates are measured against the completion of the MOOC, i.e., fulfillment of all assigned tasks and examinations and of all learning objectives intended and defined by the MOOC designer [25, 26]. But a small online pre-survey (n = 45, the whole results are currently analysed and published soon) has revealed that many MOOC learners do not share the intentions of the MOOC designer and have got their own personal goals like e.g., simple download of all available materials for their self-regulated learning and review. In many cases the MOOC learners have fulfilled their own personal goals and should be considered as successful MOOC completions but are counted as drop-outs as they have not completed the MOOC and its assigned tasks and examinations [25]. Consequently drop-out rates measured in the traditional way against completion should be high indicating the diversity of personal goals and their achievement in many different ways. In addition personalization should be manifold so that MOOC learners can select their own learning pathway according to their individual goals: By facilitation such personalization, MOOCs can pave a path for the future opening up of education to improve the learning quality [3].

Alternatively we propose that another quality indicator for MOOCs is introduced and that the concept drop-out rate should be defined differently. MOOC learners should only be counted as drop-outs if they fail to achieve their personal goals. Consequently the individual goal achievement rate would be the quality indicator for MOOCs that have to offer appropriate personalization to allow such individual learning pathways and results.

Based on our analysis above related to the three dimensions of holistic quality development and the identified need to focus the different individual goals and intentions by the MOOC learners, we can draw consequences and recommendations for MOOC designers to address this variety and to increase the fitting of MOOCs to the learners interests and demands as shown in the following Table 1.
Table 1.

MOOC learners’ requirements and design recommendations

 

Requirements by MOOC learners

Recommendation for MOOC designers

Objectives

Many different individual goals for MOOC registration

Ask for individual learning objectives and their reflection

Realization

Many different learning strategies used in the MOOC

Offer personalization of learning pathways in the MOOC

Achievements

Many different intentions what to achieve in the MOOC

Measure MOOC success according to individual goals and intentions

To research and analyse the details how to evaluate and personalize MOOCS, we have established the European initiative MOOQ for the quality of MOOCs aiming at the development and evaluation of a common Quality Reference Framework and quality indicators for improving, assessing and comparing the quality of MOOCs in close cooperation with all interested MOOC designers, learners, providers and policy makers in Europe and worldwide [17]: First activity is the launch of three online surveys for the different target groups of MOOC learners, designer and facilitators. Based on their analysis results, semi-structured interviews will ask MOOC designer, facilitator, providers and policy makers for their experiences and demands. Final aim is the validation of design patterns and tools to facilitate and improve the MOOC development for designers and the MOOC learning experiences for learners.

5 Is Open Education the Next Revolution?

According to Marx, a revolution is the complete change of the production relations and means and their new ownership and direction towards changed production power [18]. In relation to open education, the current question is whether open education is indeed a social revolution for individual learners, educational institutions, and global society, or whether MOOCs, the most prominent method of open learning, are only marketing instruments by the traditional educational providers with high reputation. The debate has started immediately after the successful launch and broad recognition of MOOCs in the mass media. The high drop-out rates have led to criticism and currently MOOCs are already declared as dead. Ensuing research on alternative quality indicators and personalization for MOOCs has already started and will provide further findings for future discussion soon.

This paper can only initiate the discussion on the impact of open education that was brought to the mass media and audiences thanks to the appearance of MOOCs. We are convinced that a change of the perspective and of the quality indicators to measure the success of MOOCs as proposed above will demonstrate the potentials that MOOCs can offer. It is necessary for future research and publications to focus on these challenges and provide more MOOC cases with personalization for all MOOC learners for further discussion.

We believe in education as a human right and public good as defined in the Sustainable Development Goal no. 4 by the United Nations [19] and that learning and education need to be changed to keep this status due to major global challenges [2]. The overview of the quality and future of open education and MOOCs has presented the needs and potential approaches to satisfy these requirements, along with how we can achieve higher learning quality by opening up education and introducing open learning innovations [3]. Current main movements in open education such as the global Open Educational Resources (OER) initiative launched with the UNESCO OER Forum (already in 2002) [20] and OER Declaration [21], the International Community for Open Research and Open Education (ICORE) [22] and Opening Up Education by the European Commission [23] are addressing the demand how to change future education. First evaluation frameworks and instruments are developed to assess the importance of open learning and open education for our future and the positive impact on our personal lives and developments as well as on all societies worldwide [24]: Future research should address and investigate the evaluation of Open Education and its impact in innovating learning experiences and quality education and in effects to improve personal development and societies. Then it can maybe be proven that Open Education and MOOCs are introducing the next revolution in learning experiences.

6 Conclusions and Future Work

Open Education and in particular MOOCs have the potential to change and improve future learning experiences. This paper identifies the need for new quality strategies and measures beyond misleading drop-out rates and for looking into all three dimensions of Open Education to meet the learners’ requirements and intentions. It is proposed that quality indicators for MOOCs have to take into consideration and evaluate the individual goals and intentions of MOOC learners. And also the MOOC design has to address and facilitate this diversity of different personal motivations, intentions and targeted achievements. That can be realized in MOOCs by asking for individual learning objectives and their reflection, offering personalization of learning pathways in MOOCs and by measuring the success of MOOCs according to individual goals and intentions. Further research is needed to investigate how the different groups of MOOC learners with their specific intentions can be addressed by providing personalized learning experiences in MOOCs as well as to assess the impact of Open Education in the society.

A first step and activity towards this vision and need was the launch of the Global Survey on the Quality of MOOCs [27]. This survey was designed and organized by MOOQ, the European Alliance for the quality of MOOCs [17]. Through the involvement and support by the leading European and international associations and institutions including four United Nation’s organizations, the survey has gained a huge recognition and was promoted worldwide. First results will be presented and discussed at the World Learning Summit 2017 [28] as well as in further workshops open for all interested stakeholders.

This debate including all stakeholders will lead to the design of a Quality Reference Framework (QRF). Based on the survey results the QRF draft with quality indicators and tools for MOOCs will be developed and discussed in close collaboration with all interested international stakeholders. It is intended to share and discuss the first draft of the QRF at the Second Open Educational Resources (OER) World Conference by UNESCO taking placing in September 2017 [29]. All discussions, contributions, review and evaluation cycles will lead to valuable instruments for designers and facilitators to improve future MOOCs for all learners worldwide.

Notes

Acknowledgments

This article is supported by MOOQ, the European Alliance for Quality of Massive Open Online Courses (www.MOOC-quality.eu). The vision of MOOQ is to foster quality in MOOCs leading to a new era of learning experiences. MOOQ is funded by the European Commission under the project number: 2015-1-NL01-KA203-008950.

References

  1. 1.
    Wiley, D.: Defining “Open” (2009). http://opencontent.org/blog/archives/1123
  2. 2.
    Stracke, C.M.: The need to change education towards open learning. In: Stracke, C.M., Shamarina-Heidenreich, T. (eds.) The Need for Change in Education: Openness as Default?, pp. 11–23. Logos, Berlin (2015). http://www.learning-innovations.eu
  3. 3.
    Stracke, C.M.: Openness for learning quality and change by open education in theory and practice - overview, history, innovations and policies: how can open learning, OER and MOOCs achieve impact for learners, organizations and in society? In: Amiel, T. (ed.) Utopias and Dystopias in Education, Sao Paulo: UNICAMP (2016, in print)Google Scholar
  4. 4.
    Stracke, C.M.: Learning innovations and learning quality: relations, interdependences, and future. In: Stracke, C.M. (ed.) The Future of Learning Innovations and Learning Quality. How do they fit together?, pp. 13–25. Gito, Berlin. http://www.learning-innovations.eu
  5. 5.
    Stracke, C.M.: Process-oriented Quality Management. In: Ehlers, U.-D., Pawlowski, J.M. (eds.) Handbook on Quality and Standardisation in E-learning, pp. 79–96. Springer, Berlin (2006)CrossRefGoogle Scholar
  6. 6.
    Stracke, C.M.: Open learning: the concept for modernizing school education and lifelong learning through the combination of learning innovations and quality. In: Stracke, C.M. (ed.) Learning Innovations and Quality: The Future of Digital Resources, pp. 15–28. Logos, Berlin (2013). http://www.learning-innovations.eu
  7. 7.
    Juran, J.M. (ed.): Quality Control Handbook. McGraw-Hill, New York (1951)Google Scholar
  8. 8.
    Deming, W.E.: Quality, productivity and competitive position. MIT, Cambridge (1982)Google Scholar
  9. 9.
    Crosby, P.B.: Quality is Free. The Art of Making Quality Certain. McGraw-Hill, New York (1980)Google Scholar
  10. 10.
    Juran, J.M.: Juran on Quality by Design. The New Steps for Planning Quality into Goods and Services. Free Press, New York (1992)Google Scholar
  11. 11.
    Deming, W.E.: Out of the Crisis. MIT, Cambridge (1986)Google Scholar
  12. 12.
    Donabedian, A.: The Definition of Quality and Approaches to Its Assessment (Explorations in Quality Assessment and Monitoring, vol. 1. Health Administration Press, Ann Arbor (1980)Google Scholar
  13. 13.
    Daniel, J.: Making Sense of MOOCs: Musings in a Maze of Myth, Paradox and Possibility (2012). http://sirjohn.ca/wordpress/wp-content/uploads/2012/08/120925MOOCspaper2.pdf. Accessed 02 Nov 2012
  14. 14.
    Downes, S.: E-learning 2.0. In: eLearn Magazine, October 2005. http://elearnmag.acm.org/featured.cfm?aid=1104968
  15. 15.
    Margaryan, A., Bianco, M., Littlejohn, A.: Instructional quality of massive open online courses (MOOCs). Comput. Educ. 80, 77–83 (2015)CrossRefGoogle Scholar
  16. 16.
    Reich, J.: Rebooting MOOC research. Science 347(6217), 34–35 (2015)CrossRefGoogle Scholar
  17. 17.
  18. 18.
    Marx, K.: Capital. A Critique of Political Economy. Volume I: Book One: The Process of Production of Capital. Progress Publishers, Moscow (1887). http://synagonism.net/book/economy/marx.1887-1867.capital-i.html Google Scholar
  19. 19.
    United Nations: Sustainable Development Goals. New York: United Nations (2016). http://sustainabledevelopment.un.org/sdgs
  20. 20.
    UNESCO: Forum on the Impact of Open Courseware for Higher Education in Developing Countries. Final Report. Paris: UNESCO (2002). http://unesdoc.unesco.org/images/0012/001285/128515e.pdf
  21. 21.
    UNESCO: 2012 Paris OER Declaration. 2012 World Open Educational Resources (OER) Congress. Paris: UNESCO (2012). http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/Events/Paris%20OER%20Declaration_01.pdf
  22. 22.
  23. 23.
    European Commission: Opening up Education: Innovative teaching and learning for all through new Technologies and Open Educational Resources. [COM(2013) 654 final] (2013) http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013DC0654&from=EN
  24. 24.
    Stracke, C.M.: Evaluation framework EFI for measuring the impact of learning, education and training. 华东师范大学学报(自然科学版). J. East China Normal Univ. 2014(2), 1–12 (2014). Shanghai: ECNU. doi:10.3969/j.issn.1000-5641.2012.02.012, http://www.opening-up.education
  25. 25.
    Gollwitzer, P.M., Sheeran, P.: Implementation intentions and goal achievement: a meta-analysis of effects and processes. Adv. Exp. Soc. Psychol. 38, 69–119 (2006)CrossRefGoogle Scholar
  26. 26.
    Baek, J., Shore, J.: Promoting student engagement in MOOCs. In: Proceedings of the Third (2016) ACM Conference on Learning @ Scale - L@S 2016, pp. 293–296. ACM Press, New York (2016). http://doi.org/10.1145/2876034.2893437
  27. 27.
    Global Survey on the Quality of MOOCs. www.survey.MOOC-Quality.eu
  28. 28.
    World Learning Summit. www.worldlearningsummit.com
  29. 29.
    UNESCO Second World Open Educational Resources (OER) Conference. http://en.unesco.org/events/2nd-world-open-educational-resources-oer-congress

Copyright information

© The Author(s) 2017

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  1. 1.Open University of the NetherlandsHeerlenThe Netherlands
  2. 2.Korean National Open UniversitySeoulKorea
  3. 3.East China Normal UniversityShanghaiChina
  4. 4.ICDE Chair in OEROsloNorway

Personalised recommendations