1 Introduction

Massive Open Online Courses (MOOCs) are changing the educational field, challenging traditional institutional strategies and recognition schemes and opening up new opportunities for learners and educators both from within and outside formal education. However, while the potential benefits and risks of the MOOCs have been discussed by scientists and policy makers, the corresponding empirical data is scarce. What’s more, the evidence that is available is usually restricted to a single course or single provider.

MOOCKnowledge (http://moocknowledge.eu/), funded by the European Commission’s Institute for Prospective Technological Studies (IPTS), aims to facilitate a shared understanding of the value and efficacy of MOOCs by developing a set of analysis tools and applying them to a wide range of MOOCs. We have already developed a three-survey (pre- post- and follow-up) tool, which compares learner’s expectations and intentions to their perceptions and the observable evidence of their actual benefits from the MOOC. We are in the process of developing a design analysis tool, which will include a set of rubrics to evaluate a MOOC’s design - from it’s overall structure to the details of specific media assets.

The most powerful outcome of the project would be the possibility to correlate different dimensions of MOOC production, execution, and learners experience. For example, identifying links between financial investment, learning design, and learner outcomes. To do this, we must first develop a conceptual model of the factors which determine or contribute to the value of a MOOC.

This paper presents our current version of this model, and invites the community to engage with it. The model was developed through a combination of desk research and expert review.

The mindmap of the model is available at:

https://atlas.mindmup.com/2016/03/f8dfb450cc3101338f4d19e3b2bc43d4/mooc_value/index.html

A version of this paper open for commenting is available at:

https://docs.google.com/document/d/1oVfZ2WGLklJNfRissjdOkkbK8A7yNZvPSJrZaKb780o/edit.

2 Method

The Model is being developed through iterations of desk research and expert review. We started by looking at the typical parameters used to list/catalogue MOOCs. We then expanded it to include factors that are often neglected, such as the institutional/individual motivations for creating a MOOC. This model was presented to experts at the RIDE conference and online, and was updated based on their feedback.

This process of calibrating literature, common practice and expert review is ongoing. Our presentation at EC TEL will be another major iteration.

3 The Model

The model currently has nine sections (Fig. 1): meta-data, cost, drivers, benefits, risks, regulatory framework, learner profile, efficacy, and figures. This model is not a taxonomy, it is simply a guide for identifying the factors that play a potential role in determining the value of a MOOC, and a starting point for exploring correlations and dependencies between these.

Fig. 1.
figure 1

Overview of the model

Full size image

  • Meta-data. Parameters typically used to index or catalogue a MOOC.

    The Meta-data parameters are:

    • Topic e.g. Java programming, web design, art history

    • Level/type educational institution type (K12, Higher education, professional development) and level (Introductory, Intermediate, Advanced)

    • Title course title

    • Timing start date and length (in weeks)

    • prerequisites

    • Institution (and faculty) providing the MOOC

    • Delivery mode Scheduled/self-paced

    • Platform e.g. Coursera, EdX, FutureLearn

    • Language e.g. English, Spanish, Arabic

    • Effort required by the student, in hours per week

    • Certification types of certificates offered (including ECTS)

    • Target audience profile of expected participants

    • Size expected number of students, including possible caps on size.

  • Cost. The various factors that determine the cost of designing, developing and delivering a MOOC.

    The cost factors we identified are:

    • Design and planning Research Design Prototyping

    • Production content production, including text, media (graphics, animations, games, and video), markup and media integration on the platform, assignments and assessments, and content maintenance (updating the content from time to time).

    • Quality Assurance

    • Marketing

    • Hosting either on an established platform or on a self-hosted/externally hosted VLE.

    • Presentation the actual “running” cost, including the time of faculty, facilitators/moderators, and tech support

    • Assessment in particular procturing and marking

    • Certification mainly the platform fees

    • Evaluation from audit of the MOOC design pre-presentation to the analysis of the feedback and analytics.

  • Drivers. Drivers are the factors that motivate institutions and individuals to offer MOOCs.

  • Benefits. Benefits are the actual positive outcomes that a MOOC may have for the individuals attending them, the institutions and individuals providing them, and society as a whole.

  • Risks. By contrast to benefits, risks enumerate the possible negative consequences of MOOCs.

  • Regulatory Framework. MOOCs (as all educational instruments) are governed by national and international regulatory frameworks, which enable and delimit their potential impact and dictate some of the practices of their providers and participants.

  • Learner Profile. The Learner Profile includes the characteristics of the MOOC participants that can be inferred from questionnaires or observations.

  • Efficacy and Learning Design. Efficacy refers to the predicted capacity of the MOOC to achieve its aims.

4 Summary

We have presented a proposed model for the value of a MOOC. Although this model admittedly still requires refinement and validation, we believe it is nevertheless of value for whoever is considering developing a MOOC, or needs to make policy decisions regarding MOOCs.

The most significant value of this model will be as a research tool for exploring the interaction and dependencies between the different dimensions. For example, to answer questions such as:

  • what is more cost-effective (in terms of learner benefits) - investment in video quality or in the quality of assignments and assessment?

  • are certain media types more appealing to specific learner profiles?

  • what are the hidden costs, benefits and risks that need to be considered when evaluating a proposal for producing a new MOOC?

We plan to collect data along these dimensions and make it available under an open licence, to facilitate research of such questions and others.