Journal of Computing in Higher Education

, Volume 29, Issue 1, pp 47–64 | Cite as

H-MOOC framework: reusing MOOCs for hybrid education

  • Mar Pérez-Sanagustín
  • Isabel Hilliger
  • Carlos Alario-Hoyos
  • Carlos Delgado Kloos
  • Saif Rayyan


Since Massive Open Online Courses (MOOCs) started to become part of the scene of Higher Education (HE), many institutions have joined the race of MOOC creation. However, producing MOOCs has shown to be a cumbersome and expensive activity for HE institutions. For this reason, many universities have started to explore and experiment with hybrid initiatives in which locally produced and third-party MOOCs are reused and integrated into traditional courses. Most of the hybrid initiatives described in the literature so far focus on flipped classroom experiences, although there are some other possibilities for integrating MOOCs in the curriculum. Moreover, few studies have reported on the institutional support required for implementing hybrid initiatives, and their benefits from a curriculum perspective. In order to shed some light on the opportunities that arise from the reuse of MOOCs, this paper presents H-MOOC, a framework that describes hybrid MOOC-based initiatives as a continuum of two factors: (1) institutional support to reuse an existing MOOC, and (2) curricular content alignment between the MOOC and the program, or the course hybridized. In addition, H-MOOC proposes indicators to measure the impact of these initiatives at both educational and institutional levels. Examples of actual hybrid initiatives and a set of guiding questions are presented to show how to apply the H-MOOC framework in different contexts.


MOOCs Higher education Hybrid initiatives Framework Indicators 


Since the appearance of Massive Open Online Courses (MOOCs), many institutions have joined the MOOC wave, generating a large number of courses. However, producing MOOCs has shown to be a cumbersome and expensive activity for Higher Education (HE) institutions (Nissenson and Shih 2015). HE decision-makers are facing important challenges derived from the rising enrollment fees, constrained budgets, and the changing educational landscape. Thus, elite universities have mainly led the MOOC development process, while the remaining institutions see MOOCs initial costs as a barrier, and need to look for alternative plans for benefiting from MOOCs (Ng’ambi and Bozalek 2015; Nissenson and Shih 2015).

In order to take advantage from MOOCs, HE entities have started to explore and experiment with hybrid learning initiatives aimed at integrating locally produced and third-party MOOCs into the curriculum (Zhang 2013; Sandeen 2013; Kloos et al. 2015; Ghadiri et al. 2013). In this context, the concept of hybrid is understood in a broad sense, including any learning initiative, strategy or model that integrates MOOCs or MOOC-related technologies into a traditional curriculum.

Most of current studies on hybrid initiatives have focused on flipped classroom experiences, analyzing learning gains in comparison with more traditional approaches (Joseph and Nath 2013; Kerr 2015; McLean et al. 2016). But, the innovation scope has expanded beyond flipped classrooms as the variety of hybrid initiatives offered by universities grows by taking advantage of the pool of existing MOOCs (Zhang 2013; Kloos et al. 2015; Pérez-Sanagustín et al. 2016a).

Some authors have started to study the impact of MOOC-based hybrid models, explaining, for instance, how hybrid models can be implemented (Zhang 2013; Kloos et al. 2015), or comparing students’ learning gains between hybrid initiatives and traditional approaches (Joseph and Nath 2013). Literature has been, however, of little relevance from an institutional perspective until the last couple of years, where indicators of hybrid initiatives success have evolved from students’ satisfaction to students’ support, costs and effort (Firmin et al. 2014; Griffiths et al. 2014). Still, few indicators inform about institutional advantages and threats of integrating MOOCs into the academic curriculum (Soffer and Cohen 2015).

Thus, there is a need to collect and analyze more information about the emerging opportunities for applying hybrid MOOC-based initiatives in HE. This paper presents the Hybrid MOOC (H-MOOC) framework, which helps understand how an existing MOOC could be reused and incorporated into the curriculum. The paper also discusses indicators that could be considered to measure the impact of including a MOOC in the curriculum from both educational and institutional perspectives. Specifically, the contribution of this paper is threefold: (1) presenting a literature review on MOOC-based hybrid initiatives where existing MOOCs are successfully incorporated in the curriculum of different institutions; (2) organizing these initiatives according to the H-MOOC framework to facilitate their comparison; and (3) offering researchers and decision-makers a set of indicators to understand the effect of different hybrid initiatives. Finally, we illustrate how the H-MOOC framework can be applied, through a set of guiding questions to anticipate what types of hybrid initiatives could be implemented by reusing an existing MOOC.

Hybrid MOOC-based models and indicators

Hybrid MOOC-based models

Prior studies have described different models to integrate MOOCs into HE curriculum. Zhang (2013) proposed 5 models based on the relevance for the institution, shifting the scope from delivery to purpose. These models are: (1) Learner services for MOOC participants, providing university services to learners that participate in MOOCs, but that are not enrolled in residential education (e.g., the use of the library); (2) MOOCs as Open Resources, using MOOC components as learning objects on residential courses; (3) Flipped classrooms, using MOOC content for residential students to study at home; (4) Challenge courses for MOOCs, developing projects as assessments of residential students’ work on a MOOC; and (5) Credit transfer from MOOCs, granting credit for completing MOOCs after passing an exam.

More recently, Kloos et al. (2015) documented 6 hybrid models for integrating MOOC technologies with face-to-face (f2f) instruction: (1) Local digital prelude, in which the first part of the course is completely online (MOOC-based) followed by a traditional f2f part; (2) Flipping the classroom, in which students are expected to use MOOC-based online content to study at home, while class time is dedicated to reinforcing their understanding and to applying the acquired knowledge; (3) Canned digital teaching with f2f tutoring, which consists of MOOC-based contents that students use to prepare their exams in semesters where there are no f2f classes, having the faculty available at office hours for tutoring; (4) Canned digital teaching in f2f courses, in which students use MOOC-based contents as textbooks in f2f residential courses; (5) Remote tutoring in f2f courses, which corresponds with digital interventions (live or canned) from experts to complement traditional courses; and (6) Canned digital teaching with remote tutoring, which refers to completely online MOOC-based courses complemented with video-conferences for tutoring.

Measuring the effect of MOOC-based hybrid initiatives

Researchers have described the results of hybrid MOOC-based initiatives using different types of indicators. Most case studies have reported students’ demographics, participation rates, success rates (pass/fail), student interaction with online content (e.g., weeks active, assignments completed, etc.), and learning gains by modality (i.e. online, face-to-face, hybrid) (Firmin et al. 2014; Konstan et al. 2015; Nissenson and Shih 2015; Wu et al. 2010). Some studies have also measured students’ use of support services, their attitudes towards MOOCs, previous online experiences, or their use of social media, among others (Firmin et al. 2014; Riffell and Sibley 2005). Indicators such as faculty self-perceptions, faculty technology literacy skills, and students’ perception of the quality of instruction have also been documented concerning the effect of MOOC usage over the teaching process (Georgina and Hosford 2009; Georgina and Olson 2008; Riffell and Sibley 2005).

Current studies have diversified the use of indicators in order to describe hybrid initiatives from an institutional perspective. Recent publications referred to infrastructure needs, student support required, course and faculty development, among other aspects that could concern HE decision-makers (Firmin et al. 2014; Moskal et al. 2013). Concerning curriculum implementation, research projects have intended to address acknowledgement of prior learning, articulation, and credit recognition (Sandeen 2013). Still, the challenge is to identify the indicators that would inform if a hybrid endeavor has achieved the expected results. Frameworks for institutional adoption and implementation of hybrid learning have been introduced to discuss key indicators related to institutional support, such as faculty incentives and technical support (Porter et al. 2016). Nevertheless, more data needs to be collected in order to understand the level of institutional adoption of hybrid initiatives and its contribution of curriculum implementation.

The H-MOOC framework

The H-MOOC framework organizes and systematically analyzes the implementation of MOOC-based hybrid initiatives as a continuum of two factors: (1) the institutional support needed (x-axis), and (2) the alignment of the hybrid initiative with the curricular content (y-axis) (Fig. 1). The framework assumes that the MOOCs used as part of the hybrid initiatives are already available (either created by the same institution or by a third party).
Fig. 1

H-MOOC framework. The ‘x-axis’ represents the institutional support needed to implement the initiative, and the ‘y-axis’ the alignment with the curricular content. The four basics models are proposed as a guide: (1) MOOC as a service, students take the MOOC voluntarily with no direct alignment with the content of any course in the curriculum; (2) MOOC as a replacement, the MOOC replaces a traditional course, with a direct alignment of the MOOC content with the existing courses in the curriculum; (3) MOOC as a driver, a traditional course in the curriculum is organized around a MOOC; and (4) MOOC as an added value, the institution provides all the support required to help students achieve the success in the course, but the course does not necessarily align with the content of a course in the curriculum

We define institutional support as the infrastructure, services and human resources needed to support the use of the MOOC (or of the MOOC content) for the learners participating in the hybrid initiative (Porter et al. 2016); we choose not to include the costs or investments in creating the MOOC since that is usually decoupled in terms of decision making at the institution. A low institutional support means that the institution invests little effort to provide infrastructure, services and human resources to launch the hybrid initiative. A high institutional support means that the institution invests much effort to provide infrastructure, human resources and associated services to help students advance on the hybrid initiative. For example, offering open and free study rooms for residential and non-residential students to work on a MOOC requires much less institutional support compared to a flipped classroom model, which will need the maximum teaching effort as well as the infrastructures typical from a f2f teaching practice.

Curricular content alignment indicates the proximity between an existing course syllabus in the institution and the syllabus of an existing MOOC (or MOOCs). A low level of alignment means that the MOOC is not aligned with the curricular content and it is used just as a complement in the hybrid initiative. However, a high level of alignment implies that the MOOC is core in the hybrid initiative. That is, the MOOC content is completely aligned with the curricular content of an existing course. Initiatives at a middle level of alignment make an indirect use of the MOOC, for example, as a reference textbook, because the content is not completely aligned with the content of the existing course, but the MOOC can still support certain aspects of the course syllabus. In some cases, and if the institution policies allow it, the institution might recognize the MOOC in form of credits, or as part of the final grade of a course in the curriculum, among other ways of acknowledging learning.

Through the continuum of these two factors, the H-MOOC framework enables the characterization of hybrid initiatives with different levels of institutional support and curricular content alignment. In Fig. 1, we present the four reference models as circles placed in the four corners of the framework.
  1. 1.

    The MOOC as service model (low on both ‘X’ and ‘Y’ axes) is typical from hybrid initiatives in which students use a MOOC (or part of it) voluntarily, and as a complement to the curriculum but without a direct alignment with the content of a course in the curriculum. For instance, universities such as Pontificia Universidad Católica de Chile (UC) or Universidad Carlos III de Madrid (UC3M) have been using MOOCs in order to leverage students’ previous knowledge for freshmen, some months before they start their first-year degree (Pérez-Sanagustín et al. 2016b; Kloos et al. 2014). In these models, taking the MOOC is a support for the students on content that is not traditionally covered by any course in the curriculum, but helps refresh concepts.

  2. 2.

    The MOOC as a replacement model (high on ‘X’ axis and low on ‘Y’ axis) is typical from hybrid initiatives in which the MOOC replaces a traditional course (or is used to extend the curriculum), with a direct alignment of the MOOC content with the course (or courses) replaced, but providing no educational nor institutional support in terms of physical infrastructure, nor services or local teaching support. In some cases, institutions might grant students with credits for taking the course. For instance, edX has recently released the initiative called “MicroMaster”. Massachusetts Institute of Technology (MIT) has joined this “MicroMaster” initiative for a pilot Master’s Degree on Supply Chain Management.1 With this initiative, learners can take a first semester fully online on edX, and a second semester on campus through traditional instruction, earning the full Master’s Degree at the end of both semesters.

  3. 3.

    The MOOC as a driver model (high on both ‘X’ and ‘Y’ axes) is typical from hybrid initiatives in which a traditional course in the curriculum is organized around a MOOC (e.g., flipped classroom), requiring high teaching and institutional support. In this case, the MOOC content is completely aligned with the content of the course in the curriculum that is hybridized. For instance, Stanford University studied how to fully integrate their Machine Learning MOOC into a graduate course on machine learning (Bruff et al. 2013), reflecting on the dimensions that should be considered and its complexity.

  4. 4.

    The MOOC as an added value model (high on ‘X’ axis and low on ‘Y’ axis) is typical from hybrid initiatives in which the institution provides all the support needed to help students achieve success in the MOOC (e.g., offering teaching classes, tutoring times, etc.), although the MOOC content does not align with the content of a course in the curriculum. The MOOC is not considered essential, but it can help acquire some extra knowledge or develop cross-curricular skills. For instance, UC3M has started to complement traditional on campus courses with MOOCs (Kloos et al. 2014), aiming at providing learners with extra materials, particularly a large number of practical exercises that cannot be covered in class time.


Apart from these four boundaries, we could also find other models that are “in between” the extremes of the two axes. An example model could be the use of MOOCs as textbooks in traditional classrooms, where the institutional support is lower than in a flipped classroom, and the content of the MOOCs are not necessarily fully aligned with the content of a course (or courses) in the curriculum.

Indicators related with H-MOOC

We have identified groups of indicators in the literature that could describe hybrid MOOC-based initiatives concerning learners’ experience, benefits for the teaching process, and institutional support requirements (Table 1). Although the importance of each of these groups of indicators may vary depending on contextual drivers and barriers, a combination of them could inform HE decision-makers about the cost-effectiveness of any hybrid initiative.
Table 1

Examples of indicators that are relevant for all MOOC-based initiatives

Learners’ experience

Teaching benefits

Institutional support requirements


Students’ satisfaction

Retention (completion rates)

Learning gains

Student use of MOOC content (interaction patterns)

Previous online experience

Prior knowledge

Students’ perception of teaching

Faculty self-perceptions

Face-to-face time (f2f)

Course and faculty development

Faculty technology literacy skills

Articulation between f2f and online components

Credit recognition

Infrastructure needs

Student support mechanisms

Technical support

Educational support

Faculty incentives

Decision-makers need information from different indicators in order to establish what the effect of combining an existing MOOC with traditional teaching practices might be. In Table 2, we have made an effort to organize the different indicators presented in the literature, and align them with the four reference models of the H-MOOC framework. This table suggests the indicators that could be used to quantify each dimension in H-MOOC, besides clarifying what indicators matter more in the four referential models of the framework. However, each institution should discuss which of these indicators are relevant according to the goals and expectations they want to achieve with the model.
Table 2

Examples of indicators whose relevance varies depending on the hybrid MOOC-based model

H-MOOC dimension


MOOC as a service

MOOC as a replacement


MOOC as added value

MOOC as a driver

Curricular content alignment

Learning gains





Interaction patterns





Face-to-face time




Articulation f2f and online components




Credit recognition





Institutional support

Infrastructure needs





Student support mechanisms





Technical support





Pedagogical support





Faculty incentives





The meaning of the ‘*’ is Relevant and ‘**’ More Relevant

For example, reducing f2f teaching time is an important incentive to use a MOOC as a driver if there is articulation between f2f components and an existing MOOC; that is why indicator f2f teaching time has ‘**’ in Table 2, meaning that it might be more relevant for HE decision makers when deciding to reuse a MOOC as a driver. Actually, in flipped classroom approaches (Kloos et al. 2015), lowering f2f teaching time for content delivery might enable teacher effort to foster active learning. In addition, in the MOOC as a driver model, aspects such as learning gains, the level of articulation with f2f and online components, and the pedagogical support needed by the teacher to carry on a flipped classroom initiative might be worth considering. In the case that traditional teaching instances are not necessarily aligned to an existing MOOC, such is the case of using a MOOC as a service, f2f time might be less important as a teaching benefit indicator (Zhang 2013). However, the use of a MOOC as a service model might consider more important student use of online content. For this reason, the interaction patterns indicator is marked with a ‘**’ in Table 2 for the MOOC as a service model, while the face-to-face time is not considered as an important indicator. Also indicators such as the students’ learning gains (usually evaluated as a final score in the course or in an exam), or the students’ support mechanism provided in the platform for students that would not have direct guidance from the teacher, might be important indicators.

Indicators of learners’ experience and teaching benefits would allow HE decision-makers to anticipate the results of the hybrid initiative in terms of curricular content alignment. This dimension could be determined by the alignment of the MOOC content with the syllabus of a course in the curriculum, or by the number of credits students receive from their participation in the MOOC, besides the improvement of students’ perception of the quality of teaching, and faculty self-perceptions. In terms of institutional support requirements, the H-MOOC framework does not consider institutional strategy and structure markers (Porter et al. 2016) by assuming the reuse of existing MOOCs within institutions that have already defined MOOC production policies and infrastructure. Nevertheless, the consideration of different support requirements is relevant for determining the costs of different hybrid models. For example, students’ support mechanisms in the form of tutoring time might be more needed in order to guide students’ learning in a MOOC as a service model, while other educational support might be more relevant in a MOOC as a driver approach. Measuring students’ learning gains would be especially important in the MOOC as a replacement or as a driver, as the MOOC is core for the methodological approach.

The H-MOOC framework in practice

This section illustrates how the H-MOOC framework works. We organize the hybrid MOOC-based initiatives described by Kloos et al. (2014) presented in section “Hybrid MOOC-based models” according to the framework. Then, we illustrate how the framework and the indicators proposed apply in actual contexts through two examples, a MOOC as a service initiative, and a MOOC as an added value initiative.

Organizing hybrid MOOC-based initiatives

The six models by Kloos et al. (2014) are classified according to the H-MOOC framework in Fig. 2. Two non-hybrid MOOC-based courses are used as a way to explain the two dimensions of the framework (see dark gray circles in Fig. 2): a fully remote course, completely aligned with the curricular content of an existing course, but with a low level of institutional support regarding infrastructures and services, and a f2f regular course, which is aligned to the curriculum and demands a significant institutional support regarding infrastructures, services and human resources.
Fig. 2

Hybrid MOOC-based models from the literature organized according to the H-MOOC framework: in dark gray two example non-hybrid MOOC-based models, in white those models that integrate online components, and in light gray hybridized remote courses

The two non-hybrid models from Fig. 2 can integrate online components in different ways. The f2f regular course can be hybridized by reusing MOOCs for a flipped classroom approach. In the flipped classroom, there is an extra effort on the teacher (and thus institutional support) to plan well ahead of class how to spend the f2f class time. However, in the local digital prelude, the number of f2f classes is reduced, and so does the institutional support. Additionally, a regular f2f course could be hybridized by reusing an existing MOOC as reference textbook without changing the course structure (i.e. canned teaching in f2f course). If the MOOC is used simply as a complement to the course (i.e. remote tutoring with f2f course), the curriculum alignment is lower than in the two models previously described (local digital prelude and canned teaching in f2f course).

The remote course model can be hybridized as well. By adding f2f tutoring to the remote course (i.e. canned digital teaching with f2f tutoring), institutional support increases. On the contrary, adding remote tutoring (i.e. canned digital teaching with remote tutoring) reduces it. As in the case of f2f regular courses, we can maintain the same structure in our remote course and hybridize it by using MOOCs as reference textbooks or complements to the course (see the two extra models added to Fig. 2 in light grey, canned teaching with remote course, remote tutoring in remote course).

Analyzing two successful hybrid initiatives

This section presents two case studies: (1) Calculus at UC, a MOOC as a service initiative, and (2) Programing at UC3M, a MOOC as an added value initiative. These two examples have been selected for being successful cases of using MOOCs in hybrid initiatives, and because the authors have been directly involved in their implementation in their respective institutions. In both cases, we explain the indicators used to analyze the impact of the initiative and how these helped decision-makers to reflect on the aspects to consider for future experiences. Specifically, we describe each of these two cases by answering four guiding questions that could serve as a reference for other institutions to apply the model:
  1. Q1

    Institutional goals. What are the main goals that the institution wants to achieve by applying this hybrid initiative?

  2. Q2

    Institutional support. What is the institutional support offered by the institution to implement this hybrid initiative and what are the main relevant indicators?

  3. Q3

    Curricular content alignment. How is the alignment of this hybrid initiative with the course curriculum and what are the main relevant indicators?

  4. Q4

    Lessons learned. What are the lessons learned in terms or the conditions that should be fulfilled for this hybrid initiative to work at an institutional level?


Analyzing a MOOC as a service: calculus at UC (Chile)

Around 700 freshmen are accepted every year to the School of Engineering of the Pontificia Universidad Católica de Chile (UC). They come with varying levels of knowledge on basic Calculus concepts. For the majority of these students, their knowledge on these topics often falls short to what is required for a first-year degree course. In the recent years, UC has offered remedial courses on Calculus for freshmen to address this problem. When students are admitted to the university, they are required to take a calculus diagnostic exam. The exam is divided into 4 modules: Algebra and Functions (M1), Trigonometry (M2), Polynomials and Complex Numbers (M3), and Successions and Summations (M4). Students who fail a specific module are required to take a 2-day intensive remedial course on each failed module. After each course, students have to take a final exam to evaluate their progress in the respective module content.

(Q1) Institutional goals

This strategy has been a way of promoting students’ calculus readiness, but has some limitations that need to be addressed: (1) low participation rates, students from regions out of Santiago had difficulties to attend; and (2) lack of personalization, not all the students need to review the same topics. In order to address these limitations, the school decided to produce 4 MOOCs, one for each module (Pérez-Sanagustín et al. 2016b). The MOOCs produced do not follow the same structure than the traditional remedial courses, but they cover the same contents and share the learning objectives. The School of Engineering encourages freshmen to take the courses as a service to review and practice the different topics of the course, both before the diagnostic exam and during the traditional courses. The aim of the initiative was to achieve two main goals: (1) providing a service for reviewing the content before the diagnosis exam; and (2) increasing the number of students passing the diagnostic exam.

(Q2) Institutional support and indicators

The support provided for the initiative was very low, since the institutional investment is dedicated to traditional remedial courses. One of the indicators is the infrastructure needs. In this case, the institution offered the Open edX platform for deploying the MOOCs; this required technical support from a part-time technician to maintain the platform. The institution also offered student support mechanisms for facilitating the registration in the MOOC platform as part of the freshmen induction process (where freshmen that pay the university fees are presented with all the services offered by the university). Neither educational support in the form of tutoring time nor student assistants nor faculty incentives were provided to run this initiative.

(Q3) Curricular content alignment and indicators

The MOOCs were open to everyone and were voluntary for the students of the School of Engineering, since their content were not aligned with any existing course in the curriculum. Therefore, for the purpose of this initiative only learning gains and interaction patterns were used as indicators related with the curricular content alignment. First, students’ interaction of the MOOC was studied in terms of the number of people that were active in the MOOC and how they used the MOOC content. Only the 589 (N = 589) students that participated in the diagnostic exam were considered for the institutional analysis. From this sample, up to 16% were active in the MOOC, having higher participation rates in some of the courses than in others. From this analysis it was observed that students interacted more with the courses before the diagnostic exam (with an average number of interactions with the course of 591 per day) than during the remedial courses (with an average of 61 interactions per day). Furthermore, the platform log files also showed that most of the students took the course for exercising and not for watching video-lectures. Second, students’ learning gains were analyzed in order to see whether using the MOOC had an effect. The analysis showed that students who were active in the MOOCs before the diagnostic exam showed better scores on this exam, but no significant effect was observed in the scores of students that were required to take final exams after traditional face-to-face courses.

(Q4) Lessons learned

The following requirements at an institutional level were needed to conduct the initiative: (1) to have a MOOC platform able to collect the information about the UC students registered in the courses; and (2) to include the registration steps within the induction process to assure that all the students could registered in the MOOCs. In addition, the institution learned that more efforts should be done in order to augment the adoption rates: (1) more support is required for marketing the courses; (2) higher efforts should be done in terms of curriculum alignment and make these courses mandatory.

This initiative on remedial courses on Calculus is classified as a “MOOC as a service” (Fig. 3). First, the alignment with the curriculum is low because the MOOCs were designed taking into account all the important topics required by students before entering the university, but without following the structure of any traditional course. Second, the institutional support is low, since UC only invested on spreading the initiative among freshmen, but did not offer any other support, such as tutors or facilities.
Fig. 3

Hybrid MOOC-based initiatives on calculus at UC (Chile) and on programming at UC3M (Spain)

Analyzing a MOOC as an added value: programming course at UC3M (Spain)

Systems Programming is a first-year, second-semester, mandatory course for the degrees of Telecommunication Technologies Engineering, Telematic Engineering, Audiovisual Systems Engineering and Communication Systems Engineering at Universidad Carlos III de Madrid (UC3M), Spain. Between 300 and 400 students annually take this mandatory course, which is offered in both Spanish and English languages. Systems Programming lasts 15 weeks, with 2 h of lecture and 2 h of laboratory class per week. This is the second programming course for students taking these degrees, after an introductory programming course in the first semester. Both programming courses in the first and second semesters use Java as the driving programming language.

(Q1) Institutional Goals

Several important problems cause a low passing rate in Systems Programming compared with other courses in the aforementioned degrees. The first problem is that the programming skill level of students that enroll in Systems Programming is lower than required, which does not allow them to keep pace with this course. This is partially due to the high number of students that fail the first semester programming course, or that drop after the first few weeks. Therefore, students taking Systems Programming need to reinforce their basic programming knowledge to cope with the challenges of this advanced programming course. The second problem teachers identified is that students make little or no use of the references provided as books or manuals on Java programming to catch up. Books and manuals are typically not very interactive, while students need to dedicate time to practice and code in order to strengthen the programming concepts explained during the course. In relation to this, a third problem is that class time is constrained by the study program, with 2 h per week for lecture in large group (with up to 100 students per class), and only 2 h for laboratory class in small group (with up to 40 students per class). This distribution limits the time that students can devote to implement their own programs, and the personalized support that the teacher can provide to learners. Therefore, the goal for using a MOOC is mainly adding value to the teaching and learning process of an existing face-to-face course, but without driving content delivery as in the case of a flipped classroom.

(Q2) Institutional Support and indicators

Students of Systems Programming 2015/2016 were highly recommended to enroll and follow an existing MOOC called “Introduction to Programming with Java—Part 1: Starting to Code in Java”. This is a popular MOOC developed by professors from different departments at UC3M and deployed in edX as a self-paced course during the 2015/2016 school year. Some of these professors also teach Systems Programming, so students can receive full support from their teachers on the contents of this MOOC. This MOOC is in English with closed captions in English and Spanish, and covers most of the contents of the first semester introductory programming course, plus the first sessions from Systems Programming. The MOOC is highly interactive with hundreds of basic and more advanced exercises for students to practice and code (Alario-Hoyos et al. 2016). In terms of indicators at an institutional support level this model considers: the infrastructure to deploy the MOOC (edX), the support mechanisms offered to the students’ enrolled in the courses, as well as the educational support provided by the teachers.

(Q3) Curricular content alignment and indicators

The interaction with online materials, although do not cover all the content of an existing course, they are completely aligned with it and can help them to prepare for the exams. There were more than 400 enrollees Systems Programming 2015/2016, about 80% of them in Spanish language, and the remaining ones in English language. These interaction patterns observed in the course served as an indicator for the creation of the second MOOC in order to guarantee the same accessibility level for learners taking Systems Programming in the two official languages, as well as to complete the part of the syllabus that was not covered with the existing MOOC. As a result of this analysis, teachers in Systems Programming 2015/2016 decided to develop a second MOOC in Spanish with the specific contents addressed in this course, recording videos with the theoretical concepts as well as solving exam problems, and including numerous exercises from previous exams adapted as automatic correction exercises for students to practice.

(Q4) Lessons learned

This initiative demands a high institutional support because of the tutoring times offered by the teachers to the university students, and because of the development of complementary online material in order to address specific contents that are covered in the regular face-to-face course but not in the MOOC.

This hybrid initiative on programing is classified as a “MOOC as an added value” (Fig. 3). The MOOC does not contain all the concepts needed to pass the course. However, the model implemented at UC3M-Spain has the highest institutional cost because of the development of complementary online material and the tutoring times offered by the teachers to the university students.


With the H-MOOC framework we aim to provide a guide to help institutions evaluate which initiatives for reusing MOOCs are more suited for their curriculum, students and faculty. Thanks to the H-MOOC framework, different MOOC-based initiatives within and across several institutions can be compared, and HE decision-makers can share what they have learned from their experiences and decision-making processes. As stated by Porter et al. (2016), who analyzed how HE institutions adopted online approaches with traditional learning, providing models, such as the H-MOOC framework, offers a very important perspective to help identify the drivers and barriers HE institutions need to face to strategically adopt educational innovations.

However, H-MOOC is only a first approach and presents some limitations that need further study. First, the H-MOOC framework might fall short to help redefine the way in which institutions deal with MOOCs and curriculum resources (e.g., figuring how to offer courses across institutions). As a first approach towards this aim, we have provided a set of guiding questions (see section “Analyzing two successful hybrid initiatives”) that can help decision-makers in Higher Education determine the most suitable model for them. Also, we illustrated with two examples of successful cases how the model was applied. But in addition to that, it would be useful to develop a manual with recommendations for adaptation purposes.

Second, the framework needs to validate how useful are the indicators proposed for other institutions to evaluate their hybrid MOOC-based initiatives and facilitate their comparison. This paper has only suggested a set of indicators based on the literature that affect the two main axes of the model, the curriculum alignment and the institutional support. However, these indicators could be extended by proposing also measures of quality in MOOCs, such as the ones proposed by Conole (2015).


This paper presents the H-MOOC frameworks, which aims to shed some light on the efforts that different HE institutions across the world have made to reuse MOOCs and integrate them as part of the traditional curricula, by providing a systematic way to define the space of hybrid learning initiatives that rely on, at least, one existing MOOC from the viewpoint of organizations. The H-MOOC framework establishes two key dimensions to describe this space: curricular content alignment and institutional support. According to these dimensions, four boundary reference hybrid MOOC-based models are proposed: (1) MOOC as a service, (2) MOOC as a replacement; (3) MOOC as a driver, and (4) MOOC as an added value. In a way, these models are a natural extension of how HE institutions think of traditional residential activities, and how they are set up: the university and the department decide on what educational activities are needed to support the curriculum they create, and what institutional support is needed. Some of these activities align strongly with the content of an existing course in the curriculum, and some could complement learning experiences, requiring more or less support from the institution.

To show how the H-MOOC framework works, we have classified some of the hybrid initiatives presented in the literature. It is noteworthy that some initiatives cannot be classified in the four boundary reference models, showing the utility of defining the framework as interplay of two axes in which the models can move vertically and horizontally. In addition, we also presented two successful case studies from two different institutions, and classified them according to the framework. These initiatives were measured in terms of indicators to understand their impact. Further, compared with other frameworks, the H-MOOC framework proposes a classification of the different initiatives from the learning goals they pursue, making it easier to analyze the institutional implications.

As future work, we plan to analyze more initiatives to validate the usage of this framework. We expect running experiments where the same MOOC is used in various models, in order to evaluate their impact and extract indicators for comparison. Also, future studies include working with different institutions and reporting the results obtained from comparing models across universities. Finally, we plan to analyze how the H-MOOC framework could be used not only as an analytical framework, but also, complemented with guides for teachers, as a means to inspire internal innovations in the use of MOOCs in HE institutions.


  1. 1.

    MITx MicroMasters on Supply Chain Management:



This work has been partially supported by the Fondecyt Project (11150231), the Project The Clover 2030 Engineering Strategy (14ENI2-26862), the Erasmus+ Programme of the European Union, projects MOOC-Maker (561533-EPP-1-2015-1-ES-EPPKA2-CBHE-JP), SHEILA (562080-EPP-1-2015-BE-EPPKA3-PI-FORWARD) and COMPETEN-SEA (574212-EPP-1-2016-1- NL-EPPKA2-CBHE-JP), by the Madrid Regional Government, through the eMadrid Excellence Network (S2013/ICE-2715), and by the Spanish Ministry of Economy and Competitiveness, project RESET (TIN2014-53199-C3-1-R) and fellowship FPDI-2013-17411.


  1. Alario-Hoyos, C., Kloos, C. D., Estévez-Ayres, I., Fernández-Panadero, C., Blasco, J., Pastrana, S., et al. (2016). Interactive activities: The key to learning programming with MOOCs. In Proceedings of the fourth European MOOCs Stakeholders Summit (EMOOCs), pp. 319–328.Google Scholar
  2. Bruff, D. O., Fisher, D. H., McEwen, K. E., & Smith, B. E. (2013). Wrapping a MOOC: Student perceptions of an experiment in blended learning. Journal of Online Learning and Teaching, 9(2), 187–199.Google Scholar
  3. Conole, G. (2015). MOOCs as disruptive technologies: Strategies for enhancing the learner experience and quality of MOOCs. Revista de Educación a Distancia, RED, 39.
  4. Firmin, R., Schiorring, E., Whitmer, J., Willett, T., Collins, E. D., & Sujitparapitaya, S. (2014). Case study: Using MOOCs for conventional college coursework. Distance Education, 35(2), 178–201.CrossRefGoogle Scholar
  5. Georgina, D. A., & Hosford, C. C. (2009). Higher education faculty perceptions on technology integration and training. Teaching and Teacher Education, 25(5), 690–696.CrossRefGoogle Scholar
  6. Georgina, D. A., & Olson, M. R. (2008). Integration of technology in higher education: A review of faculty self-perceptions. The Internet and Higher Education, 11(1), 1–8.CrossRefGoogle Scholar
  7. Ghadiri, K., Qayoumi, M. H., Junn, E., Hsu, P., & Sujitparapitaya, S. (2013). The transformative potential of blended learning using MIT edX’s 6.002 x online MOOC content combined with student team-based learning in class. Environment, 8, 14.Google Scholar
  8. Griffiths, R., Chingos, M., Mulhern, C., & Spies, R. (2014). Interactive online learning on campus: Testing MOOCs and other hybrid formats in the University System of Maryland. New York: Ithaka S + R.Google Scholar
  9. Joseph, A., & Nath, B. (2013). Integration of Massive Open Online Education (MOOC) system with in-Classroom Interaction and Assessment and Accreditation: An extensive report from a pilot study. In Proceedings of the international conference on e-learning, e-business, enterprise information systems, and e-Government (EEE), 105.Google Scholar
  10. Kerr, B. (2015). The flipped classroom in engineering education: A survey of the research. In Proceedings of the IEEE international conference on interactive collaborative learning, pp. 815–818.Google Scholar
  11. Kloos, C. D., Muñoz-Merino, P. J., Alario-Hoyos, C., Ayres, I. E., & Fernández-Panadero, C. (2015). Mixing and blending MOOC technologies with face-to-face pedagogies. In Proceedings of the IEEE Global Engineering Education Conference (EDUCON), Tallin, Estonia, pp. 967–971.Google Scholar
  12. Kloos, C. D., Muñoz-Merino, P. J., Muñoz-Organero, M., Alario-Hoyos, C., Pérez-Sanagustín, M., Ruipérez, J. A., & Sanz, J. L. (2014). Experiences running MOOCs and SPOCs at UC3M. In Proceedings of the IEEE Global Engineering Education Conference (EDUCON), pp. 884–891.Google Scholar
  13. Konstan, J. A., Walker, J. D., Brooks, D. C., Brown, K., & Ekstrand, M. D. (2015). Teaching recommender systems at large scale: Evaluation and lessons learned from a hybrid MOOC. ACM Transactions on Computer-Human Interaction, 22(2), 10.CrossRefGoogle Scholar
  14. McLean, S., Attardi, S. M., Faden, L., & Goldszmidt, M. (2016). Flipped classrooms and student learning: not just surface gains. Advances in Physiology Education, 40(1), 47–55.CrossRefGoogle Scholar
  15. Moskal, P., Dziuban, C., & Hartman, J. (2013). Blended learning: A dangerous idea? The Internet and Higher Education, 18, 15–23.CrossRefGoogle Scholar
  16. Ng’ambi, D., & Bozalek, V. (2015). Editorial: Massive open online courses (MOOCs): Disrupting teaching and learning practices in higher education. British Journal of Educational Technology, 46(3), 451–454.CrossRefGoogle Scholar
  17. Nissenson, P. M., & Shih, A. C. (2015). MOOC on a budget: Development and implementation of a low-cost MOOC at a State University. In Proceedings of the 122nd ASEE annual conference & exposition, pp. 1–25.Google Scholar
  18. Pérez-Sanagustín, M., Hernández-Correa, J., Gelmi, C., Hilliger, I., & Rodriguez, M. F. (2016b). Does taking a MOOC as a complement for remedial courses have an effect on my learning outcomes? A pilot study on calculus. In Proceedings of the European conference on technology enhanced learning (in press).Google Scholar
  19. Pérez-Sanagustín, M., Hilliger, I., Alario-Hoyos, C., Delgado Kloos, C., & Rayyan, S. (2016a). Describing MOOC-based Hybrid initiatives: The H-MOOC Framework. In Proceedings of the European MOOCs Stakeholders Summir (EMOOCs), pp. 159–172.Google Scholar
  20. Porter, W. W., Graham, C. R., Bodily, R. G., & Sandberg, D. S. (2016). A qualitative analysis of institutional drivers and barriers to blended learning adoption in higher education. Internet and Higher Education, 28, 17–27.CrossRefGoogle Scholar
  21. Riffell, S., & Sibley, D. (2005). Using web-based instruction to improve large undergraduate biology courses: An evaluation of a hybrid course format. Computers & Education, 44(3), 217–235.CrossRefGoogle Scholar
  22. Sandeen, C. (2013). Integrating MOOCs into traditional higher education: The emerging “MOOC 3.0” era. Change: The Magazine of Higher Learning, 45(6), 34–39.CrossRefGoogle Scholar
  23. Soffer, T., & Cohen, A. (2015). Implementation of Tel Aviv University MOOCs in academic curriculum: A pilot study. The International Review of Research in Open and Distributed Learning, 16(1), 80–97.CrossRefGoogle Scholar
  24. Wu, J. H., Tennyson, R. D., & Hsia, T. L. (2010). A study of student satisfaction in a blended e-learning system environment. Computers & Education, 55(1), 155–164.CrossRefGoogle Scholar
  25. Zhang, Y. (2013). Benefiting from MOOC. In World conference on educational multimedia. Hypermedia and Telecommunications, 2013(1), pp. 1372–1377.Google Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Mar Pérez-Sanagustín
    • 1
  • Isabel Hilliger
    • 1
  • Carlos Alario-Hoyos
    • 2
  • Carlos Delgado Kloos
    • 2
  • Saif Rayyan
    • 3
  1. 1.Pontificia Universidad Católica de ChileSantiagoChile
  2. 2.Universidad Carlos III De MadridMadridSpain
  3. 3.Massachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations