Introduction

Massive Open Online Courses (MOOCs) aim to provide open access to high quality learning resources for large numbers of participants, regardless of their background or geographical location. Despite their rapid rise to popularity, and the large number of registrations MOOCs attract, many people do not finish, with average completion generally acknowledged as being 15% or lower (Jordan, 2013). High attrition has been identified as one of the major problems faced by MOOC providers (Sinclair et al., 2015). MOOC learners represent a vast online learning community with diverse interests, motivational drivers and existing learning skills and, while some studies suggest that low completion rates are rooted in factors relating to learner motivation, commitment and enthusiasm, evidence is emerging that some potential learners do not possess the necessary independent learning skills required to self-regulate successfully and engage consistently within the prevailing MOOC format (Waite et al., 2013).

Learning in a MOOC environment depends on the capacity of participants to be effective, self-motivated learners. As captured by the concept of self-regulated learning (SRL), effective learning skills include aspects such as time management, prioritisation, planning, organisation of study and self-awareness (Bandura, 1997). Without such skills, learners working at a distance, in their own time and largely on their own may find it very difficult to maintain their initial momentum and to progress to completion. Despite this, most MOOCs show little awareness of the demands they are implicitly making and few provide the opportunity for learners to assess, personalised and explicitly develop their learning skills.

A common problem noted with MOOCs is their lack of pedagogic flexibility and adherence to one-size-fits-all “old-style” approaches of didactic, expert-led teaching without adequate personalisation of the content to the learners learning styles and interests (Sinclair et al., 2015). MOOCs lack more active and engaged learning strategies and the opportunity to be involved in directing one’s own learning are rarely offered in courses on most major MOOC platforms. The widely-used, more passive teaching methods, such as video lectures, are less likely to engage students in exploring deep learning.

While pedagogic issues are problematic in current MOOC platforms, some commentators have also raised the point that it may not be appropriate to use drop-out rates alone as a measure of success and that even defining what constitutes drop-out can be difficult (Clow, 2013). Learners who engage with a course at their own pace and to satisfy their individual learning objectives (rather than those of the overall course) may officially count as “dropouts” but yet have interacted with the course to their own satisfaction and achieved their objectives (Kizilcec et al., 2013). The point here is not to excuse attrition by redefining the term nor to try to paint MOOC drop-out in a more positive light but rather, as noted by Clow, that “Where we have indications of problems we have a responsibility to do what we can to address them” (Clow, 2013, p. 4). That is, if our courses are not offering suitable flexibility to support participants in their preferred ways of learning then this is a cause for concern. A learner’s goal may be to study only certain parts of a course but the current monolithic nature of most MOOCs means it is very difficult for learners to make informed choices about how this can be done and to find paths that are educationally cohesive and which meet their needs. Most MOOCs are stand-alone, give no idea of prerequisites for different topics (which would support informed decisions about accessing individual parts) and provide little navigational support for a learner making progress in their own way. Flexibility in this respect also relates to the issue of self-regulation by allowing users to take more control in directing their learning path. In a context which supports such flexibility, “success” in a MOOC can be related to learners’ own motivations and goals (Wang, 2014).

To investigate issues of self-regulated and self-directed learning in MOOCs we have developed a novel MOOC platform, known as e-learning development and adaptivity (eLDa), in which courses can be offered in a “traditional MOOC” mode (that is, as a structured, linear progression created by the instructor) but there is also the option for learners to choose their own learning paths. More additional features needed in a MOOC to inform and support learners in setting their own goals and determining a personal learning path which should include clarifying prerequisites for each unit of study (and supporting users in assessing their suitability). MOOCs platforms should incorporate adequate mechanism in assisting learners with the navigation and visualisation of their progress while engaging with the course content. This paper reports the design and development of the eLDa platform and presents preliminary results from its use in hosting a live MOOC. Data collected from participants allows us to determine preferences for different ways of learning and between externally-directed and self-directed study modes. This study aims to investigate the level of SRL skills among learners and the corresponding weaknesses observed in MOOCs. Further, using a survey instrument to investigate SRL, it is possible to investigate the MOOC learners’ strengths and weaknesses in different aspects of SRL and to relate these to their preferences for (and ultimately to their success in) different modes of study.

This paper is structured as follows. Firstly, an overview of relevant background literature is presented. Section 3 states the research questions addressed in this phase of the work and describes the methodology used. Section 4 outlines the architecture and development of the eLDa MOOC platform. Results of the data collection activity are presented in Sect. 5, followed by a discussion relating to these findings. Finally, we summarise conclusions and suggest areas for further research.

Related work

This section presents previous research relevant to the current study. First, this section will cover self-regulated learning and its conceptualisation in online contexts. Then, current thinking on aspects of good practice for MOOC platforms will be discussed.

Self-regulated learning

Developed by Bandura in the early 1980s, social cognitive theory recognises the importance of self-regulation in influencing all areas of purposeful human activity (Bandura, 1991). Bandura states: “Through exercise of forethought, people motivate themselves and guide their actions in an anticipatory, proactive way” (Bandura, 1991, p. 248). The concept of self-regulation includes factors such as self-motivation, self-monitoring and development of personal standards. Further, there is a strong relationship between self-regulation and self-efficacy (an individual’s belief in their ability to successfully complete tasks and attain goals) (Zimmerman et al., 1996). People who are effective at self-regulation are generally much more likely to take on challenging tasks and to persist in their efforts to achieve them (Zimmerman & Pons, 1986).

One of the main areas in which self-regulation has been greatly influential is education. Self-regulated learning refers to the process by which a learner takes control of, directs and evaluates their own learning (Butler & Winne, 1995). It encompasses dimensions of metacognition (reflection on one’s thinking), strategic action (planning, monitoring, and evaluating progress) and the motivation to learn. A wealth of studies conducted over 30 years has discovered a strong link between high self-regulation and effective learning: self-regulating learners learn best (Butler & Winne, 1995; Zimmerman & Schunk, 2001).

Definitions of the concept of self-regulated learning (SRL) have been expressed in slightly different ways by various authors. For example, Paris and Paris state that SRL “emphasizes autonomy and control by the individual who monitors, directs, and regulates actions toward goals of information acquisition, expanding expertise, and self-improvement” (Paris & Paris, 2001, p. 89). Zimmerman and Schunk view SRL as an approach that “seeks to explain how people improve their performance using a systematic or regular method of learning” (Zimmerman & Schunk, 2001, p. vii). In practice, SRL requires effective mastery of a range of skills generally acknowledged to include goal setting, task strategies, help seeking, environment structuring, time management, and self-evaluation (Barnard et al., 2008). These may be broken down further into explicit, concrete aspects, for example, a student’s effectiveness at environment structuring can be investigated by exploring whether they identify a distraction-free working environment for their study sessions; whether they are aware of what study environment suits them best and choose accordingly; and so on.

Some learners may implicitly recognise the need for SRL skills and demonstrate facility in developing and deploying them. More experienced learners and those who already have a background of academic study and achievement are more likely to have internalised and automatically put into practice appropriate SRL strategies which are effective for them(Zimmerman, 1998). The aspect of metacognition is important here since selfawareness of what works for oneself guides selection of the most suitable strategies (Butler & Winne, 1995). However, for many learners, explicit development of SRL skills, both early in their learning process and as an on-going process, is highly beneficial. A variety of research-informed approaches to development of SRL skills have been documented (Zimmerman et al., 1996).

Self-regulation in online learning

When studying online or at a distance, SRL skills are likely to be even more important given factors such as the greater need for learner independence, the lack of imposed structure for study times and the need to determine one’s own study environment. Students who engage in more online self-regulatory activities are associated with better academic outcomes and higher retention, and also show a more positive attitude in online course satisfaction surveys (Fisher & Baird, 2005; Howland & Moore, 2002). Conversely, lack of SRL skills is observed to prevent online learners from achieving expected learning tasks (Barnard et al., 2009).

The strategies learners need to deploy to achieve effective SRL are likely to be highly context dependent (Zimmerman & Schunk, 2001). Hence there will be differences between the skills needed in a “traditional” learning mode and those required in an online learning environment (and between different online environments also). Hence approaches needed to support learners in their development of SRL skills for an online context may overlap to some extent with those used in a traditional setting, but there will also be significant areas of difference. Some studies have sought to develop training tools which are specifically suited to online learners (and are themselves presented online) (Dabbagh & Kitsantas, 2005). Evaluating the effect of incorporating appropriate SRL skill training in an online course, Chang notes: “Students learning within a web-based environment with self-regulated learning strategies became more responsible for their own learning, more intrinsically orientated and more challengeable. They tended to value the learning material more and became more confident in course understanding and class performance.” (Chang, 2005, p. 217).

McManus explores differentiated learning approaches, finding that students with good SRL skills do not learn effectively within a strict, linear course structure (McManus, 2000). Conversely, students who are not effective self-regulators do not learn well in a highly nonlinear course where they are confronted with too many choices. The right level of autonomy in an e-learning course can empower students to develop SRL skills such as setting goals and planning a route to achieve them (Cunningham & Billingsley, 2002). In an autonomous course, learners can take control over their learning instead of being dependent on a fixed instructional path and passively consuming given content (Moore, 1993) but sufficient existing SRL skills are needed to leverage this (McManus, 2000).

Self-regulation in MOOCs

In the majority of MOOCs, the structure is highly linear and the teaching style is “top down” with content laid out by subject experts. Attempts to provide support, feedback and social contact are often made through activities such as forums and peer reviewing (Onah et al., 2014). In the context of the MOOC format, it is likely that a distinct range of SRL skills (and a high level of such skills) will be needed (Onah, 2017). Indeed, it is hardly surprising that the majority of participants in MOOCs are found to be highly educated, mature, experienced professionals with one or more existing degrees (Alcorn et al., 2014). Such learners are generally confident in exploring new ideas to extend their knowledge and expertise by following their own chosen learning paths (Pintrich, 1999). However, the rigid structure of most MOOCs takes away control from the learner, leaving a content-centred, linear course in which the instructors set all the goals. Further, the passive nature of most MOOCs means that students’ options for effective, active engagement are inadequate, their engagement and interest may suffer and dropout more probable (Academy, 2013; Reparaz et al., 2020).

Despite the need for SRL skills in order to achieve success in a MOOC, many such courses do not appear to have been constructed with any idea of building in support for fostering these skills, either implicitly or explicitly. Although MOOCs are open to all, they often do not cater for the variation of SRL levels that might be found amongst a wider range of participants, with those who do not possess the required levels of SRL skills feeling lost and failing to progress (Clara` & Barbera`, 2013). There is thus a need to ensure balance between the support and direction that some users will need, while as far as possible allowing effective self-regulating learners to control their own learning and set appropriate goals (Beaven et al., 2014). Given that effective self-regulation is associated with enhanced learning and better retention it is surprising that little attention has so far been given to this in the context of the majority of MOOCs.

Instruments to assess self-regulated learning

To evaluate students’ levels of SRL skills, a suitable approach or instrument is needed. Zimmerman and Pons used semi-structured interviews in which students were presented with a variety of learning contexts and asked what strategies they would use in each one (Zimmerman & Pons, 1986). This is a good way of obtaining rich data and generating new hypotheses, but it is not a practical means of assessment for courses with large numbers of participants, particularly online ones. An early, influential survey instrument (the Motivated Strategies for Learning questionnaire, MSLQ) was developed by Pintrich et al. (1993). This self-report, Likert-scaled instrument was designed to assess student motivation and use of learning strategies and has been widely used in other studies. A specific Self-Regulated Learning Instrument (SRLI) was introduced by Lindner and Harris, and uses a similar style of question (Lindner & Harris, 1993). A review of SRL assessment for classroom teaching conducted in 2000 indicated that surveys, interviews, teacher assessments and talk-aloud walk-throughs were all commonly used (Winne & Perry, 2000). Given the contextualized nature of SRL skills, for online and distributed environments, an appropriately targeted instrument is needed. To assess SRL in an online context, Barnard et al. (2009) developed a survey instrument which captures a conceptualization of SRL on six separate dimensions: environment structuring, goal setting, time management, help seeking, task strategies and self- evaluation. This instrument, known as the “Online Self-regulated Learning Questionnaire” (OSLQ), explores each of the six dimensions using between three and five questions. The survey instrument employed in the current research is based on OSLQ, adapted to the MOOC context.

Aspects of good practice in MOOC platforms

There are many existing MOOC platforms but there has so far been little discussion of pedagogic rationale in the development approaches of such platforms, their components and features (Sinclair et al., 2015). This section briefly reviews aspects of established good practice informing the development of our novel platform.

Pedagogic practice in MOOCs

Much e-learning development has focused on the development aspect and the provision of learning resources rather than the instructional design needed to ensure effective pedagogic content (Alexander, 2001). As noted by Alexander, “Successful e-learning takes place within a complex system involving the student experience of learning, teachers’ strategies, teachers’ planning and thinking, and the teaching/learning context” (Alexander, 2001, p. 240). Success of any e-learning course implementation requires careful consideration of the underlying pedagogy and how learners engage with the online content (Govindasamy, 2001). In the majority of MOOC learning platforms, the main instructional tool is video mini-lectures. This approach has been criticised as a major misconception of how teaching works, with MOOCs from major providers not going beyond level 1 of Bloom’s taxonomy (Bali, 2014). Some studies indicate the success of certain, specific strategies within the MOOC context include providing incentives such as badges, building activities around active learning, encouraging reflection and higher-order learning approaches and providing contact with staff (generally in necessarily impersonal form such as weekly emails) (Bali, 2014). Given the massive nature of such courses, pedagogic techniques must be scalable. Claims suggesting that current MOOCs can replicate traditional teaching for massive numbers of participants have been called naive, and the “student-facing” positioning of the major platform providers belies the reality of staff-poor, information provision which may be of benefit only to experienced, effective learners (Wiley, 2013).

Feedback

Timely feedback is generally acknowledged as being of major benefit in the learning process (Bali, 2014). It is also noted to be related to the development of SRL since the cycle of action, evaluation and reassessment benefits greatly from the input of reliable feedback (Butler & Winne, 1995). In a MOOC, with potentially many thousands of participants and very few instructors, personal feedback and direction is problematic. Current approaches include automated feedback and peer review. However, there are difficulties with both these approaches and many MOOCs appear to offer extremely limited feedback. MOOC users often feel lost and unsupported and express the opinion that there is insufficient help available (Sinclair et al., 2015).

Incorporating learning analytics

With the potential to collect and analyse large amounts of data from learning environments, learning analytics is now being used in a variety of ways, such as for the identification of students at risk of dropping out (Siemens & Long, 2011). One significant role that learning analytics can play in the context of MOOCs is to direct more personal provision of feedback to learners. Given the importance of feedback, particularly to those who are learning in a self-directed MOOC environment, receiving timely, relevant, personalized feedback and direction can help students evaluate their work, improve SRL and increase motivation in general (Arnold & Pistilli, 2012). This is another area with good potential but which is still in the early stages.

Methodology

The current work seeks to investigate issues of self-regulated learning and autonomy in the context of MOOCs. This section sets out the objectives of the work and the approach used to investigate the research questions.

Research questions

The specific research objectives addressed in the current paper are as follows:

  1. 1.

    What levels of SRL skills are demonstrated within a diverse MOOC learner group and are there particular areas of weakness which MOOCs should seek to improve?

  2. 2.

    To what extent do learners choose to direct their own study path as opposed to following a guided course?

  3. 3.

    Is there correlation between SRL skills and the learning path chosen?

This paper reports the first stage of our research to establish needs and develop strategies for fostering self-regulated learning in MOOCs.

Overarching methodology

A design science research methodology was used. This paradigm centres on the development and evaluation of an artefact to investigate a precise problem or problem domain. The methodological approach involves six steps: problem identification and motivation, definition of objectives for a solution, design and development, demonstration, evaluation and communication (Peffers et al., 2007). The approach is often incremental, with an artefact undergoing successive rounds of development, evaluation and feedback of results into the next iteration. In the present case, the eLDa MOOC platform (described in Sect. 4) was developed with the first-stage objectives of (a) supporting two modes of engagement (self-directed and instructor-led) and (b) collecting user data, in particular on SRL skills, learner preferences and chosen learning paths. Specifically, the introduction of user-directed learning paths requires support for making informed choices (such as clear preconditions for each element of the course, self-assessment of learners’ knowledge against prerequisites and information on where and how to fill in gaps if needed) and personalised learning maps of pathways taken so far, possible next steps and so on.

The platform has been trialled by using it for the development and delivery of a computing MOOC. The MOOC presents both computing concepts and provides grounding in Python programming and reuses tried materials from a previous course which had been run several times in “traditional” MOOC mode with over 900 participants in total (Sinclair et al., 2016). In addition to the novel features relating to self-regulation, the course was designed to incorporate a number of the acknowledged “good practice” approaches associated with promoting active learning and maintaining motivation in the MOOC context. The trial course developed for this stage of the research uses these existing materials as the basis for the new, experimental approach. The course was advertised to the learners via a computer science teachers’ community network and students in a blended-learning classroom setting.

A total of 107 participants were recruited for the trial run of the course by advertising the course via social networks, colleagues, the computing at school network and the local university community. The participants for the study were selected using purposeful sampling based on the registered learners. Due to the fact that this was being run as a live course, the need to provide a high quality learning experience was paramount and had to be balanced with the research needs of the exercise. A figure of around 100 participants was deemed to be a group size for which we could provide effective learning support in this initial delivery of the course. The 100 participants are active learners who have engaged with at least one or more components or resources and who required one form of support or facilitation. The course was conducted over a period of 7 months from mid May to the end of December 2015. In advance of the data collection activity, appropriate ethical approval was sought and obtained from the university’s research ethics committee.

Methods of data collection and analysis

Data collection for the current analysis was by means of a start-of-course survey administered to all course participants who enrolled on the computer science: computing concepts & python programming (online) module. As well as the more usual demographic and satisfaction information gathered by MOOCs (about the user, their aspirations, their experiences of the course and so on), an SRL survey was included together with questions relating to participants’ preferences for mode of study. The SRL survey was based on the OSLQ survey discussed in Sect. 2.1.3 which is an established SRL instrument, previously validated by its developers (Barnard et al., 2009). Our version includes slight modifications with targeted questions based on the concepts delivered and learners learning abilities to ensure suitability to the MOOC context. The survey questions were modified based on our research investigation, questions and the context of our MOOC provision. The instrument uses Likert-scale response questions covering the six SRL dimensions: environment structuring, goal setting, time management, help seeking, task strategies and self-evaluation. The full survey is presented in Sect. 5.2.1.

The general course surveys were administered to all participants. However, in order not to interfere with the participants’ learning experience, cooperation with completing the more detailed SRL surveys was sought on an optional basis. The responses to this were therefore from a subset of the overall cohort which comprises all those that opted in. The quantitative data collected in the course were exported to SPSS and the SRL results were analysed to obtain cohort statistics and learner profiles using a variety of appropriate statistical tests.

The eLDa MOOC platform

The eLDa (e-learning development and adaptivity) platform supports a novel approach to MOOC development which aims to actively involve participants in directing and regulating their own learning. It provides the necessary framework and support for participants to set their own learning goals and to access resources suitable for their needs.

Each course (or “module”) is divided into “sessions” which correspond to a coherent topic of study that (in a traditional, directed MOOC mode) might form a week’s unit of work. Each session is made up of a number of “lessons” with related concepts and content. In a directed mode of study, lessons are generally offered sequentially and mastery of all previous lessons/sessions is assumed in the current one. The eLDa platform decouples resources at the lesson level. Prerequisites are introduced to inform learners of necessary previous knowledge and, where appropriate, in which parts of the current MOOC that can be found. Learners can decide whether they wish to tackle that lesson with their current knowledge of prerequisites or whether they would prefer to review the suggested earlier lesson(s) first. A roadmap allows the user to see whether they have already studied the prerequisites. A learner can decide at any point to switch between modes. This can be useful, for example, if a learner wishes to refresh their knowledge of parts of earlier material, but then follow the course in a directed way.

Development of the eLDa computing MOOC

The course implemented on the eLDa platform to trial the approach was a computing MOOC, originally developed to provide continuing professional development for UK teachers. This course had previously been run twice in a “traditional” MOOC format, with a total of over 900 participants. It was therefore possible to use tried and trusted materials from the existing course, adapting them to the needs and format of the current context and creating additional materials as needed. The course covered computing concepts, introductory programming using Python and computing pedagogy. It comprised seven sessions and a total of 41 lessons. Figure 1 shows the standalone online course structure and visualisation of a few features.

Fig. 1
figure 1

Visualisation of the standalone online course on eLDaMOOC Platform

Full size image

Many features of “traditional” MOOCs were maintained, such as teaching videos, quizzes and forums. As noted above, care was taken to include other aspects of accepted good practice, such as incentives in the form of badges and social communication tools. Additionally, decisions were made on several aspects which could enhance the learner experience. For example, one such feature was the introduction of a facility for learner-tutor and peer-to-peer interaction. Although not the focus of the current analysis, this was thought a useful means by which to encourage social interaction and provide additional support. It was important to explore different opportunities for social learning given that participants on a self-directed path are not following a set timetable and it is therefore more difficult to coordinate interactions on, for example, a general forum.

Novel features necessary to the approach as described above were incorporated, such as the provision of information on prerequisites and the use of a road map to allow the learner to visualize their learning path (as illustrated in Fig. 3).

User response data was collected via built-in surveys. Similar to many MOOCs, general data on participant demographics, aspirations and so on was collected via a course entry survey. In addition, mini surveys were used in each session to elicit users’ feedback on the resources and on the suitability of recommendations made to them by the system. Learners then have the opportunity to provide useful information about the suitability of the recommended resources via the prerequisites system to their learning trajectory. The SRL questionnaire was administered at the start of the course to ascertain participants’ starting levels of SRL skills. Log data was also captured, recording all actions by participants throughout the course.

Design goals

The main design innovation is to support users in managing their learning if they wish to set and pursue their own study goals (Onah & Pang, 2021). There should still be the option to follow a learning path provided by the course instructor, allowing navigation of the full course in a guided, structured manner to achieve the overall course objectives. Thus, the platform should support two modes of learning: a self-study mode and the instructor-led mode in which a recommended order of topics covers the full course curriculum. To support users’ self-directed learning through informed choice, the system should offer advice on (but not enforce) recommended prerequisites for each topic and provide a map for learners to visualize the elements they have studied so far.

The platform should support good data collection and analysis features in order to evaluate participants’ SRL levels, path followed, interaction log data, attainment and evaluation responses on aspects such as satisfaction (Onah et al., 2021a). Since this is both a research tool and a platform for a live course, data collection is a particularly important aspect of the requirements but needs to be balanced with the need for learners not to be over-burdened with feedback requests.

In addition to the novel SRL features, the platform should, as far as possible, integrate a variety of acknowledged MOOC “good practice” features to support learners and mitigate against participant dropout. Again, although used as a research tool, the platform will be hosting a live course and it is important to provide a good learning experience to participants. Features include, for example, private messaging support for peer-to-peer and student-to tutor discussion to increase social learning. This is in addition to forums and provides a further support mechanism for students, allowing self-organization of smaller discussions between those students currently at a similar point. It can also encourage communication for participants who are nervous about contributing to a public forum.

The framework should encapsulate a mechanism for instructors to state lesson prerequisites and for these to be used to inform learners working in self-directed mode (Zhu & Doo, 2021). This can also provide an additional means for an instructor to monitor the learning progress and study patterns of the learners. The novel features of this architecture allow participants to self-direct their learning and to receive appropriate instructional support in order to attain their course objective, whether that is in reaching goals of their own or undertaking the full course in instructor-led mode (illustrated in Fig. 2).

Fig. 2
figure 2

Architecture of the eLDa platform

Full size image

Description of the architecture

The overall architecture is illustrated in Fig. 2. When an individual has registered and logs in, they are presented with a map visualising the whole module (or course) showing the sessions and the lessons which contribute to them. At this point, learners can decide which route to follow in order to attain optimum benefit from the course resources. The visualisation of the course and the statement of prerequisites support learners in making an informed choice of relating to their initial learning path. This is not fixed in the sense that a learner can decide at any point to switch between modes, either opting for a more structured, instructor-led path through part of the resources, or deciding to set their own objectives and change to self-directed learning.

The learner’s route

The dotted line in Fig. 2 indicates the pathway for self-directed learners. These learners can move freely from one resource to the other without any structure or restriction. Their progress and completion of individual lessons will be reflected in their personalised course map, allowing them to see at any point which areas they have completed.

The solid line indicates the pathway for the instructor-led mode of study. Learners who chose this route are led through the course in a structured, instructional manner. The learners in this route are restricted to following the course resources in a sequential order. While in instructor led mode, a student must complete all the associated prerequisites before going forward in the course. In the flow of the study, as noted above, learners may decide at any point to switch learning mode and become self-directed for the remainder or part of their study. Again, this decision is supported by the learner’s course map and by consideration of prerequisites for the different lessons in the course.

Interactive support

All learners on either mode are supported by a number of features which are regarded as general good practice within MOOCs. These features are associated with increased motivation and promotes learner interaction and engagement. The following elements are incorporated.

Quizzes

These allow learners to evaluate their understanding of the course concepts. They also provide instructors with information on learners’ progress and form the basis for awarding badges and certificates.

Exercises and solutions

Each lesson (apart from the introductory one) has programming exercises and model solutions embedded. This is another element which supports learners’ self-evaluation of their understanding. Providing model solutions for the programming elements allows students to work through (at least to some extent) problems in programming and compare their own solutions.

Forums

This interactive component enables learners to seek help from peers and tutors. It also encourages active participation and engagement, both through the act of asking questions but also through suggesting answers and contributing to general discussion of course issues.

Badges

Digital badges have been shown to provide an incentive which (for some learners at least) acts as a motivating factor and encourages participation (Gibson et al., 2015). Digital badges are described as symbols for certifying learners’ knowledge, their skills, and their competencies on several web-based platforms including MOOC (Hensiek et al., 2017; Mah, 2016). Badges are awarded when a learner starts the course and when they complete a lesson. Learners who complete the full course (following whatever mode) are awarded certificate of recognition.

Progress map

This provides the learner with an individual visualisation of the lessons and sessions completed. It indicates the concepts already studied and also shows the topics left to complete. This component helps direct and support learners in identifying their next step and accessing the appropriate resources quickly.

Surveys

These are vital for the collection of data relating to learner demographics and course satisfaction. However, they are also an important element of SRL for the learners, encouraging respondents to reflect on their learning and to be active in reviewing the provision of the course and influencing its direction for future learners.

Performance measuring metrics

The performance of the online participants was measured using the in-lesson quizzes and practical exercises.

Implementation

The eLDa course platform was implemented using WordPress—a free and open-source content management system based on PHP and MySQL. The choice of WordPress for this study was motivated by its suitability for incorporating the novel features of different learning modes and paths and for allowing the representation of learning prerequisites via compatible plugins. Before choosing WordPress, several other learning management systems were investigated. Despite their advantages in terms of learning support functionality, their structure and components made it more difficult to implement the novel features and requirements of the eLDa architecture. WordPress allowed a prototype (yet robust) system to be developed relatively quickly. For example, as illustrated in Fig. 3, WordPress creates a visual representation of the course content for mapping the session and lesson structure. This visualisation enables the learners to view an overview of the content and a representation of their own progress in a clear and simple manner.

Fig. 3
figure 3

Visualisation of course elements completed

Full size image

Additional PHP plugins were created to support further required functionality and features of the learning system. The Sensei plugin was used to create course content and lesson prerequisites; Wordfence provided security features; and Google analytics were applied to capture and represent the learner data both from the surveys and course resources such as modules, lessons, badges, videos and transcripts triggered from event log interaction. Other elements of the system include: an Apache web server; Macintosh Apache MySQL PHP; MySQL 5.5.42.cll.lve for database management.

Results

This section reports the initial results from data collected at the start of the course relating to participants’ demographics, their aspirations and their SRL skill levels.

Participant demographics

Of the 107 registered participants, 59.3% were male and 40.7% were female. Over a third (37.0%) were in the age range 35–44 and just over a quarter (25.9%) were aged between 25 and 34 (Fig. 4). In this course, less than 20% of participants were aged 45 or over.

Fig. 4
figure 4

Age of MOOC participants (n = 107)

Full size image

In line with previous research, our data indicates that the majority of participants (over 70%) were either graduates or current undergraduates (as illustrated in Fig. 5). Thus, most had existing experience of formal learning at a graduate level. It might therefore be expected that, in general, levels of SRL skills would be high.

Fig. 5
figure 5

Learners’ highest level of education

Full size image

Several questions in the pre-course survey explored the participants’ specific goals and their motivation for studying the course. In particular, they were asked what they expected to achieve by taking the course. Most frequently stated reasons are shown in Fig. 6. Most (over 60%) were motivated to learn new knowledge and skills directly related to the computing topics of the course. A further group (just under 10%) expressed their main objective as receiving a certificate rather than mastering the topic itself. A substantial minority (around 25%) were mainly driven by an interest in finding out about MOOCs and online learning and by a general curiosity to find out about the format. Over 5% of participants saw the course as a social experience in which they would be able to meet new friends.

Fig. 6
figure 6

Learners’ expectations

Full size image

The responses can be divided into two distinct categories: those participants for whom learning the subject is the primary motivation (about 60%) and those who have more abstract or tangential reasons for taking the course and who are less interested in mastering the subject (about 40%). The aims of the second group may be appropriate to their personal needs (finding out about MOOCs represents the acquisition of a different area of knowledge; making friends is a valuable social function). However, these are not directly related to the learning objectives of the course itself. This supports the view that, because goals differ between participants, no single, simple measure can be effective in judging whether the course met the students’ needs. Some of the objectives may be outside the scope and intentions of the course providers, yet these can nevertheless act as legitimate motivators for participants. Without asking each individual learner, it is not possible to know what their objectives, whether they have been met, or whether they are ones which the course would aim to fulfil.

Self-regulated learning skills

A subgroup of the enrolled participants (11 out of 107) completed the SRL survey. This study presents a small sample of data. This means that there is a lower reliability and we could not draw any logical conclusion. It is difficulty to generalise the result in the study due to the small sample size. The survey was a modified version of the OSLQ instrument adapted for use with MOOCs which will be referred to as MOSLQ. The survey questions evidence the six separate dimensions of SRL: goal setting, environment structuring, time management, task strategies, help seeking and self-evaluation.

SRL survey responses

Table 1 shows the results obtained relating to SRL skills of course participants. The first column of the table indicates the SRL dimension evidenced by that question as follows: self-evaluation (SE); goal setting (GS); time management (TM); environment structuring (ES); task strategies (TS); help seeking (HS). The percentage of participants selecting each of the Likert responses 1–5 (strongly disagree to strongly agree) is shown for each question in percentage, together with the average response for the item. As can be seen, there is a considerable variation in average responses (from 2.18 to 4.18 out of 5) indicating that some aspects of SRL are better developed than others.

Table 1 Responses to the MOSLQ survey
Full size table

The help seeking dimension is an indication that the learners were using the discussion forum provided for each of the lesson to ask for help and support from their peers and the tutors.

The lowest score related to writing down goals. While participants mostly claimed to set high standards for their work, fewer were likely to focus on articulating the objectives of their study and, even if they did, most did not keep a record. At the other end of the scale, most participants agreed or strongly agreed that they would be proactive in engaging and monitoring their progress on their chosen learning path.

The individual questions contribute to the six dimensions of SRL. Table 2 shows the results grouped according to these dimensions revealing a noticeable difference between scores on each. Respondents self-reported as being particularly effective at self-evaluation which incorporates reflecting on their own learning and reviewing their progress. However, they were much less inclined to seek help. Previous research by Onah et al. (2014, 2015) has noted the issue of low social participation by many learners. Our results show that a high proportion of learners set out with every intention of not engaging in forums (in our MOOC over 45%) or using other peer/tutor support channels (nearly 64% here). Although these results are from just one, small group of MOOC learners, they are nevertheless surprising and indicate an area for further investigation.

Table 2 Overall average result for each SRL dimension
Full size table

On any of the SRL assessment questions, it might be said that to indicate a “good” level of that particular skill, a learner should be selecting either “agree” or “strongly agree”. That is, we would view a score of 4 or above as indicating good SRL in that area. Table 2 shows that the only dimension for which the average achieves this is self-evaluation. It may be said that the results therefore indicate considerable room for improvement in all areas.

Visualisation of SRL results

Figures 7 and 8 use radar charts to emphasise the contributions of the different SRL dimensions, providing a visual presentation of the overall SRL profile. Figure 7 underlines the fact that, even in dimensions with a score of above 3, there are individual questions indicating areas in which considerable improvement could be made. For example, goal setting achieves a score of 3.35 but recording goals only achieves 2.18.

Fig. 7
figure 7

Visualisation of average SRL scores

Full size image
Fig. 8
figure 8

Visualisation of average SRL scores by dimension

Full size image

Figure 8 Clearly shows the lack of help-seeking intention amongst the group. It may be that MOOC learners do not expect this to be an effective way for them to study, however, it is concerning that some plan never to contribute or seek help in any way. The result has shown low help-seeking dimension from most of the learners

Results by individual learner

The results shown above represent the average position across the whole cohort and provide an indication of which SRL skills are under-represented in general. For each individual learner (and if the system is to provide personalised support) it is important to consider the individual profiles of each participant. Given the small number of respondents in our sample, it is possible to present here the profiles for all 11. Table 3 shows the average SRL score for each learner. While there is one outlier in the average SRL score in each direction (that is, one learner with an average of 4.33, another with average 2.67) most respondents had average scores of between 3 and 4. Given that the participants have a successful track-record in formal education, and bearing in mind that a level of 3 represents a “neutral” response to questions, these numbers are lower than might have been expected.

Table 3 Average SRL score for each learner
Full size table

Table 4 provides the descriptive statistics for the 11 participants in the study.

Table 4 Descriptive statistics for 11 participants
Full size table

Figure 9 shows the learners’ SRL scores. While two learners may have a similar average, their profiles may differ considerably with each having their own particular SRL strengths and weaknesses. Hence, to provide effective support for SRL it is necessary firstly to perform a diagnostic assessment and secondly to provide different strategies depending on which dimensions are weak. Again, the low emphasis placed by all but one learner on help-seeking is striking.

Fig. 9
figure 9

Individual learners’ SRL scores with respect to the six dimensions

Full size image

As well as investigating MOOC learners SRL levels, we were interested in finding the participants’ preferences for mode of study and hence the likely take-up of self-directed learning paths. The two basic modes of study offered were: self-directed and instructor led. However, given that the platform supports switching between modes it is also possible for learners to plan a combination of the two. This might also be regarded as a self-regulation strategy as it involves choice and direction by the learner. Further, since the SRL survey was administered at the start of the course, some learners were not yet decided. The number of learners selecting each of these four options is shown in Fig. 10. The results show that the majority of learners would like either to direct their own learning entirely or to move between modes, suggesting that for many MOOC learners more self-direction would be highly desirable.

Fig. 10
figure 10

Learners’ preferred mode of study

Full size image

Relationship between SRL and study mode

We are interested to see whether levels of SRL skills relate to participants’ choice of learning path and, ultimately, to their attainment within MOOC study. The latter question will be considered when data from the completed course is collected and analysed. At this point, the available data relates to the start of the course and students’ intentions towards mode of study. A quantitative analysis of the relationship would be preferable. However, for the small number of data points available in this preliminary study, it is not possible to meaningfully apply quantitative methods to the data. For example, although the Fisher exact test is applicable to small samples, a data set of only 11 cannot provide evidence for rejecting the null hypothesis. Therefore, we know that small sample means lower reliability, more difficult to draw conclusions or generalised the result. We therefore present the figures in a descriptive manner, viewing them as indicative only and providing suggestions for future investigation with larger numbers.

Table 5 shows, for each dimension of SRL, the number of learners who selected a self-directed path and those who did not. The results are further differentiated between learners who show a higher level or a lower level of the SRL dimension under consideration. Thus, for each dimension there is a grid representing the distribution across two separate variables (SRL dimension and choice of learning path). With more data, this format would allow investigation by the Fisher test with null hypothesis “The probability of choosing a self-directed or instructor-led path is the same for those with higher SRL skill and lower SRL skill”. For the “lower level” of learning skill we include values of ≤ 3, with values greater than 3 classified as “higher level”. Similarly, options of “self-directed” and “mixed mode” are grouped together as “learner directed” since these both indicate the intention of the learner to take control and switch as appropriate. In this study, we were able to eliminate the impact of the changing levels of the SRL skills on the analytical results based on the learner-directed route as illustrated in Table 5.

Table 5 Choice of learning path related to SRL levels
Full size table

In terms of these classifications, each dimension reveals a split between choice of study mode in which participants appear quite likely to choose either path whatever their SRL level. That is, there seems to be little indication that SRL levels are affecting choice of study mode. However, the results do indicate the more polarised positions regarding help seeking and self-evaluation. All but one participant falls into the “low” category for help seeking, but again there is little evidence of difference in choice between learning paths. Self-evaluation displays a reverse pattern, with all but one participant being classified as “high” in this dimension, although once more the choice of path seems little affected. The indications so far are that learners have definite preferences for their mode of study and the degree of autonomy they would like, however, this appears not to be related to their SRL skills. This may suggest that, although most learners would like to direct their own learning and decide on suitable objectives and learning path, many may lack the necessary skills of self-regulation to be able to do this effectively.

A further point is that the classifications used here may be over-generous. We have taken “high” to be anything above a “neutral” response”––even if only slightly. It might be argued that it would be more appropriate to include a learner in this category only if they at least “accept” the SRL strategy stated. On this measure, only one dimension (self-evaluation) would be regarded as having a “high” average and only one participant would be classified as a generally effective self-regulating learner.

Visualising SRL profiles for different study modes

Figures 11, 12 and 13 show profiles of SRL for the three study modes: self-directed, instructor led and combined respectively. In each case, the profile was created by plotting the average score on each dimension for all learners choosing that mode of study.

Fig. 11
figure 11

Average of SRL dimensions for 7 learners who preferred a self-directed learning path

Full size image
Fig. 12
figure 12

Average of SRL dimensions for 4 learners who preferred an instructor-led learning path

Full size image
Fig. 13
figure 13

Average of SRL dimensions for learners who preferred to mix self-directed and instructor-led learning paths

Full size image

Although the diagrams are based on a small number of data points they present some interesting features which suggest areas appropriate for further investigation. The profile for students choosing the instructor-led mode is notable in that no dimension is higher than 4. There is therefore no aspect of SRL in which these learners deploy strong SRL strategies. This contrasts with the participants opting for a mixed approach to study who score relatively highly in four dimensions, but with noticeably lower scores in help seeking and task strategies. This group appears confident in their self-direction but have already decided that they will not seek help or take part in social learning activity. Although further work is needed, it may be the case that some learners are so confident about their learning skills that they do not anticipate needing support, or that they do not realise the benefits of this type of interaction when engaged in online learning. The third group (those who choose self-directed learning) includes more diversity in SRL levels but in general lies somewhere between the other two. This may suggest that, on the whole, learners are choosing their mode of study wisely (that is, greater direction for those who have lower levels of SRL skill). In addition, learners with higher SRL skills recognise the benefits of blending self-direction with guidance when in unfamiliar territory and have the confidence to feel they can take control of directing their path to switch between the two modes as appropriate.

Discussion

Our first research question relates to investigating the levels of SRL skills demonstrated by MOOC learners. In the context of the trialled course, the levels of SRL overall (Table 2) showed considerable room for improvement, with self-evaluation being the only dimension scoring 4 or above. Further, the two dimensions of help seeking and task strategies both scored below 3, indicating disinclination of the participants to engage in these activities. These are perhaps surprising results given the high levels of education of the participants and their obviously successful track record of prior learning. As noted above, the concept of SRL is highly context-dependent (Zimmerman et al., 1996). A group of learners may be experienced in a more traditional learning setting, and some of the necessary SRL skills may overlap, but there may be other aspects which need further development. Both help seeking and task strategies need different approaches in a MOOC setting. For example, students used to asking questions in a class may not translate this to the need to participate in peer discussions. Effective learners are aware of the strategies for maximizing their learning gain. The stated intention of many of our learners not to participate in certain activities suggests that they may be unaware that, in an online context, activities such as engagement in course forums are not just peripheral and time-consuming but provide purposeful and effective learning mechanisms (Liu et al., 2022). The choice of learning path was another indication on how the learners make informed decisions about their independent learning routes and choices.

Even for MOOC participants with a strong learning track record, it cannot be automatically assumed that this will translate directly to the requirements for effective MOOC study. The situation is likely to be even more challenging for those without a strong learning background. In this case, they will require instructor led learning choices. Currently, the lack of support in MOOCs for developing the necessary skills may render them inaccessible to many and may be a contributory factor in cases where participants do not achieve their learning objectives.

The second research question considers the extent to which MOOC learners choose to direct their own studies. Our data shows a high demand for this to be made possible as a result of their directed learning choices. Learners were very positive about moving from the current situation of monolithic, highly directed courses to one in which they could make informed decisions on what to study next. The highly “siloed” approach of most MOOCs means that they are viewed as stand-alone and little attempt is made to provide access to constituent parts. There are some instances of linked MOOCs, but this is generally of a very basic, linear nature: such as an introductory MOOC which must be completed before the advanced topic MOOC. Our prototype allows additional metadata to be attached to a section of learning resources, such as the prerequisites needed. Further, it provides links to where material on those prerequisites may be found. Currently, this is limited to within the single course, but a useful expansion would be to introduce a general scheme for recording such information and allowing cross-referencing between (parts of) different MOOCs.

Investigating the relationship between SRL levels and choice of learning path we found that learners in our MOOC were reasonably good at selecting a mode suitable to them. This is important given McManus’ findings on the need to match SRL to the appropriate study approach (McManus, 2000). A symbiotic relationship is thus suggested between autonomy in a MOOC (that is, freedom of movement by the learner within the studying environment, without having to adhere to a predetermined order or sequence) and the development of effective SRL skills. Practicing skills of self-direction improves SRL: higher levels of SRL allow the learner to benefit more from self-directed learning. This suggests that to provide the best support for different learners, it is necessary to provide a level of adaptivity that can offer students different learning structures (and which can alter as the student’s SRL skills develop).

Major MOOC providers have been criticised as enshrining a “one-size-fits-all” approach to course development and some authors have started to explore models for more adaptive presentations (Sonwalkar, 2013). However, adaptivity on a meaningful scale is notoriously difficult to achieve both in terms of suitable platform and tools but also because of the skill needed to author effective adaptive courses. Hence, it seems that while this is an exciting prospect, it is still at an early research stage.

The lack of consideration for appropriate pedagogy in the rapid development of MOOCs means that little attention has been paid to how SRL should be contextualised and supported in this setting. Strategies for fostering SRL in e-learning can be implicit, in the sense that they are built into the course by, for example, choosing learning activities which involve exercising and developing certain skills. They may also be explicit, directing students to reflect on exercising the skill and raising metacognition of the processes involved. The first step is to identify areas of weakness which should be targeted and for this an effective diagnostic tool (such as a pre-course survey) is needed. Our work explores one aspect in which users can be allowed to take responsibility for directing their own learning and preliminary results suggest this to be a viable means of introducing learner autonomy in a MOOC (Onah et al., 2021a, b). Further aspects, such as guiding students towards explicit consideration and articulation of goal-setting would not be difficult to introduce.

The work reported here has several limitations. Firstly, the sample of learners from which data was gathered is small. It is therefore not possible to claim that these results are generalisable. However, they do provide an indication of useful areas to consider and questions to investigate in a larger scale exercise. Certainly, the overwhelming acceptance of allowing different modes of study is greatly encouraging in our work to develop the eLDa platform.

As with all self-reported data, the reliability of participants’ answers may also be an issue. Ideally, triangulation using a different form of data collection or by asking the same question in different ways could be employed. However, in a “real” course there is a need to balance the data collection activity so that it does not become burdensome (and perhaps less likely to elicit considered answers as a result). Further, it may not be reasonable to expect internal consistency between questions contributing to an SRL dimension. Learners well-practiced in SRL in a different context may display high levels on several aspects but are unfamiliar with the need to exercise others.

Finally, we note the diversity of learners’ motivations. It may not be possible for MOOC providers to satisfy all of the wide range of expectations, particularly where these are not related to academic objectives. However, developing a greater understanding of what and how participants want to study and providing the means for them to achieve this can provide more flexibility in the MOOC format and offer a learning experience which is both better matched to needs and encourages self-regulation.

Conclusions and future work

Current mainstream MOOC approaches fail to consider many aspects of pedagogy which educational research has long established to be beneficial for effective teaching and learning. In particular, many MOOCs tend to be inflexible, “one size fits all” courses which encourage passive engagement and allow little scope for students to direct and regulate their own learning. Our findings support previous research which indicates that most MOOC participants are highly educated with a track record of effective prior learning. However, our results indicate that this group of learners did not, as might be expected, score highly on many dimensions of self-regulation for effective online learning. In particular, help seeking and social interaction strategies were very low, and strategies for effective task management (such as planning) were not highly developed. This accords with the contextualised nature of SRL and we conclude that, even for participants with a track record of educational achievement, it is not safe to assume that they will be effective at self-regulation in a MOOC context. Further for other groups of learners with less formal educational background (the very learners whom it has been suggested MOOCs might cater for), the gap between SRL skills needed for success and those actually possessed is likely to be even greater. This indicates the need for MOOCs to incorporate ways to develop learners’ SRL skills.

Secondly, we found that most learners were keen to direct their own learning path and that those displaying the greatest levels of SRL planned to blend their own path-setting with following the instructor-led route in sections of the course for which that suited them. Thus, learners are demonstrating their desire to be more autonomous and to develop individual learning goals. The eLDa platform, providing support for informed goal-setting and effective navigation, has been well-received by learners. We are currently analysing data from the completed course and will investigate relationships between SRL and attainment, and choice of learning path and attainment. Given that all learners are unique in their learning preferences and approaches and in the ways they might interact with an online course, a rich adaptive model might be seen as a holy grail for MOOCs. However, this is difficult to achieve in practice. providing a format in which sections of courses can be decoupled and where learners are supported in navigating them in a path suitable to them achieves a step in the right direction by making different routes feasible. It combines MOOC technology and ethos with a more “learning object” type approach in which distinctive units of learning resources can be combined. Our platform currently allows this to be achieved within a single course. However, the approach can be extended to work between courses, allowing resources on necessary prerequisites to be referenced and obtained from other courses.

This work indicates the need to support and develop SRL skills in MOOCs. The novel feature of allowing learners to set their own goals in itself helps participants exercise and develop skills of self-determination. However, there are many other ways in which MOOCs could incorporate aspects of support. In line with a design science approach, future development of our platform will investigate ways of increasing and promoting social learning and the use of enhanced help-seeking strategies. As well as building support into the platform, it is important to increase learners’ self-awareness of their capabilities in these skills and their understanding of the importance of such skills for effective learning. Providing practical help for increasing their skill levels will provide learners with the tools to improve their SRL abilities and hence increase their effectiveness in establishing realistic learning objectives and pursuing them successfully.

The current research has gathered data from only a small number of MOOC participants but from this, themes of interest have emerged for further investigation. Our future research work will extend the trial by gathering data from a different course using two concurrent cohorts. This will allow us not only to extend the data relating to SRL amongst MOOC participants but to compare SRL skills, development and attainment between MOOCs used in a fully online mode and those used for blended learning in conjunction with classroom teaching. Further, we will investigate additional ways in which user data can be harnessed to support SRL. For example, test scores may indicate weakness in certain areas, allowing targeted feedback and personalised suggestions of appropriate remedial learning materials to be offered.