“Time is the bottleneck”: a qualitative study exploring why learners drop out of MOOCs

Why do over 90% of the learners in Massive Open Online Courses (MOOCs) never finish the course? There is a need for further studies focusing on the learners’ experiences of participating in MOOCs and factors that influence the decision to complete or drop out of the course. To deepen our understanding of why learners complete or drop out of MOOCs, we report on a qualitative case study based on in-depth interviews with 34 learners with different degrees of course completion for two MOOCs. A qualitative analysis of the interviews led to the identification of four main factors influencing dropout: (1) the learner’s perception of the course content, (2) the learner’s perception of the course design, (3) the learner’s social situation and characteristics, and (4) the learner’s ability to find and manage time effectively. How the learners conceptualized a MOOC had a strong impact on how they engaged with the contents. We discuss the implications of our results for MOOC practice in terms of time, openness and accessibility and provide recommendations for future research.


Introduction
Massive Open Online Courses (MOOCs) have increasingly captured the attention of scholars, higher education institutions, the public and the media over the past few years. While recognized as a potentially important educational practice with significant impact on the future of online learning (Siemens 2015), current MOOCs suffer from extremely low retention with dropout rates of 90% or more (Yang et al. 2013; Khalil and Ebner 2014;Liyanagunawardena et al. 2014;Veletsianos and Shepherdson 2016). In comparison with traditional university courses (online or campus based), this result appears devastating, but such a comparison is based on the assumption that enrolling in a MOOC is more or less tantamount to enrolling in a traditional course. This assumption is problematic as retention and dropout rates in MOOCs can only be meaningfully interpreted within the context of the learners' intentions and needs (Koller et al. 2013) and it has been suggested that MOOC attendance compares better to the consumption of other media channels-such as newspapers, books and TV series (e.g. Downes 2014;Reich 2013).
Researchers still have very little understanding of what learner needs MOOCs may address and how (Zheng et al. 2015). Developing a better understanding of the reasons why learners complete or drop out of MOOCs is crucial in this regard. However, MOOC research so far has mostly employed quantitative, if not positivist, methodologies and there have been surprisingly few qualitative and interpretative studies exploring learners' experiences of participating in MOOCs (Veletsianos and Shepherdson 2016). Thus, ''while researchers can say with increasing confidence what they observe learners doing in MOOCs, empirical evidence on why they do what they do, how they do what they do, and what it is like to participate in MOOCs is scarce'' (Veletsianos et al. 2015, 571). Further, learners not finishing a MOOC are unlikely to answer post-course surveys, making it difficult to gain access to this important subpopulation.
In this article, we address the need for a deeper understanding of learners' experiences of participating in MOOCs by reporting on a qualitative case study involving two MOOCs. The question we ask is: What factors influence the learners' decisions to drop out of MOOCs? From a pool of learners that agreed to be interviewed when registering for the courses, we interviewed 34 learners about their MOOC experience, and particularly their reasoning concerning MOOC completion and dropout. This qualitative research approach not only provided richer information, but also gave us better access to learners that did not finish the courses.

Research setting
Chalmers University of Technology started its MOOC initiative, ChalmersX, in 2014. The first two MOOCs, Introduction to Graphene Science and Technology (ChM001x) and Sustainability in Everyday Life (ChM002x), were released in 2015 on edX. ChM002x is an introductory-level course in sustainable development, only requiring background knowledge gained in compulsory school. ChM001x, on the other hand, is an advanced-level course requiring an adequate knowledge of general physics and university level mathematics. Both courses consist of video lectures, often followed by quizzes, weekly assignments (in the form of automatically graded multiple-choice questions), a final exam, and weekly video updates. In ChM001x, the final exam consisted of multiple-choice questions and included the interpretation of a scientific article, whereas ChM002x finished with an open response assignment graded via peer assessment. Learners needed to complete at least 60% of all graded assignments to earn an honor code certificate.
Selected course data for the two MOOCs are shown in Table 1. With 4% and 2%, the completion rate was comparatively low for both courses. Figures 1 and 2 show the participant activity over time during the run of the MOOCs. We see that only about a fourth of the learners that signed up returned to the course to start an activity, and that numbers dropped significantly as the MOOCs progressed.

Research approach
As the purpose of this study was to explore the learners' experience of participating in the Chalmers MOOCs and identify factors that influence the decision to continue or drop out, we employed a qualitative case study approach (Case and Light 2011). Data was collected through semi-structured interviews with 34 learners, 18 from ChM001x and 16 from ChM002x. The interviews were primarily conducted via Skype (three via physical meeting) and lasted about 30 min each. All interviews were audio recorded. The interviews were conducted mid-May 2015 for ChM001x, about two weeks before the deadline for the final assignment, and mid-August 2015 for ChM002x, about two weeks after the deadline for the final assignment. The interviewees were sampled by self-selection; in the pre-course survey learners could choose to be interviewed (617 did so for ChM001x, and 537 for ChM002x). We sent an e-mail to a random subset of these, asking for booking an interview time (230 and 220, respectively). In this e-mail it was stated explicitly that ''it doesn't matter to what extent you have followed the MOOC, or if you never started''. The learners we interviewed covered different levels of completion, according to Table 2.
We chose to keep the interview questions as open as possible in order to minimize the effect of leading questions on the responses. We initiated the discussion about MOOC completion and dropout with the question: ''Did you finalize the MOOC or not?''. The learner was free to interpret the word ''finalize'' from their own perspective. During the following discussion it became clear what the specific learner meant and in almost all interviews the learners implied that ''finalize'' meant completing all or almost all learning activities of the MOOC.
The interview primarily focused on the two Chalmers MOOCs, but for learners that had participated in other MOOCs, questions were also asked about those experiences.
The interviews were analysed using a general inductive approach (Thomas 2006) for qualitative analysis. The process consists of breaking up the data into smaller segments and coding each segment with a short phrase to capture its meaning. The codes that were used for categorizing the interview content were not predetermined; instead we constructed the codes during the analysis. The codes were then sorted   and sifted in an iterative way, leading to the identification of the themes. While the coding was performed by the first author, all three authors of this article were involved in developing the categories and discussing their interpretation. The two Chalmers MOOCs in this study have been produced in Sweden, Europe, while the learners have been from a global community, primarily from Europe, North America and Asia. Since both of the MOOCs used the North American edX platform, one limitation in our study is that we have not considered what impact the nationality of the MOOC platform could have on completion rates. Policies regarding certification could possibly have such an effect.
In the following, we will present the identified themes or factors that influenced the learners' decisions to drop out of MOOCs, followed by a discussion of the results against the background of previous research on the topic.

Results
A qualitative analysis of the interviews led to the identification of four overarching themes or factors that influenced the decision to drop out: (1) the learner's perception of the course content, (2) the learner's perception of the course design, Perceived level of difficulty of the content 3 The learner's perception of the course design Discouragement based on faulty assignment 5 The learner's social situation and characteristics External factors 5

Socio-economic factors
Internet access 3 English proficiency 1 Learner personality traits 5 The learner's ability to find and manage time effectively

Lack of time 21
Study techniques 3 Numbers indicate how many interviewees that mentioned each factor ''Time is the bottleneck'': a qualitative study exploring… 137 (3) the learner's social situation and characteristics, and (4) the learner's ability to find and manage time effectively. Table 3 provides a summary of the factors affecting completion and dropout rates, sorted under the four overarching themes. Note that the number of mentionings does not necessarily reflect the number of dropouts due to each factor. For example, the five mentionings of personality traits concern learners that explained their completion, not their dropout, with a specific trait in their own personality.

The learner's perception of the course content
When reasoning about completing or dropping out of MOOCs, a majority of the interviewees mentioned aspects related to the content of the MOOC.

Motivational aspects of the content
A common explanation for either dropping out or for completing a MOOC was related to motivation or interest and course content: The first reason to drop out is that the course was not what I expected, really boring and not related to what I searched for.
A mismatch between expectations and the actual content was frequently mentioned, and a few learners stated that they usually notice this quite quickly, for example during the first week of the course. If this was a dominant reason for the dropout in the two Chalmers MOOCs, then a distinct drop of participation would be clearly visible in the quantitative data. However, no such drop is apparent in Figs. 1 and 2, which suggests that a mismatch between content expectations and actual content is a factor, but not a dominating factor.
The motivational aspect of the content can be divided into two categories: utilitarian and enjoyment. In the first category, learners stated that they completed the MOOC because it was useful for their professional occupation, research, conventional studies, or what they hoped to work with in the future. If the learners believed the MOOC content will be of help in their future careers, the learners tended to have an incitement to complete the MOOC. Eight out of the 34 interviewees mentioned this as an important factor for completing respectively dropping out of a MOOC. A majority of the interviewees (and a majority of all learners in the two Chalmers MOOCs) were either university students or researchers within the content domain of the MOOCs. In the second category, enjoyment, the interviewees commented on how fun and interesting the subject of the MOOCs were: I decided to finish it anyway because it was really nice, fun and interesting.
Fourteen out of 34 interviewees made similar remarks, suggesting that enjoyment plays an important role in relation to completion rates.

Perceived level of difficulty of the content
Another quite frequent comment regarding content was the perceived level of difficulty. Out of the 34 interviewees, three learners stated that the MOOC was too difficult, while five stated the opposite, that the MOOC was very easy and this helped them to complete the MOOC. I find it very tough and difficult because in the first lessons it was all about physics and stuff. I thought it would be in some other way.
I understand the technical part because I am an engineer. It was easy for me to understand, and to evaluate if I had understood the information right. That helped me to come back.
This suggests that the level of difficulty is not a problem for the majority of MOOC learners, but for the ones that have issues with the level of difficulty it has severe consequences, making them drop out. This, of course, relates to one of the key characteristics of MOOCs: openness. If MOOCs are meant to be open for everyone, then the level of difficulty cannot be too high, neither too basic. We will return to this issue in the discussion and conclusion.

The learner's perception of the course design
There were very few comments concerning the actual design and implementation of the two Chalmers MOOCs, even to the extent of suggesting that the course design did not have any impact on dropout rates. Six out of the 34 interviewees actually claimed that completion or dropout are not highly related to what the course designers or teachers do.
It's a personal nature, it cannot be solved by the design of the MOOC.
That is not something you can do anything about.
However, as we will see in the fourth theme, the learner's ability to find and manage time effectively, there is a complex interplay between time factors inside and outside the course designers' sphere of control, and some interviewees mentioned design choices that can affect dropout rates.

Discouragement based on faulty assignments
The one distinct case of design error related to dropout is the first week assignment in ChM002x. Five out of the 16 interviewees attending this MOOC commented on the problem with this assignment. A few of these learners simply gave up. One got behind, never recovered and finally gave up. However, one of the interviewees mentioned that the weekly update video was enough to compensate for the discouragement caused by the faulty assignment: When I saw the weekly update video about the fixes I thought it was no problem. Eventually, this learner completed the MOOC, which provides an interesting case illustrating that weekly update videos can have a positive effect on completion. In this particular weekly update ''Time is the bottleneck'': a qualitative study exploring… 139 video, the teachers apologized for the problems with the assignment and promised that this and upcoming assignments would be corrected. Another case worth mentioning is the learner that started ChM002x a couple of weeks late, when the assignment was already corrected. She said: I was a lucky one as I wasn't involved in the first round of issues with the questions. I didn't get that frustration. Both of these cases suggest that quick actions to correct errors are especially important during the first week of the course.

The learner's social situation and characteristics
Even though MOOC participation is a global phenomenon, and learners come from all over the world, a majority of the learners live in North America or Europe (Veletsianos and Shepherdson 2016). This was the case for the two Chalmers MOOCs as well, though a significant number of participants also came from India. Liyanagunawardena et al. (2013) suggest that the reason for the North American and European dominance could be technological and linguistic, and this assumption corresponds well with the results we discuss below; that difficulties with Internet access and understanding English play a role in explaining dropout rates. Even though different, both a technological problem, such as Internet access, and a literacy issue, such as English proficiency, can be sorted under the umbrella of the learner's socio-economic situation. Obviously, important mechanisms behind completion and dropouts should relate to the learner herself, and the interviews provided extensive insight into these mechanisms.

External factors
An external factor in this context refers to a factor outside the control of the course designers, the teachers and the learners. A few comments were made by the interviewees concerning such external factors.
I was visiting my family and missed the first week of a MOOC, then it was difficult to finish on time.
I didn't finish one MOOC since I was sick during the exam.
These quotes succinctly illustrate how the ambition to learn clashed with other aspects of life. Even if it is not surprising, it is important to be aware of this issue. It is also worth noting that even though none of these cases could be under the control of the course designer, they all exemplify that external factors cause lack of time, which then causes dropout. Thus, the question arises, could flexibility concerning time have helped these learners to rebound and finally complete the MOOCs? After all, if the learner was sick during the exam, why could he or she not be allowed to do the exam at a later stage? The time issue will be discussed further down, but we note that more flexibility regarding time constraints could help learners to overcome external factors.

Socio-economic factors
Many of the interviewees mentioned a general problem with understanding the English spoken by the teachers in the recorded lectures. Moreover, the challenges in conducting the interviews provided us with ample indications of the low English proficiency of some of the learners. Many mentioned the video subtitles as being a great help to understand the English language in the recorded lectures. Some also mentioned that they occasionally paused the video and searched online for the meaning of specific words. One learner explicitly stated that sometimes the teacher accent is a reason for not finalizing, it is hard to understand some people's English.
Three learners mentioned another specific socio-economic factor: problems with Internet access. These learners, particularly from India, Mexico, and Venezuela, described how power cut outs or issues with a very low bandwidth caused them quite some concern during their MOOC work. It was difficult to view recorded lectures at convenient times and assignment deadlines were missed. This once again suggests that a more flexible scheduling could have helped the learners that missed an assignment deadline, and illustrates the problems that arise when the target audience for a MOOC is truly global. In a country such as Sweden, where reliable and high-speed Internet connection is almost ubiquitous, it is easy to take for granted that watching recorded lectures is not a technical issue. However, as the interviewees emphasized, residents in other countries can have substantial difficulties in this regard.

Learner personality traits
Five of the 34 interviewed learners described a particular personality trait in themselves, which they argue is part of the explanation for why they complete MOOCs, despite having issues with lack of time as so many others claimed. One of these interviewees said: I am sort of addicted to these courses, I always take more than I can handle. … Once I get started I want to get done, I want to know, as long as it is interesting. … Stopping after 50%, it was a shame to not finalize it.
The reasoning conveyed by this handful of learners is similar to what media consumers say about serialized and long-format narratives such as TV series and novels. Another possible similarity concerns collectors of different sorts; the proverbial stamp collector seems to share this attitude towards what she does. The process of performing the ritual-collecting, watching, reading, learning-overshadows the subject matter, so that this particular personality trait provides a motivation besides those mentioned before, such as utilitarian and enjoyment.
''Time is the bottleneck'': a qualitative study exploring… 141 The learner's ability to find and manage time effectively

Lack of time
When discussing MOOC completion and dropout with the interviewed learners, one factor dominated over all the others-lack of time: Time is the bottleneck. If I had more time, I would do more courses.
Lack of time was mentioned by 21 of the 34 interviewees. Family life, work and other studies were aspects of life that severely competed with the time spent on learning in MOOCs. It can be argued that lack of time is outside the sphere of control for course designers; after all, if the learner does not have time for, or does not prioritize, the MOOC, then why bother? However, why is lack of time causing massive drop outs in MOOCs but not in conventional university education? And can the course designers in some way leverage the difficulties the learners have?
The answer to the first question is not really investigated in our interviews; during our discussions with the learners we shared the assumption that MOOC learning obviously has to stand back when clashing with family life, professional work or conventional university studies. If we scrutinize this assumption, we can draw one tentative interpretation; so far, MOOC learning has the status of pastime, comparable to other hobbies or entertainment such as watching movies or playing games. Many learners motivate their MOOC learning with the utilitarian argument; that it is a help for future work or studies (and then work). However, when clashing with actual work or conventional university studies, MOOC learning is not prioritized. The reason seems obvious: attending MOOCs is so far no clear guarantee of having a specific benefit for career or employment.

Study techniques
Finding time is also a matter of managing time in an efficient way, and this is related to study techniques, which were mentioned a few times. Concerning study techniques, two learners made the following comments.
It was difficult for me to continue the course, because here in Mexico we are not accustomed take Internet courses, and to organizing our time in different time zones. It is really difficult when you have homework deadlines.
For students it's more a matter of how disciplined they are. … I would say time restrictions and time management.
The first quote is no surprise since the difficulties that many students have with managing their time in distance education in general is well documented (e.g. Levy 2007). It is interesting to note though that the learner here specifically referred to time zone difficulties. These are two distinct time issues; on one hand there is the problem with managing time when a course is more flexible than a conventional campus course; on the other hand there is the problem of synching different time zones. An obvious solution regarding the latter would be to include a feature in the MOOC platform (in our case edX) that both shows the current clock time corresponding to the time zone that edX is using, and a countdown for assignment deadlines. Interestingly, the second quote came from an assistant professor, not actually referring to himself but what he believed were the time management issues that other MOOC learners had, in particular those that are university students and not staff.
These two comments do nothing more than to suggest that study techniques play a role in dropout, an assumption that seems likely, but remains to be investigated in more depth.

Discussion and conclusions
We have used a qualitative case study approach to identify four overarching themes or factors that influenced the learners' decisions to drop out of MOOCs (see Table 3 for a summary of our main findings). Even though there is surprisingly little qualitative research on factors that influence retention in MOOCs (Liyanagunawardena et al. 2014;Veletsianos and Shepherdson 2016;Zheng et al. 2015) there are a few qualitative studies that we can compare our study with. The work by Zheng et al. (2015) is particularly interesting in this regard, since they used a similar research approach. Based on in-depth interviews with 18 learners from a range of MOOCs, they identified eight factors affecting retention in MOOCs: The first three factors are significant in our study as well, in particular factor 3, ''lack of time''. This factor has also been highlighted by Belanger and Thornton (2013). The fourth factor-''lack of pressure''-was not explicitly mentioned by our interviewees, but it is implicitly present in the arguments concerning external motivation, such as career or studies. A few learners in our study mentioned the motivation that comes from interacting with others, which corresponds to factor 5. The importance of interaction between learners as well as between learners and teachers has been emphasized in other studies as well (Mak et al. 2010;Khalil and Ebner 2014). However, a vast majority of the interviewees (31 out of 34) mentioned that they had not made extensive use of the discussion forum, primarily because they did not see the incentive to do so. Moreover, several learners mentioned explicitly that when starting their work on the MOOC, they expected to do it alone. Factors 6 and 7 were not mentioned at all in our interviews. Interestingly, factor 8 was not mentioned either. All our interviewees discussed dropout based on the assumption that a MOOC is something that you complete. None argued that they left the course because they had learnt what they needed. In other words, the learners we interviewed considered a MOOC to be quite equivalent to a traditional course, and not a collection of content to be browsed and sampled on demand.
How the learners in our study conceptualized a MOOC had strong implications for how they engaged with the contents. Ebben and Murphy (2014) argued that since MOOCs are free and easy to sign up for, learners often start a MOOC without necessarily having the incentive to complete all parts. Zheng et al. (2015) as well as Belanger and Thornton (2013) identified a kind of learner that participates in MOOCs for enjoyment only. Therefore ''typically, participants who treat MOOCs as edutainment only watch the lecture videos without completing quizzes and assignments'' (Zheng et al. 2015(Zheng et al. , 1888. This corresponds to one of the categories of engagement identified by Kizilcec et al. (2013) which they call ''auditing''. None of these findings are, however, reflected in our study. Many of the learners we interviewed expressed a strong interest in-and even passion for-the contents of the two Chalmers MOOCs, and the tendency to skip quizzes and assignments because it was ''just for fun'' was not mentioned in our study.
Our findings raise several interesting questions that merit further study. Firstly, a dominant theme in the interviews was the learners' lack of time to engage with the MOOCs. To mitigate this issue, it is therefore worth exploring aspects such as the best time during the year to run a MOOC, the duration of a MOOC, scheduled vs self-paced MOOCs, and the duration of the recorded lectures (Bruff 2013). Secondly, one set of dropout factors identified in our study is directly related to one of the hallmarks of MOOCs: the openness. When a course is open for everyone, some learners will have problems with the content being too difficult, or too basic, and some will have problems with understanding English, while others will have problems with Internet connections. Thus, we can ask two rhetorical questions: Are MOOCs really open? To what extent should the producers of MOOCs adapt the course design to learners with different levels of pre-knowledge, learners with too low English fluency, and too unreliable Internet connections? Our qualitative interview study does not provide statistics on how common it is that dropouts are caused by these three factors. However, the number of interviewees that mentioned these aspects suggests that the effect is at least noticeable on dropout rates. Finally, one of the more ambiguous results from the interviews concerns the relation between course design and dropout rates. The issue with the faulty assignment indicates that design mistakes occur and that they will have a distinct effect on dropout rates. Exploring these issues in more detail can improve the learning experience for all MOOC learners, regardless of whether they intend to complete the whole MOOC or just parts of it.
Funding This study was funded by internal funding of the Chalmers University of Technology MOOC project.
Thommy Eriksson is a Ph.D. at the Department of Applied IT (Chalmers University of Technology) within the field of digital representation. In his research he studies digital media within the fields of interaction design, mixed reality development, learning management systems and using video in higher education.
Tom Adawi is Professor in Engineering Education Research at Chalmers and Chair of the Nordic Network of Engineering Education Research. His research interests include authentic learning environments, problem solving, threshold concepts, and technology-enhanced learning.
Christian Stöhr is Senior Lecturer at the Division of Engineering Education Research at the Department for Applied IT at Chalmers, where he combines educational research with the provision of pedagogical support to faculty. He has performed extensive research and capacity building within e-learning and blended learning in engineering education, but has also contributed to STS research in environmental governance.