Advertisement

Completion, Comments and Repurposing a Digital Accessibility MOOC

  • E. A. Draffan
  • Manuel Leon
  • Abi James
  • Saud Aljaloud
  • Mike Wald
Open Access
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10896)

Abstract

The ‘massive’ and ‘open’ nature of Massive Open Online Courses (MOOCs) can provide powerful dissemination tools to raise awareness of topics in need of public attention, such as digital accessibility and its impact on assistive technology users. The subject is a wide-ranging one when taught from the point of view of those who may have sensory, physical and/or cognitive impairments coping with a myriad of digital activities on a daily basis. These may range from creating and viewing documents, surfing the web to using a mobile or a washing machine. An analysis of the interactions and learning experiences of those who completed the FutureLearn Digital Accessibility MOOC was conducted, using a combination of statistical and qualitative methods. Preliminary results indicate a progressive loss of participants over time, which is to be expected with MOOCs. However, certain measures such as the number of comments per participant, completed steps, and the “likes” count suggest a relatively high degree of engagement from this particular learning community. After examining the topics that triggered most participation, a suggestion has been formulated to repurpose parts of the course in order to exploit its most engaging sections and offer alternative forms of support for those activities that required more explanation.

Keywords

MOOCs Digital accessibility Elearning Professionals Dissemination Repurposing 

1 Introduction

The Erasmus + European project “MOOCs for Accessibility Partnership” (MOOCAP) [1] set out to provide education on accessible design in Information, Communication and Technology (ICT) covering a wide range of topics that would be of interest to all those involved in the delivery and consumption of digital products, publications and services. Over the course of three years, the project team delivered a series of Massive Open Online Courses (MOOCs) and online courses, all available as Open Educational Resources (OERs) with a Creative Commons Attribution 4.0 International Licence. The introductory course aimed to help learners understand how “those with sensory, physical and cognitive impairments may be disabled by barriers encountered when using digital technologies.” [2] The MOOC did not require any previous knowledge about the topics covered, but set out a series of accessibility skills that could be acquired if a learner completed the course over five weeks. 9642 learners registered on the three runs that were held in 2016–2017 and of those who signed up 5,604 (58%) stayed to at least view one step at any time in any course week. According to FutureLearn [3] “the average number of learners on a course is 50% of joiners”.

Research has shown there is a wide variation in the reasons for dropping out of courses [4] due to internal and external factors [5, 6]. Examples include:
  • “No real intention to complete,

  • Lack of time,

  • Course difficulty and lack of support,

  • Lack of digital skills or learning skills,

  • Bad experiences,

  • Expectations,

  • Starting late and

  • Peer Review” [7].

When developing the Digital Accessibility MOOC, the project team was aware of the high dropout rates and wanted to ensure the course platform was accessible, easy to use and encouraged interaction with tutors and mentors. Ease of use checks with assistive technologies showed that FutureLearn fulfilled the accessibility criterion and offered captions and transcriptions for all the videos or audio files uploaded to the system. There was the added bonus of an embedded discussion forum so that learners could easily comment on the content. Researchers have suggested that FutureLearn is a social learning platform because comments appear alongside each step or activity, rather than requiring the learner to work in an unrelated discussion area on the website. The former tends to encourage increased interaction between learners and this was seen as being important, where those participating on each course run outnumbered the teachers and mentors [8].

Those creating the MOOC (eight university departments in seven different countries) were aware that their contributions varied in content style and topic from discussing different types of disability with personas to complex accessibility standards related to web page design and development. The authors of this paper, coming from a partner university, were interested to see how many learners completed the various activities presented on the Digital Accessibility MOOC, as well as how many actively engaged with content and their peers. By analyzing the comments from the various steps and activities alongside completion levels it was felt there might be the possibility of seeing which topics encouraged engagement and which should be the focus of future courses and in particular suitable for repurposing into an alternative format and offered as additional Open Educational Resources.

2 Method

The initial task was to discover how many learners completed each step. Questioning whether there were any anomalies or was there a general level of dropout over the course of the five weeks? A step (as seen in a FutureLearn MOOC) is made up of a single topic page that may contain a short video or audio clip with text, possibly images and usually a question to engender discussion. Steps are grouped as activities, which are rather like a section in a chapter of a book. The chapter becomes the week of activities in the online course. There followed an analysis of the number of comments in each step compared to the number of learners. The learner’s unique ID was used due to data protection and where steps required no commenting these were omitted. It was felt this comparison might illustrate different levels of engagement to the types of content or topic. Any replies to comments were omitted from the analysis as these tended to show appreciation or question the previous comment rather than the content within the course. The results were aligned with the number of ‘likes’ on each original comment about the content to illustrate more active engagement with the topic being discussed.

Finally, an analysis of the comments found in the reflection steps, offered at the end of each week, was undertaken to find any discussion that might be related to particular topics. It was felt that this task would not only offer an insight into the learning outcomes, but also whether adaptations should be made to the way a topic was presented.

3 Results

Of the 5,604 joiners on the three runs of the Digital Accessibility MOOC only 4185 were active learners. According to FutureLearn these learners are “those (of any role) who have completed at least one step at any time in any course week, including those who go on to leave the course.” Recording the completion of a step is dependent on the learner manually selecting the ‘mark as complete’ button. When analysing the 98 steps over the five weeks it was found that the rate of completion dropped from 2465 learners per step to 839 learners. This was not unexpected as stated at the beginning of the paper and it was found that the rate of attrition was relatively uniform after the first week (see Fig. 1).
Fig. 1.

The average number of participants who completed a step per week

Moreover, despite encouragement from mentors on the Digital Accessibility MOOC to engage with the questions posed at the end of many steps or to be reflective at the end of each week and to discuss topics independently of the teaching staff, it was found that the majority of learners still failed to regularly comment. Swinneton et al. [8] discovered in their research, on average 47% of those using the discussion area only make one or two comments. On the Digital Accessibility MOOC the average number of comments per learner was 6–7 throughout the three runs.

Further inspection of the data showed that the average percentage of learners who completed an activity and commented on the topics in the reflection step provided each week, varied very little: 21–27% over the five weeks. However, when linking this to the percentage of ‘likes’ for particular comments it became clear that there was a distinct difference marking out week 4. It had the second lowest number of comments relating to the reflection step (214) with the second highest number of likes for the comments about the content (415) (Table 1).
Table 1.

Number of comments and likes compared to the number of learners per week

Reflective Step

Number of learners who completed the step

Number of comments made

Number of comment “likes”

Week 1

1699

467

489

Week 2

1308

332

352

Week 3

1085

223

325

Week 4

937

214

415

Week 5

829

199

276

On inspection of week four’s content, spread over 20 steps, there were fewer questions compared to the other weeks. Only nine steps had questions, and there were three quizzes to break up the amount of content and allow learners time to self-assess their knowledge building. Questions in the activity steps asked for answers that related directly to a web accessibility activity such as the one asked after Step 4.7 “What barriers can a web form have for people with limited dexterity?” The reflection steps at the end of each week always had the same three questions namely:
  1. 1.

    What have you found to be good, useful or interesting this week?

     
  2. 2.

    What actions will you now take to learn more about these topics and take the lessons forward in your life and work?

     
  3. 3.

    What questions, if any, have arisen for you?

     
So the question asked, was why this particular reflection step caused an increased amount of engagement? There were nearly twice the number of likes per comment, compared to other reflection steps (34% to 66%). It was decided that a thematic analysis of the comments might show what had made this particular reflection step different from the others. An initial familiarisation with the content of the 214 comments showed that learners had sometimes answered all three of the questions in one comment. Data within the comments was coded and a thematic analysis undertaken [9]. Four themes appeared, an appreciation of the week’s content in terms of helpfulness and raising awareness, mentions of complexity, the need for further materials and the wish to revisit materials with other comments relating to particular web accessibility questions. Table 2 shows the worked-out themes from the comments (with percentages) found in the week four reflection step.
Table 2.

Themes for classifying comments provided in the week four reflection step.

Theme

Comments

Percentage

Appreciate materials

118

45%

Complexity

23

9%

Further materials required

12

5%

Materials need revisiting

32

12%

Other themes related to activities

75

29%

Total

260

100%

4 Discussion

By examining the content of activities for week four, it became clear that this was not only a very full week, but also the activities might be considered quite complex, because it was about web content accessibility standards and the barriers to ease of access for those with disabilities. It included Web Content Accessibility Guidelines (WCAG), conformance and user testing often undertaken by developers or those involved with online content development. However, when posing questions for reflection, with guidance to help learners think about general aspects of the week’s activities, the most popular answers showed appreciation to those who provided the content. This gratitude was welcoming, but made the authors of this paper wonder whether the actual reflection questions needed to change in order to provide a better understanding of the learner outcomes.

However, research has shown that when learners participate actively with comments such as replying or liking, it shows more active engagement with the course [10] which could be said to be true of week four as there were comments that revealed a deep interest in the subject. Those who managed the activities often said how much they appreciated the amount of information and external links. The minority who mentioned the complexity of it all (9%) or were new to the subject often praised the way it had been broken down into steps, but still wanted an equivalent of a ‘Dummies guide’ with key points and less technical language.

There were other comments about the amount of links and resources provided and how they would need to be revisited or shared with colleagues at a later date. These comments highlighted the fact that time may have been an issue, with a sense of overload in terms of the amount of reading material provided. The idea of offering slides with key points and even smaller mini chunks of content with diagrams for the crucial elements might have helped learners cope better in this type of situation. Videos and audio recordings were purposely kept below three minutes and in the reflections, no one said they were too long. Transcripts and captions were offered and the former could be downloaded and kept for reference.

There was an awareness that the materials would only be available for 14 days after the course unless further access was purchased. On the other hand, it was pointed out that, for this particular course, all the materials had been made available and could be repurposed in different formats, once gathered from the MOOCAP project website.

When examining the comments regarding particular activities (29%) it was clear that practical advice, such as the use of web accessibility testing tools, was also highly valued. Several learners on the course shared their expertise in their comments. One of the limitations of a MOOC is the difficulty of providing hands on experience from a distance. The repurposing of this aspect of the course with more interactive online activities, illustrating how the tools worked, where they were useful and where they failed to pick up accessibility errors, would have provided improved support. This could have helped those learners who were educationalists rather than developers.

Finally, the problem of having to leave the MOOC platform to read or interact with other websites could be considered a weakness of all online learning where there are hyperlinks out to similar areas of interest. Not knowing the skills and interests of the learners who join a MOOC can make it difficult for content providers. A judgement has to be made as to the level and complexity of knowledge provided and in the case of the Digital Accessibility MOOC there was a tacit understanding that the content would be offered at several levels with the external links providing basic and/or advanced support materials. All the resources had clear explanations, but these could have been presented with indications as to the level of knowledge required, perhaps on a sliding scale from basic to advanced or college level to post graduate.

5 Conclusion

Many researchers have commented on the way learners fail to complete MOOCs based on course completion figures, the gaining of a certificate or the passing of tests. It could be said that this way of measuring the success of a MOOC fails to address the diversity of those participating and what they may wish to achieve. The Digital Accessibility MOOC had two main groups of participants namely educators and those in Information Technology professions and when reviewing the comments in the week four reflection step, it was clear that members of both groups had engaged with the content. However, the reasons for engagement included the wish to pass information on to others and use of the knowledge gained in daily work, as well as showing a genuine desire to participate with particular topics despite their complexity. Some learners were willing to share their knowledge when others included questions in amongst their comments and topic specific threads developed. This process of discussion, commenting and liking comments has resulted in researchers highlighting the potential for social learning via MOOC platforms. However, as Tubman et al. [11] point out “MOOC environments have unique challenges for pedagogy which are not present in other socioconstructivist learning environments: the scale and diversity of participation”. Nevertheless, these peer on peer interactions can be very valuable as a way of raising awareness about learners’ positive and negative feelings and when problems are arising.

Even with a very small number of comments from one reflective step taken from one week in the Digital Accessibility MOOC it was possible to prioritise the type of activity that should be repurposed into alternative formats in order to better support some learners. For example, the highlighting of crucial elements of the content into a series of online interactive slides, such as offered by SlideWiki [12], that can be made available as open education resources (using the Creative Commons licence CC-BY 4.0), would offer support to those wishing to share the information as well as collaborate on the topic. The addition of metadata to provide information to learners about the level of each activity may also be useful to prevent concerns about complexity, but a true appreciation on the part of authors as to the time it takes to absorb materials is hard, due to the diversity of those participating as mentioned before.

Most MOOCs offer content in a variety of layouts, with use of text and multimedia, as a whole course and rarely offer their content in ways that can be easily repurposed. As part of the development of the content for the Digital Accessibility MOOC as well as to facilitate the collaboration of experts within different countries, institutions and fields, a “stepwise” process for authoring was organized. Each learning object was carefully documented, with accessibility metadata and grouped in small chunks of information making up the various activities [13]. These individual artefacts such as single images, audio files to videos with their transcripts and caption files plus text based content in separate word processed documents can be adapted to suit all levels of education. The concept of offering multiple ways of learning has been recognised as an effective way of encouraging inclusive teaching and learning practices with “multi-modal opportunities in terms of representation, engagement and expression” [14].

References

  1. 1.
    MOOCAP Erasmus + Project Home page http://gpii.eu/moocap/. Accessed 31 Jan 2018
  2. 2.
    FutureLearn ‘Why Join the Course’. https://www.futurelearn.com/courses/digital-accessibility. Accessed 31 Jan 2018
  3. 3.
    FutureLearn Course Run Measures. https://partners.futurelearn.com/data/stats-dashboard/. Accessed 31 Jan 2018
  4. 4.
    Breslow, L.B., Pritchard, D.E., DeBoer, J., Stump, G.S., Ho, A.D., Seaton, D.T.: Studying learning in the worldwide classroom: research into edX’s first MOOC. Res. Pract. Assess. 8, 13–25 (2013)Google Scholar
  5. 5.
    Halawa, S., Greene, D., Mitchell, J.: Dropout prediction in MOOCs using learner activity features. Experiences Best Pract. Around MOOCs 7, 3–12 (2014)Google Scholar
  6. 6.
    Lu, X., Wang, S., Huang, J., Chen, W., Yan, Z.: What decides the dropout in MOOCs? In: Bao, Z., Trajcevski, G., Chang, L., Hua, W. (eds.) DASFAA 2017. LNCS, vol. 10179, pp. 316–327. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-55705-2_25CrossRefGoogle Scholar
  7. 7.
    Onah, D.F., Sinclair, J., Boyatt, R.: Dropout rates of massive open online courses: behavioural patterns. In: Proceedings of the EDULEARN14, pp. 5825–5834 (2014)Google Scholar
  8. 8.
    Swinnerton, B., Hotchkiss, S., Morris, N.P.: Comments in MOOCs: who is doing the talking and does it help? J. Comput. Assist. Learn. 33(1), 51–64 (2017)CrossRefGoogle Scholar
  9. 9.
    Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006)CrossRefGoogle Scholar
  10. 10.
    Ferguson, R., Clow, D.: Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs). In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, pp. 51–58. ACM, March 2015Google Scholar
  11. 11.
    Tubman, P., Oztok, M., Benachour, P.: Being social or social learning: a sociocultural analysis of the FutureLearn MOOC platform. In: 2016 IEEE 16th International Conference on Advanced Learning Technologies (ICALT), pp. 1–2. IEEE (2016)Google Scholar
  12. 12.
    SlideWiki EU Open Courseware Project. https://slidewiki.org/. Accessed 31 Jan 2018
  13. 13.
    Draffan, E.A., Wald, M., Dickens, K., Zimmermann, G., Kelle, S., Miesenberger, K., Petz, A.: Stepwise approach to accessible MOOC development. Studies in health technology and informatics 217, 227 (2015)Google Scholar
  14. 14.
    Department of Education, Inclusive Teaching and Learning in Higher Education as a route to Excellence (2017). https://www.gov.uk/government/publications/inclusive-teaching-and-learning-in-higher-education. Accessed 31 Jan 2018

Copyright information

© The Author(s) 2018

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  • E. A. Draffan
    • 1
  • Manuel Leon
    • 1
  • Abi James
    • 1
  • Saud Aljaloud
    • 1
  • Mike Wald
    • 1
  1. 1.WAIS, ECSUniversity of SouthamptonSouthamptonUK

Personalised recommendations