Design review of MOOCs: application of e-learning design principles

Abstract

The purpose of this study is to explore the pedagogical design of massive open online courses (MOOCs) using evidence-based e-learning principles. MOOCs have become an important part of discourse in higher education. However, there has been shared concern on the quality of MOOCs as learning systems for engaging learners as well as fulfilling their needs. The researchers conducted a design review of 40 computer science MOOCs from two major MOOC providers. The findings indicate a relatively low application of the principles in general, with the exception of those related to the organization and presentation of content. MOOC platforms and the difficulty level of MOOCs used the application of e-learning principles and guidelines differently. Implications for future research and design of MOOCs are discussed.

This is a preview of subscription content, log in to check access.

References

  1. Altman, D. G. (1991). Practical statistics for medical research. London: Chapman and Hall.

    Google Scholar 

  2. Barak, M., Watted, A., & Haick, H. (2016). Motivation to learn in massive open online courses: Examining aspects of language and social engagement. Computers & Education, 94, 49–60. https://doi.org/10.1016/j.compedu.2015.11.010.

    Article  Google Scholar 

  3. Beaven, T., Hauck, M., Comas-Quinn, A., Lewis, T., & de los Arcos, B. (2014). MOOCs: Striking the right balance between facilitation and self-determination. Journal of Online Learning and Teaching, 10(1), 31–43.

    Google Scholar 

  4. Burd, E. L., Smith, S. P., & Reisman, S. (2015). Exploring business models for MOOCs in Higher Education. Innovative Higher Education, 40, 37–49.

    Article  Google Scholar 

  5. Castaño-Muñoz, J., Kreijns, K., Kalz, M., & Punie, Y. (2017). Does digital competence and occupational setting influence MOOC participation? Evidence from a cross-course survey. Journal of Computing in Higher Education, 29, 28–46.

    Article  Google Scholar 

  6. Chen, C.-M., & Wu, C.-H. (2015). Effects of different video lecture types on sustained attention, emotion, cognitive load, and learning performance. Computers & Education, 80, 108–121. https://doi.org/10.1016/j.compedu.2014.08.015.

    Article  Google Scholar 

  7. Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 3, 7.

    Google Scholar 

  8. Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., & Emanuel, E. (2013). The MOOC phenomenon: Who takes massive open online courses and why? Retrieved January 7, 2018 from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2350964.

  9. Chu, H.-C., & Hwang, G.-J. (2010). Development of a project-based cooperative learning environment for computer programming courses. International Journal of Innovation and Learning, 8(3), 256–266.

    Article  Google Scholar 

  10. Chukwuemeka, E. J., Yoila, A. O., & Iscioglu, E. (2015). Instructional design quality: An evaluation of Open Education Europa Networks’ open courses using the first principles of instruction. International Journal of Science and Research, 4(11), 878–884.

    Google Scholar 

  11. Clark, R. C., & Mayer, R. E. (2011). E-Learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. San Francisco, CA: John Wiley & Sons.

    Google Scholar 

  12. Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–222.

    Article  Google Scholar 

  13. Creelman, A., Ehlers, U., & Ossiannilsson, E. (2014). Perspectives on MOOC quality: An account of the EFQUEL MOOC quality project. The International Journal for Innovation and Quality in Learning, 2(3), 78–87.

    Google Scholar 

  14. Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied Nursing Research, 5, 194–197.

    Article  Google Scholar 

  15. Dillahunt, T., Wang, Z., & Teasley, S. D. (2014). Democratizing higher education: Exploring MOOC use among those who cannot afford formal education. The International Review of Research in Open and Distance Learning. https://doi.org/10.19173/irrodl.v15i5.1841.

    Article  Google Scholar 

  16. Egloffstein, M., & Ifenthaler, D. (2017). Employee perspectives on MOOCs for workplace learning. TechTrends, 61(1), 65–67.

    Article  Google Scholar 

  17. Gonzalez, G. (2006). A systematic approach to active and cooperative learning in CS1 and its effects on CS2. In Proceedings of the 37th SIGCSE Technical Symposium on Computer Science Education (SIGCSE’06) (pp. 133–137). New York, NY: ACM Press.

  18. Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43.

    Article  Google Scholar 

  19. Gruba, P., & Sondergaard, H. (2001). A constructivist approach to communication skills instruction in computer science. Computer Science Education, 11(3), 203–219. https://doi.org/10.1076/csed.11.3.203.3833.

    Article  Google Scholar 

  20. Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the First ACM Conference on Learning @ Scale Conference (pp. 41–50). New York, NY: ACM Press.

  21. Hathaway, K. L. (2013). An application of the seven principles of good practice to online courses. Research in Higher Education Journal, 22, 1.

    Google Scholar 

  22. Hew, K. F. (2018). Unpacking the strategies of ten highly rated MOOCs: Implications for engaging students in large online courses. Teachers College Record, 120, 1–40.

    Google Scholar 

  23. Howarth, J., D’Alessandro, S., Johnson, L., & White, L. (2017). MOOCs to university: A consumer goal and marketing perspective. Journal of Marketing for Higher Education, 27(1), 144–158.

    Article  Google Scholar 

  24. Jansen, D., Rosewell, J., & Kear, K. (2017). Quality frameworks for MOOCs. In M. J. Kinshuk & M. K. Khribi (Eds.), Open education: From OERs to MOOCs (pp. 261–281). Berlin: Springer.

    Google Scholar 

  25. Konrad, A. (2017). Coursera fights to keep the promise of MOOCs alive with corporate customer push. Forbes. Retrieved January 7, 2018 from https://www.forbes.com/sites/alexkonrad/2017/12/20/coursera-goes-corporate-to-keep-alive-promise-of-moocs/#40181021543c.

  26. Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28, 563–575.

    Article  Google Scholar 

  27. Linn, M., & Dalbey, J. (1989). Cognitive consequences of programming instruction [reprinted]. In E. Soloway & J. C. Spohrer (Eds.), Studying the novice programmer (pp. 57–81). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  28. Littlejohn, A., Hooda, N., Milligan, C., & Mustain, P. (2016). Learning in MOOCs: Motivations and self-regulated learning in MOOCs. Internet and Higher Education, 29, 40–48.

    Article  Google Scholar 

  29. Lowenthal, P., & Hodges, C. (2015). In search of quality: Using quality matters to analyze the quality of massive, open, online courses (MOOCs). The International Review of Research in Open and Distributed Learning, 16(5), 83–101. https://doi.org/10.19173/irrodl.v16i5.2348.

    Article  Google Scholar 

  30. Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of Massive Open Online Courses (MOOCs). Computers & Education, 80, 77–83.

    Article  Google Scholar 

  31. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York, NY: Cambridge University Press.

    Google Scholar 

  32. Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43–59.

    Article  Google Scholar 

  33. Milligan, C., & Littlejohn, A. (2014). Supporting professional learning in a Massive Open Online Course. The International Review of Research in Open and Distance Learning. https://doi.org/10.19173/irrodl.v15i5.1855.

    Article  Google Scholar 

  34. National Academies of Sciences, Engineering, and Medicine. (2018). Assessing and responding to the growth of computer science undergraduate enrollments. Washington, D.C.: National Academies Press.

    Google Scholar 

  35. Ossiannilsson, E., Williams, K., Camilleri, A. F., & Brown, M. (2015). Quality models in online and open education around the globe: State of the art and recommendations. Oslo: International Council for Open and Distance Education.

    Google Scholar 

  36. Quality Matters. (2014). Introduction to the quality matters program. Retrieved January 10, 2018 from https://www.qualitymatters.org/sites/default/files/Introduction%20to%20the%20Quality%20Matters%20Program%20HyperlinkedFinal2014.pdf.

  37. Robins, A., Rountree, J., & Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer Science Education, 13(2), 137–172.

    Article  Google Scholar 

  38. Rohs, M., & Ganz, M. (2015). MOOCs and the claim of education for all: A disillusion by empirical data. International Review of Research in Open and Distributed Learning., 16(6), 5. https://doi.org/10.19173/irrodl.v16i6.2033.

    Article  Google Scholar 

  39. Rosewell, J., & Jansen, D. (2014). The OpenupEd quality label: Benchmarks for MOOCs. INNOQUAL: The International Journal for Innovation and Quality in Learning, 2(3), 88–100.

    Google Scholar 

  40. Sanchez-Gordon, S., & Luján-Mora, S. (2018). Technological innovations in large-scale teaching: Five roots of massive open online courses. Journal of Educational Computing Research, 56(5), 623–644. https://doi.org/10.1177/0735633117727597.

    Article  Google Scholar 

  41. Scagnoli, N. I., Choo, J., & Tian, J. (2019). Students’ insights on the use of video lectures in online classes. British Journal of Educational Technology, 50(1), 399–414. https://doi.org/10.1111/bjet.12572.

    Article  Google Scholar 

  42. Shah, D. (2017). A product at every price: A review of MOOC stats and trends in 2017. Class Central. Retrieved January 7, 2018 from https://www.class-central.com/report/moocs-stats-and-trends-2017/.

  43. Spector, J. M. (2014). Remarks on MOOCs and Mini-MOOCs. Educational Technology and Research Development, 62, 385–392.

    Article  Google Scholar 

  44. Stracke, C. M. (2019). Quality frameworks and learning design for open education. International Review of Research in Open and Distributed Learning, 20(2), 180–203.

    Article  Google Scholar 

  45. Tawfik, A. A., Reeves, T. D., Stich, A. E., Gill, A., Hong, C., McDade, J., et al. (2017). The nature and level of learner-learner interaction in a chemistry massive open online course (MOOC). Journal of Computing in Higher Education, 29(3), 411–431. https://doi.org/10.1007/s12528-017-9135-3.

    Article  Google Scholar 

  46. Tirrell, T., & Quick, D. (2012). Chickering’s seven principles of good practice: Student attrition in community college online courses. Community College Journal of Research and Practice, 36(8), 580–590.

    Article  Google Scholar 

  47. Toven-Lindsey, B., Rhoads, R. A., & Lozano, J. B. (2015). Virtually unlimited classrooms: Pedagogical practices in massive open online courses. Internet and Higher Education, 24, 1–12.

    Article  Google Scholar 

  48. Veletsianos, G., & Shepherdson, P. (2016). A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. The International Review of Research in Open and Distributed Learning. https://doi.org/10.19173/irrodl.v17i2.2448.

    Article  Google Scholar 

  49. Waltz, C. F., Strickland, O., & Lenz, E. R. (2010). Measurement in nursing and health research (5th ed.). New York, NY: Springer.

    Google Scholar 

  50. Waters, J. (2015). How nanodegrees are disrupting higher education, campus technology. Retrieved January 11, 2018 from http://campustechnology.com/articles/2015/08/05/how-nanodegrees-are-disrupting-higher-education.aspx.

  51. Watson, S. L., Loizzo, J., Watson, W. R., Mueller, C., Lim, J., & Ertmer, P. A. (2016). Instructional design, facilitation, and perceived learning outcomes: An exploratory case study of a human trafficking MOOC for attitudinal change. Educational Technology Research and Development, 64, 1273–1300.

    Article  Google Scholar 

  52. Watson, W. R., Watson, S. L., & Janakiraman, S. (2017). Instructional quality of massive open online courses: A review of attitudinal change MOOCs. International Journal of Learning Technology, 12(3), 219–224.

    Article  Google Scholar 

  53. Wexler, E. (2015). Moocs are still rising at least in numbers. The Chronicle of Higher Education. Retrieved January 7, 2018 from http://www.chronicle.com/blogs/wiredcampus/moocs-are-still-rising-at-least-in-numbers/57527.

  54. Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Addison-Wesley Professional.

    Google Scholar 

  55. Wing, J. (2011). Research notebook: Computational thinking—What and why? The Link Magazine, 6. Retrieved January 11, 2018 from https://www.cs.cmu.edu/link/research-notebook-computational-thinking-what-and-why.

  56. Yang, M., Shao, Z., Liu, Q., & Liu, C. (2017). Understanding the quality factors that influence the continuance intention of students toward participation in MOOCs. Educational Technology Research and Development, 65, 1195–1214.

    Article  Google Scholar 

  57. Yilmaz, A. B., Ünal, M., & Çakir, H. (2017). Evaluating MOOCs according to instructional design principles. Journal of Learning and Teaching in Digital Age, 2(2), 26–35.

    Google Scholar 

  58. Yoila, A. O., & Chukwuemeka, E. J. (2015). Instructional design quality evaluation of eastern mediterranean university open courses. International Journal of Scientific Research in Science, Engineering and Technology, 1(6), 1–7.

    Google Scholar 

  59. Zarb, M., & Hughes, J. (2015). Breaking the communication barrier: Guidelines to aid communication within pair programming. Computer Science Education, 25, 120–151. https://doi.org/10.1080/08993408.2015.1033125.

    Article  Google Scholar 

  60. Zhang, Q., Peck, K. L., Hristova, A., Jablokow, K. W., Hoffman, V., Park, E., et al. (2016). Exploring the communication preferences of MOOC learners and the value of preference-based groups: Is grouping enough? Educational Technology Research and Development, 64, 809–837. https://doi.org/10.1007/s12528-017-9135-3.

    Article  Google Scholar 

  61. Zhenghao, C., Alcorn, B., Christensen, G., Eriksson, N., Koller, D., & Emanuel, E. J. (2015). Who’s benefiting from MOOCs, and why. Harvard Business Review. Retrieved January 8, 2018 from https://hbr.org/2015/09/whos-benefiting-from-moocs-and-why.

  62. Zhu, M., Bonk, C. J., & Sari, A. R. (2018a). Instructor experiences designing MOOCs in higher education: Pedagogical, resource, and logistical considerations and challenges. Online Learning, 22(4), 203–241. https://doi.org/10.24059/olj.v22i4.1495.

    Article  Google Scholar 

  63. Zhu, M., Sari, A., & Lee, M. M. (2018b). A systematic review of research methods and topics of the empirical MOOC literature (2014-2016). The Internet and Higher Education, 37, 31–39.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Eunjung Grace Oh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Oh, E.G., Chang, Y. & Park, S.W. Design review of MOOCs: application of e-learning design principles. J Comput High Educ 32, 455–475 (2020). https://doi.org/10.1007/s12528-019-09243-w

Download citation

Keywords

  • MOOCs
  • E-learning design
  • Instructional quality
  • MOOCs design