Design review of MOOCs: application of e-learning design principles

  • Eunjung Grace OhEmail author
  • Yunjeong Chang
  • Seung Won Park


The purpose of this study is to explore the pedagogical design of massive open online courses (MOOCs) using evidence-based e-learning principles. MOOCs have become an important part of discourse in higher education. However, there has been shared concern on the quality of MOOCs as learning systems for engaging learners as well as fulfilling their needs. The researchers conducted a design review of 40 computer science MOOCs from two major MOOC providers. The findings indicate a relatively low application of the principles in general, with the exception of those related to the organization and presentation of content. MOOC platforms and the difficulty level of MOOCs used the application of e-learning principles and guidelines differently. Implications for future research and design of MOOCs are discussed.


MOOCs E-learning design Instructional quality MOOCs design 



  1. Altman, D. G. (1991). Practical statistics for medical research. London: Chapman and Hall.Google Scholar
  2. Barak, M., Watted, A., & Haick, H. (2016). Motivation to learn in massive open online courses: Examining aspects of language and social engagement. Computers & Education,94, 49–60. Scholar
  3. Beaven, T., Hauck, M., Comas-Quinn, A., Lewis, T., & de los Arcos, B. (2014). MOOCs: Striking the right balance between facilitation and self-determination. Journal of Online Learning and Teaching,10(1), 31–43.Google Scholar
  4. Burd, E. L., Smith, S. P., & Reisman, S. (2015). Exploring business models for MOOCs in Higher Education. Innovative Higher Education,40, 37–49.CrossRefGoogle Scholar
  5. Castaño-Muñoz, J., Kreijns, K., Kalz, M., & Punie, Y. (2017). Does digital competence and occupational setting influence MOOC participation? Evidence from a cross-course survey. Journal of Computing in Higher Education,29, 28–46.CrossRefGoogle Scholar
  6. Chen, C.-M., & Wu, C.-H. (2015). Effects of different video lecture types on sustained attention, emotion, cognitive load, and learning performance. Computers & Education,80, 108–121. Scholar
  7. Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin,3, 7.Google Scholar
  8. Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., & Emanuel, E. (2013). The MOOC phenomenon: Who takes massive open online courses and why? Retrieved January 7, 2018 from
  9. Chu, H.-C., & Hwang, G.-J. (2010). Development of a project-based cooperative learning environment for computer programming courses. International Journal of Innovation and Learning,8(3), 256–266.CrossRefGoogle Scholar
  10. Chukwuemeka, E. J., Yoila, A. O., & Iscioglu, E. (2015). Instructional design quality: An evaluation of Open Education Europa Networks’ open courses using the first principles of instruction. International Journal of Science and Research,4(11), 878–884.Google Scholar
  11. Clark, R. C., & Mayer, R. E. (2011). E-Learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. San Francisco, CA: John Wiley & Sons.CrossRefGoogle Scholar
  12. Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin,70(4), 213–222.CrossRefGoogle Scholar
  13. Creelman, A., Ehlers, U., & Ossiannilsson, E. (2014). Perspectives on MOOC quality: An account of the EFQUEL MOOC quality project. The International Journal for Innovation and Quality in Learning,2(3), 78–87.Google Scholar
  14. Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied Nursing Research,5, 194–197.CrossRefGoogle Scholar
  15. Dillahunt, T., Wang, Z., & Teasley, S. D. (2014). Democratizing higher education: Exploring MOOC use among those who cannot afford formal education. The International Review of Research in Open and Distance Learning. Scholar
  16. Egloffstein, M., & Ifenthaler, D. (2017). Employee perspectives on MOOCs for workplace learning. TechTrends,61(1), 65–67.CrossRefGoogle Scholar
  17. Gonzalez, G. (2006). A systematic approach to active and cooperative learning in CS1 and its effects on CS2. In Proceedings of the 37th SIGCSE Technical Symposium on Computer Science Education (SIGCSE’06) (pp. 133–137). New York, NY: ACM Press.CrossRefGoogle Scholar
  18. Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher,42(1), 38–43.CrossRefGoogle Scholar
  19. Gruba, P., & Sondergaard, H. (2001). A constructivist approach to communication skills instruction in computer science. Computer Science Education,11(3), 203–219. Scholar
  20. Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the First ACM Conference on Learning @ Scale Conference (pp. 41–50). New York, NY: ACM Press.Google Scholar
  21. Hathaway, K. L. (2013). An application of the seven principles of good practice to online courses. Research in Higher Education Journal,22, 1.Google Scholar
  22. Hew, K. F. (2018). Unpacking the strategies of ten highly rated MOOCs: Implications for engaging students in large online courses. Teachers College Record,120, 1–40.Google Scholar
  23. Howarth, J., D’Alessandro, S., Johnson, L., & White, L. (2017). MOOCs to university: A consumer goal and marketing perspective. Journal of Marketing for Higher Education,27(1), 144–158.CrossRefGoogle Scholar
  24. Jansen, D., Rosewell, J., & Kear, K. (2017). Quality frameworks for MOOCs. In M. J. Kinshuk & M. K. Khribi (Eds.), Open education: From OERs to MOOCs (pp. 261–281). Berlin: Springer.CrossRefGoogle Scholar
  25. Konrad, A. (2017). Coursera fights to keep the promise of MOOCs alive with corporate customer push. Forbes. Retrieved January 7, 2018 from
  26. Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology,28, 563–575.CrossRefGoogle Scholar
  27. Linn, M., & Dalbey, J. (1989). Cognitive consequences of programming instruction [reprinted]. In E. Soloway & J. C. Spohrer (Eds.), Studying the novice programmer (pp. 57–81). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  28. Littlejohn, A., Hooda, N., Milligan, C., & Mustain, P. (2016). Learning in MOOCs: Motivations and self-regulated learning in MOOCs. Internet and Higher Education,29, 40–48.CrossRefGoogle Scholar
  29. Lowenthal, P., & Hodges, C. (2015). In search of quality: Using quality matters to analyze the quality of massive, open, online courses (MOOCs). The International Review of Research in Open and Distributed Learning,16(5), 83–101. Scholar
  30. Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of Massive Open Online Courses (MOOCs). Computers & Education,80, 77–83.CrossRefGoogle Scholar
  31. Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York, NY: Cambridge University Press.CrossRefGoogle Scholar
  32. Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development,50(3), 43–59.CrossRefGoogle Scholar
  33. Milligan, C., & Littlejohn, A. (2014). Supporting professional learning in a Massive Open Online Course. The International Review of Research in Open and Distance Learning. Scholar
  34. National Academies of Sciences, Engineering, and Medicine. (2018). Assessing and responding to the growth of computer science undergraduate enrollments. Washington, D.C.: National Academies Press.Google Scholar
  35. Ossiannilsson, E., Williams, K., Camilleri, A. F., & Brown, M. (2015). Quality models in online and open education around the globe: State of the art and recommendations. Oslo: International Council for Open and Distance Education.Google Scholar
  36. Quality Matters. (2014). Introduction to the quality matters program. Retrieved January 10, 2018 from
  37. Robins, A., Rountree, J., & Rountree, N. (2003). Learning and teaching programming: A review and discussion. Computer Science Education,13(2), 137–172.CrossRefGoogle Scholar
  38. Rohs, M., & Ganz, M. (2015). MOOCs and the claim of education for all: A disillusion by empirical data. International Review of Research in Open and Distributed Learning.,16(6), 5. Scholar
  39. Rosewell, J., & Jansen, D. (2014). The OpenupEd quality label: Benchmarks for MOOCs. INNOQUAL: The International Journal for Innovation and Quality in Learning,2(3), 88–100.Google Scholar
  40. Sanchez-Gordon, S., & Luján-Mora, S. (2018). Technological innovations in large-scale teaching: Five roots of massive open online courses. Journal of Educational Computing Research,56(5), 623–644. Scholar
  41. Scagnoli, N. I., Choo, J., & Tian, J. (2019). Students’ insights on the use of video lectures in online classes. British Journal of Educational Technology,50(1), 399–414. Scholar
  42. Shah, D. (2017). A product at every price: A review of MOOC stats and trends in 2017. Class Central. Retrieved January 7, 2018 from
  43. Spector, J. M. (2014). Remarks on MOOCs and Mini-MOOCs. Educational Technology and Research Development,62, 385–392.CrossRefGoogle Scholar
  44. Stracke, C. M. (2019). Quality frameworks and learning design for open education. International Review of Research in Open and Distributed Learning, 20(2), 180–203.CrossRefGoogle Scholar
  45. Tawfik, A. A., Reeves, T. D., Stich, A. E., Gill, A., Hong, C., McDade, J., et al. (2017). The nature and level of learner-learner interaction in a chemistry massive open online course (MOOC). Journal of Computing in Higher Education,29(3), 411–431. Scholar
  46. Tirrell, T., & Quick, D. (2012). Chickering’s seven principles of good practice: Student attrition in community college online courses. Community College Journal of Research and Practice,36(8), 580–590.CrossRefGoogle Scholar
  47. Toven-Lindsey, B., Rhoads, R. A., & Lozano, J. B. (2015). Virtually unlimited classrooms: Pedagogical practices in massive open online courses. Internet and Higher Education,24, 1–12.CrossRefGoogle Scholar
  48. Veletsianos, G., & Shepherdson, P. (2016). A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. The International Review of Research in Open and Distributed Learning. Scholar
  49. Waltz, C. F., Strickland, O., & Lenz, E. R. (2010). Measurement in nursing and health research (5th ed.). New York, NY: Springer.Google Scholar
  50. Waters, J. (2015). How nanodegrees are disrupting higher education, campus technology. Retrieved January 11, 2018 from
  51. Watson, S. L., Loizzo, J., Watson, W. R., Mueller, C., Lim, J., & Ertmer, P. A. (2016). Instructional design, facilitation, and perceived learning outcomes: An exploratory case study of a human trafficking MOOC for attitudinal change. Educational Technology Research and Development,64, 1273–1300.CrossRefGoogle Scholar
  52. Watson, W. R., Watson, S. L., & Janakiraman, S. (2017). Instructional quality of massive open online courses: A review of attitudinal change MOOCs. International Journal of Learning Technology,12(3), 219–224.CrossRefGoogle Scholar
  53. Wexler, E. (2015). Moocs are still rising at least in numbers. The Chronicle of Higher Education. Retrieved January 7, 2018 from
  54. Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Addison-Wesley Professional.Google Scholar
  55. Wing, J. (2011). Research notebook: Computational thinking—What and why? The Link Magazine, 6. Retrieved January 11, 2018 from
  56. Yang, M., Shao, Z., Liu, Q., & Liu, C. (2017). Understanding the quality factors that influence the continuance intention of students toward participation in MOOCs. Educational Technology Research and Development,65, 1195–1214.CrossRefGoogle Scholar
  57. Yilmaz, A. B., Ünal, M., & Çakir, H. (2017). Evaluating MOOCs according to instructional design principles. Journal of Learning and Teaching in Digital Age,2(2), 26–35.Google Scholar
  58. Yoila, A. O., & Chukwuemeka, E. J. (2015). Instructional design quality evaluation of eastern mediterranean university open courses. International Journal of Scientific Research in Science, Engineering and Technology,1(6), 1–7.Google Scholar
  59. Zarb, M., & Hughes, J. (2015). Breaking the communication barrier: Guidelines to aid communication within pair programming. Computer Science Education,25, 120–151. Scholar
  60. Zhang, Q., Peck, K. L., Hristova, A., Jablokow, K. W., Hoffman, V., Park, E., et al. (2016). Exploring the communication preferences of MOOC learners and the value of preference-based groups: Is grouping enough? Educational Technology Research and Development,64, 809–837. Scholar
  61. Zhenghao, C., Alcorn, B., Christensen, G., Eriksson, N., Koller, D., & Emanuel, E. J. (2015). Who’s benefiting from MOOCs, and why. Harvard Business Review. Retrieved January 8, 2018 from
  62. Zhu, M., Bonk, C. J., & Sari, A. R. (2018a). Instructor experiences designing MOOCs in higher education: Pedagogical, resource, and logistical considerations and challenges. Online Learning,22(4), 203–241. Scholar
  63. Zhu, M., Sari, A., & Lee, M. M. (2018b). A systematic review of research methods and topics of the empirical MOOC literature (2014-2016). The Internet and Higher Education,37, 31–39.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Education Policy, Organization and Leadership, College of EducationUniversity of Illinois at Urbana-ChampaignChampaignUSA
  2. 2.Department of Learning and Instruction, Graduate School of EducationUniversity at Buffalo, The State University of New YorkBuffaloUSA
  3. 3.Faculty of EducationThe University of Hong KongHong KongHong Kong SAR

Personalised recommendations