How Do English Language Learners Interact with Different Content Types in MOOC Videos?

  • Judith Uchidiuno
  • Ken Koedinger
  • Jessica Hammer
  • Evelyn Yarzebinski
  • Amy Ogan


English Language Learners (ELLs) are a substantial portion of the students who enroll in MOOCs. In order to fulfill the promise of MOOCs – i.e., making higher education accessible to everyone with an internet connection – appropriate interventions should be offered to students who struggle with the language of course content. Through the analysis of clickstream log data gathered from two MOOC courses deployed on Coursera, Introduction to Psychology and Statistical Thermodynamics, we show that compared to native English speakers, ELL students have distinct behavioral patterns in how they engage with MOOC content including increased interaction with content that contains text, increased seeking away from content without visual support, and decreased video play rates. These patterns are expressed differently in response to different types of course content and domains. Our findings not only suggest more fine-grained methods for automatically identifying students who need language interventions, but also have further implications for the design of language support interventions and MOOC videos.


MOOCs English language learners MOOC behavioral analysis ELL student identification Language support interventions 


  1. Bacon, S. M. (1989). Listening for real in the foreign-language Classroom1. Foreign Language Annals, 22(6), 543–550.CrossRefGoogle Scholar
  2. Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62(2), 233–246.CrossRefGoogle Scholar
  3. Chang, A. C.-S., & Read, J. (2006). The effects of listening support on the listening performance of EFL learners. TESOL Quarterly, 40(2), 375–397. Scholar
  4. Coursera (2013). A new partnership to bring Coursera to the hundreds of millions of Chinese learners across the world. Retrieved October 20, 2015, from
  5. Crossley, S. A., Allen, D. B., & McNamara, D. S. (2011). Text readability and intuitive simplification: a comparison of readability formulas. Reading in a Foreign Language, 23(1), 86.Google Scholar
  6. Cutrell, E., Bala, S., Bansal, C., Cross, A., Datha, N., John, A., … et al. (2013). Massively empowered classroom: Enhancing technical education in india. Microsoft Research Technical Report. Retrieved from
  7. DeBoer, J., Stump, G. S., Seaton, D., & Breslow, L. (2013). Diversity in MOOC students’ backgrounds and behaviors in relationship to performance in 6.002 x. In Proceedings of the Sixth Learning International Networks Consortium Conference. Retrieved from’13.pdf.
  8. Derwing, T., & Munro, M. J. (2001). What speaking rates do non-native listeners prefer? Applied Linguistics, 22(3), 324–337. Scholar
  9. Dillahunt, T., Chen, B., & Teasley, S. (2014a). Model thinking: demographics and performance of mooc students unable to afford a formal education. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 145–146). ACM. Retrieved from
  10. Dillahunt, T. R., Wang, B. Z., & Teasley, S. (2014b). Democratizing higher education: Exploring MOOC use among those who cannot afford a formal education. The International Review of Research in Open and Distributed Learning, 15(5). Retrieved from
  11. Finkelstein, S., Yarzebinski, E., Vaughn, C., Ogan, A., & Cassell, J. (2013). The effects of culturally congruent educational technologies on student achievement. In Artificial intelligence in education (pp. 493–502). Springer. Retrieved from
  12. Guo, P. J., & Reinecke, K. (2014). Demographic differences in how students navigate through MOOCs. L@S 2014, March 4–5, 2014. Retrieved from
  13. Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the first ACM conference on learning @ scale conference (pp. 41–50). New York: ACM. doi:
  14. Hayati, A. (2010). The effect of speech rate on listening comprehension of EFL learners. Creative Education, 01(02), 107–114. Scholar
  15. Khan Academy (2015). Carlos Slim’s Foundation puts its weight behind Khan Academy. Retrieved October 20, 2015, from
  16. Kim, S., & Chang, M. (2010). Computer games for the math achievement of diverse students. Educational Technology & Society, 13(3), 224–232.Google Scholar
  17. Kim, J., Guo, P. J., Seaton, D. T., Mitros, P., Gajos, K. Z., & Miller, R. C. (2014a). Understanding in-video dropouts and interaction peaks inonline lecture videos. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 31–40). ACM. Retrieved from
  18. Kim, J., Guo, P. J., Cai, C. J., Li, S.-W. D., Gajos, K. Z., & Miller, R. C. (2014b). Data-driven interaction techniques for improving navigation of educational videos. In Proceedings of the 27th annual ACM symposium on User interface software and technology (pp. 563–572). ACM. Retrieved from
  19. Koedinger, K. R., Kim, J., Jia, J. Z., McLaughlin, E. A., & Bier, N. L. (2015). Learning is not a spectator sport: Doing is better than watching for learning from a MOOC. In Proceedings of the second (2015) ACM conference on learning @ scale (pp. 111–120). New York: ACM. doi:
  20. Kovacs, G. (2016). Effects of in-video quizzes on MOOC lecture viewing. In Proceedings of the third (2016) ACM conference on Learning@ Scale (pp. 31–40). ACM. Retrieved from
  21. Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43–52.CrossRefGoogle Scholar
  22. Mayer, R. E., Sobko, K., & Mautone, P. D. (2003). Social cues in multimedia learning: role of speaker’s voice. Journal of Educational Psychology, 95(2), 419.CrossRefGoogle Scholar
  23. McNamara, D. S., & Graesser, A. C. (2012). Coh-Metrix: An automated tool for theoretical and applied natural language processing. Applied natural language processing and content analysis: identification, investigation, and resolution. Hershey: IGI Global. Retrieved from
  24. MOOC List (2015). Find MOOCs by languages. Retrieved October 23, 2015, from
  25. Online Course Report (2015). The 50 most popular MOOCs of all time. Retrieved April 4, 2017, from
  26. Plass, J. L., Chun, D. M., Mayer, R. E., & Leutner, D. (2003). Cognitive load in reading a foreign language text with multimedia aids and the influence of verbal and spatial abilities. Computers in Human Behavior, 19(2), 221–243.CrossRefGoogle Scholar
  27. Renandya, W. A., & Farrell, T. S. (2010). Teacher, the tape is too fast! Extensive listening in ELT. ELT Journal, 65(1):52–59.Google Scholar
  28. Ross, S. (1998). Self-assessment in second language testing: a meta-analysis and analysis of experiential factors. Language Testing, 15(1), 1–20.CrossRefGoogle Scholar
  29. Seaton, D. T., Nesterko, S., Mullaney, T., Reich, J., Ho, A., & Chuang, I. (2014a). Characterizing video use in the catalogue of MITx MOOCs. Proceedings of the European MOOC Stakeholders Summit. Lausan: PAU Education, 140–146.Google Scholar
  30. Seaton, D. T., Bergner, Y., Chuang, I., Mitros, P., & Pritchard, D. E. (2014b). Who does what in a massive open online course? Communications of the ACM, 57(4), 58–65. Scholar
  31. Silverman, R., & Hines, S. (2009). The effects of multimedia-enhanced instruction on the vocabulary of English-language learners and non-English-language learners in pre-kindergarten through second grade. Journal of Educational Psychology, 101(2), 305.CrossRefGoogle Scholar
  32. Snyder, S., & Writer, I. S. (2013). Study shows who’s taking open online college courses. Retrieved October 14, 2015, from
  33. Uchidiuno, J., Ogan, A., Yarzebinski, E., & Hammer, J. (2016a). Understanding ESL students’ motivations to increase MOOC accessibility. In Proceedings of the third (2016) ACM conference on Learning@ Scale (pp. 169–172). ACM. Retrieved from
  34. Uchidiuno, J., Ogan, A., Koedinger, K. R., Yarzebinski, E., & Hammer, J. (2016b). Browser language preferences as a metric for identifying ESL Speakers in MOOCs. In Proceedings of the third (2016) ACM conference on Learning@ Scale (pp. 277–280). ACM. Retrieved from
  35. Wang, X., Yang, D., Wen, M., Koedinger, K., & Rosé, C. P. (n.d.). Investigating how student’s cognitive behavior in MOOC discussion forums affect learning gains. Retrieved from

Copyright information

© International Artificial Intelligence in Education Society 2017

Authors and Affiliations

  • Judith Uchidiuno
    • 1
  • Ken Koedinger
    • 1
  • Jessica Hammer
    • 1
  • Evelyn Yarzebinski
    • 1
  • Amy Ogan
    • 1
  1. 1.Human-Computer Interaction InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations