Advertisement

Predicting Success, Preventing Failure

Using Learning Analytics to Examine the Strongest Predictors of Persistence and Performance in an Online English Language Course
  • Danny GlickEmail author
  • Anat Cohen
  • Eitan Festinger
  • Di Xu
  • Qiujie Li
  • Mark Warschauer
Chapter

Abstract

Online learning has been recognized as a possible approach to increase students’ English language proficiency in developing countries where high-quality instructional resources are limited. Identifying factors that predict students’ performance in online courses can inform institutions and instructors of actionable interventions to improve learning processes and outcomes. Framed in Deci and Ryan’s self-determination theory (SDT) and using data from a pre-course student readiness survey, LMS log files, and a course Facebook page, this study identified key predictors of persistence and achievement among 716 Peruvian students enrolled in an online English language course. Factor analysis was used to identify latent factors from 7 behavioral variables and 18 pre-course student readiness variables. Nine factors emerged, which were classified into three categories of measures based on SDT: competence, autonomy, and relatedness. We found that factors in the categories of competence and autonomy significantly predicted persistence and achievement in online courses. Specifically, the midterm score and self-regulation skills significantly predicted students’ final test score. Counterintuitively, we also found that time spent on the course was a significantly negative predictor of the final test score and that the extent to which a student valued peer learning at the beginning of the course negatively predicted course achievement.

Keywords

Self-determination theory Student persistence Online language learning Developing countries Predictive analytics Factor analysis 

Notes

Acknowledgments

We would like to thank Betty Luz Zegarra Angulo of the Universidad Señor de Sipán for helping make available the data for this study as well as providing detailed information on the study context.

References

  1. Ai, J., & Laffey, J. (2007). Web mining as a tool for understanding online learning. MERLOT Journal of Online Learning and Teaching, 3(2), 160–169.Google Scholar
  2. Allen, D. F., & Bir, B. (2012). Academic confidence and summer bridge learning communities: Path analytic linkages to student persistence. Journal of College Student Retention: Research, Theory & Practice, 13(4), 519–548.Google Scholar
  3. Andrade, M. S., & Bunker, E. L. (2009). A model for self-regulated distance language learning. Distance Education, 30(1), 47–61.Google Scholar
  4. Angelino, L. M., Williams, F. K., & Natvig, D. (2007). Strategies to engage online students and reduce attrition rates. Journal of Educators Online, 4(2), 1–14.Google Scholar
  5. Appana, S. (2008). A review of benefits and limitations of online learning in the context of the student, the instructor, and the tenured faculty. International Journal on E-Learning, 7(1), 5–22.Google Scholar
  6. Baard, P. P. (2002). Intrinsic need satisfaction in organizations: A motivational basis of success in for-profit and not-for-profit settings. Handbook of Self-Determination Research, 2, 255–275.Google Scholar
  7. Bai, Y., Mo, D., Zhang, L., Boswell, M., & Rozelle, S. (2016). The impact of integrating ICT with teaching: Evidence from a randomized controlled trial in rural schools in China. Computers & Education, 96, 1–14.Google Scholar
  8. Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In Learning analytics (pp. 61–75). New York, NY: Springer.Google Scholar
  9. Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.Google Scholar
  10. Bakia, M., Shear, L., Toyama, Y., & Lasseter, A. (2012). Understanding the implications of online learning for educational productivity. Washington, DC: Office of Educational Technology, US Department of Education.Google Scholar
  11. Bañados, E. (2006). A blended-learning pedagogical model for teaching and learning EFL successfully through an online interactive multimedia environment. Calico Journal, 23(3), 533–550.Google Scholar
  12. Banditvilai, C. (2016). Enhancing student’s language skills through blended learning. The Electronic Journal of E-Learning, 14(3), 220–229. Available from www.ejel.org Google Scholar
  13. Bandura, A., Barbaranelli, C., Caprara, G. V., & Pastorelli, C. (2001). Selfefficacy beliefs as shapers of children’s aspirations and career trajectories. Child Development, 72, 187–206.Google Scholar
  14. Barani, G. (2011). The relationship between computer assisted language learning (CALL) and listening skill of Iranian EFL learners. Procedia-Social and Behavioral Sciences, 15, 4059–4063.Google Scholar
  15. Black, A. E., & Deci, E. L. (2000). The effects of instructors’ autonomy support and students’ autonomous motivation on learning organic chemistry: A self-determination theory perspective. Science Education, 84, 740–756.Google Scholar
  16. Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic analytics: A new tool for a new era. Educause Review, 42(4), 42–57.Google Scholar
  17. Cheng, J., Kulkarni, C., & Klemmer, S. (2013). Tools for predicting drop-off in large online classes. In Proceedings of the 2013 conference on computer supported cooperative work companion (pp. 121–124). New York, NY: ACM.Google Scholar
  18. Clay, M., Rowland, S., & Packard, A. (2009). Improving undergraduate online retention through gated advisement and redundant communication. Journal of College Student Retention, 10(1), 93–102.Google Scholar
  19. Cohen, A., & Nachmias, R. (2006). A quantitative cost effectiveness model for web-supported academic instruction. The Internet and Higher Education, 9(2), 81–90.Google Scholar
  20. Cohen, A., & Nachmias, R. (2012). The implementation of a cost effectiveness analyzer for web-supported academic instruction: An example from life science. International Journal on E-Learning, 11(2), 5–22.Google Scholar
  21. Cohen, A. (2017). Analysis of student activity in web-supported courses as a tool for predicting dropout. Educational Technology Research and Development, 65(5), 1285–1304.Google Scholar
  22. Cronquist, K., & Fiszbein, A. (2017). English Language Learning in Latin America. Inter-American Dialogue. Available from: www.dropbox.com/s/zabf293t0a12ten/English-Language-Learning-in-Latin-America-Final.pdf?dl=0
  23. Dawson, S., Gašević, D., Siemens, G., & Joksimovic, S. (2014). Current state and future trends: A citation network analysis of the learning analytics field. In Proceedings of the fourth international conference on learning analytics and knowledge (pp. 231–240). New York, NY: ACM.  https://doi.org/10.1145/2567574.2567585 Google Scholar
  24. Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York, NY: Plenum.Google Scholar
  25. Dickinson, L. (1995). Autonomy and motivation a literature review. System, 23(2), 165–174.Google Scholar
  26. Dietz-Uhler, B., & Hurn, J. E. (2013). Using learning analytics to predict (and improve) student success: A faculty perspective. Journal of Interactive Online Learning, 12(1), 17–26.Google Scholar
  27. Dominguez, M., Bernacki, M. L., & Uesbeck, P. M. (2016). Predicting STEM achievement with learning management system data: Prediction modeling and a test of an early warning system. In EDM (pp. 589–590).Google Scholar
  28. Driscoll, A., Jicha, K., Hunt, A. N., Tichavsky, L., & Thompson, G. (2012). Can online courses deliver in-class results? A comparison of student performance and satisfaction in an online versus a face-to-face introductory sociology course. Teaching Sociology, 40(4), 312–331.  https://doi.org/10.1177/0092055X12446624 Google Scholar
  29. Duffy, T. M., & Kirkley, J. R. (2003). Learner-centered theory and practice in distance education: Cases from higher education. New York, NY: Routledge.Google Scholar
  30. Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53(1), 109–132.Google Scholar
  31. EF English Proficiency Index. (2016). EF education first. Available from https://www.theewf.org/uploads/pdf/ef-epi-2016-english.pdf
  32. Erdem, M., & Kibar, P. N. (2014). Students’ opinions on facebook supported blended learning environment. TOJET: The Turkish Online. Journal of Educational Technology, 13(1), 199–206.Google Scholar
  33. Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71.Google Scholar
  34. Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84.Google Scholar
  35. Giesbers, B., Rienties, B., Tempelaar, D., & Gijselaers, W. (2013). Investigating the relations between motivation, tool use, participation, and performance in an e-learning course using web-videoconferencing. Computers in Human Behavior, 29(1), 285–292.Google Scholar
  36. Glick, D., Xu, D., Warschauer, M., Rodriguez, F., Li, Q., & Cung, B. (2016). Maximizing learning outcomes through blended learning: What research shows. Paper presented at the 2016 Association of Binational Centers of Latin America Conference, Houston, TX.Google Scholar
  37. Goldstein, P. J., & Katz, R. N. (2005). Academic analytics: The uses of management information and technology in higher education. Washington, DC: EDUCAUSE Center for Applied Research.Google Scholar
  38. Hartnett, M., George, A. S., & Dron, J. (2011). Examining motivation in online distance learning environments: Complex, multifaceted and situation-dependent. The International Review of Research in Open and Distributed Learning, 12(6), 20–38.Google Scholar
  39. Ho, J., & Crookall, D. (1995). Breaking with Chinese cultural traditions: Learner autonomy in English language teaching. System, 23(2), 235–243.Google Scholar
  40. Holec, H. (1981). Autonomy and foreign language learning. Oxford, England/New York, NY: Pergamon Press (First Published 1979, Council of Europe).Google Scholar
  41. Jaggers, S. S., & Xu, D. (2016). How do online course design features influence student performance? Computers & Education, 95, 270–284.Google Scholar
  42. Järvelä, S., Volet, S., & Järvenoja, H. (2010). Research on motivation in collaborative learning: Moving beyond the cognitive–situative divide and combining. Educational Psychologist, 45, 15–27.Google Scholar
  43. Johnson, L., Becker, S., Estrada, V., & Freeman, A. (2015). The NMC Horizon Report: 2015 Higher Education Edition. Austin, TX: New Media Consortium.Google Scholar
  44. Komarraju, M., & Nadler, D. (2013). Self-efficacy and academic achievement: Why do implicit beliefs, goals, and effort regulation matter? Learning and Individual Differences, 25, 67–72.Google Scholar
  45. Lee, L. (2016). Autonomous learning through task-based instruction in fully online language courses. Language Learning & Technology, 20(2), 81–97.Google Scholar
  46. Levi-Gamlieli, H., Cohen, A., & Nachmias, R. (2015). Detection of overly intensive learning by using weblog of course website. Technology, Instruction, Cognition and Learning (TICL), 10(2), 151–171.Google Scholar
  47. Levy, Y. (2007). Comparing dropouts and persistence in e-learning courses. Computers & Education, 48, 185–204.  https://doi.org/10.1016/j.compedu.2004.12.004 Google Scholar
  48. Lim, J. M. (2016). Predicting successful completion using student delay indicators in undergraduate self-paced online courses. Distance Education, 37(3), 317–332.  https://doi.org/10.1080/01587919.2016.1233050 Google Scholar
  49. Lu, J., Yu, C. S., & Liu, C. (2003). Learning style, learning patterns, and learning performance in a WebCT-based MIS course. Information & Management, 40(6), 497–507.  https://doi.org/10.1016/S0378-7206(02)00064-2 Google Scholar
  50. Lykourentzou, I., Giannoukos, I., Nikolopoulos, V., Mpardis, G., & Loumos, V. (2009). Dropout prediction in e-learning courses through the combination of machine learning techniques. Computers & Education, 53, 950–965.  https://doi.org/10.1016/j.compedu.2009.05.010 Google Scholar
  51. Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop “early warning system” for educators: A proof of concept. Computers & Education, 54, 588–599.  https://doi.org/10.1016/j.compedu.2009.09.008 Google Scholar
  52. Marzban, A. (2011). Improvement of reading comprehension through computer-assisted language learning in Iranian intermediate EFL students. Procedia Computer Science, 3, 3–10.Google Scholar
  53. Maki, R., & Maki, W. (2003). Prediction of learning and satisfaction in Webbased and lecture courses. Journal of Educational Computing Research, 28(3), 197–219.Google Scholar
  54. Mandernach, B. J. (2009). Effect of instructor-personalized multimedia in the online classroom. International Review of Research in Open and Distance Learning, 10(3), 19.Google Scholar
  55. Massengale, L. R., & Vasquez, E. (2016). Assessing accessibility: Are online courses better than face-to-face instruction at providing access to course content for students with disabilities? Journal of the Scholarship of Teaching and Learning, 16(1), 69–79.  https://doi.org/10.14434/josotl.v16i1.19101 Google Scholar
  56. Miltiadou, M., & Savenye, W. C. (2003). Applying social cognitive constructs of motivation to enhance student success in online distance education. AACE Journal, 11(1), 78–95.Google Scholar
  57. Ministério de Educación Perú. (2016a). Currículo nacional de la educación básica. Retrieved from http://www.minedu.gob.pe/curriculo/pdf/curriculo-nacional-2016-2.pdf.
  58. Ministério de Educación Perú. (2016b). Plan de implementación al 2021 de la política nacional de enseñanza, aprendizaje y uso de idioma Inglés – Política “Inglés, puertas al mundo. Retrieved from http://www.minedu.gob.pe/ingles-puertas-al-mundo.
  59. Ministério de Educación Perú. (2017). Jornada escolar completa: Secundaria. Retrieved from http://www.minedu.gob.pe/jec/escuela-jec.php.
  60. Moore, M. G., & Kearsley, G. (2011). Distance education: A systems view of online learning. Belmont, CA: Cengage Learning.Google Scholar
  61. Rios, S. M., & Cabrera, A. F. (2008). La efectividad de un model de aprendizaje combinado para la enseñanza del inglés como lengua extranjera: Estudio empírico. RLA, Revista de Linguistica Teórica y Aplicada, 46(2), 95–118. Retrieved from http://www.scielo.cl/scielo.php?script=sci_arttext&pid=S0718-48832008000200006 Google Scholar
  62. Muilenburg, L. Y., & Berge, Z. L. (2005). Student barriers to online learning: A factor analytic study. Distance Education, 26(1), 29–48.  https://doi.org/10.1080/01587910500081269 Google Scholar
  63. Nistor, N., & Neubauer, K. (2010). From participation to dropout: Quantitative participation patterns in online university courses. Computers & Education, 55, 663–672.  https://doi.org/10.1016/j.compedu.2010.02.026 Google Scholar
  64. OECD. (2015). Education at a glance 2015: OECD indicators. Paris, France: OECD Publishing.  https://doi.org/10.1787/eag-2015-en Google Scholar
  65. Otter, R. R., Seipel, S., Graeff, T., Alexander, B., Boraiko, C., Gray, J., … Sadler, K. (2013). Comparing student and faculty perceptions of online and traditional courses. The Internet and Higher Education, 19, 27–35.  https://doi.org/10.1016/j.iheduc.2013.08.001 Google Scholar
  66. Palmer, S., & Holt, D. (2010). Students’ perceptions of the value of the elements of an online learning environment: Looking back in moving forward. Interactive Learning Environments, 18(2), 135–151.Google Scholar
  67. Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the motivated strategies questionnaire (MSLQ). Ann Arbor, MI: University of Michigan, National Center for Research to Improve Postsecondary Teaching and Learning.Google Scholar
  68. Park, J.-H., & Choi, H. J. (2009). Factors influencing adult learners’ decision to drop out or persist in online learning. Educational Technology & Society, 12(4), 207–217.Google Scholar
  69. Rienties, B., Tempelaar, D., Van den Bossche, P., Gijselaers, W., & Segers, M. (2009). The role of academic motivation in computer-supported collaborative learning. Computers in Human Behavior, 25(6), 1195–1206.Google Scholar
  70. Roby, T., Ashe, S., Singh, N., & Clark, C. (2013). Shaping the online experience: How administrators can influence student and instructor perceptions through policy and practice. The Internet and Higher Education, 17, 29–37.Google Scholar
  71. Rodriguez, M. C., Rooms, A., & Montañez, M. (2008). Students’ perceptions of online-learning quality given comfort, motivation, satisfaction, and experience. Journal of Interactive Online Learning, 7(2), 105–125.Google Scholar
  72. Romero, C., López, M. I., Luna, J. M., & Ventura, S. (2013). Predicting students’ final performance from participation in on-line discussion forums. Computers & Education, 68, 458–472.  https://doi.org/10.1016/j.compedu.2013.06.009 Google Scholar
  73. Romero, C., & Ventura, S. (2007). Educational data mining: A survey from 1995 to 2005. Expert Systems with Applications, 33(1), 135–146.  https://doi.org/10.1016/j.eswa.2006.04.005 Google Scholar
  74. Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67.Google Scholar
  75. Santana, M. A., Costa, E. B., Neto, B. F. D. S., Silva, I. C. L., & Rego, J. B. (2015). A predictive model for identifying students with dropout profiles in online courses. In Proceeding of the 8th international conference on educational data mining, EDM Workshops.Google Scholar
  76. Schunk, D. H. (2012). Learning theories, an educational perspective (6th ed.). Boston, MA: Pearson Education Inc.Google Scholar
  77. Siemens, G., & Gašević, D. (2012). Special issue on learning and knowledge analytics. Educational Technology & Society, 15(3), 1–163.Google Scholar
  78. Sife, A., Lwoga, E., & Sanga, C. (2007). New technologies for teaching and learning: Challenges for higher learning institutions in developing countries. International Journal of Education and Development Using ICT, 3(2), 57–67.Google Scholar
  79. Vahdat, S., & Eidipour, M. (2016). Adopting CALL to improve listening comprehension of iranian junior high school students. Theory and Practice in Language Studies, 6(8), 1609–1617.Google Scholar
  80. Vansteenkiste, M., Lens, W., & Deci, E. L. (2006). Intrinsic versus extrinsic goal contents in self-determination theory: Another look at the quality of academic motivation. Educational Psychologist, 41(1), 19–31.Google Scholar
  81. Willging, P. A., & Johnson, S. D. (2009). Factors that influence students’ decision to dropout of online courses. Journal of Asynchronous Learning Networks, 13(3), 115–127.Google Scholar
  82. Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In Handbook of self-regulation (pp. 531–566).Google Scholar
  83. Wladis, C., & Samuels, J. (2016). Do online readiness surveys do what they claim? Validity, reliability, and subsequent student enrollment decisions. Computers & Education, 98, 39–56.  https://doi.org/10.1016/j.compedu.2016.03.001 Google Scholar
  84. Xie, K. U. I., Debacker, T. K., & Ferguson, C. (2006). Extending the traditional classroom through online discussion: The role of student motivation. Journal of Educational Computing Research, 34(1), 67–89.Google Scholar
  85. Xu, D., & Jaggars, S. S. (2014). Performance gaps between online and face-to-face courses: Differences across types of students and academic subject areas. The Journal of Higher Education, 85(5), 633–659.Google Scholar
  86. You, J. W. (2016). Identifying significant indicators using LMS data to predict course achievement in online learning. The Internet and Higher Education, 29, 23–30.  https://doi.org/10.1016/j.iheduc.2015.11.003 Google Scholar
  87. Yuan, J., & Kim, C. (2014). Guidelines for facilitating the development of learning communities in online courses. Journal of Computer Assisted Learning, 30(3), 220–232.Google Scholar
  88. Zakrzewska, D. (2009). Cluster analysis in personalized e-learning systems. In N. T. Nguyen & E. Szczerbicki (Eds.), Intelligent systems for knowledge management (pp. 229–250). Berlin, Germany: Springer.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Danny Glick
    • 1
    Email author
  • Anat Cohen
    • 2
  • Eitan Festinger
    • 2
  • Di Xu
    • 3
  • Qiujie Li
    • 3
  • Mark Warschauer
    • 3
  1. 1.Edusoft, a Subsidiary of ETS, and UC Irvine’s Digital Learning LabUniversity of CaliforniaIrvineUSA
  2. 2.Tel-Aviv UniversityTel AvivIsrael
  3. 3.University of CaliforniaIrvineUSA

Personalised recommendations