Skip to main content

Predicting Success, Preventing Failure

Using Learning Analytics to Examine the Strongest Predictors of Persistence and Performance in an Online English Language Course

Abstract

Online learning has been recognized as a possible approach to increase students’ English language proficiency in developing countries where high-quality instructional resources are limited. Identifying factors that predict students’ performance in online courses can inform institutions and instructors of actionable interventions to improve learning processes and outcomes. Framed in Deci and Ryan’s self-determination theory (SDT) and using data from a pre-course student readiness survey, LMS log files, and a course Facebook page, this study identified key predictors of persistence and achievement among 716 Peruvian students enrolled in an online English language course. Factor analysis was used to identify latent factors from 7 behavioral variables and 18 pre-course student readiness variables. Nine factors emerged, which were classified into three categories of measures based on SDT: competence, autonomy, and relatedness. We found that factors in the categories of competence and autonomy significantly predicted persistence and achievement in online courses. Specifically, the midterm score and self-regulation skills significantly predicted students’ final test score. Counterintuitively, we also found that time spent on the course was a significantly negative predictor of the final test score and that the extent to which a student valued peer learning at the beginning of the course negatively predicted course achievement.

Keywords

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The CEFR is an international standard for describing language ability ranging from A1 (basic) up to C2 (proficient).

  2. 2.

    The MSLQ is in the public domain; therefore, permission to use this instrument is not needed.

References

  • Ai, J., & Laffey, J. (2007). Web mining as a tool for understanding online learning. MERLOT Journal of Online Learning and Teaching, 3(2), 160–169.

    Google Scholar 

  • Allen, D. F., & Bir, B. (2012). Academic confidence and summer bridge learning communities: Path analytic linkages to student persistence. Journal of College Student Retention: Research, Theory & Practice, 13(4), 519–548.

    Article  Google Scholar 

  • Andrade, M. S., & Bunker, E. L. (2009). A model for self-regulated distance language learning. Distance Education, 30(1), 47–61.

    Article  Google Scholar 

  • Angelino, L. M., Williams, F. K., & Natvig, D. (2007). Strategies to engage online students and reduce attrition rates. Journal of Educators Online, 4(2), 1–14.

    Article  Google Scholar 

  • Appana, S. (2008). A review of benefits and limitations of online learning in the context of the student, the instructor, and the tenured faculty. International Journal on E-Learning, 7(1), 5–22.

    Google Scholar 

  • Baard, P. P. (2002). Intrinsic need satisfaction in organizations: A motivational basis of success in for-profit and not-for-profit settings. Handbook of Self-Determination Research, 2, 255–275.

    Google Scholar 

  • Bai, Y., Mo, D., Zhang, L., Boswell, M., & Rozelle, S. (2016). The impact of integrating ICT with teaching: Evidence from a randomized controlled trial in rural schools in China. Computers & Education, 96, 1–14.

    Article  Google Scholar 

  • Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In Learning analytics (pp. 61–75). New York, NY: Springer.

    Chapter  Google Scholar 

  • Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3–17.

    Google Scholar 

  • Bakia, M., Shear, L., Toyama, Y., & Lasseter, A. (2012). Understanding the implications of online learning for educational productivity. Washington, DC: Office of Educational Technology, US Department of Education.

    Google Scholar 

  • Bañados, E. (2006). A blended-learning pedagogical model for teaching and learning EFL successfully through an online interactive multimedia environment. Calico Journal, 23(3), 533–550.

    Article  Google Scholar 

  • Banditvilai, C. (2016). Enhancing student’s language skills through blended learning. The Electronic Journal of E-Learning, 14(3), 220–229. Available from www.ejel.org

    Google Scholar 

  • Bandura, A., Barbaranelli, C., Caprara, G. V., & Pastorelli, C. (2001). Selfefficacy beliefs as shapers of children’s aspirations and career trajectories. Child Development, 72, 187–206.

    Article  Google Scholar 

  • Barani, G. (2011). The relationship between computer assisted language learning (CALL) and listening skill of Iranian EFL learners. Procedia-Social and Behavioral Sciences, 15, 4059–4063.

    Article  Google Scholar 

  • Black, A. E., & Deci, E. L. (2000). The effects of instructors’ autonomy support and students’ autonomous motivation on learning organic chemistry: A self-determination theory perspective. Science Education, 84, 740–756.

    Article  Google Scholar 

  • Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic analytics: A new tool for a new era. Educause Review, 42(4), 42–57.

    Google Scholar 

  • Cheng, J., Kulkarni, C., & Klemmer, S. (2013). Tools for predicting drop-off in large online classes. In Proceedings of the 2013 conference on computer supported cooperative work companion (pp. 121–124). New York, NY: ACM.

    Chapter  Google Scholar 

  • Clay, M., Rowland, S., & Packard, A. (2009). Improving undergraduate online retention through gated advisement and redundant communication. Journal of College Student Retention, 10(1), 93–102.

    Article  Google Scholar 

  • Cohen, A., & Nachmias, R. (2006). A quantitative cost effectiveness model for web-supported academic instruction. The Internet and Higher Education, 9(2), 81–90.

    Article  Google Scholar 

  • Cohen, A., & Nachmias, R. (2012). The implementation of a cost effectiveness analyzer for web-supported academic instruction: An example from life science. International Journal on E-Learning, 11(2), 5–22.

    Google Scholar 

  • Cohen, A. (2017). Analysis of student activity in web-supported courses as a tool for predicting dropout. Educational Technology Research and Development, 65(5), 1285–1304.

    Article  Google Scholar 

  • Cronquist, K., & Fiszbein, A. (2017). English Language Learning in Latin America. Inter-American Dialogue. Available from: www.dropbox.com/s/zabf293t0a12ten/English-Language-Learning-in-Latin-America-Final.pdf?dl=0

  • Dawson, S., Gašević, D., Siemens, G., & Joksimovic, S. (2014). Current state and future trends: A citation network analysis of the learning analytics field. In Proceedings of the fourth international conference on learning analytics and knowledge (pp. 231–240). New York, NY: ACM. https://doi.org/10.1145/2567574.2567585

    Chapter  Google Scholar 

  • Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York, NY: Plenum.

    Book  Google Scholar 

  • Dickinson, L. (1995). Autonomy and motivation a literature review. System, 23(2), 165–174.

    Article  Google Scholar 

  • Dietz-Uhler, B., & Hurn, J. E. (2013). Using learning analytics to predict (and improve) student success: A faculty perspective. Journal of Interactive Online Learning, 12(1), 17–26.

    Google Scholar 

  • Dominguez, M., Bernacki, M. L., & Uesbeck, P. M. (2016). Predicting STEM achievement with learning management system data: Prediction modeling and a test of an early warning system. In EDM (pp. 589–590).

    Google Scholar 

  • Driscoll, A., Jicha, K., Hunt, A. N., Tichavsky, L., & Thompson, G. (2012). Can online courses deliver in-class results? A comparison of student performance and satisfaction in an online versus a face-to-face introductory sociology course. Teaching Sociology, 40(4), 312–331. https://doi.org/10.1177/0092055X12446624

    Article  Google Scholar 

  • Duffy, T. M., & Kirkley, J. R. (2003). Learner-centered theory and practice in distance education: Cases from higher education. New York, NY: Routledge.

    Book  Google Scholar 

  • Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53(1), 109–132.

    Article  Google Scholar 

  • EF English Proficiency Index. (2016). EF education first. Available from https://www.theewf.org/uploads/pdf/ef-epi-2016-english.pdf

  • Erdem, M., & Kibar, P. N. (2014). Students’ opinions on facebook supported blended learning environment. TOJET: The Turkish Online. Journal of Educational Technology, 13(1), 199–206.

    Google Scholar 

  • Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71.

    Article  Google Scholar 

  • Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84.

    Article  Google Scholar 

  • Giesbers, B., Rienties, B., Tempelaar, D., & Gijselaers, W. (2013). Investigating the relations between motivation, tool use, participation, and performance in an e-learning course using web-videoconferencing. Computers in Human Behavior, 29(1), 285–292.

    Article  Google Scholar 

  • Glick, D., Xu, D., Warschauer, M., Rodriguez, F., Li, Q., & Cung, B. (2016). Maximizing learning outcomes through blended learning: What research shows. Paper presented at the 2016 Association of Binational Centers of Latin America Conference, Houston, TX.

    Google Scholar 

  • Goldstein, P. J., & Katz, R. N. (2005). Academic analytics: The uses of management information and technology in higher education. Washington, DC: EDUCAUSE Center for Applied Research.

    Google Scholar 

  • Hartnett, M., George, A. S., & Dron, J. (2011). Examining motivation in online distance learning environments: Complex, multifaceted and situation-dependent. The International Review of Research in Open and Distributed Learning, 12(6), 20–38.

    Article  Google Scholar 

  • Ho, J., & Crookall, D. (1995). Breaking with Chinese cultural traditions: Learner autonomy in English language teaching. System, 23(2), 235–243.

    Article  Google Scholar 

  • Holec, H. (1981). Autonomy and foreign language learning. Oxford, England/New York, NY: Pergamon Press (First Published 1979, Council of Europe).

    Google Scholar 

  • Jaggers, S. S., & Xu, D. (2016). How do online course design features influence student performance? Computers & Education, 95, 270–284.

    Article  Google Scholar 

  • Järvelä, S., Volet, S., & Järvenoja, H. (2010). Research on motivation in collaborative learning: Moving beyond the cognitive–situative divide and combining. Educational Psychologist, 45, 15–27.

    Article  Google Scholar 

  • Johnson, L., Becker, S., Estrada, V., & Freeman, A. (2015). The NMC Horizon Report: 2015 Higher Education Edition. Austin, TX: New Media Consortium.

    Google Scholar 

  • Komarraju, M., & Nadler, D. (2013). Self-efficacy and academic achievement: Why do implicit beliefs, goals, and effort regulation matter? Learning and Individual Differences, 25, 67–72.

    Article  Google Scholar 

  • Lee, L. (2016). Autonomous learning through task-based instruction in fully online language courses. Language Learning & Technology, 20(2), 81–97.

    Google Scholar 

  • Levi-Gamlieli, H., Cohen, A., & Nachmias, R. (2015). Detection of overly intensive learning by using weblog of course website. Technology, Instruction, Cognition and Learning (TICL), 10(2), 151–171.

    Google Scholar 

  • Levy, Y. (2007). Comparing dropouts and persistence in e-learning courses. Computers & Education, 48, 185–204. https://doi.org/10.1016/j.compedu.2004.12.004

    Article  Google Scholar 

  • Lim, J. M. (2016). Predicting successful completion using student delay indicators in undergraduate self-paced online courses. Distance Education, 37(3), 317–332. https://doi.org/10.1080/01587919.2016.1233050

    Article  Google Scholar 

  • Lu, J., Yu, C. S., & Liu, C. (2003). Learning style, learning patterns, and learning performance in a WebCT-based MIS course. Information & Management, 40(6), 497–507. https://doi.org/10.1016/S0378-7206(02)00064-2

    Article  Google Scholar 

  • Lykourentzou, I., Giannoukos, I., Nikolopoulos, V., Mpardis, G., & Loumos, V. (2009). Dropout prediction in e-learning courses through the combination of machine learning techniques. Computers & Education, 53, 950–965. https://doi.org/10.1016/j.compedu.2009.05.010

    Article  Google Scholar 

  • Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop “early warning system” for educators: A proof of concept. Computers & Education, 54, 588–599. https://doi.org/10.1016/j.compedu.2009.09.008

    Article  Google Scholar 

  • Marzban, A. (2011). Improvement of reading comprehension through computer-assisted language learning in Iranian intermediate EFL students. Procedia Computer Science, 3, 3–10.

    Article  Google Scholar 

  • Maki, R., & Maki, W. (2003). Prediction of learning and satisfaction in Webbased and lecture courses. Journal of Educational Computing Research, 28(3), 197–219.

    Article  Google Scholar 

  • Mandernach, B. J. (2009). Effect of instructor-personalized multimedia in the online classroom. International Review of Research in Open and Distance Learning, 10(3), 19.

    Article  Google Scholar 

  • Massengale, L. R., & Vasquez, E. (2016). Assessing accessibility: Are online courses better than face-to-face instruction at providing access to course content for students with disabilities? Journal of the Scholarship of Teaching and Learning, 16(1), 69–79. https://doi.org/10.14434/josotl.v16i1.19101

    Article  Google Scholar 

  • Miltiadou, M., & Savenye, W. C. (2003). Applying social cognitive constructs of motivation to enhance student success in online distance education. AACE Journal, 11(1), 78–95.

    Google Scholar 

  • Ministério de Educación Perú. (2016a). Currículo nacional de la educación básica. Retrieved from http://www.minedu.gob.pe/curriculo/pdf/curriculo-nacional-2016-2.pdf.

  • Ministério de Educación Perú. (2016b). Plan de implementación al 2021 de la política nacional de enseñanza, aprendizaje y uso de idioma Inglés – Política “Inglés, puertas al mundo. Retrieved from http://www.minedu.gob.pe/ingles-puertas-al-mundo.

  • Ministério de Educación Perú. (2017). Jornada escolar completa: Secundaria. Retrieved from http://www.minedu.gob.pe/jec/escuela-jec.php.

  • Moore, M. G., & Kearsley, G. (2011). Distance education: A systems view of online learning. Belmont, CA: Cengage Learning.

    Google Scholar 

  • Rios, S. M., & Cabrera, A. F. (2008). La efectividad de un model de aprendizaje combinado para la enseñanza del inglés como lengua extranjera: Estudio empírico. RLA, Revista de Linguistica Teórica y Aplicada, 46(2), 95–118. Retrieved from http://www.scielo.cl/scielo.php?script=sci_arttext&pid=S0718-48832008000200006

    Google Scholar 

  • Muilenburg, L. Y., & Berge, Z. L. (2005). Student barriers to online learning: A factor analytic study. Distance Education, 26(1), 29–48. https://doi.org/10.1080/01587910500081269

    Article  Google Scholar 

  • Nistor, N., & Neubauer, K. (2010). From participation to dropout: Quantitative participation patterns in online university courses. Computers & Education, 55, 663–672. https://doi.org/10.1016/j.compedu.2010.02.026

    Article  Google Scholar 

  • OECD. (2015). Education at a glance 2015: OECD indicators. Paris, France: OECD Publishing. https://doi.org/10.1787/eag-2015-en

    Book  Google Scholar 

  • Otter, R. R., Seipel, S., Graeff, T., Alexander, B., Boraiko, C., Gray, J., … Sadler, K. (2013). Comparing student and faculty perceptions of online and traditional courses. The Internet and Higher Education, 19, 27–35. https://doi.org/10.1016/j.iheduc.2013.08.001

    Article  Google Scholar 

  • Palmer, S., & Holt, D. (2010). Students’ perceptions of the value of the elements of an online learning environment: Looking back in moving forward. Interactive Learning Environments, 18(2), 135–151.

    Article  Google Scholar 

  • Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the motivated strategies questionnaire (MSLQ). Ann Arbor, MI: University of Michigan, National Center for Research to Improve Postsecondary Teaching and Learning.

    Google Scholar 

  • Park, J.-H., & Choi, H. J. (2009). Factors influencing adult learners’ decision to drop out or persist in online learning. Educational Technology & Society, 12(4), 207–217.

    Google Scholar 

  • Rienties, B., Tempelaar, D., Van den Bossche, P., Gijselaers, W., & Segers, M. (2009). The role of academic motivation in computer-supported collaborative learning. Computers in Human Behavior, 25(6), 1195–1206.

    Article  Google Scholar 

  • Roby, T., Ashe, S., Singh, N., & Clark, C. (2013). Shaping the online experience: How administrators can influence student and instructor perceptions through policy and practice. The Internet and Higher Education, 17, 29–37.

    Article  Google Scholar 

  • Rodriguez, M. C., Rooms, A., & Montañez, M. (2008). Students’ perceptions of online-learning quality given comfort, motivation, satisfaction, and experience. Journal of Interactive Online Learning, 7(2), 105–125.

    Google Scholar 

  • Romero, C., López, M. I., Luna, J. M., & Ventura, S. (2013). Predicting students’ final performance from participation in on-line discussion forums. Computers & Education, 68, 458–472. https://doi.org/10.1016/j.compedu.2013.06.009

    Article  Google Scholar 

  • Romero, C., & Ventura, S. (2007). Educational data mining: A survey from 1995 to 2005. Expert Systems with Applications, 33(1), 135–146. https://doi.org/10.1016/j.eswa.2006.04.005

    Article  Google Scholar 

  • Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67.

    Article  Google Scholar 

  • Santana, M. A., Costa, E. B., Neto, B. F. D. S., Silva, I. C. L., & Rego, J. B. (2015). A predictive model for identifying students with dropout profiles in online courses. In Proceeding of the 8th international conference on educational data mining, EDM Workshops.

    Google Scholar 

  • Schunk, D. H. (2012). Learning theories, an educational perspective (6th ed.). Boston, MA: Pearson Education Inc.

    Google Scholar 

  • Siemens, G., & Gašević, D. (2012). Special issue on learning and knowledge analytics. Educational Technology & Society, 15(3), 1–163.

    Google Scholar 

  • Sife, A., Lwoga, E., & Sanga, C. (2007). New technologies for teaching and learning: Challenges for higher learning institutions in developing countries. International Journal of Education and Development Using ICT, 3(2), 57–67.

    Google Scholar 

  • Vahdat, S., & Eidipour, M. (2016). Adopting CALL to improve listening comprehension of iranian junior high school students. Theory and Practice in Language Studies, 6(8), 1609–1617.

    Article  Google Scholar 

  • Vansteenkiste, M., Lens, W., & Deci, E. L. (2006). Intrinsic versus extrinsic goal contents in self-determination theory: Another look at the quality of academic motivation. Educational Psychologist, 41(1), 19–31.

    Article  Google Scholar 

  • Willging, P. A., & Johnson, S. D. (2009). Factors that influence students’ decision to dropout of online courses. Journal of Asynchronous Learning Networks, 13(3), 115–127.

    Google Scholar 

  • Winne, P. H., & Perry, N. E. (2000). Measuring self-regulated learning. In Handbook of self-regulation (pp. 531–566).

    Chapter  Google Scholar 

  • Wladis, C., & Samuels, J. (2016). Do online readiness surveys do what they claim? Validity, reliability, and subsequent student enrollment decisions. Computers & Education, 98, 39–56. https://doi.org/10.1016/j.compedu.2016.03.001

    Article  Google Scholar 

  • Xie, K. U. I., Debacker, T. K., & Ferguson, C. (2006). Extending the traditional classroom through online discussion: The role of student motivation. Journal of Educational Computing Research, 34(1), 67–89.

    Article  Google Scholar 

  • Xu, D., & Jaggars, S. S. (2014). Performance gaps between online and face-to-face courses: Differences across types of students and academic subject areas. The Journal of Higher Education, 85(5), 633–659.

    Article  Google Scholar 

  • You, J. W. (2016). Identifying significant indicators using LMS data to predict course achievement in online learning. The Internet and Higher Education, 29, 23–30. https://doi.org/10.1016/j.iheduc.2015.11.003

    Article  Google Scholar 

  • Yuan, J., & Kim, C. (2014). Guidelines for facilitating the development of learning communities in online courses. Journal of Computer Assisted Learning, 30(3), 220–232.

    Article  Google Scholar 

  • Zakrzewska, D. (2009). Cluster analysis in personalized e-learning systems. In N. T. Nguyen & E. Szczerbicki (Eds.), Intelligent systems for knowledge management (pp. 229–250). Berlin, Germany: Springer.

    Chapter  Google Scholar 

Download references

Acknowledgments

We would like to thank Betty Luz Zegarra Angulo of the Universidad Señor de Sipán for helping make available the data for this study as well as providing detailed information on the study context.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Danny Glick .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Glick, D., Cohen, A., Festinger, E., Xu, D., Li, Q., Warschauer, M. (2019). Predicting Success, Preventing Failure. In: Ifenthaler, D., Mah, DK., Yau, J.YK. (eds) Utilizing Learning Analytics to Support Study Success. Springer, Cham. https://doi.org/10.1007/978-3-319-64792-0_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64792-0_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64791-3

  • Online ISBN: 978-3-319-64792-0

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics