Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Beyond engagement analytics: which online mixed-data factors predict student learning outcomes?

Abstract

This mixed-method study focuses on online learning analytics, a research area of importance. Several important student attributes and their online activities are examined to identify what seems to work best to predict higher grades. The purpose is to explore the relationships between student grade and key learning engagement factors using a large sample from an online undergraduate business course at an accredited American university (n = 228). Recent studies have discounted the ability to predict student learning outcomes from big data analytics but a few significant indicators have been found by some researchers. Current studies tend to use quantitative factors in learning analytics to forecast outcomes. This study extends that work by testing the common quantitative predictors of learning outcome, but qualitative data is also examined to triangulate the evidence. Pre and post testing of information technology understanding is done at the beginning of the course. First quantitative data is collected, and depending on the hypothesis test results, qualitative data is collected and analyzed with text analytics to uncover patterns. Moodle engagement analytics indicators are tested as predictors in the model. Data is also taken from the Moodle system logs. Qualitative data is collected from student reflection essays. The result was a significant General Linear Model with four online interaction predictors that captured 77.5 % of grade variance in an undergraduate business course.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

References

  1. Agudo-Peregrina, Ã. F., Iglesias-Pradas, S., Conde-Gonzalez, M. Ã., & Hernandez-Garcia, Ã. (2014). Can we predict success from log data in vles? Classification of interactions for learning analytics and their relation with performance in vle-supported f2f and online learning. Computers in Human Behavior, 31(1), 542–550.

  2. Beattie, S., Woodley, C., & Souter, K. (2014). Creepy analytics and learner data rights. In B. Hegarty, J. McDonald & S.-K. Loke (Eds.), Rhetoric and reality: Critical perspectives on educational techology - conference proceedings (pp. 422–425). Dunedin, NZ: ASCILITE.

  3. Carlson, W. L., Thorne, B., & Krehbiel, T. C. (2004). Statistical business and economics. Upper Saddle River, NJ: Prentice-Hall.

  4. Chatti, M. A., Dyckhoff, A. L., Schroeder, U., & Thüs, H. (2012). A reference model for learning analytics. International Journal of Technology Enhanced Learning, 4(5/6), 310–331.

  5. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates

  6. Dougiamas, M., & Taylor, P. C. (2003). Moodle: Using learning communities to create an open source course management system. Paper presented at the Proceedings of the EDMEDIA 2003. In Conference. HA: Honolulu.

  7. Fidalgo-Blanco, Ã., Sein-Echaluce, M. L., Garcia-Peealvo, F. J., & Conde, M. Ã. (2015). Using learning analytics to improve teamwork assessment. Computers in Human Behavior, 47(2), 149–156.

  8. Gomez-Aguilar, D. A., Hernandez-Garcia, Ã., Garcia-Pealvo, F. J., & Theren, R. (2015). Tap into visual analysis of customization of grouping of activities in elearning. Computers in Human Behavior, 47(2), 60–67.

  9. Gunn, C. (2014). Defining an agenda for learning analytics. In B. Hegarty, J. McDonald, & S.-K. Loke (Eds.), Rhetoric and reality: critical perspectives on educational techology - conference proceedings (pp. 638–637). Dunedin, NZ: ASCILITE.

  10. Iglesias-Pradas, S., Ruiz-de-Azcarate, C., & Agudo-Peregrina, Ã. F. (2015). Assessing the suitability of student interactions from Moodle data logs as predictors of cross-curricular competencies. Computers in Human Behavior, 47(2), 81–89.

  11. Nersesian, R., & Strang, K. D. (2013). Risk planning with discrete distribution analysis applied to petroleum spills. International Journal of Risk and Contingency Management, 2(4), 61–78.

  12. Nieto-Acevedo, Y., Vanessa, M. M., & Enrique, C. (2015). Towards a decision support system based on learning analytics. Advances in Information Sciences & Service Sciences, 7(1), 1–12.

  13. Reyes, J. (2015). The skinny on big data in education: learning analytics simplified. TechTrends: Linking Research & Practice to Improve Learning, 59(2), 75–80.

  14. Ruiparez-Valiente, J. A., Mua-Merino, P. J., Leony, D., & Delgado Kloos, C. (2015). Alas-ka: a learning analytics extension for better understanding the learning process in the khan academy platform. Computers in Human Behavior, 47(2), 139–148.

  15. Scheffel, M., Drachsler, H., Stoyanov, S., & Specht, M. (2014). Quality indicators for learning analytics. Journal of Educational Technology & Society, 17(4), 117–132.

  16. Shum, S. B. (2012). Learning analytics [policy brief]. Moscow, RU: united nations educational, Scientific and Cultural Organization (UNESCO) Instittute for Information Technologies in Education. Retrieved from http://iite.unesco.org/pics/publications/en/files/3214711.pdf

  17. Snee, R. D. (1973). Some aspects of nonorthogonal data analysis, part 1. Developing prediction equations. Journal of Quality Technology, 5(1), 67–79.

  18. Strang, K. D. (2012). Applied financial nonlinear programming models for decision making. International Journal of Applied Decision Sciences, 5(4), 370–395. Retrieved from http://www.inderscience.com/info/inarticletoc.php?jcode=ijads&year=2012&vol=5&issue=4

  19. Strang, K. D. (2015). Selecting Research techniques for a Method and Strategy. In K. D. Strang (Ed.), Palgrave Handbook of Research Design in Business and Management (ch. 5, pp. 63–80). New York: Palgrave Macmillan. ISBN: 978–1137379924.

  20. Strang, K. D., & Sun, Z. (2015). Analyzing relationships in terrorism big data using Hadoop and statistics. The Journal of Computer Information Systems, 55(4), 55–72. Retrieved from http://www.iacis.org/jcis/forthcoming.php

  21. Sun, Z., Strang, K. D., & Yearwood, J. (2014). Analytics service oriented architecture for enterprise information systems. In I. Khalil & A. M. Tjoa (Eds.), ACM International Proceedings of 8th International Conference on Research and Practical Issues of Enterprise Information Systems (CONFENIS 2014) (Vol. 1, pp. 10–19). Hanoi, Vietnam: IFIP TC8 Working Group 8.9. ISBN: 978–1-4503-3001-5. Retrieved from http://www.iiwas.org/conferences/confenis2014

  22. Tamhane, A. C., & Dunlop, D. D. (2000). Statistics and data analysis from elementary to intermediate. Upper Saddle River, NJ: Prentice-Hall.

  23. Vajjhala, N. R., Strang, K. D., & Sun, Z. (2015). Statistical modeling and visualizing of open big data using a terrorism case study. Paper presented at the Open Big Data Conference, Rome, Italy. Retrieved from http://www.ficloud.org/obd2015/

  24. Xing, W., Guo, R., Petakovic, E., & Goggins, S. (2015). Participation-based student final performance prediction model through interpretable genetic programming: integrating learning analytics, educational data mining and theory. Computers in Human Behavior, 47(2), 168–181.

  25. Yahya, A.-A., Messoussi, R., & Touahni, R. (2015). Analytical tools for visualisation of interactions in online e-learning activities on lms and semantic similarity measures on text. Journal of Theoretical & Applied Information Technology, 73(1), 102–118.

  26. Zacharis, N. Z. (2015). A multivariate approach to predicting student outcomes in web-enabled blended learning courses. The Internet and Higher Education, 27(1), 44–53.

Download references

Author information

Correspondence to Kenneth David Strang.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Strang, K.D. Beyond engagement analytics: which online mixed-data factors predict student learning outcomes?. Educ Inf Technol 22, 917–937 (2017). https://doi.org/10.1007/s10639-016-9464-2

Download citation

Keywords

  • ICT in higher education
  • Moodle engagement analytics
  • Big data analytics
  • Online undergraduate business course
  • Student learning performance
  • Grades