Advertisement

Predictive analytics in education: a comparison of deep learning frameworks

  • Tenzin DoleckEmail author
  • David John Lemay
  • Ram B. Basnet
  • Paul Bazelais
Article
  • 47 Downloads

Abstract

Large swaths of data are readily available in various fields, and education is no exception. In tandem, the impetus to derive meaningful insights from data gains urgency. Recent advances in deep learning, particularly in the area of voice and image recognition and so-called complete knowledge games like chess, go, and StarCraft, have resulted in a flurry of research. Using two educational datasets, we explore the utility and applicability of deep learning for educational data mining and learning analytics. We compare the predictive accuracy of popular deep learning frameworks/libraries, including, Keras, Theano, Tensorflow, fast.ai, and Pytorch. Experimental results reveal that performance, as assessed by predictive accuracy, varies depending on the optimizer used. Further, findings from additional experiments by tuning network parameters yield similar results. Moreover, we find that deep learning displays comparable performance to other machine learning algorithms such as support vector machines, k-nearest neighbors, naive Bayes classifier, and logistic regression. We argue that statistical learning techniques should be selected to maximize interpretability and should contribute to our understanding of educational and learning phenomena; hence, in most cases, educational data mining and learning analytics researchers should aim for explanation over prediction.

Keywords

Machine learning Deep learning Educational data mining Learning analytics Classification Predictive analytics 

Notes

References

  1. Aldowah, H., Al-Samarraie, H., & Fauzy, W. (2019). Educational data mining and learning analytics for 21st century higher education: A review and synthesis. Telematics and Informatics, 37, 13–49.  https://doi.org/10.1016/j.tele.2019.01.007.CrossRefGoogle Scholar
  2. Avella, J., Kebritchi, M., Nunn, S., & Kanai, T. (2016). Learning analytics methods, benefits, and challenges in higher education: A systematic literature review. Online Learning, 20(2), 13–29.Google Scholar
  3. Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In Learning analytics (pp. 61–75). New York, NY: Springer.Google Scholar
  4. Batmaz, Z., Yurekli, A., Bilge, A., & Kaleli, C. (2018). A review on deep learning for recommender systems: Challenges and remedies. Artificial Intelligence Review, 52(1), 1–37.  https://doi.org/10.1007/s10462-018-9654-y.CrossRefGoogle Scholar
  5. Bazelais, P., Lemay, D. J., Doleck, T., Hu, X. S., Vu, A., & Yao, J. (2018). Grit, mindset, and academic performance: A study of pre-University science students. Eurasia Journal of Mathematics, Science and Technology Education, 14(12), 1–10.  https://doi.org/10.29333/ejmste/94570.
  6. Botelho, A. F., Baker, R. S., & Heffernan, N. T. (2017). Improving sensor-free affect detection using deep learning. In E. André, R. Baker, X. Hu, M. M. T. Rodrigo, & B. Boulay (Eds.), Proceedings of the 18th international conference on artificial intelligence in education (pp. 40–51). London, UK: Springer International Publishing.CrossRefGoogle Scholar
  7. Brinton, C. G., & Chiang, M. (2015). MOOC performance prediction via clickstream data and social learning networks. IEEE Conference on Computer Communications (INFOCOM), 2299–2307.Google Scholar
  8. Costa, E., Fonseca, B., Santana, M., de Araújo, F., & Rego, J. (2017). Evaluating the effectiveness of educational data mining techniques for early prediction of students’ academic failure in introductory programming courses. Computers in Human Behavior, 73, 247–256.  https://doi.org/10.1016/j.chb.2017.01.047.CrossRefGoogle Scholar
  9. Deng, L., & Yu, D. (2014). Deep learning: Methods and applications. Foundations And Trends In Signal Processing, 7(3–4), 197–387.  https://doi.org/10.1561/2000000039.MathSciNetCrossRefzbMATHGoogle Scholar
  10. Doleck, T., Basnet, R. B., Poitras, E. G., & Lajoie, S. P. (2015). Mining learner-system interaction data: Implications for modeling learner behaviors and improving overlay models. Journal of Computers in Education, 2(4), 421–447.  https://doi.org/10.1007/s40692-015-0040-3.CrossRefGoogle Scholar
  11. Doleck, T., Jarrell, A., Poitras, E. G., Chaouachi, M., & Lajoie, S. P. (2016). A tale of three cases: Examining accuracy, efficiency, and process differences in diagnosing virtual patient cases. Australasian Journal of Educational Technology, 36(5), 61–76.  https://doi.org/10.14742/ajet.2759.
  12. Doleck, T., Poitras, E., & Lajoie, S. (2019). Assessing the utility of deep learning: Using learner-system interaction data from BioWorld. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + innovate learning (pp. 734–738). Amsterdam, Netherlands: AACE.Google Scholar
  13. Fast.ai. (2019). Retrieved from https://www.fast.ai/
  14. Home-Keras Documentation. (2019). Retrieved from https://keras.io/
  15. Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P. (2019). Deep learning for time series classification: A review. Data Mining and Knowledge Discovery, 33(4), 917–963.  https://doi.org/10.1007/s10618-019-00619-1.MathSciNetCrossRefGoogle Scholar
  16. Jiang, Y., Bosch, N., Baker, R., Paquette, L., Ocumpaugh, J., Andres, J. M. A. L., Moore, A. L., & Biswas, G. (2018). Expert feature-engineering vs. deep neural networks: Which is better for sensor-free affect detection? In Proceedings of the 19th international conference on artificial intelligence in education (pp. 198–211). London, UK: Springer.CrossRefGoogle Scholar
  17. Jordan, M., & Mitchell, T. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260.  https://doi.org/10.1126/science.aaa8415.MathSciNetCrossRefzbMATHGoogle Scholar
  18. Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. In I. Maglogiannis et al. (Eds.), Emerging artificial intelligence applications in computer engineering (pp. 3–24). Amsterdam, Netherlands: IOS Press.Google Scholar
  19. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.  https://doi.org/10.1038/nature14539.CrossRefGoogle Scholar
  20. Lemay, D. J., & Doleck, T. (2019). Grade prediction of weekly assignments in MOOCs: Mining Video-Viewing Behavior. Education and Information Technologies.  https://doi.org/10.1007/s10639-019-10022-4.
  21. Lykourentzou, I., Giannoukos, I., Nikolopoulos, V., Mpardis, G., & Loumos, V. (2009). Dropout prediction in e-learning courses through the combination of machine learning techniques. Computers & Education, 53(3), 950–965.  https://doi.org/10.1016/j.compedu.2009.05.010.CrossRefGoogle Scholar
  22. Mao, Y., Lin, C., & Chi, M. (2018). Deep learning vs. Bayesian knowledge tracing: Student models for interventions. JEDM | Journal of Educational Data Mining, 10(2), 28–54.Google Scholar
  23. Marcus, G. (2018). Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631.Google Scholar
  24. Nguyen, G., Dlugolinsky, S., Bobák, M., Tran, V., López García, Á., Heredia, I., et al. (2019). Machine learning and deep learning frameworks and libraries for large-scale data mining: A survey. Artificial Intelligence Review, 52(1), 77–124.  https://doi.org/10.1007/s10462-018-09679-z.CrossRefGoogle Scholar
  25. Pang, B., Nijkamp, E., & Wu, Y. (2019). Deep learning with TensorFlow: A Review. Journal of Educational and Behavioral Statistics.  https://doi.org/10.3102/1076998619872761.
  26. Papamitsiou, Z., & Economides, A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64.Google Scholar
  27. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.MathSciNetzbMATHGoogle Scholar
  28. Peña-Ayala, A. (2014). Educational data mining: A survey and a data mining-based analysis of recent works. Expert Systems with Applications, 41(4), 1432–1462.  https://doi.org/10.1016/j.eswa.2013.08.042.CrossRefGoogle Scholar
  29. Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L. J., & Sohl-Dickstein, J. (2015). Deep knowledge tracing. In Advances in Neural Information Processing Systems (pp. 505–513).Google Scholar
  30. Poitras, E. G., Lajoie, S. P., Doleck, T., & Jarrell, A. (2016). Subgroup discovery with user interaction data: An empirically guided approach to improving intelligent tutoring systems. Educational Technology & Society, 19(2), 204–214.Google Scholar
  31. PyTorch. (2019). Retrieved from https://pytorch.org/
  32. Rajni, J., & Malaya, D. (2015). Predictive analytics in a higher education context. IT Professional, 17(4), 24–33.  https://doi.org/10.1109/mitp.2015.68.CrossRefGoogle Scholar
  33. Romero, C., & Ventura, S. (2016). Educational data science in massive open online courses. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 7(1), e1187.  https://doi.org/10.1002/widm.1187.Google Scholar
  34. Siemens, G., & Baker, R. S. (2012). Learning analytics and educational data mining: Towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). ACM.Google Scholar
  35. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), 1929–1958.MathSciNetzbMATHGoogle Scholar
  36. Sünderhauf, N., Brock, O., Scheirer, W., Hadsell, R., Fox, D., Leitner, J., et al. (2018). The limits and potentials of deep learning for robotics. The International Journal of Robotics Research, 37(4–5), 405–420.  https://doi.org/10.1177/0278364918770733.CrossRefGoogle Scholar
  37. TensorFlow. (2019). Retrieved from https://www.tensorflow.org/
  38. Theano 1.0.0 documentation. (2019). Retrieved from http://deeplearning.net/software/theano/
  39. Wilson, K. H., Karklin, Y., Han, B., Ekanadham, C. (2016). Back to the basics: Bayesian extensions of IRT outperform neural networks for proficiency estimation. In Proceedings of Educational Data Mining (pp. 539–544).Google Scholar
  40. Xiao, C., Choi, E., & Sun, J. (2018). Opportunities and challenges in developing deep learning models using electronic health records data: A systematic review. Journal of the American Medical Informatics Association, 25(10), 1419–1428.  https://doi.org/10.1093/jamia/ocy068.CrossRefGoogle Scholar
  41. Xiong, X., Zhao, S., Van Inwegen, E. G., & Beck, J. E. (2016). Going deeper with deep knowledge tracing. In Proceedings of 9th International Conference on Educational Data Mining (pp. 545–550).Google Scholar
  42. Zhang, Q., Yang, L., Chen, Z., & Li, P. (2018). A survey on deep learning for big data. Information Fusion, 42, 146–157.  https://doi.org/10.1016/j.inffus.2017.10.006.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Tenzin Doleck
    • 1
    Email author
  • David John Lemay
    • 2
  • Ram B. Basnet
    • 3
  • Paul Bazelais
    • 2
  1. 1.University of Southern CaliforniaLos AngelesUSA
  2. 2.McGill UniversityMontrealCanada
  3. 3.Colorado Mesa UniversityGrand JunctionUSA

Personalised recommendations