Predictive analytics in education: a comparison of deep learning frameworks
- 47 Downloads
Large swaths of data are readily available in various fields, and education is no exception. In tandem, the impetus to derive meaningful insights from data gains urgency. Recent advances in deep learning, particularly in the area of voice and image recognition and so-called complete knowledge games like chess, go, and StarCraft, have resulted in a flurry of research. Using two educational datasets, we explore the utility and applicability of deep learning for educational data mining and learning analytics. We compare the predictive accuracy of popular deep learning frameworks/libraries, including, Keras, Theano, Tensorflow, fast.ai, and Pytorch. Experimental results reveal that performance, as assessed by predictive accuracy, varies depending on the optimizer used. Further, findings from additional experiments by tuning network parameters yield similar results. Moreover, we find that deep learning displays comparable performance to other machine learning algorithms such as support vector machines, k-nearest neighbors, naive Bayes classifier, and logistic regression. We argue that statistical learning techniques should be selected to maximize interpretability and should contribute to our understanding of educational and learning phenomena; hence, in most cases, educational data mining and learning analytics researchers should aim for explanation over prediction.
KeywordsMachine learning Deep learning Educational data mining Learning analytics Classification Predictive analytics
- Avella, J., Kebritchi, M., Nunn, S., & Kanai, T. (2016). Learning analytics methods, benefits, and challenges in higher education: A systematic literature review. Online Learning, 20(2), 13–29.Google Scholar
- Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In Learning analytics (pp. 61–75). New York, NY: Springer.Google Scholar
- Bazelais, P., Lemay, D. J., Doleck, T., Hu, X. S., Vu, A., & Yao, J. (2018). Grit, mindset, and academic performance: A study of pre-University science students. Eurasia Journal of Mathematics, Science and Technology Education, 14(12), 1–10. https://doi.org/10.29333/ejmste/94570.
- Botelho, A. F., Baker, R. S., & Heffernan, N. T. (2017). Improving sensor-free affect detection using deep learning. In E. André, R. Baker, X. Hu, M. M. T. Rodrigo, & B. Boulay (Eds.), Proceedings of the 18th international conference on artificial intelligence in education (pp. 40–51). London, UK: Springer International Publishing.CrossRefGoogle Scholar
- Brinton, C. G., & Chiang, M. (2015). MOOC performance prediction via clickstream data and social learning networks. IEEE Conference on Computer Communications (INFOCOM), 2299–2307.Google Scholar
- Costa, E., Fonseca, B., Santana, M., de Araújo, F., & Rego, J. (2017). Evaluating the effectiveness of educational data mining techniques for early prediction of students’ academic failure in introductory programming courses. Computers in Human Behavior, 73, 247–256. https://doi.org/10.1016/j.chb.2017.01.047.CrossRefGoogle Scholar
- Doleck, T., Jarrell, A., Poitras, E. G., Chaouachi, M., & Lajoie, S. P. (2016). A tale of three cases: Examining accuracy, efficiency, and process differences in diagnosing virtual patient cases. Australasian Journal of Educational Technology, 36(5), 61–76. https://doi.org/10.14742/ajet.2759.
- Doleck, T., Poitras, E., & Lajoie, S. (2019). Assessing the utility of deep learning: Using learner-system interaction data from BioWorld. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + innovate learning (pp. 734–738). Amsterdam, Netherlands: AACE.Google Scholar
- Fast.ai. (2019). Retrieved from https://www.fast.ai/
- Home-Keras Documentation. (2019). Retrieved from https://keras.io/
- Jiang, Y., Bosch, N., Baker, R., Paquette, L., Ocumpaugh, J., Andres, J. M. A. L., Moore, A. L., & Biswas, G. (2018). Expert feature-engineering vs. deep neural networks: Which is better for sensor-free affect detection? In Proceedings of the 19th international conference on artificial intelligence in education (pp. 198–211). London, UK: Springer.CrossRefGoogle Scholar
- Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. In I. Maglogiannis et al. (Eds.), Emerging artificial intelligence applications in computer engineering (pp. 3–24). Amsterdam, Netherlands: IOS Press.Google Scholar
- Lemay, D. J., & Doleck, T. (2019). Grade prediction of weekly assignments in MOOCs: Mining Video-Viewing Behavior. Education and Information Technologies. https://doi.org/10.1007/s10639-019-10022-4.
- Mao, Y., Lin, C., & Chi, M. (2018). Deep learning vs. Bayesian knowledge tracing: Student models for interventions. JEDM | Journal of Educational Data Mining, 10(2), 28–54.Google Scholar
- Marcus, G. (2018). Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631.Google Scholar
- Nguyen, G., Dlugolinsky, S., Bobák, M., Tran, V., López García, Á., Heredia, I., et al. (2019). Machine learning and deep learning frameworks and libraries for large-scale data mining: A survey. Artificial Intelligence Review, 52(1), 77–124. https://doi.org/10.1007/s10462-018-09679-z.CrossRefGoogle Scholar
- Pang, B., Nijkamp, E., & Wu, Y. (2019). Deep learning with TensorFlow: A Review. Journal of Educational and Behavioral Statistics. https://doi.org/10.3102/1076998619872761.
- Papamitsiou, Z., & Economides, A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64.Google Scholar
- Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L. J., & Sohl-Dickstein, J. (2015). Deep knowledge tracing. In Advances in Neural Information Processing Systems (pp. 505–513).Google Scholar
- Poitras, E. G., Lajoie, S. P., Doleck, T., & Jarrell, A. (2016). Subgroup discovery with user interaction data: An empirically guided approach to improving intelligent tutoring systems. Educational Technology & Society, 19(2), 204–214.Google Scholar
- PyTorch. (2019). Retrieved from https://pytorch.org/
- Siemens, G., & Baker, R. S. (2012). Learning analytics and educational data mining: Towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). ACM.Google Scholar
- TensorFlow. (2019). Retrieved from https://www.tensorflow.org/
- Theano 1.0.0 documentation. (2019). Retrieved from http://deeplearning.net/software/theano/
- Wilson, K. H., Karklin, Y., Han, B., Ekanadham, C. (2016). Back to the basics: Bayesian extensions of IRT outperform neural networks for proficiency estimation. In Proceedings of Educational Data Mining (pp. 539–544).Google Scholar
- Xiong, X., Zhao, S., Van Inwegen, E. G., & Beck, J. E. (2016). Going deeper with deep knowledge tracing. In Proceedings of 9th International Conference on Educational Data Mining (pp. 545–550).Google Scholar