Diagrammatic Student Models: Modeling Student Drawing Performance with Deep Learning

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9146)


Recent years have seen a growing interest in the role that student drawing can play in learning. Because drawing has been shown to contribute to students’ learning and increase their engagement, developing student models to dynamically support drawing holds significant promise. To this end, we introduce diagrammatic student models, which reason about students’ drawing trajectories to generate a series of predictions about their conceptual knowledge based on their evolving sketches. The diagrammatic student modeling framework utilizes deep learning, a family of machine learning methods based on a deep neural network architecture, to reason about sequences of student drawing actions encoded with temporal and topological features. An evaluation of the deep-learning-based diagrammatic student models suggests that it can predict student performance more accurately and earlier than competitive baseline approaches.


Student modeling Intelligent tutoring systems Deep learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Van Meter, P., Garner, J.: The Promise and Practice of Learner-Generated Drawing: Literature Review and Synthesis. Educational Psychology Review. 17, 285–325 (2005)CrossRefGoogle Scholar
  2. 2.
    Schwamborn, A., Mayer, R.E., Thillmann, H., Leopold, C., Leutner, D.: Drawing as a Generative Activity and Drawing as a Prognostic Activity. Journal of Educational Psychology 102, 872–879 (2010)CrossRefGoogle Scholar
  3. 3.
    Schmeck, A., Mayer, R.E., Opfermann, M., Pfeiffer, V., Leutner, D.: Drawing Pictures During Learning from Scientific Text: Testing the Generative Drawing Effect and the Prognostic Drawing Effect. Contemporary Educational Psychology 39, 275–286 (2014)CrossRefGoogle Scholar
  4. 4.
    Verhoeven, L., Schnotz, W., Paas, F.: Cognitive Load in Interactive Knowledge Construction. Learning and Instruction 19, 369–375 (2009)CrossRefGoogle Scholar
  5. 5.
    Zhang, H., Linn, M.: Using drawings to support learning from dynamic visualizations. In: Proceedings of the 8th International Conference of the Learning Sciences, Utrecht, The Netherlands, pp. 161–162 (2008)Google Scholar
  6. 6.
    Corbett, A.T., Anderson, J.R.: Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge. User Modelling and User-Adapted Interaction 4, 253–278 (1994)CrossRefGoogle Scholar
  7. 7.
    Gong, Y., Beck, J.E.: Looking beyond transfer models: finding other sources of power for student models. In: Konstan, J.A., Conejo, R., Marzo, J.L., Oliver, N. (eds.) UMAP 2011. LNCS, vol. 6787, pp. 135–146. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  8. 8.
    Sabourin, J., Mott, B., Lester, J.: Utilizing dynamic bayes nets to improve early prediction models of self-regulated learning. In: Carberry, S., Weibelzahl, S., Micarelli, A., Semeraro, G. (eds.) UMAP 2013. LNCS, vol. 7899, pp. 228–241. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  9. 9.
    Chi, M., VanLehn, K., Litman, D., Jordan, P.: Inducing effective pedagogical strategies using learning context features. In: De Bra, P., Kobsa, A., Chin, D. (eds.) UMAP 2010. LNCS, vol. 6075, pp. 147–158. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Sao Pedro, M.A., De Baker, R.S.J., Gobert, J.D., Montalvo, O., Nakama, A.: Leveraging Machine-learned Detectors of Systematic Inquiry Behavior to Estimate and Predict Transfer of Inquiry Skill. User Modelling and User-Adapted Interaction 23, 1–39 (2013)CrossRefGoogle Scholar
  11. 11.
    Muldner, K., Burleson, W., Van De Sande, B., Vanlehn, K.: An Analysis of Students’ Gaming Behaviors in an Intelligent Tutoring System: Predictors and Impacts. User Modelling and User-Adapted Interaction 21, 99–135 (2011)CrossRefGoogle Scholar
  12. 12.
    Armentano, M.G., Amandi, A.A.: Modeling Sequences of User Actions for Statistical Goal Recognition. User Modelling and User-Adapted Interaction 22, 281–311 (2012)CrossRefGoogle Scholar
  13. 13.
    Min, W., Ha, E.Y., Rowe, J.P., Mott, B.W., Lester, J.C.: Deep learning-based goal recognition in open-ended digital games. In: Tenth Artificial Intelligence and Interactive Digital Entertainment Conference, Raleigh, NC, pp. 37–43 (2014)Google Scholar
  14. 14.
    Valentine, S., Vides, F., Lucchese, G., Turner, D., Kim, H., Li, W., Linsey, J., Hammond, T.: Mechanix: a sketch-based tutoring system for statics courses. In: Proceedings of the Twenty-Fourth Conference on Innovative Applications of Artificial Intelligence, Toronto, Ontario, Canada (2012)Google Scholar
  15. 15.
    van Joolingen, W.R., Bollen, L., Leenaars, F.A.: Using drawings in knowledge modeling and simulation for science teaching. In: Nkambou, R., Bourdeau, J., Mizoguchi, R. (eds.) Advances in Intelligent Tutoring Systems. SCI, vol. 308, pp. 249–264. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  16. 16.
    Jee, B., Gentner, D.: Drawing on experience: use of sketching to evaluate knowledge of spatial scientific concepts. In: Proceedings of the 31st Annual Conference of the Cognitive Science Society, Amsterdam, The Netherlands (2009)Google Scholar
  17. 17.
    Ruiz‐Primo, M.a., Li, M., Ayala, C., Shavelson, R.J.: Evaluating Students’ Science Notebooks as an Assessment Tool. International Journal of Science Education 26, 1477–1506 (2004)CrossRefGoogle Scholar
  18. 18.
    Wiebe, E.N., Madden, L.P., Bedward, J.C., Carter, M.: Examining Science Inquiry Practices in the Elementary Classroom through Science Notebooks. Presented at NARST Annual Meeting, Garden Grove (2009)Google Scholar
  19. 19.
    Smith, A., Wiebe, E., Mott, B., Lester, J.: SketchMiner: mining learner-generated science drawings with topological abstraction. In: Proceedings of the Seventh International Conference on Educational Data Mining, London, UK, pp. 288–291 (2014)Google Scholar
  20. 20.
    Bengio, Y., Lamblin, P.: Greedy Layer-wise Training of Deep Networks. Advances in neural information processing systems 19, 153 (2007)Google Scholar
  21. 21.
    Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105 (2012)Google Scholar
  22. 22.
    Socher, R., Pennington, J., Huang, E.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Linguistics, A. for C. (ed.) Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 151–161 (2011)Google Scholar
  23. 23.
    Srivastava, N., Hinton, G.: Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15, 1929–1958 (2014)zbMATHMathSciNetGoogle Scholar
  24. 24.
    Palm, R.: Prediction as a candidate for learning deep hierarchical models of data. Technical University of Denmark. (2012)Google Scholar
  25. 25.
    Hsu, C., Chang, C., Lin, C.: A Practical Guide to Support Vector Classification. 1, 1–16 (2003)Google Scholar
  26. 26.
    Blaylock, N., Allen, J.: Hierarchical Goal Recognition. Plan, Activity, and Intent Recognition Theory and Practice, pp. 3–31 (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Center for Educational InformaticsNorth Carolina State UniversityRaleighUSA

Personalised recommendations