Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques

Article
  • 168 Downloads

Abstract

Learner modeling is a basis of personalized, adaptive learning. The research literature provides a wide range of modeling approaches, but it does not provide guidance for choosing a model suitable for a particular situation. We provide a systematic and up-to-date overview of current approaches to tracing learners’ knowledge and skill across interaction with multiple items, focusing in particular on the widely used Bayesian knowledge tracing and logistic models. We discuss factors that influence the choice of a model and highlight the importance of the learner modeling context: models are used for different purposes and deal with different types of learning processes. We also consider methodological issues in the evaluation of learner models and their relation to the modeling context. Overall, the overview provides basic guidelines for both researchers and practitioners and identifies areas that require further clarification in future research.

Keywords

Learner modeling Skill modeling Overview Evaluation Methodology Knowledge-learning-instruction framework 

References

  1. Aleven, V., Koedinger, K.R.: Knowledge component (kC) approaches to learner modeling. In: Sottilare, R.A., Graesser, A., Hu, X., Holden, H. (eds) Design Recommendations for Intelligent Tutoring Systems, vol. 1, pp. 165–182. US Army Research Laboratory, Orlando, FL (2013)Google Scholar
  2. Aleven, V., Sewall, J.: The frequency of tutor behaviors: a case study. In: Proceedings of Intelligent Tutoring Systems, pp. 396–401. Springer (2016)Google Scholar
  3. Aleven, V., McLaren, B.M., Sewall, J., Koedinger, K.R.: The cognitive tutor authoring tools (CTAT): preliminary evaluation of efficiency gains. In: International Conference on Intelligent Tutoring Systems, pp. 61–70. Springer (2006)Google Scholar
  4. Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167–207 (1995)CrossRefGoogle Scholar
  5. Arroyo, I., Mehranian, H., Woolf, B.P.: Effort-based tutoring: an empirical approach to intelligent tutoring. In: Proceedings of Educational Data Mining (2010)Google Scholar
  6. Ayers, E., Junker, B.: Do skills combine additively to predict task difficulty in eighth grade mathematics. In: Proceedings of Educational Data Mining: Papers from the AAAI Workshop (2006)Google Scholar
  7. Baker, R.S.: Stupid tutoring systems, intelligent humans. Int. J. Artif. Intell. Educ. 26(2), 600–614 (2016)CrossRefGoogle Scholar
  8. Baker, R.S., Yacef, K.: The state of educational data mining in 2009: a review and future visions. J. Educ. Data Min. 1(1), 3–17 (2009)Google Scholar
  9. Baker, R.S., Corbett, A.T., Aleven, V.: More accurate student modeling through contextual estimation of slip and guess probabilities in Bayesian knowledge tracing. In: Proceedings of Intelligent Tutoring Systems, pp. 406–415. Springer (2008)Google Scholar
  10. Barnes, T.: The q-matrix method: mining student response data for knowledge. In: American Association for Artificial Intelligence 2005 Educational Data Mining Workshop, pp. 1–8 (2005)Google Scholar
  11. Beck, J.E., Chang, K.: Identifiability: a fundamental problem of student modeling. In: Conati, C., McCoy, K., Paliouras, G. (eds.) User Modeling 2007: 11th International Conference, UM 2007, Corfu, Greece, July 25–29, 2007. Proceedings, pp. 137–146. Springer, Berlin (2007)Google Scholar
  12. Beck, J., Xiong, X.: Limits to accuracy: how well can we do at student modeling? In: Proceedings of Educational Data Mining (2013a)Google Scholar
  13. Beck, J.E., Gong, Y.: Wheel-spinning: students who fail to master a skill. In: Proceedings of Artificial Intelligence in Education, pp. 431–440. Springer (2013b)Google Scholar
  14. Beck, J.E., Chang, K.M., Mostow, J., Corbett, A.: Does help help? Introducing the Bayesian evaluation and assessment methodology. In: Proceedings of Intelligent Tutoring Systems, pp. 383–394. Springer (2008a)Google Scholar
  15. Beck, J.E., Pardos, Z.A., Heffernan, N.T., Ruiz, C.: The composition effect: conjunctive or compensatory? An analysis of multi-skill math questions in its. In: Proceedings of Educational Data Mining (2008b)Google Scholar
  16. Beel, J., Breitinger, C., Langer, S., Lommatzsch, A., Gipp, B.: Towards reproducibility in recommender-systems research. User Model. User Adapt. Interact. 26(1), 69–101 (2016)CrossRefGoogle Scholar
  17. Boroš, P., Nižnan, J., Pelánek, R., Řihák, J.: Automatic detection of concepts from problem solving times. In: Proceedings of Artificial Intelligence in Education, LNCS, vol. 7926, pp. 595–598. Springer (2013)Google Scholar
  18. Bull, S., Kay, J.: Student models that invite the learner in: the smili:() open learner modelling framework. Int. J. Artif. Intell. Educ. 17(2), 89–120 (2007)Google Scholar
  19. Bull, S., Kay, J.: Open learner models. In: Nkambou, R., Bourdeau, J., Mizoguchi, R. (eds.) Advances in Intelligent Tutoring Systems, pp. 301–322. Springer, Berlin (2010)Google Scholar
  20. Bull, S., Ginon, B., Boscolo, C., Johnson, M.: Introduction of learning visualisations and metacognitive support in a persuadable open learner model. In: Proceedings of Learning Analytics & Knowledge, pp. 30–39. ACM (2016)Google Scholar
  21. Carlson, R., Genin, K., Rau, M., Scheines, R.: Student profiling from tutoring system log data: when do multiple graphical representations matter? In: Proceedings of Educational Data Mining (2013)Google Scholar
  22. Carmona, C., Millán, E., Pérez-de-la Cruz, J.L., Trella, M., Conejo, R.: Introducing prerequisite relations in a multi-layered Bayesian student model. In: User Modeling, pp. 347–356. Springer (2005)Google Scholar
  23. Cen, H., Koedinger, K., Junker, B.: Learning factors analysis—a general method for cognitive model evaluation and improvement. In: Proceedings of Intelligent Tutoring Systems, pp. 164–175. Springer (2006)Google Scholar
  24. Cen, H., Koedinger, K.R., Junker, B.: Is over practice necessary? Improving learning efficiency with the cognitive tutor through educational data mining. Front. Artif. Intell. Appl. 158, 511 (2007)Google Scholar
  25. Cen, H., Koedinger, K., Junker, B.: Comparing two IRT models for conjunctive skills. In: Proceedings of Intelligent Tutoring Systems, pp. 796–798. Springer (2008)Google Scholar
  26. Chen, Y., Wuillemin, P.H., Labat, J.M.: Discovering prerequisite structure of skills through probabilistic association rules mining. In: Proceedings of Educational Data Mining, pp. 117–124 (2015)Google Scholar
  27. Chen, Y., Gonzlez-Brenes, J., Tian, J.: Joint discovery of skill prerequisite graphs and student models. In: Proceedings of Educational Data Mining (2016)Google Scholar
  28. Chi, M., Koedinger, K., Gordon, G., Jordan, P., Van Lehn, K.: Instructional factors analysis. In: Proceedings of Educational Data Mining (2011)Google Scholar
  29. Chrysafiadi, K., Virvou, M.: Student modeling approaches: a literature review for the last decade. Expert Syst. Appl. 40(11), 4715–4729 (2013)CrossRefGoogle Scholar
  30. Conati, C., Gertner, A., Vanlehn, K.: Using Bayesian networks to manage uncertainty in student modeling. User Model. User Adapt. Interact. 12(4), 371–417 (2002)MATHCrossRefGoogle Scholar
  31. Corbett, A.T., Anderson, J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. User Model. User Adapt. Interact. 4(4), 253–278 (1994)CrossRefGoogle Scholar
  32. David, Y.B., Segal, A., Gal, Y.K.: Sequencing educational content in classrooms using Bayesian knowledge tracing. In: Proceedings of Learning Analytics and Knowledge, pp. 354–363. ACM (2016)Google Scholar
  33. De La Torre, J.: Dina model and parameter estimation: a didactic. J. Educ. Behav. Stat. 34(1), 115–130 (2009)CrossRefGoogle Scholar
  34. Desmarais, M.: Conditions for effectively deriving a q-matrix from data with non-negative matrix factorization. In: Proceedings of Educational Data Mining (2011)Google Scholar
  35. Desmarais, M.C., Baker, R.S.: A review of recent advances in learner and skill modeling in intelligent learning environments. User Model. User Adapt. Interact. 22(1–2), 9–38 (2012)CrossRefGoogle Scholar
  36. Desmarais, M., Lemieux, F.: Clustering and visualizing study state sequences. In: Proceedings of Educational Data Mining (2013a)Google Scholar
  37. Desmarais, M.C., Naceur, R.: A matrix factorization method for mapping items to skills and for enhancing expert-based q-matrices. In: Proceedings of Artificial Intelligence in Education, pp. 441–450. Springer (2013b)Google Scholar
  38. Desmarais, M.C., Meshkinfam, P., Gagnon, M.: Learned student models with item to item knowledge structures. User Model. User Adapt. Interact. 16(5), 403–434 (2006)CrossRefGoogle Scholar
  39. Desmarais, M., Beheshti, B., Xu, P.: The refinement of a q-matrix: assessing methods to validate tasks to skills mapping. In: Proceedings of Educational Data Mining (2014)Google Scholar
  40. Doignon, J.P., Falmagne, J.C.: Knowledge Spaces. Springer Science & Business Media, Berlin (2012)MATHGoogle Scholar
  41. Essa, A.: A possible future for next generation adaptive learning systems. Smart Learn. Environ. 3(1), 16 (2016)CrossRefGoogle Scholar
  42. Falakmasir, M.H., Pardos, Z.A., Gordon, G.J., Brusilovsky, P.: A spectral learning approach to knowledge tracing. In: Proceedings of Educational Data Mining (2013)Google Scholar
  43. Feng, M., Heffernan, N., Mani, M., Heffernan, C.: Using mixed-effects modeling to compare different grain-sized skill models. In: Proceedings of Educational Data Mining: Papers from the AAAI Workshop (2006)Google Scholar
  44. Gasparetti, F., Limongelli, C., Sciarrone, F.: Exploiting wikipedia for discovering prerequisite relationships among learning objects. In: Proceedings of Information Technology Based Higher Education and Training, pp. 1–6. IEEE (2015)Google Scholar
  45. Goldhammer, F.: Measuring ability, speed, or both? Challenges, psychometric solutions, and what can be gained from experimental control. Meas. Interdiscip. Res. Perspect. 13(3–4), 133–164 (2015)CrossRefGoogle Scholar
  46. Goldin, I., Koedinger, K., Aleven, V.: Hints: you can’t have just one. In: Proceedings of Educational Data Mining (2013)Google Scholar
  47. Gong, Y., Beck, J.: Items, skills, and transfer models: which really matters for student modeling? In: Proceedings of Educational Data Mining (2011)Google Scholar
  48. Gong, Y., Beck, J.E., Heffernan, N.T.: Comparing knowledge tracing and performance factor analysis by using multiple model fitting procedures. In: Proceedings of Intelligent Tutoring Systems, pp. 35–44. Springer (2010)Google Scholar
  49. Gong, Y., Beck, J.E., Heffernan, N.T.: How to construct more accurate student models: comparing and optimizing knowledge tracing and performance factor analysis. Int. J. Artif. Intell. Educ. 21(1–2), 27–46 (2011)Google Scholar
  50. Gong, Y., Beck, J.E., Ruiz, C.: Modeling multiple distributions of student performances to improve predictive accuracy. In: Masthoff, J., Mobasher, B., Desmarais, M.C., Nkambou, R. (eds.) User Modeling, Adaptation, and Personalization: 20th International Conference, UMAP 2012, Montreal, Canada, July 16–20, 2012. Proceedings, pp. 102–113. Spinger, Berlin (2012)Google Scholar
  51. González-Brenes, J., Huang, Y., Brusilovsky, P.: General features in knowledge tracing: applications to multiple subskills, temporal item response theory, and expert knowledge. In: Proceedings of Educational Data Mining, pp. 84–91 (2014)Google Scholar
  52. González-Brenes, J.P., Huang, Y.: Your model is predictive—but is it useful? Theoretical and empirical considerations of a new paradigm for adaptive tutoring evaluation. In: Proceedings of Educational Data Mining (2015)Google Scholar
  53. Hämäläinen, W., Kumpulainen, V., Mozgovoy, M.: Evaluation of clustering methods for adaptive learning systems. In: Kose, U., Koc, D. (eds.) Artificial Intelligence Applications in Distance Education, pp. 237–260. IGI Global, Hershey, PA (2015)Google Scholar
  54. Hawkins, W.J., Heffernan, N.T., Baker, R.S.: Learning Bayesian knowledge tracing parameters with a knowledge heuristic and empirical probabilities. In: Proceedings of Intelligent Tutoring Systems, pp. 150–155. Springer International Publishing (2014)Google Scholar
  55. Hershkovitz, A., de Baker, R.S.J., Gobert, J., Wixon, M., Sao Pedro, M.: Discovery with models a case study on carelessness in computer-based science inquiry. Am. Behav. Sci. 57(10), 1480–1499 (2013)CrossRefGoogle Scholar
  56. Huang, Y., González-Brenes, J.P., Brusilovsky, P.: Challenges of using observational data to determine the importance of example usage. In: Proceedings of Artificial Intelligence in Education, pp. 633–637. Springer (2015a)Google Scholar
  57. Huang, Y., González-Brenes, J.P., Kumar, R., Brusilovsky, P.: A framework for multifaceted evaluation of student models. In: Proceedings of Educational Data Mining (2015b)Google Scholar
  58. Junker, B.W., Sijtsma, K.: Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Appl. Psychol. Meas. 25(3), 258–272 (2001)MathSciNetCrossRefGoogle Scholar
  59. Käser, T., Busetto, A.G., Solenthaler, B., Baschera, G.M., Kohn, J., Kucian, K., von Aster, M., Gross, M.: Modelling and optimizing mathematics learning in children. Int. J. Artif. Intell. Educ. 23(1–4), 115–135 (2013a)CrossRefGoogle Scholar
  60. Käser, T., Busetto, A.G., Solenthaler, B., Kohn, J., von Aster, M., Gross, M.: Cluster-based prediction of mathematical learning patterns. In: Proceedings of Artificial Intelligence in Education, pp. 389–399. Springer (2013b)Google Scholar
  61. Käser, T., Klingler, S., Schwing, A.G., Gross, M.: Beyond knowledge tracing: modeling skill topologies with Bayesian networks. In: Proceedings of ITS, pp. 188–198 (2014a)Google Scholar
  62. Käser, T., Koedinger, K.R., Gross, M.: Different parameters—same prediction: an analysis of learning curves. In: Proceedings of Educational Data Mining, pp. 52–59 (2014b)Google Scholar
  63. Käser, T., Klingler, S., Gross, M.: When to stop? Towards universal instructional policies. In: Proceedings of Learning Analytics and Knowledge, pp. 289–298. ACM (2016)Google Scholar
  64. Khajah, M., Wing, R.M., Lindsey, R.V., Mozer, M.C.: Integrating latent-factor and knowledge-tracing models to predict individual differences in learning. In: Proceedings of Educational Data Mining (2014a)Google Scholar
  65. Khajah, M.M., Huang, Y., González-Brenes, J.P., Mozer, M.C., Brusilovsky, P.: Integrating knowledge tracing and item response theory: a tale of two frameworks. In: Workshop on Personalization Approaches in Learning Environments (2014b)Google Scholar
  66. Khajah, M., Lindsey, R.V., Mozer, M.C.: How deep is knowledge tracing? In: Proceedings of Educational Data Mining (2016)Google Scholar
  67. Klingler, S., Käser, T., Solenthaler, B., Gross, M.: On the performance characteristics of latent-factor and knowledge tracing models. In: Proceedings of Educational Data Mining (2015)Google Scholar
  68. Klingler, S., Käser, T., Solenthaler, B., Gross, M.: Temporally coherent clustering of student data. In: Proceedings of Educational Data Mining (2016)Google Scholar
  69. Klinkenberg, S., Straatemeier, M., Van der Maas, H.: Computer adaptive practice of maths ability using a new item response model for on the fly ability and difficulty estimation. Comput. Educ. 57(2), 1813–1824 (2011)CrossRefGoogle Scholar
  70. Koedinger, K.R., McLaughlin, E.A.: Closing the loop with quantitative cognitive task analysis. In: Proceedings of Educational Data Mining (2016)Google Scholar
  71. Koedinger, K.R., Pavlik, Jr P.I., Stamper, J.C., Nixon, T., Ritter, S.: Avoiding problem selection thrashing with conjunctive knowledge tracing. In: Proceedings of Educational Data Mining, pp. 91–100 (2011)Google Scholar
  72. Koedinger, K.R., Corbett, A.T., Perfetti, C.: The knowledge-learning-instruction framework: bridging the science-practice chasm to enhance robust student learning. Cogn. Sci. 36(5), 757–798 (2012)CrossRefGoogle Scholar
  73. Koedinger, K.R., Stamper, J.C., McLaughlin, E.A., Nixon, T.: Using data-driven discovery of better student models to improve student learning. In: Proceedings of Artificial Intelligence in Education, pp. 421–430. Springer (2013)Google Scholar
  74. Koedinger, K.R., Yudelson, M.V., Pavlik, P.I.: Testing theories of transfer using error rate learning curves. Top. Cogn. Sci. 8(3), 589–609 (2016)CrossRefGoogle Scholar
  75. Lan, A.S., Waters, A.E., Studer, C., Baraniuk, R.G.: Sparse factor analysis for learning and content analytics. J. Mach. Learn. Res. 15(1), 1959–2008 (2014)MathSciNetMATHGoogle Scholar
  76. Lee, J.I., Brunskill, E.: The impact on individualizing student models on necessary practice opportunities. In: Proceedings of Educational Data Mining, pp. 118–125 (2012)Google Scholar
  77. Li, N., Cohen, W., Koedinger, K.R., Matsuda, N.: A machine learning approach for automatic student model discovery. In: Proceedings of Educational Data Mining (2011)Google Scholar
  78. Lindsey, R., Khajah, M., Mozer, M.C.: Automatic discovery of cognitive skills to improve the prediction of student learning. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberge, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 1386–1394. Curran Associates Inc., La Jolla, CA (2014)Google Scholar
  79. Liu, R., Koedinger, K.R., McLaughlin, E.A.: Interpreting model discovery and testing generalization to a new dataset. In: Proceedings of Educational Data Mining, pp. 107–113 (2014)Google Scholar
  80. Liu, R., Patel, R., Koedinger, K.R.: Modeling common misconceptions in learning process data. In: Proceedings of Learning Analytics and Knowledge, pp. 369–377. ACM (2016)Google Scholar
  81. MacHardy, Z., Pardos, Z.A.: Evaluating the relevance of educational videos using BKT and big data. In: Proceedings of Educational Data Mining (2015)Google Scholar
  82. Mayo, M., Mitrovic, A.: Optimising its behaviour with Bayesian networks and decision theory. Int. J. Artif. Intell. Educ. 12(2), 124–153 (2001)Google Scholar
  83. McTavish, T.S., Larusson, J.A.: Labeling mathematical errors to reveal cognitive states. In: Open Learning and Teaching in Educational Communities, pp. 446–451. Springer (2014)Google Scholar
  84. Meadows, D.H.: Leverage Points: Places to Intervene in a System. Sustainability Institute Hartland, VT (1999)Google Scholar
  85. Merceron, A., Yacef, K.: Clustering students to help evaluate learning. In: Courtiat, J.-P., Davarakis, C., Villemur, T. (eds.) Technology Enhanced Learning: IFIP TC3 Technology Enhanced Learning Workshop (TeL’04), World Computer Congress, August 22–27, 2004, Toulouse, France, pp. 31–42. Springer, US, Boston, MA (2005)CrossRefGoogle Scholar
  86. Mettler, E., Massey, C.M., Kellman, P.J.: Improving adaptive learning technology through the use of response times. In: Proceedings of Conference of the Cognitive Science Society, pp. 2532–2537 (2011)Google Scholar
  87. Millán, E., Loboda, T., Pérez-de-la Cruz, J.L.: Bayesian networks for student model engineering. Comput. Educ. 55(4), 1663–1683 (2010)CrossRefGoogle Scholar
  88. Mitrovic, A., Martin, B.: Evaluating the effect of open student models on self-assessment. Int. J. Artif. Intell. Educ. 17(2), 121–144 (2007)Google Scholar
  89. Mitrovic, A., Mayo, M., Suraweera, P., Martin, B.: Constraint-based tutors: a success story. In: Proceedings of Industrial, Engineering and Other Applications of Applied Intelligent Systems, pp. 931–940. Springer (2001)Google Scholar
  90. Mitrovic, A., Koedinger, K.R., Martin, B.: A comparative analysis of cognitive tutoring and constraint-based modeling. In: Proceedings of User Modeling, pp. 313–322. Springer (2003)Google Scholar
  91. Mitrovic, A., Martin, B., Suraweera, P.: Intelligent tutors for all: the constraint-based approach. IEEE Intell. Syst. 22, 38–45 (2007)CrossRefGoogle Scholar
  92. Murray, R.C., Ritter, S., Nixon, T., Schwiebert, R., Hausmann, R.G., Towle, B., Fancsali, S.E., Vuong, A.: Revealing the learning in learning curves. In: Proceedings of Artificial Intelligence in Education, pp. 473–482. Springer (2013)Google Scholar
  93. Nakic, J., Granic, A., Glavinic, V.: Anatomy of student models in adaptive learning systems: a systematic literature review of individual differences from 2001 to 2013. J. Educ. Comput. Res. 51(4), 459–489 (2015)Google Scholar
  94. Nižnan, J., Pelánek, R., Papoušek, J.: Exploring the role of small differences in predictive accuracy using simulated data. In: AIED Workshop on Simulated Learners (2015a)Google Scholar
  95. Nižnan, J., Pelánek, R., Řihák, J.: Student models for prior knowledge estimation. In: Proceedings of Educational Data Mining, pp. 109–116 (2015b)Google Scholar
  96. Nixon, T., Fancsali, S., Ritter, S.: The complex dynamics of aggregate learning curves. In: Proceedings of Educational Data Mining (2013)Google Scholar
  97. Nižnan, J., Pelánek, R., Řihák, J.: Mapping problems to skills combining expert opinion and student data. In: Proceedings of Mathematical and Engineering Methods in Computer Science, LNCS, vol. 8934, pp. 113–124. Springer (2014)Google Scholar
  98. Ohlsson, S.: Constraint-based student modeling. In: Greer, J.E., McCalla, G.I. (eds.) Student Modelling: The Key to Individualized Knowledge-Based Instruction, pp. 167–189. Springer, Berlin (1994)Google Scholar
  99. Papoušek, J., Pelánek, R., Stanislav, V.: Adaptive practice of facts in domains with varied prior knowledge. In: Proceedings of Educational Data Mining, pp. 6–13 (2014)Google Scholar
  100. Papoušek, J., Stanislav, V., Pelánek, R.: Evaluation of an adaptive practice system for learning geography facts. In: Proceedings of Learning Analytics and Knowledge. ACM (2016)Google Scholar
  101. Pardos, Z.A., Heffernan, N.T.: Determining the significance of item order in randomized problem sets. In: Proceedings of Educational Data Mining (2009)Google Scholar
  102. Pardos, Z.A., Heffernan, N.T.: Modeling individualization in a Bayesian networks implementation of knowledge tracing. In: Proceedings of User Modeling, Adaptation, and Personalization, pp. 255–266. Springer (2010a)Google Scholar
  103. Pardos, Z.A., Heffernan, N.T.: Navigating the parameter space of Bayesian knowledge tracing models: visualizations of the convergence of the expectation maximization algorithm. In: Proceedings of Educational Data Mining, pp. 161–170 (2010b)Google Scholar
  104. Pardos, Z.A., Heffernan, N.T.: KT-IDEM: introducing item difficulty to the knowledge tracing model. In: Konstan, J.A., Conejo, R., Marzo, J.L. Oliver, N. (eds.) User Modeling, Adaption and Personalization: 19th International Conference, UMAP 2011, Girona, Spain, July 11–15, 2011. Proceedings, pp. 243–254. Springer, Berlin (2011)Google Scholar
  105. Pardos, Z.A., Xu, Y.: Improving efficacy attribution in a self-directed learning environment using prior knowledge individualization. In: Proceedings of Learning Analytics and Knowledge, pp. 435–439. ACM (2016)Google Scholar
  106. Pardos, Z.A., Heffernan, N.T., Anderson, B., Heffernan, C.L., Schools, W.P.: Using fine-grained skill models to fit student performance with Bayesian networks. In: Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.D. (eds.) Handbook of Educational Data Mining, pp. 417–426. Chapman & Hall (2010)Google Scholar
  107. Pardos, Z.A., Gowda, S.M., Baker, R.S., Heffernan, N.T.: The sum is greater than the parts: ensembling models of student knowledge in educational software. ACM SIGKDD Explor. Newsl. 13(2), 37–44 (2012a)CrossRefGoogle Scholar
  108. Pardos, Z.A., Trivedi, S., Heffernan, N.T., Sárközy, G.N.: Clustered knowledge tracing. In: Proceedings of Intelligent Tutoring Systems, pp. 405–410. Springer (2012b)Google Scholar
  109. Pardos, Z.A., Wang, Q.Y., Trivedi, S.: The real world significance of performance prediction. In: Proceedings of Educational Data Mining (2012c)Google Scholar
  110. Pavlik, P.I., Anderson, J.R.: Practice and forgetting effects on vocabulary memory: an activation-based model of the spacing effect. Cogn. Sci. 29(4), 559–586 (2005)CrossRefGoogle Scholar
  111. Pavlik, P.I., Anderson, J.R.: Using a model to compute the optimal schedule of practice. J. Exp. Psychol. Appl. 14(2), 101 (2008)CrossRefGoogle Scholar
  112. Pavlik, Jr P., Bolster, T., Wu, S.M., Koedinger, K., Macwhinney, B.: Using optimally selected drill practice to train basic facts. In: Proceedings of Intelligent Tutoring Systems, pp. 593–602. Springer (2008)Google Scholar
  113. Pavlik, P.I., Cen, H., Koedinger, K.R.: Performance factors analysis—a new alternative to knowledge tracing. In: Proceedings of Artificial Intelligence in Education, pp. 531–538. IOS Press (2009)Google Scholar
  114. Pavlik, P.I., Brawner, K., Olney, A., Mitrovic, A.: A review of student models used in intelligent tutoring systems. In: Sottilare, R.A., Graesser, A., Hu, X., Holden, H. (eds) Design Recommendations for Intelligent Tutoring Systems, vol. 1, pp. 39–68. US Army Research Laboratory, Orlando, FL (2013)Google Scholar
  115. Pelánek, R.: Application of time decay functions and Elo system in student modeling. In: Proceedings of Educational Data Mining, pp. 21–27 (2014)Google Scholar
  116. Pelánek, R.: Metrics for evaluation of student models. J. Educ. Data Min. 7(2), 1–19 (2015a)Google Scholar
  117. Pelánek, R.: Modeling students’ memory for application in adaptive educational systems. In: Proceedings of Educational Data Mining, pp. 480–483 (2015b)Google Scholar
  118. Pelánek, R.: Applications of the Elo rating system in adaptive educational systems. Comput. Educ. 98, 169–179 (2016)CrossRefGoogle Scholar
  119. Pelánek, R., Jarušek, P.: Student modeling based on problem solving times. Int. J. Artif. Intell. Educ. 25(4), 493–519 (2015)Google Scholar
  120. Pelánek, R., Řihák, J.: Properties and applications of wrong answers in online educational systems. In: Proceedings of Educational Data Mining (2016)Google Scholar
  121. Pelánek, R., Řihák, J., Papoušek, J.: Impact of data collection on interpretation and evaluation of student model. In: Proceedings of Learning Analytics and Knowledge, pp. 40–47. ACM (2016)Google Scholar
  122. Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L.J., Sohl-Dickstein, J.: Deep knowledge tracing. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 505–513. Curran Associates, Inc. (2015). http://papers.nips.cc/paper/5654-deep-knowledge-tracing.pdf
  123. Qiu, Y., Qi, Y., Lu, H., Pardos, Z.A., Heffernan, N.T.: Does time matter? Modeling the effect of time with Bayesian knowledge tracing. In: EDM, pp. 139–148 (2011)Google Scholar
  124. Reddy, S., Labutov, I., Banerjee, S., Joachims, T.: Unbounded human learning: optimal scheduling for spaced repetition. In: Proceedings of Knowledge Discovery and Data Mining. ACM (2016)Google Scholar
  125. Ritter, S., Yudelson, M., Fancsali, S.E., Berman, S.R.: How mastery learning works at scale. In: Proceedings of Learning@Scale, pp. 71–79. ACM (2016)Google Scholar
  126. Rollinson, J., Brunskill, E.: From predictive models to instructional policies. In: Proceedings of Educational Data Mining (2015)Google Scholar
  127. Rubin, D.C., Hinton, S., Wenzel, A.: The precise time course of retention. J. Exp. Psychol. Learn. Mem. Cogn. 25(5), 1161 (1999)CrossRefGoogle Scholar
  128. Sao Pedro, M., Baker, R.S., Gobert, J.D.: Incorporating scaffolding and tutor context into Bayesian knowledge tracing to predict inquiry skill acquisition. In: Proceedings of Educational Data Mining, pp. 185–192 (2013a)Google Scholar
  129. Sao Pedro, M.A., Baker, R.S., Gobert, J.D.: What different kinds of stratification can reveal about the generalizability of data-mined skill assessment models. In: Proceedings of Learning Analytics and Knowledge, pp. 190–194. ACM (2013b)Google Scholar
  130. Scheines, R., Silver, E., Goldin, I.: Discovering prerequisite relationships among knowledge components. In: Proceedings of Educational Data Mining, pp. 355–356 (2014)Google Scholar
  131. Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., Dennison, D.: Hidden technical debt in machine learning systems. In: Proceedings of the 28th International Conference on Neural Information Processing Systems. NIPS’15, Montreal, Canada, pp. 2503–2511. MIT Press, Cambridge, MA (2015). http://dl.acm.org/citation.cfm?id=2969442.2969519
  132. Sense, F., Behrens, F., Meijer, R.R., Rijn, H.: An individual’s rate of forgetting is stable over time but differs across materials. Top. Cogn. Sci. 8(1), 305–321 (2016)CrossRefGoogle Scholar
  133. Smith, A.C., Frank, L.M., Wirth, S., Yanike, M., Hu, D., Kubota, Y., Graybiel, A.M., Suzuki, W.A., Brown, E.N.: Dynamic analysis of learning in behavioral experiments. J. Neurosci. 24(2), 447–461 (2004)CrossRefGoogle Scholar
  134. Stamper, J.C., Koedinger, K.R., McLaughlin, E.A.: A comparison of model selection metrics in datashop. In: Proceedings of Educational Data Mining, pp. 284–287 (2013)Google Scholar
  135. Stone, M.: An asymptotic equivalence of choice of model by cross-validation and Akaike’s criterion. J. R. Stat. Soc. Ser. B (Methodol.) 39(1), 44–47 (1977)Google Scholar
  136. Straatemeier, M.: Math Garden: A New Educational and Scientific Instrument. PhD thesis, Universiteit van Amsterdam, Faculty of Social and Behavioural Sciences (2014)Google Scholar
  137. Streeter, M.: Mixture modeling of individual learning curves. In: Proceedings of Educational Data Mining (2015)Google Scholar
  138. Tang, S., McBride, E., Gogel, H., Pardos, Z.A.: Item ordering effects with qualitative explanations using online adaptive tutoring data. In: Proceedings of Learning@Scale, pp. 313–316. ACM (2015)Google Scholar
  139. Tatsuoka, K.K.: Rule space: an approach for dealing with misconceptions based on item response theory. J. Educ. Meas. 20(4), 345–354 (1983)CrossRefGoogle Scholar
  140. Thai-Nghe, N., Drumond, L., Horváth, T., Krohn-Grimberghe, A., Nanopoulos, A., Schmidt-Thieme, L.: Factorization techniques for predicting student performance. In: Santos, O., Boticario, J. (eds.) Educational Recommender Systems and Technologies: Practices and Challenges, pp. 129–153. IGI Global, Hershey, PA (2012)Google Scholar
  141. Toscher, A., Jahrer, M.: Collaborative filtering applied to educational data mining. In: KDD cup (2010)Google Scholar
  142. Trivedi, S., Pardos, Z.A., Heffernan, N.T.: Clustering students to generate an ensemble to improve standard test score predictions. In: Proceedings of Artificial Intelligence in Education, pp. 377–384. Springer (2011)Google Scholar
  143. Valdés Aguirre, B., Ramírez Uresti, J.A., Boulay, B.D.: An analysis of student model portability. Int. J. Artif. Intell. Educ. 26(3), 932–974 (2016)CrossRefGoogle Scholar
  144. Van Der Linden, W.: Conceptual issues in response-time modeling. J. Educ. Meas. 46(3), 247–272 (2009)CrossRefGoogle Scholar
  145. Van Inwegen, E.G., Adjei, S.A., Wang, Y., Heffernan, N.T.: Using partial credit and response history to model user knowledge. In: Proceedings of Educational Data Mining (2015)Google Scholar
  146. Vanlehn, K.: The behavior of tutoring systems. Int. J. Artif. Intell. Educ. 16(3), 227–265 (2006)Google Scholar
  147. Vanlehn, K., Lynch, C., Schulze, K., Shapiro, J.A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., Wintersgill, M.: The Andes physics tutoring system: lessons learned. Int. J. Artif. Intell. Educ. 15(3), 147–204 (2005)MATHGoogle Scholar
  148. van de Sande, B.: Properties of the Bayesian knowledge tracing model. J. Educ. Data Min. 5(2), 1 (2013)Google Scholar
  149. Řihák, J.: Use of time information in models behind adaptive system for building fluency in mathematics. In: Proceedings of Educational Data Mining, Doctoral Consortium (2015)Google Scholar
  150. Řihák, J., Pelánek, R.: Choosing a student model for a real world application. In: Building ITS Bridges Across Frontiers (ITS Workshop) (2016)Google Scholar
  151. Wang, Y., Beck, J.: Class vs. student in a Bayesian network student model. In: Proceedings of Artificial Intelligence in Education, pp. 151–160. Springer (2013a)Google Scholar
  152. Wang, Y., Heffernan, N.: Extending knowledge tracing to allow partial credit: using continuous versus binary nodes. In: Proceedings of Artificial Intelligence in Education, pp. 181–188. Springer (2013b)Google Scholar
  153. Wang, X., Berger, J.O., Burdick, D.S., et al.: Bayesian analysis of dynamic item response models in educational testing. Ann. Appl. Stat. 7(1), 126–153 (2013)MathSciNetMATHCrossRefGoogle Scholar
  154. Wang, Y., Heffernan, N.T., Heffernan, C.: Towards better affect detectors: effect of missing skills, class features and common wrong answers. In: Proceedings of Learning Analytics and Knowledge, pp. 31–35. ACM (2015)Google Scholar
  155. Wauters, K., Desmet, P., Van Den Noortgate, W.: Item difficulty estimation: an auspicious collaboration between data and judgment. Comput. Educ. 58(4), 1183–1193 (2012)CrossRefGoogle Scholar
  156. White, K.G.: Forgetting functions. Anim. Learn. Behav. 29(3), 193–207 (2001)CrossRefGoogle Scholar
  157. Xu, Y., Mostow, J.: Comparison of methods to trace multiple subskills: is LR-DBN best? In: Proceedings of Educational Data Mining (2012)Google Scholar
  158. Yudelson, M.V., Koedinger, K.R.: Estimating the benefits of student model improvements on a substantive scale. In: Educational Data Mining Workshops Proceedings (2013a)Google Scholar
  159. Yudelson, M.V., Koedinger, K.R., Gordon, G.J.: Individualized Bayesian knowledge tracing models. In: Proceedings of Artificial Intelligence in Education, pp. 171–180. Springer (2013b)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  1. 1.Faculty of InformaticsMasaryk UniversityBrnoCzechia

Personalised recommendations