Skip to main content
SpringerLink
Log in

Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques

  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

Learner modeling is a basis of personalized, adaptive learning. The research literature provides a wide range of modeling approaches, but it does not provide guidance for choosing a model suitable for a particular situation. We provide a systematic and up-to-date overview of current approaches to tracing learners’ knowledge and skill across interaction with multiple items, focusing in particular on the widely used Bayesian knowledge tracing and logistic models. We discuss factors that influence the choice of a model and highlight the importance of the learner modeling context: models are used for different purposes and deal with different types of learning processes. We also consider methodological issues in the evaluation of learner models and their relation to the modeling context. Overall, the overview provides basic guidelines for both researchers and practitioners and identifies areas that require further clarification in future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Aleven, V., Koedinger, K.R.: Knowledge component (kC) approaches to learner modeling. In: Sottilare, R.A., Graesser, A., Hu, X., Holden, H. (eds) Design Recommendations for Intelligent Tutoring Systems, vol. 1, pp. 165–182. US Army Research Laboratory, Orlando, FL (2013)

  • Aleven, V., Sewall, J.: The frequency of tutor behaviors: a case study. In: Proceedings of Intelligent Tutoring Systems, pp. 396–401. Springer (2016)

  • Aleven, V., McLaren, B.M., Sewall, J., Koedinger, K.R.: The cognitive tutor authoring tools (CTAT): preliminary evaluation of efficiency gains. In: International Conference on Intelligent Tutoring Systems, pp. 61–70. Springer (2006)

  • Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167–207 (1995)

    Article  Google Scholar 

  • Arroyo, I., Mehranian, H., Woolf, B.P.: Effort-based tutoring: an empirical approach to intelligent tutoring. In: Proceedings of Educational Data Mining (2010)

  • Ayers, E., Junker, B.: Do skills combine additively to predict task difficulty in eighth grade mathematics. In: Proceedings of Educational Data Mining: Papers from the AAAI Workshop (2006)

  • Baker, R.S.: Stupid tutoring systems, intelligent humans. Int. J. Artif. Intell. Educ. 26(2), 600–614 (2016)

    Article  Google Scholar 

  • Baker, R.S., Yacef, K.: The state of educational data mining in 2009: a review and future visions. J. Educ. Data Min. 1(1), 3–17 (2009)

    Google Scholar 

  • Baker, R.S., Corbett, A.T., Aleven, V.: More accurate student modeling through contextual estimation of slip and guess probabilities in Bayesian knowledge tracing. In: Proceedings of Intelligent Tutoring Systems, pp. 406–415. Springer (2008)

  • Barnes, T.: The q-matrix method: mining student response data for knowledge. In: American Association for Artificial Intelligence 2005 Educational Data Mining Workshop, pp. 1–8 (2005)

  • Beck, J.E., Chang, K.: Identifiability: a fundamental problem of student modeling. In: Conati, C., McCoy, K., Paliouras, G. (eds.) User Modeling 2007: 11th International Conference, UM 2007, Corfu, Greece, July 25–29, 2007. Proceedings, pp. 137–146. Springer, Berlin (2007)

  • Beck, J., Xiong, X.: Limits to accuracy: how well can we do at student modeling? In: Proceedings of Educational Data Mining (2013a)

  • Beck, J.E., Gong, Y.: Wheel-spinning: students who fail to master a skill. In: Proceedings of Artificial Intelligence in Education, pp. 431–440. Springer (2013b)

  • Beck, J.E., Chang, K.M., Mostow, J., Corbett, A.: Does help help? Introducing the Bayesian evaluation and assessment methodology. In: Proceedings of Intelligent Tutoring Systems, pp. 383–394. Springer (2008a)

  • Beck, J.E., Pardos, Z.A., Heffernan, N.T., Ruiz, C.: The composition effect: conjunctive or compensatory? An analysis of multi-skill math questions in its. In: Proceedings of Educational Data Mining (2008b)

  • Beel, J., Breitinger, C., Langer, S., Lommatzsch, A., Gipp, B.: Towards reproducibility in recommender-systems research. User Model. User Adapt. Interact. 26(1), 69–101 (2016)

    Article  Google Scholar 

  • Boroš, P., Nižnan, J., Pelánek, R., Řihák, J.: Automatic detection of concepts from problem solving times. In: Proceedings of Artificial Intelligence in Education, LNCS, vol. 7926, pp. 595–598. Springer (2013)

  • Bull, S., Kay, J.: Student models that invite the learner in: the smili:() open learner modelling framework. Int. J. Artif. Intell. Educ. 17(2), 89–120 (2007)

    Google Scholar 

  • Bull, S., Kay, J.: Open learner models. In: Nkambou, R., Bourdeau, J., Mizoguchi, R. (eds.) Advances in Intelligent Tutoring Systems, pp. 301–322. Springer, Berlin (2010)

  • Bull, S., Ginon, B., Boscolo, C., Johnson, M.: Introduction of learning visualisations and metacognitive support in a persuadable open learner model. In: Proceedings of Learning Analytics & Knowledge, pp. 30–39. ACM (2016)

  • Carlson, R., Genin, K., Rau, M., Scheines, R.: Student profiling from tutoring system log data: when do multiple graphical representations matter? In: Proceedings of Educational Data Mining (2013)

  • Carmona, C., Millán, E., Pérez-de-la Cruz, J.L., Trella, M., Conejo, R.: Introducing prerequisite relations in a multi-layered Bayesian student model. In: User Modeling, pp. 347–356. Springer (2005)

  • Cen, H., Koedinger, K., Junker, B.: Learning factors analysis—a general method for cognitive model evaluation and improvement. In: Proceedings of Intelligent Tutoring Systems, pp. 164–175. Springer (2006)

  • Cen, H., Koedinger, K.R., Junker, B.: Is over practice necessary? Improving learning efficiency with the cognitive tutor through educational data mining. Front. Artif. Intell. Appl. 158, 511 (2007)

    Google Scholar 

  • Cen, H., Koedinger, K., Junker, B.: Comparing two IRT models for conjunctive skills. In: Proceedings of Intelligent Tutoring Systems, pp. 796–798. Springer (2008)

  • Chen, Y., Wuillemin, P.H., Labat, J.M.: Discovering prerequisite structure of skills through probabilistic association rules mining. In: Proceedings of Educational Data Mining, pp. 117–124 (2015)

  • Chen, Y., Gonzlez-Brenes, J., Tian, J.: Joint discovery of skill prerequisite graphs and student models. In: Proceedings of Educational Data Mining (2016)

  • Chi, M., Koedinger, K., Gordon, G., Jordan, P., Van Lehn, K.: Instructional factors analysis. In: Proceedings of Educational Data Mining (2011)

  • Chrysafiadi, K., Virvou, M.: Student modeling approaches: a literature review for the last decade. Expert Syst. Appl. 40(11), 4715–4729 (2013)

    Article  Google Scholar 

  • Conati, C., Gertner, A., Vanlehn, K.: Using Bayesian networks to manage uncertainty in student modeling. User Model. User Adapt. Interact. 12(4), 371–417 (2002)

    Article  MATH  Google Scholar 

  • Corbett, A.T., Anderson, J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. User Model. User Adapt. Interact. 4(4), 253–278 (1994)

    Article  Google Scholar 

  • David, Y.B., Segal, A., Gal, Y.K.: Sequencing educational content in classrooms using Bayesian knowledge tracing. In: Proceedings of Learning Analytics and Knowledge, pp. 354–363. ACM (2016)

  • De La Torre, J.: Dina model and parameter estimation: a didactic. J. Educ. Behav. Stat. 34(1), 115–130 (2009)

    Article  Google Scholar 

  • Desmarais, M.: Conditions for effectively deriving a q-matrix from data with non-negative matrix factorization. In: Proceedings of Educational Data Mining (2011)

  • Desmarais, M.C., Baker, R.S.: A review of recent advances in learner and skill modeling in intelligent learning environments. User Model. User Adapt. Interact. 22(1–2), 9–38 (2012)

    Article  Google Scholar 

  • Desmarais, M., Lemieux, F.: Clustering and visualizing study state sequences. In: Proceedings of Educational Data Mining (2013a)

  • Desmarais, M.C., Naceur, R.: A matrix factorization method for mapping items to skills and for enhancing expert-based q-matrices. In: Proceedings of Artificial Intelligence in Education, pp. 441–450. Springer (2013b)

  • Desmarais, M.C., Meshkinfam, P., Gagnon, M.: Learned student models with item to item knowledge structures. User Model. User Adapt. Interact. 16(5), 403–434 (2006)

    Article  Google Scholar 

  • Desmarais, M., Beheshti, B., Xu, P.: The refinement of a q-matrix: assessing methods to validate tasks to skills mapping. In: Proceedings of Educational Data Mining (2014)

  • Doignon, J.P., Falmagne, J.C.: Knowledge Spaces. Springer Science & Business Media, Berlin (2012)

    MATH  Google Scholar 

  • Essa, A.: A possible future for next generation adaptive learning systems. Smart Learn. Environ. 3(1), 16 (2016)

    Article  Google Scholar 

  • Falakmasir, M.H., Pardos, Z.A., Gordon, G.J., Brusilovsky, P.: A spectral learning approach to knowledge tracing. In: Proceedings of Educational Data Mining (2013)

  • Feng, M., Heffernan, N., Mani, M., Heffernan, C.: Using mixed-effects modeling to compare different grain-sized skill models. In: Proceedings of Educational Data Mining: Papers from the AAAI Workshop (2006)

  • Gasparetti, F., Limongelli, C., Sciarrone, F.: Exploiting wikipedia for discovering prerequisite relationships among learning objects. In: Proceedings of Information Technology Based Higher Education and Training, pp. 1–6. IEEE (2015)

  • Goldhammer, F.: Measuring ability, speed, or both? Challenges, psychometric solutions, and what can be gained from experimental control. Meas. Interdiscip. Res. Perspect. 13(3–4), 133–164 (2015)

    Article  Google Scholar 

  • Goldin, I., Koedinger, K., Aleven, V.: Hints: you can’t have just one. In: Proceedings of Educational Data Mining (2013)

  • Gong, Y., Beck, J.: Items, skills, and transfer models: which really matters for student modeling? In: Proceedings of Educational Data Mining (2011)

  • Gong, Y., Beck, J.E., Heffernan, N.T.: Comparing knowledge tracing and performance factor analysis by using multiple model fitting procedures. In: Proceedings of Intelligent Tutoring Systems, pp. 35–44. Springer (2010)

  • Gong, Y., Beck, J.E., Heffernan, N.T.: How to construct more accurate student models: comparing and optimizing knowledge tracing and performance factor analysis. Int. J. Artif. Intell. Educ. 21(1–2), 27–46 (2011)

    Google Scholar 

  • Gong, Y., Beck, J.E., Ruiz, C.: Modeling multiple distributions of student performances to improve predictive accuracy. In: Masthoff, J., Mobasher, B., Desmarais, M.C., Nkambou, R. (eds.) User Modeling, Adaptation, and Personalization: 20th International Conference, UMAP 2012, Montreal, Canada, July 16–20, 2012. Proceedings, pp. 102–113. Spinger, Berlin (2012)

  • González-Brenes, J., Huang, Y., Brusilovsky, P.: General features in knowledge tracing: applications to multiple subskills, temporal item response theory, and expert knowledge. In: Proceedings of Educational Data Mining, pp. 84–91 (2014)

  • González-Brenes, J.P., Huang, Y.: Your model is predictive—but is it useful? Theoretical and empirical considerations of a new paradigm for adaptive tutoring evaluation. In: Proceedings of Educational Data Mining (2015)

  • Hämäläinen, W., Kumpulainen, V., Mozgovoy, M.: Evaluation of clustering methods for adaptive learning systems. In: Kose, U., Koc, D. (eds.) Artificial Intelligence Applications in Distance Education, pp. 237–260. IGI Global, Hershey, PA (2015)

  • Hawkins, W.J., Heffernan, N.T., Baker, R.S.: Learning Bayesian knowledge tracing parameters with a knowledge heuristic and empirical probabilities. In: Proceedings of Intelligent Tutoring Systems, pp. 150–155. Springer International Publishing (2014)

  • Hershkovitz, A., de Baker, R.S.J., Gobert, J., Wixon, M., Sao Pedro, M.: Discovery with models a case study on carelessness in computer-based science inquiry. Am. Behav. Sci. 57(10), 1480–1499 (2013)

    Article  Google Scholar 

  • Huang, Y., González-Brenes, J.P., Brusilovsky, P.: Challenges of using observational data to determine the importance of example usage. In: Proceedings of Artificial Intelligence in Education, pp. 633–637. Springer (2015a)

  • Huang, Y., González-Brenes, J.P., Kumar, R., Brusilovsky, P.: A framework for multifaceted evaluation of student models. In: Proceedings of Educational Data Mining (2015b)

  • Junker, B.W., Sijtsma, K.: Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Appl. Psychol. Meas. 25(3), 258–272 (2001)

    Article  MathSciNet  Google Scholar 

  • Käser, T., Busetto, A.G., Solenthaler, B., Baschera, G.M., Kohn, J., Kucian, K., von Aster, M., Gross, M.: Modelling and optimizing mathematics learning in children. Int. J. Artif. Intell. Educ. 23(1–4), 115–135 (2013a)

    Article  Google Scholar 

  • Käser, T., Busetto, A.G., Solenthaler, B., Kohn, J., von Aster, M., Gross, M.: Cluster-based prediction of mathematical learning patterns. In: Proceedings of Artificial Intelligence in Education, pp. 389–399. Springer (2013b)

  • Käser, T., Klingler, S., Schwing, A.G., Gross, M.: Beyond knowledge tracing: modeling skill topologies with Bayesian networks. In: Proceedings of ITS, pp. 188–198 (2014a)

  • Käser, T., Koedinger, K.R., Gross, M.: Different parameters—same prediction: an analysis of learning curves. In: Proceedings of Educational Data Mining, pp. 52–59 (2014b)

  • Käser, T., Klingler, S., Gross, M.: When to stop? Towards universal instructional policies. In: Proceedings of Learning Analytics and Knowledge, pp. 289–298. ACM (2016)

  • Khajah, M., Wing, R.M., Lindsey, R.V., Mozer, M.C.: Integrating latent-factor and knowledge-tracing models to predict individual differences in learning. In: Proceedings of Educational Data Mining (2014a)

  • Khajah, M.M., Huang, Y., González-Brenes, J.P., Mozer, M.C., Brusilovsky, P.: Integrating knowledge tracing and item response theory: a tale of two frameworks. In: Workshop on Personalization Approaches in Learning Environments (2014b)

  • Khajah, M., Lindsey, R.V., Mozer, M.C.: How deep is knowledge tracing? In: Proceedings of Educational Data Mining (2016)

  • Klingler, S., Käser, T., Solenthaler, B., Gross, M.: On the performance characteristics of latent-factor and knowledge tracing models. In: Proceedings of Educational Data Mining (2015)

  • Klingler, S., Käser, T., Solenthaler, B., Gross, M.: Temporally coherent clustering of student data. In: Proceedings of Educational Data Mining (2016)

  • Klinkenberg, S., Straatemeier, M., Van der Maas, H.: Computer adaptive practice of maths ability using a new item response model for on the fly ability and difficulty estimation. Comput. Educ. 57(2), 1813–1824 (2011)

    Article  Google Scholar 

  • Koedinger, K.R., McLaughlin, E.A.: Closing the loop with quantitative cognitive task analysis. In: Proceedings of Educational Data Mining (2016)

  • Koedinger, K.R., Pavlik, Jr P.I., Stamper, J.C., Nixon, T., Ritter, S.: Avoiding problem selection thrashing with conjunctive knowledge tracing. In: Proceedings of Educational Data Mining, pp. 91–100 (2011)

  • Koedinger, K.R., Corbett, A.T., Perfetti, C.: The knowledge-learning-instruction framework: bridging the science-practice chasm to enhance robust student learning. Cogn. Sci. 36(5), 757–798 (2012)

    Article  Google Scholar 

  • Koedinger, K.R., Stamper, J.C., McLaughlin, E.A., Nixon, T.: Using data-driven discovery of better student models to improve student learning. In: Proceedings of Artificial Intelligence in Education, pp. 421–430. Springer (2013)

  • Koedinger, K.R., Yudelson, M.V., Pavlik, P.I.: Testing theories of transfer using error rate learning curves. Top. Cogn. Sci. 8(3), 589–609 (2016)

    Article  Google Scholar 

  • Lan, A.S., Waters, A.E., Studer, C., Baraniuk, R.G.: Sparse factor analysis for learning and content analytics. J. Mach. Learn. Res. 15(1), 1959–2008 (2014)

    MATH  MathSciNet  Google Scholar 

  • Lee, J.I., Brunskill, E.: The impact on individualizing student models on necessary practice opportunities. In: Proceedings of Educational Data Mining, pp. 118–125 (2012)

  • Li, N., Cohen, W., Koedinger, K.R., Matsuda, N.: A machine learning approach for automatic student model discovery. In: Proceedings of Educational Data Mining (2011)

  • Lindsey, R., Khajah, M., Mozer, M.C.: Automatic discovery of cognitive skills to improve the prediction of student learning. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberge, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 1386–1394. Curran Associates Inc., La Jolla, CA (2014)

  • Liu, R., Koedinger, K.R., McLaughlin, E.A.: Interpreting model discovery and testing generalization to a new dataset. In: Proceedings of Educational Data Mining, pp. 107–113 (2014)

  • Liu, R., Patel, R., Koedinger, K.R.: Modeling common misconceptions in learning process data. In: Proceedings of Learning Analytics and Knowledge, pp. 369–377. ACM (2016)

  • MacHardy, Z., Pardos, Z.A.: Evaluating the relevance of educational videos using BKT and big data. In: Proceedings of Educational Data Mining (2015)

  • Mayo, M., Mitrovic, A.: Optimising its behaviour with Bayesian networks and decision theory. Int. J. Artif. Intell. Educ. 12(2), 124–153 (2001)

    Google Scholar 

  • McTavish, T.S., Larusson, J.A.: Labeling mathematical errors to reveal cognitive states. In: Open Learning and Teaching in Educational Communities, pp. 446–451. Springer (2014)

  • Meadows, D.H.: Leverage Points: Places to Intervene in a System. Sustainability Institute Hartland, VT (1999)

  • Merceron, A., Yacef, K.: Clustering students to help evaluate learning. In: Courtiat, J.-P., Davarakis, C., Villemur, T. (eds.) Technology Enhanced Learning: IFIP TC3 Technology Enhanced Learning Workshop (TeL’04), World Computer Congress, August 22–27, 2004, Toulouse, France, pp. 31–42. Springer, US, Boston, MA (2005)

    Chapter  Google Scholar 

  • Mettler, E., Massey, C.M., Kellman, P.J.: Improving adaptive learning technology through the use of response times. In: Proceedings of Conference of the Cognitive Science Society, pp. 2532–2537 (2011)

  • Millán, E., Loboda, T., Pérez-de-la Cruz, J.L.: Bayesian networks for student model engineering. Comput. Educ. 55(4), 1663–1683 (2010)

    Article  Google Scholar 

  • Mitrovic, A., Martin, B.: Evaluating the effect of open student models on self-assessment. Int. J. Artif. Intell. Educ. 17(2), 121–144 (2007)

    Google Scholar 

  • Mitrovic, A., Mayo, M., Suraweera, P., Martin, B.: Constraint-based tutors: a success story. In: Proceedings of Industrial, Engineering and Other Applications of Applied Intelligent Systems, pp. 931–940. Springer (2001)

  • Mitrovic, A., Koedinger, K.R., Martin, B.: A comparative analysis of cognitive tutoring and constraint-based modeling. In: Proceedings of User Modeling, pp. 313–322. Springer (2003)

  • Mitrovic, A., Martin, B., Suraweera, P.: Intelligent tutors for all: the constraint-based approach. IEEE Intell. Syst. 22, 38–45 (2007)

    Article  Google Scholar 

  • Murray, R.C., Ritter, S., Nixon, T., Schwiebert, R., Hausmann, R.G., Towle, B., Fancsali, S.E., Vuong, A.: Revealing the learning in learning curves. In: Proceedings of Artificial Intelligence in Education, pp. 473–482. Springer (2013)

  • Nakic, J., Granic, A., Glavinic, V.: Anatomy of student models in adaptive learning systems: a systematic literature review of individual differences from 2001 to 2013. J. Educ. Comput. Res. 51(4), 459–489 (2015)

    Google Scholar 

  • Nižnan, J., Pelánek, R., Papoušek, J.: Exploring the role of small differences in predictive accuracy using simulated data. In: AIED Workshop on Simulated Learners (2015a)

  • Nižnan, J., Pelánek, R., Řihák, J.: Student models for prior knowledge estimation. In: Proceedings of Educational Data Mining, pp. 109–116 (2015b)

  • Nixon, T., Fancsali, S., Ritter, S.: The complex dynamics of aggregate learning curves. In: Proceedings of Educational Data Mining (2013)

  • Nižnan, J., Pelánek, R., Řihák, J.: Mapping problems to skills combining expert opinion and student data. In: Proceedings of Mathematical and Engineering Methods in Computer Science, LNCS, vol. 8934, pp. 113–124. Springer (2014)

  • Ohlsson, S.: Constraint-based student modeling. In: Greer, J.E., McCalla, G.I. (eds.) Student Modelling: The Key to Individualized Knowledge-Based Instruction, pp. 167–189. Springer, Berlin (1994)

  • Papoušek, J., Pelánek, R., Stanislav, V.: Adaptive practice of facts in domains with varied prior knowledge. In: Proceedings of Educational Data Mining, pp. 6–13 (2014)

  • Papoušek, J., Stanislav, V., Pelánek, R.: Evaluation of an adaptive practice system for learning geography facts. In: Proceedings of Learning Analytics and Knowledge. ACM (2016)

  • Pardos, Z.A., Heffernan, N.T.: Determining the significance of item order in randomized problem sets. In: Proceedings of Educational Data Mining (2009)

  • Pardos, Z.A., Heffernan, N.T.: Modeling individualization in a Bayesian networks implementation of knowledge tracing. In: Proceedings of User Modeling, Adaptation, and Personalization, pp. 255–266. Springer (2010a)

  • Pardos, Z.A., Heffernan, N.T.: Navigating the parameter space of Bayesian knowledge tracing models: visualizations of the convergence of the expectation maximization algorithm. In: Proceedings of Educational Data Mining, pp. 161–170 (2010b)

  • Pardos, Z.A., Heffernan, N.T.: KT-IDEM: introducing item difficulty to the knowledge tracing model. In: Konstan, J.A., Conejo, R., Marzo, J.L. Oliver, N. (eds.) User Modeling, Adaption and Personalization: 19th International Conference, UMAP 2011, Girona, Spain, July 11–15, 2011. Proceedings, pp. 243–254. Springer, Berlin (2011)

  • Pardos, Z.A., Xu, Y.: Improving efficacy attribution in a self-directed learning environment using prior knowledge individualization. In: Proceedings of Learning Analytics and Knowledge, pp. 435–439. ACM (2016)

  • Pardos, Z.A., Heffernan, N.T., Anderson, B., Heffernan, C.L., Schools, W.P.: Using fine-grained skill models to fit student performance with Bayesian networks. In: Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.D. (eds.) Handbook of Educational Data Mining, pp. 417–426. Chapman & Hall (2010)

  • Pardos, Z.A., Gowda, S.M., Baker, R.S., Heffernan, N.T.: The sum is greater than the parts: ensembling models of student knowledge in educational software. ACM SIGKDD Explor. Newsl. 13(2), 37–44 (2012a)

    Article  Google Scholar 

  • Pardos, Z.A., Trivedi, S., Heffernan, N.T., Sárközy, G.N.: Clustered knowledge tracing. In: Proceedings of Intelligent Tutoring Systems, pp. 405–410. Springer (2012b)

  • Pardos, Z.A., Wang, Q.Y., Trivedi, S.: The real world significance of performance prediction. In: Proceedings of Educational Data Mining (2012c)

  • Pavlik, P.I., Anderson, J.R.: Practice and forgetting effects on vocabulary memory: an activation-based model of the spacing effect. Cogn. Sci. 29(4), 559–586 (2005)

    Article  Google Scholar 

  • Pavlik, P.I., Anderson, J.R.: Using a model to compute the optimal schedule of practice. J. Exp. Psychol. Appl. 14(2), 101 (2008)

    Article  Google Scholar 

  • Pavlik, Jr P., Bolster, T., Wu, S.M., Koedinger, K., Macwhinney, B.: Using optimally selected drill practice to train basic facts. In: Proceedings of Intelligent Tutoring Systems, pp. 593–602. Springer (2008)

  • Pavlik, P.I., Cen, H., Koedinger, K.R.: Performance factors analysis—a new alternative to knowledge tracing. In: Proceedings of Artificial Intelligence in Education, pp. 531–538. IOS Press (2009)

  • Pavlik, P.I., Brawner, K., Olney, A., Mitrovic, A.: A review of student models used in intelligent tutoring systems. In: Sottilare, R.A., Graesser, A., Hu, X., Holden, H. (eds) Design Recommendations for Intelligent Tutoring Systems, vol. 1, pp. 39–68. US Army Research Laboratory, Orlando, FL (2013)

  • Pelánek, R.: Application of time decay functions and Elo system in student modeling. In: Proceedings of Educational Data Mining, pp. 21–27 (2014)

  • Pelánek, R.: Metrics for evaluation of student models. J. Educ. Data Min. 7(2), 1–19 (2015a)

  • Pelánek, R.: Modeling students’ memory for application in adaptive educational systems. In: Proceedings of Educational Data Mining, pp. 480–483 (2015b)

  • Pelánek, R.: Applications of the Elo rating system in adaptive educational systems. Comput. Educ. 98, 169–179 (2016)

    Article  Google Scholar 

  • Pelánek, R., Jarušek, P.: Student modeling based on problem solving times. Int. J. Artif. Intell. Educ. 25(4), 493–519 (2015)

  • Pelánek, R., Řihák, J.: Properties and applications of wrong answers in online educational systems. In: Proceedings of Educational Data Mining (2016)

  • Pelánek, R., Řihák, J., Papoušek, J.: Impact of data collection on interpretation and evaluation of student model. In: Proceedings of Learning Analytics and Knowledge, pp. 40–47. ACM (2016)

  • Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L.J., Sohl-Dickstein, J.: Deep knowledge tracing. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 505–513. Curran Associates, Inc. (2015). http://papers.nips.cc/paper/5654-deep-knowledge-tracing.pdf

  • Qiu, Y., Qi, Y., Lu, H., Pardos, Z.A., Heffernan, N.T.: Does time matter? Modeling the effect of time with Bayesian knowledge tracing. In: EDM, pp. 139–148 (2011)

  • Reddy, S., Labutov, I., Banerjee, S., Joachims, T.: Unbounded human learning: optimal scheduling for spaced repetition. In: Proceedings of Knowledge Discovery and Data Mining. ACM (2016)

  • Ritter, S., Yudelson, M., Fancsali, S.E., Berman, S.R.: How mastery learning works at scale. In: Proceedings of Learning@Scale, pp. 71–79. ACM (2016)

  • Rollinson, J., Brunskill, E.: From predictive models to instructional policies. In: Proceedings of Educational Data Mining (2015)

  • Rubin, D.C., Hinton, S., Wenzel, A.: The precise time course of retention. J. Exp. Psychol. Learn. Mem. Cogn. 25(5), 1161 (1999)

    Article  Google Scholar 

  • Sao Pedro, M., Baker, R.S., Gobert, J.D.: Incorporating scaffolding and tutor context into Bayesian knowledge tracing to predict inquiry skill acquisition. In: Proceedings of Educational Data Mining, pp. 185–192 (2013a)

  • Sao Pedro, M.A., Baker, R.S., Gobert, J.D.: What different kinds of stratification can reveal about the generalizability of data-mined skill assessment models. In: Proceedings of Learning Analytics and Knowledge, pp. 190–194. ACM (2013b)

  • Scheines, R., Silver, E., Goldin, I.: Discovering prerequisite relationships among knowledge components. In: Proceedings of Educational Data Mining, pp. 355–356 (2014)

  • Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., Dennison, D.: Hidden technical debt in machine learning systems. In: Proceedings of the 28th International Conference on Neural Information Processing Systems. NIPS’15, Montreal, Canada, pp. 2503–2511. MIT Press, Cambridge, MA (2015). http://dl.acm.org/citation.cfm?id=2969442.2969519

  • Sense, F., Behrens, F., Meijer, R.R., Rijn, H.: An individual’s rate of forgetting is stable over time but differs across materials. Top. Cogn. Sci. 8(1), 305–321 (2016)

    Article  Google Scholar 

  • Smith, A.C., Frank, L.M., Wirth, S., Yanike, M., Hu, D., Kubota, Y., Graybiel, A.M., Suzuki, W.A., Brown, E.N.: Dynamic analysis of learning in behavioral experiments. J. Neurosci. 24(2), 447–461 (2004)

    Article  Google Scholar 

  • Stamper, J.C., Koedinger, K.R., McLaughlin, E.A.: A comparison of model selection metrics in datashop. In: Proceedings of Educational Data Mining, pp. 284–287 (2013)

  • Stone, M.: An asymptotic equivalence of choice of model by cross-validation and Akaike’s criterion. J. R. Stat. Soc. Ser. B (Methodol.) 39(1), 44–47 (1977)

  • Straatemeier, M.: Math Garden: A New Educational and Scientific Instrument. PhD thesis, Universiteit van Amsterdam, Faculty of Social and Behavioural Sciences (2014)

  • Streeter, M.: Mixture modeling of individual learning curves. In: Proceedings of Educational Data Mining (2015)

  • Tang, S., McBride, E., Gogel, H., Pardos, Z.A.: Item ordering effects with qualitative explanations using online adaptive tutoring data. In: Proceedings of Learning@Scale, pp. 313–316. ACM (2015)

  • Tatsuoka, K.K.: Rule space: an approach for dealing with misconceptions based on item response theory. J. Educ. Meas. 20(4), 345–354 (1983)

    Article  Google Scholar 

  • Thai-Nghe, N., Drumond, L., Horváth, T., Krohn-Grimberghe, A., Nanopoulos, A., Schmidt-Thieme, L.: Factorization techniques for predicting student performance. In: Santos, O., Boticario, J. (eds.) Educational Recommender Systems and Technologies: Practices and Challenges, pp. 129–153. IGI Global, Hershey, PA (2012)

  • Toscher, A., Jahrer, M.: Collaborative filtering applied to educational data mining. In: KDD cup (2010)

  • Trivedi, S., Pardos, Z.A., Heffernan, N.T.: Clustering students to generate an ensemble to improve standard test score predictions. In: Proceedings of Artificial Intelligence in Education, pp. 377–384. Springer (2011)

  • Valdés Aguirre, B., Ramírez Uresti, J.A., Boulay, B.D.: An analysis of student model portability. Int. J. Artif. Intell. Educ. 26(3), 932–974 (2016)

    Article  Google Scholar 

  • Van Der Linden, W.: Conceptual issues in response-time modeling. J. Educ. Meas. 46(3), 247–272 (2009)

    Article  MathSciNet  Google Scholar 

  • Van Inwegen, E.G., Adjei, S.A., Wang, Y., Heffernan, N.T.: Using partial credit and response history to model user knowledge. In: Proceedings of Educational Data Mining (2015)

  • Vanlehn, K.: The behavior of tutoring systems. Int. J. Artif. Intell. Educ. 16(3), 227–265 (2006)

    Google Scholar 

  • Vanlehn, K., Lynch, C., Schulze, K., Shapiro, J.A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., Wintersgill, M.: The Andes physics tutoring system: lessons learned. Int. J. Artif. Intell. Educ. 15(3), 147–204 (2005)

    MATH  Google Scholar 

  • van de Sande, B.: Properties of the Bayesian knowledge tracing model. J. Educ. Data Min. 5(2), 1 (2013)

    Google Scholar 

  • Řihák, J.: Use of time information in models behind adaptive system for building fluency in mathematics. In: Proceedings of Educational Data Mining, Doctoral Consortium (2015)

  • Řihák, J., Pelánek, R.: Choosing a student model for a real world application. In: Building ITS Bridges Across Frontiers (ITS Workshop) (2016)

  • Wang, Y., Beck, J.: Class vs. student in a Bayesian network student model. In: Proceedings of Artificial Intelligence in Education, pp. 151–160. Springer (2013a)

  • Wang, Y., Heffernan, N.: Extending knowledge tracing to allow partial credit: using continuous versus binary nodes. In: Proceedings of Artificial Intelligence in Education, pp. 181–188. Springer (2013b)

  • Wang, X., Berger, J.O., Burdick, D.S., et al.: Bayesian analysis of dynamic item response models in educational testing. Ann. Appl. Stat. 7(1), 126–153 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  • Wang, Y., Heffernan, N.T., Heffernan, C.: Towards better affect detectors: effect of missing skills, class features and common wrong answers. In: Proceedings of Learning Analytics and Knowledge, pp. 31–35. ACM (2015)

  • Wauters, K., Desmet, P., Van Den Noortgate, W.: Item difficulty estimation: an auspicious collaboration between data and judgment. Comput. Educ. 58(4), 1183–1193 (2012)

    Article  Google Scholar 

  • White, K.G.: Forgetting functions. Anim. Learn. Behav. 29(3), 193–207 (2001)

    Article  Google Scholar 

  • Xu, Y., Mostow, J.: Comparison of methods to trace multiple subskills: is LR-DBN best? In: Proceedings of Educational Data Mining (2012)

  • Yudelson, M.V., Koedinger, K.R.: Estimating the benefits of student model improvements on a substantive scale. In: Educational Data Mining Workshops Proceedings (2013a)

  • Yudelson, M.V., Koedinger, K.R., Gordon, G.J.: Individualized Bayesian knowledge tracing models. In: Proceedings of Artificial Intelligence in Education, pp. 171–180. Springer (2013b)

Download references

Acknowledgements

The author thanks members of the Adaptive Learning group at Masaryk University for valuable discussions and for their work on the development of practical educational system, which inspired many aspects of this overview. The author also thanks anonymous reviewers for their valuable comments, which significantly improved the structure and presentation of the paper.

Author information

Authors and Affiliations

  1. Faculty of Informatics, Masaryk University, Brno, Czechia

    Radek Pelánek

Authors
  1. Radek Pelánek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Radek Pelánek.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pelánek, R. Bayesian knowledge tracing, logistic models, and beyond: an overview of learner modeling techniques. User Model User-Adap Inter 27, 313–350 (2017). https://doi.org/10.1007/s11257-017-9193-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-017-9193-2

Keywords

Search

Navigation