Stealth Assessment in ITS - A Study for Developmental Dyscalculia

  • Severin Klingler
  • Tanja Käser
  • Alberto-Giovanni Busetto
  • Barbara Solenthaler
  • Juliane Kohn
  • Michael von Aster
  • Markus Gross
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9684)


Intelligent tutoring systems are adapting the curriculum to the needs of the student. The integration of stealth assessments of student traits into tutoring systems, i.e. the automatic detection of student characteristics has the potential to refine this adaptation. We present a pipeline for integrating automatic assessment seamlessly into a tutoring system and apply the method to the case of developmental dyscalculia (DD). The proposed classifier is based on user inputs only, allowing non-intrusive and unsupervised, universal screening of children. We demonstrate that interaction logs provide enough information to identify children at risk of DD with high accuracy and validity and reliability comparable to traditional assessments. Our model is able to adapt the duration of the screening test to the individual child and can classify a child at risk of DD with an accuracy of 91 % after 11 min on average.


Automatic assessment Feature processing Bayesian network Pairwise clustering Computer-based screening Dyscalculia 



This work was supported by ETH Grant ETH-23 13-2.


  1. 1.
    Arroyo, I., Woolf, B.P.: Inferring learning and attitudes from a bayesian network of log file data. In: Proceedings of AIED, pp. 33–40 (2005)Google Scholar
  2. 2.
    von Aster, M.G., Shalev, R.: Number development and developmental dyscalculia. Dev. Med. Child Neurol. 49, 868–873 (2007)CrossRefGoogle Scholar
  3. 3.
    von Aster, M.G.: Developmental cognitive neuropsychology of number processing and calculation: varieties of developmental dyscalculia. Eur. Child Adolesc. Psychiatry 9, S41–S57 (2000)CrossRefGoogle Scholar
  4. 4.
    von Aster, M., Zulauf, M.W., Horn, R.: Neuropsychologische Testbatterie für Zahlenverarbeitung und Rechnen bei Kindern: ZAREKI-R. Pearson, Frankfurt (2006)Google Scholar
  5. 5.
    Attali, Y.: Reliability-based feature weighting for automated essay scoring. Appl. Psychol. Meas. 39(4), 303–313 (2015)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Baker, R.S., Corbett, A.T., Koedinger, K.R.: Detecting student misuse of intelligent tutoring systems. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 531–540. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Beacham, N., Trott, C.: Screening for dyscalculia within HE. MSOR 5, 1–4 (2005)Google Scholar
  8. 8.
    Beck, J.E.: Engagement tracing: using response times to model student disengagement. In: Proceedings of AIED, pp. 88–95 (2005)Google Scholar
  9. 9.
    Beck, J.E., Gong, Y.: Wheel-spinning: students who fail to master a skill. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 431–440. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  10. 10.
    Butterworth, B.: Dyscalculia Screener. Nelson Publishing Company Ltd., London (2003)Google Scholar
  11. 11.
    Butterworth, B., Varma, S., Laurillard, D.: Dyscalculia: from brain to education. Science 332(6033), 1049–1053 (2011)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Cisero, C., Royer, J., Marchant, H., Jackson, S.: Can the computer-based academic assessment system (CAAS) be used to diagnose reading disability in college students? J. Educ. Psychol. 89(4), 599–620 (1997)CrossRefGoogle Scholar
  13. 13.
    Kadosh, R.C., Kadosh, K.C., Schuhmann, T., Kaas, A., Goebel, R., Henik, A., Sack, A.T.: Virtual dyscalculia induced by parietal-lobe TMs impairs automatic magnitude processing. Current Biol. 17, 689–693 (2007)CrossRefGoogle Scholar
  14. 14.
    Cooper, D.G., Muldner, K., Arroyo, I., Woolf, B.P., Burleson, W.: Ranking feature sets for emotion models used in classroom based intelligent tutoring systems. In: De Bra, P., Kobsa, A., Chin, D. (eds.) UMAP 2010. LNCS, vol. 6075, pp. 135–146. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  15. 15.
    Desoete, A., Grégoire, J.: Numerical competence in young children and in children with mathematics learning disabilities. Learn. Individ. Differ. 16(4), 351–367 (2006)CrossRefGoogle Scholar
  16. 16.
    Esser, G., Wyschkon, A., Ballaschk, K.: BUEGA: Basisdiagnostik Umschriebener Entwicklungsstörungen im Grundschulalter. Hogrefe, Göttingen (2008)Google Scholar
  17. 17.
    Geary, D.C., Brown, S.C., Samaranayake, V.A.: Cognitive addition: a short longitudinal study of strategy choice and speed-of-processing differences in normal and mathematically disabled children. Dev. Psychol. 27(5), 787–797 (1991)CrossRefGoogle Scholar
  18. 18.
    Graf, E.A., Fife, J.H.: Difficulty modeling and automatic generation of quantitative items: recent advances and possible next steps. In: Gierl, M.J., Haladyna, T.M. (eds.) Automatic Item Generation: Theory and Practice, pp. 157–179. Routledge, London (2013)Google Scholar
  19. 19.
    Haffner, J., Baro, K., Parzer, P., Resch, F.: Heidelberger Rechentest (HRT): Erfassung mathematischer Basiskomptenzen im Grundschulalter. Hogrefe Verlag, Goettingen (2005)Google Scholar
  20. 20.
    Hao, J., Shu, Z., Davier, A.: Analyzing process data from game/scenario- based tasks: an edit distance approach. JEDM 7, 33–50 (2015)Google Scholar
  21. 21.
    Hofmann, T., Buhmann, J.M.: Pairwise data clustering by deterministic annealing. IEEE Trans. Pattern Anal. Mach. Intell. 19(1), 1–14 (1997)CrossRefGoogle Scholar
  22. 22.
    Käser, T., Baschera, G.M., Kohn, J., Kucian, K., Richtmann, V., Grond, U., Gross, M., von Aster, M.: Design and evaluation of the computer-based training program calcularis for enhancing numerical cognition. Front. Dev. Psychol. 4, 489 (2013)Google Scholar
  23. 23.
    Käser, T., Busetto, A.G., Solenthaler, B., Kohn, J., von Aster, M., Gross, M.: Cluster-based prediction of mathematical learning patterns. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 389–399. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  24. 24.
    Käser, T.: Modeling and Optimizing Computer-Assisted Mathematics Learning in Children. Ph.D. thesis, Diss., ETH Zürich, Nr. 22145 (2014)Google Scholar
  25. 25.
    Kucian, K., Grond, U., Rotzer, S., Henzi, B., Schönmann, C., Plangger, F., Gälli, M., Martin, E., von Aster, M.: Mental number line training in children with developmental dyscalculia. NeuroImage 57(3), 782–795 (2011)CrossRefGoogle Scholar
  26. 26.
    Landerl, K., Bevan, A., Butterworth, B.: Developmental dyscalculia and basic numerical capacities: a study of 8-9-year-old students. Cognition 93, 99–125 (2004)CrossRefGoogle Scholar
  27. 27.
    Noël, M.P., Rousselle, L.: Developmental changes in the profiles of dyscalculia: an explanation based on a double exact-and-approximate number representation model. Front. Hum. Neurosci. 5, 165 (2011)CrossRefGoogle Scholar
  28. 28.
    Ostad, S.A.: Developmental differences in addition strategies: a comparison of mathematically disabled and mathematically normal children. Br. J. Educ. Psychol. 67, 345–357 (1997)CrossRefGoogle Scholar
  29. 29.
    Reshef, D.N., Reshef, Y.A., Finucane, H.K., Grossman, S.R., McVean, G., Turnbaugh, P.J., Lander, E.S., Mitzenmacher, M., Sabeti, P.C.: Detecting novel associations in large data sets. Science 334(6062), 1518–1524 (2011)CrossRefGoogle Scholar
  30. 30.
    Shalev, R., von Aster, M.G.: Identification, classification, and prevalence of developmental dyscalculia. Encyclopedia of Language and Literacy, Development, pp. 1–9 (2008)Google Scholar
  31. 31.
    Shute, V.J.: Stealth assessment in computer-based games to support learning. In: Computer Games and Instruction (2011)Google Scholar
  32. 32.
    Von Aster, M., Rauscher, L., Kucian, K., Käser, T., McCaskey, U., Kohn, J.: Calcularis - evaluation of a computer-based learning program for enhancing numerical cognition for children with developmental dyscalculia. In: 62nd Annual Meeting of the American Academy of Child and Adolescent Psychiatry (2015)Google Scholar
  33. 33.
    Woolger, C.: Wechsler intelligence scale for children-third edition (WISC-III). In: Dorfman, W.I., Hersen, M. (eds.) Understanding Psychological Assessment. Perspectives on Individual Differences, pp. 219–233. Springer, New York (2001)CrossRefGoogle Scholar
  34. 34.
    Zhang, H.: The optimality of naive bayes. In: Proceedings of FLAIRS (2004)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Severin Klingler
    • 1
  • Tanja Käser
    • 1
  • Alberto-Giovanni Busetto
    • 2
  • Barbara Solenthaler
    • 1
  • Juliane Kohn
    • 3
  • Michael von Aster
    • 3
    • 4
    • 5
  • Markus Gross
    • 1
  1. 1.Department of Computer ScienceETH ZurichZürichSwitzerland
  2. 2.Department of Electrical and Computer EngineeringUniversity of CaliforniaSanta BarbaraUSA
  3. 3.Department of PsychologyUniversity of PotsdamPotsdamGermany
  4. 4.Center for MR-ResearchUniversity Children’s Hospital ZurichZürichSwitzerland
  5. 5.Department of Child Adolescent PsychiatryDRK Kliniken Berlin WestendBerlinGermany

Personalised recommendations