Automated Feedback on the Structure of Hypothesis Tests

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11626)


Hypothesis testing is a challenging topic for many students in introductory university statistics courses. In this paper we explore how automated feedback in an Intelligent Tutoring System can foster students’ ability to carry out hypothesis tests. Students in an experimental group (N = 163) received elaborate feedback on the structure of the hypothesis testing procedure, while students in a control group (N = 151) only received verification feedback. Immediate feedback effects were measured by comparing numbers of attempted tasks, complete solutions, and errors between the groups, while transfer of feedback effects was measured by student performance on follow-up tasks. Results show that students receiving elaborate feedback solved more tasks and made fewer errors than students receiving only verification feedback, which suggests that students benefited from the elaborate feedback.


Domain reasoner Hypothesis testing Intelligent tutoring systems Statistics education 



We thank teachers Jeltje Wassenberg-Severijnen and Corine Geurts for their collaboration in designing teaching tasks and delivering the course. Furthermore, we thank Noeri Huisman, Martijn Fleuren, Peter Boon and Wim van Velthoven who helped with developing the domain reasoner.


  1. 1.
    Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167–207 (1995)CrossRefGoogle Scholar
  2. 2.
    Carver, R., et al.: Guidelines for Assessment and Instruction in Statistics Education College Report 2016. American Statistical Association (2016).
  3. 3.
    Drijvers, P., Boon, P., Doorman, M., Bokhove, C., Tacoma, S.: Digital design: RME principles for designing online tasks. In: Margolinas, C. (ed.) Proceedings of ICMI Study 22 Task Design in Mathematics Education, pp. 55–62. ICMI, Clermont-Ferrand (2013)Google Scholar
  4. 4.
    Falk, R., Greenbaum, C.W.: Significance tests die hard: the amazing persistence of a probabilistic misconception. Theory Psychol. 5(1), 75–98 (1995)CrossRefGoogle Scholar
  5. 5.
    Field, A.: Discovering Statistics Using SPSS, 3rd edn. Sage Publications, London (2009)zbMATHGoogle Scholar
  6. 6.
    Garfield, J.B., Ben-Zvi, D., Chance, B., Medina, E., Roseth, C., Zieffler, A.: Learning to reason about statistical inference. In: Developing Students’ Statistical Reasoning, pp. 261–288. Springer, Dordrecht (2008).
  7. 7.
    Goguadze, G.: ActiveMath - generation and reuse of interactive exercises using domain reasoners and automated tutorial strategies (Doctoral dissertation, Saarland University, Saarbrücken, Germany) (2011).
  8. 8.
    Heeren, B., Jeuring, J.: Feedback services for stepwise exercises. Sci. Comput. Program. 88, 110–129 (2014). Scholar
  9. 9.
    Kodaganallur, V., Weitz, R.R., Rosenthal, D.: A comparison of model-tracing and constraint-based intelligent tutoring paradigms. Int. J. Artif. Intell. Educ. 15(2), 117–144 (2005)Google Scholar
  10. 10.
    Mitrovic, A., Martin, B., Suraweera, P.: Intelligent tutors for all: the constraint-based approach. IEEE Intell. Syst. 4, 38–45 (2007)CrossRefGoogle Scholar
  11. 11.
    Nwana, H.S.: Intelligent tutoring systems: an overview. Artif. Intell. Rev. 4(4), 251–277 (1990)CrossRefGoogle Scholar
  12. 12.
    Shute, V.J.: Focus on formative feedback. Rev. Educ. Res. 78(1), 153–189 (2008)CrossRefGoogle Scholar
  13. 13.
    Tacoma, S., Heeren, B., Jeuring, J., Drijvers, P.: Automated feedback on the structure of hypothesis tests. In: Jankvist, U.T., van den Heuvel-Panhuizen, M., Veldhuis, M. (eds.) Proceedings of the Eleventh Congress of the European Society for Research in Mathematics Education, pp. xx–yy. Freudenthal Group & Freudenthal Institute, Utrecht University and ERME, Utrecht, the Netherlands (2019)Google Scholar
  14. 14.
    Vallecillos, A.: Some empirical evidence on learning difficulties about testing hypotheses. In: Bulletin of the International Statistical Institute: Proceedings of the Fifty-Second Session of the International Statistical Institute, vol. 58, pp. 201–204 (1999)Google Scholar
  15. 15.
    Van der Kleij, F., Feskens, R., Eggen, T.: Effects of feedback in a computer-based learning environment on students’ learning outcomes. A meta-analysis. Rev. Educ. Res. 85(4), 475–511 (2015). Scholar
  16. 16.
    VanLehn, K.: The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educ. Psychol. 46(4), 197–221 (2011). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Utrecht UniversityUtrechtThe Netherlands
  2. 2.Open University of the NetherlandsHeerlenThe Netherlands

Personalised recommendations