Smart Learner Support Through Semi-automatic Feedback

  • Paul Libbrecht
  • Wolfgang Müller
  • Sandra Rebholz
Part of the Lecture Notes in Educational Technology book series (LNET)


Learning tools that produce automated feedback are becoming commodity, from multiple-choice questions to intelligent tutoring systems, and from direct manipulations to exploratory environments. In this paper, we argue how such learning tools can become smart by applying the semi-automatic feedback paradigm where the teacher complements the feedback capabilities of the learning tool. The approach employs analytics as a central awareness mechanism for teacher to provide guidance in a way that is most relevant to the past usage of the learning tool, including what it provided as feedback. The SMALA approach we describe is realized as an open-source software which has been evaluated in a number of undergraduate studies, leveraging the default learning management system’s architecture of the universities. This software delivers visualizations of the activities at each level of interaction (the group of all users, the group of users in a classroom, the individual learner). The different levels support the teacher in adjusting his or her strategy and respond to individual requests.


Semi-automatic Feedback Formative Assessment Teaching analytics 


  1. Ainsworth, L., & Viegut, D. (2006). Common formative assessments. Thousand Oaks, CA: Corwin Press.Google Scholar
  2. Anderson, J. R., Boyle, C. F., Corbett, A. T., & Lewis, M. W. (1990). Cognitive modeling and intelligent tutoring. Artificial Intelligence, 42, 7–49.CrossRefGoogle Scholar
  3. Anderson, J. R., & Pelletier, R. (1991). A developmental system for model-tracing tutors. In The International Conference on the Learning Sciences (pp. 1–8), Charlottesville, Virginia.Google Scholar
  4. Arnold, K. E. (2010). Signals: Applying academic analytics. Educause Quarterly, 33(1).
  5. Bannan-Ritland, B. (2003). The role of design in research: The integrative learning design framework. Educational Researcher, 32(1), 21–24.CrossRefGoogle Scholar
  6. Bescherer, C., Kortenkamp, U., Müller, W., & Spannagel, C. (2009). Intelligent computer-aided assessment in math classrooms. In A. E. A. McDougall (Ed.), Researching IT in education: Theory, practice and future directions (pp. 200–205). Milton Park, New York: Routledge.Google Scholar
  7. Bescherer, C., & Spannagel, C. (2009). Design patterns for the use of technology in introductory mathematics tutorials. In A. Tatnall & A. Jones (Eds.), Education and technology for a better world (pp. 427–435). Berlin: Springer.CrossRefGoogle Scholar
  8. Bescherer, C., Herding, D., Kortenkamp, U., Müller, W., & Zimmermann, M. (2011). e-learning tools with intelligent assessment and feedback. In K. Sabine Graf, F. Lin, & R. McGreal (Eds.), e-learning tools with intelligent assessment and feedback (pp. 151–163). Hershey: IGI Global.Google Scholar
  9. Brown, S. (2004). Assessment for learning. Learning and Teaching in Higher Education 1, 81–89.Google Scholar
  10. Buchem, I., & Pérez-Sanagustín, M. (2013). Personal learning environments in smart cities: Current approaches and future scenarios. eLearning Papers, 35.
  11. Butler, A. C., Karpicke, J. D., & Roediger III, H. L. R. (2007). The effect of type and timing of feedback on learning from multiple-choice tests. Journal of Experimental Psychology: Applied, 13(4), 273–281.
  12. Carnegie Mellon University (2013). CTAT—Cognitive Tutor Authoring Tools. Retrieved January 3rd, 2014 from
  13. Clark, R. C., & Mayer, R. E. (2011). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning (3rd ed.). San Francisco, CA: Wiley.CrossRefGoogle Scholar
  14. Corbett, A. T., Koedinger, K. R., & Anderson, J. R. (1997a). Chapter 37: Intelligent tutoring systems. In M. Helander, T. K. Landauer, & P. Prabhu (Eds.), Handbook of human–computer interaction (pp. 849–874). Elsevier Science B. V, Amsterdam.Google Scholar
  15. Corbett, A. T., Koedinger, K. R., & Anderson, J. R. (1997b). Intelligent tutoring systems. In M. Helander, T. K. Landauer, & P. Prabhu (Eds.), Handbook of human-computer interaction. Amsterdam: Elsevier Science B. V.Google Scholar
  16. Dodds, P., & Fletcher, J. D. (2004). Opportunities for new “smart’’ learning environments enabled by next generation web capabilities. Journal of Educational Multimedia and Hypermedia, 13(4), 391–404.Google Scholar
  17. Essa, A., & Ayad, H. (2012). Improving student success using predictive models and data visualisations. Research in Learning Technology, 20(0).
  18. Feng, M., Heffernan, N. T., & Koedinger, K. R. (2009). Addressing the assessment challenge in an Intelligent Tutoring System that tutors as it assesses. User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI Journal), 19(3), 243–266.
  19. Guéraud, V., & Cagnat, J.-M. (2006). Automatic semantic activity monitoring of distance learners guided by pedagogical scenarios. In Proceedings of EC-TEL (pp. 476–481). Springer, Berlin.Google Scholar
  20. Hattie, J. (2008). Visible learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. London: Routledge.Google Scholar
  21. Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 81–112(1).
  22. Heffernan, N., Heffernan, C., Dietz, K., Soffer, D., Pellegrino, J. W., & Goldman, S. R. (2012). Improving mathematical learning outcomes through automatic reassessment and relearning. In AERA Annual Meeting 2012.Google Scholar
  23. Jovanovic, J., Devedzic, V., Gasevic, D., Hatala, M., Eap, T., Richards, G., et al. (2007). Using semantic web technologies to analyze learning content. IEEE Internet Computing, 11(5), 45–53.CrossRefGoogle Scholar
  24. Jovanovic, J., Gasevic, D., Brooks, C., Devedzic, V., Hatala, M., Eap, T., et al. (2008). LOCO-analyst: Semantic web technologies in learning content usage analysis. International Journal of Continuing Engineering Education and Life Long Learning, 18(1), 54–76.CrossRefGoogle Scholar
  25. Koedinger, K. R., Aleven, V. A. W. M. M., & Heffernan, N. T. (2003). Toward a rapid development environment for cognitive tutors. In Proceedings of the 11th International Conference on Artificial Intelligence in Education, AI-ED 2003, Amsterdam, 2003 (pp. 455–457). IOS Press.Google Scholar
  26. Koedinger, K. R., Aleven, V., Heffernan, T., McLaren, B., & Hockenberry, M. (2004). Opening the door to non-programmers: authoring intelligent tutor behavior by demonstration. In Proceedings of 7th Annual Intelligent Tutoring Systems Conference, Maceio, Brazil, 2004.Google Scholar
  27. Kulik, J. A., & Kulik, C.-L. C. (1988). Timing of Feedback and Verbal Learning. Review of Educational Research58, 79–97.Google Scholar
  28. Mikulecký, P. (2012). Smart environments for smart learning. In Proceedings of DIVAI 2012—9th International Scientific Conference on Distance Learning in Applied Informatics (pp. 213–222).Google Scholar
  29. Müller, W., Bescherer, C., Kortenkamp, U., & Spannagel, C. (2006). Intelligent computer-aided assessment in math classrooms: State-of-the-art and perspectives. In Proceedings of the IFIP WG 3.1, 3.3, and 3.5 Joint Conference Imagining the future for ICT and Education, Alesund, Norway, 2006.Google Scholar
  30. NEXT-TELL (2013). Call For Papers 3rd International Workshop on Teaching Analytics (IWTA) at E-TEL 2013. Retrieved from
  31. Ravet, S. (2009). Building my ePortfolio, an ePortolano publication. Available from Accessed on 15 May 2014.
  32. Rebholz, S., Libbrecht, P., & Müller, W. (2012). Learning analytics as an investigation tool for teaching practicioners. In Proceedings of the Workshop Towards Theory and Practice of Teaching Analytics 2012 (TaPTA-2012), Saarbrücken, Germany, 2012. CEUR-WS.Google Scholar
  33. Rebholz, S., Libbrecht, P., & Müller, W. (2013). Interactive visual exploration of learning data: the role of teachers as learning analysts. In Proceedings of 10th IFIP World Conference on Computers in Education (WCCE 2013), Torun, Poland, 2013.Google Scholar
  34. Rebholz, S., & Zimmermann, M. (2013). Applying computer-aided intelligent assessment in the context of mathematical induction. In Z. Pan, A.-D. Cheok, W. Müller, I. Iurgel, P. Petta, & B. Urban (eds), Transactions on Edutainment X (pp. 191–201), Springer, Berlin, LNCS 7775.
  35. Reeves, T. C., Herrington, J., & Oliver, R. (2005). Design research: A socially responsible approach to instructional technology research in higher education. Journal of Computing in Higher Education, 16(2), 97–116.CrossRefGoogle Scholar
  36. Reusser, K. (1993). Tutoring systems and pedagogical theory: Representational tools for understanding, planning, and reflection in problem solving. In S. P. Lajoie & S. J. Derry (Eds.), Computers as cognitive tools (pp. 143–177). London: Routledge.Google Scholar
  37. Siemens, G. (2011). What are learning analytics? Retrieved from
  38. Singh, R., Saleem, M., Pradhan, P., Heffernan, C., Heffernan, N. T., Razzaq, L., Dailey, M. D., O’Connor, C., & Mulcahy, C. (2011). Feedback during web-based homework: The role of hints. In Artificial Intelligence in Education—15th International Conference, AIED 2011 (pp. 328–336). Springe, Berlin/Heidelberg.Google Scholar
  39. Sleeman, D., B. J. S. (1982). Introduction: Intelligent tutoring systems. In D. S. J. S. Brown (Ed.), Intelligent tutoring systems (pp. 1–11). New York: Academic Press.Google Scholar
  40. Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research78(1),153–189.
  41. Tabach, M. (2013). Developing a general framework for instrumental orchestration. In Conference on European Research on Mathematics Education (CERME8).Google Scholar
  42. VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.
  43. Vatrapu, R., Teplovs, C., Fujita, N., & Bull, S. (2011). Towards visual analytics for teachers’ dynamic diagnostic pedagogical decision-making. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 93–98), New York, NY, USA, 2011. ACM.Google Scholar
  44. Worcester Polytechnic Institute (2013). Assistments. Retrieved January, 3rd, 2014 from
  45. Yesner, R. (2012). The next-generation classroom: smart, interactive and connected learning environments. IDC Government Insights.Google Scholar
  46. Zhang, H. & Almeroth, K. (2010). Moodog: tracking student activity in online course management systems. Journal of Interactive Learning Research, 21(3), 407–429.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Computer ScienceUniversity of Education WeingartenWeingartenGermany
  2. 2.Media Education and Visualization GroupUniversity of Education WeingartenWeingartenGermany

Personalised recommendations