EC-TEL 2016: Adaptive and Adaptable Learning pp 340-354 | Cite as

How Teachers Use Data to Help Students Learn: Contextual Inquiry for the Design of a Dashboard

  • Françeska Xhakaj
  • Vincent Aleven
  • Bruce M. McLaren
Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9891)

Abstract

Although learning with Intelligent Tutoring Systems (ITS) has been well studied, little research has investigated what role teachers can play, if empowered with data. Many ITSs provide student performance reports, but they may not be designed to serve teachers’ needs well, which is important for a well-designed dashboard. We investigated what student data is most helpful to teachers and how they use data to adjust and individualize instruction. Specifically, we conducted Contextual Inquiry interviews with teachers and used Interpretation Sessions and Affinity Diagramming to analyze the data. We found that teachers generate data on students’ concept mastery, misconceptions and errors, and utilize data provided by ITSs and other software. Teachers use this data to drive instruction and remediate issues on an individual and class level. Our study uncovers how data can support teachers in helping students learn and provides a solid foundation and recommendations for designing a teacher’s dashboard.

Keywords

Intelligent Tutoring Systems Dashboard Contextual Inquiry 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

We thank Gail Kusbit, Carnegie Learning, Jae-Won Kim, and the teachers we interviewed for their help with this project. NSF Award #1530726 supported this work.

References

  1. 1.
    Abel, T.D., Evans, M.A.: Cross-disciplinary participatory & contextual design research: creating a teacher dashboard application. Interact. Des. Archit. J. 19, 63–76 (2013)Google Scholar
  2. 2.
    Aleven, V., McLaren, B.M., Sewall, J., van Velsen, M., Popsecu, O., Demi, S., Ringenberg, M., Koedinger, K.R.: Example-tracing tutors: Intelligent tutor development for non-programmers. Int. J. Artif. Intell. Educ. 26, 224–269 (2016)CrossRefGoogle Scholar
  3. 3.
    Ali, L., Hatala, M., Gasevic, D., Jovanovic, J.: A qualitative evaluation of evolution of a learning analytics tool. Comput. Educ. 58(1), 470–489 (2012)CrossRefGoogle Scholar
  4. 4.
    van Alphen, E., Bakker, S.: Lernanto: an ambient display to support differentiated instruction. In: Proceedings of the CSCL 2015 Conference on Computer Supported Collaborative Learning, pp. 759–760 (2015)Google Scholar
  5. 5.
    Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167–207 (1995)CrossRefGoogle Scholar
  6. 6.
    Arnold, K.E., Pistilli, M.D.: Course signals at Purdue: using learning analytics to increase student success. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, pp. 267–270. ACM (2012)Google Scholar
  7. 7.
    Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gasevic, D., Mulder, R., Williams, D., Dawson, S., Lockyer, L.: A conceptual framework linking learning design with learning analytics. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 329–338 (2016)Google Scholar
  8. 8.
    Beyer, H., Holtzblatt, K.: Contextual Design: Defining Customer-Centered Systems. Morgan Kaufmann, San Francisco (1997)Google Scholar
  9. 9.
    Bull, S., Kay, J.: Open learner models. In: Nkambou, R., Bourdeau, J., Mizoguchi, R. (eds.) Advances in Intelligent Tutoring Systems. SCI, vol. 308, pp. 301–322. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
  11. 11.
    Hamilton, L., Halverson, R., Jackson, S.S., Mandinach, E., Supovitz, J.A., Wayman, J.C.: Using student achievement data to support instructional decision making, IES Practice Guide, NCEE 2009-4067, National Center for Education Evaluation and Regional Assistance (2009)Google Scholar
  12. 12.
    Heffernan, N.T., Heffernan, C.L.: The ASSISTments ecosystem: building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. Int. J. Artif. Intell. Educ. 24(4), 470–497 (2014)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Kamin, S., Capitanu, B., Twidale, M., Peiper, C.: A teacher’s dashboard for a high school algebra class. In: Reed, R.H., Berque, D.A., Prey, J.C. (eds.) The Impact of Tablet PCs and Pen-based Technology on Education: Evidence and Outcomes, pp. 63–72. Purdue University Press, West Lafayette (2008)Google Scholar
  14. 14.
    Kelly, K., Heffernan, N., Heffernan, C., Goldman, S., Pellegrino, J., Soffer Goldstein, D.: Estimating the effect of web-based homework. In: Lane, H., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 824–827. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  15. 15.
    Kulik, C.-L.C., Kulik, J.A., Bangert-Drowns, R.L.: Effectiveness of mastery learning programs: a meta-analysis. Rev. Educ. Res. 60(2), 265–299 (1990)CrossRefGoogle Scholar
  16. 16.
    Kulik, J.A., Fletcher, J.D.: Effectiveness of intelligent tutoring systems: a meta-analytic review. Rev. Educ. Res. 86(1), 42–78 (2015)Google Scholar
  17. 17.
    Long, Y., Aleven, V.: Mastery-oriented shared student/system control over problem selection in a linear equation tutor. In: Micarelli, A., Stamper, J., Panourgia, K., Krouwel, M.R. (eds.) ITS 2016. LNCS, vol. 9684, pp. 90–100. Springer, Heidelberg (2016). doi: 10.1007/978-3-319-39583-8_9 CrossRefGoogle Scholar
  18. 18.
    Lovett, M., Meyer, O., Thille, C.: The open learning initiative: measuring the effectiveness of the OLI statistics course in accelerating student learning. J. Interact. Media Educ. 2008(1), Art. 13 (2008)Google Scholar
  19. 19.
    Ma, W., Adesope, O.O., Nesbit, J.C., Liu, Q.: Intelligent tutoring systems and learning outcomes: a meta-analysis. J. Educ. Psychol. 106(4), 901–918 (2014)CrossRefGoogle Scholar
  20. 20.
    Maldonado, R.M., Kay, J., Yacef, K., Schwendimann, B.: An interactive teacher’s dashboard for monitoring groups in a multi-tabletop learning environment. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 482–492. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  21. 21.
    Mazza, R., Dimitrova, V.: CourseVis: a graphical student monitoring tool for supporting instructors in web-based distance courses. Int. J. Hum. Comput. Stud. 65(2), 125–139 (2007)CrossRefGoogle Scholar
  22. 22.
    McLaren, B.M., Scheuer, O., Miksatko, J.: Supporting collaborative learning and e-discussions using artificial intelligence techniques. Int. J. Artif. Intell. Educ. 20(1), 1–46 (2010)Google Scholar
  23. 23.
    Steenbergen-Hu, S., Cooper, H.: A meta-analysis of the effectiveness of intelligent tutoring systems on K–12 students’ mathematical learning. J. Educ. Psychol. 105(4), 970–987 (2013)CrossRefGoogle Scholar
  24. 24.
    Steenbergen-Hu, S., Cooper, H.: A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. J. Educ. Psychol. 106(2), 331–347 (2014)CrossRefGoogle Scholar
  25. 25.
    van Leeuwen, A., Janssen, J., Erkens, G., Brekelmans, M.: Supporting teachers in guiding collaborating students: effects of learning analytics in CSCL. Comput. Educ. 79, 28–39 (2014)CrossRefGoogle Scholar
  26. 26.
    VanLehn, K.: The behavior of tutoring systems. Int. J. Artif. Intell. Educ. 16(3), 227–265 (2006)Google Scholar
  27. 27.
    VanLehn, K.: The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educ. Psychol. 46(4), 197–221 (2011)CrossRefGoogle Scholar
  28. 28.
    Waalkens, M., Aleven, V., Taatgen, N.: Does supporting multiple student strategies lead to greater learning and motivation? Investigating a source of complexity in the architecture of intelligent tutoring systems. Comput. Educ. 60(1), 159–171 (2013)CrossRefGoogle Scholar
  29. 29.
    Woolf, B.P.: Building Intelligent Interactive Tutors: Student-Centered Strategies for Revolutionizing e-Learning. Morgan Kauffman, Burlington (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Françeska Xhakaj
    • 1
  • Vincent Aleven
    • 1
  • Bruce M. McLaren
    • 1
  1. 1.Human Computer Interaction InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations