Skip to main content
Book cover

International Conference on Artificial Intelligence in Education

AIED 2018: Artificial Intelligence in Education pp 154–168Cite as

Student Learning Benefits of a Mixed-Reality Teacher Awareness Tool in AI-Enhanced Classrooms

Part of the Lecture Notes in Computer Science book series (LNAI,volume 10947)

Abstract

When used in K-12 classrooms, intelligent tutoring systems (ITSs) can be highly effective in helping students learn. However, they might be even more effective if designed to work together with human teachers, to amplify their abilities and leverage their complementary strengths. In the present work, we designed a wearable, real-time teacher awareness tool: mixed-reality smart glasses that tune teachers in to the rich analytics generated by ITSs, alerting them to situations the ITS may be ill-suited to handle. A 3-condition experiment with 286 middle school students, across 18 classrooms and 8 teachers, found that presenting teachers with real-time analytics about student learning, metacognition, and behavior had a positive impact on student learning, compared with both business-as-usual and classroom monitoring support without advanced analytics. Our findings suggest that real-time teacher analytics can help to narrow the gap in learning outcomes across students of varying prior ability. This is the first experimental study showing that real-time teacher analytics can enhance student learning. This research illustrates the promise of AIED systems that integrate human and machine intelligence to support student learning.

Keywords

  • Real-time analytics
  • Classroom orchestration
  • Teacher-in-the-loop
  • Human-AI hybrid
  • Intelligent tutors
  • Dashboards
  • Classroom evaluation

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-93843-1_12
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-93843-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

References

  1. Alavi, H.S., Dillenbourg, P.: An ambient awareness tool for supporting supervised collaborative problem solving. IEEE TLT 5, 264–274 (2012)

    Google Scholar 

  2. Aleven, V.: Help seeking and intelligent tutoring systems: theoretical perspectives and a step towards theoretical integration. In: Azevedo, R., Aleven, V. (eds.) International Handbook of Metacognition and Learning Technologies, pp. 311–335. Springer, New York (2013). https://doi.org/10.1007/978-1-4419-5546-3_21

    CrossRef  Google Scholar 

  3. Aleven, V., McLaren, B.M., Sewall, J., Koedinger, K.R.: A new paradigm for intelligent tutoring systems: example-tracing tutors. IJAIED 19(2), 105–154 (2009)

    Google Scholar 

  4. Aleven, V., Roll, I., McLaren, B.M., Koedinger, K.R.: Help helps, but only so much: research on help seeking with intelligent tutoring systems. IJAIED 26, 205–223 (2016)

    Google Scholar 

  5. Alphen, E.V., Bakker, S.: Lernanto: using an ambient display during differentiated instruction. In: CHI EA (2016)

    Google Scholar 

  6. Baker, R.S.: Modeling and understanding students off-task behavior in intelligent tutoring systems. In: CHI, pp. 1059–1068 (2007)

    Google Scholar 

  7. Baker, R.S.: Stupid tutoring systems, intelligent humans. IJAIED 26(2), 600–614 (2016)

    Google Scholar 

  8. Baker, R.S., Corbett, A.T., Roll, I., Koedinger, K.R.: Developing a generalizable detector of when students game the system. UMUAI 18(3), 287–314 (2008)

    Google Scholar 

  9. Beck, J.E., Gong, Y.: Wheel-spinning: students who fail to master a skill. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 431–440. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_44

    CrossRef  Google Scholar 

  10. Bull, S., Kay, J.: Open learner models. In: Nkambou, R., Bourdeau, J., Mizoguchi, R. (eds.) Advances in Intelligent Tutoring Systems, pp. 301–322. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14363-2_15

    CrossRef  Google Scholar 

  11. Chromebook Management Software for Schools. https://www.goguardian.com/

  12. Cohen, J.: A power primer. Psychol. Bull. 112(1), 155–159 (1992)

    CrossRef  Google Scholar 

  13. Corbett, A.T., Anderson, J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. UMUAI 4(4), 253–278 (1995)

    Google Scholar 

  14. Desmarais, M.C., Baker, R.S.: A review of recent advances in learner and skill modeling in intelligent learning environments. UMUAI 22(1–2), 9–38 (2012)

    Google Scholar 

  15. Dillenbourg, P., Jermann, P.: Technology for classroom orchestration. In: Khine, M., Saleh, I. (eds.) New Science of Learning, pp. 525–552. Springer, New York (2010). https://doi.org/10.1007/978-1-4419-5716-0_26

    CrossRef  Google Scholar 

  16. Dillenbourg, P.: Trends in classroom orchestration. STELLAR 1, 5 (2011)

    Google Scholar 

  17. Hanington, B., Martin, B.: Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport (2012)

    Google Scholar 

  18. Hapara | Making Learning Visible. https://hapara.com/

  19. Holstein, K., Hong, G., Tegene, M., McLaren, B. M., Aleven, V.: The classroom as a dashboard: co-designing wearable cognitive augmentation for K-12 teachers. In: LAK, pp. 79–88. ACM (2018)

    Google Scholar 

  20. Holstein, K., McLaren, B.M., Aleven, V.: Informing the design of teacher awareness tools through causal alignment analysis. In: ICLS (in press)

    Google Scholar 

  21. Holstein, K., McLaren, B.M., Aleven, V.: Intelligent tutors as teachers’ aides: exploring teacher needs for real-time analytics in blended classrooms. In: LAK, pp. 257–266. ACM (2017)

    Google Scholar 

  22. Holstein, K., McLaren, B.M., Aleven, V.: SPACLE: investigating learning across virtual and physical spaces using spatial replays. In: LAK, pp. 358–367. ACM (2017)

    Google Scholar 

  23. Kai, S., Almeda, V.A., Baker, R.S., Shechtman, N., Heffernan, C., Heffernan, N.: Modeling wheel-spinning and productive persistence in skill builders. In: JEDM (in press)

    Google Scholar 

  24. Kelly, K., Heffernan, N., Heffernan, C., Goldman, S., Pellegrino, J., Soffer Goldstein, D.: Estimating the effect of web-based homework. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 824–827. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_122

    CrossRef  Google Scholar 

  25. Kulik, J.A., Fletcher, J.D.: Effectiveness of intelligent tutoring systems: a meta-analytic review. RER 86(1), 42–78 (2016)

    Google Scholar 

  26. LanSchool Classroom Management Software. https://www.lenovosoftware.com/lanschool

  27. Long, Y., Aleven, V.: Supporting students’ self-regulated learning with an open learner model in a linear equation tutor. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 219–228. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_23

    CrossRef  Google Scholar 

  28. Long, Y., Aleven, V.: Gamification of joint student/system control over problem selection in a linear equation tutor. In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds.) ITS 2014. LNCS, vol. 8474, pp. 378–387. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07221-0_47

    CrossRef  Google Scholar 

  29. LookWhosTalking. bitbucket.org/dadamson/lookwhostalking

  30. Martinez-Maldonado, R., Clayphan, A., Yacef, K., Kay, J.: MTFeedback: providing notifications to enhance teacher awareness of small group work in the classroom. IEEE TLT 8(2), 187–200 (2015)

    Google Scholar 

  31. Mavrikis, M., Gutierrez-Santos, S., Poulovassilis, A.: Design and evaluation of teacher assistance tools for exploratory learning environments. In: LAK, pp. 168–172. ACM (2016)

    Google Scholar 

  32. Molenaar, I., Knoop-van Campen, C.: Teacher dashboards in practice: usage and impact. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds.) EC-TEL 2017. LNCS, vol. 10474, pp. 125–138. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66610-5_10

    CrossRef  Google Scholar 

  33. Nye, B.D.: Barriers to ITS adoption: a systematic mapping study. In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds.) ITS 2014. LNCS, vol. 8474, pp. 583–590. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07221-0_74

    CrossRef  Google Scholar 

  34. Pelánek, R., Řihák, J.: Experimental analysis of mastery learning criteria. In: UMAP, pp. 156–163. ACM (2017)

    Google Scholar 

  35. Prieto, L.P.: Supporting orchestration of blended CSCL scenarios in distributed learning environments. Unpublished doctoral thesis (2012)

    Google Scholar 

  36. Prieto, L.P., Sharma, K., Dillenbourg, P., Jesús, M.: Teaching analytics: towards automatic extraction of orchestration graphs using wearable sensors. In: LAK, pp. 148–157. ACM (2016)

    Google Scholar 

  37. Ritter, S., Yudelson, M., Fancsali, S.E., Berman, S.R.: How mastery learning works at scale. In: L@S, pp. 71–79. ACM (2016)

    Google Scholar 

  38. Ritter, S., Yudelson, M., Fancsali, S., Berman, S.R.: Towards integrating human and automated tutoring systems. In: EDM, pp. 626–627 (2016)

    Google Scholar 

  39. Rodríguez-Triana, M.J., Prieto, L.P., Vozniuk, A., Boroujeni, M.S., Schwendimann, B.A., Holzer, A., Gillet, D.: Monitoring, awareness and reflection in blended technology enhanced learning: a systematic review. IJTEL 9(23), 126–150 (2017)

    CrossRef  Google Scholar 

  40. Segal, A., Hindi, S., Prusak, N., Swidan, O., Livni, A., Palatnic, A., Schwarz, B., Gal, Y.: Keeping the teacher in the loop: technologies for monitoring group learning in real-time. In: André, E., Baker, R., Hu, X., Rodrigo, M.M.T., du Boulay, B. (eds.) AIED 2017. LNCS (LNAI), vol. 10331, pp. 64–76. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61425-0_6

    CrossRef  Google Scholar 

  41. Schofield, J.W.: Computers and Classroom Culture. University Press, Cambridge (1995)

    CrossRef  Google Scholar 

  42. Stang, J.B., Roll, I.: Interactions between teaching assistants and students boost engagement in physics labs. Phys. Rev. Phys. Educ. Res. 10(2), 020117 (2014)

    CrossRef  Google Scholar 

  43. Tissenbaum, M., Matuk, C.: Real-time visualization of student activities to support classroom orchestration. In: ICLS, pp. 1120–1127 (2016)

    Google Scholar 

  44. Xhakaj, F., Aleven, V., McLaren, B.M.: Effects of a teacher dashboard for an intelligent tutoring system on teacher knowledge, lesson planning, lessons and student learning. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds.) EC-TEL 2017. LNCS, vol. 10474, pp. 315–329. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66610-5_23

    CrossRef  Google Scholar 

  45. Yacef, K.: Intelligent teaching assistant systems. In: ICCE, pp. 136–140. IEEE (2002)

    Google Scholar 

Download references

Acknowledgements

This work was supported by NSF Award #1530726, and IES Grant R305B150008 to CMU. The opinions expressed are those of the authors and do not represent the views of NSF, IES or the U.S. ED. Special thanks to Gena Hong, Octav Popescu, Jonathan Sewall, Mera Tegene, Cindy Tipper, and all participating students and teachers.

Author information

Authors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Kenneth Holstein, Bruce M. McLaren & Vincent Aleven

Authors
  1. Kenneth Holstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruce M. McLaren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vincent Aleven
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kenneth Holstein .

Editor information

Editors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, PA, USA

    Carolyn Penstein Rosé

  2. University of Technology, Sydney, NSW, Australia

    Roberto Martínez-Maldonado

  3. University of Duisburg-Essen, Duisburg, Germany

    H. Ulrich Hoppe

  4. UCL Institute of Education, London, UK

    Rose Luckin

  5. UCL Institute of Education, London, UK

    Manolis Mavrikis

  6. UCL Institute of Education, London, UK

    Dr. Kaska Porayska-Pomsta

  7. Carnegie Mellon University, Pittsburgh, PA, USA

    Bruce McLaren

  8. University of Sussex, Brighton, UK

    Benedict du Boulay

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Holstein, K., McLaren, B.M., Aleven, V. (2018). Student Learning Benefits of a Mixed-Reality Teacher Awareness Tool in AI-Enhanced Classrooms. In: , et al. Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science(), vol 10947. Springer, Cham. https://doi.org/10.1007/978-3-319-93843-1_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-93843-1_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-93842-4

  • Online ISBN: 978-3-319-93843-1

  • eBook Packages: Computer ScienceComputer Science (R0)