Studying Teacher Orchestration Load in Technology-Enhanced Classrooms

A Mixed-Method Approach and Case Study
  • Luis P. PrietoEmail author
  • Kshitij Sharma
  • Pierre Dillenbourg
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9307)


Teacher orchestration of technology-enhanced learning (TEL) processes plays a major role in students’ outcomes, especially in face-to-face classrooms. However, few studies look into the fine-grained details of how such orchestration unfolds, the challenges and cognitive overload that using technologies at a classroom level pose for teachers. This paper proposes a mixed-method approach to the study of orchestration cognitive load, combining physio-behavioural (eye-tracking) and subjective measurements (questionnaires, stimulated recall interviews). We illustrate the approach by applying it to study the orchestration of two technology-enhanced geometry lessons, by a secondary school teacher. The results of our mixed-method analyses highlight the difficulty of classroom-level (as opposed to individual- or group-level) interactions, especially in modelling students’ progress and understanding. Such insights can be useful in the design of new classroom technologies, and to focus researchers’ attention on critical orchestration episodes during their evaluation.


Orchestration load Eye-tracking Stimulated recall Cognitive load Classroom studies 



This research was supported by a Marie Curie Fellowship within the 7th European Community Framework Programme (MIOCTI, FP7–PEOPLE–2012–IEF project no. 327384). Special thanks to Wendy Farden and the International School of Lausanne for their collaboration.


  1. 1.
    Alavi, H.S., Dillenbourg, P.: An ambient awareness tool for supporting supervised collaborative problem solving. IEEE Trans. Learn. Technol. 5(3), 264–274 (2012)CrossRefGoogle Scholar
  2. 2.
    Anderson, J.R.: Spanning seven orders of magnitude: a challenge for cognitive modeling. Cogn. Sci. 26(1), 85–112 (2002)CrossRefGoogle Scholar
  3. 3.
    Bailey, B.: Usability testing: an early history. Accessed on 17 March 2015
  4. 4.
    Beers, P.J., Boshuizen, H.P., Kirschner, P.A., Gijselaers, W., Westendorp, J.: Cognitive load measurements and stimulated recall interviews for studying the effects of information and communications technology. Edu. Technol. Res. Develop. 56(3), 309–328 (2008)CrossRefGoogle Scholar
  5. 5.
    Brunken, R., Plass, J.L., Leutner, D.: Direct measurement of cognitive load in multimedia learning. Edu. Psychol. 38(1), 53–61 (2003)CrossRefGoogle Scholar
  6. 6.
    Buettner, R.: Cognitive workload of humans using artificial intelligence systems: towards objective measurement applying eye-tracking technology. In: Timm, I.J., Thimm, M. (eds.) KI 2013. LNCS, vol. 8077, pp. 37–48. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  7. 7.
    Creswell, J.W.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage publications, Thousand Oaks (2013) Google Scholar
  8. 8.
    Cuendet, S., Bonnard, Q., Do-Lenh, S., Dillenbourg, P.: Designing augmented reality for the classroom. Comput. Edu. 68, 557–569 (2013)CrossRefGoogle Scholar
  9. 9.
    DeLeeuw, K.E., Mayer, R.E.: A comparison of three measures of cognitive load: evidence for separable measures of intrinsic, extraneous, and germane load. J. Edu. Psychol. 100(1), 223 (2008)CrossRefGoogle Scholar
  10. 10.
    Dillenbourg, P.: Design for classroom orchestration. Comput. Edu. 69, 485–492 (2013)CrossRefGoogle Scholar
  11. 11.
    Dillenbourg, P., Järvelä, S., Fischer, F.: The evolution of research on computer-supported collaborative learning. In: Balacheff, N., Ludvigsen, S., de Jong, T., Lazonder, A., Barnes, S. (eds.) TEL, pp. 3–19. Springer, The Netherlands (2009)CrossRefGoogle Scholar
  12. 12.
    Dillenbourg, P., Zufferey, G., Alavi, H., Jermann, P., Do-Lenh, S., Bonnard, Q., Cuendet, S., Kaplan, F.: Classroom orchestration: the third circle of usability. In: Proceedings of CSCL 2011, vol. 1, pp. 510–517 (2011)Google Scholar
  13. 13.
    Doyle, W.: Ecological approaches to classroom management. In: Evertson, C.M., Weinstein, C.S. (eds.) Handbook of Classroom Management: Research, Practice, and Contemporary Issues , pp. 97–125 (2006)Google Scholar
  14. 14.
    Gómez, F., Nussbaum, M., Weitz, J.F., Lopez, X., Mena, J., Torres, A.: Co-located single display collaborative learning for early childhood education. Int. J. Comput. Support. Collab. Learn. 8(2), 225–244 (2013)CrossRefGoogle Scholar
  15. 15.
    Gould, J.D., Lewis, C.: Designing for usability: key principles and what designers think. Commun. ACM 28(3), 300–311 (1985)CrossRefGoogle Scholar
  16. 16.
    Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv. Psychol. 52, 139–183 (1988)CrossRefGoogle Scholar
  17. 17.
    Kharrufa, A., Martinez-Maldonado, R., Kay, J., Olivier, P.: Extending tabletop application design to the classroom. In: Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces, pp. 115–124. ACM (2013)Google Scholar
  18. 18.
    Newell, A.: Unified Theories of Cognition. Harvard University Press, Cambridge (1994)Google Scholar
  19. 19.
    Onrubia, J., Engel, A.: The role of teacher assistance on the effects of a macro-script in collaborative writing tasks. Int. J. Comput. Support. Collab. Learn. 7(1), 161–186 (2012)CrossRefGoogle Scholar
  20. 20.
    Paas, F., Renkl, A., Sweller, J.: Cognitive load theory: instructional implications of the interaction between information structures and cognitive architecture. Instr. Sci. 32(1), 1–8 (2004)CrossRefGoogle Scholar
  21. 21.
    Paas, F., Tuovinen, J.E., Tabbers, H., Van Gerven, P.W.: Cognitive load measurement as a means to advance cognitive load theory. Edu. Psychol. 38(1), 63–71 (2003)CrossRefGoogle Scholar
  22. 22.
    Prieto, L.P., Dlab, M.H., Gutiérrez, I., Abdulwahed, M., Balid, W.: Orchestrating technology enhanced learning: a literature review and a conceptual framework. Int. J. Technol. Enhanced Learn. 3(6), 583–598 (2011)CrossRefGoogle Scholar
  23. 23.
    Prieto, L.P., Sharma, K., Wen, Y., Dillenbourg, P.: The burden of facilitating collaboration: towards estimation of teacher orchestration load using eye-tracking measures. In: Proceedings of the 11th International Conference on Computer-Supported Collaborative Learning, Vol. I, pp. 212–219 (2015)Google Scholar
  24. 24.
    Prieto, L.P., Wen, Y., Caballero, D., Sharma, K., Dillenbourg, P.: Studying teacher cognitive load in multi-tabletop classrooms using mobile eye-tracking. In: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, pp. 339–344. ACM (2014)Google Scholar
  25. 25.
    Roschelle, J., Dimitriadis, Y., Hoppe, U.: Classroom orchestration: synthesis. Comput. Edu. 69, 523–526 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Luis P. Prieto
    • 1
    Email author
  • Kshitij Sharma
    • 1
  • Pierre Dillenbourg
    • 1
  1. 1.CHILI LabÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland

Personalised recommendations