Skip to main content
View expanded cover

Machine Learning Paradigms pp 131–156Cite as

Multimodal Learning Analytics in a Laboratory Classroom

Part of the Intelligent Systems Reference Library book series (ISRL,volume 158)

Abstract

Sophisticated research approaches and tools can help researchers to investigate the complex processes involved in learning in various settings. The use of video technology to record classroom practices, in particular, can be a powerful way for capturing and studying learning and related phenomena within a social setting such as the classroom. This chapter outlines several multimodal techniques to analyze the learning activities in a laboratory classroom. The video and audio recordings were processed automatically to obtain information rather than requiring manual coding. Moreover, these automated techniques are able to extract information with an efficiency that is beyond the capabilities of human-coders, providing the means to deal analytically with the multiple modalities that characterize the classroom. Once generated, the information provided by the different modalities is used to explain and predict high-level constructs such as students’ attention and engagement. This chapter not only presents the results of the analysis, but also describes the setting, hardware and software needed to replicate this analytical approach.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-13743-4_8
  • Chapter length: 26 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-13743-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   169.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 8.1
Fig. 8.2
Fig. 8.3
Fig. 8.4
Fig. 8.5
Fig. 8.6
Fig. 8.7

Notes

  1. 1.

    HyperFace: Face landmarks and pose detection. https://github.com/takiyu/hyperface.

  2. 2.

    DockerFace: Face detection https://github.com/natanielruiz/dockerface.

References

  1. Amidon, E., Flanders, N.A.: The effects of direct and indirect teacher influence on dependent-prone students learning geometry. J. Educ. Psychol. 52(6), 286–291 (1961). https://doi.org/10.1037/h0043674

    CrossRef  Google Scholar 

  2. Antonakis, J., Bendahan, S., Jacquart, P., Lalive, R.: On making causal claims: a review and recommendations. Leadersh. Q. 21(6), 1086–1120 (2010). https://doi.org/10.1016/j.leaqua.2010.10.010

    CrossRef  Google Scholar 

  3. Beeby, T., Burkhardt, H., Fraser, R.: Systematic Classroom Analysis Notation. Shell Centre for Mathematics Education, Nottingham, England (1979)

    Google Scholar 

  4. Bezdek, J.C., Ehrlich, R., Full, W.: FCM: the fuzzy c-means clustering algorithm. Comput. Geosci. 10(2), 191–203 (1984). https://doi.org/10.1016/0098-3004(84)90020-7

    CrossRef  Google Scholar 

  5. Blikstein, P.: Multimodal learning analytics. In: Suthers, D., Verbert, K., Duval, E., Ochoa, X. (eds.) Proceedings of the Third International Conference on Learning Analytics and Knowledge, Leuven, Belgium, pp. 102–106 (2013)

    Google Scholar 

  6. Bourke, S.F.: The teaching and learning of mathematics: national report of the second phase of the IEA Classroom Environment Study (ACER research monograph No. 25). Australian Council for Educational Research, Hawthorn, Victoria (1984)

    Google Scholar 

  7. Campbell, D.T.: Assessing the impact of planned social change (Occasional Paper Series No. 8). Western Michigan University, College of Education, Evaluation Center, Kalamazoo, MI (1976). https://eric.ed.gov/?id=ED303512

  8. Chan, M.C.E., Baker, A., Slee, R., Williamson, J.: Educational engagement through the middle years of schooling: Report for the In2COmmuinity Applied Learning Project. Victoria Institute, Melbourne, Australia (2015). https://www.vu.edu.au/sites/default/files/victoria-institute/pdfs/Educational%20Engagement%20through%20the%20Middle%20Years%20of%20Schooling%20%28web-version2%29.pdf

  9. Chan, M.C.E., Clarke, D.J.: Video-based research in a laboratory classroom. In: Xu, L., Aranda, G., Clarke, D. (eds.) Video-Based Research in Education: Cross-Disciplinary Perspectives. Routledge, New York (in press)

    Google Scholar 

  10. Chan, M.C.E., Clarke, D.J.: Learning research in a laboratory classroom: Complementarity and commensurability in juxtaposing multiple interpretive accounts. In: Dooley, T., Gueudet, G. (eds.) Proceedings of the Congress of European Research in Mathematics Education, Dublin, Ireland, pp. 2713–2720 (2017)

    Google Scholar 

  11. Chan, M.C.E., Clarke, D.J.: Structured affordances in the use of open-ended tasks to facilitate collaborative problem solving. ZDM Int. J. Math. Educ. 49, 951–963 (2017). https://doi.org/10.1007/s11858-017-0876-2

    CrossRef  Google Scholar 

  12. Chan, M.C.E., Clarke, D.J., Cao, Y.: The social essentials of learning: an experimental investigation of collaborative problem solving and knowledge construction in mathematics classrooms in Australia and China. Math. Educ. Res. J. 30(1), 39–50 (2017). https://doi.org/10.1007/s13394-017-0209-3

    CrossRef  Google Scholar 

  13. Clarke,D.J.: International comparative studies in mathematics education. Chapter 5 In: Bishop, A.J.,Clements, M.A.,Keitel, C.,Kilpatrick, J., Leung, F.K.S. (eds.) Second InternationalHandbook of Mathematics Education, pp. 145–186. Dordrecht, The Netherlands: Kluwer Academic Publishers (2003)

    Google Scholar 

  14. Clarke, D.J.: Studying the classroom negotiation of meaning: Complementary accounts methodology, Chapter 7. In: Teppo, A. (ed.) Qualitative Research Methods in Mathematics Education. Monograph Number 9 of the Journal for Research in Mathematics Education, pp. 98–111. NCTM, Reston, VA (1998)

    Google Scholar 

  15. Clarke, D.J. (ed.): Perspectives on Practice and Meaning in Mathematics and Science Classrooms. Kluwer Academic Publishers, Dordrecht, The Netherlands (2001)

    Google Scholar 

  16. Clarke, D.J.: Using cross-cultural comparison to interrogate the logic of classroom research in mathematics education. In: Kaur, B., Ho, W.K., Toh, T.L., Choy, B.H. (eds.) Proceedings of the 41st Conference of the International Group for the Psychology of Mathematics Education, vol. 1, pp. 13–28. PME, Singapore (2017)

    Google Scholar 

  17. Clarke, D.J., Keitel, C., Shimizu, Y. (eds.): Mathematics Classrooms in Twelve Countries: The Insider’s Perspective. Sense Publishers, Rotterdam, The Netherlands (2006)

    Google Scholar 

  18. Clarke, D.J., Mitchell, C., Bowman, P.: Optimising the use of available technology to support international collaborative research in mathematics classrooms. In: Janik, T., Seidel, T. (eds.) The Power of Video Studies in Investigating Teaching and Learning in the Classroom, pp. 39–60. Waxmann, New York (2009)

    Google Scholar 

  19. Clarke, D.J., Xu, L.H., Wan, M.E.V.: Spoken mathematics as an instructional strategy: The public discourse of mathematics classrooms in different countries. In: Kaur, B., Anthony, G., Ohtani, M., Clarke, D. (eds.) Student Voice in Mathematics Classrooms Around the World, pp. 13–31. Sense Publishers, Rotterdam, The Netherlands (2013)

    CrossRef  Google Scholar 

  20. Cobb, P., Bauersfeld, H.: The Emergence of Mathematical Meaning: Interaction in Classroom Cultures. L. Erlbaum Associates, Hillsdale, NJ (1995)

    Google Scholar 

  21. Dringus, L.P.: Learning analytics considered harmful. J. Asynchronous Learn. Netw. 16(3), 87–100 (2012)

    Google Scholar 

  22. Erlwanger, S.H.: Case studies of children’s conceptions of mathematics. J. Child. Math. Behav. 1(3), 157–283 (1975)

    Google Scholar 

  23. Ferguson, R.: Learning analytics: drivers, developments and challenges. Int. J. Technol. Enhanc. Learn. 4(5–6), 304–317 (2012)

    CrossRef  Google Scholar 

  24. Ferreiro, E., Teberosky, A.: Literacy before schooling (K. Goodman Castro Trans.) [Los sistemas de escritura en el desarrollo del niño]. Heinemann Educational Books, Exeter, NH (Original work published 1979) (1982)

    Google Scholar 

  25. Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: potential of the concept, state of the evidence. Rev. Educ. Res. 74(1), 59–109 (2004)

    CrossRef  Google Scholar 

  26. Frydenberg, E., Ainley, M., Russell, V.J.: Student motivation and engagement. Sch. Issues Dig. 2 (2005)

    Google Scholar 

  27. Gibbs, R., Poskitt, J.: Student Engagement in the Middle Years of Schooling (Years 7–10): A Literature Review. New Zealand Ministry of Education, Wellington (2010). http://www.educationcounts.govt.nz/publications/schooling/student-engagement-in-the-middle-years-of-schooling-years-7-10-a-literature-review/introduction

  28. Good, T.L., Grouws, D.A.: Teaching effects: a process-product study in fourth-grade mathematics classrooms. J. Teach. Educ. 28(3), 49–54 (1977). https://doi.org/10.1177/002248717702800310

    CrossRef  Google Scholar 

  29. Hart, L.E.: Classroom processes, sex of student, and confidence in learning mathematics. J. Res. Math. Educ. 20(3), 242–260 (1989). https://doi.org/10.2307/749514

    CrossRef  Google Scholar 

  30. Helme, S., Clarke, D.: Cognitive engagement in the mathematics classroom. In: Clarke, D. (ed.) Perspectives on Practice and Meaning in Mathematics and Science Classrooms, pp. 131–153. Kluwer Academic Publishers, Dordrecht, The Netherlands (2001)

    Google Scholar 

  31. Hiebert, J., Gallimore, R., Garnier, H., Givvin, K.B., Hollingsworth, H., Jacobs, J., et al.: Teaching Mathematics in Seven Countries: Results from the TIMSS 1999 Video Study (NCES 2003–013 Revised). U.S. Department of Education, National Center for Education Statistics, Washington, DC (2003). http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2003013

  32. Janík, T., Seidel, T. (eds.): The Power of Video Studies in Investigating Teaching and Learning in the Classroom. Waxmann, Münster, Germany (2009)

    MATH  Google Scholar 

  33. Kleinsmith, A., Bianchi-Berthouze, N.: Recognizing affective dimensions from body posture. In: Paiva, A., Prada, R., Picard, R.W. (eds.) Proceedings of the Affective Computing and Intelligent Interaction Conference, Lisbon, Portugal, pp. 48–58 (2007)

    Google Scholar 

  34. Lim, F.V., O’Halloran, K.L., Podlasov, A.: Spatial pedagogy: mapping meanings in the use of classroom space. Camb. J. Educ. 42(2), 235–251 (2012)

    CrossRef  Google Scholar 

  35. Marshall, S.: Exploring the ethical implications of MOOCs. Distance Educ. 35(2), 250–262 (2014). https://doi.org/10.1080/01587919.2014.917706

    CrossRef  Google Scholar 

  36. Mota, S., Picard, R.W.: Automated posture analysis for detecting learner’s interest level. In: Martinez, A.M., Tan, H.Z. (eds.) Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop, Madison, WI, vol. 5, pp. 49–54, June 2003

    Google Scholar 

  37. Ochoa, X., Chiluiza, K., Méndez, G., Luzardo, G., Guamán, B., Castells, J.: Expertise estimation based on simple multimodal features. In: Proceedings of the 15th ACM on International Conference on Multimodal Interaction, pp. 583–590. ACM, Dec 2013

    Google Scholar 

  38. Ochoa, X.: Multimodal learning analytics. In: Lang, C., Siemens, G., Wise, A.F., Gaševic, D. (eds.) The Handbook of Learning Analytics, pp. 129–141. Society for Learning Analytics Research (SoLAR), Alberta, Canada (2017)

    CrossRef  Google Scholar 

  39. Ochoa, X., Worsley, M.: Augmenting learning analytics with multimodal sensory data. J. Learn. Anal. 3(2), 213–219 (2016)

    CrossRef  Google Scholar 

  40. Overholt, G.: Ethnography and education: limitations and sources of error. J. Thought 15(3), 11–20 (1980)

    Google Scholar 

  41. Paiva, A., Prada, R., Picard, R.W. (eds.): Conference Proceedings of the Second Affective Computing and Intelligent Interaction Conference. Springer, Lisbon, Portugal (2007)

    Google Scholar 

  42. Piaget, J.: The Language and Thought of the Child. Routledge & Kegan Paul, London (1926)

    Google Scholar 

  43. Peterson, P.L., Fennema, E.: Effective teaching, student engagement in classroom activities, and sex-related differences in learning mathematics. Am. Educ. Res. J. 22(3), 309–335 (1985). https://doi.org/10.2307/1162966

    CrossRef  Google Scholar 

  44. Raca, M., Dillenbourg, P.: System for assessing classroom attention. In: Proceedings of the Third International Conference on Learning Analytics and Knowledge, pp. 265–269. ACM, Apr 2013

    Google Scholar 

  45. Raca, M., Dillenbourg, P.: System for assessing classroom attention. In: Suthers, D., Verbert, K., Duval, E., Ochoa, X. (eds.) Proceedings of the Third International Conference on Learning Analytics and Knowledge, Leuven, Belgium, pp. 265–269, Apr 2013

    Google Scholar 

  46. Raca, M., Tormey, R., Dillenbourg, P.: Sleepers’ lag-study on motion and attention. In: Proceedings of the Fourth International Conference on Learning Analytics and Knowledge, Indianapolis, IN, pp. 36–43, Mar 2014

    Google Scholar 

  47. Sfard, A., Kieran, C.: Cognition as communication: Rethinking learning-by-talking through multi-faceted analysis of students’ mathematical interactions. Mind Cult. Act. 8(1), 42–76 (2001). https://doi.org/10.1207/s15327884mca0801_04

    CrossRef  Google Scholar 

  48. Shernoff, D.J.: Optimal Learning Environments to Promote Student Engagement. Springer, New York (2013)

    CrossRef  Google Scholar 

  49. Skinner, B.F.: The science of learning and the art of teaching. Harv. Educ. Rev. 24, 86–97 (1954)

    Google Scholar 

  50. Skinner, E.A., Belmont, M.J.: Motivation in the classroom: reciprocal effects of teacher behavior and student engagement across the school year. J. Educ. Psychol. 85(4), 571–581 (1993)

    CrossRef  Google Scholar 

  51. Stigler, J.W., Hiebert, J.: The Teaching Gap: Best Ideas from the World’s Teachers for Improving Education in the Classroom. Free Press, New York (2009)

    Google Scholar 

  52. Taylor, L., Parsons, J.: Improving student engagement. Curr. Issues Educ. 14(1) (2011)

    Google Scholar 

  53. Ulewicz, M., Beatty, A.: The Power of Video Technology in International Comparative Research in Education. National Academy Press, Washington, DC (2001)

    Google Scholar 

  54. Vygotsky, L.S.: Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge, MA (1978)

    Google Scholar 

Download references

Acknowledgements

This research was conducted with Science of Learning Research Centre funding provided by the Australian Research Council Special Initiatives Grant (SR120300015) and the Discovery Projects funding scheme (DP170102541). We would like to thank the students, parents, teachers, and school staff for their invaluable support of this project. We are also very grateful to our technical team, Cameron Mitchell and Peter (Reggie) Bowman, for their expertise in operating the Science of Learning Research Classroom facility for the project.

Author information

Authors and Affiliations

  1. Melbourne Graduate School of Education, The University of Melbourne, Victoria, Australia

    Man Ching Esther Chan & David Clarke

  2. New York University, New York, USA

    Xavier Ochoa

Authors
  1. Man Ching Esther Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xavier Ochoa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Clarke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Man Ching Esther Chan .

Editor information

Editors and Affiliations

  1. Department of Informatics, University of Piraeus, Piraeus, Greece

    Prof. Maria Virvou

  2. Department of Informatics, University of Piraeus, Piraeus, Greece

    Dr. Efthimios Alepis

  3. University of Piraeus, Pireas, Greece

    Prof. Dr. George A. Tsihrintzis

  4. Faculty of Science, Technology and Mathematics, University of Canberra, Canberra, SA, Australia

    Prof. Lakhmi C. Jain

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Chan, M.C.E., Ochoa, X., Clarke, D. (2020). Multimodal Learning Analytics in a Laboratory Classroom. In: Virvou, M., Alepis, E., Tsihrintzis, G., Jain, L. (eds) Machine Learning Paradigms. Intelligent Systems Reference Library, vol 158. Springer, Cham. https://doi.org/10.1007/978-3-030-13743-4_8

Download citation