Skip to main content
SpringerLink
Log in

Introduction to Multimodal Learning Analytics

  • Chapter
  • First Online:
The Multimodal Learning Analytics Handbook

Abstract

This chapter provides an introduction and an overview of this edited book on Multimodal Learning Analytics (MMLA). The goal of this book is to introduce the reader to the field of MMLA and provide a comprehensive overview of contemporary MMLA research. The contributions come from diverse contexts to support different objectives and stakeholders (e.g., learning scientists, policymakers, technologists). In this first introductory chapter, we present the history of MMLA and the various ongoing challenges, giving a brief overview of the contributions of the book, and conclude by highlighting the potential emerging technologies and practices connected with MMLA.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    What is Learning Analytics?: https://www.solaresearch.org/about/what-is-learning-analytics/

  2. 2.

    CrossMMLA: https://www.solaresearch.org/community/sigs/crossmmla-sig/

  3. 3.

    We looked at Scopus simply because the metadata were available (compared to Google Scholar).

  4. 4.

    MMLA concerns with validity related to data collection, sometimes called “instrumentation validity”, other important types of validity are: internal, external and statistical conclusion validity (see Straub et al., 2004).

References

  • Abrahamson, D., Worsley, M., Pardos, Z., & Ou, L. (2021). Learning analytics of embodied design: Enhancing synergy. International Journal of Child-Computer Interaction. https://doi.org/10.1016/j.ijcci.2021.100409

  • Amft, O., Favela, J., Intille, S., Musolesi, M., & Kostakos, V. (2020). Personalized pervasive health. IEEE Pervasive Computing, 19(3), 11–13.

    Article  Google Scholar 

  • Baker, R. S., & Ocumpaugh, J. (2015). Interaction-based affect detection in educational software. The Oxford handbook of affective computing, 233–245.

    Google Scholar 

  • Bholat, D. (2015). Big data and central banks. Big Data & Society, 2(1). https://doi.org/10.1177/2053951715579469

  • Blikstein, P., & Worsley, M. (2016). Multimodal learning analytics and education data mining: Using computational technologies to measure complex learning tasks. Journal of Learning Analytics, 3(2), 220–238.

    Article  Google Scholar 

  • Cacioppo, J. T., & Tassinary, L. G. (1990). Inferring psychological significance from physiological signals. American Psychologist, 45(1), 16.

    Article  Google Scholar 

  • Chatti, M. A., Muslim, A., & Schroeder, U. (2017). Toward an open learning analytics ecosystem. In Big data and learning analytics in higher education (pp. 195–219). Springer.

    Chapter  Google Scholar 

  • Cukurova, M., Giannakos, M., & Martinez - Maldonado, R. (2020). The promise and challenges of multimodal learning analytics. British Journal of Educational Technology, 51(5), 1441–1449.

    Article  Google Scholar 

  • D’Mello, S. (2017). Emotional learning analytics. In Handbook of learning analytics (p. 115). SOLAR.

    Chapter  Google Scholar 

  • Di Mitri, D., Schneider, J., Specht, M., & Drachsler, H. (2018). From signals to knowledge: A conceptual model for multimodal learning analytics. Journal of Computer Assisted Learning, 34(4), 338–349.

    Article  Google Scholar 

  • D’mello, S. K., & Kory, J. (2015). A review and meta-analysis of multimodal affect detection systems. ACM Computing Surveys (CSUR), 47(3), 1–36.

    Article  Google Scholar 

  • Dominguez, F., Ochoa, X., Zambrano, D., Camacho, K., & Castells, J. (2021). Scaling and adopting a multimodal learning analytics application in an institution-wide setting. IEEE Transactions on Learning Technologies, 14(3), 400–414.

    Article  Google Scholar 

  • Drachsler, H., & Schneider, J. (2018). JCAL special issue on multimodal learning analytics. Journal of Computer Assisted Learning, 34(4), 335–337.

    Article  Google Scholar 

  • Echeverria, V., Martinez-Maldonado, R., & Buckingham Shum, S. (2019). Towards collaboration translucence: Giving meaning to multimodal group data. In Proceedings of the 2019 chi conference on human factors in computing systems (pp. 1–16).

    Google Scholar 

  • Ferguson, R., & Clow, D. (2017). Where is the evidence? A call to action for learning analytics. In Proceedings of the seventh international learning analytics & knowledge conference (pp. 56–65).

    Chapter  Google Scholar 

  • Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., … Warschauer, M. (2020). Mining big data in education: Affordances and challenges. Review of Research in Education, 44(1), 130–160.

    Article  Google Scholar 

  • Giannakos, M. N., Sharma, K., Papavlasopoulou, S., Pappas, I. O., & Kostakos, V. (2020a). Fitbit for learning: Towards capturing the learning experience using wearable sensing. International Journal of Human-Computer Studies, 136, 102384.

    Article  Google Scholar 

  • Giannakos, M., Spikol, D., Molenaar, I., Di Mitri, D., Sharma, K., Ochoa, X., & Hammad, R. (2020b). CrossMMLA in practice: Collecting, annotating and analyzing multimodal data across spaces. In CEUR Workshop Proceedings (Vol. 2610). CEUR-WS.

    Google Scholar 

  • Giannakos, M.N., Lee-Cultura, S., Sharma, K.: Sensing-based analytics in education: The rise of multimodal data enabled learning systems. IT Professional pp. –(2021).

    Google Scholar 

  • Hatfield, J. L., Cryder, M., & Basso, B. (2020). Remote sensing: Advancing the science and the applications to transform agriculture. IT Professional, 22(3), 42–45.

    Article  Google Scholar 

  • Lee-Cultura, S., Sharma, K., & Giannakos, M. (2022). Children’s play and problem-solving in motion-based learning technologies using a multi-modal mixed methods approach. International Journal of Child-Computer Interaction, 31, 100355.

    Article  Google Scholar 

  • Mangaroska, K., & Giannakos, M. (2019). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534.

    Article  Google Scholar 

  • Ochoa, X., & Worsley, M. (2016). Augmenting learning analytics with multimodal sensory data. Journal of Learning Analytics, 3(2), 213–219.

    Article  Google Scholar 

  • Ochoa, X., Lang, A. C., & Siemens, G. (2017). Multimodal learning analytics. The Handbook of Learning Analytics, 1, 129–141.

    Article  Google Scholar 

  • Ochoa, X., Domínguez, F., Guamán, B., Maya, R., Falcones, G., & Castells, J. (2018). The RAP system: Automatic feedback of oral presentation skills using multimodal analysis and low-cost sensors. In Proceedings of the 8th international conference on learning analytics and knowledge (pp. 360–364).

    Google Scholar 

  • O'Donnell, R. D., & Eggemeier, F. T. (1986). Workload assessment methodology. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance (Cognitive processes and performance) (Vol. 2, pp. 41–49). Wiley.

    Google Scholar 

  • Papamitsiou, Z., Giannakos, M. N., & Ochoa, X. (2020). From childhood to maturity: are we there yet? Mapping the intellectual progress in learning analytics during the past decade. In Proceedings of the tenth international conference on learning analytics & knowledge (pp. 559–568).

    Google Scholar 

  • Papavlasopoulou, S., Sharma, K., Melhart, D., Schellekens, J., Lee-Cultura, S., Giannakos, M. N., & Yiannakakis, G. N. (2021). Investigating gaze interaction to support children’s gameplay. International Journal of Child-Computer Interaction, 30, 100349.

    Article  Google Scholar 

  • Pfleging, B., Fekety, D. K., Schmidt, A., & Kun, A. L. (2016). A model relating pupil diameter to mental workload and lighting conditions. In Proceedings of the 2016 CHI conference on human factors in computing systems (pp. 5776–5788).

    Google Scholar 

  • Riedl, R., Davis, F. D., & Hevner, A. R. (2014). Towards a NeuroIS research methodology: Intensifying the discussion on methods, tools, and measurement. Journal of the Association for Information Systems, 15(10), 4.

    Google Scholar 

  • Schneider, J., Börner, D., Van Rosmalen, P., & Specht, M. (2015). Presentation trainer, your public speaking multimodal coach. In Proceedings of the 2015 ACM on international conference on multimodal interaction (pp. 539–546).

    Google Scholar 

  • Sharma, K., & Giannakos, M. (2020). Multimodal data capabilities for learning: What can multimodal data tell us about learning? British Journal of Educational Technology, 51(5), 1450–1484.

    Article  Google Scholar 

  • Sharma, K., Papamitsiou, Z., & Giannakos, M. (2019). Building pipelines for educational data using AI and multimodal analytics: A “grey-box” approach. British Journal of Educational Technology, 50(6), 3004–3031.

    Article  Google Scholar 

  • Spikol, D., Ruffaldi, E., Dabisias, G., & Cukurova, M. (2018). Supervised machine learning in multimodal learning analytics for estimating success in project-based learning. Journal of Computer Assisted Learning, 34(4), 366–377.

    Article  Google Scholar 

  • Straub, D. W., Ang, S., & Evaristo, R. (1994). Normative standards for MIS research. DATA BASE for Advances in Information Systems, 25(1), 21–34.

    Article  Google Scholar 

  • Straub, D. W., Gefen, D., & Boudreau, M.-C. (2004). Validation guidelines for IS positivist research. Communications of the Association for Information Systems, 13(24), 380–427.

    Google Scholar 

  • Worsley, M. (2012). Multimodal learning analytics: enabling the future of learning through multimodal data analysis and interfaces. In Proceedings of the 14th ACM international conference on multimodal interaction (pp. 353–356).

    Google Scholar 

  • Worsley, M. (2018). Multimodal learning analytics’ past, present, and potential futures. CrossMMLA@ LAK.

    Google Scholar 

In This Book

  • Alwahaby, H., Cukurova, M., Papamitsiou, Z., & Giannakos, M. (2022). The evidence of impact and ethical considerations of multimodal learning analytics: A systematic literature review. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Bleck, M., & Le, N. T. (2022). A physiology-aware learning analytics framework. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Cloude, E. B., Wiedbusch, M. D., Dever, D. A., Dario Torre, D., & Azevedo, R. (2022). The role of metacognition and self-regulation on clinical reasoning: Leveraging multimodal learning analytics to transform medical education. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Di Mitri, D., Schneider, J., Limbu, B., Mat Sanusi, K. A., & Klemke, R. (2022). Multimodal learning experience for deliberate practice. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Giannakos, M., Cukurova, M., & Papavlasopoulou, S. (2022). Sensor-based analytics in education: Lessons learned from research in multimodal learning analytics. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Hammad, R., Bahja, M., & Kuhail, M. A. S. (2022). Bridging the gap between informal learning pedagogy and multimodal learning analytics. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Kubsch, M., Caballero, D., & Uribe, P. (2022). Once more with feeling - emotions in multimodal learning analytics. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Malmberg, J., Saqr, M., Järvenoja, H., Haataja, E., Pijeira-Díaz, H. J., & Järvelä, S. (2022). Modeling the complex interplay between monitoring events for regulated learning with psychological networks. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Ochoa, X. (2022). Multimodal systems for automated oral presentation feedback: A comparative analysis. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Shankar, S. K., Rodríguez-Triana, M. J., Prieto, L. P., Calleja, A. R., & Chejara, P. (2022). CDM4MMLA: Contextualized data model for multimodal learning analytics. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Tancredi, T., Abdu, R., Balasubramaniam, R., & Abrahamson, D. (2022). Intermodality in multimodal learning analytics for cognitive theory development: A case from embodied design for mathematics learning. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Vujovic, M., Hernandez-Leo, D., Martinez-Maldonado, R., Cukurova, M., & Spikol, D. (2022). Multimodal learning analytics and the design of learning spaces. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

  • Worsley, M. (2022). Framing the future of multimodal learning analytics. In the Multimodal Learning Analytics Handbook. Springer.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Norwegian University of Science and Technology, Trondheim, Norway

    Michail Giannakos & Kshitij Sharma

  2. University of Copenhagen, Copenhagen, Denmark

    Daniel Spikol

  3. DIPF, Frankfurt am Main, Germany

    Daniele Di Mitri

  4. New York University, New York, NY, USA

    Xavier Ochoa

  5. University of East London, London, UK

    Rawad Hammad

Authors
  1. Michail Giannakos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Spikol
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniele Di Mitri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kshitij Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xavier Ochoa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rawad Hammad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michail Giannakos .

Editor information

Editors and Affiliations

  1. Norwegian University of Science and Technology, Trondheim, Norway

    Michail Giannakos

  2. University of Copenhagen, Copenhagen, Denmark

    Daniel Spikol

  3. DIPF, Frankfurt am Main, Germany

    Daniele Di Mitri

  4. Norwegian University of Science and Technology, Trondheim, Norway

    Kshitij Sharma

  5. New York University, New York, NY, USA

    Xavier Ochoa

  6. University of East London, London, UK

    Rawad Hammad

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Giannakos, M., Spikol, D., Di Mitri, D., Sharma, K., Ochoa, X., Hammad, R. (2022). Introduction to Multimodal Learning Analytics. In: Giannakos, M., Spikol, D., Di Mitri, D., Sharma, K., Ochoa, X., Hammad, R. (eds) The Multimodal Learning Analytics Handbook. Springer, Cham. https://doi.org/10.1007/978-3-031-08076-0_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-08076-0_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-08075-3

  • Online ISBN: 978-3-031-08076-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation