Multimodal Learning Hub: A Tool for Capturing Customizable Multimodal Learning Experiences

  • Jan SchneiderEmail author
  • Daniele Di Mitri
  • Bibeg Limbu
  • Hendrik Drachsler
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11082)


Studies in Learning Analytics provide concrete examples of how the analysis of direct interactions with learning management systems can be used to optimize and understand the learning process. Learning, however, does not necessarily only occur when the learner is directly interacting with such systems. With the use of sensors, it is possible to collect data from learners and their environment ubiquitously, therefore expanding the use cases of Learning Analytics. For this reason, we developed the Multimodal Learning Hub (MLH), a system designed to enhance learning in ubiquitous learning scenarios, by collecting and integrating multimodal data from customizable configurations of ubiquitous data providers. In this paper, we describe the MLH and report on the results of tests where we explored its reliability to integrate multimodal data.


Multimodal Learning Analytics Sensor-based learning System design 


  1. 1.
    Siemens, G., Long, P.: Penetrating the fog: analytics in learning and education. Educ. Rev. 46, 30 (2011)Google Scholar
  2. 2.
    Arnold, K.E., Pistilli, M.D.: Course signals at purdue: using learning analytics to increase student success. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, pp. 267–270. ACM, New York (2012)Google Scholar
  3. 3.
    Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z.: Off-task behavior in the cognitive tutor classroom: when students “game the system.” In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 383–390. ACM, New York (2004)Google Scholar
  4. 4.
    Kizilcec, R.F., Piech, C., Schneider, E.: Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In: Proceedings of the Third International Conference on Learning Analytics and Knowledge, pp. 170–179. ACM, New York (2013)Google Scholar
  5. 5.
    Swan, M.: Sensor Mania! the internet of things, wearable computing, objective metrics, and the quantified self 2.0. J. Sens. Actuator Networks 1, 217–253 (2012)CrossRefGoogle Scholar
  6. 6.
    Worsley, M.: (Dis) engagement matters: identifying efficacious learning practices with multimodal learning analytics. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge, pp. 365–369 (2018)Google Scholar
  7. 7.
    Blikstein, P.: Multimodal learning analytics. In: Proceedings of the Third International Conference on Learning Analytics and Knowledge, pp. 102–106 (2013)Google Scholar
  8. 8.
    Schneider, J., Börner, D., van Rosmalen, P., Specht, M.: Augmenting the senses: a review on sensor-based learning support. Sensors 15, 4097–4133 (2015)CrossRefGoogle Scholar
  9. 9.
    Dermody, F., Sutherland, A.: A multimodal system for public speaking with real time feedback, pp. 369–370 (2015)Google Scholar
  10. 10.
    Ochoa, X., Domínguez, F., Guamán, B., Maya, R., Falcones, G., Castells, J.: The RAP system: automatic feedback of oral presentation skills using multimodal analysis and low-cost sensors. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge, pp. 360–364 (2018)Google Scholar
  11. 11.
    Schneider, J., Börner, D., van Rosmalen, P., Specht, M.: Can you help me with my pitch? studying a tool for real-time automated feedback. IEEE Trans. Learn. Technol. 9, 318–327 (2016)CrossRefGoogle Scholar
  12. 12.
    Hoque, M. (Ehsan), Courgeon, M., Martin, J.-C., Mutlu, B., Picard, R.W.: MACH: my automated conversation coach. In: Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing - UbiComp 2013, p. 697. ACM Press, New York (2013)Google Scholar
  13. 13.
    Alexandersson, J., Aretoulaki, M., Campbell, N., Gardner, M., Girenko, A., Klakow, D., Koryzis, D., Petukhova, V., Specht, M., Spiliotopoulos, D., Stricker, A., Taatgen, N.: Metalogue: a multiperspective multimodal dialogue system with metacognitive abilities for highly adaptive and flexible dialogue management. In: 2014 International Conference on Intelligent Environments, pp. 365–368 (2014)Google Scholar
  14. 14.
    Rodríguez-Triana, M.J., Prieto, L.P., Martínez-Monés, A., Asensio-Pérez, J.I., Dimitriadis, Y.: The teacher in the loop: customizing multimodal learning analytics for blended learning. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge, pp. 417–426 (2018)Google Scholar
  15. 15.
    Dillenbourg, P.: The evolution of research on digital education. Int. J. Artif. Intell. Educ. 26, 544–560 (2016)CrossRefGoogle Scholar
  16. 16.
    Di Mitri, D., Schneider, J., Specht, M., Drachsler, H.: From signals to knowledge. A conceptual model for multimodal learning analytics, JCAL (2018)Google Scholar
  17. 17.
    King, P.E., Young, M.J., Behnke, R.R.: Public speaking performance improvement as a function of information processing in immediate and delayed feedback interventions. Commun. Educ. 49, 365–374 (2000)CrossRefGoogle Scholar
  18. 18.
    Coulter, G.A., Grossen, B.: The effectiveness of in-class instructive feedback versus after-class instructive feedback for teachers learning direct instruction teaching behaviors. Eff. Sch. Pract. 16, 21–35 (1997)Google Scholar
  19. 19.
    Hattie, J., Timperley, H.: The power of feedback. Rev. Educ. Res. 77, 81–112 (2007)CrossRefGoogle Scholar
  20. 20.
    Börner, D., Kalz, M., Specht, M.: Beyond the channel: a literature review on ambient displays for learning. Comput. Educ. 60, 426–435 (2013)CrossRefGoogle Scholar
  21. 21.
    Guest, W., et al.: Affordances for capturing and re-enacting expert performance with wearables. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds.) EC-TEL 2017. LNCS, vol. 10474, pp. 403–409. Springer, Cham (2017). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Jan Schneider
    • 1
    Email author
  • Daniele Di Mitri
    • 2
  • Bibeg Limbu
    • 2
  • Hendrik Drachsler
    • 1
  1. 1.DIPFFrankfurt am MainGermany
  2. 2.Welten InstituteOpen University of the NetherlandsHeerlenThe Netherlands

Personalised recommendations