Quantitative Multimodal Interaction Analysis for the Assessment of Problem-Solving Skills in a Collaborative Online Game

  • Alejandro AndradeEmail author
  • Bryan Maddox
  • David Edwards
  • Pravin Chopade
  • Saad Khan
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1112)


We propose a novel method called Quantitative Multimodal Interaction Analysis to understand the meaning of interactions from a set of multimodal observable behavior. We apply this method for the measurement of collaborative problem-solving skills in a dyadic online game specially designed for this purpose. We outline our assumptions and describe the machine learning approach that help us tag multimodal behaviors connecting the theoretical construct with the empirical evidence.


Interaction Analysis Multimodal learning analytics Collaborative Problem-Solving skills 


  1. 1.
    McNeill, D.: Gesture and Thought. University of Chicago press, Chicago (2008)Google Scholar
  2. 2.
    Jordan, B., Henderson, A.: Interaction analysis: Foundations and practice. J. Learn. Sci. 4, 39–103 (1995)CrossRefGoogle Scholar
  3. 3.
    Goffman, E.: Forms of Talk. University of Pennsylvania Press, Philadelphia (1981)Google Scholar
  4. 4.
    Goodwin, C.: Action and embodiment within situated human interaction. J. Pragmat. 32, 1489–1522 (2000)CrossRefGoogle Scholar
  5. 5.
    Kaptelinin, V., Nardi, B.A.: Acting with Technology: Activity Theory and Interaction Design. MIT press, Cambridge (2006)Google Scholar
  6. 6.
    Bavelas, J., Gerwing, J., Healing, S.: Doing mutual understanding. Calibrating with micro-sequences in face-to-face dialogue. J. Pragmat. 121, 91–112 (2017)CrossRefGoogle Scholar
  7. 7.
    Chopade, P., Edwards, D., Khan, S.: Designing a digital jigsaw game based measurement of collaborative problem-solving skills. In: Cunningham, J., et al. (eds.) Companion Proceedings of the 9th International Learning Analytics and Knowledge Conference (LAK 2019), Tempe, Arizona, pp. 26–31. Society for Learning Analytics Research (SoLAR) (2019)Google Scholar
  8. 8.
    Fiore, S.M., Smith-Jentsch, K.A., Salas, E., Warner, N., Letsky, M.: Towards an understanding of macrocognition in teams: developing and defining complex collaborative processes and products. Theor. Issues Ergon. Sci. 11, 250–271 (2010)CrossRefGoogle Scholar
  9. 9.
    Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., Sloetjes, H.: ELAN: a professional framework for multimodality research. In: 5th International Conference on Language Resources and Evaluation (LREC 2006), pp. 1556–1559. (2006)Google Scholar
  10. 10.
    Chopade, P., Khan, S., Edwards, D., Davier, A.A.V.: Machine learning for efficient assessment and prediction of human performance in collaborative learning environments. In: 2018 IEEE International Symposium on Technologies for Homeland Security (HST), pp. 1–6 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Alejandro Andrade
    • 1
    Email author
  • Bryan Maddox
    • 2
  • David Edwards
    • 1
  • Pravin Chopade
    • 1
  • Saad Khan
    • 1
  1. 1.ACTNext by ACT, Inc.Iowa CityUSA
  2. 2.Norwich Research ParkUniversity of East AngliaNorwich, NorfolkUK

Personalised recommendations