Using Motion Capture Technologies to Provide Advanced Feedback and Scaffolds for Learning

  • Andreja Istenic Starcic
  • William Mark Lipsmeyer
  • Lin LinEmail author
Part of the Educational Communications and Technology: Issues and Innovations book series (ECTII)


In this chapter, we take the stand that cognition and learning are embodied in psychomotor activities and socio-cultural contexts, and they are mediated by technologies on the enactive, iconic, and symbolic representational levels. We discuss motion or body movements as an integral part of cognition and learning. The particular focus is on the role of motion capture technologies in integrating body, sensorimotor engagement, and feedback in learning. Motion capture technologies may help assist learning in several ways: (1) fascilitating seamless human–computer interaction; (2) connecting the enactive learning to observation and to model-based learning; (3) linking body motion to psychological reactions and states. Traditionally, computer-based learning has supported visual and symbolic representations. Advanced motion capture technologies connect physical and virtual environments, support enactive representations, connect different types of representations, and provide smart and sophisticated feedback to improve learning.


Cognitive dissonance Model-based learning Motion capture Scaffolding Feedback The zone of proximal development 


  1. Alexander, C. (1964). Notes on the synthesis of form. Cambridge, MA: Harvard University Press.Google Scholar
  2. Anderson, M. L. (2003). Embodied cognition: A field guide. Artificial inteligence, 149, 91–130.CrossRefGoogle Scholar
  3. Asada, M., MacDorman, K., Ishiguro, H., & Kuniyoshi, Y. (2001). Cognitive developmental robotics as a new paradigm for the design of humanoid robots. Robotics and Autonomous Systems, 37(2–3), 185–193.CrossRefGoogle Scholar
  4. Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
  5. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.CrossRefGoogle Scholar
  6. Bianchi-Berthouze, N., Kim, W. W., & Patel, D. (2007). Does body movement engage you more in digital game play? And why? In A. Paiva, R. Prada, & R. W. Picard (Eds.), ACII 2007, LNCS (Vol. 4738, pp. 102–113). New York: ACM.Google Scholar
  7. Black, P., & Wiliam, D. (2009). Developing the theory of formal assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5.CrossRefGoogle Scholar
  8. Box, G. E. P. (1976). Science and Statistics. Journal of the American Statistical Association, 71, 791–799. CrossRefGoogle Scholar
  9. Bruner, J. S. (1966). Toward a theory of instruction. Cambridge, MA: Harvard University Press.Google Scholar
  10. Bodner, G. (1986). Constructivism: A theory of knowledge. Journal of Chemical Education, 63, 873–878. Scholar
  11. Castellano, G., Villalba, S. D., & Camurri, A. (2007). Recognising human emotions from body movement and gesture dynamics. In A. Paiva, R. Prada, & R. W. Picard (Eds.), ACII 2007, LNCS (Vol. 4738, pp. 71–82). Heidelberg: Springer.Google Scholar
  12. Cheng, X., & Davis, J. (2000). Camera placement considering occlusion for robust motion capture (Stanford Computer Science Technical Report, CS-TR-2000-07). Stanford University.Google Scholar
  13. Cordes, C., & Miller, E. (Eds.). (2000). Fool’s gold: A critical look at computers in childhood. College Park, MD: Alliance for Childhood.Google Scholar
  14. Crane, E., & Gross, M. (2007). Motion capture and emotion: Affect detection in whole body movement. In A. Paiva, R. Prada, & R. W. Picard (Eds.), ACII 2007, LNCS (Vol. 4738, pp. 95–101). Heidelberg: Springer.Google Scholar
  15. D’Mello, S., Dieterle, E., & Duckworth, A. (2017). Advanced, analytic, automated (AAA) measurement of engagement during learning. Educational Psychologist, 52(2), 104–123. CrossRefGoogle Scholar
  16. Festinger, L. (1962). Cognitive dissonance. Scientific American, 207(4), 93–106.CrossRefGoogle Scholar
  17. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. CrossRefGoogle Scholar
  18. Istenic Starcic, A., Lipsmeyer, W. M., & Lin, L. (2018). Motion capture technology supporting cognitive, psychomotor, and affective-social learning. In Wu, Huang, Istenic Starcic, Shadijev, Lin (Eds.), 2018 International Conference of Innovative Technologies in Learning, LNCS (pp. 311–322).Google Scholar
  19. Johnson-Glenberg, M. C., Birchfield, D. A., Tolentino, L., & Koziupa, T. (2014). Collaborative embodied learning in mixed reality motion-capture environments: Two science studies. Journal of Educational Psychology, 106(1), 86–104. CrossRefGoogle Scholar
  20. Kaiser, H. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23(3), 187–200. CrossRefGoogle Scholar
  21. Kim, K., Maloney, D., Bruder, G., Bailenson, J. N., & Welch, G. F. (2017). The effects of virtual human’s spatial and behavioral coherence with physical objects on social presence in AR. Computer Animation and Virtual Worlds, 28, e1771.CrossRefGoogle Scholar
  22. Kleinsmith, A., & Bianchi-Berthouze, N. (2007). Recognizing affective dimensions from body posture. In A. Paiva, R. Prada, & R. W. Picard (Eds.), ACII 2007, LNCS (Vol. 4738, pp. 48–58). Heidelberg: Springer.Google Scholar
  23. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: Historical review, a meta-analysis and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254–284.CrossRefGoogle Scholar
  24. Kozulin, A. (1999). Vygotsky’s psychology. Cambridge: Harvard University Press.Google Scholar
  25. Lindgren, R., & Moshell, J. (2011). Supporting children’s learning with body-based metaphors in a mixed reality environment. In Proceedings of the 10th International Conference on Interaction Design and Children (pp. 177–180). New York: ACM.Google Scholar
  26. Luft, J. & Ingham, H. (1955). The Johari window, a graphic model of interpersonal awareness. In Proceedings of the Western Training Laboratory in Group Development. Los Angeles: University of California, Los Angeles.Google Scholar
  27. Piaget, J. (1952). The origins of intelligence in children. New York: International Universities Press.CrossRefGoogle Scholar
  28. Rosati, G., Oscari, F., Spagnol, S., Avanzini, F., & Masiero, S. (2012). Effect of task-related continuous auditory feedback during learning of tracking motion exercises. Journal of Neuroengineering and Rehabilitation, 9(79), 2–13. CrossRefGoogle Scholar
  29. Russell, S., & Norvig, P. (2016). Artificial intelligence: A modern approach. Malaysia: Pearson Education Limited. Tin hoc Collection. Global Edition.Google Scholar
  30. Schwartz, D. L., Tsang, J. T., & Blair, K. P. (2016). The ABCs of how we learn: 26 scientifically proven approaches, how they work, and when to use them. New York: W. W. Norton & Company.Google Scholar
  31. Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and measuring student engagement in science. Educational Psychologist, 50(1), 1–13.CrossRefGoogle Scholar
  32. Troy, J., Erignac, C., & Murray, P. (2006). Closed-loop feedback control using motion capture systems. US Grant – US7643893B2. Boeing Aircraft Corporation. Retried August 18, from
  33. Trungpa, C., & Fremantle, F. (2000). Tibetan book of the dead: The great liberation through hearing in the Bardo. Boston: Shambhala Publications.Google Scholar
  34. von Glasersfeld, E. (1995). Radical constructivism: A way of knowing and learning, Studies in Mathematics Education, Series: 6. Bristol, PA: Falmer Press.CrossRefGoogle Scholar
  35. Vygotsky, L. (1978). Mind in society: The development of higher mental process. Cambridge, MA: Harvard University Press.Google Scholar
  36. Waibel, A., Vo, M. T., Duchnowski, P., & Manke, S. (1996). Multimodal interfaces. Artificial Intelligence Review, 10(3–4), 299–319.CrossRefGoogle Scholar
  37. Walchli, M., Ruffieux, J., Bourquin, Y., Keller, M., & Taube, W. (2016). Maximizing performance: Augmented feedback, focus of attention, and/or reward? Medicine and Science in Sports and Exercise, 48(4), 714–719. CrossRefGoogle Scholar
  38. Wallon, R. C., & Lindgren, R. (2017). Considerations for the design of gesture-augmented learning environments. In M. J. Spector, B. B. Lockee, & M. D. Childress (Eds.), Learning, design, and technology: An international compendium of theory, research, practice and policy (pp. 1–21). Cham: Springer. CrossRefGoogle Scholar

Copyright information

© Association for Educational Communications and Technology 2019

Authors and Affiliations

  • Andreja Istenic Starcic
    • 1
    • 2
  • William Mark Lipsmeyer
    • 3
  • Lin Lin
    • 4
    Email author
  1. 1.Faculty of EducationUniversity of PrimorskaKoperSlovenia
  2. 2.University of LjubljanaLjubljanaSlovenia
  3. 3.Zoic Studios, LACulver CityUSA
  4. 4.University of North TexasDentonUSA

Personalised recommendations