Using Gesture Analysis to Assess Students’ Developing Representational Competence

  • Matthew E. LiraEmail author
  • Mike Stieff
Part of the Models and Modeling in Science Education book series (MMSE, volume 11)


Assessments of representational competence traditionally take one of two formats: those that ask students to generate external representations and those that ask students to interpret a given representation. Using either format extant studies have focused primarily on assessing the quality of students’ representational competence before and after instruction or analyzing how representational competence differs between experts and novices. This chapter will discuss a novel approach for assessing students' developing representational competence using a micro-genetic approach. Specifically, we illustrate how student-generated gestures hold unique affordances for assessing representational competence that compliment traditional pen and paper assessments. We demonstrate the application of this approach with a simple experiment that assesses student reasoning with multi-representational learning technologies. We show how the specificity (i.e. the informational properties) of different learning technologies influence students’ developing representational competence as evident in the gestures that students produce before, during, and after interacting with a learning technology. We conclude by contrasting the distinct informational and computational properties of external representations depicted in inscriptions and those depicted in gesture.


  1. Agrawala, M., Phan, D., Heiser, J., Haymaker, J., Klingner, J., Hanrahan, P., & Tversky, B. (2003). Designing effective step-by-step assembly instructions. ACM Transactions on Graphics (TOG), 22(3), 828–837.CrossRefGoogle Scholar
  2. Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183–198.CrossRefGoogle Scholar
  3. Ainsworth, S., Galpin, J., & Musgrove, S. (2007). Learning about dynamic systems by drawing for yourself and for others. In EARLI conference 2007. Google Scholar
  4. Alibali, M. W. (2005). Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial Cognition and Computation, 5(4), 307–331.CrossRefGoogle Scholar
  5. Alibali, M. W., & Nathan, M. J. (2012). Embodiment in mathematics teaching and learning evidence from learners’ and teachers’ gestures. Journal of the Learning Sciences, 21(2), 247–286.CrossRefGoogle Scholar
  6. Azevedo, F. S. (2000). Designing representations of terrain: A study in meta-representational competence. The Journal of Mathematical Behavior, 19(4), 443–480.CrossRefGoogle Scholar
  7. Becvar, L. A., Hollan, J., & Hutchins, E. (2005). Hands as molecules: Representational gestures used for developing theory in a scientific laboratory. Semiotica, 156, 89–112.Google Scholar
  8. Chi, M. T., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices*. Cognitive Science, 5(2), 121–152.CrossRefGoogle Scholar
  9. Chu, M., & Kita, S. (2011). The nature of gestures’ beneficial role in spatial problem solving. Journal of Experimental Psychology: General, 140(1), 102.CrossRefGoogle Scholar
  10. Church, R. B., & Goldin-Meadow, S. (1986). The mismatch between gesture and speech as an index of transitional knowledge. Cognition, 23(1), 43–71.CrossRefGoogle Scholar
  11. Cook, M., Carter, G., & Wiebe, E. N. (2008). The interpretation of cellular transport graphics by students with low and high prior knowledge. International Journal of Science Education, 30(2), 239–261.CrossRefGoogle Scholar
  12. Crick, F. H., & Watson, J. D. (1954). The complementary structure of deoxyribonucleic acid. Proceedings of the Royal Society of London Series A: Mathematical and Physical Sciences, 223(1152), 80–96.CrossRefGoogle Scholar
  13. Demir, Ö. E. (2009). A tale of two hands: Development of narrative structure in children's speech and gesture and its relation to later reading skill (Doctoral dissertation). Retrieved from ProQuest. (Accession No. 3369323).Google Scholar
  14. Ehrlich, S. B., Levine, S. C., & Goldin-Meadow, S. (2006). The importance of gesture in children's spatial reasoning. Developmental Psychology, 42(6), 1259.CrossRefGoogle Scholar
  15. Flood, V. J., Amar, F. G., Nemirovsky, R., Harrer, B. W., Bruce, M. R., & Wittmann, M. C. (2014). Paying attention to gesture when students talk chemistry: Interactional resources for responsive teaching. Journal of Chemical Education, 92(1), 11–22.CrossRefGoogle Scholar
  16. Garber, P., & Goldin-Meadow, S. (2002). Gesture offers insight into problem-solving in adults and children. Cognitive Science, 26(6), 817–831.CrossRefGoogle Scholar
  17. Glaser, B. G. (1965). The constant comparative method of qualitative analysis. Social.problems, 12(4), 436–445.CrossRefGoogle Scholar
  18. Göksun, T., Hirsh-Pasek, K., & Golinkoff, R. M. (2010). How do preschoolers express cause in gesture and speech? Cognitive Development, 25(1), 56–68.CrossRefGoogle Scholar
  19. Hammer, D., Sherin, B., & Kolpakowski, T. (1991). Inventing graphing: Meta-representational expertise in children. Journal of Mathematical Behavior, 10(2), 117–160.Google Scholar
  20. Hinze, S. R., Rapp, D. N., Williamson, V. M., Shultz, M. J., Deslongchamps, G., & Williamson, K. C. (2013). Beyond ball-and-stick: Students’ processing of novel STEM visualizations. Learning and Instruction, 26, 12–21.CrossRefGoogle Scholar
  21. Iverson, J. M., & Goldin-Meadow, S. (2005). Gesture paves the way for language development. Psychological Science, 16(5), 367–371.CrossRefGoogle Scholar
  22. Johnstone, A. H. (1991). Why is science difficult to learn? Things are seldom what they seem. Journal of Computer Assisted Learning, 7(2), 75–83.CrossRefGoogle Scholar
  23. Jolley, R. P. (2010). Children and pictures: Drawing and understanding. Chichester: Wiley.Google Scholar
  24. Kastens, K. A., Agrawal, S., & Liben, L. S. (2008). Research methodologies in The role of gestures in geoscience teaching and learning. Journal of Geoscience Education, 56(4), 362–368.Google Scholar
  25. Kozma, R. B., & Russell, J. (1997). Multimedia and understanding: Expert and novice responses to different representations of chemical phenomena. Journal of Research in Science Teaching, 34(9), 949–968.CrossRefGoogle Scholar
  26. Kozma, R., & Russell, J. (2005). Students becoming chemists: Developing representational competence. In J. K. Gilbert (Ed.), Visualization in science education (pp. 121–145).CrossRefGoogle Scholar
  27. Latour, B. (1986). Visualization and cognition. Knowledge and Society, 6, 1–40.Google Scholar
  28. Lira, M., Stieff, M., & Scopelitis, S. (2012). The role of gesture in solving spatial problems in STEM. In J. van Aalst, K. Thompson, M. J. Jacobson, & P. Reimann (Eds.), The future of learning: Proceedings of the tenth international conference of the learning sciences (ICLS) - volume 2, short papers, Symposia, and abstracts (pp. 406–410). Sydney: International Society of the Learning Sciences.Google Scholar
  29. McCloud, S. (2006). Making comics: Storytelling secrets of comics, manga, and graphic novels. William Morrow.Google Scholar
  30. McDermott, L. C., Rosenquist, M. L., & Van Zee, E. H. (1987). Student difficulties in connecting graphs and physics: Examples from kinematics. American Journal of Physics, 55(6), 503–513.CrossRefGoogle Scholar
  31. McNeill, D. (1985). So you think gestures are nonverbal? Psychological Review, 92(3), 350.CrossRefGoogle Scholar
  32. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. University of Chicago Press.Google Scholar
  33. McNeill, D. (2005). Gesture and thought. University of Chicago Press.Google Scholar
  34. Nitz, S., Ainsworth, S. E., Nerdel, C., & Prechtl, H. (2014). Do student perceptions of teaching predict the development of representational competence and biological knowledge? Learning and Instruction, 31, 13–22.CrossRefGoogle Scholar
  35. Parnafes, O. (2007). What does “fast” mean? Understanding the physical world through computational representations. The Journal of the Learning Sciences, 16(3), 415–450.CrossRefGoogle Scholar
  36. Pedelty, L. (1985). Gestures in aphasia (doctoral dissertation). Unpublished doctoral dissertation.Google Scholar
  37. Perry, M., Church, R. B., & Goldin-Meadow, S. (1988). Transitional knowledge in the acquisition of concepts. Cognitive Development, 3(4), 359–400.CrossRefGoogle Scholar
  38. Roth, W. (2000). From gesture to scientific language. Journal of Pragmatics, 32(11), 1683–1714.CrossRefGoogle Scholar
  39. Scherr, R. E. (2008). Gesture analysis for physics education researchers. Physical Review Special Topics-Physics Education Research, 4(1). 010101.
  40. Schnotz, W., & Lowe, R. (2008). A unified view of learning from animated and static graphics. In R. Lowe & W. Schnotz (Eds.), Learning with animation research implications for design (pp. 304–356). Cambridge: Cambridge University Press.Google Scholar
  41. Schwartz, D. L., & Black, J. B. (1996). Shuttling between depictive models and abstract rules: Induction and fallback. Cognitive Science, 20(4), 457–497.CrossRefGoogle Scholar
  42. diSessa, A. A. (2004). Metarepresentation: Native competence and targets for instruction. Cognition and Instruction, 22(3), 293–331.CrossRefGoogle Scholar
  43. Shah, P., & Hoeffner, J. (2002). Review of graph comprehension research: Implications for instruction. Educational Psychology Review, 14(1), 47–69.CrossRefGoogle Scholar
  44. Sherin, B. L. (2000). How students invent representations of motion: A genetic account. The Journal of Mathematical Behavior, 19(4), 399–441.CrossRefGoogle Scholar
  45. Srivastava, A., & Ramadas, J. (2013). Analogy and gesture for mental visualization of DNA structure. In D. F. Treagust & C. Y. Tsui (Eds.), Multiple representations in biological education (pp. 311–329). New York: Springer.CrossRefGoogle Scholar
  46. Stieff, M. (2011). When is a molecule three dimensional? A task-specific role for imagisticreasoning in advanced chemistry. Science Education, 95(2), 310–336.CrossRefGoogle Scholar
  47. Stieff, M., Hegarty, M., & Deslongchamps, G. (2011). Identifying representational competence with multi-representational displays. Cognition and Instruction, 29(1), 123–145.CrossRefGoogle Scholar
  48. Uttal, D. H., & O’Doherty, K. (2008). Comprehending and learning from ‘visualizations’: A developmental perspective. In J. K. Gilbert, M. Reiner, & M. Nakhleh (Eds.), Visualization: Theory and practice in science education (pp. 53–72). Dordrecht: Springer.CrossRefGoogle Scholar
  49. Weiss, T. F. (1996). Cellular biophysics (Vol. 1). Cambridge: MIT press.Google Scholar
  50. Zohar, A., & Tamir, P. (1991). Assessing students’ difficulties in causal reasoning in biology—a diagnostic instrument. Journal of Biological Education, 25(4), 302–307.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.The University of IowaIowaUSA
  2. 2.University of Illinois at ChicagoChicagoUSA

Personalised recommendations