Using Machine Learning to Overcome the Expert Blind Spot for Perceptual Fluency Trainings

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11625)


Most STEM domains use multiple visual representations to illustrate complex concepts. While much research has focused on helping students make sense of visuals, students also have to become perceptually fluent at translating among visuals fast and effortlessly. Because perceptual fluency is acquired via implicit, nonverbal processes, perceptual fluency trainings provide simple classification tasks that vary visual features across numerous examples. Prior research shows that learning from such trainings is strongly affected by the sequence of the examples. Further, prior research shows that perceptual fluency trainings are most effective for high-performing students but may confuse low-performing students. We propose that a lack of benefits for low-performing students may result from a perceptual expert blind spot of instructors who typically develop perceptual fluency trainings: expert instructors may be unable to anticipate the needs of students who do not see meaningful information in the visuals. In prior work, we used a machine-learning approach to develop a sequence of example visuals of chemical molecules for low-performing students. This study tested the effectiveness of this sequence in comparison to an expert-generated sequence in a randomized experiment as part of an undergraduate chemistry course. We determined students’ performance based on log data from an educational technology they used in the course. Results show that the machine-learned sequence was more effective for low-performing students. The expert sequence was more effective for high-performing students. Our results can inform the development of perceptual-fluency trainings for adaptive educational technologies.


Multiple visuals Perceptual fluency Sequencing Machine learning 



This research was funded by NSF IIS 1623605 and by NSF ITP 1545481. We thank Yuzi Yu, Blake Mason, John Moore, Rob Nowak, Purav Patel, and Tim Rogers.


  1. 1.
    Gilbert, J.K.: Visualization: an emergent field of practice and inquiry in science education. In: Gilbert, J.K., Reiner, M., Nakhleh, M.B. (eds.) Visualization: Theory and Practice in Science Education, vol. 3, pp. 3–24. Springer, Dordrecht (2008). Scholar
  2. 2.
    Ainsworth, S.: The educational value of multiple-representations when learning complex scientific concepts. In: Gilbert, J.K., Reiner, M., Nakama, A. (eds.) Visualization: Theory and Practice in Science Education, pp. 191–208. Springer, Dordrecht (2008). Scholar
  3. 3.
    NRC: Learning to Think Spatially. National Academies Press, Washington, D.C. (2006)Google Scholar
  4. 4.
    Rau, M.A., Michaelis, J.E., Fay, N.: Connection making between multiple graphical representations: a multi-methods approach for domain-specific grounding of an intelligent tutoring system for chemistry. Comput. Educ. 82, 460–485 (2015)CrossRefGoogle Scholar
  5. 5.
    Ainsworth, S.: DeFT: a conceptual framework for considering learning with multiple representations. Learn. Instr. 16, 183–198 (2006)CrossRefGoogle Scholar
  6. 6.
    McElhaney, K.W., Chang, H.Y., Chiu, J.L., Linn, M.C.: Evidence for effective uses of dynamic visualisations in science curriculum materials. Stud. Sci. Educ. 51, 49–85 (2015)CrossRefGoogle Scholar
  7. 7.
    Rau, M.A.: Conditions for the effectiveness of multiple visual representations in enhancing STEM learning. Educ. Psychol. Rev. 29, 717–761 (2017)CrossRefGoogle Scholar
  8. 8.
    Rau, M.A., Aleven, V., Rummel, N.: Successful learning with multiple graphical representations and self-explanation prompts. J. Educ. Psychol. 107, 30–46 (2015)CrossRefGoogle Scholar
  9. 9.
    Seufert, T., Brünken, R.: Cognitive load and the format of instructional aids for coherence formation. Appl. Cogn. Psychol. 20, 321–331 (2006)CrossRefGoogle Scholar
  10. 10.
    Bodemer, D., Faust, U.: External and mental referencing of multiple representations. Comput. Hum. Behav. 22, 27–42 (2006)CrossRefGoogle Scholar
  11. 11.
    Kellman, P.J., Massey, C.M.: Perceptual learning, cognition, and expertise. In: Ross, B.H. (ed.) The Psychology of Learning and Motivation, vol. 558, pp. 117–165. Elsevier Academic Press, New York (2013)Google Scholar
  12. 12.
    Goldstone, R.L., Barsalou, L.W.: Reuniting perception and conception. Cognition 65, 231–262 (1998)CrossRefGoogle Scholar
  13. 13.
    Kellman, P.J., Garrigan, P.B.: Perceptual learning and human expertise. Physics Life Rev. 6, 53–84 (2009)CrossRefGoogle Scholar
  14. 14.
    Goldstone, R.L., Schyns, P.G., Medin, D.L.: Learning to bridge between perception and cognition. Psychol. Learn. Motiv. 36, 1–14 (1997)CrossRefGoogle Scholar
  15. 15.
    Gibson, E.J.: Perceptual learning in development: some basic concepts. Ecol. Psychol. 12, 295–302 (2000)CrossRefGoogle Scholar
  16. 16.
    Goldstone, R.: Perceptual Learning. Academic Press, San Diego (1997)Google Scholar
  17. 17.
    Rau, M.A., Wu, S.P.W.: Support for sense-making processes and inductive processes in connection-making among multiple visual representations. Cogn. Instr. 36, 361–395 (2018)CrossRefGoogle Scholar
  18. 18.
    Rau, M.A.: Sequencing support for sense making and perceptual induction of connections among multiple visual representations. J. Educ. Psychol. 110, 811–833 (2018)CrossRefGoogle Scholar
  19. 19.
    Schooler, J.W., Fiore, S., Brandimonte, M.A.: At a loss from words: verbal overshadowing of perceptual memories. Psychol. Learn. Motiv. Adv. Res. Theor. 37, 291–340 (1997)CrossRefGoogle Scholar
  20. 20.
    Goldstone, R.L., Landy, D.H., Son, J.Y.: The education of perception. Top. Cogn. Sci. 2, 265–284 (2010)CrossRefGoogle Scholar
  21. 21.
    Gibson, E.J.: Principles of Perceptual Learning and Development. Prentice Hall, New York (1969)Google Scholar
  22. 22.
    Chase, W.G., Simon, H.A.: Perception in chess. Cogn. Psychol. 4, 55–81 (1973)CrossRefGoogle Scholar
  23. 23.
    Richman, H.B., Gobet, F., Staszewski, J.J., Simon, H.A.: Perceptual and memory processes in the acquisition of expert performance: the EPAM model. In: Ericsson, K.A. (ed.) The Road to Excellence? The Acquisition of Expert Performance in the Arts and Sciences, Sports and Games, pp. 167–187. Erlbaum Associates, Mahwah (1996)Google Scholar
  24. 24.
    Taber, K.S.: Revisiting the chemistry triplet: drawing upon the nature of chemical knowledge and the psychology of learning to inform chemistry education. Chem. Educ. Res. Pract. 14, 156–168 (2013)CrossRefGoogle Scholar
  25. 25.
    Koedinger, K.R., Corbett, A.T., Perfetti, C.: The knowledge-learning-instruction framework: bridging the science-practice chasm to enhance robust student learning. Cogn. Sci. 36, 757–798 (2012)CrossRefGoogle Scholar
  26. 26.
    Shanks, D.: Implicit learning. In: Lamberts, K., Goldstone, R. (eds.) Handbook of Cognition, pp. 202–220. Sage, London (2005)Google Scholar
  27. 27.
    Massey, C.M., Kellman, P.J., Roth, Z., Burke, T.: Perceptual learning and adaptive learning technology - developing new approaches to mathematics learning in the classroom. In: Stein, N.L., Raudenbush, S.W. (eds.) Developmental Cognitive Science Goes to School, pp. 235–249. Routledge, New York (2011)Google Scholar
  28. 28.
    Rau, M.A., Aleven, V., Rummel, N.: Interleaved practice in multi-dimensional learning tasks: which dimension should we interleave? Learn. Instr. 23, 98–114 (2013)CrossRefGoogle Scholar
  29. 29.
    Rau, M.A., Aleven, V., Rummel, N., Pardos, Z.: How should intelligent tutoring systems sequence multiple graphical representations of fractions? A multi-methods study. Int. J. Artif. Intell. Educ. 24, 125–161 (2014)CrossRefGoogle Scholar
  30. 30.
    Bradley, J.-C., Lancashire, R., Lang, A., Williams, A.: The spectral game: leveraging open data and crowdsourcing for education. J. Cheminform. 1, 1–10 (2009)CrossRefGoogle Scholar
  31. 31.
    Rau, M.A., Aleven, V., Rummel, N.: Supporting students in making sense of connections and in becoming perceptually fluent in making connections among multiple graphical representations. J. Educ. Psychol. 109, 355–373 (2017)CrossRefGoogle Scholar
  32. 32.
    Kellman, P.J., et al.: Perceptual learning and the technology of expertise: studies in fraction learning and algebra. Pragmat. Cogn. 16, 356–405 (2008)CrossRefGoogle Scholar
  33. 33.
    Wise, J.A., Kubose, T., Chang, N., Russell, A., Kellman, P.J.: Perceptual learning modules in mathematics and science instruction. In: Hoffman, P., Lemke, D. (eds.) Teaching and Learning in a Network World, pp. 169–176. IOS Press, Amsterdam (2000)Google Scholar
  34. 34.
    Kellman, P.J., Massey, C.M., Son, J.Y.: Perceptual learning modules in mathematics: enhancing students’ pattern recognition, structure extraction, and fluency. Top. Cogn. Sci. 2, 285–305 (2009)CrossRefGoogle Scholar
  35. 35.
    Nathan, M.J., Koedinger, K.R., Alibali, M.W.: Expert blind spot: when content knowledge eclipses pedagogical content knowledge. In: Chen, L., et al. (eds.) Proceedings of the Third International Conference on Cognitive Science, pp. 644–648. USTC Press, Beijing (2001)Google Scholar
  36. 36.
    Chin, J.M., Schooler, J.W.: Why do words hurt? Content, process, and criterion shift accounts of verbal overshadowing. Eur. J. Cogn. Psychol. 20, 396–413 (2008)CrossRefGoogle Scholar
  37. 37.
    Zhu, X.: Machine teaching: an inverse problem to machine learning and an approach toward optimal education. In: AAAI Conference on Artificial Intelligence (2015)Google Scholar
  38. 38.
    Sen, A., et al.: Machine beats human at finding the optimal sequence of visual representations for students learning of perceptual fluency. In: Boyer, K.E., Yudelson, M. (eds.) Proceedings of the 11th International Conference on Educational Data Mining, pp. 137–146. International Educational Data Mining Society, Buffalo (2018)Google Scholar
  39. 39.
    VanLehn, K.: The relative effectiveness of human tutoring, intelligent tutoring systems and other tutoring systems. Educ. Psychol. 46, 197–221 (2011)CrossRefGoogle Scholar
  40. 40.
    Rau, M.A., Mason, B., Nowak, R.: How to model implicit knowledge? Use of metric learning to assess student perceptions of visual representations. In: Barnes, T., Chi, M., Feng, M. (eds.) Proceedings of the 9th International Conference on Educational Data Mining, pp. 199–206. International Educational Data Mining Society, Raleigh (2016)Google Scholar
  41. 41.
    Van Gog, T., Paas, F.: Instructional efficiency: revisiting the original construct in educational research. Educ. Psychol. 43, 16–26 (2008)CrossRefGoogle Scholar
  42. 42.
    Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, Hillsdale (1988)zbMATHGoogle Scholar
  43. 43.
    Park, O., Lee, J.: Adaptive instructional systems. In: Jonassen, D.H. (ed.) Handbook of Research for Educational Communications and Technology, pp. 651–658. Erlbaum, Mahwah (2003)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of Wisconsin – MadisonMadisonUSA

Personalised recommendations