Combining Multiple External Representations and Refutational Text: An Intervention on Learning to Interpret Box Plots

  • Stephanie LemEmail author
  • Goya Kempen
  • Eva Ceulemans
  • Patrick Onghena
  • Lieven Verschaffel
  • Wim Van Dooren


Box plots are frequently misinterpreted and educational attempts to correct these misinterpretations have not been successful. In this study, we used two instructional techniques that seemed powerful to change the misinterpretation of the area of the box in box plots, both separately and in combination, leading to three experimental conditions, next to a control condition. First, we used multiple external representations: Histograms were used as an overlay on box plots in order to give students a better insight in the way box plots represent data distributions. Second, we used refutational text to explicitly name and invalidate the area misinterpretation of box plots. Third, we combined multiple external representations and refutational text. A box plot test showed that students in the refutation and combination condition scored statistically significant better than students in the control condition with respect to the misinterpretation of interest. The condition with multiple external representations scored in between. The implications of these results for theory and educational practice are discussed.


Intervention Multiple external representations Refutational text Box plots 



Stephanie Lem holds a post-doctoral fellowship of the Research Foundation - Flanders (FWO). This research was partially supported by grant GOA 2006/01 “Developing adaptive expertise in mathematics education” from the Research Fund KU Leuven, Belgium.

Supplementary material

10763_2014_9604_MOESM1_ESM.doc (886 kb)
ESM 1 (DOC 886 kb)


  1. Acevedo Nistal, A., Van Dooren, W., Clarebout, G., Elen, J. & Verschaffel, L. (2009). Conceptualising, investigating and stimulating representational flexibility in mathematical problem solving and learning: A critical review. ZDM-The International Journal on Mathematics Education, 41, 627–636.CrossRefGoogle Scholar
  2. Ainsworth, S. (1999). The functions of multiple representations. Computers & Education, 33, 131–152.CrossRefGoogle Scholar
  3. Ainsworth, S. (2008). The educational value of multiple-representations when learning complex scientific concepts. In J. K. Gilbert, M. Reiner & M. Nakhleh (Eds.), Visualization: Theory and practice in science education (pp. 191–208). Dordrecht, The Netherlands: Springer.Google Scholar
  4. Babai, R., Brecher, T., Stavy, R. & Tirosh, D. (2006). Intuitive interference in probabilistic reasoning. International Journal of Science and Mathematics Education, 4, 627–639.CrossRefGoogle Scholar
  5. Bakker, A. (2004). Design research in statistics education: On symbolizing and computer tools. Utrecht: Freudenthal Institute.Google Scholar
  6. Bakker, A., Biehler, R. & Konold, C. (2005). Should young students learn about box plots? In G. Burrill & M. Camden (Eds.), Curricular development in statistics education: International Association for Statistical Education 2004 Roundtable (pp. 163–173). Voorburg, The Northland: International Statistical Institute.Google Scholar
  7. Cobb, P., Gravemeijer, K. P. E., Bowers, J. & Doorman, M. (1997). Statistical minitools [Computer software]. Nashville & Utrecht: Vanderbult University, TN & Freudenthal Institute, Utrecht University. Retrieved from
  8. Cobb, P., McClain, K. & Gravemeijer, K. (2003). Learning about statistical covariation. Cognition and Instruction, 21, 1–78.CrossRefGoogle Scholar
  9. Corradi, D., Elen, J. & Clarebout, G. (2012). Understanding and enhancing the use of multiple external representations in chemistry education. Journal of Science Education and Technology, 21, 780–795.CrossRefGoogle Scholar
  10. Craeghs, B. (2012). Box plot, quartile plot en offset quartile plot: Verschillende externe representaties, dezelfde interpretatiemoeilijkheden? [Box plot, quartile plot and offset quartile plot: Different representations, the same interpretation difficulties?] (Unpublished master thesis). KU Leuven, Belgium.Google Scholar
  11. De Vries, E., Demetriadis, S. & Ainsworth, S. (2009). External representations for learning: Headed towards a digital culture. In N. Balacheff, S. Ludvigsen, T. De Jong, A. Lazonder & S. Barnes (Eds.), External representations for learning (pp. 137–153). Dordrecht, The Netherlands: Springer.Google Scholar
  12. Evans, J., St., B. T. & Over, D. E. (1996). Rationality and Reasoning. Hove, England: Psychology Press.Google Scholar
  13. Faul, F. & Erdfelder, E. (1992). G*Power (version 3.1.7) [Computer software]. Bonn: Bonn University.Google Scholar
  14. Gill, M. G., Ashton, P. T. & Algina, J. (2004). Changing preservice teachers’ epistemological beliefs about teaching and learning in mathematics: An intervention study. Contemporary Educational Psychology, 29, 164–185.CrossRefGoogle Scholar
  15. Hynd, C. R. (2001). Refutational texts and the change process. International Journal of Educational Research, 35, 699–714.CrossRefGoogle Scholar
  16. Kahneman, D. (2000). A psychological point of view: Violations of rational rules as a diagnostic of mental processes. Behavioral and Brain Sciences, 23, 681–683. doi: 10.1037/0003-066X.58.9.697.CrossRefGoogle Scholar
  17. Landis, J. R. & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometric, 33, 159–174. doi: 10.2307/2529310.
  18. Lem, S., Onghena, P., Verschaffel, L. & Van Dooren, W. (2012). On the misinterpretation of histograms and box plots. Educational Psycholog, 33, 155–174.CrossRefGoogle Scholar
  19. Lem, S., Onghena, P., Verschaffel, L. & Van Dooren, W. (2013a). External representations for data distributions: In search of cognitive fit. Statistics Education Research Journal, 12(1), 4–19.Google Scholar
  20. Lem, S., Onghena, P., Verschaffel, L. & Van Dooren, W. (2013b). The heuristic interpretation of box plots. Learning and Instruction, 26(4), 22–35.CrossRefGoogle Scholar
  21. Lem, S., Onghena, P., Verschaffel, L. & Van Dooren, W. (2013c). Experts’ misinterpretation of box plots - a dual processing approach. Manuscript submitted for publication.Google Scholar
  22. Lem, S., Onghena, P., Verschaffel, L., Van Dooren, W. (2014). Interpreting histograms. As easy as it seems? European Journal of Psychology of Education. Advance online publication. doi: 10.1007/s10212-014-0213-x
  23. McCrudden, M. T. & Kendeou, P. (2012). Exploring the link between cognitive processes and learning from refutational text. Journal of Research in Reading, 37(S1), 1–25.Google Scholar
  24. Palmer, D. H. (2003). Investigating the relationship between refutational text and conceptual change. Science Education, 87, 663–684.CrossRefGoogle Scholar
  25. Pierce, R. & Chick, H. (2013). Workplace statistical literacy for teachers: Interpreting box plots. Mathematics Education Research Journal, 25, 189–205.CrossRefGoogle Scholar
  26. Pierce, R., Chick, H. & Wander, R. (2014). Improving teachers’ professional statistical literacy. In H. MacGillivray, M. Martin & B. Phillips (Eds.), Topics from Australian Conferences on Teaching Statistics: OZCOTS 2008–2012 (pp. 295–309). New York, NY: Springer.Google Scholar
  27. Sinatra, G. M. & Broughton, S. H. (2011, August). Understanding the refutation text effect in conceptual change research: Multiple perspectives. In W. Van Dooren (Chair), The Role of Refutational Texts in Achieving Conceptual Change. Symposium presented at the biannual meeting of the European Association for Research on Learning and Instruction, Exeter, England.Google Scholar
  28. Stavy, R. & Tirosh, D. (2000). How students (mis-)understand science and mathematics. New York, NY: Teachers College Press.Google Scholar
  29. Tippett, C. D. (2010). Refutation text in science education: A review of two decades of research. International Journal of Science and Mathematics Education, 8, 951–970.CrossRefGoogle Scholar
  30. Tsui, C.-Y. & Treagust, D. F. (2003). Genetics reasoning with multiple external representations. Research in Science Education, 33, 111–135.CrossRefGoogle Scholar
  31. Tufte, E. R. (1983). The visual display of quantitative information. Cheshire, England: Graphic Press.Google Scholar
  32. Van den Broek, P. & Kendeou, P. (2008). Cognitive processes in comprehension of science texts: The role of co-activation in confronting misconceptions. Applied Cognitive Psychology, 22, 335–351.CrossRefGoogle Scholar
  33. Van Dooren, W., De Bock, D., Hessels, A., Janssens, D. & Verschaffel, L. (2004). Remedying secondary school students’ illusion of linearity: A teaching experiment aiming at conceptual change. Learning and Instruction, 14, 485–501.CrossRefGoogle Scholar
  34. Waldrip, B. & Prain, V. (2012). Developing an understanding of ions in junior secondary school chemistry. International Journal of Science and Mathematics Education, 10, 1191–1213.CrossRefGoogle Scholar

Copyright information

© Ministry of Science and Technology, Taiwan 2014

Authors and Affiliations

  • Stephanie Lem
    • 1
    Email author
  • Goya Kempen
    • 1
  • Eva Ceulemans
    • 2
  • Patrick Onghena
    • 2
  • Lieven Verschaffel
    • 1
  • Wim Van Dooren
    • 1
  1. 1.Centre for Instructional Psychology and TechnologyKU LeuvenLeuvenBelgium
  2. 2.Methodology of Educational Sciences Research GroupKU LeuvenLeuvenBelgium

Personalised recommendations