# Delayed Learning Effects with Erroneous Examples: a Study of Learning Decimals with a Web-Based Tutor

- First Online:

## Abstract

Erroneous examples – step-by-step problem solutions with one or more errors for students to find and fix – hold great potential to help students learn. In this study, which is a replication of a prior study (Adams et al. 2014), but with a much larger population (390 vs. 208), middle school students learned about decimals either by working with interactive, web-based erroneous examples or with more traditional supported problems to solve. The erroneous examples group was interactively prompted to find, explain, and fix errors in decimal problems, while the problem-solving group was prompted to solve the same decimal problems and explain their solutions. Both groups were given correctness feedback on their work by the web-based program. Although the two groups did not differ on an immediate post-test, the erroneous examples group performed significantly better on a delayed test, given a week after the initial post-test (*d* = .33, for gain scores), replicating the pattern of the prior study. Interestingly, the problem solving group reported liking the intervention more than the erroneous examples group (*d* = .21 for liking rating in a questionnaire) and found the user interface easier to interact with (*d* = .37), suggesting that what students like does not always lead to the best learning outcomes. This result is consistent with that of *desirable difficulty* studies, in which a more cognitively challenging learning task results in deeper and longer-lasting learning.

### Keywords

Erroneous examples Problem solving Mathematics learning Intelligent tutoring systems### References

- Adams, D., McLaren, B. M., Durkin, K., Mayer, R.E., Rittle-Johnson, B., Isotani, S., & Van Velsen, M. (2014). Using erroneous examples to improve mathematics learning with a web-based tutoring system.
*Computers in Human Behavior, 36C (2014),*401–411. Elsevier. doi: 10.1016/j.chb.2014.03.053. - Aleven, V., McLaren, B. M., Sewall, J., & Koedinger, K. R. (2009). A new paradigm for intelligent tutoring systems: example-tracing tutors.
*International Journal of Artificial Intelligence in Education, 19*(2), 105–154.Google Scholar - Aleven, V., Myers, E., Easterday, M., & Ogan, A. (2010). Toward a framework for the analysis and design of educational games. In:
*Proceedings of the 2010 I.E. International Conference on Digital Game and Intelligent Toy Enhanced Learning*. (pp. 69–76). doi:10.1109/DIGITEL.2010.55. - Anthony, L. (2008).
*Developing handwriting-based Intelligent Tutors to enhance mathematics learning*. Unpublished doctoral dissertation, Carnegie Mellon University, USA.Google Scholar - Atkinson, R. K., Renkl, A., & Merrill, M. M. (2003). Transitioning from studying examples to solving problems: combining fading with prompting fosters learning.
*Journal of Educational Psychology, 95*, 774–783.CrossRefGoogle Scholar - Booth, J. L., Lange, K. E., Koedinger, K. R., & Newton, K. J. (2013). Using example problems to improve student learning in algebra: differentiating between correct and incorrect examples.
*Learning and Instruction, 25*, 24–34.CrossRefGoogle Scholar - Borasi, R. (1996).
*Reconceiving mathematics instruction: A focus on errors*. Ablex Publishing Corporation.Google Scholar - Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999).
*How people learn: Brain, mind, experience, and school*. Washington: National Academy Press.Google Scholar - Brueckner, L. J. (1928). Analysis of difficulties in decimals.
*Elementary School Journal, 29*, 32–41.CrossRefGoogle Scholar - Catrambone, R. (1998). The subgoal learning model: Creating better examples so that students can solve novel problems.
*Journal of Experimental Psychology: General, 127*(4), 355–376.CrossRefGoogle Scholar - Chi, M. T. H. (2000). Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. In R. Glaser (Ed.),
*Advances in instructional psychology*(pp. 161–238). Mahwah: Lawrence Erlbaum Associates, Inc.Google Scholar - Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, R., & Glaser, R. (1989). Self explanations: how students study and used examples in learning to solve problems.
*Cognitive Science, 13*, 145–182.CrossRefGoogle Scholar - Chi, M. T. H., DeLeeuw, N., Chiu, M.-H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding.
*Cognitive Science, 25*(4), 471–533.MATHCrossRefGoogle Scholar - College Board (2015). Identifying sentence errors. From the College Board PSAT/NMSQT website: https://www.collegeboard.org/psat-nmsqt/preparation/writing-skills/sentence-errors.
- Dunlowsky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: promising directions from cognitive and educational psychology.
*Psychological Science in the Public Interest, 14*(1), 4–58.CrossRefGoogle Scholar - Durkin, K., & Rittle-Johnson, B. (2012). The effectiveness of using incorrect examples to support learning about decimal magnitude.
*Learning and Instruction, 22*, 206–214.CrossRefGoogle Scholar - Fiorella, L., & Mayer, R. E. (2015).
*Learning as a generative activity: Eight learning strategies that improve understanding*. New York: Cambridge University Press.CrossRefGoogle Scholar - Gee, J. P. (2003).
*What video games have to teach us about learning and literacy*(1st ed.). New York: Palgrave Macmillan.Google Scholar - Glasgow, R., Ragan, G., Fields, W. M., Reys, R., & Wasman, D. (2000). The decimal dilemma.
*Teaching Children Mathematics, 7*(2), 89–93.Google Scholar - Graeber, A., & Tirosh, D. (1988). Multiplication and division involving decimals: preservice elementary teachers’ performance and beliefs.
*Journal of Mathematics Behavior, 7*, 263–280.Google Scholar - Groβe, C. S., & Renkl, A. (2007). Finding and fixing errors in worked examples: can this foster learning outcomes?
*Learning and Instruction, 17*(6), 612–634.CrossRefGoogle Scholar - Gunderman, R. B., & Burdick, E. J. (2007). Error and opportunity.
*American Journal of Roentgenology, 188*(4), 901–903.CrossRefGoogle Scholar - Guthrie, E. R. (1952).
*The psychology of learning*. New York: Harper & Brothers.Google Scholar - Hausmann, R. G. M., & Chi, M. T. H. (2002). Can a computer interface support self-explanation?
*International Journal of Cognitive Technology, 7*, 4–14.Google Scholar - Hiebert, J. (1992). Mathematical, cognitive, and instructional analyses of decimal fractions. Chapter 5 in
*Analysis of arithmetic for mathematics teaching*, pp 283–322. Lawrence Erlbaum.Google Scholar - Hiebert, J., & Wearne, D. (1985). A model of students’ decimal computation procedures.
*Cognition and Instruction, 2*, 175–205.CrossRefGoogle Scholar - Huang, T.-H., Liu, Y.-C., & Shiu, C.-Y. (2008). Construction of an online learning system for decimal numbers through the use of cognitive conflict strategy.
*Computers & Education, 50*, 61–76.CrossRefGoogle Scholar - Hull, C. L. (1952).
*A behavior system: An introduction to behavior theory concerning the individual organism*. New Haven: Yale University Press.Google Scholar - Irwin, K. C. (2001). Using everyday knowledge of decimals to enhance understanding.
*Journal for Research in Mathematics Education, 32*(4), 399–420.MathSciNetCrossRefGoogle Scholar - Isotani, S., McLaren, B. M., & Altman, M. (2010). Towards intelligent tutoring with erroneous examples: A taxonomy of decimal misconceptions. In V. Aleven, J. Kay, & J. Mostow (Eds.),
*Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS-10), Lecture Notes in Computer Science, 6094*(pp. 346–348). Berlin: Springer.Google Scholar - Johnson, C. I., & Mayer, R. E. (2010). Applying the self-explanation principle to multimedia learning in a computer-based game-like environment.
*Computers in Human Behavior, 26*, 1246–1252.CrossRefGoogle Scholar - Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples.
*Journal of Educational Psychology, 93*, 579–588.CrossRefGoogle Scholar - King, A. (1994). Guiding knowledge construction in the classroom: effects of teaching children how to question and how to explain.
*American Educational Research Journal, 31*(2), 338–368.CrossRefGoogle Scholar - Lomas, J. D., Patel, K., Forlizzi, J., & Koedinger, K. (2013).
*Optimizing challenge in an educational game using large-scale design experiments. Proceedings of CHI2013*. New York: ACM Press.Google Scholar - Mayer, R. E., & Johnson, C. I. (2010). Adding instructional features that promote learning in a game-like environment.
*Journal of Educational Computing Research, 42*, 241–265.CrossRefGoogle Scholar - McLaren, B. M., Lim, S., & Koedinger, K. R. (2008). When and how often should worked examples be given to students? New results and a summary of the current state of research. In
*Proceedings of the 30th Annual Conference of the Cognitive Science Society*(pp. 2176–2181). Austin: Cognitive Science Society.Google Scholar - Moreno, R., & Park, B. (2010). Cognitive load theory: Historical development and relation to other theories. In J. L. Plass, R. Moreno, & R. Brünken (Eds.),
*Cognitive Load Theory*. Cambridge: Cambridge University Press.Google Scholar - National Health Care, U.K. (2013), Intrathecal injection error video: https://www.youtube.com/watch?v=cipFuDxiF2Y.
- National Mathematics Advisory Panel. (2008).
*Foundations for success: The final report of the National Mathematics Advisory Panel*. Washington: U.S. Department of Education.Google Scholar - Paas, F., & van Merriënboer, J. (1994). Variability of worked examples and transfer of geometrical problem-solving skills: a cognitive-load approach.
*Journal of Educational Psychology, 86*(1), 122–133.CrossRefGoogle Scholar - Putt, I. J. (1995). Preservice teachers ordering of decimal numbers: when more is smaller and less is larger!
*Focus on Learning Problems in Mathematics, 17*(3), 1–15.Google Scholar - Renkl, A. (2002). Worked-out examples: instructional explanations support learning by self explanation.
*Learning and Instruction, 12*, 529–556.CrossRefGoogle Scholar - Renkl, A. (2014). The worked examples principle in multimedia learning. In R. E. Mayer (Ed.),
*The Cambridge handbook of multimedia learning*(2nd ed., pp. 391–412). New York: Cambridge University Press.Google Scholar - Renkl, A., & Atkinson, R. K. (2010). Learning from worked-out examples and problem solving. In J. L. Plass, R. Moreno, & R. Brünken (Eds.),
*Cognitive Load Theory*. Cambridge: Cambridge University Press.Google Scholar - Resnick, L. B., Nesher, P., Leonard, F., Magone, M., Omanson, S., & Peled, I. (1989). Conceptual bases of arithmetic errors: the case of decimal fractions.
*Journal for Research in Mathematics Education, 20*(1), 8–27.CrossRefGoogle Scholar - Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: an iterative process.
*Journal of Educational Psychology, 93*, 346–362.CrossRefGoogle Scholar - Sackur-Grisvard, C., & Léonard, F. (1985). Intermediate cognitive organizations in the process of learning a mathematical concept: the order of positive decimal numbers.
*Cognition and Instruction, 2*, 157–174.CrossRefGoogle Scholar - Salden, R. J. C. M., Aleven, V., Schwonke, R., & Renkl, A. (2010). The expertise reversal effect and worked examples in tutored problem solving.
*Instructional Science, 38*, 289–307.CrossRefGoogle Scholar - Schwonke, R., Renkl, A., Krieg, C., Wittwer, J., Aleven, V., & Salden, R. (2009). The worked-example effect: not an artefact of lousy control conditions.
*Computers in Human Behavior, 25*(2009), 258–266.CrossRefGoogle Scholar - Shoebottom, P. (2015). Error correction. From the Frankfurt International School website: http://esl.fis.edu/grammar/correctText/.
- Siegler, R. S. (2002). Microgenetic studies of self-explanation. In N. Granott & J. Parziale (Eds.),
*Microdevelopment, Transition Processes in Development and Learning*(pp. 31–58). Cambridge: Cambridge University Press.CrossRefGoogle Scholar - Skinner, B. F. (1938).
*The behavior of organisms: An experimental analysis*. New York: Appleton-Century.Google Scholar - Stacey, K. (2005). Travelling the road to expertise: A longitudinal study of learning. In. H. Chick & J. Vincent (Eds.),
*Proceedings of the 29th Conference of the International Group for the Psychology of Mathematics Education*(vol 1, pp.19–36). University of Melbourne: PME.Google Scholar - Stacey, K., Helme, S., & Steinle, V. (2001). Confusions between decimals, fractions and negative numbers: A consequence of the mirror as a conceptual metaphor in three different ways. In M. v. d. Heuvel-Panhuizen (Ed.),
*Proceedings of the 25th Conference of the International Group for the Psychology of Mathematics Education*(Vol. 4, pp. 217–224). Utrecht: PME.Google Scholar - Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra.
*Cognition and Instruction, 2*, 59–89.CrossRefGoogle Scholar - Sweller, J., Van Merriënboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design.
*Educational Psychology Review, 10*, 251–296.CrossRefGoogle Scholar - Sweller, J., Ayres, P., & Kalyuga, S. (2011).
*Cognitive load theory*. New York: Springer.CrossRefGoogle Scholar - Swigger, K. M., & Wallace, L. F. (1988). A discussion of past programming errors and their effect on learning Assembly language.
*The Journal of Systems and Software, 8*, 395–399.CrossRefGoogle Scholar - The Doctor’s Company (2013). Video on learning from errors. https://www.youtube.com/watch?v=-ol5jM7YHH0.
- Tsamir, P. & Tirosh, D. (2003). In-service mathematics teachers’ views of errors in the classroom. In International Symposium: Elementary Mathematics Teaching, Prague.Google Scholar
- Tsovaltzi, D., Melis, E., & McLaren, B. M. (2012). Erroneous examples: Effects on learning fractions in a web-based setting.
*International Journal of Technology Enhanced Learning (IJTEL).*V4 N3/4 2012 pp 191–230.Google Scholar - WHO (2014). World Health Organization (WHO): “Learning from Errors to Prevent Harm” workshop. http://www.who.int/patientsafety/education/curriculum/PSP_mpc_topic-05.pdf.
- Widjaja, W., Stacey, K., & Steinle, V. (2011). Locating Negative Decimals on the Number Line: Insights into the Thinking of Pre-service Primary Teachers.
*Journal of Mathematical Behavior*. 30, 80–91. http://dx.doi.org/10.1016/j.jmathb.2010.11.004. - Wylie, R., & Chi, M. T. H. (2014). The self-explanation principle in multimedia learning. In R. E. Mayer (Ed.),
*The Cambridge handbook of multimedia learning*(2nd ed., pp. 413–432). New York: Cambridge University Press.Google Scholar - Yue, C. L., Bjork, E. L., & Bjork, R. A. (2013). Reducing verbal redundancy in multimedia learning: an undesired desirable difficulty.
*Journal of Educational Psychology, 105*, 266–277.CrossRefGoogle Scholar - Zhu, X., & Simon, H. A. (1987). Learning mathematics from examples and by doing.
*Cognition and Instruction, 4*(3), 137–66.CrossRefGoogle Scholar