Productive failure in learning the concept of variance


In a study with ninth-grade mathematics students on learning the concept of variance, students experienced either direct instruction (DI) or productive failure (PF), wherein they were first asked to generate a quantitative index for variance without any guidance before receiving DI on the concept. Whereas DI students relied only on the canonical formulation of variance taught to them, PF students generated a diversity of formulations for variance but were unsuccessful in developing the canonical formulation. On the posttest however, PF students significantly outperformed DI students on conceptual understanding and transfer without compromising procedural fluency. These results challenge the claim that there is little efficacy in having learners solve problems targeting concepts that are novel to them, and that DI needs to happen before learners should solve problems on their own.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3


  1. 1.

    Initially, a condition (PF vs. DI) by teacher (teacher A vs. teacher B) MANCOVA was carried out with prior knowledge as covariate. However, the main and interaction effects of teacher were not significant. Hence, the teacher (or class) factor was collapsed, and a more straightforward MANCOVA with condition as the sole between-subjects factor has been reported.


  1. Bielaczyc, K., & Kapur, M. (2010). Playing epistemic games in science and mathematics classrooms. Educational Technology, 50(5), 19–25.

    Google Scholar 

  2. Brown, A., & Campione, J. (1994). Guided discovery in a community of learners. In K. McGilly (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp. 229–270). Cambridge: MIT Press.

    Google Scholar 

  3. Carroll, W. (1994). Using worked examples as an instructional support in the algebra classroom. Journal of Educational Psychology, 86, 360–367.

    Article  Google Scholar 

  4. Chi, M. T. H., Glaser, R., & Farr, M. J. (1988). The nature of expertise. Hillsdale: Erlbaum.

    Google Scholar 

  5. Clifford, M. M. (1984). Thoughts on a theory of constructive failure. Educational Psychologist, 19(2), 108–120.

    Article  Google Scholar 

  6. Cooper, G., & Sweller, J. (1987). The effects of schema acquisition and rule automation on mathematical problem-solving transfer. Journal of Educational Psychology, 79, 347–362.

    Article  Google Scholar 

  7. diSessa, A. A., Hammer, D., Sherin, B. L., & Kolpakowski, T. (1991). Inventing graphing: Meta-representational expertise in children. Journal of Mathematical Behavior, 10(2), 117–160.

    Google Scholar 

  8. Hardiman, P., Pollatsek, A., & Weil, A. (1986). Learning to understand the balance beam. Cognition and Instruction, 3, 1–30.

    Article  Google Scholar 

  9. Kapur, M. (2008). Productive failure. Cognition and Instruction, 26(3), 379–424.

    Article  Google Scholar 

  10. Kapur, M. (2009). Productive failure in mathematical problem solving. Instructional Science, 38(6), 523–550. doi:10.1007/s11251-009-9093-x.

    Article  Google Scholar 

  11. Kapur, M. (2010). A further study of productive failure in mathematical problem solving: Unpacking the design components. Instructional Science, 39(4), 561–579. doi:10.1007/s11251-010-9144-3.

    Article  Google Scholar 

  12. Kapur, M., & Bielaczyc, K. (2011). Classroom-based experiments in productive failure. In L. Carlson, C. Hölscher, & T. Shipley (Eds.), Proceedings of the 33rd annual conference of the cognitive science society (pp. 2812–2817). Austin: Cognitive Science Society.

    Google Scholar 

  13. Kapur, M., & Bielaczyc, K. (2012). Designing for productive failure. The Journal of the Learning Sciences, 21(1), 45–83.

    Article  Google Scholar 

  14. Kapur, M., & Kinzer, C. (2009). Productive failure in CSCL groups. International Journal of Computer-Supported Collaborative Learning (ijCSCL), 4(1), 21–46.

    Article  Google Scholar 

  15. Kapur, M., & Rummel, N. (2009). The assistance dilemma in CSCL. In A. Dimitracopoulou, C. O’Malley, D. Suthers, & P. Reimann (Eds.), Computer supported collaborative learning practices-CSCL2009 community events proceedings, Vol. 2 (pp. 37–42). International Society of the Learning Sciences.

  16. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work. Educational Psychologist, 41(2), 75–86.

    Article  Google Scholar 

  17. Klahr, D., & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. Psychological Science, 15(10), 661–667.

    Article  Google Scholar 

  18. Mathan, S., & Koedinger, K. (2003). Recasting the feedback debate: Benefits of tutoring error detection and correction skills. In U. Hoppe, F. Verdejo, & J. Kay (Eds.), Artificial intelligence in education: Shaping the future of education through intelligent technologies (pp. 13–20). Amsterdam: IOS Press.

    Google Scholar 

  19. Paas, F. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84, 429–434.

    Article  Google Scholar 

  20. Paas, F., & van Merriënboer, J. (1994). Variability of worked examples and transfer of geometrical problem solving skills: A cognitive-load approach. Journal of Educational Psychology, 86, 122–133.

    Article  Google Scholar 

  21. Roll, I. (2009). Structured invention activities to prepare students for future learning: Means, mechanisms, and cognitive processes. Pittsburgh: Thesis.

    Google Scholar 

  22. Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3(4), 207–217.

    Article  Google Scholar 

  23. Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–522.

    Article  Google Scholar 

  24. Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.

    Article  Google Scholar 

  25. Strand-Cary, M., & Klahr, D. (2008). Developing elementary science skills: Instructional effectiveness and path independence. Cognitive Development, 23(4), 488–511.

    Article  Google Scholar 

  26. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.

    Article  Google Scholar 

  27. Sweller, J. (2010). What human cognitive architecture tells us about constructivism. In S. Tobias & T. M. Duffy (Eds.), Constructivist instruction: Success or failure (pp. 127–143). New York: Routledge.

    Google Scholar 

  28. Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2, 59–89.

    Article  Google Scholar 

  29. Tobias, S., & Duffy, T. M. (2010). Constructivist instruction: Success or failure. New York: Routledge.

    Google Scholar 

  30. Trafton, J. G., & Reiser, R. J. (1993). The contribution of studying examples and solving problems to skill acquisition. In M. Polson (Ed.), Proceedings of the 15th annual conference of the cognitive science society (pp. 1017–1022). Hillsdale: Erlbaum.

    Google Scholar 

  31. Tuovinen, J. E., & Sweller, J. (1999). A comparison of cognitive load associated with discovery learning and worked examples. Journal of Educational Psychology, 91, 334–341.

    Article  Google Scholar 

  32. Van Lehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21(3), 209–249.

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Manu Kapur.


Appendix A: The complex problem scenario

Mr. Fergusson, Mr. Merino, and Mr. Eriksson are the mangers of the Supreme Football Club. They are on the lookout for a new striker, and after a long search, they short-listed three potential players: Mike Arwen, Dave Backhand, and Ivan Right. All strikers asked for the same salary, so the managers agreed that they should base their decisions on the players’ performance in the Premier League for the last 20 years. Table 4 shows the number of goals that each striker had scored between 1988 and 2007.

Table 4 Number of goals scored by three strikers in the premier league

The managers agreed that the player they hire should be a consistent performer. They decided that they should approach this decision mathematically, and would want a formula for calculating the consistency of performance for each player. This formula should apply to all players and help provide a fair comparison. The managers decided to get your help.

Please come up with a formula for consistency and show which player is the most consistent striker. Show all working and calculations on the paper provided.

Appendix B: Examples of pretest items

Central tendencies

The table below shows the timing (in minutes) for a 2.4 km run for 40 students in Class 2E1. Calculate the mean, median and mode of the timing of Class 2E1.

$$ \begin{gathered} 1 1,{ 11},{ 12},{ 12},{ 12},{ 12},{ 13},{ 13},{ 13},{ 13},{ 13},{ 14},{ 14},{ 14},{ 14},{ 14},{ 14},{ 15},{ 15},{ 15},{ 15},{ 15}, \, \hfill \\ 1 5,{ 15},{ 15},{ 15},{ 15},{ 15},{ 16},{ 16},{ 16},{ 16},{ 16},{ 16},{ 16},{ 16},{ 17},{ 17},{ 17},{ 17} \hfill \\ \end{gathered} $$


The heart rate per minute of a group of 20 adults is displayed in the dot diagram below. For example, 3 adults have a rate of 60 beats per minute. Based on this data set, how many individuals from a similar group of 40 adults would be expected to have a heart rate of 90 beats or more per minute?

Dot diagram for the heart rate per minute for a group of 20 adults


The owners of two cinemas, A and B, argue that their respective cinema enjoys a more consistent attendance. They collected the daily attendance of their cinemas for 11 days. The results of their data collection are shown below.


  Cinema A Cinema B
Day 1 69 61
Day 2 70 65
Day 3 75 91
Day 4 52 55
Day 5 57 58
Day 6 92 95
Day 7 71 67
Day 8 73 81
Day 9 74 89
Day 10 72 70
Day 11 87 93

Based on the above attendance data and statistics, which cinema do you think enjoys a more consistent attendance? Please explain mathematically and show your working.

Appendix C: Examples of posttest items

Procedural fluency item 1

Q1. Marks scored by 10 students on a test on statistics are shown below. As a measure of the variance, calculate the standard deviation of the test scores.

$$ 30, \, 50, \, 50, \, 55, \, 60, \, 60, \, 60, \, 70, \, 80, \, 90 $$

Conceptual understanding item 1

Q2. For Q1, one student came up with another measure of variance by taking the average of the sum of the difference between adjacent scores as shown below:\( \frac{{\left( {50 - 30} \right) + \left( {50 - 50} \right) + \left( {55 - 50} \right) + \left( {60 - 55} \right) + \left( {60 - 60} \right) + \left( {60 - 60} \right) + \left( {70 - 60} \right) + \left( {80 - 70} \right) + \left( {90 - 80} \right)}}{10 - 1} \) \( = 6.67 \)

How does the student’s measure of variance compare with the standard deviation as a measure of variance? Which one is better? Please explain your answer.

Procedural fluency item 2

In preparing for the Youth Olympics in 2010, the Ministry of Community, Youth and Sports had to decide the month in which to hold the games. They narrowed their options to July and August, and decided to examine rainfall data for ten randomly selected days in July and August in 2007 to make a choice. The amounts of rainfall (in millimeters) for the 2 months are shown below.


Day Rainfall in July (mm) Rainfall in August (mm)
Week 1, Day 1 32 25
Week 1, Day 3 35 31
Week 2, Day 2 35 35
Week 2, Day 4 37 37
Week 2, Day 7 37 37
Week 3, Day 2 37 37
Week 3, Day 5 38 38
Week 3, Day 7 39 39
Week 4, Day 5 40 42
Week 4, Day 6 40 49
  1. i.

    Based on the information, which month should the Ministry choose, given that they would want a month that has a consistently low amount of rainfall?

Conceptual Understanding Item 2

  1. ii.

    A few days later, the Ministry re-looked at the data and realized that they made a mistake for the figure recorded Week 4, Day 6 in July. Instead for 40 mm, the rainfall should be 60 mm. Given this new figure, which month should the Ministry choose now, if they want one that has a consistently low amount of rainfall?

Transfer item

Two Secondary Four students were nominated for the “Best Science Student” award for 2009. Muthu Kumaran is the top Physics student, while Alicia Kuan is the top Chemistry student for 2009. The table below shows the Physics and Chemistry top scorers between 1998 and 2009, with their scores presented in ascending order.


Top physics scorers for the past 12 years Top chemistry scorers for the past 12 years
Name Year Score Name Year Score
Yap Pei Ling 2006 81 Lim Jen Yi 1998 80
Cho Ying Ming 1999 83 Charissa Tan 2001 81
Bala Ayanan 2001 83 Allan Wu 2000 83
Mohammad Azhar 2000 84 Ali Salim 2002 85
Matilda Tay 2002 84 Derick Chan 1999 89
Louis Ho 2005 85 David Tan 2003 90
Tham Jing Ling 2004 85 Abdul Basher 2005 90
Jodie Ang 1998 85 Fredrick Chay 2004 94
Jeremy Goh 2003 85 Linda Siew 2006 95
Chee Haw Ren 2006 85 Terry Lee 2008 96
Susan Teo 2005 86 Low Ming Lee 2007 98
Muthu Kumaran 2009 94 Alicia Kwan 2009 99
Mean 85 Mean 90

Both Muthu and Alicia are the best performers in their respective subjects for the past 12 years. Because there is only one “Best Science Student” award, who do you think deserves the award more? Please explain your decision mathematically and show your working.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Kapur, M. Productive failure in learning the concept of variance. Instr Sci 40, 651–672 (2012).

Download citation


  • Problem solving
  • Productive failure
  • Multiple representations
  • Mathematics
  • Classroom-based research