Skip to main content
SpringerLink
Log in

Productive failure in CSCL groups

  • Published:
International Journal of Computer-Supported Collaborative Learning Aims and scope Submit manuscript

Abstract

This study was designed as a confirmatory study of work on productive failure (Kapur, Cognition and Instruction, 26(3), 379–424, 2008). N = 177, 11th-grade science students were randomly assigned to solve either well- or ill-structured problems in a computer-supported collaborative learning (CSCL) environment without the provision of any external support structures or scaffolds. After group problem solving, all students individually solved well-structured problems followed by ill-structured problems. Compared to groups who solved well-structured problems, groups who solved ill-structured problems expectedly struggled with defining, analyzing, and solving the problems. However, despite failing in their collaborative problem-solving efforts, these students outperformed their counterparts from the well-structured condition on the individual near and far transfer measures subsequently, thereby confirming the productive failure hypothesis. Building on the previous study, additional analyses revealed that neither preexisting differences in prior knowledge nor the variation in group outcomes (quality of solutions produced) seemed to have had any significant effect on individual near and far transfer measures, lending support to the idea that it was the nature of the collaborative process that explained productive failure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. As a rule of thumb, partial η2 = .01 is considered a small, .06 medium, and .14 a large effect size (Cohen, 1977).

  2. The software program Multiple Episode Protocol Analysis (MEPA) developed by Dr. Gijsbert Erkens was used for carrying out the LSA. See http://edugate.fss.uu.nl/mepa/index.htm.

  3. It is important to note that in the initial study (Kapur, 2008), LSA analysis was triangulated through an interactional analysis of discussion excerpts explaining the various transitions and feedback loops.

References

  • Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1–14.

    Article  Google Scholar 

  • Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359.

    Article  Google Scholar 

  • Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis. New York: Cambridge University Press.

    Google Scholar 

  • Bielaczyc, K. (2006). Designing social infrastructure: Critical issues in creating learning environments with technology. The Journal of the Learning Sciences, 15(3), 301–329.

    Article  Google Scholar 

  • Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. In A. Iran-Nejad, & P. D. Pearson (Eds.), Review of research in education, 24 (pp. 61–101). Washington, DC: American Educational Research Association.

    Google Scholar 

  • Bromme, R., Hesse, F. W., & Spada, H. (2005). Barriers and biases in computer-mediated knowledge communication-and how they may be overcome. New York, NY: Springer.

    Google Scholar 

  • Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42.

    Google Scholar 

  • Chatterji, M. (2003). Designing and using tools for educational assessment. Boston: Allyn & Bacon.

    Google Scholar 

  • Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6(3), 271–315.

    Article  Google Scholar 

  • Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121–152.

    Article  Google Scholar 

  • Cho, K. L., & Jonassen, D. H. (2002). The effects of argumentation scaffolds on argumentation and problem solving. Educational Technology, Research and Development, 50(3), 5–22.

    Article  Google Scholar 

  • Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York: Academic Press.

    Google Scholar 

  • Collins, H. (1985). Changing order. London: Sage.

    Google Scholar 

  • Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods: A handbook for research on interactions. New York: Irvington.

    Google Scholar 

  • de Groot, A. D. (1965). Thought and choice in chess. The Hague, NL: Mouton.

    Google Scholar 

  • Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can we support CSCL (pp. 61–91). Heerlen, NL: Open Universiteit Nederland.

    Google Scholar 

  • Dillenbourg, P., & Jermann, P. (2007). Designing integrative scripts. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 275–302). New York, NY: Springer.

    Chapter  Google Scholar 

  • Erkens, G., Kanselaar, G., Prangsma, M., & Jaspers, J. (2003). Computer support for collaborative and argumentative writing. In E. De Corte, L. Verschaffel, N. Entwistle, & J. van Merrienboer (Eds.), Powerful learning environments: Unravelling basic components and dimensions (pp. 157–176). Amsterdam: Elsevier Science.

    Google Scholar 

  • Ertl, B., Kopp, B., & Mandl, H. (2007). Supporting collaborative learning in videoconferencing using collaboration scripts and content schemes. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 213–236). New York, NY: Springer.

    Chapter  Google Scholar 

  • Fischer, F., Kollar, I., Mandl, H., & Haake, J. (2007). Perspectives on collaboration scripts. In F. Fischer, H. Mandl, J. Haake, & I. Kollar (Eds.), Scripting computer-supported collaborative learning (pp. 1–10). New York, NY: Springer.

    Chapter  Google Scholar 

  • Fischer, F., & Mandl, H. (2005). Knowledge convergence in computer-supported collaborative learning: The role of external representation tools. The Journal of the Learning Sciences, 14(3), 405–441.

    Article  Google Scholar 

  • Ge, X., & Land, S. M. (2003). Scaffolding students’ problem-solving processes in an ill-structured task using question prompts and peer interactions. Educational Technology, Research and Development, 51(1), 21–38.

    Article  Google Scholar 

  • Giles, J. (2006). The trouble with replication. Nature, 442, 344–347.

    Article  Google Scholar 

  • Goel, V., & Pirolli, P. (1992). The structure of design problem spaces. Cognitive Science, 16, 395–429.

    Article  Google Scholar 

  • Hatano, G., & Inagaki, K. (1986). Two courses of expertise. In H. Stevenson, H. Azuma, & K. Hakuta (Eds.), Child Development and Education in Japan (pp. 262–272). New York: Freeman.

    Google Scholar 

  • Hewitt, J. (2005). Towards an understanding of how threads die in asynchronous computer conferences. The Journal of the Learning Sciences, 14(4), 567–589.

    Article  Google Scholar 

  • Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235–266.

    Article  Google Scholar 

  • Holland, J. H. (1995). Hidden order: How adaptation builds complexity. New York: Addison-Wesley.

    Google Scholar 

  • Jonassen, D. H. (2000). Towards a design theory of problem solving. Educational Technology, Research and Development, 48(4), 63–85.

    Article  Google Scholar 

  • Jonassen, D. H., & Kwon, H. I. (2001). Communication patterns in computer-mediated vs. face-to-face group problem solving. Educational Technology, Research and Development, 49(1), 35–52.

    Article  Google Scholar 

  • Kapur, M. (2008). Productive failure. Cognition and Instruction, 26(3), 379–424.

    Article  Google Scholar 

  • Kapur, M., Dickson, L., & Toh, P. Y. (2008). Productive failure in mathematical problem solving. In B. C. Love, K. McRae, & V. M. Sloutsky (Eds.), Proceedings of the 30th Annual Conference of the Cognitive Science Society (pp. 1717–1722). Austin, TX: Cognitive Science Society.

    Google Scholar 

  • Kapur, M., & Kinzer, C. (2007). The effect of problem type on collaborative problem solving in a synchronous computer-mediated environment. Educational Technology, Research and Development, 55(5), 439–459.

    Article  Google Scholar 

  • Kapur, M., Voiklis, J., & Kinzer, C. (2007). Sensitivities to early exchange in synchronous computer-supported collaborative learning (CSCL) groups. Computers and Education, 51, 54–66.

    Article  Google Scholar 

  • Kauffman, S. (1995). At home in the universe. New York: Oxford University Press.

    Google Scholar 

  • King, A. (2007). Scripting collaborative learning processes: A cognitive perspective. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 13–38). New York, NY: Springer.

    Chapter  Google Scholar 

  • Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86.

    Article  Google Scholar 

  • Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hamalainen, R., Hakkinen, P., & Fischer, F. (2007). Specifying computer-supported collaboration scripts. International Journal of Computer-Supported Collaborative Learning, 2(2–3), 211–224.

    Article  Google Scholar 

  • Kyllonen, P. C., & Lajoie, S. P. (2003). Reassessing aptitude: Introduction to a special issue in honor of Richard E. Snow. Educational Psychologist, 38, 79–83.

    Article  Google Scholar 

  • Lin, X., Hmelo, C., Kinzer, C., & Secules, T. J. (1999). Designing technology to support reflection. Educational Technology, Research and Development, 47(3), 43–62.

    Article  Google Scholar 

  • Lund, K., Molinari, G., Sejourne, A., & Baker, M. (2007). How do argumentation diagrams compare when students pairs use them as a means for debate or as a tool for representing debate? International Journal of Computer-Supported Collaborative Learning, 2(2–3), 273–296.

    Article  Google Scholar 

  • Marton, F. (2007). Sameness and difference in transfer. The Journal of the Learning Sciences, 15(4), 499–535.

    Article  Google Scholar 

  • McNamara, D. S. (2001). Reading both high-coherence and low-coherence texts: Effects of text sequence and prior knowledge. Canadian Journal of Experimental Psychology, 55(1), 51–62.

    Google Scholar 

  • McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14(1), 1–43.

    Article  Google Scholar 

  • Mestre, J. P. (2005). Transfer of learning from a modern multidisciplinary perspective. Greenwich, CT: Information Age.

    Google Scholar 

  • Mirza, N. M., Tartas, V., Perret-Clermont, A., & de Pietro, J. (2007). Using graphical tools in a phased activity for enhancing dialogical skills: An example with Digalo. International Journal of Computer-Supported Collaborative Learning (ijCSCL), 2(2–3), 247–272.

    Article  Google Scholar 

  • Petroski, H. (2006). Success through failure: The paradox of design. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Poole, M. S., & Holmes, M. E. (1995). Decision development in computer-assisted group decision making. Human Communications Research, 22(1), 90–127.

    Article  Google Scholar 

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models. Thousand Oaks, CA: Sage Publications.

    Google Scholar 

  • Reiser, B. J. (2004). Scaffolding complex learning: The mechanisms of structuring and problematizing student work. The Journal of the Learning Sciences, 13(3), 423–451.

    Article  Google Scholar 

  • Rummel, N., & Spada, H. (2007). Can people learn in computer-mediated collaboration by following a script? In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 39–56). New York, NY: Springer.

    Chapter  Google Scholar 

  • Sandberg, I. (1994). Human competence at work: An interpretative approach. Göteborg, Sweden: BAS.

    Google Scholar 

  • Sandoval, W. A., & Millwood, K. A. (2005). The quality of students’ use of evidence in written scientific explanations. Cognition and Instruction, 23(1), 23–55.

    Article  Google Scholar 

  • Scardamalia, M., & Bereiter, C. (2003). Knowledge building. In J. W. Guthrie (Ed.), Encyclopedia of Education. New York, NY: Macmillan Reference.

    Google Scholar 

  • Schellens, T., Van Keer, H., De Wever, B., & Valcke, M. (2007). Scripting by assigning roles: Does it improve knowledge construction in asynchronous discussion groups? International Journal of Computer-Supported Collaborative Learning (ijCSCL), 2(2–3), 225–246.

    Article  Google Scholar 

  • Schwartz, D. L. (1995). The emergence of abstract dyad representations in dyad problem solving. The Journal of the Learning Sciences, 4(3), 321–354.

    Article  Google Scholar 

  • Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–522.

    Article  Google Scholar 

  • Schwartz, D. L., Bransford, J. D., & Sears, D. (2005). Efficiency and innovation in transfer. In J. P. Mestre (Ed.), Transfer of learning from a modern multidisciplinary perspective (pp. 1–52). Greenwich, CT: Information Age Publishing.

    Google Scholar 

  • Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.

    Article  Google Scholar 

  • Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis. London: Sage Publications.

    Google Scholar 

  • Spiro, R. J., Feltovich, R. P., Jacobson, M. J., & Coulson, R. L. (1992). Cognitive flexibility, constructivism, and hypertext. In T. M. Duffy, & D. H. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation (pp. 1–5). Hillsdale, NJ : Erlbaum.

    Google Scholar 

  • Stahl, G. (2005). Group cognition in computer-assisted collaborative learning. Journal of Computer Assisted Learning, 21, 79–90.

    Article  Google Scholar 

  • Stahl, G. (2007). Scripting group cognition. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 327–336). New York, NY: Springer.

    Chapter  Google Scholar 

  • Suthers, D. D. (2006). Technology affordances for intersubjective meaning making: A research agenda for CSCL. International Journal of Computer-Supported Collaborative Learning (ijCSCL), 1(3), 315–337.

    Article  Google Scholar 

  • Suthers, D. D., & Hundhausen, C. (2003). An empirical study of the effects of representational guidance on collaborative learning. The Journal of the Learning Sciences, 12(2), 183–219.

    Article  Google Scholar 

  • VanLehn, K. (1999). Rule learning events in the acquisition of a complex skill: An evaluation of cascade. The Journal of the Learning Sciences, 8(1), 71–125.

    Article  Google Scholar 

  • VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21(3), 209–249.

    Article  Google Scholar 

  • Voss, J. F. (1988). Problem solving and reasoning in ill-structured domains. In C. Antaki (Ed.), Analyzing everyday explanation: A casebook of methods pp. 74–93. London: Sage Publications.

    Google Scholar 

  • Voss, J. F. (2005). Toulmin’s model and the solving of ill-structured problems. Argumentation, 19, 321–329.

    Article  Google Scholar 

  • Wampold, B. E. (1992). The intensive examination of social interaction. In T.R. Kratochwill, & J.R. Levin (Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 93–131). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Weinberger, A., Stegmann, K., Fischer, F., & Mandl, H. (2007). Scripting argumentative knowledge construction in computer-supported learning environments. In F. Fischer, I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborative learning (pp. 191–212). New York, NY: Springer.

    Chapter  Google Scholar 

  • Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry and Allied Disciplines, 17, 89–100.

    Article  Google Scholar 

Download references

Acknowledgements

The research reported in this paper was funded in part by the Spencer Research Training Grant and the Education Policy Research Fellowship from Teachers College, Columbia University to the first author. The authors would like to thank the students, teachers, and principals of the participating schools for their support for this project. We are also grateful to David Hung, Donald J. Cunningham, Katerine Bielaczyc, Katherine Anderson, Liam Rourke, Michael Jacobson, Rebecca Mancy, Rogers Hall, Sarah Davis, Steven Zuiker, and John Voiklis for their insightful comments and suggestions.

Author information

Authors and Affiliations

  1. National Institute of Education, Nanyang Technological University, 1 Nanyang Walk, Singapore, 637616, Singapore

    Manu Kapur

  2. Teachers College, Columbia University, 525 W 120th Street, New York, NY, 10027, USA

    Charles K. Kinzer

Authors
  1. Manu Kapur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Charles K. Kinzer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manu Kapur.

Appendices

Appendix A–Sample Items from the 25-item MCQ Pretest

Two cars having different weights are traveling on a level surface with different but constant velocities. Within the same distance, greater force will always be required to stop the car with the greater

(A) weight (B) velocity

(C) kinetic energy (D) momentum

A 5 kg block is resting on a rough horizontal plane. The coefficient of friction between the block and the plane is 0.8. A 50 N force parallel to the plane is applied on the block for 10 s and then removed. The block eventually comes to a stop. Assuming g = 10 ms−2 and that the coefficient of friction does not change, the total distance traveled by the block equals

(A) 20 m (B) 25 m

(C) 100 m (D) 125 m

A car starts moving from rest in a straight line with a constant acceleration of 5 ms−2, then at constant velocity, and finally decelerating at the rate of 5 ms−2 before coming to a stop. If the total time of motion equals 5 s and the average speed for the entire motion equals 4 ms−1, how long does the car move at constant velocity?

(A) 1 s (B) 2 s

(C) 3 s (D) 4 s

Appendix B–Collaborative Phase Problem Scenarios

Ill-structured Problem 1

You have recently been hired as a lawyer for a prestigious law firm. On your first day, you are sent to meet with an important client who has been fined for speeding. Opening your work file, you find your assignment:

Dear new lawyer,

This morning, I received a call from Mr. Gupta asking me for help. According to him, he almost ran over a small boy this morning in downtown Ghaziabad and was fined for speeding. He insists that he was not. He says that the boy suddenly ran on to the road and he braked very hard and managed to avoid an accident. However, this was enough for a policeman who happened to be there to fine him Rs. 20,000 for speeding. Mr. Gupta is a very important client of our firm and we must do our best to help him. I trust you will give this case your best effort. I am attaching his file for your reference.

I am meeting with Mr. Gupta later this evening. So, I need you to investigate this case and submit your report to me with your analyses and recommendation by today.

Sincerely,Nitin Sharma

Senior Partner

PS–Please note that the word of law is very clear on this. A person is speeding if and only if he is driving above the legal speed limit of the road. No exceptions.

CLIENT FILE

Name: Mr. Amit Gupta

Age: 52 years

Driving Experience: 34 years

Prior Traffic Violations: 1981 (Fined for speeding, Rs. 500),

1993 (Fined for drunk driving, Rs. 10,000)

To carry out your investigation, you go through a number of steps such as a) interviewing an eyewitness, b) analyzing the incident report filed by traffic police, c) accessing the medical examination reports, and d) interviewing the mechanic who inspected the car after the incident.

EYEWITNESS’ ACCOUNT

“I was walking on the roadside pavement. I don’t recall the traffic on the road to be particularly heavy. Suddenly, I noticed a small boy run out on to the road chasing a cricket ball. The next thing I heard was a loud screeching sound. I realized that it came from an Ambassador car skidding to a stop in order to avoid running the boy over. The boy was very lucky to have escaped any injury. I think the boy took about 3 s to cross the road, but I don’t think he looked at the traffic before crossing the road. He was just chasing the ball!”

TRAFFIC POLICE INCIDENT REPORT

  • Traffic conditions: Normal

  • Weather conditions: Bright and sunny; dry road

  • No evidence of a collision between the car and the boy.

  • Number of passengers in the car besides the driver: None

  • Evidence of skid marks: about 15 m

  • Speed limit on the road: 55kmph

  • Width of the road: about 4.5 m

MEDICAL EXAMINATION REPORT

General Comments:

Neither the driver nor the boy sustained any physical injury whatsoever.

Results of the car driver’s medical tests

  • BP (Blood Pressure) = 110/80

  • HR (Heart Rate) = 80

  • Weight = 75 kg

  • Reaction Time = 0.8 s on an average

  • Drug/Alcohol Screen = Negative

INTERVIEW WITH THE MECHANIC

You: What can you say about the condition of the car from your inspection?

Mechanic: Well, this is a heavy car weighing about 1,570 kg and I can clearly see some wear and tear of the tires and the braking system. The braking fluid is also running out. As a result, the traction between the tires and the road does not seem to be as good as it can be.

You: Oh! Does this mean the car was not maintained properly?

Mechanic: Not really. You see, the traction also depends on the condition of the road. The coefficient of friction between the car’s tires and the road is usually between 0.6 and 0.7. So, given the city’s roads, the level of traction not being as good is quite understandable.

You: So, what are you saying?

Mechanic: What I’m saying is that although the traction is not as good as it could have been, this is quite normal in Ghaziabad. Also, it is hard to tell how much of the wear and tear happened during the skidding itself.

You: OK. Thank you for your time.

Well-structured version of ill-structured problem 1

You are a lawyer in a prestigious law firm. You’ve been assigned the following case:

A man was driving his car when, suddenly, a small boy ran out on to the road chasing a ball. He slammed on the brakes and skidded to a stop, leaving a 15 m long skid mark on the road. Luckily the boy was not hurt, but a policeman watching from the sidewalk walked over and fined the man for speeding. An investigation found out that the speed limit on the road is 55kmph. It also determined that the coefficient of friction between the tires and the road was 0.6. The man’s mass was 75 kg and his reaction time, on average, was found to be about 0.8 s. The car’s information manual indicated the mass of the car to be 1,570 kg. Witnesses say that the boy took about 3 s to cross the 4.5 m wide road.

As the man’s lawyer, will you fight the fine in court? Present your case as best you can.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kapur, M., Kinzer, C.K. Productive failure in CSCL groups. Computer Supported Learning 4, 21–46 (2009). https://doi.org/10.1007/s11412-008-9059-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11412-008-9059-z

Keywords

Search

Navigation