Skip to main content

Advertisement

SpringerLink
Log in

Thresholds of knowledge development in complex problem solving: a multiple-case study of advanced learners’ cognitive processes

  • Research Article
  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

This multiple-case study examined how advanced learners solved a complex problem, focusing on how their frequency and application of cognitive processes contributed to differences in performance outcomes, and developing a mental model of a problem. Fifteen graduate students with backgrounds related to the problem context participated in the study. Data sources included direct observation of solution operations, participants’ think aloud and stimulated recalls as they solved the problem, as well as solution scores indicating how well each participant solved the problem. A grounded theory approach was used to analyze stimulated recall and think aloud data. A set of thirteen cognitive processes emerged in the coding and were tallied for each participant. Individual cases were then grouped into clusters that shared similar frequencies of prior knowledge activation, performance outcomes, and tool use behaviors. Each cluster was profiled from least to most successful with descriptive accounts of each cluster’s approach to solving the problem. A cross cluster analysis indicated how learners’ cognitive processes corresponded with problem solving operations that revealed thresholds of knowledge development and formed an integrated mental model of the problem. The findings suggested that mastering problem solving operations within each threshold enhanced the learners’ conceptual awareness of where to apply cognitive processes and increased the combinations of cognitive processes they activated at higher thresholds of knowledge development. The findings have implications for anticipating where novices need support within each threshold of knowledge development during complex problem solving.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Alibali, M. W., Phillips, K. M. O., & Fischer, A. D. (2009). Learning new problem-solving strategies leads to changes in problem representation. Cognitive Development, 24(2), 89–101. doi:10.1016/j.cogdev.2008.12.005.

    Article  Google Scholar 

  • Anderson, J. R. (2009). Expertise. In Cognitive psychology and its implications (7th ed., pp. 242–271). New York: W. H. Freeman.

  • Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.

    Google Scholar 

  • Azevedo, R., & Hadwin, A. (2005, November). Scaffolding self-regulated learning and metacognition: Implications for the design of computer-based scaffolds. Instructional Science, 33(5), 367–379. doi:10.1007/s11251-005-1272-9.

  • Bannert, M., & Reimann, P. (2011). Supporting self-regulated hypermedia learning through prompts. Instructional Science, 40(1), 193–211.

    Article  Google Scholar 

  • Belland, B. (2010). Portraits of middle school students constructing evidence-based arguments during problem-based learning: The impact of computer-based scaffolds. Educational Technology Research and Development, 58(3), 285–309. doi:10.1007/s11423-009-9139-4.

    Article  Google Scholar 

  • Belland, B., Glazewski, K., & Richardson, J. (2011). Problem-based learning and argumentation: Testing a scaffolding framework to support middle school students’ creation of evidence-based arguments. Instructional Science, 39(5), 667–694. doi:10.1007/s11251-010-9148-z.

    Article  Google Scholar 

  • Bixler, B. A., & Land, S. M. (2010). Supporting college students’ ill-structured problem solving in a web-based learning environment. Journal of Educational Technology Systems, 39(1), 3–15. doi:10.2190/ET.39.1.b.

    Article  Google Scholar 

  • Bloom, B. S., Englehart, M. B., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives, the classification of educational goals: Handbook I—Cognitive Domain. New York: McKay.

    Google Scholar 

  • Borkowski, J. G., Johnston, M. B., & Reid, M. K. (1987). Metacognition, motivation, and controlled performance. In S. J. Ceci (Ed.), Handbook of cognitive, social, and neuropsychological aspects of learning disabilities (Vol. 2, pp. 147–174). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How experts differ from novices. In J. D. Bransford, A. L. Brown, & R. R. Cocking (Eds.), How people learn: Brain, mind, experience, and school (pp. 31–50). Washington, D.C.: National Academy Press.

    Google Scholar 

  • Bransford, J. D., & Stein, B. S. (1984). The IDEAL problem solver: A guide for improving thinking, learning, and creativity. New York: W. H. Freeman.

    Google Scholar 

  • Calderhead, J. (1981). Stimulated recall: A method for research on reaching. British Journal of Educational Psychology, 51(2), 211–217. doi:10.1111/j.2044-8279.1981.tb02474.x.

    Article  Google Scholar 

  • Charness, N. (1976). Memory of chess positions: Resistance to interference. Journal of Experimental Psychology, 2(6), 641–653. doi:10.1037/0278-7393.2.6.641.

    Google Scholar 

  • Chase, W. G., & Ericsson, K. A. (1982). Skill and working memory. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 16, pp. 1–58). New York: Academic Press.

    Google Scholar 

  • Chase, W. G., & Simon, H. A. (1973). The mind’s eye in chess. In W. G. Chase (Ed.), Visual information processing (pp. 215–281). New York: Academic Press.

    Google Scholar 

  • Chi, M. T. H., & Bassock, M. (1991). Learning from examples vs. self-explanations. In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 251–282). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Chi, M. T. H., Feltovitch, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121–152. doi:10.1207/s15516709cog0502_2.

    Article  Google Scholar 

  • Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. J. Sternberg (Ed.), Advances in the psychology of human intelligence (pp. 7–75). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Cho, K., & Jonassen, D. H. (2002, September). The effects of argumentation scaffolds on argumentation and problem solving. Educational Technology Research and Development, 50(3), 5–22. doi:10.1007/BF02505022.

  • Corbin, J. M., & Strauss, A. M. (2008). Basics of qualitative research: Techniques and procedures for developing grounded theory. Los Angeles: Sage.

    Google Scholar 

  • Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. Los Angeles: Sage.

    Google Scholar 

  • de Jong, T., & Ferguson-Hessler, M. G. M. (1991). Knowledge of problem situations in physics: A comparison of good and poor novice problem solvers. Learning and Instruction, 1(4), 289–302. doi:10.1016/0959-4752(91)90010-6.

    Article  Google Scholar 

  • Derry, S. J. (1996). Cognitive schema theory in the constructivist debate. Educational Psychologist, 31(3), 163–174.

    Article  Google Scholar 

  • Dreyfus, H. L., & Dreyfus, S. E. (1986). Mind over machine: The power of human intuition and expertise in the era of the computer. Oxford: B. Blackwell.

    Google Scholar 

  • Dunkle, M. E., Schraw, G., & Bendixen, L. D. (1995). Cognitive processes in well-defined and ill-defined problem solving. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.

  • Efklides, A. (2008). Metacognition: Defining its facets and levels of functioning in relation to self-regulation and co-regulation. European Psychologist, 13(4), 277–287.

    Article  Google Scholar 

  • Ericsson, K. A., Prietula, M. J., & Cokely, E. T. (2007). The making of an expert. Harvard Business Review, 85(7/8), 114.

    Google Scholar 

  • Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.

    Google Scholar 

  • Ericsson, K. A., & Staszewsli, J. J. (1989). Skilled memory and expertise: Mechanisms of exceptional performance. In D. Klahr & K. Kotovsky (Eds.), Complex information processing: The impact of Herbert A. Simon (pp. 235–267). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Flavell, J. H. (1971). Stage-related properties of cognitive development. Cognitive Psychology, 2, 421–453. doi:10.1016/0010-0285(71)90025-9.

    Article  Google Scholar 

  • Foshay, R., & Kirkley, J. (1998). Principles for teaching problem solving. Edina, MN: TRO Learning.

    Google Scholar 

  • Funke, J., & Frensch, P. A. (1995). Complex problem solving research in North America and Europe: An integrative review. Foreign Psychology, 5, 42–47.

    Google Scholar 

  • Gillespie, M. (2002). EFF research principle: An approach to teaching and learning that builds expertise. In Equipped for the future (Vol. 2, pp. 1–8). Washington, D.C.: National Institute for Literacy.

  • Glaser, R. (1989). Expertise and learning: How do we think about instructional processes now that we have discovered knowledge structures? In D. Klahr & K. Kotovsky (Eds.), Complex information processing: The impact of Herbert A. Simon (pp. 269–282). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Henning, P. H. (2004). Everyday cognition and situated learning. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (2nd ed., pp. 143–168). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Iiyoshi, T., Hannifin, M. J., & Wang, F. (2005). Cognitive tools and student-centered learning: Rethinking tools, functions, and applications. Educational Media International, 42, 281–296. doi:10.1080/09523980500161346.

    Article  Google Scholar 

  • Ionescu, T. (2012). Exploring the nature of cognitive flexibility. New Ideas in Psychology, 30(2), 190–200. doi:10.1016/j.newideapsych.2011.11.001.

    Article  Google Scholar 

  • Jonassen, D. H. (1996). Computers in the classroom: Mindtools for critical thinking. Englewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  • Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 48(4), 63–85. doi:10.1007/BF02300500.

    Article  Google Scholar 

  • Jonassen, D. H. (2003). Using cognitive tools to represent problems. Journal of Research on Technology in Education, 35(3), 362–381.

    Google Scholar 

  • Jonassen, D. H. (2005). Tools for representing problems and the knowledge required to solve them. In S. Tergan & T. Keller (Eds.), Lecture notes in computer science (Vol. 3426, pp. 93–95). Berlin: Springer.

    Google Scholar 

  • Jonassen, D. H. (2006). Modeling with technology: Mindtools for conceptual change. Upper Saddle River, NJ: Merrill Prentice Hall.

    Google Scholar 

  • Jonassen, D. H., & Reeves, T. C. (1996). Learning with technology: Using computers as cognitive tools. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 693–719). New York: Macmillan.

    Google Scholar 

  • Jonassen, D. H., & Strobel, J. (2006). Modeling for meaningful learning. In D. Hung & M. S. Khine (Eds.), Engaged learning with emerging technologies (pp. 1–27). Dordrecht, South Holland: Springer.

    Chapter  Google Scholar 

  • Kapur, M. (2008). Productive failure. Cognition and Instruction, 26(3), 379–424. doi:10.1080/07370000802212669.

    Article  Google Scholar 

  • Kapur, M., & Rummel, N. (2012). Productive failure in learning from generation and invention activities. Instructional Science, 40(4), 645–650. doi:10.1007/s11251-012-9235-4.

    Article  Google Scholar 

  • Kim, M. K. (2012). Theoretically grounded guidelines for assessing learning progress: Cognitive changes in ill-structured complex problem-solving contexts. Educational Technology Research and Development, 60(4), 601–622. doi:10.1007/s11423-012-9247-4.

    Article  Google Scholar 

  • Kim, B., & Reeves, T. (2007). Reframing research on learning with technology: In search of the meaning of cognitive tools. Instructional Science, 35(3), 207–256. doi:10.1007/s11251-006-9005-2.

    Article  Google Scholar 

  • Kleinmuntz, D. N., & Schkade, D. A. (1993). Information displays and decision processes. Psychological Science, 4(4), 221–227. doi:10.1111/j.1467-9280.1993.tb00265.x.

    Article  Google Scholar 

  • Kluwe, R. H. (1995). Single case studies and models of complex problem solving. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European perspective (pp. 269–291). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Kozma, R. (2003). The material features of multiple representations and their cognitive and social affordances for science understanding. Learning and Instruction, 13(2), 205–226. doi:10.1016/S0959-4752(02)00021-X.

    Article  Google Scholar 

  • Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212–218. doi:10.1207/s15430421tip4104_2.

    Article  Google Scholar 

  • Lajoie, S. P. (1993). Computer environments as cognitive tools for enhancing learning. In S. P. Lajoie & S. J. Derry (Eds.), Computers as cognitive tools (pp. 261–288). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Lajoie, S. P. (2000). Computers as cognitive tools: No more walls (Vol. 2). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Lajoie, S. P. (2008). Metacognition, self-regulation, and self-regulated learning: A rose by any other name? Educational Psychology Review, 20(4), 469–475. doi:10.1007/s10648-0089088-1.

    Article  Google Scholar 

  • Larkin, J. H. (1983). The role of problem representation in physics. In D. Gentner & A. L. Stevens (Eds.), Mental models (pp. 75–98). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Lee, H., Lim, K., & Grabowski, B. (2010). Improving self-regulation, learning strategy use, and achievement with metacognitive feedback. Educational Technology Research and Development, 58(6), 629–648.

    Article  Google Scholar 

  • Liu, M., & Bera, S. (2005). An analysis of cognitive tool use patterns in a hypermedia learning environment. Educational Technology Research and Development, 53(1), 5–21.

    Article  Google Scholar 

  • Liu, M., Bera, S., Corliss, S., Svinicki, M., & Beth, A. (2004). Understanding the connection between cognitive tool use and cognitive processes as used by sixth graders in a problem-based hypermedia learning environment. Journal of Educational Computing Research, 31(3), 309–334.

    Article  Google Scholar 

  • Liu, M., Horton, L., Corliss, S. B., Svinicki, M. D., Bogard, T., Kim, J., et al. (2009). Students’ problem-solving as mediated by their cognitive tool use: A study of tool use patterns. Journal of Educational Computing Research, 40(1), 111–139.

    Article  Google Scholar 

  • Liu, M., Horton, L., Kang, J., Kimmons, R. & Lee, J. (2013a). Using a Ludic simulation to make learning of middle school space science fun. The International Journal of Gaming and Computer-Mediated Simulations, 5(1).

  • Liu, M., Yuen, T. T. Horton, L., Lee, J., Toprac, P. and Bogard, T. (2013b). Designing technology-enriched cognitive tools to support young learners’ problem solving. The International Journal of Cognitive Technology.

  • Livingston, E., & Borko, H. (1989). Expert-novice differences in teaching: A cognitive analysis and implications for teacher education. Journal of Teacher Education, 40(4), 36–42. doi:10.1177/002248718904000407.

    Article  Google Scholar 

  • Lyle, J. (2003). Stimulated recall: A report on its use in naturalistic research. British Educational Research Journal, 29(6), 861–878. doi:10.1080/0141192032000137349.

    Article  Google Scholar 

  • Mayer, R. E. (1989). Models for understanding. Review of Educational Research, 59, 43–64. doi:10.3102/00346543059001043.

    Article  Google Scholar 

  • Mayer, R. E. (1998). Cognitive, metacognitive, and motivational aspects of problem solving. Instructional Science, 26(1–2), 49–63.

    Article  Google Scholar 

  • Mayer, R. E., & Wittrock, M. C. (1996). Problem-solving transfer. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 47–62). New York: Macmillan.

    Google Scholar 

  • Merriënboer, J., & Stoyanov, S. (2008). Learners in a changing learning landscape: Reflection from an instructional design perspective. In J. Visser, M. Visser-Valfrey, D. N. Aspin, & J. D. Chapman (Eds.), Lifelong learning book series (Vol. 12, pp. 69–90). Dordrecht, South Holland: Springer.

    Google Scholar 

  • Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81–97. doi:10.1037/h0043158.

    Article  Google Scholar 

  • Patel, V. L., & Groen, G. J. (1991). The general and specific nature of medical expertise: A critical look. In K. A. Ericsson & J. Smith (Eds.), Toward a general theory of expertise: Prospects and limits (pp. 93–125). New York: Cambridge University Press.

    Google Scholar 

  • Pea, R. D. (1985). Beyond amplification: Using the computer to reorganize mental functioning. Educational Psychologist, 20(4), 167–182. doi:10.1207/s15326985ep2004_2.

    Article  Google Scholar 

  • Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341. doi:10.1007/s10648-006-9029-9.

    Article  Google Scholar 

  • Pintrich, P. R. (2000). An achievement goal theory perspective on issues in motivation terminology, theory, and research. Contemporary Educational Psychology, 25(1), 92–104. doi:10.1006/ceps.1999.1017.

    Article  Google Scholar 

  • Poitras, E., Lajoie, S., & Hong, Y. J. (2011). The design of technology-rich learning environments as metacognitive tools in history education. Instructional Science, 1–29.

  • Prawat, R. S. (1989). Promoting access to knowledge, strategy, and disposition in students: A research synthesis. Review of Educational Research, 59, 1–41. doi:10.2307/1170445.

    Article  Google Scholar 

  • Puntambekar, S., & Hübscher, R. (2005). Tools for scaffolding students in a complex learning environment: What have we gained and what have we missed? Educational Psychologist, 40(1), 1–12. doi:10.1207/s15326985ep4001_1.

    Article  Google Scholar 

  • Ramdass, D., & Zimmerman, B. J. (2011). Developing self-regulation skills: The important role of homework. Journal of Advanced Academics, 22(2), 194–218.

    Article  Google Scholar 

  • Salomon, G., Perkins, D. N., & Globerson, T. (1991). Partners in cognition: Extending human intelligent technologies. Educational Researcher, 20(3), 2–9.

    Article  Google Scholar 

  • Savelsbergh, E. R., de Jong, T., & Ferguson-Hessler, M. G. M. (1998). Competence-related differences in problem representations. In M. van Sommeren, P. Reimann, T. de Jong, & H. Boshuizen (Eds.), The role of multiple representations in learning and problem solving (pp. 262–282). Amsterdam: Elsevier.

    Google Scholar 

  • Schkade, D. A., & Kleinmuntz, D. N. (1994). Information displays and choice processes: Differential effects of organization, form, and sequence. Organizational Behavior and Human Decision Processes, 57(3), 319–337. doi:10.1006/obhd.1994.1018.

    Article  Google Scholar 

  • Simons, K., & Klein, J. (2007, January). The impact of scaffolding and student achievement levels in a problem-based learning environment. Instructional Science, 35(1), 41–72. doi:10.1007/s11251-006-9002-5.

  • Spector, J. M. (2008). Expertise and dynamic tasks. In H. Qudrat-Ullah, J. M. Spector, & P. I. Davidsen (Eds.), Understanding complex systems series: Complex decision making 2008 (pp. 25–37). Berlin: Springer. doi:10.1007/978-3-540-73665-3_2.

  • Spector, J. M. (2010). Mental representations and their analysis: An epistemological perspective. In D. Ifenthaler, P. Pirnay-Dummer, & N. M. Seel (Eds.), Computer-based diagnostics and systematic analysis of knowledge 2010 (Part 1, pp. 27–40). New York: Springer. doi:10.1007/978-1-4419-5662-0_3.

  • Spiro, R. J., Coulson, R. L., Feltovich, P. J., & Anderson, D. K. (1988). Cognitive flexibility theory: Advanced knowledge acquisition in ill-structured domains (Tech. Rep. No. 441). Urbana-Champaign, IL: University of Illinois, Center for the Study of Reading.

  • Stake, R. E. (2006). Multiple case study analysis. New York: Guilford Press.

    Google Scholar 

  • van Gog, T., Ericsson, K., Rikers, R., & Paas, F. (2005, September). Instructional design for advanced learners: Establishing connections between the theoretical frameworks of cognitive load and deliberate practice. Educational Technology Research and Development, 53(3), 73–81. doi:10.1007/BF02504799.

  • Vygotsky, L. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Xun, G., & Land, S. (2004). A conceptual framework for scaffolding ill-structured problem-solving processes using question prompts and peer interaction. Educational Technology Research and Development, 52(2), 5–22. doi:10.1007/BF02504836.

    Article  Google Scholar 

  • Yin, R. K. (2003). Case study research: Design and methods (3rd ed., Vol. 5). Thousand Oaks, CA: Sage.

  • Zhang, J. (1997). The nature of external representations in problem solving. Cognitive Science, 21(2), 179–217.

    Article  Google Scholar 

  • Zhang, J., & Norman, D. A. (1994). Representations in distributed cognitive tasks. Cognitive Science, 18(1), 87–122. doi:10.1016/0364-0213(94)90021-3.

    Article  Google Scholar 

  • Zimmerman, B. J., & Campillo, M. (2003). Motivating self-regulated problem solvers. In J. E. Davidson & R. Sternberg (Eds.), The nature of problem solving (pp. 233–262). New York: Cambridge University Press.

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank Tim Yuen and Paul Toprac for assisting with the coding process for this study.

Author information

Authors and Affiliations

  1. School of Education and Health Sciences, University of Dayton, 300 College Park, Dayton, OH, 45469, USA

    Treavor Bogard

  2. The University of Texas at Austin, Austin, TX, USA

    Min Liu

  3. Dharma Drum Mountain World Center for Buddhist Education, New Taipei City, Taiwan

    Yueh-hui Vanessa Chiang

Authors
  1. Treavor Bogard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yueh-hui Vanessa Chiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Treavor Bogard.

Appendix

Appendix

See Table 5.

Table 5 List of cognitive codes and definitions
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bogard, T., Liu, M. & Chiang, Yh.V. Thresholds of knowledge development in complex problem solving: a multiple-case study of advanced learners’ cognitive processes. Education Tech Research Dev 61, 465–503 (2013). https://doi.org/10.1007/s11423-013-9295-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-013-9295-4

Keywords

Search

Navigation