Skip to main content
SpringerLink
Log in

It is Not Enough to be Smart: On Explaining the Relation Between Intelligence and Complex Problem Solving

  • Original research
  • Published:
Technology, Knowledge and Learning Aims and scope Submit manuscript

Abstract

The main aim of this study was to (a) test the construct validity of complex problem solving (CPS); (b) examine the ability to acquire knowledge as a mediator of the relationship between intelligence and CPS performance; and (c) investigate the personal need for structure as a moderator of the relationship between intelligence and knowledge acquisition. A total of 128 participants completed the self-report Personal Need for Structure scale; the Vienna Matrix Test to assess intelligence; and a new multiple complex systems approach method to assess CPS skills. When analyzing the internal structure of CPS, we found that a two-dimensional model consisting of knowledge acquisition and knowledge application best fitted the data. We also found that the relationship between intelligence and CPS performance was partially mediated by the ability to acquire knowledge. Finally, personal need for structure did not moderate the relationship between intelligence and the ability to acquire knowledge. Our results indicate a need to further investigate other cognitive abilities in interaction with contextual situational factors that could additionally explain variance in CPS performance. Moreover, we also highlight the importance of deeper observation of the knowledge application phase of CPS process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. Since the terminology rule identification, rule knowledge, and rule application is very common in the literature (e.g., Funke and Greiff, 2017; Schweizer, Wüstenberg and Greiff, 2013; Wüstenberg, Greiff, and Funke 2012), we decided to use this terminology when addressing a three-faceted model of CPS.

  2. An eigendynamic is a specific effect that can be programmed into a dependent variable. It refers to a constant increase or decrease of value of this variable itself, independent of other influences or variables. This effect creates an impression that the situation is worsening over time if the proper intervention is not provided.

References

  • ALLEA—All European Academies (2017). The European Code of Conduct for Research Integrity Revised Version. Berlin: ALLEA—All European Academies.

  • Baggen, Y., Mainert, J., Kretzschmar, A., Lans, T., Biemans, H. J. A., & Greiff, S. (2017). Complex problems in entrepreneurship education: Examining complex problem-solving in the application of opportunity identification. Educational Research International, 2017, 1–13.

    Article  Google Scholar 

  • Bar-Tal, Y. (2010). When the need for cognitive structure does not cause heuristic thinking: The moderating effect of the perceived ability to achieve cognitive structure. Psychology, 1, 96–105.

    Article  Google Scholar 

  • Bar-Tal, Y., Raviv, R., & Spitzer, A. (1999). The need and ability to achieve cognitive structuring: Individual differences that moderate the effect of stress on information processing. Journal of Personality and Social Psychology, 77(1), 33–51.

    Article  Google Scholar 

  • Beckmann, J. F., & Guthke, J. (1995). Complex problem solving, intelligence and learning ability. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European perspective (pp. 3–25). Hillsdale: Erlbaum.

    Google Scholar 

  • Blais, A. R., Thompson, M. M., & Baranski, J. V. (2005). Individual differences in decision processing and confidence judgments in comparative judgment tasks: The role of cognitive styles. Personality and Individual Differences, 38, 1701–1713.

    Article  Google Scholar 

  • Bühner, M., Kröner, S., & Ziegler, M. (2008). Working memory, visual–spatial-intelligence and their relationship to problem-solving. Intelligence, 36(6), 672–680.

    Article  Google Scholar 

  • Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86(1), 79–122.

    Article  Google Scholar 

  • Fischer, A., Greiff, S., Wüstenberg, S., Fleischer, J., Buchwald, F., & Funke, J. (2015). Assessing analytic and interactive aspects of problem solving competency. Learning and Individual Differences, 39, 172–179.

    Article  Google Scholar 

  • Frensch, P. A., & Funke, J. (1995). Definitions, traditions, and a general framework for understanding complex problem solving. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European perspective (pp. 3–25). NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Frensch, P. A., & Sternberg, R. J. (1991). Skill-related differences in game playing. In R. J. Sternberg & P. A. Frensch (Eds.), Complex problem solving: Principles and mechanisms (pp. 343–381). Hillsdale: Lawrence Erlbaum Associates.

    Google Scholar 

  • Funke, J. (1983). Einige Bemerkungen zu Problemen der Problemlöseforschung oder: Ist Testintelligenz doch ein Prädiktor? [Issues in problem solving research: Is test intelligence a predictor after all?]. Diagnostica, 29, 283–302.

    Google Scholar 

  • Funke, J. (2001). Dynamic systems as tools for analysing human judgement. Thinking and Reasoning, 7(1), 69–89.

    Article  Google Scholar 

  • Funke, J. (2010). Complex problem solving: A case for complex cognition? Cognitive Processing, 11, 133–142.

    Article  Google Scholar 

  • Funke, J., & Frensch, P. A. (2007). Complex problem solving: The European perspective-10 years after. In D. H. Jonassen (Ed.), Learning to solve complex scientific problems (pp. 25–47). New York: Lawrence Erlbaum.

    Google Scholar 

  • Funke, J., & Greiff, S. (2017). Dynamic problem solving: Multiple-item testing based on minimally complex systems. In D. Leutner, J. Fleischer, J. Grünkorn, & E. Klieme (Eds.), Competence assessment in education: Research, models and instruments (pp. 427–443). Cham: Springer.

    Chapter  Google Scholar 

  • Gocłowska, M. A., Baas, M., Crisp, R. J., & De Dreu, C. K. W. (2014). Whether social schema violations help or hurt creativity depends on need for structure. Personality and Social Psychology Bulletin, 40(8), 959–971.

    Article  Google Scholar 

  • Gonzalez, C., Thomas, R., & Vanyukov, P. (2005a). The relationships between cognitive ability and dynamic decision making. Intelligence, 33(2), 169–186.

    Article  Google Scholar 

  • Gonzalez, C., Vanyukov, P., & Martin, M. K. (2005b). The use of microworlds to study dynamic decision making. Computers in Human Behavior, 21(2), 273–286.

    Article  Google Scholar 

  • Goodwin, L. D., & Leech, N. L. (2006). Understanding correlation: Factors that affect the size of r. The Journal of Experimental Education, 74(3), 251–266.

    Article  Google Scholar 

  • Greiff, S., & Fischer, A. (2013). Measuring complex problem solving: An educational application of psychological theories. Journal for Educational Research Online, 5(1), 38–58.

    Google Scholar 

  • Greiff, S., Fischer, A., Stadler, M., & Wüstenberg, S. (2015). Assessing complex problem-solving skills with multiple complex systems. Thinking and Reasoning, 21(3), 356–382.

    Article  Google Scholar 

  • Greiff, S., Fischer, A., Wüstenberg, S., Sonnleitner, P., Brunner, M., & Martin, R. (2013a). A multitrait–multimethod study of assessment instruments for complex problem solving. Intelligence, 41(5), 579–596.

    Article  Google Scholar 

  • Greiff, S., & Funke, J. (2017). Interactive problem solving: Exploring the potential of minimal complex systems. In B. Csapó & J. Funke (Eds.), The nature of problem solving: Using research to inspire 21st century learning (pp. 93–105). Paris: OECD Publishing.

    Google Scholar 

  • Greiff, S., Krkovic, K., & Hautamäki, J. (2016). The prediction of problem-solving assessed via microworlds. A study on the relative relevance of fluid reasoning and working memory. European Journal of Psychological Assessment, 32(4), 298–306.

    Article  Google Scholar 

  • Greiff, S., & Neubert, J. C. (2014). On the relation of complex problem solving, personality, fluid intelligence, and academic achievement. Learning and Individual Differences, 36, 37–48.

    Article  Google Scholar 

  • Greiff, S., Wüstenberg, S., & Funke, J. (2012). Dynamic problem solving: A new measurement perspective. Applied Psychological Measurement, 36(3), 189–213.

    Article  Google Scholar 

  • Greiff, S., Wüstenberg, S., Molnár, G., Fischer, A., Funke, J., & Csapó, B. (2013b). Complex problem solving in educational contexts—something beyond g: Concept, assessment, measurement invariance, and construct validity. Journal of Educational Psychology, 105(2), 364–379.

    Article  Google Scholar 

  • Grežo, M., & Sarmány-Schuller, I. (2015). The need for structure and the recognition heuristic in financial decision making. Československá psychologie: časopis pro psychologickou teorii a praxi, 59(6), 534–548.

    Google Scholar 

  • Gros, B. (2007). Digital games in education: The design of games-based learning environments. Journal of Research on Technology in Education, 40(1), 23–38.

    Article  Google Scholar 

  • Gros, B. (2015). Integration of digital games in learning and e-learning environments: Connecting experiences and context. In T. Lowrie & R. JorgensenZevenbergen (Eds.), Digital games mathematics learning mathematics education in the digital era (pp. 35–53). Dordrecht: Springer.

    Chapter  Google Scholar 

  • Hainey, T., Connolly, T. M., Stansfield, M., & Boyle, E. A. (2011). Evaluation of a game to teach requirements collection and analysis in software engineering at tertiary education level. Computers and Education, 56(1), 21–35.

    Article  Google Scholar 

  • Halama, P., & Gurňáková, J. (2014). Need for structure and big five personality traits as predictors of decision-making styles in health professionals. Studia Psychologica, 56(3), 171–179.

    Article  Google Scholar 

  • Herborn, K., Mustafić, M., & Greiff, S. (2017). Mapping an experiment-based assessment of collaborative behavior onto collaborative problem solving in PISA 2015: A cluster analysis approach for collaborator profiles. Journal of Educational Measurement, 54(1), 103–122.

    Article  Google Scholar 

  • Herde, C. N., Wüstenberg, S., & Greiff, S. (2016). Assessment of complex problem solving: What we know and what we don’t know. Applied Measurement in Education, 29(4), 265–277.

    Article  Google Scholar 

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.

    Article  Google Scholar 

  • Joslyn, S., & Hunt, E. (1998). Evaluating individual differences in response to time-pressure situations. Journal of Experimental Psychology, 4, 16–43.

    Google Scholar 

  • Kanyongo, G. Y., Brook, G. P., Kyei-Blankson, L., & Gocmen, G. (2007). Reliability and statistical power: How measurement fallibility affects power and required sample sizes for several parametric and nonparametric statistics. Journal of Modern Applied Statistical Methods, 6(1), 81–90.

    Article  Google Scholar 

  • Kline, P. (1999). The handbook of psychological testing (2nd ed.). London: Routledge.

    Google Scholar 

  • Klose, J., Černochová, D., & Král, P. (2002). Vídeňský maticový test. Praha: Testcentrum.

    Google Scholar 

  • Kluwe, R. H., Misiak, C., & Haider, H. (1991). The control of complex systems and performance in intelligence tests. In H. Rowe (Ed.), Intelligence: Reconceptualization and measurement (pp. 227–244). Hillsdale: Lawrence Erlbaum.

    Google Scholar 

  • Kretzschmar, A., Neubert, J. C., Wüstenberg, S., & Greiff, S. (2016). Construct validity of complex problem solving: A comprehensive view on different facets of intelligence and school grades. Intelligence, 54, 55–69.

    Article  Google Scholar 

  • Kröner, S., Plass, J. L., & Leutner, D. (2005). Intelligence assessment with computer simulations. Intelligence, 33(4), 347–368.

    Article  Google Scholar 

  • Kruglanski, A. W. (1989). Lay epistemics and human knowledge: Cognitive and motivational bases. New York: Plenum.

    Book  Google Scholar 

  • Lotz, C., Scherer, R., Greiff, S., & Sparfeldt, J. R. (2017). Intelligence in action—Effective strategic behaviors while solving complex problems. Intelligence, 64, 98–112.

    Article  Google Scholar 

  • Lotz, C., Sparfeldt, J. R., & Greiff, S. (2016). Complex problem solving in educational contexts—Still something beyond a “good g”? Intelligence, 59, 127–138.

    Article  Google Scholar 

  • McDonald, R. P. (1999). Test theory: A unified treatment. New Jersey: Lawrence Erlbaum Associates.

    Google Scholar 

  • Messick, S. (1984). The nature of cognitive styles: Problems and promises in educational research. Educational Psychologist, 19(2), 59–74.

    Article  Google Scholar 

  • Molnár, G., Greiff, S., Wüstenberg, S., & Fischer, A. (2017). Empirical study of computer-based assessment of domain-general complex problem-solving skills. In B. Csapó & J. Funke (Eds.), The nature of problem solving: Using research to inspire 21st century learning (pp. 125–138). Paris: OECD Publishing.

    Google Scholar 

  • Molnár, G., & Csapó, B. (2018). The efficacy and development of students’ problem-solving strategies during compulsory schooling: Logfile analyses. Frontiers in Psychology, 9, 302.

    Article  Google Scholar 

  • Neuberg, S. L., & Newsom, J. T. (1993). Personal need for structure: Individual differences in the desire for simple structure. Journal of Personality and Social Psychology, 65(1), 113–131.

    Article  Google Scholar 

  • OECD. (2005). The definition and selection of key competencies. Executive summary. Retrieved from: https://www.oecd.org/pisa/35070367.pdf

  • OECD. (2013). PISA 2012 Assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. Paris: OECD Publishing.

    Book  Google Scholar 

  • OECD. (2014). PISA 2012 Results: Creative problem solving: Students’ skills in tackling real-life problems (Vol. V). Paris: OECD Publishing.

    Google Scholar 

  • Prensky, M. (2001). Digital game-based learning. New York: McGraw-Hill Education.

    Google Scholar 

  • Prensky, M. (2010). Teaching digital natives: Partnering for real learning. California: Corwin—A SAGE Company.

    Google Scholar 

  • Putz-Osterloh, W. (1981). Über die Beziehung zwischen Testintelligenz und Problemlöseerfolg [On the relation between test intelligence and problem solving success]. Zeitschrift für Psychologie, 189, 79–100.

    Google Scholar 

  • Rigas, G., & Brehmer, B. (1999). Mental processes in intelligence tests and dynamic decision-making tasks. In P. Juslin & H. Montgomery (Eds.), Judgement and decision making: Neo-Brunswikean and process-tracing approaches (pp. 45–65). Hillsdale: Lawrence Erlbaum Associates Inc.

    Google Scholar 

  • Sarmány-Schuller, I. (1999). Procrastination, need for cognition and sensation seeking. Studia Psychologica, 41(1), 73–85.

    Google Scholar 

  • Sarmány-Schuller, I. (2000). Need for structure and coping processes. Ansiedad y Estrés, 6(1), 39–45.

    Google Scholar 

  • Sarnataro-Smart, S. (2013). Personal need for structure: Indiscriminate classification systems as barriers to processing mathematical complexity. Honors Thesis Collection, 110. http://repository.wellesley.edu/thesiscollection/110

  • Schaller, M., Boyd, C., Yohannes, J., & O’Brien, M. (1995). The prejudiced personality revisited: Personal need for structure and formation of erroneous group stereotypes. Journal of Personality and Social Psychology, 68(3), 544–555.

    Article  Google Scholar 

  • Scherer, R., Greiff, S., & Hautamäki, J. (2015). Exploring the relation between time on task and ability in complex problem solving. Intelligence, 48, 37–50.

    Article  Google Scholar 

  • Schultz, P. W., & Searleman, A. (1998). Personal need for structure, the einstellung task, and the effects of stress. Personality and Individual Differences, 24(3), 305–310.

    Article  Google Scholar 

  • Schult, J., Stadler, M., Becker, N., Greiff, S., & Sparfeldt, J. R. (2017). Home alone: Complex problem-solving performance benefits from individual online assessment. Computers in Human Behavior, 68, 513–519.

    Article  Google Scholar 

  • Schweizer, F., Wüstenberg, S., & Greiff, S. (2013). Validity of the MicroDYN approach: Complex problem solving predicts school grades beyond working memory capacity. Learning and Individual Differences, 24, 42–52.

    Article  Google Scholar 

  • Stadler, M., Becker, N., Gödker, M., Leutner, D., & Greiff, S. (2015). Complex problem solving and intelligence: A meta-analysis. Intelligence, 53, 92–101.

    Article  Google Scholar 

  • Stadler, M., Becker, N., Schult, J., Niepel, C., Spinath, F. M., & Greiff, S. (2017). The logic of success: The relation between complex problem-solving skills and university achievement. Higher Education, 55, 1–15.

    Google Scholar 

  • Stranovská, E., Munková, D., Munk, M., & Sarmány-Schuller, I. (2013). Cognitive-individual, linguistic and demographic variables, and syntactic abilities in foreign language. Studia Psychologica, 55(4), 273–287.

    Article  Google Scholar 

  • Svecova, V., & Pavlovicova, G. (2016). Screening the personal need for the structure and solving word problems with fractions. Springerplus, 5, 652.

    Article  Google Scholar 

  • Thompson, M. M., Naccarato, M. E., & Parker, K. E. (1989). Assessing cognitive need: The development of the personal need for structure and personal fear of invalidity scales. Halifax: Annual Meeting of the Canadian Psychological Association.

    Google Scholar 

  • Tschirgi, J. E. (1980). Sensible reasoning: a hypothesis about hypotheses. Child Development, 51(1), 1–10.

    Article  Google Scholar 

  • Van Eck, R. (2006). Digital game-based learning: It’s not just the digital natives who are restless. Educause Review, 41(2), 1–16.

    Google Scholar 

  • Vollmeyer, R., Burns, B. D., & Holyoak, K. J. (1996). The impact of goal specificity on strategy use and the acquisition of problem structure. Cognitive Science, 20(1), 75–100.

    Article  Google Scholar 

  • Wojtowicz, A., & Wojtowicz, B. (2015). The personal need for structure as a factor affecting the understanding and projecting of complex spatial structures. Technical Transactions Architecture, 11, 63–72.

    Google Scholar 

  • Wüstenberg, S., Greiff, S., & Funke, J. (2012). Complex problem solving more than reasoning? Intelligence, 40(1), 1–14.

    Article  Google Scholar 

  • Wüstenberg, S., Stadler, M., Hautamäki, J., & Greiff, S. (2014). The role of strategy knowledge for the application of strategies in complex problem-solving tasks. Technology, Knowledge and Learning, 19(1–2), 127–146.

    Article  Google Scholar 

Download references

Funding

This work was supported by the grant agency VEGA under Grant no. 2/0116/15 and Grant no. 2/0035/20.

Author information

Authors and Affiliations

  1. Institute of Experimental Psychology, Centre of Social and Psychological Sciences, Slovak Academy of Sciences, Dúbravská cesta 9, 841 04, Bratislava, Slovakia

    Matúš Grežo & Ivan Sarmány-Schuller

Authors
  1. Matúš Grežo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ivan Sarmány-Schuller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matúš Grežo.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Grežo, M., Sarmány-Schuller, I. It is Not Enough to be Smart: On Explaining the Relation Between Intelligence and Complex Problem Solving. Tech Know Learn 27, 69–89 (2022). https://doi.org/10.1007/s10758-021-09498-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10758-021-09498-2

Keywords

Search

Navigation