Skip to main content

Advertisement

Springer Nature Link
Log in

To Clarity and Beyond: Situating Higher-Order, Critical, and Critical-Analytic Thinking in the Literature on Learning from Multiple Texts

  • REVIEW ARTICLE
  • Published:
Educational Psychology Review Aims and scope Submit manuscript

Abstract

For this systematic review, learning from multiple texts served as the specific context for investigating the constructs of higher-order (HOT), critical (CT), and critical-analytic (CAT) thinking. Examining the manifestations of HOT, CT, and CAT within the specific context of learning from multiple texts allowed us to clarify and disentangle these valued modes of thought. We begin by identifying the mental activities underlying the processes and outcomes of learning from multiple texts. We then juxtapose these mental activities with definitions of HOT, CT, and CAT drawn from the literature. Through this juxtaposition, we define HOT as multi-componential, including evaluation; CT as requiring both evaluation and its justification or substantiation; and CAT as considering the extent to which evaluation and justification may be consistently and systematically applied. We further generate a number of insights, described in the final section of this article. These include the frequent manifestations of HOT, CT, and CAT within the context of students learning from multiple texts and the co-occurring demand for these valued modes of thinking. We propose an additional mode of valued thought, that we refer to as devising, when learners synthetically and systematically use knowledge and strategies gained within one multiple text learning situation to produce an original product or solution in another novel learning situation. We consider such devising to demand HOT, CT, and CAT.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Finland)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data Availability

Data available upon request.

Notes

  1. Although comprehension has been referred to as the result of both bottom-up (or passive) and top-down (i.e., purposeful and active) knowledge activation processes (Kurby et al., 2005; Wolfe & Goldman, 2005), we refer here to top-down processes or students’ deliberate engagement of prior knowledge, as reflective of HOT (Richter & Maier, 2017).

  2. Bloom et al. (1956), in their original introduction of this taxonomy, did not refer to higher vis-à-vis lower levels.

  3. This study was beyond the scope of our review as it was a dissertation; however, serves as an illustrative example here.

  4. The multiple text literature can also benefit from considering objectives specified within Marzano and Kendall (2008) self-system. These include asking students to consider the importance of tasks to them, their efficacy for task completion, emotional responses to tasks or texts, as well as overall motivation. As suggested by a recent review from Anmarkrud et al. (2021), rarely have these self-system components been examined in the literature on learning from multiple texts, with interest constituting the motivational construct most analyzed. Recent work has started to look at the role of epistemic emotions in learning from multiple texts (Danielson et al., 2022; Muis et al., 2015).

  5. This paper was excluded from our review as it did not have a learning outcome.

  6. Kammerer et al. (2013) were unique in asking students to apprise their certainty in a solution to a medical controversy, described across multiple texts. We considered students’ certainty appraisals to reflect metacognition; however this was a mental activity not included among the multiple text outcomes we coded for (i.e., and was placed into the Other category), as Kammerer et al. (2013) were unique in including this as an assessment.

References

*Indicate references included in the review

  • Adams, N. E. (2015). Bloom’s Taxonomy of cognitive learning objectives. Journal of the Medical Library Association, 103(3), 152–153. https://doi.org/10.3163/1536-5050.103.3.010

    Article  Google Scholar 

  • Afflerbach, P., Cho, B.-Y., & Kim, J.-Y. (2011). The assessment of higher order thinking in reading. In G. Schraw & D. R. Robinson (Eds.), Assessment of higher order thinking skills (pp. 185–217). IAP Information Age Publishing.

    Google Scholar 

  • Afflerbach, P., Cho, B.-Y., & Kim, J.-Y. (2015). Conceptualizing and assessing higher-order thinking in reading. Theory into Practice, 54(3), 203–212. https://doi.org/10.1080/00405841.2015.1044367

    Article  Google Scholar 

  • Alexander, P. A. (2014). Thinking critically and analytically about critical-analytic thinking: An introduction. Educational Psychology Review, 26, 469–476. https://doi.org/10.1007/s10648-014-9283-1

    Article  Google Scholar 

  • Alexander, P. A., Dinsmore, D. L., Fox, E., Grossnickle, E. M., Loughlin, S. M., Maggioni, L., Parkinson, M. M., & Winters, F. I. (2011). Higher order thinking and knowledge: Domain-general and domain-specific trends and future directions. In G. Schraw & D. R. Robinson (Eds.), Assessment of higher order thinking skills (pp. 47–88). IAP Information Age Publishing.

    Google Scholar 

  • Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of Educational Objectives. Longman.

    Google Scholar 

  • Anmarkrud, Ø., Bråten, I., Florit, E., & Mason, L. (2021). The role of individual differences in sourcing: A systematic review. Educational Psychology Review, 1–44. https://doi.org/10.1007/s10648-021-09640-7

  • *Anmarkrud, Ø., Bråten, I., & Strømsø, H. I. (2014). Multiple-documents literacy: Strategic processing, source awareness, and argumentation when reading multiple conflicting documents. Learning and Individual Differences, 30, 64–76. https://doi.org/10.1016/j.lindif.2013.01.007

  • *Anmarkrud, Ø., McCrudden, M. T., Bråten, I., & Strømsø, H. I. (2013). Task-oriented reading of multiple documents: Online comprehension processes and offline products. Instructional Science, 41(5), 873–894. https://doi.org/10.1007/s11251-013-9263-8

  • *Barzilai, S., Tal-Savir, D., Abed, F., Mor-Hagani, S., & Zohar, A. R. (2021). Mapping multiple documents: From constructing multiple document models to argumentative writing. Reading and Writing: An Interdisciplinary Journal. https://doi.org/10.1007/s11145-021-10208-8

  • *Barzilai, S., Tzadok, E., & Eshet-Alkalai, Y. (2015). Sourcing while reading divergent expert accounts: Pathways from views of knowing to written argumentation. Instructional Science, 43(6), 737–766. https://doi.org/10.1007/s11251-015-9359-4

  • Barzilai, S., & Zohar, A. (2012). Epistemic thinking in action: Evaluating and integrating online sources. Cognition and Instruction, 30(1), 39–85. https://doi.org/10.1080/07370008.2011.636495

    Article  Google Scholar 

  • Barzilai, S., Zohar, A. R., & Mor-Hagani, S. (2018). Promoting integration of multiple texts: A review of instructional approaches and practices. Educational Psychology Review, 30(3), 973–999. https://doi.org/10.1007/s10648-018-9436-8

    Article  Google Scholar 

  • Bloom, B. S. (Ed.), Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. David McKay.

  • *Brand‐Gruwel, S., Kammerer, Y., van Meeuwen, L., & van Gog, T. (2017). Source evaluation of domain experts and novices during Web search. Journal of Computer Assisted Learning, 33(3), 234–251. https://doi.org/10.1111/jcal.12162

  • *Brand-Gruwel, S., Wopereis, I., & Vermetten, Y. (2005). Information problem solving by experts and novices: Analysis of a complex cognitive skill. Computers in Human Behavior, 21(3), 487–508. https://doi.org/10.1016/j.chb.2004.10.005

  • Brand-Gruwel, S., Wopereis, I., & Walraven, A. (2009). A descriptive model of information problem solving while using internet. Computers & Education, 53(4), 1207–1217. https://doi.org/10.1016/j.compedu.2009.06.004

    Article  Google Scholar 

  • Brante, E. W., & Strømsø, H. I. (2018). Sourcing in text comprehension: A review of interventions targeting sourcing skills. Educational Psychology Review, 30(3), 773–799. https://doi.org/10.1007/s10648-017-9421-7

    Article  Google Scholar 

  • Bråten, I., Britt, M. A., Strømsø, H. I., & Rouet, J.-F. (2011). The role of epistemic beliefs in the comprehension of multiple expository texts: Toward an integrated model. Educational Psychologist, 46(1), 48–70. https://doi.org/10.1080/00461520.2011.538647

    Article  Google Scholar 

  • *Bråten, I., Ferguson, L. E., Anmarkrud, Ø., & Strømsø, H. I. (2013). Prediction of learning and comprehension when adolescents read multiple texts: The roles of word-level processing, strategic approach, and reading motivation. Reading and Writing: An Interdisciplinary Journal, 26(3), 321–348. https://doi.org/10.1007/s11145-012-9371-x

  • *Bråten, I., Ferguson, L. E., Strømsø, H. I., & Anmarkrud, Ø. (2014). Students working with multiple conflicting documents on a scientific issue: Relations between epistemic cognition while reading and sourcing and argumentation in essays. British Journal of Educational Psychology, 84(1), 58–85. https://doi.org/10.1111/bjep.12005

  • *Bråten, I., & Strømsø, H. I. (2003). A longitudinal think-aloud study of spontaneous strategic processing during the reading of multiple expository texts. Reading and Writing, 16:195–218. https://doi.org/10.1023/A:1022895207490

  • Bråten, I., & Strømsø, H. I. (2011). Measuring strategic processing when students read multiple texts. Metacognition and Learning, 6(2), 111–130. https://doi.org/10.1007/s11409-011-9075-7

    Article  Google Scholar 

  • *Britt, M. A., & Aglinskas, C. (2002). Improving students’ ability to identify and use source information. Cognition and Instruction, 20(4), 485–522. https://doi.org/10.1207/S1532690XCI2004_2

  • Britt, M. A., Perfetti, C. A., Sandak, R., & Rouet, J. F. (1999). Content integration and source separation in learning from multiple texts. In S. R. Goldman, A. C. Graesser, & P. van den Broek (Eds.), Narrative comprehension, causality, and coherence: Essays in honor of Tom Trabasso (pp. 209–233). Lawrence Erlbaum Associates.

    Google Scholar 

  • Brown, N. J., Afflerbach, P. P., & Croninger, R. G. (2014). Assessment of critical-analytic thinking. Educational Psychology Review, 26(4), 543–560. https://doi.org/10.1007/s10648-014-9280-4

    Article  Google Scholar 

  • Buehl, M. M., & Alexander, P. A. (2005). Motivation and performance differences in students’ domain-specific epistemological belief profiles. American Educational Research Journal, 42(4), 697–726. https://doi.org/10.3102/00028312042004697

    Article  Google Scholar 

  • Butterfuss, R., & Kendeou, P. (2021). KReC-MD: Knowledge revision with multiple documents. Educational Psychology Review, 33(4), 1475–1497. https://doi.org/10.1007/s10648-021-09603-y

    Article  Google Scholar 

  • Byrnes, J. P., & Dunbar, K. N. (2014). The nature and development of critical-analytic thinking. Educational Psychology Review, 26(4), 477–493. https://doi.org/10.1007/s10648-014-9284-0

    Article  Google Scholar 

  • *Cerdán, R., & Vidal-Abarca, E. (2008). The effects of tasks on integrating information from multiple documents. Journal of Educational Psychology, 100(1), 209–222. https://doi.org/10.1037/0022-0663.100.1.209

  • *Cho, B.-Y., Han, H., & Kucan, L. L. (2018). An exploratory study of middle-school learners’ historical reading in an Internet environment. Reading and Writing: An Interdisciplinary Journal, 31(7), 1525–1549. https://doi.org/10.1007/s11145-018-9847-4

  • *Cho, B.-Y., Woodward, L., Li, D., & Barlow, W. (2017). Examining adolescents’ strategic processing during online reading with a question-generating task. American Educational Research Journal, 54(4), 691–724

  • Cleary, T. J., Callan, G. L., & Zimmerman, B. J. (2012). Assessing self-regulation as a cyclical, context-specific phenomenon: Overview and analysis of SRL microanalytic protocols. Education Research International, 2012, 428639. https://doi.org/10.1155/2012/428639

    Article  Google Scholar 

  • *Daher, T. A., & Kiewra, K. A. (2016). An investigation of SOAR study strategies for learning from multiple online resources. Contemporary Educational Psychology, 46, 10–21. https://doi.org/10.1016/j.cedpsych.2015.12.004

  • Danielson, R. W., Sinatra, G. M., Trevors, G., Muis, K. R., Pekrun, R., & Heddy, B. C. (2022). Can multiple texts prompt causal thinking? The role of epistemic emotions. The Journal of Experimental Education, 1–15. https://doi.org/10.1080/00220973.2022.2107604

  • Danvers, E. C. (2016). Criticality’s affective entanglements: Rethinking emotion and critical thinking in higher education. Gender and Education, 28(2), 282–297. https://doi.org/10.1080/09540253.2015.1115469

    Article  Google Scholar 

  • Dinsmore, D. L., & Alexander, P. A. (2012). A critical discussion of deep and surface processing: What it means, how it is measured, the role of context, and model specification. Educational Psychology Review, 24(4), 499–567. https://doi.org/10.1007/s10648-012-9198-7

    Article  Google Scholar 

  • *Du, H., & List, A. (2020). Researching and writing based on multiple texts. Learning and Instruction, 66, 101297. https://doi.org/10.1016/j.learninstruc.2019.101297

  • *Du, H., & List, A. (2021). Evidence use in argument writing based on multiple texts. Reading Research Quarterly, 56(4), 715–735. https://doi.org/10.1002/rrq.366

  • Dumas, D., & Dong, Y. (2021). Focusing the relational lens on critical thinking: How can relational reasoning support critical and analytic thinking? In D. Fasko & F. Fair (Eds.) Critical thinking and reasoning: Theory development, instruction, and assessment (pp. 47–63). Brill. https://doi.org/10.1163/9789004444591_004

  • Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12, 43–52. https://doi.org/10.1016/j.tsc.2013.12.004

    Article  Google Scholar 

  • Eber, P. A., & Parker, T. S. (2007). Assessing student learning: Applying Bloom's Taxonomy. Human Service Education, 27(1), 45–53. Retrieved February 20, 2023, from https://go.gale.com/ps/i.do?id=GALE%7CA280993786&sid=googleScholar&v=2.1&it=r&linkaccess=abs&issn=08905428&p=AONE&sw=w&userGroupName=anon%7E9c8f4eb

  • Elder, L., & Paul, R. W. (2013). Critical thinking: Intellectual standards essential to reasoning well within every domain of thought. Journal of Developmental Education, 36(3), 34–35. Retrieved February 20, 2023, from https://files.eric.ed.gov/fulltext/EJ1067273.pdf

  • Ennis, R. H. (1962). A concept of critical thinking. Harvard Educational Review, 32(1), 81–111.

    Google Scholar 

  • Ennis, R. H. (1993). Critical thinking assessment. Theory into Practice, 32(3), 179–186.

    Article  Google Scholar 

  • Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Executive Summary: The Delphi Report. The California Academic Press.

  • Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906–911. https://doi.org/10.1037/0003-066X.34.10.906

    Article  Google Scholar 

  • Frerejean, J., Velthorst, G. J., van Strien, J. L. H., Kirschner, P. A., & Brand-Gruwel, S. (2019). Embedded instruction to learn information problem solving: Effects of a whole task approach. Computers in Human Behavior, 90, 117–130. https://doi.org/10.1016/j.chb.2018.08.043

    Article  Google Scholar 

  • *Gerjets, P., Kammerer, Y., & Werner, B. (2011). Measuring spontaneous and instructed evaluation processes during Web search: Integrating concurrent thinking-aloud protocols and eye-tracking data. Learning and Instruction, 21(2), 220–231. https://doi.org/10.1016/j.learninstruc.2010.02.005

  • Gil, L., Bråten, I., Vidal-Abarca, E., & Strømsø, H. I. (2010a). Summary versus argument tasks when working with multiple documents: Which is better for whom? Contemporary Educational Psychology, 35(3), 157–173. https://doi.org/10.1016/j.cedpsych.2009.11.002

    Article  Google Scholar 

  • Gil, L., Bråten, I., Vidal-Abarca, E., & Strømsø, H. I. (2010b). Understanding and integrating multiple science texts: Summary tasks are sometimes better than argument tasks. Reading Psychology, 31(1), 30–68. https://doi.org/10.1080/02702710902733600

    Article  Google Scholar 

  • *Goldman, S. R., Braasch, J. L. G., Wiley, J., Graesser, A. C., & Brodowinska, K. (2012). Comprehending and learning from internet sources: Processing patterns of better and poorer learners. Reading Research Quarterly, 47(4), 356–381. https://doi.org/10.1002/RRQ.027

  • Granello, D. H. (2001). Promoting cognitive complexity in graduate written work: Using Bloom’s taxonomy as a pedagogical tool to improve literature reviews. Counselor Education and Supervision, 40(4), 292–307. https://doi.org/10.1002/j.1556-6978.2001.tb01261.x

    Article  Google Scholar 

  • Greene, J. A., Muis, K. R., & Pieschl, S. (2010). The role of epistemic beliefs in students’ self-regulated learning with computer-based learning environments: Conceptual and methodological issues. Educational Psychologist, 45(4), 245–257. https://doi.org/10.1080/00461520.2010.515932

    Article  Google Scholar 

  • *Grossnickle Peterson, E., & Alexander, P. A. (2020). Navigating print and digital sources: Students’ selection, use, and integration of multiple sources across mediums. Journal of Experimental Education, 88(1), 27–46. https://doi.org/10.1080/00220973.2018.1496058

  • *Hagen, Å. M., Braasch, J. L. G., & Bråten, I. (2014). Relationships between spontaneous note-taking, self-reported strategies and comprehension when reading multiple texts in different task conditions. Journal of Research in Reading, 37(S1), S141–S157. https://doi.org/10.1111/j.1467-9817.2012.01536.x

  • Hofer, B. K., & Bendixen, L. D. (2012). Personal epistemology: Theory, research, and future directions. In K. R. Harris, S. Graham, T. Urdan, C. B. McCormick, G. M. Sinatra, & J. Sweller (Eds.), APA educational psychology handbook, Vol. 1. Theories, constructs, and critical issues (pp. 227–256). American Psychological Association. https://doi.org/10.1037/13273-009

  • Jiménez-Aleixandre, M. P., & Puig, B. (2012). Argumentation, evidence evaluation and critical thinking. In B. J. Fraser, K. Tobin, & C. J. McRobbie (Eds.), Second international handbook of science education (pp. 1001–1015). Springer. https://doi.org/10.1007/978-1-4020-9041-7_66

  • Jones, K. O., Harland, J., Reid, J. M. V.,& Bartlett, R. (2009). Relationship between examination questions and bloom's taxonomy. IEEE Frontiers in Education Conference (pp. 1–6). San Antonio, Texas. https://doi.org/10.1109/FIE.2009.5350598

  • *Kammerer, Y., Bråten, I., Gerjets, P., & Strømsø, H. I. (2013). The role of Internet-specific epistemic beliefs in laypersons’ source evaluations and decisions during Web search on a medical issue. Computers in human behavior, 29(3), 1193–1203. https://doi.org/10.1016/j.chb.2012.10.012

  • *Kammerer, Y., Gottschling, S., & Bråten, I. (2021). The role of internet-specific justification beliefs in source evaluation and corroboration during web search on an unsettled socio-scientific issue. Journal of Educational Computing Research, 59(2), 342–378. https://doi.org/10.1177/0735633120952731

  • *Kammerer, Y., Kalbfell, E., & Gerjets, P. (2016). Is this information source commercially biased? How contradictions between web pages stimulate the consideration of source information. Discourse Processes, 53(5-6), 430–456. https://doi.org/10.1080/0163853X.2016.1169968

  • Kiili, C., & Leu, D. J. (2019). Exploring the collaborative synthesis of information during online reading. Computers in Human Behavior, 95, 146–157. https://doi.org/10.1016/j.chb.2019.01.033

    Article  Google Scholar 

  • Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction-integration model. Psychological Review, 95(2), 163–182. https://doi.org/10.1037/0033-295X.95.2.163

    Article  Google Scholar 

  • Kintsch, W., & van Dijk, T. A. (1978). Toward a model of text comprehension and production. Psychological Review, 85(5), 363–394. https://doi.org/10.1037/0033-295X.85.5.363

    Article  Google Scholar 

  • *Kobayashi, K. (2009a). Comprehension of relations among controversial texts: Effects of external strategy use. Instructional Science, 37(4), 311–324. https://doi.org/10.1007/s11251-007-9041-6

  • *Kobayashi, K. (2009b). The influence of topic knowledge, external strategy use, and college experience on students’ comprehension of controversial texts. Learning and Individual Differences, 19(1), 130–134. https://doi.org/10.1016/j.lindif.2008.06.001

  • *Kobayashi, K. (2014). Students’ consideration of source information during the reading of multiple texts and its effect on intertextual conflict resolution. Instructional Science, 42(2), 183–205. https://doi.org/10.1007/s11251-013-9276-3

  • Krathwohl, D.R., Bloom, B.S., & Masia, B.B. (1964). Taxonomy of educational objectives: The classification of educational goals. Handbook II: The affective domain. David McKay.

  • Kuhn, D. (2019). Critical thinking as discourse. Human Development, 62(3), 146–164. https://doi.org/10.1159/000500171

    Article  Google Scholar 

  • Kurby, C. A., Britt, M. A., & Magliano, J. P. (2005). The role of top-down and bottom-up processes in between-text integration. Reading Psychology, 26(4–5), 335–362. https://doi.org/10.1080/02702710500285870

    Article  Google Scholar 

  • Lai, E. R. (2011). Critical thinking: A literature review. Pearson’s Research Reports, 6(1), 40–41.

    Google Scholar 

  • Lee, Y. (2022). Examining students’ help-seeking when learning from multiple texts. Pennsylvania State University.

    Google Scholar 

  • Lewis, A., & Smith, D. (1993). Defining higher order thinking. Theory into Practice, 32(3), 131–137. https://doi.org/10.1080/00405849309543588

    Article  Google Scholar 

  • *Linderholm, T., Therriault, D. J., & Kwon, H. (2014). Multiple science text processing: Building comprehension skills for college student readers. Reading Psychology, 35(4), 332–356. https://doi.org/10.1080/02702711.2012.726696

  • List, A., & Alexander, P. A. (2015). Examining response confidence in multiple text tasks. Metacognition and Learning, 10, 407–436. https://doi.org/10.1007/s11409-015-9138-2

    Article  Google Scholar 

  • List, A., & Alexander, P. A. (2017). Analyzing and integrating models of multiple text comprehension. Educational Psychologist, 52(3), 143–147. https://doi.org/10.1080/00461520.2017.1328309

    Article  Google Scholar 

  • List, A., & Alexander, P. A. (2018). Corroborating students’ self-reports of source evaluation. Behaviour & Information Technology, 37(3), 198–216. https://doi.org/10.1080/0144929X.2018.1430849

    Article  Google Scholar 

  • List, A., & Alexander, P. A. (2019). Toward an integrated framework of multiple text use. Educational Psychologist, 54(1), 20–39. https://doi.org/10.1080/00461520.2018.1505514

    Article  Google Scholar 

  • *List, A., Alexander, P. A., & Stephens, L. A. (2017). Trust but verify: Examining the association between students’ sourcing behaviors and ratings of text trustworthiness. Discourse Processes, 54(2), 83–104. https://doi.org/10.1080/0163853X.2016.1174654

  • *List, A., Campos Oaxaca, G. S., Lee, E., Du, H., & Lee, H. Y. (2021). Examining perceptions, selections, and products in undergraduates’ learning from multiple resources. British Journal of Educational Psychology, 91(4), 1555–1584. https://doi.org/10.1111/bjep.12435

  • *List, A., & Du, H. (2021). Reasoning beyond history: Examining students’ strategy use when completing a multiple text task addressing a controversial topic in education. Reading and Writing: An Interdisciplinary Journal. https://doi.org/10.1007/s11145-020-10095-5

  • List, A., Du, H., & Wang, Y. (2019a). Understanding students’ conceptions of task assignments. Contemporary Educational Psychology, 59, 101801. https://doi.org/10.1016/j.cedpsych.2019.101801

    Article  Google Scholar 

  • *List, A., Du, H., Wang, Y., & Lee, H. Y. (2019b). Toward a typology of integration: Examining the documents model framework. Contemporary Educational Psychology, 58, 228–242. https://doi.org/10.1016/j.cedpsych.2019.03.003

  • *List, A., Grossnickle, E. M., & Alexander, P. A. (2016a). Profiling students’ multiple source use by question type. Reading Psychology, 37(5), 753–797. https://doi.org/10.1080/02702711.2015.1111962

  • *List, A., Grossnickle, E. M., & Alexander, P. A. (2016b). Undergraduate students’ justifications for source selection in a digital academic context. Journal of Educational Computing Research, 54(1), 22–61. https://doi.org/10.1177/0735633115606659

  • Marzano, R. J., & Kendall, J. S. (2008). Designing and assessing educational objectives: Applying the new taxonomy. Corwin Press.

  • Mason, L., Boldrin, A., & Ariasi, N. (2010). Searching the Web to learn about a controversial topic: Are students epistemically active? Instructional Science, 38, 607–633. https://doi.org/10.1007/s11251-008-9089-y

    Article  Google Scholar 

  • *Mason, L., Junyent, A. A., & Tornatora, M. C. (2014). Epistemic evaluation and comprehension of web-source information on controversial science-related topics: Effects of a short-term instructional intervention. Computers & Education, 76, 143–157. https://doi.org/10.1016/j.compedu.2014.03.016

  • *Mason, L., Zaccoletti, S., Scrimin, S., Tornatora, M. C., Florit, E., & Goetz, T. (2020). Reading with the eyes and under the skin: Comprehending conflicting digital texts. Journal of Computer Assisted Learning, 36(1), 89–101. https://doi.org/10.1111/jcal.12399

  • *Mateos, M., & Solé, I. (2009). Synthesising information from various texts: A study of procedures and products at different educational levels. European Journal of Psychology of Education, 24(4), 435–451. https://doi.org/10.1007/BF03178760

  • Mayer, R. E. (2002). A taxonomy for computer-based assessment of problem solving. Computers in Human Behavior, 18(6), 623–632. https://doi.org/10.1016/S0747-5632(02)00020-1

    Article  Google Scholar 

  • *McCrudden, M. T., Kulikowich, J. M., Lyu, B., & Huynh, L. (2022). Promoting integration and learning from multiple complementary texts. Journal of Educational Psychology. Advance online publication. https://doi.org/10.1037/edu0000746

  • Miri, B., David, B. C., & Uri, Z. (2007). Purposely teaching for the promotion of higher-order thinking skills: A case of critical thinking. Research in Science Education, 37(4), 353–369. https://doi.org/10.1007/s11165-006-9029-2

    Article  Google Scholar 

  • Muis. K. R., Chevrier, M., Denton, C. A., & Losenno, K. M. (2021). Epistemic emotions and epistemic cognition predict critical thinking about socio-scientific issues. Frontiers in Education, 6, Article 669908. https://doi.org/10.3389/feduc.2021.669908

  • *Muis, K. R., Pekrun, R., Sinatra, G. M., Azevedo, R., Trevors, G., Meier, E., & Heddy, B. C. (2015). The curious case of climate change: Testing a theoretical model of epistemic beliefs, epistemic emotions, and complex learning. Learning and Instruction, 39, 168–183. https://doi.org/10.1016/j.learninstruc.2015.06.003

  • Murphy, P. K., Rowe, M. L., Ramani, G., & Silverman, R. (2014). Promoting critical-analytic thinking in children and adolescents at home and in school. Educational Psychology Review, 26(4), 561–578. https://doi.org/10.1007/s10648-014-9281-3

    Article  Google Scholar 

  • Newmann, F. M. (1991). Promoting higher order thinking in social studies: Overview of a study of 16 high school departments. Theory & Research in Social Education, 19(4), 324–340. https://doi.org/10.1080/00933104.1991.10505645

    Article  Google Scholar 

  • Paul, R., & Elder, L. (2006). The miniature guide to critical thinking concepts and tools (4th ed.). The Foundation for Critical Thinking. Retrieved February 20, 2023, from https://www.criticalthinking.org/files/Concepts_Tools.pdf

  • Paul, R. W., & Nosich, G. M. (1991). A proposal for the national assessment of higher-order thinking at the community college, college, and university levels. National Center for Education Statistics, Office of Educational Research and Improvement, the United States Department of Education. Retrieved February 20, 2023, from https://files.eric.ed.gov/fulltext/ED340762.pdf

  • Perfetti, C. A., Rouet, J.-F., & Britt, M. A. (1999). Towards a theory of documents representation. In H. van Oostendorp & S. R. Goldman (Eds.), The Construction of mental representations during reading (pp. 99–122). Lawrence Erlbaum Associates.

    Google Scholar 

  • Petty, R. E., & Briñol, P. (2015). Emotion and persuasion: Cognitive and meta-cognitive processes impact attitudes. Cognition and Emotion, 29(1), 1–26. https://doi.org/10.1080/02699931.2014.967183

    Article  Google Scholar 

  • Rapp, D. N., & Mensink, M. C. (2011). Focusing effects from online and offline reading tasks. In M. T. McCrudden, J. P. Magliano, & G. Schraw (Eds.), Text relevance and learning from text (pp. 141–164). IAP Information Age Publishing.

    Google Scholar 

  • Resnick, L. B. (1987). Education and learning to think. National Academies Press. https://doi.org/10.17226/1032

    Book  Google Scholar 

  • Reznitskaya, A., Anderson, R. C., Dong, T., Li, Y., Kim, I.-H., & Kim, S.-Y. (2008). Learning to think well: Application of argument schema theory to literacy instruction. In C. C. Block & S. R. Parris (Eds.), Comprehension instruction: Research-based best practices (pp. 196–213). The Guilford Press.

    Google Scholar 

  • Richland, L. E., & Simms, N. (2015). Analogy, higher order thinking, and education. Wiley Interdisciplinary Reviews: Cognitive Science, 6(2), 177–192. https://doi.org/10.1002/wcs.1336

    Article  Google Scholar 

  • Richter, T., & Maier, J. (2017). Comprehension of multiple documents with conflicting information: A two-step model of validation. Educational Psychologist, 52(3), 148–166. https://doi.org/10.1080/00461520.2017.1322968

    Article  Google Scholar 

  • *Rodicio, H. G. (2015). Students’ evaluation strategies in a Web research task: Are they sensitive to relevance and reliability? Journal of Computing in Higher Education, 27(2), 134–157. https://doi.org/10.1007/s12528-015-9098-1

  • Roeser, S., & Todd, C. (2015). Emotion and value: Introduction. In S. Roeser & C. Todd (Eds.), Emotion and Value (pp. 1–4). Oxford University Press.

    Google Scholar 

  • Rouet, J.-F. (2006). The skills of document use: From text comprehension to web-based learning. Lawrence Erlbaum Associates.

    Book  Google Scholar 

  • Rouet, J.-F., & Britt, M. A. (2011). Relevance processes in multiple document comprehension. In M. T. McCrudden, J. P. Magliano, & G. Schraw (Eds.), Relevance instructions and goal-focusing in text learning (pp. 19–52). Information Age.

    Google Scholar 

  • *Rouet, J.-F., Britt, M. A., Mason, R. A., & Perfetti, C. A. (1996). Using multiple sources of evidence to reason about history. Journal of Educational Psychology, 88(3), 478–493. https://doi.org/10.1037/0022-0663.88.3.478

  • *Rouet, J.-F., Favart, M., Britt, M. A., & Perfetti, C. A. (1997). Studying and using multiple documents in history: Effects of discipline expertise. Cognition and Instruction, 15(1), 85–106. https://doi.org/10.1207/s1532690xci1501_3

  • Rudd, R., Baker, M., & Hoover, T. (2000). Undergraduate agriculture student learning styles and critical thinking abilities: Is there a relationship? Journal of Agricultural Education, 41(3), 2–12.

    Google Scholar 

  • Sadler, T. D., & Zeidler, D. L. (2004). The morality of socioscientific issues: Construal and resolution of genetic engineering dilemmas. Science Education, 88(1), 4–27. https://doi.org/10.1002/sce.10101

    Article  Google Scholar 

  • *Salmerón, L., Gil, L., Bråten, I., & Strømsø, H. (2010). Comprehension effects of signalling relationships between documents in search engines. Computers in Human Behavior, 26(3), 419–426. https://doi.org/10.1016/j.chb.2009.11.013

  • Samuelstuen, M. S., & Bråten, I. (2007). Examining the validity of self-reports on scales measuring students’ strategic processing. British Journal of Educational Psychology, 77(2), 351–378. https://doi.org/10.1348/000709906X106147

    Article  Google Scholar 

  • Schoor, C., Rouet, J. F., Artelt, C., Mahlow, N., Hahnel, C., Kroehne, U., & Goldhammer, F. (2021). Readers’ perceived task demands and their relation to multiple document comprehension strategies and outcome. Learning and Individual Differences, 88, 102018. https://doi.org/10.1016/j.lindif.2021.102018

    Article  Google Scholar 

  • Schraw, G., & Robinson, D. R. (2011). Conceptualizing and assessing higher order thinking skills. In G. Schraw & D. R. Robinson (Eds.), Assessment of higher order thinking skills (pp. 47–88). IAP Information Age Publishing.

    Google Scholar 

  • Scriven, M., & Paul, R. (1987, August). Critical thinking as defined by the National Council for Excellence in Critical Thinking. In 8th Annual International Conference on Critical Thinking and Education Reform, Rohnert Park, CA (pp. 25–30)

  • Seaman, M. (2011). Bloom’s Taxonomy: Its evolution, revision, and use in the field of education. In D. J. Flinders & P. B. Uhrmacher (Eds.), Curriculum & Teaching Dialogue (pp. 29–45). Information Age Publishing Inc.

    Google Scholar 

  • Sockett, H. (1971). Bloom’s Taxonomy: A philosophical critique (I). Cambridge Journal of Education, 1(1), 16–25. https://doi.org/10.1080/0305764710010103

    Article  Google Scholar 

  • *Solé, I., Miras, M., Castells, N., Espino, S., & Minguela, M. (2013). Integrating information: An analysis of the processes involved and the products generated in a written synthesis task. Written Communication, 30(1), 63–90. https://doi.org/10.1177/0741088312466532

  • Stromer-Galley, J., & Muhlberger, P. (2009). Agreement and disagreement in group deliberation: Effects on deliberation satisfaction, future engagement, and decision legitimacy. Political Communication, 26(2), 173–192. https://doi.org/10.1080/10584600902850775

    Article  Google Scholar 

  • *Strømsø, H. I., Bråten, I., Britt, M. A., & Ferguson, L. E. (2013). Spontaneous sourcing among students reading multiple documents. Cognition and Instruction, 31(2), 176–203. https://doi.org/10.1080/07370008.2013.769994

  • Tarchi, C., & Mason, L. (2020). Effects of critical thinking on multiple-document comprehension. European Journal of Psychology of Education, 35(2), 289–313. https://doi.org/10.1007/s10212-019-00426-8

    Article  Google Scholar 

  • Tsai, C.-C. (2004). Beyond cognitive and metacognitive tools: The use of the Internet as an “epistemological” tool for instruction. British Journal of Educational Technology, 35(5), 525–536. https://doi.org/10.1111/j.0007-1013.2004.00411.x

    Article  Google Scholar 

  • *Tsai, M.-J., & Wu, A.-H. (2021). Visual search patterns, information selection strategies, and information anxiety for online information problem solving. Computers & Education, 172, 104236. https://doi.org/10.1016/j.compedu.2021.104236

  • *van Strien, J. L. H., Kammerer, Y., Brand-Gruwel, S., & Boshuizen, H. P. A. (2016). How attitude strength biases information processing and evaluation on the web. Computers in Human Behavior, 60, 245–252. https://doi.org/10.1016/j.chb.2016.02.057

  • *Vandermeulen, N., van den Broek, B., van Steendam, E., & Rijlaarsdam, G. (2020). In search of an effective source use pattern for writing argumentative and informative synthesis texts. Reading and Writing, 33(2), 239–266. https://doi.org/10.1007/s11145-019-09958-3

  • Vijayaratnam, P. (2012). Developing higher order thinking skills and team commitment via group problem solving: A bridge to the real world. Procedia-Social and Behavioral Sciences, 66, 53–63. https://doi.org/10.1016/j.sbspro.2012.11.247

    Article  Google Scholar 

  • *Walraven, A., Brand-Gruwel, S., & Boshuizen, H. P. A. (2009). How students evaluate information and sources when searching the World Wide Web for information. Computers & Education, 52(1), 234–246. https://doi.org/10.1016/j.compedu.2008.08.003

  • *Walraven, A., Brand-Gruwel, S., & Boshuizen, H. P. A. (2010). Fostering transfer of websearchers’ evaluation skills: A field test of two transfer theories. Computers in Human Behavior, 26(4), 716–728. https://doi.org/10.1016/j.chb.2010.01.008

  • Wang, Y., & List, A. (2019). Calibration in multiple text use. Metacognition and Learning, 14(2), 131–166. https://doi.org/10.1007/s11409-019-09201-y

    Article  Google Scholar 

  • Wentzel, K. R. (2014). Commentary: The role of goals and values in critical-analytic thinking. Educational Psychology Review, 26(4), 579–582. https://doi.org/10.1007/s10648-014-9285-z

    Article  Google Scholar 

  • Weiss, R. E. (2003). Designing problems to promote higher-order thinking. New Directions for Teaching and Learning, 2003(95), 25–31. https://doi.org/10.1002/tl.109

    Article  Google Scholar 

  • *Wiley, J., Goldman, S. R., Graesser, A. C., Sanchez, C. A., Ash, I. K., & Hemmerich, J. A. (2009). Source evaluation, comprehension, and learning in Internet science inquiry tasks. American Educational Research Journal, 46(4), 1060–1106. https://doi.org/10.3102/0002831209333183

  • Wiley, J., Griffin, T. D., Steffens, B., & Britt, M. A. (2020). Epistemic beliefs about the value of integrating information across multiple documents in history. Learning and Instruction, 65, 101266. https://doi.org/10.1016/j.learninstruc.2019.101266

    Article  Google Scholar 

  • *Wiley, J., & Voss, J. F. (1999). Constructing arguments from multiple sources: Tasks that promote understanding and not just memory for text. Journal of Educational Psychology, 91(2), 301–311. https://doi.org/10.1037/0022-0663.91.2.301

  • Willingham, D. T. (2007). Critical thinking: Why it is so hard to teach? American Educator, 31(2), 8–19. https://www.aft.org/sites/default/files/media/2014/Crit_Thinking.pdf

  • Wolf, A. B. (2017). “Tell me how that makes you feel”: Philosophy’s reason/emotion divide and epistemic pushback in philosophy classrooms. Hypatia, 32(4), 893–910. https://doi.org/10.1111/hypa.12378

    Article  Google Scholar 

  • *Wolfe, M. B. W., & Goldman, S. R. (2005). Relations between adolescents’ text processing and reasoning. Cognition and Instruction, 23(4), 467–502. https://doi.org/10.1207/s1532690xci2304_2

  • *Yang, F. (2017). Examining the reasoning of conflicting science information from the information processing perspective—An eye movement analysis. Journal of Research in Science Teaching, 54(10), 1347–1372. https://doi.org/10.1002/tea.21408

  • Zeidler, D.L., & Lewis, J. (2003). Unifying themes in moral reasoning on socioscientific issues and discourse. In D. L. Zeidler (ed.), The Role of Moral Reasoning on Socioscientific Issues and Discourse in Science Education (pp. 289–306). Springer. https://doi.org/10.1007/1-4020-4996-X_15

  • Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329–339. https://doi.org/10.1037/0022-0663.81.3.329

    Article  Google Scholar 

  • Zimmerman, B. J. (2013). From cognitive modeling to self-regulation: A social cognitive career path. Educational Psychologist, 48(3), 135–147. https://doi.org/10.1080/00461520.2013.794676

    Article  Google Scholar 

  • Zohar, A., & Dori, Y. J. (2003). Higher-order thinking and low-achieving students: Are they mutually exclusive? Journal of the Learning Sciences, 12(2), 145–181. https://doi.org/10.1207/S15327809JLS1202_1

    Article  Google Scholar 

  • Braasch, J. L., Rouet, J. F., Vibert, N., & Britt, M. A. (2012). Readers’ use of source information in text comprehension. Memory & Cognition, 40, 450–465. https://doi.org/10.3758/s13421-011-0160-6

    Article  Google Scholar 

  • Muis, K. R. (2007). The role of epistemic beliefs in selfregulated learning. Educational Psychologist, 42(3), 173–190. https://doi.org/10.1080/00461520701416306

    Article  Google Scholar 

  • Facione, P. A. (2000). The disposition toward critical thinking: Its character, measurement, and relation to critical thinking skill. Informal Logic, 20(1), 61–84. https://doi.org/10.22329/il.v20i1.2254

Download references

Author information

Authors and Affiliations

  1. Department of Educational Psychology, Counseling, and Special Education, The Pennsylvania State University, 227 Cedar Building, PA, 16820, University Park, USA

    Alexandra List

  2. Department of Human Development and Quantitative Methodology, University of Maryland, 3242 Benjamin Building, College Park, MD, 20742, College Park, USA

    Yuting Sun

Authors
  1. Alexandra List
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuting Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexandra List.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

Table 5 Descriptive information for studies included in our review
Full size table

5.

Appendix 2. Literature search and screening process

Literature Search

We conducted keyword searches in the PsycINFO and Educational Resources Information Center (ERIC) databases. We restricted the searches to empirical, quantitative studies published in peer-reviewed, English-language journals. To capture examinations of HOT, CT, and CAT within the multiple text literature, we combined two sets of search terms. The first set included keywords or phrases for multiple text research, such as “multiple source*”, “multiple text*”, “multiple document*”, “intertext*”, “conflicting texts”, “complementary texts”, and “information problem solving”. In the second set, we selected a range of keywords and phrases pertaining to HOT, CT, and CAT. These included broader terms such as “higher order thinking”, “critical thinking”, “critical analytic*”, “reasoning”, “metacognit*”, as well as more specific cognitive processes relevant for multiple text learning that potentially manifest HOT, CT, or CAT, such as “integrat*”, “synthes*”, “analys*”, “corroborat*”, “validat*”, “evaluat*”, “justif*”, “sourcing”, “argument*”, and “refutation*”. A full list of search terms is as follows:

(multiple source* OR multiple text* OR multiple document* OR intertext* OR conflicting information OR conflicting views OR conflicting texts OR conflicting sources OR complementary text* OR information problem solving) AND (higher order thinking OR critical thinking OR critical analytic OR metacogni* OR analys* OR analytical OR reasoning OR analog* OR refutation* OR integrat* OR synthes* OR corroborat* OR argument* OR validat* OR justif* OR evaluat* OR sourcing)

These initial searches yielded 1814 non-duplicate records. We narrowed down this initial pool by applying research area classification filters to constrain the literature to areas more relevant for educational and academic contexts (e.g., curriculum & programs & teaching methods, academic learning & achievement, cognitive processes, educational psychology, human experimental psychology, learning and memory). The remaining 482 records after this narrowing were then subject to title and abstract screening.

Title and Abstract Screening

We screened the titles and abstracts based on the following inclusion criteria: The study (a) involved a main task that required reading two or more texts, and (b) was conducted in an educational or academically relevant context. (c) The participants were proficient in the language in which the task was conducted, and (d) had no psychological, physiological, or neurological disorders or disabilities that could affect their text processing. This screening left us with 161 records.

Additional Searches

From these 161 records, we identified authors with five or more included articles on multiple text learning and journals that published five or more included articles. We then did a physical search of additional studies by going through these authors’ Google Scholar pages and the tables of contents of the selected journals in the last five years. We also identified additional relevant studies from the works referenced in these remaining studies following a backward snowballing procedure. These additional searches led us to another 30 non-duplicate records, making for a total of 191 records assessed in full-text form for eligibility.

Full-Text Assessment

As we read the full texts of these remaining 191 articles, we ensured the studies met the following eligibility criteria: (a) The multiple-text task was completed by participants independently rather than as a collaborative group activity; (b) The study included at least one process measure, that was not a self-report strategy question, and (c) one (non-self-report) outcome measure that assessed some form of higher-order cognition potentially reflective of HOT, CT, or CAT. Studies employing self-report questionnaire measures, to capture process data, were only included if these also used other forms of process or outcome measures (e.g., think-alouds, eye-tracking). This left us with a final set of 54 records, with 57 studies eligible for the systematic review.

Appendix 3. Coding for processes and outcomes of multiple text learning

Table 6 Process coding
Full size table

6 

Table 7 Outcome coding
Full size table

7.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

List, A., Sun, Y. To Clarity and Beyond: Situating Higher-Order, Critical, and Critical-Analytic Thinking in the Literature on Learning from Multiple Texts. Educ Psychol Rev 35, 40 (2023). https://doi.org/10.1007/s10648-023-09756-y

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10648-023-09756-y

Keywords

Search

Navigation