Skip to main content
SpringerLink
Log in

Students’ evaluation of information during online inquiry: Working individually or in pairs

  • Published:
The Australian Journal of Language and Literacy Aims and scope Submit manuscript

Abstract

Varying information quality and an increase of misinformation on the Internet accentuates the importance of supporting students’ competencies to critically evaluate information. This study compared how individuals and pairs of secondary students worked to evaluate the quality of online information across two inquiry topics. Two similar studies were conducted with 140 Finnish (Study I) and 52 US (Study II) students. Students were asked to conduct an online inquiry and then write an essay about one of two topics: allowing the genetic modification of organisms (GMO) or the effects of social media on people’s quality of life (SM). Students worked either individually or in pairs. Their work was supported with a digital tool that prompted them to evaluate the credibility of online texts they selected as sources for their essays. Three separate analyses of covariance were conducted to examine differences in evaluation of online texts between individual and paired readers as well as between the inquiry topics across three dimensions: 1) total number of relevant justifications for credibility evaluations, 2) different types of relevant justifications, and 3) overall quality of students’ evaluations and justifications across all responses captured by the digital tool. Results showed that working in pairs appeared to support the evaluation of online information in all assessed aspects in one context (Study I) but not in another (Study II). In Study 1, the GMO topic appeared to stimulate students’ evaluation of information slightly more than SM, while there was no difference in evaluation performance across topics in Study 2. Findings suggest that discussing the credibility of online texts with a partner is a promising practice. Future research should explore more qualitative dimensions of how partners work together as they evaluate online texts and how instruction could be used to support collaborative evaluation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Applebee, A.N., Langer, J.A., Nystrand, M., & Gamoran, A. (2003). Discussion-based approaches to developing understanding: Classroom instruction and student performance in middle and high school English. American Educational Research Journal, 40, 685–730. https://doi.org/10.3102/00028312040003685

    Article  Google Scholar 

  • Australian Curriculum Assessment and Reporting Authority. (n.d.). The Australian Curriculum, v6.0. Retrieved from http://www.australiancurriculum.edu.au/Home

  • Barzilai, S., Tzadok, E., & Eshet-Alkalai, Y. (2015). Sourcing while reading divergent expert accounts: Pathways from views of knowing to written argumentation. Instructional Science, 43, 737–766. https://doi.org/10.1007/s11251-015- 9359-4

    Article  Google Scholar 

  • Barzilai, S. & Zohar, A. (2012). Epistemic thinking in action: Evaluating and integrating online sources. Cognition and Instruction, 30, 39–85. https://doi.org/10.1080/07370008.2011.636495

    Article  Google Scholar 

  • Bogdan, R.C., & Biklen, S.K. (2003). Qualitative research for education: An introduction to theories and methods (4th ed.). Boston, MA: Allyn & Bacon.

    Google Scholar 

  • Brand-Gruwel, S., Wopereis, I., & Vermetten, Y. (2005). Information problem solving by experts and novices: Analysis of a complex cognitive skill. Computers in Human Behavior, 21, 487–508. https://doi.org/10.1016/j. chb.2004.10.005

    Article  Google Scholar 

  • Bråten, I., & Braasch, J.L.G. (2017). Key issues on students’ critical reading and learning in the 21st century information society. In C. Ng & B. Bartlett (Eds.), Improving reading and reading engagement in the 21st century: International research and innovations (pp. 77–98). Singapore: Springer.

    Chapter  Google Scholar 

  • Bråten, I., McCrudden, M.T., Stang Lund, E., Brante, E.W., & Strømsø, H.I. (2018). Task-oriented learning with multiple documents: Effects of topic familiarity, author expertise, and content relevance on document selection, processing, and use. Reading Research Quarterly, 53, 345–365. https://doi.org/10.1002/rrq.197

    Article  Google Scholar 

  • Bråten, I., Stadtler, M., & Salmerón, L. (2018). The role of sourcing in discourse comprehension. In M.F. Schober, M.A. Britt, & D.N. Rapp (Eds.), Handbook of Discourse Processes (2nd. ed., pp. 141–166). New York, NY: Routledge.

    Google Scholar 

  • Bråten, I., Strømsø, H.I., & Britt, M.A. (2009). Trust matters: Examining the role of source evaluation in students’ construction of meaning within and across multiple texts. Reading Research Quarterly, 44, 6–28. https://doi.org/10.1598/RRQ.44.1.1

    Article  Google Scholar 

  • Britt, M.A., & Aglinskas, C. (2002). Improving students’ ability to identify and use source information. Cognition and Instruction, 20, 485–522. https://doi.org/10.1207/S1532690XCI2004_2

    Article  Google Scholar 

  • Britt, M.A., Rouet, J.F., & Durik, A.M. (2017). Literacy beyond text comprehension: A theory of purposeful reading. New York, NY: Routledge.

    Book  Google Scholar 

  • Castek, J., Coiro, J., Guzniczak, L., & Bradshaw, C. (2012). Examining peer collaboration in online inquiry. The Educational Forum, 76, 479–496. https://doi.org/10.1080/00131725.2012.707756

    Article  Google Scholar 

  • Cho, B.-Y., Woodward, L., Li, D., Barlow, W. (2017). Examining adolescents’ strategic processing during online reading with a question-generating task. American Educational Research Journal, 54, 691–724. doi:10.3102/0002831217701694

    Article  Google Scholar 

  • Coiro, J. (2011). Talking about reading as thinking: Modeling the hidden complexities of online reading comprehension. Theory Into Practice, 50, 107–115. https://doi.org/10.1080/00405841.2011.558435

    Article  Google Scholar 

  • Coiro, J., Castek, J., & Guzniczak, L. (2011). Uncovering online reading comprehension processes: Two adolescents reading independently and collaboratively on the Internet. 60th Yearbook of the Literacy Research Association, 354–369.

    Google Scholar 

  • Coiro, J., Coscarelli, C., Maykel, C., & Forzani, E. (2015). Investigating criteria that seventh graders use to evaluate the quality of online information. Journal of Adolescent & Adult Literacy, 59, 287–297. https://doi.org/10.1002/jaal.448

    Article  Google Scholar 

  • Craig, S.D., Sullins, J., Witherspoon, A., & Gholson, B. (2006). The deep-level reasoning questions effect: The role of dialogue and deep-level reasoning questions during vicarious learning. Cognition and Instruction, 24, 565–591. https://doi.org/10.1207/s1532690xci2404_4

    Article  Google Scholar 

  • Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg (Ed.), Collaborative-learning: Cognitive and computational approaches (pp. 1–19). Oxford: Elsevier.

    Google Scholar 

  • Dillenbourg, P., & Schneider D. (1995). Mediating the mechanisms which make collaborative learning sometimes effective. International Journal of Educational Telecommunications, 1 (2-3), 131–146.

    Google Scholar 

  • European Commission, Directorate-General for Communication Networks, Content and Technology. (2018). A multi-dimensional approach to disinformation: report of the independent high level group on fake news and online disinformation. Brussels: European Union. doi:10.2759/739290 Retrieved from https://www.cato.org/publications/policy-analysis/ risky-business-role-arms-sales-us-foreign-policy

    Google Scholar 

  • Fischer, F., Kollar, I. Mandl, H. & Haake J.M. (Eds.), Scripting computer-supported collaborative learning: Cognitive, computational and educational approaches (pp. 263-271). New York, NY: Springer.

  • Forzani, E. (2018). How well can students evaluate online science information? Contributions of prior knowledge, gender, socioeconomic status, and offline reading ability. Reading Research Quarterly, 53, 385–390. https://doi.org/10.1002/rrq.218

    Article  Google Scholar 

  • Gerjets, P., Kammerer, Y., & Werner, B. (2011). Measuring spontaneous and instructed evaluation processes during Web search: Integrating concurrent thinkingaloud protocols and eye-tracking data. Learning and Instruction, 21, 220–231. https://doi.org/10.1016/j.learninstruc.2010.02.005

    Article  Google Scholar 

  • Goldman, S.R., Braasch, J.L.G., Wiley, J., Graesser, A.C., & Brodowinska, K. (2012). Comprehending and learning from internet sources: Processing patterns of better and poorer learners. Reading Research Quarterly, 47, 356–381. https://doi.org/10.1002/RRQ.027

    Google Scholar 

  • Hogan, N., & Vernhagen, C. (2012). Critical appraisal of information on the Web in practice: Undergraduate students’ knowledge, reported use, and behaviour. Canadian Journal of Learning and Technology, 38(1), 1–14. http://dx.doi.org/10.21432/T23K5P

    Article  Google Scholar 

  • Häkkinen, P., & Mäkitalo-Siegl, K. (2007). Educational perspectives on scripting CSCL. In Fischer, F., Kollar, I., Mandl, H. & Haake, J.M. (Eds.) Scripting computersupported collaborative learning. (pp. 263–271). Springer.

    Chapter  Google Scholar 

  • Jeong, H., & Hmelo-Silver, C.E. (2016). Seven affordances of computer-supported collaborative learning: How to support collaborative learning? How can technologies help? Educational Psychologist, 51, 247–265. https://doi.org/10.1080/00461520.2016.1158654

    Article  Google Scholar 

  • Johnson, F., Sbaffi, L., & Rowley, J. (2016). Students’ approaches to the evaluation of digital information: Insights from their trust judgments. British Journal of Educational Technology, 47, 1243–1258. https://doi.org/10.1111/bjet.12306

    Article  Google Scholar 

  • Kammerer, Y., Meier, N., & Stahl, E. (2016). Fostering secondary-school students’ intertext model formation when reading a set of websites: The effectiveness of source prompts. Computers & Education, 102, 52–64. https://doi.org/10.1016/j.compedu.2016.07.001

    Article  Google Scholar 

  • Kiili, C., Coiro, J., & Hämäläinen, J. (2016). An online inquiry tool to support the exploration of controversial issues on the Internet. Journal of Literacy and Technology, 17(1-2), 31–52.

    Google Scholar 

  • Kiili, C., Laurinen, L., & Marttunen, M. (2008). Students evaluating Internet sources - From versatile evaluators to uncritical readers. Journal of Educational Computing Research, 39, 75–95. https://doi.org/10.2190/EC.39.1.e

    Article  Google Scholar 

  • Kiili, C., Laurinen, L., Marttunen, M., & Leu, D.J. (2012). Working on understanding during collaborative online reading. Journal of Literacy Research, 44, 448–483. https://doi.org/10.1177/1086296X12457166

    Article  Google Scholar 

  • Kiili, C., Leu, D.J., Marttunen, M., Hautala, J., & Leppänen, P.H.T. (2018). Exploring early adolescents’ evaluation of academic and commercial online resources related to health. Reading and Writing, 31, 533–557. https://doi.org/10.1007/s11145-017-9797-2

    Article  Google Scholar 

  • King, A. (2002). Structuring peer interaction to promote high-level cognitive processing. Theory into Practice, 41, 33–39.

    Article  Google Scholar 

  • King, A., & Rosenshine, B. (1993). Effect of guided cooperative questioning on children’s knowledge construction. Journal of Experimental Education, 61, 127–148. https://doi.org/10.1080/00220973.1993.9943857

    Article  Google Scholar 

  • Knight, S., & Mercer, N. (2017). Collaborative epistemic discourse in classroom information-seeking tasks. Technology, Pedagogy and Education, 26, 33–50. https://doi.org/10.1080/1475939X.2016.1159978

    Article  Google Scholar 

  • Kreijns, K., Kirschner, P.A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computersupported collaborative learning environments: A review of the research. Computers in Human Behaviour, 19, 335–353. https://doi.org/10.1016/S0747-5632(02)00057-2

    Article  Google Scholar 

  • Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Lazonder, A.W. (2005). Do two heads search better than one? Effects of student collaboration on web search behaviour and search outcomes. British Journal of Educational Technology, 36, 465–475. https://doi.org/10.1111/j.1467-8535.2005.00478.x

    Article  Google Scholar 

  • Leu, D.J., Kinzer, C.K., Coiro, J., Castek, J., & Henry, L.A. (2013). New literacies: A dual level theory of the changing nature of literacy, instruction, and assessment. In D.E. Alvermann, N.J. Unrau & R.B. Ruddell (Eds.), Theoretical models and processes of reading (6th ed., pp. 1150–1181). Newark, DE: International Reading Association.

    Chapter  Google Scholar 

  • Lou, Y., Abrami, P.C., & d’Apollonia, S. (2001). Small group and individual learning with technology: A meta-analysis. Review of Educational Research, 71, 449–521. https://doi.org/10.3102/00346543071003449

    Article  Google Scholar 

  • McCrudden, M.T., Bråten, I., & Braasch, J.L. (2018). Introduction to research on multiple source use. In J.L.G. Braasch, I. Bråten & M.T. McCrudden (Eds.), Handbook of multiple source use (pp. 1–13). New York, NY: Routledge.

    Google Scholar 

  • McCrudden, M.T., Stenseth, T., Bråten, I., & Strømsø, H.I. (2016). The effects of topic familiarity, author expertise, and content relevance on Norwegian students’ document selection: A mixed methods study. Journal of Educational Psychology, 108, 147–162. http://dx.doi.org/10.1037/edu0000057

    Article  Google Scholar 

  • McGrew, S., Ortega, T., Breakstone, J., & Wineburg, S. (2017). The challenge that’s bigger than fake news: Civic reasoning in a social media environment. American Educator, Fall Issue, 4–7.

    Google Scholar 

  • Metzger, M.J. (2007). Making sense of credibility on the Web: Models for evaluating online information and recommendations for future research. Journal of the American Society for Information Science and Technology, 58, 2078–2091. https://doi.org/10.1002/asi.20672

    Article  Google Scholar 

  • National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards for English language arts and literacy in history/social studies, science, and technical subjects. Washington, DC.

    Google Scholar 

  • Nicol, D.J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31, 199–218. https://doi.org/10.1080/03075070600572090

    Article  Google Scholar 

  • Nokes-Malach, T.J., Richey, J.E., & Gadgil, S. (2015). When is it better to learn together? Insights from research on collaborative learning. Educational Psychology Review, 27, 645–656. https://doi.org/10.1007/s10648-015-931

    Article  Google Scholar 

  • Paul, J., Macedo-Rouet, M., Rouet, J-F., & Stadtler, M. (2017). Why attend to source information when reading online? The perspective of ninth grade students from two different countries. Computers & Education, 113, 339–354. https://doi.org/10.1016/j.compedu.2017.05.020

    Article  Google Scholar 

  • Paul, J., Stadtler, M., & Bromme, R. (2017). Effects of a sourcing prompt and conflicts in reading materials on elementary students’ use of source information. Discourse Processes. Advance online publication. https://doi.org/10.1080/0163853X.2017.1402165

    Google Scholar 

  • Pérez, A., Potocki, A., Stadtler, M., Macedo-Rouet, M., Paul, J., Salmerón, L., & Rouet, J.F. (2018). Fostering teenagers’ assessment of information reliability: Effects of a classroom intervention focused on critical source dimensions. Learning and Instruction, 58, 53–64.

    Article  Google Scholar 

  • Perfetti, C.A., Rouet, J.-F., & Britt, M.A. (1999). Towards a theory of documents representation. In H. van Oostendorp & S. Goldman (Eds.), The construction of mental representations during reading (pp. 99–122). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Roschelle, J., & Teasley, S. (1995). The construction of shared knowledge in collaborative problem solving. In C. O’Malley (Ed.), Computer-supported collaborative learning (pp. 69–197). Berlin, Germany: Springer Verlag.

    Chapter  Google Scholar 

  • Springer, L., Stanne, M.E., & Donovan, S.S. (1999). Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational Research, 69, 21–51. https://doi.org/10.3102/00346543069001021

    Article  Google Scholar 

  • Teasley, S.D. (1995). The role of talk in children’s peer collaborations. Developmental Psychology, 31, 207–220. https://doi.org/10.1037/0012-1649.31.2.207

    Article  Google Scholar 

  • Thomm, E., & Bromme, R. (2012). ‘It should at least seem scientific!’ Textual features of ‘scientificness’ and their impact on lay assessments of online information. Science Education, 96, 187–211. https://doi.org/10.1002/sce.20480

    Article  Google Scholar 

  • Tseng, A.S. (2018). Students and evaluation of web-based misinformation about vaccination: Critical reading or passive acceptance of claims? Advanced online publication. International Journal of Science Education, Part B. https://doi.org/10.1080/21548455.2018.1479800

    Google Scholar 

  • Tseng, S., & Fogg, B.J. (1999). Credibility and computing technology. Communications of the ACM, 42 (5), 39–44. https://doi.org/10.1145/301353.301402

    Article  Google Scholar 

  • van den Boom, G., Paas, F., van Merrienboer, J.J.G., & van Gog, T. (2004). Reflection prompts and tutor feedback in a web-based learning environment: Effects on students’ self-regulated learning competence. Computers in Human Behavior, 20, 551–567. https://doi.org/10.1016/j.chb.2003.10.001

    Article  Google Scholar 

  • Vygotsky, L.S. (1986). Thought and language. Cambridge, MA: MIT Press.

    Google Scholar 

  • Walraven, A., Brand-Gruwel, S., & Boshuizen, H.P. (2009). How students evaluate information and sources when searching the World Wide Web for information. Computers & Education, 52, 234–246. https://doi.org/10.1016/j.compedu.2008.08.003

    Article  Google Scholar 

  • Walraven, A., Brand-Gruwel, S., & Boshuizen, H.P. (2013). Fostering students’ evaluation behaviour while searching the Internet. Instructional Science, 41, 125–146. https://doi.org/10.1007/s11251-012-9221-x

    Article  Google Scholar 

  • Wigfield, A., & Guthrie, J.T. (1997). Relations of children’s motivation for reading to the amount and breadth or their reading. Journal of Educational Psychology, 89, 420–432. http://dx.doi.org/10.1037/0022-0663.89.3.420

    Article  Google Scholar 

  • Wiley, J., Goldman, S.R., Graesser, A.C., Sanchez, C.A., Ash, I.K., & Hemmerich, J.A. (2009). Source evaluation, comprehension, and learning in Internet science inquiry tasks. American Educational Research Journal, 46, 1060–1106. https://doi.org/10.3102/0002831209333183

    Article  Google Scholar 

  • Zhang, M. (2013). Supporting middle school students’ online reading of scientific resources: Moving beyond cursory, fragmented, and opportunistic reading. Journal of Computer Assisted Learning, 29, 138–152. https://doi.org/10.1111/j.1365-2729.2012.00478.x

    Article  Google Scholar 

  • Zhang, M., & Quintana, C. (2012). Scaffolding strategies for supporting middle school students’ online inquiry processes. Computers & Education, 58, 181–196. https://doi.org/10.1016/j.compedu.2011.07.016

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Oslo, Norway

    Carita Kiili

  2. University of Rhode Island, USA

    Julie Coiro

  3. University of Jyvaskyla, Finland

    Eija Räikkönen

Authors
  1. Carita Kiili
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Julie Coiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eija Räikkönen
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kiili, C., Coiro, J. & Räikkönen, E. Students’ evaluation of information during online inquiry: Working individually or in pairs. AJLL 42, 167–183 (2019). https://doi.org/10.1007/BF03652036

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03652036

Search

Navigation