Beyond the Passive Absorption of Information: Engaging Students in the Critical Reading of Scientific Articles

Abstract

There is a consensus within the science education community that primary scientific literature is a legitimate and desirable educational resource. Moreover, critical reading of scientific articles is widely recognized as a key aspect of scientific literacy. However, university science courses rarely provide students with explicit opportunities to cultivate their critical reading skills. Much of the reason for this is that instructors tend to hold a passive learning view of reading in which students are expected to absorb information from scientific articles. The purpose of this study was to provide research evidence that an active learning scenario (ALS) combining (1) argumentation, (2) peer critique (also referred to as peer assessment), and (3) the Task-Oriented Reading Instruction framework (Ritchey & List, College Teaching, 70(3), 280–295, 2022) could be a concrete and realistic possibility for engaging students in the critical reading of scientific papers. The data analyzed in this study were the written critiques of scientific research articles and written peer feedback produced by sixty-one university students (38 females and 23 males, 19–25 years old). The results indicate that the ALS effectively offered students explicit opportunities to become more active and more critical readers of scientific articles, producing arguments, anticipating counterarguments, and constructing rebuttals. Implications related to critical reading instruction in science education and supporting students’ development of critical reading skills are discussed.

1 Introduction

The preparation of critical readers is a crucial but neglected aspect of higher education institutions. This is a common reflection made by the authors of the twelve chapters of Reading Across the Disciplines (Manarin, 2022). Critical reading is an umbrella term which means different things to different people and in different contexts. In the case of academic settings, becoming a critical reader implies developing skills about the identification of patterns of textual elements, the distinction between main and subordinate ideas, the evaluation of credibility, judgement about the argumentation of a text, and production of relevant inferences (Manarin et al., 2015). This definition gives us an idea of how challenging and complex the goal of preparing critical readers is. Unfortunately, there is a lack of training among faculty about how to nurture critical reading skills (Sutherland & Incera, 2021). One consequence of this is that instructors limit their didactic action to (1) giving reading assignments and (2) expecting students to read independently rather than acting as reading mentors (Hubbard, 2021; Hubbard et al., 2022). Hence, it is exceedingly difficult, if not impossible, for students to enrich their critical reading skills when they do not receive real guidance and support.

In the field of science education, there is a notable consensus that primary scientific literature—reports of novel observations, theories, or opinions, communicated in writing for peers in the scientific community (Van Lacum et al., 2014)—is a legitimate and desirable educational resource that contributes to authentic scientific literacy. Griffiths and Davila (2022) reiterate that “critical reading of scientific texts is a key practice in science, as the primary literature constructs knowledge and communicates research in the field” (p. 143). Moreover, these scholars emphasize the need to create research-based teaching and learning strategies that help university students to become engaged in the critical reading of scientific articles (also referred to as scientific research articles, or scientific papers). Naturally, such creation entails moving away from the passive learning view of reading in which students are expected to absorb information towards more active learning practices that provide them with explicit opportunities to go beyond a superficial reading of scientific articles to critically engage with these readings.

It may be obvious to point out that critical reading of scientific papers needs argumentative reasoning. Nevertheless, “university science courses give undergraduates limited opportunities to strengthen their argumentation skills” (Archila et al., 2023a, p. 635). Previous attempts to cultivate critical reading of scientific research articles (e.g., Lammers et al., 2019; Van Lacum et al., 2014, 2016; Wijayanti & Adi, 2022) have focused on giving students explicit opportunities to identify components of the authors’ argumentation (e.g., supports, counterarguments, refutations). In the current study, we explore an under-researched possibility: to engage undergraduates in the critical reading of scientific articles through the construction of arguments, counterarguments, and rebuttals. Additionally, we maintain that this possibility is a rational and reasonable scenario to involve students in peer assessment (also known as peer critique)—peer grading and providing useful feedback (Winstone & Carless, 2020)—while critiquing the argumentation of other equal-status students.

Recently, Ritchey and List (2022) proposed a theoretically based framework called Task-Oriented Reading Instruction (TORI). This framework was the bedrock of our study. Much of the reason for this is that TORI was created to explicitly provide goals for reading, to specify strategies to help students meet these goals, and to align assessment with assigned reading goals. Thus, this study was aimed at providing evidence for the claim that an active learning scenario (ALS) combining (1) argumentation, (2) peer critique, and (3) the TORI could be a concrete and realistic possibility for engaging students in the critical reading of scientific research articles. With this in mind, we created an ALS. The central question addressed in this article is as follows: To what extent does the ALS engage science (Biology and Microbiology) students and engineering (Biomedical engineering, Chemical Engineering, and Food Engineering) students in the critical reading of scientific papers assigned in a Food Microbiology course?

2 Literature Review

Scientific reading is a fundamental component in university science education. Nonetheless, many students do not complete course readings. In a study conducted in the California State University (Gorzycki et al., 2019), 206 undergraduate open-ended comments about student priorities and beliefs about academic reading were analyzed. One key result was that although undergraduates tended to declare that reading is important, they do not read. Another outcome was that this situation seems to be exacerbated by deficient educational practices such as the lack of explicit opportunities for undergraduate students to cultivate their discrete academic reading skills (e.g., identifying implications, synthesizing sources, and critiquing assertions). Also, in California State University, Desa et al. (2020) analyzed the written comments about academic reading in the undergraduate experience of thirty-three instructors. Interestingly, these researchers found that even though faculty seemed to be convinced that academic reading is crucial for undergraduates’ success, they seldom integrated academic reading pedagogies into their practices. Moreover, these scholars concluded that there were contextual factors that reinforced this issue, namely, (1) faculty epistemologies, (2) faculty assumptions about learning in higher education (instructors tend to teach as they were taught), and (3) the institution’s cultural leanings. These factors undoubtedly evoke Kampourakis’s (2017) reflection that one fundamental cause of the issues in university science courses is the way “science teachers themselves were taught science in their undergraduate studies” (p. 202). Furthermore, he insists that nothing can change until higher education institutions become aware that instructors’ publications in prestigious journals is important but not their principal mission, which is teaching and learning.

In a recent survey study (N = 249), Hairston-Dotson and Incera (2022) discovered a paradox: although undergraduate students acknowledged that complex critical reading skills (e.g., applying) are more useful than simpler reading skills (e.g., skimming), they reported practicing skimming more often than applying when working on their assignments. These academics recommend that instructors, instead of merely asking students “to read” a text, should create time and space within the classroom so students have concrete and realistic opportunities to practice and develop complex critical reading skills while actively engaging with the text. It is noteworthy that in another study (N = 128), Sutherland and Incera (2021) sought to investigate the opinions about critical reading of faculty at Eastern Kentucky University. One outcome was that instructors considered more complex critical reading skills (e.g., applying) as more useful, while they considered simpler skills (e.g., skimming) less useful. Additionally, participants appeared to be more willing to invest more time of their classes to teach more complex critical reading skills than simpler ones. Arguably, it is this willingness that science educators can use as leverage to encourage and help science teachers take advantage of educational resources to promote critical reading in their courses.

There is empirical research that supports the claim that primary scientific literature is one legitimate and desirable educational resource than can be exploited in multiple ways, such as the use of this type of literature to introduce students to the nature of science. For example, Wenk and Tronsky (2011) maintain that research articles make explicit aspects of nature of science (e.g., collection, analysis, and interpretation of data) that many textbooks tend to ignore. Specifically, these scholars provide research evidence for the claim that introductory science classrooms can be transformed into educational spaces in which first year students find opportunities to strengthen their skills to read primary scientific literature and that doing so enriches their understanding of the process of science. In similar fashion, Schmid et al. (2021) explored the usefulness of an approach combining early exposure to this type of literature and interactions with scientists. The soundness of this approach was evidenced in the United States with 12 science undergraduate students from a research-intensive university who were enrolled in a seminar-style introduction to biological literature course. An interesting result of this study was that although explicit nature of science instruction was not part of this approach, these academics found that participants developed certain understandings about the nature of science. Of course, they acknowledged that explicit instruction would have benefited their approach.

Argumentation is intimately related to the nature of science since this skill is essential in the process of doing science as well as the communication in science (Lemke, 1990). Hence, Khishfe (2014, 2023a, b) highlights the relevance of improving science education practices by integrating argumentation skills and nature of science ideas. In this sense, the results of intervention studies conducted by Khishfe (2021 (N = 36), 2023b (N = 42)) with grade 10 students suggest that explicit instruction of argumentation connected with explicit instruction of the nature of science contributes to the promotion of students’ argumentation as well as the development of conceptions about the nature of science. As these scholars comment, one explanation for this is that argumentation and the nature of science are closely linked.

Additionally, it is important to recognize that in 2007, Hoskins and her colleagues developed the Consider, Read, Elucidate the hypotheses, Analyze and interpret the data, and Think of the next Experiment (CREATE) method. This is an educational tool for teaching science and introducing undergraduates to issues regarding the nature of science through primary scientific literature. They maintain that CREATE works as a supplement and complement of traditional lecture-based science teaching and inquiry laboratory classes. Likewise, they report that this method not only increased understanding of scientific research among undergraduate students but also their interest in this type of research. Also, improvements in the undergraduates’ ability to critically read and interpret data were found. CREATE was originally designed and implemented within the context of genetics and cell biology and has been applied or modified in various university courses such as introductory biology courses (Chatzikyriakidou et al., 2021) and ecological courses (Carter & Wiles, 2017; Smith & Paradise, 2022). Despite the benefits of CREATE, as Lennox et al. (2020) point out, some of its disadvantages are that this method was designed for students who have achieved a certain mastery of disciplinary knowledge and the application of this pedagogical tool requires a considerable amount of work by instructors. In addition, Segura-Totten and Dalman (2013) question the effectiveness of this method to lead to higher learning gains in comparison with more traditional ways of reading primary scientific literature.

Recently, Chatzikyriakidou et al. (2022) demonstrated that increasing student engagement with primary scientific literature contributed to the promotion of research skills. They used the five core concepts of biology as a framework for student engagement with this type of literature. The analyses included the responses of 19 undergraduates to a questionnaire before and after an educational intervention in a discussion-based introductory biology course. One of the major findings of this study was that asking participants to complete a five core concept matrix table, connecting the biological content from a piece of primary scientific literature to at least three corresponding boxes of the table, encouraged students to produce notes and/or summaries of the content contained within research articles in their own words. This is a clear example of how to move beyond the passive absorption of information. Furthermore, the results of another study (Chatzikyriakidou et al., 2021) reaffirmed that involving students in purposeful activities such as the completion of a five core concepts of biology matrix table as they read a selected scientific paper and prompted them to begin to think of scientific facts contained within that paper as part of larger biological concepts. Importantly, one of the conclusions of a study conducted by Lee et al. (2022) was that simply reading a PDF® does little to enrich reading strategies in students learning to read primary scientific literature.

Instructors should not assume that undergraduate students have the skills and strategies to critically engage with primary research literature. This was one of the conclusions of a recent study carried out in a UK university (Hubbard et al., 2022). Data were collected through thirty-three interviews which were conducted with second year undergraduates (n = 9), final year undergraduates (n = 7), PhD students (n = 5), postdoctoral researchers (n = 6) and academics (n = 6). Furthermore, Bjorn et al. (2022) observe that even doctoral students struggle to engage critically with primary literature. Thus, instructors should explicitly create opportunities for students to actively engage with research literature. To illustrate this, consider the positive results reported by Heiss and Liu (2022) in analytical chemistry courses where students were asked to read scientific articles which were then debated in a discussion led by a team of students. According to the authors, this contributed to the reinforcement of the importance of topics covered in these courses. Another example related to this link between active learning and primary scientific literature is work by Palavalli-Nettimi et al. (2022) who provided biology undergraduates with opportunities to (1) read a piece of primary research literature, (2) develop annotations to dig deeper into the research communicated in the article, and (3) produce a podcast episode to share the research results with a general audience.

Even though peer assessment is widely accepted as a strategic ally of active learning practices (Amo & Jareño, 2011; de-Armas-González et al., 2023; Delgado Rodríguez, 2017; Reuse-Durham, 2005), the use of peer critique to promote critical reading is an under-researched possibility in science education. Indeed, there is not much evidence about the use of this type of assessment to foster critical reading of primary research literature in undergraduate science education. The novel results reported by Deng and his colleagues (2019) provide encouragement to embrace peer assessment. These academics tested an educational strategy with twenty-two undergraduate chemistry students at a Chinese university. The outcomes indicate that the integration of reading, anonymous peer critique, and discussion helped participants to engage with scientific writing practice. Besides, evidence led the researchers to conclude that peer evaluation was an effective means to provide opportunities for undergraduate students to (1) share their scientific writing productions, (2) compare each other’s production, and (3) engage in self-reflection. It should be pointed out that the participants in this Chinese study were asked to read four entire organic synthesis papers.

At this point, it is important to bear in mind that asking students to read an entire article is not the only option to introduce primary scientific literature in university science classrooms. Asking students to read specific sections (e.g., the introduction) is another acceptable option (Bogucka & Wood, 2009). Hunter and Kovarik (2022), for instance, have explored the use of excerpts and data from primary literature to help students focus on applying what they have learned in class. This exploration was carried out using active learning in analytical chemistry courses offered by The College of New Jersey and Trinity College. One of the reasons mentioned by these academics against recommending that instructors ask the students to read one full article is that “reading the entire paper makes the assignments more laborious and likely too time-intensive for the frequent practice that is desired” (p. 1242). In our case, undergraduates were required to read two entire scientific articles outside class, while specific parts of these papers were addressed during class time. In the next section, this and other features of our ALS will be discussed in detail.

3 Conceptual Framing

In this section, we discuss the conceptual framework on which the pillars (TORI, argumentation, and peer critique) of our ALS were based. To begin with, it is important to define learning, active learning, and ALS. Archila et al. (2022a) “define learning as constructing meaning and reflecting critically on this meaning” (p. 3). Therefore, we assume that critical reading is a prime contributor to students’ learning achievement since critical readers are able to actively construct meaning from textual (and visual) information as they think critically about this information, going beyond a surface reading to read between the lines (Griffiths & Davila, 2022). While numerous definitions exist, active learning in tertiary education can be described broadly as abandoning lecture-based instruction (passive learning), engaging students in activities where they do and/or produce something, cultivate their higher-order thinking skills (e.g., argumentation), and critically reflect on what they are doing and/or producing (Mizokami, 2018).

Archila et al. (2022a) stress that active learning is a valuable option to counter the centuries old instructor-centered lecture format that has dominated science teaching and learning in higher education institutions around the globe, since active learning focuses on higher-order thinking development rather than the transmission, memorization, and recall of information. They further point out that it is more likely that science instructors will embrace active learning practices if these are presented as a sequence of concrete and realistic research-based actions instead of abstract and idealized discourses. From this perspective, in this article, we coin the term ALS to refer to a series of activities purposeful planned and implemented in a classroom to involve students in the process of learning.

To reiterate, the TORI is a vital pillar of our ALS. It is worth mentioning that Ritchey and List (2022) proposed this framework, drawing on decade-long research advances in education as well as in psychology. Importantly, the TORI framework highlights that tertiary education-level reading has three characteristics. The first characteristic results from the fact that tertiary education-level reading is domain specific. Texts are informed by the epistemic aims and practices of particular domains (e.g., business, chemistry, and history). Clearly, the content communicated in a text of a particular domain (e.g., business) differs from one from another particular domain (e.g., chemistry). This means that in the process of making sense of this content, students should make use of domain-specific strategies. Scientific research articles, for instance, are constructed on the basis of scientific argumentation processes. It is therefore reasonable to point out that this type of text follows an argumentative structure (Van Lacum et al., 2014, 2016) which should be considered in the creation of strategies to help students critically engage with scientific articles.

The second characteristic is variability. Students in courses at tertiary level may be asked to read different types of texts (e.g., chapters from handbooks, news articles, scientific articles, and textbooks). The importance of this characteristic is that students are likely to be more familiarized with some type of texts (e.g., news articles) than others (e.g., scientific articles) which are not necessarily written with a student audience in mind. This leads to the claim that students should be specially assisted in the reading process of expert-level texts. And the third characteristic of tertiary education-level reading is autonomy. Tertiary education students are regularly expected to demonstrate a high level of autonomy during the completion of course reading assignments. It should be pointed out that “autonomy” does not mean that instructors can assume that students have the skills to be able to critically engage with disciplinary texts. Research evidence, for example, shows that students seldom use critical reading skills. Instead, skimming and skipping sections and information when working on reading assignments are very common actions among students (Hairston-Dotson & Incera, 2022; Lennox et al., 2020). In this regard, Hubbard (2021) claims that giving students support is essential in order to cultivate their critical reading skills.

Ritchey and List (2022) maintain that an attractive strength of the TORI is that this can be applied in a relatively simple way and in any course and discipline. An important reason for this is that the instructional framework consists of only five pragmatic steps all related to the instructor. The first step involves selecting the text that will be assigned to the students. This step tends to be familiar to instructors since text selection is an integral part of their course design process (Collins-Dogrul & Saldaña, 2019). It is important to stress that this selection should be guided by the goal(s) for reading as some text types (e.g., scientific article) may be more linked to specific types of reading goals (e.g., evaluating the coherence of an experimental design). The second step concerns the identification of the specific goal(s) that is expected students should achieve. It is recommended that this goal should be included in a short statement in declarative style, using a verb (e.g., judge) at the beginning. This helps students to have more clarity about the process of reasoning entailed in the reading goal. Of course, the accomplishment of such a goal would contribute to the strengthening of the students’ reading routines.

The third step addresses the definition of the means by which students’ reading will be assessed. Both format (e.g., open questions) and cognitive demand (e.g., argumentative questions) are aspects that can vary. Thus, the challenge for instructors in this step is to ensure that the assessment(s) is aligned with the assigned reading goal(s). Likewise, faculty should make clear to students how the assessment is influenced by the reading goal. The fourth step consists of identifying effective strategies to help students meet the goal of the assigned reading and prepare for assessment. This step is a good opportunity for faculty to not only present students with effective and pragmatic reading strategies to use to tackle expectations regarding reading but also to help them become aware of the importance of using strategies during the reading process. A prime reason for promoting the use of reading strategies among students is that these strategies can help them to better explore the text (e.g., preview the headings, highlight topic sentences, and write down main ideas) and reflect on the ideas of this text (e.g., think about what the highlighted sentences have in common). By the same token, these strategies should be acknowledged as concrete actions carried out to engage productively with the text. Accordingly, the major purpose of the fifth step of the TORI is devoted to the explicit communication of the information from steps 1 to 4 to the students. Instructors can use this step to show students the articulation and coherence between each of the four previous steps. Likewise, this step serves to help students recognize how their reading process is being supported. We will explain later how these steps were adopted in our ALS.

Another major feature of the TORI is that this is guided by the premise that presenting students with explicit, specific goals for reading is a condition sine qua non for the promotion of productive task-oriented reading behaviors. This premise makes sense if we acknowledge that students educated in passive science learning classrooms tend to read with the sole purpose of retaining information that they feel will be useful to do well in exams, rather than reading to analyze, understand, interpret, and criticize scientific texts (Hubbard & Dunbar, 2017). As Hubbard et al. (2022) observe, students’ passive and exam-oriented view of reading can be countered if instructors start to become aware that scientific texts are a powerful educational resource. In addition, they maintain that faculty should exploit the educational potential of scientific research literature by providing students with explicit opportunities to become more active and more critical readers of this type of literature.

Argumentation—defined as “the justification of claims with reasons and/or evidence” (Erduran et al., 2022, p. 655)—is another pillar of our educational scenario. This cognitive-linguistic skill is intimately connected and vital to critically read scientific articles. One reason for this is that, as Van Lacum et al. (2014) point out, various elements of scientific argumentation (e.g., argument, counterargument, and rebuttal) are communicated in scientific papers. In fact, the production of research articles is a prime example of why scientific argumentation is considered not only an intellectual but also a communicative activity. Given that argumentation contributes to the production of any reason-based and/or evidence-based scientific construct (e.g., explanations, models, and theories), it is no exaggeration to say this skill plays a vital role in the progress of science. Finocchiaro (2021), for instance, reminds us that Galileo, the father of modern science, was an authentic practitioner of argumentation. Another example of the substantial value of argumentation in the scientific enterprise is documented in the recent book The Pandemic of Argumentation (Oswald et al., 2022). A common claim in its eighteen chapters is that scientific argumentation is a powerful skill to fight disinformation and/or misinformation about COVID-19. By the same token, it is an undeniable fact that argumentation is beneficial to the promotion of COVID-19 literacy—“the functional understanding of Covid-19 as well as making informed decisions based upon this understanding” (Archila et al., 2021a).

The production of arguments, counterarguments, and rebuttals is the specific argumentation skill targeted in our study. The reason for this is that these skills are crucial elements of critical argumentation—the questioning of argumentation in order to develop a balanced (critical) point of view (Walton, 2006). Here, we assume that the production of counterarguments is a rational way to question arguments, while questioning counterarguments requires the formulation of rebuttals. It should be pointed out that critical argumentation is in line with the idea that argumentation benefits from critical thinking and vice versa (Andrews, 2015). Osborne (2010) maintains that scientists routinely engage in critical argumentation. Also, he notes that the construction of counterarguments and rebuttals is an example of the critical side of argumentation. The importance of critical argumentation is that it allows students to avoid the development of biased argumentation (Archila et al., 2023a, 2023b). Therefore, it does not sound surprising that critical argumentation has become a key ally of critical reading which is widely acknowledged as a branch of critical thinking (Archila et al., 2019; Lin, 2014; Ritchey & List, 2022; Wilson, 2016). Wallace and Wray (2021) emphasize the importance of always reading academic journal articles with a critical eye, adopting a reasonable skeptical stance towards the authors’ claims. They explain that reasonable skepticism entails being open-minded and willing to recognize the robustness of the authors’ claims, but only if they can adequately support their claims.

Peer critique is the third pillar of our ALS. The reason why we separate peer critique from argumentation (the second pillar of the ALS) is that it is expected that first of all, students will produce arguments, counterarguments, and rebuttals, and then, they will become engaged in peer critique of the argumentation produced by others. Peer assessment is one of the various sources of assessment for learning (also referred to as formative assessment) which is defined as “a process in which teachers and students recognize and respond to student learning, during that learning” (Cowie, 2012, p. 679). We agree with Slater (2020) that successful implementation of active learning in university science courses requires instructors to become aware that lecture-based instruction is the perpetuation of passive learning and that they should provide students with more meaningful learning experiences such as formative assessment. In this sense, peer assessment emerges as a powerful means to make such experiences a reality. Harris and Brown (2013) remind us that fundamental to successful implementation of peer assessment is the construction of awareness of the feature that this type of assessment entails, that students are free from instructor dependence, since peer assessment is essentially a student-led processes. Hence, students need to be thoroughly instructed on how to make useful judgements (useful feedback) about the work of others. It is true that simply instructing students is insufficient. Instructors should provide students with genuine opportunities to criticize the work of their partners in order for them to cultivate peer assessment skills. In our case, instruction and opportunities were even more explicit because the study was conducted in a formative assessment environment.

According to Topping (2018), there is a wide spectrum of products that can be used to engage students in peer assessment practices. Some examples include oral presentations, portfolios, test performance, and writing. Furthermore, he underlines the versatility of peer assessment. For instance, peer critique can be (i) one-way (e.g., an older class assessing a younger class), reciprocal (e.g. same-ability pairs in one class), or mutual within a group; (ii) voluntary or compulsory; (iii) anonymous or not; (iv) on single or multiple pieces of work; (v) on the same or different kind of products; (vi) guided or not by rubrics or structured formats for feedback; (vii) expected or not to lead to opportunities to rework the product in the light of feedback; (viii) organized by matching of students in deliberate and selective ways or random or accidental ways; (ix) carried out in class or outside class; and (x) supported or not by information technology. Specifically, the structure of peer assessment in our ALS was (i) reciprocal, (ii) voluntary, (iii) anonymous, (iv) carried out on multiple pieces of work, (v) carried out on the same kind of products, (vi) guided by rubrics, (vii) articulated with opportunities to make revisions after receiving feedback, (viii) organized by matching students in random ways, (ix) carried out in class, and (x) supported by information technology. We will describe more fully later the way in which these specificities were exploited in our scenario.

4 The Active Learning Scenario

Bearing in mind the idea that it is more likely that instructors will become receptive to implementing an ALS if, at the beginning, this is planned for a limited number of classes, we designed our scenario as two 75-min class sessions and followed a step-by-step structure (three steps in Session 1 and four steps in Session 2) (Fig. 1). It is important to clarify that before these two sessions, undergraduates were given 3 weeks to read two assigned scientific articles (Article 1 (Iacumin et al., 2022); Article 2 (Ayeni et al., 2011)). The instructor explained to the students that Articles 1 and 2 would be the backbone of two in-class activities and that the goal of these readings was to carry out an initial exploration of these papers, at their own pace. Also, during these 3 weeks, they were free to ask the instructor any question about any feature (e.g., scientific content) of the articles. Furthermore, before students started the activity of the first session, they received instruction about how to read and critique scientific articles (Yeong, 2014). As recommended by Rybarczyk (2006) and Van Lacum et al. (2014), emphasis was placed on the role of each element of the conventional structure of research articles (e.g., abstract, introduction, method, results, and discussion).

Fig. 1

Major elements of the active learning scenario

Articles 1 and 2 were chosen for four reasons. The first is that these papers present various of the concepts (e.g., food spoilage microorganisms, molecular biology) discussed during lectures beforehand, thus helping “students [not to] get frustrated with unfamiliar terminology” (Rybarczyk, 2006, p. 166). Second, these manuscripts had standard scientific research article structures (Article 1 (Abstract, Introduction, Materials and methods, Results and discussion, and Conclusion); Article 2 (Abstract, Introduction, Material and Methods, Results, and Discussion)). Third, both articles followed a single-anonymized peer review process before being accepted for publication (Tomkins et al., 2017). And the fourth reason is that these research articles were published in specialized scientific journals.

The ALS revolved around two sections of the standard scientific research article structures, namely, the Introduction section and the Material and Methods section. Much of the reason for this is that we were particularly interested in the valuable opportunities that these sections could give the students to get a summarized idea of the articles (Introduction section) and go deeper into the way the authors conducted the studies (Material and Methods section). These two sections were addressed gradually throughout the two sessions of the ALS to avoid students becoming saturated with information. The first session of the ALS dealt with the Introduction section of Article 1. Students were told that the goal of this session was to read the information communicated in the Introduction section from a critical point of view. In the first step, each student was asked to answer the thought-provoking (and ambiguous) question, “You consider that the quality of the Introduction section of Iacumin et al.’s (2022) article is…”. The five options were Excellent, Very Good, Good, Fair, and Poor. In order to engage students in critical reading, they were required to produce (1) at least two valid arguments—based on reasons and/or evidence (Erduran et al., 2022)—and coherent—effectively supporting her/his claim (Archila et al., 2022b)—; (2) at least two valid and coherent counterarguments; and (3) at least one valid and coherent rebuttal while answering the question, “Why did you make that decision?”. This question is expressly recommended by Archila et al., (2019, 2021b) to prompt students to read critically. We would like to stress that the instructor made it very clear that there was no one right answer (e.g., very good, fair). In other words, whatever students decided, what was most important was the argumentation they constructed.

In the second step, participants were engaged in reciprocal peer critique (Topping, 2018). To this end, students were given access to a Google Sheet® in which each graded and provided anonymous useful feedback on the argumentation of two anonymous peers. This means that each student played the role of critics (giving feedback) as well as being criticized (receive feedback). Following Archila et al. (2022b), the instructor encouraged undergraduates to focus their criticisms on the quality of the argumentation of their peers and to avoid commenting “on trivial problems and errors (e.g., spelling)” (p. 2290). The importance of offering feedback respectfully, avoiding harmful comments was explained to them.

In the third step, students were provided with a final opportunity to read the Introduction section critically. Specifically, each undergraduate was asked to reflect upon the comments written by two peers in the previous step about her/his argumentation and to develop a (final) standpoint about the thought-provoking question mentioned in Step 1. It is worth adding here that students were free to put (or not) the feedback comments they received into practice, thus making Step 3 more about engaging in evaluative judgement practice and less about passive consumption of feedback.

With respect to the second session of our ALS, this addressed the Material and Methods section of Article 2. The instructor made it clear to the undergraduates that the goal of this session was to critically judge the information communicated in this section. In the first step, each student answered the thought-provoking question, “You consider that the quality of the Material and Methods section of Ayeni et al.’s (2011) article is…”; they could choose one of the following options: excellent, very good, good, fair, and poor. Moreover, students were asked to answer the question, “Why did you make that decision?”. They were required to construct at least two arguments, two counterarguments, and one rebuttal for their answer.

In the second step, students discussed in groups of three or four the decision made by each undergraduate in Step 1 with a view to making a group decision. In response to Aikin and Casey’s (2022) invitation to move from the disagreement-dependent argumentation (disagreement-only and disagreement-always) format to (dis)agreement-centered argumentation, the instructor encouraged students to interact argumentatively not only in possible dissensus situations but also in situations of agreement, since “one can give reasons, for example, to reinforce, maintain, or intensify someone’s agreement on a mutually shared proposition” (p. 3). In addition, each group was asked to co-construct at least two arguments, two counterarguments, and one rebuttal for their decision. Current research suggests that student–student argumentative interaction seems to be one possibility to foster productive co-construction of (counter) arguments (Archila et al., 2022c). Thus, this step offered an opportunity for students to experience an important feature of argumentation, namely, the fact that it can be co-constructed by different individuals.

In the third step, students were involved in reciprocal peer assessment practice. In other words, the argumentation of each group was criticized by two other groups, and members had to discuss and interchange impressions. Each group completed a Google Sheet®, grading, and giving anonymous useful feedback on the argumentation of two anonymous groups. This step was intended to enable students to practice their small peer assessment-group discussion skills (e.g., to articulate and evaluate their ideas). Finally, in the fourth step, undergraduates had a final opportunity to critically read the Material and Methods section. Each student was required to think about the written feedback produced by the two small groups in Step 3 about the argumentation of her/his group and to individually construct a (final) view about the thought-provoking question they answered in Step 1. Each student was encouraged to adopt an attitude based on evaluative judgement, while reflecting on the feedback comments her/his group had received earlier. It should be pointed out that the inclusion of a “Group decision” step (Step 2 in Fig. 1) in the second session of our ALS explains why this is a 4-step session while the first session is a 3-step session.

5 Research Design and Method

To demonstrate the extent to which the ALS engaged participants in the critical reading of scientific papers, the present study used a mixed methods approach (Mertens, 2023), collecting and interpreting data by means of quantitative and qualitative measures. As such, we expected to provide empirical evidence to support the claim that the ALS, which combines argumentation, peer critique, and the TORI (Ritchey & List, 2022), can be a rational and reasonable possibility for engaging students in the critical reading of scientific papers. Specifically, quantitative analyses were based on frequency counts, mean, standard deviation, and effect size calculations. On the other hand, the qualitative analyses involved a coding process of the participants’ responses to open-ended questions.

5.1 Context and Participants

The ALS reported in this article was implemented in a university bilingual (Spanish–English) science course—a type of course in which two languages are used and treated as valuable resources for science teaching and learning (Archila & Truscott de Mejía, 2020a)—called Food Microbiology. A common translanguaging practice (Mazak & Herbas-Donoso, 2015) in this course is to assign readings in English and discuss them in Spanish in order to cultivate student bilingual scientific literacy—scientific literacy in two languages (Airey & Linder, 2011). Moreover, one educational aim of the Food Microbiology course is that students would be able to critically read primary scientific literature. It is important to clarify that this course was chosen through convenience sampling—“a sample that is selected because of its availability to the researcher” (Clark et al., 2021, p. 606), as the second author was one of the two course instructors. The course is offered every semester by the Department of Chemical and Food Engineering of a major university in Bogotá, Colombia. This small-enrollment course (20–40 students per semester) serves undergraduate students from diverse academic programs such as, Biology, Biomedical engineering, Chemical Engineering, Food Engineering, and Microbiology. Usually, students enrolled in this course have completed about 60% of their undergraduate studies.

The study was designed with due consideration to the ethical guidelines of the American Psychological Association (APA 2017), and approval was granted from the University’s Research Ethics Committee before starting. Among the 65 eligible students, 63 (96.9%) consented to participate. However, two students did not fully participate in at least one of the two sessions of the ALS. Therefore, they were excluded from the analysis, resulting in a total sample size of 61 undergraduates (38 females and 23 males), aged 19 to 25 years old (M_age = 20.9, SD_age = 1.55). The first language of the students was Spanish. All participants were fully informed about the purpose and the voluntary nature of the study. It was made clear to them that participation was completely anonymous and that they were free to withdraw at any time. Written informed consent was obtained from all participants. In addition, all information collected was treated confidentially, and students were assigned codes, for example, 1U28 means Class 1, undergraduate number 28.

These 61 participants were grouped into two classes with the ALS carried out in the following order:

Class 1: Undergraduates taking Food Microbiology during the spring semester (average age 21.1 years), 33 students (21 females and 12 males).

Class 2: Undergraduates taking Food Microbiology during the fall semester (average age 20.6 years), 28 students (17 females and 11 males).

It is important to clarify that in Class 1 as well as in Class 2, science students and engineering students studied together. We decided to implement the ALS in two classes to have a larger sample size, but it is beyond the scope of this paper to examine the similarities and differences between these classes.

5.2 The Role of the Instructors

Instructors play an essential role in making active learning practices a reality rather than mere rhetoric. Of course, as Wenzel et al. (2022) emphasize, this implies that instructors should relinquish their outmoded role as transmitters of disciplinary knowledge and be convinced of the potential benefits of active learning (e.g., productive student learning experience). In our case, the two instructors who were responsible for the Food Microbiology course participated in the creation of the ALS and were wholly committed to implementing the scenario. Although the instructors were not experts on how to foster critical reading, they were aware of the importance of promoting this skill and spontaneously used to ask students not only to identify key features of scientific articles but also to read between the lines. The constructive participation of these instructors in the creation of the scenario helped us to propose a concrete, pragmatic, and realistic ALS. In the implementation of the scenario, they played the role of facilitators, giving room for undergraduates to assume responsibility for their learning. The sole function of these instructors was to encourage the undergraduates and engage them in the critical reading of the two scientific articles. While doing this, they assumed a neutral attitude without giving their impressions as much as possible and taking care not to use their authority to influence students’ decisions. “Some students may lack the confidence to critique papers […] because they are unsure that they understand the work presented” (Muench, 2000, p. 256). With this in mind, the instructors created a symmetric instructor-student classroom atmosphere to help students feel free to ask what was confusing for them.

5.3 Data Collection

As already mentioned, the ALS consists of two sessions (Fig. 1). The data corpus of this study was derived from the written responses of the 61 participants. To be precise, students answered one questionnaire per session (Questionnaire 1, Session 1 in Appendix 1; Questionnaire 2, Session 2 in Appendix 2). These questionnaires were distributed at the beginning of each session. Questionnaire 1 contained four questions presented in two parts. The students answered the same questionnaire individually in both parts. A small-group decision section was included in Questionnaire 2 (Part two in Appendix 2). Each student had a copy of Articles 1 and 2 which s/he could refer to during each session. Article 1 had a length of ∼ 8,000 words while Article 2 was about 6000 words long. The Flesch Reading Ease Score (Flesch, 1948)—a score that ranges from zero (very difficult to read) to 100 (very easy to read)—for the Introduction section (∼ 900 words) of Article 1 and the Material and Methods section (∼ 1500 words) of Article 2 were 20.8 and 17.1, respectively. This means that these sections were “very difficult” to read. Values between zero and 30 are typical of specialized texts such as scientific articles. The Flesch Reading Ease Score was calculated via the Microsoft Word® software (Stockmeyer, 2009).

Additionally, a Google Sheet® was used to collect the anonymous grading and the written peer critique produced by the students individually (Session 1, Step 2 in Fig. 1) and in small groups (Session 2, Step 3 in Fig. 1) about the argumentation (arguments, counterarguments, and rebuttals) of peers. Clearly, anonymity facilitates a willingness in students to communicate their critiques. The instructors made sure students understood how to use the assessment criteria (Table 1) and had as much time as they needed. Importantly, they asked students to assume a respectful attitude, avoiding comments that could be harmful. Likewise, online peer assessment offered us a major advantage, namely, the possibility of automatically recording data (Lu & Law, 2012, cited in Topping, 2018). It is worth noting here that in the Food Microbiology course, students are regularly (1) provided with opportunities to produce (counter) arguments and rebuttals and (2) involved in peer assessment practices as part of the creation process of argument maps (for detailed examples from this course, see Archila et al., 2022b, 2022d) and scientific argument podcasts (for detailed examples from this course, see Archila et al., 2023c). Therefore, participants were familiar with argumentation and peer assessment practices. Another point to clarify here is that students received written feedback from the instructors after Session 2 of the ALS; however, this feedback was communicated to the students after the data collection period to avoid any influence of instructor feedback on the results of our study.

Table 1 Rubric used in the scoring of the students’ responses to open-ended questions of Questionnaires 1 and 2 (taken from Archila et al., 2022b, p. 2292)

Finally, as in previous studies with undergraduates (Bennett & Taubman, 2013 (N = 20); Oliver, 2022 (N = 34)), students were asked to give us feedback for future improvements since we considered that our ALS was an unfinished and open educational resource that should be continually refined. Thus, at the end of each session, we administered a voluntary and anonymous online survey (Session 1, Survey 1 in Appendix 3; Session 2, Survey 2 in Appendix 4). The items were adapted from intervention feedback surveys developed by Archila et al., (2021a, 2021b, 2021c), Bennett and Taubman (2013), and Oliver (2022). This adaptation did not change the purpose of the items as this was solely nominal. For example, the question, “Was the small-group debate useful for you to make a decision?” (Archila et al., 2021b, p. 288), was adapted to “Were the opportunities of giving written feedback to two peers useful for you to make a decision?” (Question 3 in Appendix 3). Fifty-six out of the 61 participants (32/33 in class 1 and 24/28 in class 2) answered Survey 1, while 47 students (26/33 in class 1 and 21/28 in class 2) answered the second survey.

5.4 Data Analysis

In response to our research question, “To what extent does the ALS engage science (Biology and Microbiology) students and engineering (Biomedical engineering, Chemical Engineering, and Food Engineering) students in the critical reading of scientific papers assigned in a Food Microbiology course?” the responses to closed-ended questions of Questionnaires 1 (Questions 1 and 3 in Appendix 1) and 2 (Questions 1, 3, and 5 in Appendix 2) were placed on a rating scale range of frequency: “poor (1),” “fair (2),” “good (3),” “very good (4),” or “excellent (5).” Note that this scale was presented to the students in both questionnaires. Our analysis continued by coding (scoring) the participants’ responses to open-ended questions in the questionnaires (Questions 2 and 4 in Appendix 1; Questions 2, 4, and 6 in Appendix 2). This was carried out according to four criteria as described below in Table 1. For example, the following is the argumentation produced by 1U28 as response to Question 2 of the Questionnaire 2 and coded with the maximum score: six.

[Claim]: I consider that the Material and Methods section is good. [Argument 1]: A valuable feature of this section is that the authors mention the primers (Y1–Y2) they used in the partial sequencing of the ribosomal subunit 16 (16S rRNA) to identify the microorganisms. This contributes to the replicability of the experiments. [Counterargument 1]: However, in this section the sequence of the primers is not reported. These primers are crucial for the identification of the microorganisms. Even though the primers are reported in external sources, the sequence used must be detailed to avoid confusion or factory changes of the primers. Moreover, providing this information would benefit the replicability of the experiments.

[Argument 2]: In addition, I find it very useful that the authors indicate the commercial kits they used to determine the minimal inhibitory concentration. In this case, they used VetMicTM Lact-I microdilution tests to determine the MIC [referring to minimal inhibitory concentration] to some aminoglycosides, lincosamides, tetracyclines, and amphenicols. [Argument 3]: Also, it is very well described how the authors used hand-made plates to determine the MIC to other antibiotics such as ampicillin, nitrofurantoin, T-S [referring to trimethoprim–sulfamethoxazole], and fosfomycin. [Counterargument 2]: Yet, the use of hand-made plates can result in a technical difficulty to ensure the replicability of the experiments. This is not a problem for the methods that involved standard identification kits. The use of hand-made plates can influence the results of the study since the quality of this type of plates may be affected by several conditions (human errors, measurement errors, reactants of poor quality, etc.). [Rebuttal]: Despite this, I feel that the hand-made plates protocol is well explained throughout the subsection section, “Antibiotic resistance pattern”. This allows other researchers to replicate the protocol used by the authors (1U28).

In our coding process, we assumed that a (counter) argument or rebuttal is valid when it is supported by reason and/or evidence (Erduran et al., 2022) while its coherence depends on the fact that it effectively defends a claim (argument), opposes an argument (counterargument), or rebuts a counterargument (rebuttal) (Archila et al., 2022b). The process of coding was carried out in a collaborative way (Saldana, 2021). The first and second authors worked independently to code the students’ responses and they then got together to compare their results. The Kappa statistic was used to evaluate the reliability of this process. Cohen’s kappa coefficient (Cohen, 1960) was 0.95 for Question 2 of Questionnaire 1, 0.99 for Question 4 of Questionnaire 1, 0.90 for Question 2 of Questionnaire 2, 0.95 for Question 4 of Questionnaire 2, and 0.96 for Question 6 of Questionnaire 2. According to Bryman (2016), “a coefficient of 0.75 or above is considered very good” (p. 276) inter-coder agreement. We used the Statistical Package for the Social Sciences (SPSS®, v28) for all statistical calculations. Any disagreements were discussed, and a consensus score was reached for all participants. Moreover, Cohen’s d effect size was calculated using students’ argumentation of the “Initial decision” (Question 2 in Appendixes 1 and 2) as the “control” condition and argumentation of the “Final decision” (Questions 4 and 6 in Appendixes 1 and 2, respectively) was treated as the “experimental” condition. We adopted the benchmarks for interpreting effect size proposed by Cohen (1988, pp. 25–26): small (d = 0.2), medium (d = 0.5), and large (d = 0.8). It should be pointed out that this control-experimental assumption is a legitimate option in cases in which a control group is not available to researchers (Creswell & Creswell, 2018; Ferron et al., 2023; Tanious & Onghena, 2021). Thus, effect size can be calculated using both the responses of the students at the beginning of the intervention (control condition) and after this (experimental condition) (Archila et al., 2021b, 2023a, 2023b; Meli et al., 2022).

Additionally, the first and the second author coded the participants’ responses to Questions 4 and 6 of Questionnaires 1 and 2, respectively. This coding process was based on two criteria presented in Table 2. Wu and Schunn (2023) maintain that engagement with peer feedback is a key element of authentic assessment of learning practices. Thus, the purpose of this coding was to get an idea of the type of engagement (passive or active) that occurred in our ALS. To be clear, “the MSWord Compare Documents function was utilized to identify revisions” (Wu & Schunn, 2023, p. 6). The revisions identified then followed the coding process which was guided by a single passive-active coding method. Cohen’s kappa coefficient calculated was 0.87 for Question 4 of Questionnaire 1 and 0.84 for Question 6 of Questionnaire 2. Where there were differences, it was deemed necessary to re-examine the participants’ responses and then to discuss these until a consensus was reached.

Table 2 Rubric used in the coding of the students’ responses to Question 4 of Questionnaire 1 and Question 6 of Questionnaire 2 (inspired by the work of Wu & Schunn, 2023)

In order to get an idea of the quality of the feedback given by the students, feedback in writing produced individually (Session 1, Step 2 in Fig. 1) and in small groups (Session 2, Step 3 in Fig. 1) was coded. Based on the coding rubric (Table 3) proposed by Archila et al., (2022b, p. 2294), the first and the second author carried out this coding process independently and attained a Cohen’s kappa coefficient of 0.87 and 0.91 for individual written peer feedback and small-group written peer feedback, respectively. Where there were disagreements, these were resolved by discussion. Finally, we used frequency counts to analyze participants’ responses to the intervention feedback surveys. Some answers to open-ended questions (Questions 3 and 4 in Appendix 3; Questions 2 and 3 in Appendix 4) are commented on in the next section.

Table 3 Rubric used in the coding process of the written peer feedback (adapted from Archila et al., 2022b)

6 Results

We report the findings of the implementation of the ALS in the following three sections; the first section is about students’ judgements of the quality of specific parts of Articles 1 and 2; the second section focuses on participants’ production of arguments, counterarguments, and rebuttals; and the third section addresses peer feedback. Throughout these three sections, we present the results of the intervention feedback surveys to offer a deeper understanding of the contribution of our study.

6.1 Participants’ Judgements of the Quality of Specific Sections of Articles 1 and 2

In our ALS, the participating students were asked to judge the quality of the Introduction section of Article 1 as well as the Material and Methods section of Article 2. Table 4 shows the students’ average judgements (scores) along with the standard deviations. The maximum and the minimum possible average scores for each stage of judgement (e.g., initial individual judgement) were 5 (excellent) and 1 (poor), respectively. In general, the results indicate that students considered that the quality of these sections was very good/good. Although these outcomes provide a valuable overview of the students’ judgements, one might ask, were undergraduates effectively engaged in critical reading? Clearly, to make a judgement does not necessarily mean to become engaged in critical reading when this judgement is not accompanied by sound argumentation.

Table 4 Students’ judgements of the quality of specific sections of Articles 1 and 2

6.2 Participants’ Argumentation

Results suggest that students effectively became engaged in critical reading during our ALS. As demonstrated by Table 5, students accompanied their judgement by valid and coherent arguments, counterarguments, and rebuttals. The fact that all the participants’ average scores were higher than 4.90/6 can be seen as a positive result that should be interpreted in the light of five aspects. First, as part of the Food Microbiology course, students received instruction about how to read and critique scientific articles. Second, a large fraction of the undergraduates (27/32 in class 1; 22/24 in class 2) who answered Survey 1 had received instruction in critical reading before taking this course (Question 1 in Appendix 3). Third, students were provided with strategies to help them meet the reading goals (Fig. 1). Fourth, before Sessions 1 and 2, undergraduates were given 3 weeks to read the two assigned articles. It is important to note that the great majority of the students considered that they had had sufficient time for reading Articles 1 (32/32 in class 1; 24/24 in class 2; Question 2 in Appendix 3) and 2 (24/26 in class 1; 21/21 in class 2; Question 1 in Appendix 4). And fifth, in the Food Microbiology course, undergraduates were regularly given explicit opportunities to produce (counter) arguments and rebuttals while creating argument maps and scientific argument podcasts.

Table 5 Means, standard deviations, and total scores of the argumentation produced per session

These aspects largely explain why the participants’ average scores were higher than 4.90 even in the initial individual judgement. Given this situation it makes sense to have found only small (d = 0.32) and medium (d = 0.50) effect sizes (Table 5) when initial individual judgements were compared with final individual judgements. In other words, the students had not only had training in the production of (counter) arguments and rebuttals before they started Session 1 but also training in critical reading. Additionally, they had sufficient time for reading. Clearly, the contribution of our study is that it is the first in which the production of (counter) arguments and rebuttals is used as a platform to engage students in the critical reading of scientific articles. By the same token, Table 5 indicates that students went beyond the passive absorption of information; they actively engaged with Articles 1 and 2.

To illustrate active engagement in critical reading, consider the following argumentations produced by 2U7 and 2U20 as response to Question 4 of the Questionnaire 1:

[Claim]: I think that this introduction is good, [Argument 1]: as initially there is a brief explanation of the mixtures involved in the preparation of ice cream, as well as how these are used. This is helpful in order to be able to understand in more detail the reasons why this food is one of the most affected by the action of altering microorganisms. [Counterargument 1]: However, one of the shortcomings regarding this is that there is no adequate conceptualization of the Zygosaccharomyces rouxii microorganism because this is only seen as one of the most common osmophilic yeasts which can be found in food with high concentrations of sugar. [Rebuttal]: In spite of this, the introduction refers to the most important techniques and strategies used to control and slow the growth of osmophilic yeast, and this is very relevant to be able to understand the rest of the article.

[Argument 2]: On the other hand, the introduction is quite coherent, showing that the aim is clear in relation to the existing problem and the methods which are proposed to solve it. [Counterargument 2]: Nevertheless, these clarifications are short and only found at the end of the introduction. From my point of view, this is not sufficient, as the reader can lose interest in these important details (2U7).

[Claim]: I consider that the introduction is good [Argument 1]: because it contextualizes the topic to be developed and its importance, as well as clarifying the issue of interest. First of all, there is a definition of MBICs at the beginning of the introduction and a focus on important aspects such as the fact that these are “intermediates for the production of artisanal or industrial ice creams and desserts” (Iacumin et al., 2022), which helps to identify their role in the frozen food industry. [Counterargument 1]: However, in the contextualization section, there is no mention of information or numbers that help to better establish the importance of this topic in industry, or to position information at national or international level in order to improve the understanding of this issue.

[Argument 2]: Besides, [in the introduction section] the issue of interest is clarified by explaining that “the high number of organic compounds contained in these preparations favors contamination and, in some cases, even the development of a rich yeast population” (Iacumin et al., 2022). Moreover, there is an indication of the conditions which favor their growth and the type of yeasts which can develop at the base of the ice creams. These are osmotolerant and osmophilic yeasts. In addition, there is a connection with the yeast of interest – the Zygosaccharomyces rouxii. [Counterargument 2]: Nevertheless, there is no mention of the effect of the yeast at the base of ice cream and which is very important to understand the seriousness of the problem. [Rebuttal]: In any case, there is some information about the effect of yeasts in general in this type of substratum, which helps to give an idea (2U20).

It is encouraging to see that the arguments [Arguments 1 and 2] produced by 2U7 and 2U20 demonstrate that they read the introduction section of Iacumin et al.’s (2022) article with a purpose in mind: to produce arguments to support their decision that the introduction was good. In particular, it is interesting to note that 2U20 used citations in both arguments to make clearer what s/he was referring to. This indicates her/his valuable commitment to navigate the ideas of this section. Likewise, the counterarguments communicated by these students suggest that they strived to take into consideration alternative views. The importance of this result is that this shows that 2U7 and 2U20 were able to object their own arguments. The relevance of this is even more evident once we recognize that the “task [of anticipating counterarguments] is generally difficult” (Mercier, 2016, p. 691). Active engagement in critical reading is also evidenced in the fact that 2U7 and 2U20 went further in the production of arguments and the anticipation of counterarguments; each generated one valid and coherent rebuttal as opposition to one of the counterarguments. One aspect to comment on the rebuttals produced by these students is that both focused on specific and desirable features of an introduction section (e.g., “the most important techniques and strategies […] relevant to be able to understand the rest of the article” (2U7) and “some information […] to give an idea” (2U20)). This indicates that they seem to have understood what it is to be expected from the introduction section of a research article. In short, the critical argumentation (argument, counterargument, and rebuttal) produced by these participants illustrates a concrete and realistic possibility for going beyond a passive and surface reading to read in a more active and more critical way.

6.3 Quality of the Written Peer Feedback

Peer critique is one of the pillars of our ALS (Fig. 1). The findings suggest that a relevant number of participants offered useful written peer feedback to their peers (Table 6). Topping (2018) invites us to embrace the idea that peer assessment is a valuable source of formative assessment. He argues that students demonstrate that they have developed an authentic understanding of the product subject of judgement when they are able to assume the role of peer assessors and provide useful peer feedback. Furthermore, we found that almost all the participants who answered Surveys 1 (29/32 in class 1; 20/24 in class 2; Question 3 in Appendix 3) and 2 (24/26 in class 1; 19/21 in class 2; Question 2 in Appendix 4), acknowledged that giving feedback to peers was useful for them to develop a judgement. Comments made in these surveys included: “I could compare my own text while reading the text of a peer, and thus I could realize my own weak points,” “being an assessor forced me to become more aware of what the other peer wrote. This helped me to dig deeper into my argumentation,” and “I could develop a broader judgement and not just focus on what I believed.” Furthermore, more than half of the respondents felt that receiving peer feedback was useful for them in making a decision ((Survey 1: 25/32 in class 1; 19/24 in class 2; Question 4 in Appendix 3) (Survey 2: (19/26 in class 1; 19/21 in class 2; Question 3 in Appendix 4)). Some of their reasons include the following: “It was good to receive peer feedback as this helped me to revise my judgements,” “the comments that I received led me to identify what I had to improve,” and “this feedback helped me to analyze my decision and clarify my argumentation in more detail.”

Table 6 Quality of the written peer feedback

Finally, the number of participants who were actively engaged with peer feedback is higher than those who were passively engaged (Table 7). As Wu and Schunn (2023) point out, providing students with opportunities to (1) offer, (2) receive, and (3) reflect on peer feedback is an important but (still in the twenty-first century) neglected element of constructive learning experiences. Therefore, the inclusion of peer critique in our ALS is a legitimate and desirable contribution if we take into account that the results of the surveys revealed that many of the respondents never or infrequently had the opportunity to give written peer individual feedback (26/32 in class 1; 21/24 in class 2; Question 5 in Appendix 3) or written peer group feedback (22/26 in class 1; 18/21 in class 2; Question 4 in Appendix 4) in other university courses.

Table 7 Types of engagements with peer feedback

7 Discussion

In 1983, Beverley Bell insisted that promoting student active engagement with scientific texts should be an imperative within science education. The problem is that 40 years later, instructor-centered lecture formats dominate science teaching and learning in too many universities around the globe. Recently, Idsardi et al. (2023) stressed that institutions of higher education should invest more efforts and resources in switching from instructor-centered learning (passive learning) to student-centered learning (active learning). In this regard, Hubbard (2021) and Hubbard et al. (2022) observe that instructors commonly ask students to read assigned scientific articles, but rarely engage students in the critical reading of these articles. In addition, students tend to hold the limited view that reading is necessary solely to earn good grades (Marchant, 2002; Theriault, 2022), overlooking its contributions to several scientific domains (e.g., authentic inquiry) (Xiang, 2022). The TORI proposed by Ritchey and List (2022) is in essence an invitation for instructors to go beyond the mere role of reading assigners. Argumentation skills are intimately connected and fundamental to critically read scientific articles. Unfortunately, one consequence of the hegemony of lecture-based instructional practices is that “as we begin the third decade of the twenty-first century, argument and debate are not habitual practices of university science education” (Archila et al., 2022c, p. 236). Another consequence is that lecture-based instruction does not encourage peer assessment (Winstone & Carless, 2020). For all the reasons just mentioned, here, we provide evidence for the claim that an ALS combining the TORI, argumentation, peer assessment can be a concrete and realistic possibility for engaging students in the critical reading of scientific articles. In what follows, we discuss the results in terms in which they answer our research question, “To what extent does the ALS engage science (Biology and Microbiology) students and engineering (Biomedical engineering, Chemical Engineering, and Food Engineering) students in the critical reading of scientific papers assigned in a Food Microbiology course?”.

The great majority of the participating students became engaged in the critical reading of the Introduction section of Article 1 as well as the Material and Methods section of Article 2. To be clear, our results seem to indicate that most students produced valid and coherent argumentation while judging the quality of these sections (Table 5). As previously mentioned, students were asked to make a decision about the quality of each section (excellent, very good, good, fair, and poor). This is a thought-provoking (and ambiguous) task. Much of the reason for this is that there is no one right answer unless we have provided students with quite specific criteria to, for example, distinguish between “very good” and “good.” And we did not do so. We left it purposely ambiguous. This ambiguity encouraged students to prioritize the production of valid and coherent (counter) arguments and rebuttals over making a “right” decision. In other words, students were challenged to accompany their judgement by sound argumentation. This is in line with the claim that presenting students with thought-provoking and ambiguous questions is a legitimate means to engage them in genuine argumentation practices (Archila, 2015; Archila et al., 2021c, 2022e).

The ALS effectively provided students with explicit opportunities to produce arguments, counterarguments, and rebuttals. This is particularly valuable since “arguments containing rebuttals are thought to be of the highest quality, as they require the ability to compare, contrast and distinguish different lines of reasoning” (Osborne, 2010, p. 464). Recently, Erduran et al. (2022) reported that students are seldom encouraged to anticipate counterarguments, considering alternatives to their own arguments. Using the production of arguments, counterarguments, and rebuttals as a platform to engage undergraduates in the critical reading of scientific journal articles is an under-researched possibility. Earlier studies have focused on giving students explicit opportunities to identify components of the authors’ argumentation (e.g., supports, counterarguments, refutations) (e.g., Lammers et al., 2019; Van Lacum et al., 2014, 2016); Wijayanti & Adi, 2022).

Also, our findings appear to reinforce the idea that the TORI can be applied in a relatively simple way and in any course and discipline (Ritchey & List, 2022). This idea is important due to the fact that nowadays the use of primary scientific literature is gaining attention in undergraduate science courses (Verkade & Lim, 2016). Essentially, this theoretically based framework helped us to give shape to our ALS. It may be obvious to point out that (1) stating explicitly the goals of assigned readings and (2) providing students with strategies to help them meet these goals are key conditions for productive reading. Nonetheless, as Hubbard (2021) and Hubbard et al. (2022) note, this rarely happens at the tertiary education level. One reason for this is that instructors commonly (prefer to) assume that students have the skills to be able to critically engage with the assigned readings. As Kerr and Frese (2017) suggest, instructors should guide students to understand the rational for reading (e.g., communicate the goals of reading) and how to capitalize effectively on reading assignments.

In our ALS, given that we followed the TORI framework, the two instructors not only made clear to the students what the goals of reading were but also equipped them with practical and functional strategies (e.g., Before reading, identify what prime characteristics it is expected to find in the Introduction section) (Fig. 1). Likewise, in Sessions 1 and 2 of the ALS, these instructors integrated “empathy” (Hubbard, 2021, p. 60) in their practices, creating a symmetric instructor-student classroom atmosphere which allowed them to “act[ing] as reading mentors rather than expect[ing] students to read independently” (Hubbard, 2021, p. 60). Arguably, the fact that we adopted an active learning approach instead of an instructor-centered lecture format facilitated this type of classroom atmosphere.

Additionally, the TORI highlights the importance of linking reading goals to assessment practices (Ritchey & List, 2022). In the ALS, we included peer assessment. The outcomes indicate that our scenario provided students with concrete opportunities to (1) give, (2) receive, and (3) reflect on written peer feedback (Table 6 and 7). Therefore, it is plausible to claim that our ALS contributes to the construction of responses to the call of Wu and Schunn (2023) for the creation of activities in which students have opportunities to receive comments from their peers about their work, provide constructive comments while reviewing and criticizing the work of others, and think about peer feedback. Furthermore, our results seem to corroborate Topping’s (2018) view that there is a wide battery of products that can be used to engage students in peer feedback assessment practices. To the best of our knowledge, this study is the first to combine critical reading of scientific articles, argumentation, and peer critique in the same scenario. It is worth adding here that the majority of the participants confirmed that they had never or had infrequently had the opportunity to offer written peer individual feedback or written peer group feedback in other university courses. This is why we agree with Campbell and Batista (2023) and Noroozi et al. (2023) that much work remains to be done to consolidate peer feedback as a recurrent educational practice in institutions of higher education.

Finally, taken together, the findings appear to suggest that our ALS is one legitimate and desirable possibility to move away from passive (and information absorption-based) learning views of reading to active learning practices of critical reading. The ALS is an original contribution as this integrates argumentation, peer critique, and the TORI framework (Ritchey & List, 2022). Thus, our results expand the literature on the use of primary scientific literature as an educational resource (Chatzikyriakidou & McCartney, 2022; Chatzikyriakidou et al., 2021, 2022; Griffiths & Davila, 2022; Hoskins et al., 2007; Hunter & Kovarik, 2022; Lee et al., 2022; Muench, 2000; Palavalli-Nettimi et al., 2022; Smith & Paradise, 2022) by adding evidence supporting the idea that university students should be provided with a variety of opportunities to cultivate their critical reading skills. Likewise, this study sheds light on the importance of taking advantage of active learning principles to involve students in activities that require the application of discrete academic reading skills while engaging with scientific journal articles (Bennett & Taubman, 2013; Bogucka & Wood, 2009; Heiss & Liu, 2022). In our case, active learning occurred in the form of argumentation and peer critique practices. As Mizokami (2018) reminds us, active learning becomes a reality when students are engaged in activities where they do (e.g., critique the work of peers) and/or produce something (e.g., arguments, counterarguments, rebuttals), as well as cultivate their higher-order thinking skills (e.g., argumentation), and critically reflect on what they are doing and/or producing.

8 Conclusions

Students should be provided with opportunities to not only understand scientific articles but also to read them critically (Raimondi et al., 2020). Accordingly, the research question of our study revolved around the extent to which the ALS engaged undergraduate students in the critical reading of scientific journal articles. In the light of the above findings and discussion, one overarching conclusion that emerges from our research is that the ALS seems to be an original educational tool that offers promising potential for cultivating critical reading of scientific articles since the merits of this scenario are supported by the fact that the majority of the participating students became engaged in the critical reading of this type of articles. Another conclusion is that the integration of argumentation, peer critique, and the TORI framework (Ritchey & List, 2022) is effectively a concrete, realistic, and innovative possibility for providing students with explicit opportunities to practice critical reading skills.

9 Limitations and Future Directions

Despite the utility of the results, there are also various limitations which should be considered when interpreting these findings. First, one noticeable shortcoming of our study is the small number of participating students. This seriously limits the quality of our mixed method procedure since it does not allow us to make evidence-based generalizations of the eventual benefits of the ALS. Undeniably, had the number of students been larger, we would have been able to provide more robust research evidence relating to the benefits of our scenario. A second limitation is that this study was carried out in just one university science course with students from different science and engineering majors (e.g., food engineering, microbiology). This shows bias in our research design since we did not implement the ALS with a wider diversity of majors (e.g., law, psychology). Besides, many of the participating students, apart from the Food Microbiology course, had previously received instruction about how to read and critique scientific articles. Additionally, argumentation and peer assessment are common educational practices of this course. It must be admitted that these contextual factors could have largely influenced the promising results of our ALS. It would therefore be interesting to implement our scenario in other university courses in order to corroborate the findings reported in this article. Likewise, we did not collect data related to participants’ prior experience with other critical reading instructional initiatives, academic level, and prior content knowledge, which may be factors that influence our outcomes. Another main limitation of this study is that although undergraduates were given three weeks to read Articles 1 and 2, the two sessions of the ALS focused only on two specific sections of these articles, namely, the Introduction and the Material and Methods sections. Arguably, this limitation could be seen as a strength, if we acknowledge the positive experiences documented by Bogucka and Wood (2009), Hunter and Kovarik (2022), Spiegelberg (2014), and Vroom (2022), while implementing reading activities that required students to be focused just on quite specific parts of scientific journal articles.

It is our hope that the results presented here can contribute to enrich the global discussion of how to engage students in the critical reading of research articles. This is a discussion that is particularly motivated by the complex issue that students are often not reading the assigned texts (Gorzycki et al., 2019; Oliver, 2022; Sutherland & Incera, 2021) and instructors seldom integrate academic reading pedagogies into their courses (Desa et al., 2020). The situation is even more complex in countries where English is the second or foreign language since reading in this language can often constitute an extra challenge for some students (Archila & Truscott de Mejía, 2020b). Of course, whatever means used to promote critical reading will be unproductive if students do not understand or complete their assigned texts. With this in mind, it should be pointed out that although our ALS seems to offer valuable benefits for both students (e.g., explicit opportunities to go beyond a superficial reading of research papers to critically engage with these readings) and instructors (e.g., pragmatic ways to move away from the passive learning view of reading towards more active learning practices), this does not mean that this scenario is finished. Put simply, we created this scenario as an unfinished and open possibility for instructors interested in cultivating students’ critical reading skills. Hence, further work on how to integrate this ALS with other active learning-based initiatives could be a particularly relevant research direction. For example, Gomez-Marin (2023) nowadays maintains that podcasts are becoming increasingly a strategic ally of scientific communication. It would therefore be interesting to explore the articulation of the ALS with the idea of involving undergraduates in the creation of podcast episodes in which they communicate the research outcomes of a piece of primary research literature with a general audience (Palavalli-Nettimi et al., 2022). Such articulation would result in students developing and producing scientific argument podcasts in which they communicate their judgements (arguments, counterarguments, rebuttals) of the quality of a scientific article.

An essential issue in the future will be to compare the ALS to other instructional initiatives (e.g., CREATE (Hoskins et al., 2007)) and/or articulate our scenario with those initiatives. This could provide more clarity about the impact of the ALS as well as the ways how this scenario can benefit from the utilities of these initiatives and vice versa. Moreover, it would be relevant to study how to use the ALS as one possibility to prepare students to critically engage with multiple reading activities such as the search and selection of original research articles (Russo & Jankowski, 2023), the reading of annotated primary scientific literature (Kararo & McCartney, 2019), and participation in academic reading seminars (Afdal et al., 2023) and virtual journal clubs (Stengel et al., 2021). Furthermore, future research could involve developing specific active learning-based strategies that foster student skills to critically read fraudulent academic articles (Pflugfelder, 2022) as well as news articles that present people with false and/or inaccurate scientific information (Archila et al., 2019, 2021b). Clearly, there is much work to be done and the challenge and the need for future studies is increased by the fact that “the development of students’ critical reading skills is an important and urgent issue facing institutions of higher education today” (Sutherland & Incera, 2021, p. 267).

Appendices

Appendix 1. Questionnaire 1

1.1 Part One: Initial Decision

• 1. You consider that the quality of the Introduction section of Iacumin et al.’s (2022) article is…

  • a. Excellent.

  • b. Very Good.

  • c. Good.

  • d. Fair.

  • e. Poor.

• 2. Why did you make that decision? Provide at least two arguments, two counterarguments, and one rebuttal for your answer.

1.2 Part Two: Final Decision

• 3. Having reflected upon the written feedback provided by two peers, you consider that the quality of the Introduction section of Iacumin et al.’s (2022) article is…

  • a. Excellent.

  • b. Very Good.

  • c. Good.

  • d. Fair.

  • e. Poor.

• 4. Why did you make that decision? Provide at least two arguments, two counterarguments, and one rebuttal for your answer.

Appendix 2. Questionnaire 2

2.1 Part One: Initial Decision

• 1. You consider that the quality of the Material and Methods section of Ayeni et al.’s (2011) article is…

  • a. Excellent.

  • b. Very Good.

  • c. Good.

  • d. Fair.

  • e Poor.

• 2. Why did you make that decision? Provide at least two arguments, two counterarguments, and one rebuttal for your answer.

2.2 Part Two: Small-group Decision

• 3. Your group consider that the quality of the Material and Methods section of Ayeni et al.’s (2011) article is…

  • a. Excellent.

  • b. Very Good.

  • c. Good.

  • d. Fair.

  • e. Poor.

• 4. Why did your group make that decision? Provide at least two arguments, two counterarguments, and one rebuttal for your answer.

2.3 Part Three: Final Decision

• 5. Having reflected upon the written feedback provided by two small groups, you consider that the quality of the Material and Methods section of Ayeni et al.’s (2011) article is…

  • a. Excellent.

  • b. Very Good.

  • c. Good.

  • d. Fair.

  • e. Poor.

• 6. Why did you make that decision? Provide at least two arguments, two counterarguments, and one rebuttal for your answer.

Appendix 3. Survey 1

• 1. Apart from the Food Microbiology course, have you ever received instruction about how to read and critique scientific articles?

  • a. Yes.

  • b. No.

• 2. Did you have sufficient time to read Article 1?

  • a. Yes.

  • b. No.

• 3. Were the opportunities of giving written feedback to two peers useful for you to make a decision? Explain why or why not.

• 4. Were the written feedback provided by peers useful for you to make a decision? Explain why or why not.

• 5. How often do you have the opportunity to give written peer feedback in other university courses?

  • a. Very frequently.

  • b. Fairly frequently.

  • c. Infrequently.

  • d. Never.

Appendix 4. Survey 2

• 1. Did you have sufficient time to read Article 2?

  • a. Yes.

  • b. No.

• 2. Were the opportunities of providing written group feedback to two small groups useful for you to make a decision? Explain why or why not.

• 3. Was the written feedback given by small groups useful for you to make a decision? Explain why or why not.

• 4. How often do you have the opportunity to provide written peer group feedback in other university courses?

  • a. Very frequently.

  • b. Fairly frequently.

  • c. Infrequently.

  • d. Never.