Fostering noticing of classroom discussion features through analysis of contrasting cases

Blair, Kristen P.; Banes, Leslie C.; Martin, Lee

doi:10.1007/s11251-024-09661-z

Fostering noticing of classroom discussion features through analysis of contrasting cases

Original Research
Open access
Published: 17 April 2024

(2024)
Cite this article

Download PDF

You have full access to this open access article

Instructional Science Aims and scope Submit manuscript

Fostering noticing of classroom discussion features through analysis of contrasting cases

Download PDF

152 Accesses
Explore all metrics

Abstract

Productive classroom discussion has been shown to support student learning across academic domains. Facilitating successful discussion hinges on the teacher’s ability to make adept in-the-moment observations of various aspects of student talk and classroom dynamics. In two studies, we explore a pedagogical intervention using contrasting cases to support novice teachers in learning to notice key features of classroom discussion. Study 1 involves preservice teachers in a bilingual teaching methods course in a university-based credential program. Study 2 involves undergraduates in an education psychology course, many of whom are prospective teachers. Study participants engaged in analyzing transcript-based contrasting cases of discussion vignettes as they collaboratively developed guiding principles for effective class discussion. Data include pre- and post-instruction video noticing task reflections, principles identified, and transcribed partner discussions during the activity. Post-instruction, learners displayed increased student-centered noticing when watching videos of classroom discussions. Additionally, there was increased awareness of the absence of productive features or missed opportunities within the discourse. In this proof-of-concept set of studies, we explore the potential of contrasting cases-based activities to help prepare teachers for the complex task of orchestrating discussion by supporting them in learning to notice.

Examining Science Education in ChatGPT: An Exploratory Study of Generative Artificial Intelligence

Article Open access 22 March 2023

What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education

Article Open access 22 February 2023

Incorporating AI in foreign language education: An investigation into ChatGPT’s effect on foreign language learners

Article Open access 19 March 2024

Across disciplines, educators recognize productive class discussions as instrumental to learning (Gee, 2004; Lemke 1990), and studies over the last two decades have consistently demonstrated positive effects on student achievement (Banes et al., 2020; Mercer et al., 2004; Murphy et al., 2009). Despite this, productive discussion remains relatively rare in K-12 classrooms in the US, especially in schools characterized by high poverty rates and significant numbers of culturally and linguistically diverse learners (Gallimore et al., 2014; Pianta & Hamre, 2009). By productive discussion, we refer to the collaborative co-construction of knowledge among participants sharing responsibility over content and discourse (Reznitskaya, 2012). The quantity of student talk in a discussion is not enough to create a productive discussion. Classroom research has shown the quality of student talk is equally important (Smith & Stein, 2011; Truxaw & DeFranco, 2008). Orchestrating such high-quality student talk creates a high demand on teachers who must attend to several facets of classroom activity simultaneously (Walshaw & Anthony, 2008). Initiating and sustaining a productive discussion has been described as “among the most challenging activities for an instructor,” (Davis, 1993), in part, because classroom talk is unpredictable and contingent upon each talk turn shaping and reshaping discourse.

Within this complexity, one way to begin to understand the learning problem is to consider a simpler analog. At one level, the learning problem might be conceptualized as a matching problem, learning to appropriately pair inputs A, B, C and responses X, Y, Z. Teachers learn to map student speech acts that occur as the discussion unfolds (the inputs) to appropriate responses they might make. For example, if a student puts forward a partially-formed idea, the teacher might probe them to elaborate or provide evidence. The challenge is increased because this occurs not only at single utterance level, but at the level of larger talk patterns unfolding during the discussion. For example, if there is a pattern of “popcorning” of unrelated ideas, a teacher might implement a discussion move to encourage students to listen to and build on one another’s ideas. A further challenge is that the space of student inputs is infinite, and talk is not labeled with the kind of input it represents. Teachers need to learn to notice across several dimensions to evaluate features of the discussion and decide which, if any, response is appropriate to enact (Barnhart & van Es, 2015). In other words, they need to recognize A, B, C in context.

According to perceptual learning theorist Gibson (1969), learning to notice involves increased precision in recognizing the underlying structure of a phenomenon, such as a new surfer learning to ‘read’ the features of incoming waves to decide which to pursue and how to position themselves. With this increased precision comes the ability to differentiate situations that one might initially gloss as similar, such as two waves of the same height that might look the same to a beginner, but which peak in different locations, indicating important distinctions in how they will break to the experienced surfer. In the context of learning to notice student talk during a discussion, this might involve two situations where students are speaking, but the quality of talk patterns differs. Novice teachers might notice broadly that students are talking to each other, while more experienced teachers might have learned to differentiate whether there is equality in who is talking or whether the talk includes genuine uptake of ideas. More precise differentiation should enable more finely tuned responses (e.g., Bransford & Schwartz, 1999). While perceptual learning theories generally focus on increased precision in noticing what is present in the stimuli (e.g., Gibson, 1969), teachers also need to notice what is absent. For example, if a student’s idea is not acknowledged and taken up, this absence should be noticed and potentially acted on.

Successful mapping additionally requires teachers to develop a flexible repertoire of possible responses, such as talk moves, to draw upon in facilitating student talk - the X, Y, Zs in the analogy above (Michaels & O’Connor, 2015). This is not as straightforward as it sounds (e.g., Herbel-Eisenmann et al., 2009), and we posit that it also requires a form of learning to notice, in this case learning to differentiate features of the response moves themselves. For example, educators are taught about revoicing as a talk move they might use. Revoicing has been defined as “the reuttering of another person’s speech through repetition, expansion, rephrasing, and reporting” (Herbel-Eisenmann et al., 2009, p. 268). In practice, revoicing can take on many forms, from restating what the student said, to rephrasing in academic language, to including additional information not directly stated by the student. It can also serve many functions, from enabling the speaker to agree with or clarify a restatement, to helping students feel heard, to moving an idea along (Herbel-Eisenmann et al., 2009; O’Connor & Michaels, 2019). When observing model teachers enacting revoicing, or enacting it themselves, it is important for teachers to be able to differentiate revoicing from other kinds of talk moves and to notice the features of how it is being enacted in a given situation, which will influence the function it serves and the consequences it might have. This is true of the full space of possible response moves.

While useful for highlighting certain aspects of the learning problem, the simplified analogy of a matching problem does not capture other key aspects of learning to lead discussion. Among them is that discussion occurs in social and cultural contexts and is shaped by factors such as teacher and student values and relational histories at personal and societal levels (e.g., Hufferd-Ackles et al., 2004). For example, with respect to values, teachers often report needing to handle multiple goals that can require navigating tradeoffs, especially in classroom settings, where time is always limited. An example tradeoff is wanting to make space for and validate student interpretations of a literary text, while also wanting to ensure students are exposed to and understand more canonical interpretations (Athanases & Sanchez, 2019). These values may not be surfaced for the teacher, and we posit that a third form of noticing required is for teachers to learn to identify and refine their understanding of their own values and priorities and whether a discussion is unfolding in ways that are consistent with those commitments.

While the full challenge of learning to facilitate discussion is complex, in this paper we tackle three aspects we believe are important for developing teachers, building on Gibson’s (1969) theory of learning to notice as developing more differentiated perception of the structure of a situation. First is increased differentiation with respect to the structure and features of student talk, including both what is present and what is missing. Second is increased differentiation with respect to the features and dimensions of teachers’ potential responses to that talk, such as when teachers observe other teachers or they themselves apply talk moves. Third is an identification and refinement of one’s values and commitments with respect to discussion and more differentiated noticing with respect to alignment with those values.

We present a proof of concept of an instructional approach that uses contracting cases (Bransford & Schwartz, 1999) within a designed activity structure to support noticing. In Study 1, we engaged a small group of pre-service teachers and audio recorded their interactions with the learning materials. In study 2, we engaged a larger group of undergraduates, and included an active control condition who completed a more standard transcript annotation activity for comparison. In both studies, we look at shifts in noticing from before to after the learning intervention and explore how the intervention may have encouraged those shifts.

Prior work on classroom discussion

Features of productive discussion

Discussion has been shown to support student learning across content areas and is often lauded as an evidence-based, high-leverage practice for teacher education (e.g., Forzani, 2014; TeachingWorks, 2023). In math, discussion has been shown to support students’ motivation and engagement and help students process mathematical content (Cirillo, 2013). In English language arts and literacy, a meta-analysis revealed that many, but not all, approaches to discussion increased students’ comprehension and critical thinking (Murphy et al., 2009). In science, discussion has been shown to help make models of scientific thinking available to students (Duschl & Osborne, 2002) and result in improved reasoning, problem-solving, and understanding (Mercer et al., 2004).

In a productive discussion, teachers and students use one another’s ideas as resources to build collective knowledge related to instructional goals. Teachers often have an idealized notion of class discussion, but may be unaware of how specific teacher and student actions can facilitate or hinder it (Adler et al., 2003). Across disciplines, several features of productive discussion have been established in the literature. In contrast to initiate, respond, evaluate (IRE) sequences of classroom talk in which the teacher asks a known-answer question, the student replies, and the teacher evaluates (Mehan, 1982), productive discussion includes authentic open-ended questions, uptake promoting cohesive discourse (Nystrand, 1997), and students taking on leading roles. In high-quality discussions, teachers are not positioned as “knowledge holders,” but instead students contribute the majority of ideas (Hufferd-Ackles, Fuson, & Sherin, 2014). Research suggests novice teachers may struggle with eliciting rich student explanations, tending instead to provide the explanations themselves (Banes et al., 2018). They also struggle with positioning students to listen to, build on, and value one another’s thinking, not only the teacher’s (Hakuta et al., 2013). Teachers and students may use revoicing to repeat, verify, or clarify an idea, facilitating a shared understanding (Herbel-Eisenmann et al., 2009). Ideally students engage in “building on” ideas from one talk turn to another and “building up” of central ideas to develop a complete idea across the discussion (Hakuta et al., 2013). Moreover, in productive discussion, teachers engage diverse student voices (Banes et al., 2018) and probe student reasoning with evidence (Michaels & O’Connor, 2012). Based these ideas, we operationalized five categories of features to focus on as we developed the contrasting cases that form the basis of our instruction: engaging multiple student voices, eliciting and probing student ideas, revoicing, uptake and building of student ideas, and bringing evidence into explanations.

While some facets of class discussion may differ across disciplines, notably, what counts as “evidence” in argumentation (Wolfe, 2011), the features described here were selected as foci for this study of teacher noticing because they have been shown to be effective elements of productive class discussion in studies in math, science, history, and English language arts and can be showcased in short discussion transcripts. Note that these are not the only possible features we could have focused on, but they represent a subset known to be important toward the goal of rich student-centered discussion.

Teacher noticing and professional development

Professional development projects have supported in-service teachers’ development of discussion practice, including using talk moves (Michaels & O’Connor, 2015), setting norms, and fostering meta-talk within discussions (Kuhn & Zillmer, 2015). More pedagogical innovations are needed that engage future teachers in thinking deeply about the features of effective class discussion and the teacher and student moves that facilitate it. Tools and approaches that support novice teachers in learning to orchestrate productive discussion early in their careers are crucial, especially when they may not have had opportunities to observe or engage in rich class discussion themselves. Walshaw and Anthony (2008) suggest teachers’ beliefs about teaching and learning play an important role in developing complex class discussion practices. Thus, learning experiences that highlight what is possible in class discussion and offer opportunities for reflective analysis may be especially useful. Our approach engages preservice and prospective future teachers in refining their noticing across varied discussions while they reflect on their own values about what makes for a productive discussion.

Some approaches to professional development have focused on teacher noticing. Teacher noticing includes attending to features of classroom interactions, reasoning about what was observed, and deciding how to act (Jacob, Lamb, & Phillip, 2010). Much of the research on teacher noticing has come from the domains of mathematics and science education and has focused on teachers’ abilities to attend to and interpret students’ mathematical or scientific thinking (e.g., van Es & Sherin, 2021). Research suggests that when noticing in videos of classroom interactions, preservice teachers (PSTs) typically focus on superficial features of classroom practice, such as hand raising, following procedures, and staying on task (Star & Strickland, 2008; Erickson, 2011), and they focus more on teachers’ actions than students’ thinking and learning (Levin et al., 2009). Research in mathematics and science education has shown that extensive professional development with structured supports, such as reflection and feedback (Davis, 2006), use of contrasting examples (Kisa, 2013), and appropriate framing (Star & Strickland, 2008; Barnhart & Van Es, 2014) can shift what PSTs notice, helping them learn to attend to the features that will better position them to enact rigorous and responsive instruction that builds on student thinking in their domains.

In these studies, our aim is to support future teachers’ abilities to notice features of productive classroom discussion that may be applicable across content domains, namely the five feature categories described above. We present these studies as a proof of concept that a relatively short, contrasting-cases based approach can lead to shifts in what participants notice while watching videos of classroom discussion in ways consistent with perceptual learning theories and prior research on productive teacher noticing. We describe the instructional approach and rationale behind it in a way that we hope will be generative for others wanting to develop a similar model.

Instructional approach

History of contrasting cases-based instruction

Our approach to helping future teachers develop noticing skills involves an instructional method derived from perceptual learning theories (Gibson, 1969). These theories initially focused on discriminating visual, auditory, or sensory information, such as detecting differences in the loudness of two tones, and have expanded to examine more ‘conceptual’ differentiation, such as distinguishing the quality of two writing samples (Lin-Siegler et al., 2015) Contrasting cases are juxtaposed examples that are chosen to highlight distinctive features or relationships (Gibson, 1969; Schwartz & Bransford, 1998; Bransford & Schwartz, 1999). Generally, a majority of features are held constant between examples, so that key differences stand out. For example, pairing wines side-by-side can help people learn to perceive subtle differences they might otherwise gloss. The purpose of contrasting cases is to help people learn to notice important features or dimensions so they can develop a more differentiated understanding of a concept or phenomenon. This can help people learn to distinguish one thing from another, recognize what features or elements are important, and better understand conditions of applicability (Schwartz et al., 2016). Contrasting cases have been shown to support such learning across many contexts including physics (Schwartz et al., 2011; Shemwell et al., 2015), statistics (Schwartz & Martin, 2004; Kapur, 2014), mathematics (Rittle-Johnson & Star, 2007), management science (Roelle & Berthold, 2015), and writing (Lin-Siegler et al., 2015).

A few studies have looked at the use of contrasting cases in the context of teacher education. For example, Schenke and Richland (2017) looked at the relationship between PSTs’ mathematical content knowledge and their spontaneous use of contrasting examples of student answers in their instruction. Derry et al. (2007) engaged teachers in contrasting their own solutions to mathematics problems to help them learn to notice opportunities for algebraic thinking in student responses. Two studies have used video-based contrasting cases^{Footnote 1}. Nagarajan & Hmelo-Silver (2006) examined effects of different scaffolding prompts, including compare and contrast and a metacognitive prompt, on what undergraduates in an educational psychology course noticed from contrasting videos of formative assessment interactions between a teacher and student. In Kisa (2013) five biology teachers were tasked with comparing and contrasting across two videos, one showing a small group solving a complex science problem with high levels of student thinking and a second showing a group solving the same task with low levels of student thinking. Kisa found shifts in teacher noticing while engaging with the professional development sessions, with teachers making more linkages between teacher actions and student thinking and more interpretations of student thinking.

The ways the contrasting cases are constructed and the activities by which learners engage with them has varied across studies, depending on purpose. A common activity involves asking learners to compare and contrast between the cases with the goal that they will notice key distinctions (e.g.,. Lin- Siegler et al., 2015, Kisa, 2013). For example, Roelle and Berthold (2015) had learners engage with two contrasting examples of company production styles. They compared the effects of having the similarities and differences pre-identified for the learners to having the learners identify the similarities and differences themselves. However, Chin et al. (2016) discuss some limitations of compare-and-contrast task demands, in favor of task demands that encourage synthesis across a set of cases.

An illustrative example of a synthesis task comes from the domain of statistics. Schwartz and Martin (2004) developed instruction where high school students were tasked with inventing a mathematical procedure that would capture the reliability of a baseball pitching machine based on data about where the machine’s balls hit relative to a target. To aid them in inventing their procedures, students were provided examples of data from different pitching machines that their procedure should be able to handle, as shown in Fig. 1. These carefully chosen examples - the contrasting cases - were selected to help students notice important features of distributions that a solution would need to take into account. For example, comparing across cases, students might realize that Big Bruiser involved a different number of balls, highlighting the need to consider sample size. Comparing Smyths and Fireball highlights the need to differentiate reliability from accuracy, while comparing Ronco to the other cases alerts students to the concept of outliers. Students invented a range of solutions, which they iteratively refined as peers or the teacher pointed out features of the stimuli that the solution did not account for (e.g., different sample sizes). The instructional goal of the activity was to help students notice important features of the problem space that a solution would need to take into account, such that when students are later introduced to a canonical solution (the mean deviation formula), they better understand the mathematical work it is doing. Through the process of iteratively refining their solutions as they related to the features of the stimuli, increased precision in differentiating the stimuli reinforced differentiation of the solutions, and vice versa. This preparatory noticing activity helped students be more adaptive to new kinds of problems that relied on the same underlying concepts compared to groups of comparison students who learned the mean deviation formula in more traditional ways.

In line with the approach taken by Schwartz and Martin (2004), our intervention presents multiple carefully chosen contrasting cases and a task that requires learners to synthesize across them. The cases are transcripts of classroom talk, representing a space of stimuli (features of student talk) and possible responses (features of teacher moves). Learners in our interventions are asked to synthesize across transcripts to generate three principles that identify what they consider the most important features of productive discussion.

A brief note on terminology: although the participants in our studies are simultaneously students at the university and pre-service or prospective future teachers, in the presentation of our study design and data we will refer to them as “learners” or “participants.” This choice allows us to reserve the terms “student” and “teacher” for the discursive roles within the transcripts of classroom talk.

Design rationale for the instructional intervention

Design of the activity structure

We first describe the design of the overall activity structure and how it relates to the three areas of development that are the focus of the intervention.

a)
Increased differentiation with respect to the structure and features of student talk, including both what is present and what is missing.
b)
Increased differentiation with respect to the features and dimensions of teachers’ potential responses to that talk.
c)
Identification and refinement of one’s values and commitments with respect to discussion and more differentiated noticing with respect to alignment with those values.

As the basis of the activity, we developed sets of brief, contrasting, fictitious transcripts of classroom interaction. The content of these will be described in more detail below. Their overarching function was to highlight (through contrasts) key features of both student talk and teacher responses to help study participants learn to notice and differentiate these features.

Each study participant received copies of the six transcripts printed on half sheets of paper. They worked with a partner to rank the dialogs from what they thought was the best to the least good example of classroom dialog, however they defined it, and to identify and record two to three principles that guided their ranking. They were instructed that taken together, their principles should explain the ranking of all the dialogs (e.g., for any dialog pair, someone should be able to look at their principles to decide which one they likely thought was better), and they should be able to imagine applying these principles to a new video or transcript of a discussion. There was no single right answer in the ranking or identification of principles. The primary function of the ranking activity was to engage learners in making multiple, close, pairwise comparisons across the cases to help the built-in contrasts stand out. A need to commit to a single ranking among partners ensured that disagreements would be identified, discussed, and resolved. The function of identifying principles was to (a) explicitly name the values related to discussion and (b) discuss how the features of the cases related to those values. Identifying principles helped learners synthesize the ranking of individual cases into a few higher-order principles (Chin et al., 2016). This may support transfer, as learners both differentiate features at the level of the specific stimuli and distill a deep structure that can generalize. Additionally, we hypothesized that identifying what features of a productive discussion should be present would help learners notice instances when those features were absent.

Once they completed the main activity, participants were provided with three additional cases and told that they could modify their principles if they liked. Working with new cases helped learners determine if their principles could handle new examples and refine them if needed.

Finally, participants engaged in a whole class discussion, led by the instructor, about their rankings and their principles. They first entered their group’s rankings into a shared document that was projected on screen in front of the class. The instructor then facilitated an open discussion of the rankings and the principles they came up with in their groups, what they meant, and why they were important. This discussion lasted about 10 min. The instructor then gave a 10-minute pre-prepared lecture on features of classroom discourse, making connections to the just-completed discussion when possible. The rationale for the group discussion and lecture was that activities that ask learners to invent principles or explanations often benefit from an expert recap and create a time-for-telling (Schwartz & Bransford, 1998). Participants may not all notice every relevant feature during the contrasting cases activity. The discussion and expert recap provided an opportunity to see features they missed and consolidate the things they noticed into an explanatory framework that can help organize their understanding. Additionally, the lecture offered a window into research-based, canonical understandings of productive features of discussion, offering an opportunity for learners to refine their values.

Design of the cases

The core learning activity made use of a set of six fictitious transcripts of classroom dialog in a middle school science classroom. (See Appendix A for the full set of dialogs cases and activity prompts). Though fictional, the dialogs were developed based on the authors’ experiences with teachers and youth in classrooms and thus, represent a range of discussion features that occur in real classrooms. The transcripts presented a classroom discussion focused on whether or not the placement of toxic waste sites is an environmental justice issue, and the six dialogs represented similar but distinct contrasting cases of how the dialog could have gone. The dialogs were piloted with a group of teacher educators and revised before they were presented to participants.

As noted above, we developed these transcripts to contrast (and thus make salient) selected discussion features based on a review of the literature, focusing on engaging multiple student voices, eliciting and probing student ideas, revoicing, uptake and building of student ideas, and bringing evidence into explanations. For example, across cases, different numbers of students contribute, teachers or students drive the source of ideas, and evidence is brought to bear to varied degrees. Design principles included the need to keep the dialogs short while presenting several variations of the selected discussion features and ensuring dialogs were as realistic as possible. We also wanted to ensure that pairwise comparisons of cases could isolate particular features to make them salient (for example, Dialogs A and F differ only in revoicing), while across the cases, features showed up in different combinations and variations to help participants learn to identify features across instantiations and support discussing and refining principles for ranking (for example, the relative importance of ideas being generated by students and discussions including correct evidence). Appendix B provides a table of the relationship between features and cases.

As an example of the dialog contrasts, consider the first two turns of Dialog A:

S1: I talked about how toxic waste sites are kinda like the lead paint we read about last week—they both make people sick.

T: Okay. Good! Someone on the other side of the room want to share? Can you add something new?

Dialog B had the same opening, but the teacher gave a different response:

S1: I talked about how toxic waste sites are kinda like the lead paint we read about last week—they both make people sick.

T: So, what I hear you saying is that the location makes it an environmental justice issue because they don’t make people sick evenly, right? So, not everyone is impacted equally by the toxic waste sites. Someone on the other side of the room want to share? Can you add something new?

In this paired example, the teacher in Dialog A affirmed the student’s response and moved on, whereas the teacher in Dialog B paraphrased the student’s contribution and then offered his or her own language and analysis. This particular contrast was designed to highlight aspects of teacher revoicing.

While we certainly aimed to highlight features meaningful to classroom discussion -- and built the fictitious transcripts from the authors’ experiences as teachers and teacher educators to represent kinds of dialog present in real classrooms -- we note that the goal of the present studies was to assess the utility of contrasting cases for promoting noticing. The features contrasted are drawn from the research literature but are not the only features we could have chosen to highlight. For example, we did not in this instance highlight aspects of relational histories among discussion participants or contextual features of the classroom. Among the designed cases, we also did not presuppose which of these features the learners would find most valuable, problematic, and so forth, and this is something we examine in the data. Additionally, the relatively short length of the contrasting cases was chosen to enable the certain features to be made salient without overburdening learners with excessive reading across cases. If this initial proof-of-concept proves fruitful, additional research could consider how choosing different contrasts or longer or shorter cases might influence effectiveness.

Overview of the studies

Study 1 focused on teacher candidates nearing the end of a post baccalaureate teacher credential program to explore their noticing and reasoning around identified features of classroom discussion and examine how the activity might influence their noticing. Study 2 was a replication and extension of Study 1 with a different population (undergraduates -- most of whom were interested in teaching, but who did not necessarily have classroom teaching experience). The population for study 2 was chosen largely because it afforded access to a large enough group of participants to enable a control condition. While Study 1 involved a small sample and no matched control, Study 2 enabled us to introduce an experimental contrast involving a contrasting cases condition and a control condition who completed a transcript annotation task designed to represent a more conventional though still active learning activity.

Based on the research literatures on perceptual learning and teacher noticing, we might expect specific patterns of shifts following instruction. These include:

a)
a general shift away from focusing on teacher behaviors toward focusing on student talk and student ideas, consistent with prior documented changes in noticing with increasing expertise (e.g., Levin et al., 2009).
b)
increased precision in differentiating aspects of student talk, such as noticing more kinds of features and c) an increase in precision of differentiating the teacher’s responses, such as how a talk move is being implemented, which might include noticing more missed opportunities in its implementation.

With respect to noticing of student talk, patterns (a) and (b) would both predict overall increases in student-focused noticing from pre- to post-intervention. With respect to noticing of teacher behaviors and talk moves, pattern a) would predict an overall decrease in teacher-focused noticing as learners shift toward student noticing, while (c) would predict a shift in the quality of teacher-focused noticing, such as noticing more absences.

Study 1 provides an initial test case among advanced pre-service teachers nearing the end of their program. While the population for Study 2 was chosen largely for access, seeing similar effects among a population with less teaching experience would additionally strengthen our proof-of-concept.

Study 1

Participants

Participants included 18 preservice teachers (13 women, 5 men) who were participating in a university course on bilingual teaching methodology and classroom inquiry as part of a post-baccalaureate teacher credential program. Twelve were multiple subject teacher candidates preparing to be elementary school teachers or all content areas, and 6 were single subject candidates preparing to be high school teachers focusing on a specific content area (2 social science, 2 science, 1 English, 1 math). Discussion is relevant in elementary through high school teaching across domains, and all students in the class participated in the learning activity as part of their coursework. All participants were bilingual, taking the course to fulfill requirements to have a bilingual authorization added to their credentials. Seventeen PSTs were Spanish-English bilinguals and one was a Mandarin-English bilingual. All multiple subject PSTs were student teaching in dual language immersion elementary school placements and single subject PSTs were student teaching in English-as-the-language-of instruction high school classrooms. This study took place in the final month of the credential program, and all PSTs had been student teaching for 9 months. All were placed in classrooms with high percentages of culturally and linguistically diverse learners. Their coursework and readings in the credential program had provided some prior exposure to features of effective class discussion, including the importance of eliciting student talk and supporting language learners. Most candidates had not yet engaged with the ideas of uptake or revoicing in class discussion. The quality of class discussion they had observed in their student teaching placements varied.

Materials

Dialog Cases. As previously described, the primary set of cases were six fictitious transcripts of classroom dialog in middle school science that included both teacher and student talk. There was an additional supplemental set of three shorter transcripts that included a few turns of student talk only. We paired these cases with a worksheet that asked participants to rank the cases from what they thought was the worst to best example of productive discussion and to identify three principles that explain their ranking. Full materials can be found in Appendix A. Each dialog case was printed on a piece of 8.5 by 5.5 inch paper, and the worksheet was printed on 8 × 11 paper. The primary cases ranged from 94 to 232 words long (M = 160) and were labeled with the letters A through F. Though the activity was done in pairs, each participant had their own copy of the materials.

Videos for Noticing. As part of an experimental noticing assessment, we asked participants to view and comment on videos of classroom discussion before and after engaging with the dialog cases. Video-based assessments in which teachers detect effective teaching practices are often used as measures of teacher learning (e.g. Kersting, 2008; Santagata & Guarino, 2011; Wiens et al., 2013) and have been shown to be able predict the effectiveness of teachers’ pedagogical implementation and student achievement (Kersting et al., 2012). For our video noticing task, we selected four 90 s clips: two clips each from two videos available online through teachingchannel.com and the MA Department of Elementary and Secondary Education^{Footnote 2}. We selected two similar clips from each video, which were counterbalanced, so participants could comment on different clips at pre and post, rather than commenting twice on the same clip in a short span of time, which may have influenced responses. These videos were selected because they were publicly available, had good quality audio and video, and showed students actively participating in a discussion. We note that these conditions were challenging to meet, and we make no claim that these videos were optimal, but selected them only to provide opportunities for noticing. The contrasting cases instructional activity used written transcripts, and so videos of classroom discussion would represent transfer to a more authentic context. The videos differed in the age of the students featured (middle school vs. high school), the ways in which the teachers facilitated discussion (more teacher-driven in the case of the middle school video), and the ways that discussion unfolded (e.g., in patterns of teacher-student vs. student-student talk, with more teacher-student talk in the middle school example). Our intent was to select videos that would allow learners to engage in noticing of the uptake of ideas and other features emphasized in the transcript cases. We refer to the middle school video clips as MS1 and MS2, and the high school clips as HS1 and HS2, with 1 and 2 referring to time order in which they were pulled from the larger video.

Procedure

Preservice teacher participants completed activities as part of the last 90 min of a session of their regular bilingual teaching methods and inquiry course. All participants completed activities as part of regular coursework, but were also given the opportunity to volunteer as research participants, allowing us to collect data on their participation. Everyone in the class consented to be part of the study.

Participants first completed the Video Noticing Task. To begin, we told them, “In a class discussion, when someone says something, there are a variety of ways their idea can be taken up by others. It can be built on, repeated, paraphrased, disagreed with, and so on.” We told them their task was to “write down the things you noticed about the ways the teacher and the students take up each other’s ideas.” Each participant watched a 90 s video clip (MS 1 or 2) and had 3 min to write comments on what they noticed. They then repeated this process for a second clip (HS 1 or 2). To ensure even distribution of each version of the video clips at pre and post, half of the participants were randomly assigned to watch Clips MS1 and HS2, and half were assigned to watch Clip MS2 and HS1. Participants completed this activity individually on a laptop while wearing headphones. There was no debriefing on the noticing activity.

Participants then moved to work on the Contrasting Cases activity in pairs. Participants had 30 min to complete this activity. We audio recorded their conversations. They then participated in a short discussion and lecture (approximately 10 min each, described previously) focused on productive classroom discussion.

Finally, participants completed the Video Noticing Task a second time, as a post-test. The process was identical except that participants who previously watched clips MS1 and HS2 now watched MS2 and HS1, and vice versa.

Coding process

We began our analysis with an inductive coding pass of participant responses on the Video Noticing Tasks. We proceeded iteratively through an open coding pass, where all members of the research team completed open codes on a randomly selected subset of the data. We then collaboratively discussed our open codes, debating typical and boundary cases, before reaching consensus on a smaller set of axial codes. Our axial codes included five codes focused on noticing teacher talk and six codes focused on student talk (See Table 1). Based on prior research, we were a priori interested in differences between the teacher and student-focused categories. Additionally, we found that participants sometimes commented both on the presence of a feature (e.g., “the students make points and other students build on those points.”), and sometimes actively commented on the absence of a feature or noted a missed opportunity (e.g., “there is a lack of building on a topic from one student to another. Instead, it seems like each new student is offering a new idea to the table”). As such, for each of our eleven codes, we also included an additional tag to indicate if the code was noted as “present” “absent” or both. We call this dimension valence. We did not have a priori expectations about presence vs. absence codes, but wanted to examine whether there were differences in shifts between them, as we hypothesized that noticing absences may be more difficult than noticing presence.

After reaching consensus on this coding scheme and developing a codebook with definitions and examples, two senior researchers coded 25% of the video noticing reflections independently and reached interrater reliability of 83% (exact agreement/total codes applied). Discrepancies were then discussed and the coding scheme further refined the definitions in the codebook until both coders reached agreement.

Two junior researchers and one of the senior researchers then applied the coding scheme to the PST responses. In all coding, researchers were blind to whether a response was from the pre or post video noticing activity. Next, all researchers met to compare coding results and discussed areas with low inter-coder reliability to dispel any differences in interpretations of the coding scheme. This process was repeated until all responses had been coded by all researchers and discussed in meetings (Cohen’s kappa average final two rounds of coding: 0.83 presence, 0.82 absence).

Table 1 Codes and Examples for Video Noticing Task. Note: Within descriptions, “T” represents teacher and “S” represents student

Full size table

Results

Pre- to post- shifts on video noticing task

After coding the Video Noticing Task (see Table 1), we examined differences in participants’ responses before and after the Contrasting Cases Activity. We first summed the total number of teacher-focused codes in which the presence of a feature was noted (valence = present) and summed the number of teacher-focused codes in which the absence of a feature/missed opportunity was actively noted (valence = absent). We did the same for the student-focused codes. We then conducted a repeated measures ANCOVA with the within-subject factors of time (pre-post), subject-focus (teacher-focused or student-focused), and valence (noting presence or absence of features). Video clip order was included as a covariate. There was no significant main effect of time (F(1,15) = 0.37, p > .5), but there was a significant interaction of time by subject-focus (F(1,15) = 14.01, p < .01, η_p² = 0.48). Overall the total number of codes noted did not increase significantly from pre to post, but there were shifts in the focus of codes. Video noticing responses showed different patterns of change for student-focused codes than teacher-focused codes (Fig. 2). Follow-up analysis found a marginal increase in the total number of student-focused codes from pre-to-post (t(16) = -1.77, p = .096, Cohen’s d = 0.43), consistent with one of our hypothesized patterns of shifts. There were no changes in the total number of teacher-focused codes, t(16) = 0.24, p > .5, Cohen’s d = 0.06). However, small sample size limits interpretation of results. Nominally, there was an interaction within the teacher-focused codes, such that the number of codes noting the presence of a feature in the teacher’s behavior decreased while those noting absence of a feature or missed opportunity increased, though this interaction was not significant.

We next examined specific codes along the teacher-focused versus student-focused dimension. The following analyses are exploratory, and do not control for multiple comparisons. Though the overall number of teacher-focused codes did not significantly change, participants were marginally more likely to talk about the presence and/or absence of teacher’s open-ended questions at post (exact McNemar’s test, p = .06). Before the contrasting cases activities, 65% (11) of PSTs mentioned something coded in this category. After the activities, 95% (16) did. With respect to student-focused codes, there was a nominal trend toward an increase in talking about student-centeredness of the discussion (exact McNemar’s test, p = .12), with 47% (8) of PSTs mentioning something coded in this category at pre and 75% (13) at post. Participants were more likely to notice the absence of students building on each other’s ideas (exact McNemar’s test, p = .03), mentioned by 6% (1) of PSTs at pre and 41% (7) of PSTs at post.

What happened during the intervention

As pairs ranked cases and developed their principles, they identified and grappled with what they prioritized in a discussion and how it played out in the dialogs. There was variability in how the groups ranked different dialogs, as can be seen in Fig. 3. Participant pairs uniformly rated Dialog E, which was completely student-led, as the best, followed by dialog C, which involved extensive teacher eliciting of student ideas. Ratings of the remaining dialogs showed more variability. Dialog B, referenced earlier, where the teacher did a considerable amount of elaboration while revoicing student comments, was particularly controversial (mean rank 4.11, SD = 1.45). Three of the nine groups ranked it as worst (sixth), while five groups ranked it as third best. While groups varied in how they ranked it, all noticed a key feature it was intended to highlight: whether the teacher interjected additional ideas into the conversation when revoicing. The differences in the rankings of Dialog B reflect differences in valuing, rather than differences in noticing the key feature.

To illustrate the process behind the rankings, consider the discussion that one pair of participants, Carla and Maggy, both multiple subject PSTs, had around Dialog B as they discussed teacher rephrasing and revoicing, which they noted can be positive, but can also represent the teacher leading the conversation at the expense of student voice. We selected this audio-transcript as a representative example of how, through ranking and closely comparing the contrasting cases, PSTs reflected on tensions they experienced in their moment-to-moment decision making and their developing beliefs about the goals of productive discussion.

The pair first compared Dialog B to Dialog D, which included many teacher-led short answer questions.

Carla: The teacher also rephrased it in B. I like rephrasing because sometimes you understand things [inaudible]. Yeah, so I like when the teacher rephrases something, because one, it’ll probably be louder, and two, it’ll be concise, whereas the students tend to ramble sometimes.

Maggy: Yeah.

Carla: So, to me, that was a good thing, when a teacher rephrases. So I like that D and B rephrase. But again, D has no sentences from the students, although the teacher is asking them questions.

Maggy: Yeah, but this is a….

Carla: But it’s not high level? What do you think?

Maggy: I just don’t like how the teacher is leading them to the… It’s kind of a yes and no question. “Do you think that this is bad?” “Yes.” “Why?” You know? It’s very low level.

Carla: Do you agree that the teacher’s doing that in both of these, the teacher’s leading the conversation?

Later, this pair contrasted Dialog B with Dialog A, continuing to consider the role of students and the teacher in leading the discussion and contributing ideas, and referring back to their own classroom instruction to further problematize rephrasing.

Carla: For me, it goes back to, do you want students to lead or something? If our goal is for students to lead and cooperate on this question or come up with their own idea, then A is gonna be better than B, but that brings in the fact that A would be better than B [inaudible] because again, the teacher isn’t creating the conclusion for them.

Maggy: This is fun! I’m having a whole load of fun, I don’t know why.

Carla: Well, ‘cause….

Maggy: From this answer the student gave, the teacher is like, “So I hear you’re saying that the location makes it an environmental justice issue because they don’t make people sick evenly,” right? And it’s like, of course the student’s not gonna say, “No, it’s not an environmental justice issue”.

Carla: Yeah, right.

Maggy: Well I’d hope they’d say that, but you’re right, they probably won’t.

Carla: So not everyone is impacted… and then, the teacher’s just, you know, making conclusions for the students, so not everyone is affected equally by the toxic waste sites.

Maggy: Yeah.

Carla: So that, “Someone on the other side of the room want to share,” they already shared their opinion, or like, supposedly what the student’s opinion was, but it sounds like that’s what the teacher thinks, too. And I know when I do that in my classes, when I’m like, “Oh, so do you think that Fidel Castro [was a hero] because he did this and this?” And then the students are like, “Uhhh, yeah, yeah he was a hero.”

Maggy: Because of what you already said.

Carla: I don’t know, yeah.

Carla: Okay, so then… For me, I feel like A is a little better, even though there’s not a lot of talking, they’re still coming up with their own conclusions here.

Maggy: Right.

Carla: I don’t know. What do you think?

Maggy: Honestly….

Maggy: It really depends on what I want them to take from this discussion, but if we do think that a discussion… ‘cause we also have to define how….

Carla: So one of the ways you’re defining a discussion is students need to learn how to create their own ideas and put those into a discussion. I think B is worse. A is better than B.

For Carla and Maggy, Dialog B prompted them to consider several important concepts. Comparing Dialog B to Dialog D brought up ideas around rephrasing and high- versus low-level questions. Later, when they compared Dialog B to Dialog A, they talked more about the sources of their ideas and conclusions. After relating the teacher moves in Dialog B to their own experiences leading discussion, they came to the conclusion that, for them, a central component of a discussion was that students are creating their own ideas and coming to their own conclusions, which they then included as a principle for ranking the dialogs. In this example, comparison across the dialogs helped the participants to differentiate their perception of a teacher talk move (revoicing), as well reflect on and refine their own values and commitments when it comes to discussion.

A second exemplar illustrates study participants attending to and differentiating features of student talk across contrasting cases and identifying them in their guiding principles. To illustrate how this may have plausibly influenced noticing, we present a conversation around the three supplemental dialogs (G, H, and I).

Across the three dialogs, student one says the same opening line:

S1: Like White people and people with money don’t get sick as much from the toxic waste. So that means it’s not fair.

The dialogs differ in student two’s response. Dialogs G and H contrast ‘I agree because’ sentence frame use with responses that differ in degree of uptake and building. Dialog I presents an example of uptake that does not use a sentence frame.

Dialog G.

S2: I agree with David because rich people don’t live by the toxic waste sites. The toxic waste sites are mostly near low-income neighborhoods like we can see on the map.

Dialog H:

S2: I agree with David because pollution is really gross, like when there’s garbage in the ocean and it kills the dolphins. People shouldn’t pollute the earth.

Dialog I:

S2: People shouldn’t get punished and get sick more because of where they live. They can’t afford to move.

Consider the following dialog between Brandon (single subject English) and Javier (single subject science) as they negotiate meaning and unpack their understanding about what it means for students to “build on each others ideas.” While all PSTs negotiated meaning with their partners, this transcript was selected as an exemplar of partners jointly constructing a nuanced understanding of one of their key principles, using examples from the contrasting cases.

Javier: I’m going to add direct responses [to our guiding principles]. Or responses that build on the other.

Brandon: Yeah.

Javier: This one’s not a good one because [the student] is not really responding to the other student. He’s just like “The ocean’s really gross.” That’s not what [the first student] was saying. He wasn’t saying that [pollution] was gross.

Brandon: Well, they’re saying that [the placement of toxic waste sites] is not fair.

Javier: Yeah.

Brandon: But it’s not building off of what he was saying. Yeah. They’re just bringing-.

Javier: It’s like he wasn’t paying attention.

Brandon: -In another point.

…

Brandon: It’s like, how well were they able to improve… So it looks like students that were actually able to improve the first person’s argument. The first person on the top dialogue was saying “White people and people with money don’t get sick as much from toxic waste, so that means it’s not fair.” And then student two is saying “I agree with David because rich people…”.

Javier: Yeah.

Brandon: And then they say that the toxic waste sites are mostly near low income neighborhoods like we can see on the map.

Javier: Yeah.

Brandon: Instead of just saying “that means it’s not fair.” Yeah.

Javier: Yeah.

Brandon: So that is a big leap, and they are building off of-.

Javier: Yeah, they’re directly building off of it. Like the third one-.

Brandon: Yeah, it’s a real improvement.

Javier: “People shouldn’t get punished.” He kind of took it a little further.

….

Javier: Yeah. And the other one’s just like “The ocean’s gross.”

In the above conversation, Brandon and Javier reason that for students to “build on each other’s ideas” their conversation turns must be closely related. Brandon and Javier decide, despite the use of marked linking language or sentence frame (“I agree with ___ because…”), Dialog H is actually not an example of building on student ideas. Comparing dialogs offered opportunities for Brandon and Javier to deepen their understanding of discussion as they jointly decided responses must “improve” an argument or “take it a little further” to count as “building.” Moreover, they reflect an understanding of how “building” can sometimes serve to fill in gaps (“big leaps”) in the class’s collective argument by clarifying ideas and adding evidence. They contrast this example of strong student-student uptake to the pseudo-uptake they identified in the first transcript, which may have influenced their attending to student ideas. In this case, as the pair were refining their principles, they were also plausibly refining what they were noticing in the student talk sequence.

In all, the principles that participants generated were related to the contrasts built into the cases. Each of the nine pairs of participants identified three principles, yielding 27 in total. We combined like principles and categorized them into seven groups (guiding principle categories), some of which were teacher-focused, and some of which were student-focused (see Table 2). The most common principle identified by the PST pairs was that the discussion should be student-centered or focused, which is perhaps reflected in the pattern of a larger increase in student-focused noticing codes post-instruction. With a larger sample size in Study 2, we will examine how principles identified during the activity relate to shifts in noticing patterns after instruction.

Table 2 Most frequent categories of guiding principles in Study 1

Full size table

Study 2

Study 1 showed that the Contrasting Cases Activity led to fruitful conversations among preservice teachers, with evidence of shifts in noticing on the Video Noticing Task. However, a small sample size and lack of a comparison condition limited interpretation of results. In Study 2, we sought to replicate and extend Study 1 with a larger sample and addition of an active control as an experimental contrast. We also moved to a different population of participants, in this case undergraduates who were taking a course on educational psychology. This shift afforded the opportunity to have a sample large enough to include a control condition, and it allowed us to explore how more novice learners, who had not gone through a teacher credential program, would respond to the instructional intervention. Previous surveys of undergraduates in education courses at this university found that more than two-thirds reported teaching as a definite or possible career goal. Because of this, we refer to these undergraduate as prospective teachers, though not all may choose to pursue teaching. Compared to the preservice teachers in Study 1, they had less knowledge and experience about teaching.