Data Collection: Participants
Data were collected from three teachers’ classrooms, representing eight sections of middle school physical science. The school was located in rural, northern New England. Forty-eight percent of students in this school district were eligible for free and reduced lunch, higher than the state average. Two of the teachers transferred to other schools after the first year of the study; therefore, we collected two academic years worth of data from one teacher and only one year from the other two teachers. All three teachers were using the Project Based Inquiry Science Curriculum (Kolodner et al. 2010) for physical sciences. The data collection occurred after students had been formally taught the material, and thus, the intent was for the questions to cover material that was familiar to the students.
One-hundred and thirty-four students (70 male, 64 female) participated in the study. They answered between two and four sets of multiple-choice clicker questions individually for a total of 250 data points. During peer discussion opportunities, students worked in groups of 2–5, and a total of 72 small-group peer conversations were recorded. All student groups were designated by the teacher. We did not specify how teachers should form the groups, but found they did so largely to minimize disruptions in the classroom. Since we were interested in what the students could learn from one another, we asked teachers to only intervene in student discussions if needed to maintain order.
We used both quantitative and qualitative approaches in order to examine student performance on clicker questions and evidence for co-construction during peer discussion. A matched-pair clicker question design was used (e.g., Smith et al. 2009), and all student peer discussions during clicker questions were audio recorded.
For the matched-pair question study design, we wrote pairs of clicker questions on the concepts of kinetic energy, electrical energy, thermal energy, and forces. Teachers reviewed the questions prior to the study, and we revised the questions based on their comments (see Appendix for questions). The experimental design followed a preexisting methodological approach (e.g., Smith et al. 2009, 2011; Fig. 1). First, students answered one clicker question individually (Q1), and then, they participated in a conversation with peers about that question. Following the discussion, they had an opportunity to answer the same question again (Q1 After Discussion, Q1AD). Finally, they were presented with a second matched-pair question (Q2) that was conceptually similar to the first question. The students answered Q2 individually without consulting their peers. After Q2 had been answered, the teachers discussed Q1/Q1AD and Q2 with the class and showed the students’ voting frequency graphs. Neither the answers nor graphs that show the frequency of Q1/Q1AD or Q2 answers were revealed to students until after voting for Q2 was completed.
Questions (Q1/Q1AD and Q2) were randomized using a coin flip to decide which question would be asked initially and which would be asked as a matched-pair question (Q2). Throughout this paper, these questions are referred to by the science content topic: thermal energy, kinetic energy, electrical energy, and forces. There were no incentives given to students for participating in the voting and discussions associated with these questions; however, students were accustomed to discussing questions as part of the Project Based Inquiry Science Curriculum (Kolodner et al. 2010), but were new to the use of clicker questions.
A digital voice recorder was placed at each desk to record the student peer conversation about Q1. The clickers were assigned to each student, so when listening to the audio files, we knew the number of students sitting together, and each student’s clicker performance.
For the quantitative analysis, all statistical analyses were performed with SPSS (IBM, Armonk, NY) or Excel (Microsoft, Redmond, WA).
To analyze peer conversations during clicker questions, we used a grounded approach (e.g., Glaser and Strauss 2009; Shkedi 2005) that focused on evidence for collaborative knowledge co-construction. To conduct this analysis, we transcribed all of the peer discussions, read through each conversation, and generated initial elements of collaborative knowledge co-construction in each conversation. The unit of analysis was the group discussion related to each of the four clicker questions. There were 72 conversations, with an average length of time on task of 57 ± 36 s. We then discussed the initial elements with the research team, and through these discussions, consensus emerged about documenting similar elements of co-construction including: contributions to the science content from multiple individuals, acknowledgment of ideas, asking questions, and revision of ideas. Each of these elements of co-construction is described in more detail below.
Contributions to the Science Content from Multiple Individuals
Prior research has acknowledged the benefits of collaborative group discussion in which multiple individuals contribute ideas (Dillenbourg 1999; Stahl 2006). In collaborative group discussions, ideas are shared, explored, modified, improved, and expanded upon by multiple individuals. By emphasizing contributions to the science content from multiple individuals in the analysis, we explicitly distinguish among situations in which one student tells peers an answer (correct or incorrect) and situations in which multiple students contribute to the science content. For this element, we are not evaluating the accuracy of any individual’s contribution or when individuals tell their peers incorrect answers. Rather, when coding for this element, we focused on whether or not multiple students made verbal statements about the science content. It was not sufficient for multiple students to say only “yes” or “no” or to mention their choice for the multiple-answer question. Here is an example of a conversation about the forces on a propeller car moving at constant speed in which two students discussed the science content: “Student
1: I think its C. Student
2: Yeah. S1: Because there’s friction and like the wind and the air. S2: I just always pick C when I don’t know the answer. S1: Same here. S2: And it can’t be equal [Answer E]. S1: Yeah. S2: Because it’s moving forward so there is more force forward. S1: Yup. S2: What are you thinking? S1: Yes.”
Acknowledgment of Ideas
For knowledge co-construction to be successful, students need to listen to one another and not talk past each other; students need to acknowledge their own ideas and those of peers. Students need to listen to each others ideas with the aim of eventually incorporating these ideas into their own thinking. Less important is evaluating the correctness or incorrectness of other ideas. Practically, in the analysis, we were interested in instances when a student mentioned his or her own ideas or a peer’s ideas. Mentioning other ideas could involve evaluating the correctness or incorrectness of a statement or stating some uncertainty or a hole in one’s thinking. For instance, in this conversation about the same propeller car question, the second student in this excerpt points out a hole in her thinking while acknowledging the first student’s idea: “Student 1: I said D because it’s going at a constant speed. [Laughter] Student 2: Oh, I forgot about that.” And, in this next statement, also about the propeller car, a student acknowledges a hole in her own thinking: “Student 1: Yeah, I thought it was going to be the overall forward force, but it’s not speeding up so …” In these examples, students acknowledged peers’ or their own ideas and this could instigate contributions to the science content from multiple students.
Asking a Question About Science Content
Additionally, for successful knowledge co-construction, students need to ask questions. Asking questions is one of the eight practices highlighted as essential elements in K-12 science curriculums (NGSS Lead States 2013). Questioning helps develop habits of mind and is an important part of scientific literacy and helpful in the growth of scientific knowledge. Asking questions can lead to modifications in a student’s understanding and, in the case of peer discussion, can lead to a change in answer choice. Less important is the nature of the question, open or closed, and whether the question is on or off the main science content topic. In the analysis for this element, we focused on questions explicitly about the science content (e.g., “Did you pick number two because of the temperature?”). If a student simply asked about answer choice (e.g., “What did you pick?” “Why did you pick C?”), the question was not counted in this element because our goal was to capture questions that could support contributions to the science content from multiple students.
Revision of Ideas
Finally, for knowledge co-construction in this setting, it was insufficient for there to simply be discourse in which multiple people contribute ideas; there needed to be some revision of ideas. Revision is important due to the nature of clicker conservations. Students are typically first asked to answer a question individually, and then, after the peer discussion, they are given an opportunity to stick with their initial answers or revise their answers. Therefore, a fundamental instructional goal of peer discussion is for students who may have initially chosen an incorrect answer to revise their answers based on what they learned (Mazur 1997). We expected that there will be both revision in the direction of an incorrect idea and revision in the direction of a correct idea. A recent study, using a similar experimental design, found that of the students who changed their clicker question answer, the majority of changed answers were from the wrong answer to the correct answer (Miller et al. 2015). However, given that we are less interested in correctness or evaluation, we focused on the existence of revision not necessarily the direction to the correct answer. Practically, we focused on verbal statements that implied a change in any direction about either the correct answer or the science content. We included both verbal revision of a multiple-choice answer (e.g., “I used to think it was A, now I think it’s D.”) and verbal revision of an explanation for the science content (e.g., “I used to think only temperature influences amount of thermal energy, but now I think both mass and temperature influence thermal energy.”).
These four elements capture the kind of knowledge co-construction seen in these settings given the inherent constraints. Once the elements were defined, two researchers then independently coded all of the conversations and differences were resolved through discussion. In several cases, the substance of the disagreement was due to ambiguities in the transcript and in those cases three of the researchers listened to the audio together to resolve the disagreement.
After coding all the conversations, we also investigated whether different co-construction elements were present in peer discussion where student groups improved their performance from Q1 to Q1AD. To determine which groups showed improvement, we looked at mean Q1 and Q1AD scores for each student group. Groups were labeled according to whether their mean Q1 to Q1 AD scores increased (Q1 < Q1AD), decreased (Q1 > Q1AD), or remained the same (Q1 = Q1AD). If the group’s mean Q1 was equal to 100 %, the group was labeled “ceiling.”
Finally, to address potential concerns about clickers, namely conversations that are off-topic and conversations in which one student told the rest of the group an answer with no discussion, we further examined the conversations that included none of the co-construction elements described above. We systematically read through those conversations and examined whether these conversations included any mention of science content, and if they did, how the other students in the group responded to a student offering an explanation to the clicker question.