Aligning Teaching to Learning: A 3-Year Study Examining the Embedding of Language and Argumentation into Elementary Science Classrooms

  • Brian Hand
  • Lori A. Norton-Meier
  • Murat Gunel
  • Recai Akkus
Article

Abstract

How can classrooms become communities of inquiry that connect intellectually challenging science content with language-based activities (opportunities to talk, listen, read, and write) especially in settings with diverse populations? This question guided a 3-year mixed-methods research study using the Science Writing Heuristic (SWH) approach in cooperation with 2 universities, area education agencies, 6 school districts, 32 elementary teachers, and over 700 students each year. The participating teachers engaged in a yearly summer institute, planned units, implemented this curriculum in the classroom, and contributed to ongoing data collection and analysis. Findings demonstrate that critical embedded language opportunities contribute to an increase in student Iowa Tests of Basic Skills (ITBS) scores in science and language based on level of implementation particularly for elementary students who receive free and reduced lunch (an indicator of living at the poverty level).

Keywords

Argumentation Argument-based inquiry Elementary school science Literacy practices in science Science learning Science writing heuristic Teaching practices in science learning 

There is no science without language. Science cannot exist without some form of language, that is, one cannot explain old or construct new science knowledge without language (mathematical, graphical, verbal, or iconic). In this 3-year mixed-methods research study, the researchers engaged in an exploration of this very basic philosophical underpinning in relation to the implementation of the Science Writing Heuristic (SWH) approach, a form of argument-based inquiry, by teachers in elementary classroom settings. The focus was on how the language connections are not a “sidebar” activity, but are an integral essential element to the work of scientists as well as students taking up the work of scientists in classrooms. In attempting to re-frame the emphasis in the elementary classroom away from the teaching of science to a more student-oriented language and learning view of science, there was a need for the researchers to draw on a number of separate theoretical frames to build an approach to teaching science that could be adopted by all teachers. These theoretical frames draw on learning theory, argument, and language-to-learn knowledge bases as means to build a teaching in the service of learning approach to elementary science.

The Role of Language in Science

A major focus of argument-based approaches to inquiry is the importance and role of language in science. Norris & Phillips (2003) have clearly defined two essential senses of literacy that frame science. The first is the derived sense of literacy in which “reading and writing do not stand only in a functional relationship with respect to science, as simply tools for the storage and transmission of science. Rather, the relationship is a constitutive one, wherein reading and writing are constitutive parts of science” (p. 226). For Norris & Phillips, this is critical because these constituents are the “essential elements of the whole” (p. 226), that is, remove these language elements and there is no science. Science is not something that can be done without language. To this derived sense of science literacy, we would expand Norris & Phillips’ definition to include the different modes of representation. While this is implicit within reading and writing, there is a need to understand that different modes of science are integral to the concept of reading and writing, that is, science is more than just text. Other modes used by scientists to construct understanding include graphs, equations, tables, diagrams, and models.

The second essential sense of literacy is the fundamental sense of science literacy. For Norris and Phillips, the fundamental sense involves the “reasoning required to comprehend, interpret, analyze, and criticize any text” (p. 237). Importantly, they argue that science has to move past oracy and the oral traditions because “without text, the social practices that make science possible could not be engaged with” (p. 233). The important recording, presentation, and re-presentation of ideas and debates and arguments that constitute the nature of the discipline are not possible without text. These two essential senses of literacy are critical to the development of scientific literacy.

The authors (Hand, 2008; Norton-Meier, Hand & Ardasheva, 2013; Norton-Meier, Hand, Hockenberry & Wise, 2008) in discussing the role of language in science have suggested that students learn about language, while they learn through language while experiencing language (adapted from Halliday, 1975). Halliday’s work provides a framework that deals with the concept of authentic language use in the science classroom. Rather than having to learn the language of science separate to its use, that is, having to learn the vocabulary of the topic before actually doing the topic, his work suggest that students will learn the language of the science topic through actively using this language while they are experiencing how it should be used. Yore & Treagust (2006) have highlighted this engagement of student language by suggesting that there is, in fact, a need to recognize that similar to English Language Learners, there is Science Language Learning (ScLL) where students, in fact, may engage with three different forms of language associated with science language. These three forms include the home language of the student, the language of instruction, and the language of science. Recognizing these different forms of language requires that we provide opportunities for students to move between these forms in the context of their learning, further highlighting the work of Halliday. Importantly, such movement between these language forms will require students to engage with multiple forms of negotiation as a means to construct understanding.

For Klein (2006, p. 171), negotiation needs to represent a balance between the denotative meanings of science as outlined in first-generation cognitive science and the fuzzy, perceptual reasoning language use of second-generation cognitive science. As such, he believes that students need to be provided with opportunities to negotiate between everyday, narrative speech and scientific explanation and argumentation through combined talk and writing experiences; have opportunities to write informal, speech-like texts and narrative–argument blends; engage in the pragmatic and dialogical aspects of argumentation; and should be explicitly taught science text genre. We believe that the Science Writing Heuristic (SWH) approach encompasses the features emphasized by Klein.

The Science Writing Heuristic Approach

The Science Writing Heuristic (SWH) approach, a form of argument-based inquiry, is intended to provide students with opportunities to be engaged in doing science through understanding both the argument structure of science and the importance of language in science. As such, the SWH approach is intended to provide students with opportunities to “grasp the practices” of science (Ford & Forman, 2006). This means that students are required to pose questions, gather data, make claims, produce evidence to support their claims, check to see what others say (both peers and experts), and to reflect on how their ideas have changed. For this approach to succeed, teachers have to be prepared to enter into, or provide space for, a series of different negotiations with students ranging from individual to small group to whole class. There are two templates that have been developed to support this approach (Table 1).
Table 1

The two templates for the SWH: The teacher template and the student template

The science writing heuristic, Part I

The science writing heuristic, Part II

A template for teacher-designed activities to promote laboratory understanding

A template for students

1. Exploration of pre-instruction understanding through individual or group concept mapping

1. Beginning ideas—What are my questions?

2. Pre-laboratory activities, including informal writing, making observations, brainstorming, and posing questions

2. Tests—What did I do?

3. Participation in laboratory activity

3. Observations—What did I see?

4. Negotiation phase I—writing personal meanings for laboratory activity (for example, writing journals)

4. Claims—What can I claim?

5. Negotiation phase II—sharing and comparing data interpretations in small groups (for example, making group charts)

5. Evidence—How do I know? Why am I making these claims?

6. Negotiation phase III—comparing science ideas to textbooks for other printed resources. (for example, writing group notes in response to focus questions)

6. Reading—How do my ideas compare with other ideas?

7. Negotiation phase IV—individual reflection and writing. (for example, creating a presentation such as a poster or report for a larger audience.)

7. Reflection—How have my ideas changed?

8. Exploration of post-instruction understanding through concept mapping.

 

While the SWH approach recognizes the need for students to conduct laboratory investigations that develop their understandings of scientific methods and procedures, the teachers’ template also seeks to provide a stronger pedagogical focus for this learning. In other words, the SWH approach is based on the assumption that science-writing genres in school should reflect some of the characteristics of scientists’ writing, but also be shaped as pedagogical tools to encourage students to “unpack” scientific meaning and reasoning. The SWH approach is intended to promote both scientific thinking and reasoning in the laboratory, as well as metacognition, where learners become aware of the basis of their knowledge, and are able to monitor more explicitly their learning. Because the SWH approach focuses on canonical forms of scientific thinking, such as the development of links between claims and evidence, it also has the potential to build learners’ understandings of the nature of science, strengthen conceptual understandings, and engage them in the authentic argumentation process of science, that is, in authentic practices of science (Ford, 2008).

The SWH approach emphasizes the collaborative nature of scientific activity, that is, scientific argumentation, where learners are expected to engage in a continuous cycle of negotiating and clarifying meanings and explanations with their peers and teacher. In other words, the SWH approach is designed to promote classroom discussion where students’ personal explanations and observations are tested against the perceptions and contributions of the broader group, and to nature. Learners are encouraged to make explicit and defensible connections between questions, observations, data, claims, and evidence. When students state a claim for an investigation, they are expected to describe a pattern, make a generalization, state a relationship, or construct an explanation, that is, they are required to both construct knowledge and critique this knowledge (Ford, 2008).

In the SWH approach, students are required to construct knowledge about nature through the practices of science. Of critical importance for the researchers is not in determining how the adoption of such practices impacts on students’ understanding of argument, but rather, how does the use of argument impact on students learning of science. Thus, teaching is viewed as being in the service of learning.

The Study

Previous research using the SWH approach has centered on studies at the middle/high school level and the freshman university level. These studies have shown that the level of implementation of the approach has an impact on student scores on standardized tests (Greenbowe & Burke, 2008; Akkus, Gunel & Hand, 2007), on closing both the achievement and gender gaps in freshman chemistry (Poock, Burke, Greenbowe & Hand, 2007), as well as promoting the use of high-order cognitive thinking skills in all students including low-achieving science students (Grimberg & Hand, 2009). One study has shown that high-level implementation of the SWH approach has a significant advantage over high-level traditional teaching in closing the gap between high- and low-achieving science students (Gunel, 2006). The intent of this study was to examine the applicability of this approach with elementary students and to see if the results for the higher-grade levels could be replicated with younger children. Thus, the researchers were guided by the questions:
  1. 1.

    Does the level of implementation of the SWH approach impact on elementary students’ (grades 3, 4, 5, and 6) grade growth scores measured by the Iowa Test of Basic Skills (ITBS) for the science and language tests?

     
  2. 2.

    Does the SWH approach, a language-embedded approach to science inquiry, impact learning for low socio-economic status (SES) students?

     

Methodology

This study employed a mixed methodology to examine the complexity of the research questions because they cannot be satisfactorily asked and answered with a single-research approach. Mixed methodology research is a term used to describe “studies or projects that employ at least one quantitative and one qualitative method to produce knowledge claims” (Smith, 2006, p. 459). Howe (2003) points out that qualitative and quantitative methodologies are not distinct, but part of a whole thinking process, and thus, mixing methodologies is simply what researchers already do—mixing both qualitative and quantitative knowing.

Research Setting and Participants

The researchers analyzed the results of a 3-year professional development project focused on elementary teachers to develop strong understandings of science teaching and of pedagogical practices to implement inquiry-based approach. Even though the majority of the participants in the first year continued the project in the second and third years, there were few teachers who dropped out the study at the end of the first and second years, and new teachers joined in the second and third years. There were 31, 32, and 32 teachers, along with their students, in year 1, 2, and 3, respectively, across K-6 grade levels from five school districts in the Midwest USA. Four of the school districts were in rural settings with two of those schools receiving a designation by the federal government as rural poverty areas. The fifth school district was a large urban district. Table 2 shows the distribution of the teachers and students across grade levels and 3 years.
Table 2

Distribution of teachers and students across 3 years, with implementation level

Grade level

Teacher implementation levels

Year 1

Total 1

Year 2

Total 2

Year 3

Total 3

L

M

Tr* (Sts**)

L

M

Tr (Sts)

L

M

H

Tr (Sts)

Pre-school

 

1

1 (13)

 

1

1 (13)

   

Kindergarten

1

 

1 (19)

1e

1

2 (38)

2

1e

 

3 (50)

1

  

  

3

  

3 (59)

2

4

1

5 (98)

2e

4

6 (112)

3

3

 

6 (111)

3

2

2b

4 (82)

1

2

3 (54)

 

3

 

3 (70)

4

4a

3c

7 (153)

1e

5

6 (123)

2

4

 

6 (119)

5

5

2

7 (231)

1

6

7 (237)

 

2

3

5 (214)

6

3

3d

6 (184)

3

4f

7 (205)

1

4

1

6 (157)

Total

19

12

31 (780)

9

23

32 (782)

11

17

4

32 (780)

L low implementation, M medium implementation, H high implementation

aTwo fourth grade teachers dropped out

bOne third grade teacher dropped out

cOne fourth grade teacher dropped out

dOne sixth grade teacher dropped out

eOne teacher is new

fTwo teachers are new

*Number of teachers

**Number of students

The teachers in this research study were all white—29 females and 3 males. The teachers volunteered for participation in five school districts and represented varied levels of teaching experience (2 – 23 years). The students from the study demographically can be described in the following ways: 69 % White, 14 % Black, 12 % Hispanic, 4 % Asian, 1 % American Indian, 52 % male, 48 % female, 11 % of the students receive special education services, and 23 % receive free or reduced lunch (an indicator of living at the poverty level in the USA). The participating teachers were involved in a 3-year implementation and analysis cycle, which involved a yearly 10-day summer institute focused on examining the SWH approach in the following ways:
  1. 1.

    Science content knowledge update (teacher as learner);

     
  2. 2.

    Learning theory knowledge update;

     
  3. 3.

    Pedagogical content knowledge update;

     
  4. 4.

    Embedded language practices.

     

More details about the professional development training contents are provided in Appendix A as electronic supplementary material.

In addition to the summer workshop, the teachers received 3 days of professional development during the school year focused on planning units, reflecting on implementation, discussing problems and obstacles, and exploring language strategies. The underlying theme to all of the professional development work with teachers was—learning—what is learning and how do teachers support the learning of every student in the classroom. Each teacher was visited at least two times in a year by a project staff, area education content area consultants, or researchers, in their classrooms for on-site support, field note taking, modeling, and co-teaching.

Data Collection

Two main data sources were used to examine teacher implementation and student learning for the purposes of this research study. The first data source was quantitative and focused on the use of the Iowa Test of Basic Skills (ITBS) scores (a standardized test adopted by the school districts) for students grade 3 and up. The ITBS was developed by the University of Iowa and provided as a service to schools by the Riverside Publishing Company. The tests were first administered in 1935 in Iowa to improve instruction. Since then, along with almost all schools in Iowa, the ITBS tests have become a widely accepted instrument to measure educational progress annually in the USA. Tests offer educators a diagnostic look at how their students in kindergarten through eighth grade are progressing in key academic areas such as science, math, social studies, and language, and offers diagnostic data that can be used to create intervention groups and to drive curricular decisions. More about ITBS can be found at: http://www.riversidepublishing.com/products/itbs/details.html.

The second data source was qualitative in nature and involved observing teachers in classrooms as they implemented the SWH approach. For each teacher, two types of observations occurred, on-site and videotaped observations. During the on-site observations, an observer who had SWH teaching experience was physically present in the classroom, following the teacher, and taking field notes on teacher-student interactions. After the lesson, the observer had a short debriefing session with the teacher, where constructive feedback was provided after the teacher’s identification/self evaluation of strengths, weaknesses, and difficulties with implementation. Each teacher was videotaped twice during implementation and the videos were used to make a detailed analysis of implementation level by independent observers. The independent observer was someone in the project who did not interact directly with the teacher. The on-site and videotape observations were conducted by three graduate students, two consultants (one with a specialization in literacy and the other in science), and two professors.

Data Analysis

The intent of this study is to provide detailed analysis about the impact of implementation level on ITBS grade growth scores of science and language as well as the impact of implementation on grade level growth scores when social-economic status is taken into consideration. Therefore, we will briefly state the qualitative findings of implementation level and move on to quantitative findings.

Qualitative Data Analysis

The analyses consisted of the constant comparative method of data analysis. In terms of the qualitative data, the on-site and videotape observations were analyzed multiple times in order to identify the teachers’ levels of implementation through the use of an observation matrix developed for the project (see Appendix B, in electronic supplementary material). The criteria matrix consisted of four major areas of pedagogical practice. The criteria placed a teacher in one of the three categories for defining the quality of the implementation of the SWH approach: low, medium, and high. The teachers who continued the project for 3 years were also followed in terms of the changes in their implementation levels.

The implementation matrix is based on the theoretical facets of the SWH approach. First of all, the dialogical interaction among students is crucial for learning in the SWH approach because students learn by arguing with peers and themselves. For this, they need an opportunity to see different perspectives. Therefore, the teacher’s attempt to create such an environment is one of the observations during the implementation. Secondly, the SWH approach focuses on learning in the classroom, not teaching by the teacher. This means that the authority of the knowledge occurring in the classroom is the argumentation made by the students and the teacher collaboratively, not the teacher. With this criterion, teachers who attempt to lecture all the time are captured. Third, teachers’ making connections among different disciplines, especially language connection, is important during implementation because without language, there is no science. This means that we do and learn science by any form of language. This should be integrated in the classroom. Lastly, science argumentation is based on the question-claim-evidence structure. Therefore, teachers should monitor students’ argumentation structure and try to support their scientific thinking. Students in such a classroom are given opportunities to create their own questions, to test their initial ideas, to make scientific claims, and to construct evidence using their observations and data.

The type inter-rater agreement adopted for this study was percentage of absolute agreement calculated by “ … the number of times raters agree on a rating, then divides by the total number of ratings. Thus, this measure can be vary between 0 and 100 %.” (Graham, Milanowski & Miller, 2012, p. 7) In this study, percentage of absolute agreement between any pairs of observers for teachers’ level of implementation was from 90 to 95 %. More information about agreement calculation can be found at Tinsley & Weiss (2000). Since reaching the high agreement rate gives rise to the reliability, it may also indicate systematic diagnostic/measurement/scoring error. In this study, in order to formulate more rigorous reliability in coding and to ensure validity, researchers worked to eliminate variations in the extraneous circumstances of the measuring process. The extraneous variations such as lack of understanding related to the coding task and context of data, scaling, unit of analytical process, were worked through in the coding team prior to the scoring in order to enhance judgment dimension of the reliability for the video the analyses and study (Krippendorff, 2011).

Dialogical interaction is the first of the four criteria. Types of questions asked by teacher and students, teacher’s response to students’ answer and questions, and the direction of communication (e.g. from teacher to student) are of essential importance for creating dialogical interaction. The questions teachers ask in classrooms can either promote or limit classroom conversation.

The second criterion is focus of learning. Focus of learning was defined in the SWH approach as creating a non-threatening environment, choosing an inquiry investigation, and promoting public sharing of knowledge, an important step away from traditional science classroom practice. Teachers allow students to ask their own investigation questions, build models, and support their claims using the evidence found during the investigation.

Unit preparation and making connections is the third criterion. Unit preparation refers to identifying the big ideas of the units, which reflects teachers’ understanding of the content knowledge. In deciding the big ideas, teachers are engaged in an inquiry about their students’ prior knowledge on which students build new concepts. Making those connections requires centering the concepts of the units on the big ideas and students’ prior knowledge and supporting students in learning scientific language.

The last criterion is science argumentation. For the SWH approach, it is crucial that teachers encourage students to make a scientific argument among themselves by providing evidence for their knowledge claims. One of the roles of the teacher in an SWH classroom is to create dialogical interaction (which is the first criterion) to promote scientific debate. Students argue based on the big ideas negotiated in the classroom.

Quantitative Data Analysis

Analysis of the quantitative data was an essential element to understanding one piece of student learning related to our research questions. In terms of the quantitative analysis, analysis of covariance (ANCOVA) models was estimated to control for other variables such as mathematic achievement that might impact students’ science and language achievement (Agresti & Finlay, 1997). To ensure the accuracy of the data that were collected, both frequency distributions and descriptive statistics were obtained using the SPSS Frequencies procedure (Mertler & Vannatta, 2002). The SPSS Casewise Diagnostic procedure was employed to examine whether outliers possibly affect the results of the study (Levine & Roos, 2002). Also, for all models, Levene’s tests yielded non-significant equality of variance, which means that the assumption was not violated. The normality assumption is not too crucial for large samples; therefore, no testing was done for this assumption. Homogeneity of regression slopes was violated only for the year (independent variable) when the science scores were the dependent variable.

Given the sample size at each grade level and in order to examine the impact of the teacher implementation effect regardless of grade level, students’ grade equivalent growth scores were used instead of raw scores. The grade equivalent (GE) is a number that describes a student’s location on an achievement continuum. The GE is a decimal number that describes performance in terms of grade level and months. For example, if a sixth grade student obtains a GE of 8.4 on the vocabulary test, his score is likely the one a typical student finishing the fourth month of eighth grade would likely get on the vocabulary test. To calculate the growth, we also obtained the date of the test. For example, suppose that the test is taken after 6 months of schooling, indicating that the student’s grade equivalency is 6.6. Thus, the grade equivalent growth (GEG) for this student would be 1.8 (8.4 – 6.6).

In order to investigate the two research questions, as stated above, this study used ANCOVA models where the implementation level, year, and SES status served as independent variables. The first research question was addressed by analyzing the effects of the year and implementation level on the GEG science and language scores. The second research question was addressed by investigating differentiation of the GEG science and language scores within the implementation levels based on students who receive free/reduced lunch (a measure of poverty and those who do not. While the interaction effect results within ANCOVA gives some level of indication for such differentiation, the significance level is mainly dependent upon cross differentiations (implementation levels and SES status are all taken together). Yet, differentiation of the GEG scores between the SES status for the different implementation levels needed closer examination. Therefore, the authors applied pairwise comparisons (post hoc) to tease out such discriminations even when the interaction effect appears non-significant within the ANCOVA results.

In this study, we reported effect sizes using the Cohen’s d index, which is widely used in social science. There are three advantages of reporting effect sizes. First, reporting effect size makes meta-analyses possible for a given report. Second, effect size reporting allows a researcher to determine more appropriate study expectations in future studies. Third, reporting and interpreting effect sizes facilitate assessment and comparison of study results across existing related studies (Wilkinson & Affairs, 1999). The criteria for identifying the magnitude of an effect size are as follows: (a) a small effect size is between 0.2 and 0.5 standard deviation units; (b) a medium effect size is between 0.5 and 0.8 standard deviation units; and (c) a large effect size is 0.8 or more standard deviation units (Rosenthal & Rosnow, 1984; Sheskin, 2004). Effect sizes smaller than 0.2 standard deviation units are named trivial (Kulik, 2002).

Results

The results of the study consisted of two parts: the analysis of the teachers’ implementation levels and the impact of such implementation levels on student ITBS performance. In other words, findings from the first part of the qualitative analysis and teachers’ levels of implementation were used in the statistical analysis as an independent variable in the second part of the analysis. Further, in the second quantitative part of the analysis, the impact of implementation level on the ITBS performance when student SES status was taken into consideration was also investigated. Therefore, the results will be reported accordingly.

Qualitative Results

Based on the analysis of the videotapes of the teachers and the field notes kept during the on-site observations, each teacher was placed on an implementation scale of three levels (low, medium, and high) depending on how well they implemented the SWH. This analysis indicated that regardless of their grade levels, there were 19 teachers in low- and 12 teachers in medium-level implementation in the first year. In the second year, regardless of the grade level or being a new teacher, there were 9 low- and 23 medium-level implementers. No teacher in either years 1 or 2 reached the high-level of implementation. However, in the third year, 4 teachers reached this high level of implementation while 11 and 17 teachers were at low and medium level, respectively. (see Table 2 in the “Research Setting and Participants” section for the distribution of teachers according to level of implementation across grades and 3 years of the study.)

The teachers who continued the project for 3 years were also followed in terms of their implementation change over the 3 years. The results indicate that from years 1 to 2, one teacher regressed in her implementation, five teachers stayed at the low level, 12 teachers moved from low to medium level, and 8 teachers stayed at the medium level. Table 3 displays these results. The teacher who regressed in her implementation had personal problems that restricted her implementation of the SWH approach, so she ranked as low in the second year.
Table 3

Teacher implementation level shifts from years 1 to 2 and years 2 to 3

Grade level

Teacher implementation level shifts across years

Med → low

Low → low

Low → med

Med → med

Med → high

Pre-school

   

(1) –

 

Kindergarten

(–) 1

 

(1) –

  

2

(–) 1

(1) –

(3) 1

(1) 2

 

3

(1) –

 

(2) –

(–) 2

 

4

  

(2) 1

(2) 4

 

5

 

(1) –

(4) 1

(2) 1

(–) 3

6

 

(3) 1

(–) 1

(2) 3

(–) 1

Total

(1) 2

(5) 1

(12) 4

(8) 12

(–) 4

Parentheses show the shifts from years 1 to 2

Table 3 also illustrates implementation shifts from years 2 to 3, which also includes the teachers who participated in the second year of the project. While two teachers regressed and one teacher stayed at a low level, most of the teachers either improved their implementation or stayed at the medium level. Eight teachers from each grade level showed movements toward a better implementation of the SWH approach (low to medium and medium to high).

Quantitative Results

Quantitative analyses included qualitative findings stated above which was essential to the mixed methodology design. Several ANCOVA models were estimated to examine the effects of implementation level and year on students’ grade equivalency growth (GEG) in ITBS science and language and to explore the relationship between level of teacher implementation and SES status of students on the GEG scores across 3 years. Two separate sets of ANCOVA models were conducted for the GEG science and language. The first set of the models targeted the year and implementation effect on the GEG science and language; whereas, the second set investigated effects of SES and individualized education program (IEP) status of students, year, and implementation level on the GEG science and language. However, ANCOVA models with either the IEP status or the year were not estimated since the cell sizes became very small (e.g. 10) in some implementation levels. Therefore, the researchers only conducted models to estimate effect of SES status and implementation levels. In short, due to cell size limitations, in this study, we investigated the effect of SES status and the implementation level on the GEG science and language scores regardless of the year.

In both sets of analyses, ITBS mathematics scores were used as the covariate. The reason for choosing ITBS mathematics scores was twofold. First, ITBS mathematics scores were found to be one of the strongest covariates during the preliminary analysis. Second, other available ITBS scores such as ITBS reading has strong correlation with the independent variable implementation level.

Effect of Implementation Level and Year on Science Grade Equivalency Growth Scores

In the models of the first set, the students’ ITBS mathematics scores were used as the covariate and year and implementation level were used as independent variables. The first model estimated the effects of year and implementation level on students’ grade equivalency growth (GEG) of science scores. The ITBS mathematics score as a covariate was significant in this model (F (1, 1480) = 710.954, p = .001, η2 = .324). The model yielded a significant main effect of implementation level (F (2, 1480) = 8.400, p = .001, η2 = .011). That is, the growth of the students in medium-implementation classrooms was significantly greater than that of students in low implementation classrooms for each year of the project. Further pairwise comparisons were made within each year. In this model, the year main effect (F (2, 1480) = 1.645, p = .193, η2 = .002) and the interaction effect (F (2, 1480) = .517, p = .597, η2 = .001) were not significant. The adjusted R2 for this model was .329. Table 4 shows the means, standard errors, Cohen’s d effect sizes, and t values in parentheses.
Table 4

Adjusted means and Cohen’s d effect sizes for science

 

Imp. level

Adj. mean

Adj. std. error

Number

Cohen’s d

Med – low

High – med

High – low

Year 1

Low

0.782

0.075

337

0.172*

(2.045**)

Med

1.020

0.089

243

Year 2

Low

0.541

0.127

118

0.293*

(2.765**)

Med

0.946

0.073

358

Year 3

Low

0.558

0.187

55

0.297*

(1.998**)

0.107

(.661**)

Med

0.970

0.087

255

−0.189

(−1.718**)

High

0.707

0.126

122

*Significant mean difference at p < 0.05 level

**t values for the pair wise comparisons

Effect of Implementation Level and Year on Language Grade Equivalency Growth Scores

The second model of the first set was conducted to estimate the effects of year and implementation level on the GEG language scores using the ITBS mathematics scores as the covariate. Even though the interaction effect (F (2, 1412) = 1.405, p = .246, η2 = .002) was not significant, there were significant main effects for the year and implementation level (F (2, 1412) = 12.817, p = .001, η2 = .018 and F (2, 1412) = 5.343, p = .005, η2 = .008, respectively), indicating that there was a decline as the year progressed and a higher level of implementation impacted the language scores. Pairwise comparisons yielded that students’ grade equivalency growth in language was higher in medium-implementation classrooms compared to low-implementation classrooms. While the differences between medium and low levels of implementation were not significant in the first 2 years, medium and high teachers’ students significantly outperformed the students of low level of implementation (d = .384 SD and d = .457 SD, respectively). There was no difference between medium and high teachers in the third year. The adjusted R2 for this model was .267. These results can be seen in Table 5.
Table 5

Adjusted means and Cohen’s d effect sizes for language

 

Imp. level

Adj. mean

Adj. std. error

Number

Cohen’s d

Med – low

High – med

High – low

Year 1

Low

0.960

0.084

325

0.113

(1.287**)

Med

1.130

0.102

216

Year 2

Low

0.631

0.138

119

0.102

(.969**)

Med

0.785

0.079

363

Year 3

Low

0.140

0.203

55

0.384*

(2.560**)

0.457*

(2.759**)

Med

0.717

0.098

235

0.075

(.643**)

High

0.830

0.146

107

*Significant mean difference at p < 0.05 level

**t values for the pair wise comparisons

Effect of Implementation Level and Social-Economic Status On Science And Language Grade Equivalency Growth

For the second set of analysis, two separate 3 × 2 ANCOVA models were estimated to investigate the impact of implementation level, and SES status on students’ ITBS science and language GEG scores by using the ITBS mathematics as the covariates. The covariate was significant in these models, F (1, 1481) = 655.760, p < .001, η2 = .307 and F (1, 1413) = 457.381, p < .001, η2 = .245, respectively. In terms of the science GEG scores, the results also indicated significant main effects for the independent variables of implementation level, F (2, 1481) = 6.091, p = .002, η2 = .008; and of SES, F (1, 1481) = 14.696, p < .001, η2 = .010). The interaction effect (F (2, 1481) = .381, p = .683, η2 = .001) was not significant in this model. Similarly, for the language GEG scores, the results indicated significant main effects for the independent variable of SES status, F (1, 1413) = 9.792, p = .002, η2 = .007. The other independent variable implementation level, F (2, 1413) = .210, p = .811, η2 < .001, was not significant in the model. The interaction effect was also not significant in this model (F (2, 1413) = .003, p = .997, η2 < .001).

For science scores, pairwise comparisons yielded that there were some significant grade equivalency growth differences between low SES and higher SES students within implementation levels. Table 6 shows the adjusted mean scores, standard errors, t values and Cohen’s d values. While higher SES students scored significantly higher than students with low SES within low and medium teacher implementation levels, such significant differences were not noted within the high teacher implementation group. Further, when higher SES and low SES students’ scores were compared in terms of Cohen’s d values within the different implementation levels, the Cohen’s d values for low, medium, and high implementation levels were 0.4, 0.3, and 0.2, respectively. That is, when considering the GEG score in science within the low-implementation group, the effect size difference between higher and lower SES status students was 0.4 SD in favor of higher SES students. The effect size differences within medium and high implementation classrooms were 0.3 and 0.2 SDs in favor of higher SES students (in other words, those who do not live at the poverty level). Similarly, in terms of the language GEG scores, pairwise comparisons yielded that there were some significant grade equivalency growth differences between higher SES and low SES students within implementation levels. Table 6 also shows the adjusted mean scores, standard errors, t values and Cohen’s d values in parentheses.
Table 6

Adjusted means-Cohen’s d values for implementation-SES effect on ITBS GEG science and language

Implementation level

SES status

Adj. mean

Adj. std. error

Number

t value

Cohen’s d value

Low

Standard

0.858 (0.907)

0.072 (0.081)

358 (345)

4.014* (2.601*)

.4 (.3)

Free/red

0.327 (0.528)

0.111 (0.121)

152 (154)

Medium

Standard

1.099 (0.974)

0.056 (0.063)

612 (570)

4.175* (3.406*)

.3 (.3)

Free/red

0.660 (0.580)

0.089 (0.097)

244 (244)

High

Standard

0.798 (0.956)

0.147 (0.175)

88 (75)

1.032 (1.209)

.2 (.3)

Free/red

0.521 (0.571)

0.235 (0.266)

34 (32)

Parentheses show language results

*Significant mean difference at p < 0.05 level

Before the discussion of the results, it is important to illuminate the limitations of this study. Aside from all efforts to increase the validity and the reliability of the rater training and the coding process, the scoring calls for rater and scoring matrix limitations. In addition, this study employed purposeful sampling by reaching out to administrators who were interested in participating in the SWH project and the administrators picked which elementary teachers would participate in the 3-year project. The school districts that chose to participate in the study as well as the teachers and students who work and attend the participating schools limit the findings of the study. Finally, the quantitative measurement is limited with the ITBS test and in future studies should be expanded to include other measures not limited by a single standardized test (see Lamb, Cavagnetto & Akmal, 2014).

Discussion and Implications

In considering the results from this study, the authors are encouraged that an emphasis by teachers on implementing argument-based inquiry in their elementary classrooms can lead to interesting developments in student learning. The results presented here indicate that a focus on teaching in the service of learning can increase students’ science and language scores depending on the teacher’s level of implementation. In addition, this data contributes to an ongoing discussion in the educational community about what might close the achievement gap. In this study in particular, we were able to focus on a measure of poverty (a student’s free/reduced lunch status), and find that in high levels of implementation, the gap between the students was diminishing. This leads us to ask, “Why might this phenomenon is occurring in the high implementation classrooms?” In reviewing the results from this study, the researchers would like to reiterate the context of the study. Teachers and students were involved in using the SWH approach to learn science with the argumentation and language strategies embedded as integral components within the SWH approach. The results from the study do indicate three critical issues that emerged from our analysis.

The first is related to teacher implementation. As part of the professional development activities, teachers were asked to shift their orientation on learning, argument and language use within their classrooms. Not all teachers were able to achieve strong implementation of the necessary pedagogical elements for a shift to a “grasp of practice” orientation. Importantly, there was not a consistent pattern for lack of implementation. However, one consistent thread among these low-implementation teachers was their reluctance to shift their locus of control of knowledge. Who controls learning? Teachers in low-implementation classrooms believe they control learning and knowledge. These teachers struggled with recognizing that science is a constructed activity, and that students need opportunities to be part of the construction process—even though they are constantly constructing regardless of what the teacher was doing. It is also important to note that in the results, student scores in science shifted based on implementation level more quickly than the language scores. It took the entire 3 years to see this shift happen for teachers so it is important to consider the types of professional development experiences will best support the ongoing learning of teachers. In this study, a 3-year implementation phase was critical to success.

The second critical issue arising from the study was the practice of argument and how this impacts on results. Teachers who began to implement the SWH approach well became comfortable in promoting strong negotiation opportunities for students. As students generated their questions, investigated these and arrived at claims and evidence, they were required to constantly negotiate the strengths and outcomes of these activities. An essential component of this practice was that the data was their own, that is, the data were derived as a consequence of their own investigations and thus were embedded in the science inquiry rather than being artificial as from a textbook. Students who were in classrooms with teachers successful at promoting these strategies scored much better on the Iowa Test of Basic Skills.

The third critical element arising from the study is the writing opportunities presented to students in completing their inquiry activities. Within the SWH approach, writing is a critical form of language. At the completion of the public negotiations over claims and evidence, students were required to complete an individual report. Again, students were required to link questions, claims, and evidence together to construct a sound argument from their inquiry activities. Having participated in the public negotiation of claims and evidence, students were expected to use the feedback and develop their own argument as a consequence of this final negotiating opportunity. The improving scores on the ITBS language results note the outcome of these efforts.

When considering the implications of this study for future research and practice, it is essential to return to the two theoretical areas underpinning this research: authentic embedded experiences and the senses of science literacy. Embedding authentic science and language activities into the science curriculum means that students have opportunities to build their derived and fundamental senses of science literacy. The fundamental sense of science literacy requires that students understand the practices of science or at least have opportunities to “grasp” these practices. Science inquiry is not something that is provided by teachers to students, but rather needs to be an authentic activity that requires students to practice science argumentation through constructing questions, claims, and evidence and then having to publicly debate and defend their position. The SWH approach is centered on providing these opportunities to students to learn about language, while they learn through language all while living the language of science, and the resulting student performances on ITBS would indicate the benefits of this argument-based inquiry approach. Future research particularly on the nuances of teacher practices at the different levels of implementation will aid in understanding how to support new and established teachers to continue to contribute to our understanding of how to teach in the service of learning.

Acknowledgments

This project was funded through a Math-Science Partnership grant and the National Science Foundation (ESI - 0537035). The opinions and interpretations herein are solely that of the authors.

Supplementary material

10763_2015_9622_MOESM1_ESM.doc (33 kb)
ESM 1(DOC 33 kb)
10763_2015_9622_MOESM2_ESM.doc (36 kb)
ESM 2(DOC 35 kb)

Copyright information

© Ministry of Science and Technology, Taiwan 2015

Authors and Affiliations

  • Brian Hand
    • 1
  • Lori A. Norton-Meier
    • 2
  • Murat Gunel
    • 3
  • Recai Akkus
    • 4
  1. 1.The University of IowaIowa CityUSA
  2. 2.University of LouisvilleLouisvilleUSA
  3. 3.Faculty of Education, Department of Elementary Education, Chair Primary Education Program Ziya Gokalp CaddesiTED UniversityAnkaraTurkey
  4. 4.College of Education, Department of Mathematics EducationAbant Izzet Baysal UniversityBoluTurkey

Personalised recommendations