The research design can be understood as a series of design experiments with a pre-test post one-group only design. The effects of four homework assignments were examined in two groups of 25 students (see Table 1). In each group, two pre-tests were carried out to set a baseline of time on task and the level of student participation. In these two lessons, students were provided with homework assignments in a regular manner: a combination of text reading and textbook assignments. In both groups, four post-tests were administered; one in each instructional strategy, and a delayed post-test was administered for group 1 only. The study was carried out in teaching history.
Table 1 The research design
Intervention
The four homework assignments were developed in a pilot study. Eight students, five secondary school teachers, and two teacher trainers were interviewed on design principles of effective homework assignments. These interviews revealed four main design principles that were used to setup the four homework assignments of the current study:
-
Students benefit from the possibility to choose their assignments. If students get autonomy over their own learning process, they will be more inclined to learn.
-
Students should be aware of the aim of each homework assignment. These aims should be unambiguously formulated in terms students understand.
-
Homework assignments should be straightforward. Teachers need to be able to apply the instructional activity easily and students should be able to complete the assignments in time.
-
Homework should be preparatory for the lesson. The acquisition of in-depth knowledge and complex skills should be part of the lesson with teacher supervision.
Prior to the research, the four homework assignments were piloted with a panel of five teachers, including a teacher trainer, to examine if it is doable and similar in difficulty.
Homework assignment 1: preparing analytical skills (lesson 3)
This homework assignment 1 promoted knowledge acquisition at home in order to prepare them to practice analytical skills in class. The strategy included four phases, two at home and two in class. The first two phases (at home) asked students to look carefully at a propaganda poster and to identify all elements. This means answering questions as; ‘what is happening in the poster’, ‘who are the people in het poster’, and ‘do you recognize any symbols used’. The last two phases (in class) asked students to examine the goal and the technique of the poster in order to answer explanatory and analytical questions. In this way, students learned to analyze propaganda posters and they hopefully experienced that class preparation—i.e., going through all phases—resulted in a more profound analysis.
Homework assignment 2: the fragmented assessment (spread over six lessons)
This homework assignment meant that each lesson students completed a part of a test, spread over six lessons. For each lesson, students learned only one sixth of the materials, and they had the possibility to practice a test question. In class, they had the opportunity to ask questions before completing the part of the test about the materials they studied at home.
Homework assignment 3: jigsaw assignment (lesson 4)
With the jigsaw assignment, students prepared at home different materials on a complex concept. For example, the concept of National Socialism is spread over four questions (What is an ideology, and what are the characteristics of fascism, racial doctrine and Lebensraum). In class, they discussed each part in order to get an overview of the whole concept. Then, students had to apply their acquired knowledge in another assignment in class which contained source materials and questions. Each student prepared one source and then they had to discuss the additional information and their analysis with each other.
Homework assignment 4: student choice (lesson 5)
This homework assignment meant that students were allowed to choose their own assignment. All assignments were grouped on topic and difficulty. Students started at home and completed the assignments in class. Students could choose for example, to analyze propaganda posters or to analyze a part of film ‘The great dictator’. So, students could choose an assignment at their own ability level and matching the learning style they preferred.
Participants
In this study, two groups of 50 grade 11 students in total (group 1, 22 students; group 2, 29 students) participated. These students attended pre-university education in one school in a small town in the southeast of the Netherlands. The students were 16 or 17 years old and 35 of them were female.
Data and measures
Data was gathered by means of class observations and a self-report questionnaire. Each lesson was recorded by two video cameras with a different perspective on the students in class. With the observational data, four variables were measured: students’ time on task in class, their level of student participation, the questions students asked in class and the teaching formats applied in class. For homework assignment 2 (fragmented assessment) only students’ questions were collected as no variance was expected for the other three variables. A delayed post-test was administered in lesson 7 for group 1 only. Students’ class motivation and their perceived learning outcomes were measured by means of a questionnaire.
Time on task
The coding units were 3 × 6 units of 30 s of each lesson. This means that there were eighteen 30-s coding units per lesson. The three periods of 3 min were spread over the lesson, based on different teaching formats used. Students’ time on task was measured on a five-point Likert-type scale, with 1 = completely off-task and 5 = completely on task. Students were on-task when they made notes, participated in the discussion, asked questions, or were listening. Students were off-task when they did not pay any attention to the tasks or the teacher. The time on task was registered for each individual student in class. Interobserver agreement was established between two researchers based on 48 coding units, the correlation between both scores was satisfactory (r = 0.74).
Level of participation
For the level of student participation in class, we used the same coding units as with time on task. Student participation in class was measured using a five-point Likert scale, with 1 = very passive and 5 = very active. Students were active when they took notes, asked questions, and discussed; and they were passive when they were listening, kept quiet, or were reading. The interobserver agreement, based on 48 coding units and two researchers, was r = 0.72.
Student questions
For each lesson, all student questions were coded with the question as coding unit. Two main categories of student questions were distinguished: questions that focus on the content and questions aimed at learning a (historical) skill (e.g., How can I see the propaganda technique ‘prestige’ from the source?). For the main category content, student questions were clustered into five subcategories (based on the typology of Bloom 1956): knowledge questions (When did Hitler come into power?), comprehension questions (How did Hitler come into power?), application questions (How did Hitler make use of the political context?), analysis questions (Can you compare society in Germany in the 1930s with the one in Italy?), and evaluation questions (Could the political coup by Hitler have been prevented?).
Motivation and perceived learning outcomes
Students’ motivation and their perceived learning outcomes with respect to the four homework assignments were measured in both groups by means of a questionnaire in lessons 7 and 11, respectively (i.e. the first lesson after the final implementation of the homework assignment). The questionnaire was piloted with grade 10 students. Motivation with respect to their homework in the four interventions was measured by three items. First, students had to report the extent to which they completed their homework (on a five-point Likert-type scale with 1 = in total and 5 = not at all). Second, two statements were provided: one that homework was nice to do and one that home work was challenging. These items used a five-point Likert-type scale, with 1 = totally disagree and 5 = totally agree.
Perceived learning outcomes were measured by four statements (on a five-point Likert-type scale, with 1 = totally disagree and 5 = totally agree), for each of the four interventions (By doing my homework it improved my understanding of the lesson, doing homework was a meaningful activity, I learned a lot of doing my homework, and it would be good to repeat this homework next time).
Teaching format
Based on an inspection of the observation data, five teaching formats were distinguished: (1) self-regulated individual work, (2) self-regulated group work, (3) teacher-task instructions, (4) teacher-led classroom discussion, and (5) teacher explanation of the subject matter. Each time on task/level of participation unit received a teaching format code (W1, W2, W3, W4, or W5)
Data analysis
As the two groups did not differ significantly in time on task and level of participation, we merged the data from both classes. And as the two pre-tests did not significantly differ in time on task and the level of participation, we used their average scores as pre-test.
Paired t tests were used between the pre-test, on the one hand, and each of the post-tests and the delayed post-test, on the other hand, to answer research question 1 (with time-on task and level of class participation as dependent variables) and research question 4 (with student motivation and perceived learning outcomes as dependent variables). Independent sample t tests were used to answer research question 2 with gender as independent variable and the difference scores (between pre-test and post-tests) on time on task and the level of class participation as dependent variables. As we performed a series of t test in these cases, we adapted the original significance level of 5 % based on the Bonferroni correction method (5 % divided by the number of analyses).
Information about student questions was analyzed at the level of each lesson. Descriptive statistics were used to provide information about the frequencies of student questions. Univariate analysis of variance with Scheffé post hoc analyses were used to answer the research question about differences between teaching formats (question 5). Teaching format was the independent variable and time on task and level of class participation were the dependent variables.