Homework is an important part of how students develop fluency with the ideas and techniques in mathematics in general, and in calculus in particular. This is especially true at the undergraduate level, where the expectation is that students spend considerable time outside of class working on homework and studying course material. For example, Rasmussen and Ellis (2013) found that post-secondary science, technology, engineering, and mathematics (STEM) intending students in the United States (US) reported spending approximately six h a week preparing (studying, reading, and doing homework or lab work) for Calculus I, a course with typically four contact hours per week. The time undergraduate students spend outside of class is an important venue to grapple with the material and interact with the content. In this study, we investigate the relationship between the nature of Calculus I homework at five PhD-granting universities identified as having a more successful Calculus I program than comparable universities.

The Calculus I programs at the five universities were selected as part of a large, US national study at over 500 institutions, including Associates-granting 2-year colleges, Bachelors-granting 4-year colleges, Masters-granting universities, and PhD-granting universities (as determined by the highest degree offered in mathematics). These five sites were selected because students at these universities were more successful in Calculus I when compared to students at comparable universities when controlling for student background and other factors. Student success variables included increased confidence, interest, and enjoyment of mathematics, grade in Calculus I, and persistence to Calculus II. This project enabled us to ask the following questions that shed light on how homework is related to student success in Calculus I:

  1. 1.

    What characterizes the nature of homework at the selected PhD-granting universities with demonstrated successful Calculus I programs?

  2. 2.

    What differentiates the nature of Calculus I homework at selected and non-selected PhD-granting universities?

As detailed in the methods section, we answer the first research question by analyzing student focus group interviews that were conducted as part of the case studies at the five selected universities. These universities were selected for case studies because the survey results indicated that their students retained confidence, enjoyment, and interest in mathematics more than comparable universities involved in the survey. To answer our second research question, we draw on the large-scale survey data to compare the nature of homework at selected universities to non-selected PhD-granting universities. We complement the quantitative analysis with illustrative findings from the student focus group interviews at the selected universities. These interviews shed light on students’ accounts of the role of homework on their success. This study contributes to the literature examining the role of mathematics homework in students’ learning and success, further extends a theoretical lens in which to view the role of homework, and offers pragmatic insights for other institutions into the nature of homework systems of successful calculus programs.

Theoretical Background

We draw on Herbst and Chazan’s (2012) elaboration of the instructional triangle (Cohen et al. 2003) to understand the nature of homework within the broader context of instruction. This theoretical tool employs the concepts of didactical contract and milieu (Brousseau 1997) to describe the interactions between teachers, students, and knowledge at stake through instruction (see Fig. 1). The knowledge at stake is used by Herbst and Chazan synonymously with content, and refers to the potential mathematical knowledge one might learn through instruction. In this context, the milieu is “a counterpart environment that provides feedback on the actions of the students” (Herbst and Chazan 2012, p. 607). Included within the milieu are the goals for students and the resources available to students while working on tasks. The teacher is responsible for maintaining harmony between the milieu and the students by ensuring proper functioning of the milieu, shown in Fig. 1 by the arrow from the teacher’s component to the milieu within the student subsystem.

Fig. 1
figure 1

Instructional Triangle adapted by Herbst and Chazan (2012) describing the interactions between the teacher, knowledge at stake (content), students, and milieu around the completion of tasks

Full size image

Herbst and Chazan (2012) argue that successful instruction occurs when all aspects of this system work together. They employ the notion of didactical contract (Brousseau 1997) to understand the ways in which teachers, students, and the knowledge at stake (content) are bound to each other and the environment through predominantly implicit responsibilities. To satisfy the didactical contract teachers are responsible for students’ development of the knowledge and students are responsible for partaking in the tasks needed to construct the knowledge.

Differences in the didactical contract exist between education at the K-12 level and education at the undergraduate level. At the K-12 level teachers tend to guide students through the development of knowledge with tasks largely completed in class. At the undergraduate level, however, there is a shift in the didactical contract and students are expected to construct knowledge outside of the classroom after being presented the material during lecture. This shift in the didactical contract at the undergraduate level places a heightened importance on resources, such as homework, that can support the construction of knowledge. At this level, homework becomes a vehicle used by instructors to provide students with the practice needed for developing understanding, but leaves much of the responsibility of the construction of knowledge to the student. Therefore, studying the nature of homework at the undergraduate level can aid in the understanding of student success and the interactions that come into play between the instructor, the student, and the content.

Herbst and Chazan’s (2012) elaboration on the instructional triangle and the didactical contract (Brousseau 1997) allow us to view homework assignments as an important part of the milieu for the Calculus I course. This particular perspective provides a lens in which homework is examined as part of a system that directly interacts with teachers, students, and content to support instruction and student success.

Literature Review

Studies of the role of homework span several decades and have demonstrated the potential positive influence homework can have on student learning as well as on student confidence (Black and Wiliam 1998; Cartledge and Sasser 1981; Halcrow and Dunnigan 2012; Kitsantas et al. 2011; Lenz 2010; Morrel 2007; White and Mesa 2014; Young et al. 2011). Our review of these studies suggests that these positive influences are often implicitly dependent on aspects of the instructional environment and the interactions that take place between the teacher, student, and content within the instructional environment.

Research relating student learning to homework indicates that the mode of delivery of the homework is less important than the content of the homework itself and how it may serve as a form of formative assessment for the student. For instance, Lenz (2010) found no significant difference in student learning whether students were assigned online based homework or traditional paper-based homework with similar content. From our perspective, this indicates that variation in the milieu and homework structure may exist if the teacher interacts accordingly with the knowledge at stake and creates tasks that students may successfully interact with when constructing knowledge. In other research, Young et al. (2011) studied how modifying content to include an application rich treatment of Calculus I and Calculus II related to students’ success in these courses by assessing the mathematical knowledge developed through pre and post exams on six different calculus topics. Students in the experimental group took a one-credit course in tandem with their Calculus I and Calculus II courses that was solely dedicated to the applications of the content from each course in an attempt to create deeper understanding and increase student success. Young and colleagues found that students eventually benefited from the introduction of applications, but the benefit was not realized until the students were in Calculus II, as reflected by the scores on the pre and post exams. This study highlights the connections between the knowledge at stake and student success both within the immediate classroom environment, as well as in subsequent courses.

A number of researchers have connected specific aspects of either written homework or online homework to students’ affect toward mathematics. For example, Morrel (2007) investigated the effects of written homework on her Calculus I students’ confidence. The content of these assignments focused on arguments and explanations, and involved many multi-step and/or context-free problems. She asserted that these assignments encouraged students to interact with the content at a deeper level by gaining skills in communicating and writing about mathematics. She argued that the written homework specifically contributed to students’ confidence by encouraging them to tackle cognitively difficult problems. Further, she noted that including cognitively demanding written homework allowed students who had seen calculus before to remain interested. This is an especially applicable benefit for Calculus I, as approximately two-thirds of US students enrolled in post-secondary mainstream Calculus I have previously taken a calculus class at the secondary level (Bressoud et al. 2013).

Halcrow and Dunnigan (2012) also reported an increase in students’ confidence when they looked at the benefits of online homework, and argued that online homework provided additional opportunities for students to think about and practice mathematics outside of the immediate classroom environment. The authors attributed the gain in students’ confidence to students’ ability to attempt problems multiple times and to receive immediate feedback. This finding illustrates how online homework can contribute to a milieu in which more practice and resources, such as immediate feedback or hints, are available to students while completing assignments that ultimately can increase students’ construction of knowledge and success in the course. These resources position online homework as an assessment with both formative and summative purposes. In many instructional environments, summative assessments are unavoidable, especially in high enrollment undergraduate courses with multiple sections. However, incorporating feedback (both immediate feedback for online homework and written feedback) turns these assessments into tools to improve learning in the short term while retaining the purpose of measuring the students’ current understanding (Black and Wiliam 1998).

White and Mesa (2014) further investigated the specific content of homework problems (in addition to in-class problems and exam problems) at a community collegeFootnote 1 that had demonstrated a higher than expected level of student success in Calculus I. They found that instructors at this more successful institution assigned more problems that required students to learn more cognitively demanding skills, but that instructors varied on the amount of rich tasks and complex procedure problems they assigned.

The previous studies point to various ways in which homework has been related to student success and illustrates the importance of interactions between the teacher, content, and student when constructing and completing the assignments. Specifically, this body of research indicates that the content of homework is especially important, and that there are aspects of both written and online homework that have been connected to student success. Our study contributes to this literature by exploring the nature of homework assignments at selected universities that were identified as having a successful Calculus I program. We ground this exploration within the perspective of the instructional triangle as adapted by Herbst and Chazan (2012), which allows us to understand and focus on relationships between the aspects of homework as part of a system involving content, students, and the teacher.

Methods

To answer our research questions we conducted a mixed method analysis on data from a large, national study focused on successful calculus programs in the US, the Characteristics of Successful Programs in College Calculus (CSPCC) project.Footnote 2 The CSPCC project study involved over 500 institutions, including AS-granting 2-year colleges, BA-granting 4-year colleges, MA-granting universities, and PhD-granting universities (as determined by the highest degree offered in mathematics) (Bressoud et al. 2013). For this report we focus on the PhD-granting universities because it is these institutions that are contributing the most STEM graduates to the workforce (Blair et al. 2012) and hence understanding the features of these institutions is of national importance. Further, by focusing on one institution type rather than analyzing data from all institution types we are able to remove this contextual variable. Future analyses should be conducted to situate how our findings about homework at the PhD-granting universities relate to the homework systems of other institution types.

The larger CSPCC study consisted of two phases, the first of which was a survey given to Calculus I students and their instructors at the beginning and end of the fall term. The goal of this phase was to identify institutions with more successful Calculus I programs, as well as to learn more about the landscape of Calculus I in the US. The second phase of this study included explanatory case studies at five PhD-granting universities. These universities were selected because the responses to our surveys indicated that they had more successful Calculus I programs (compared to counterpart universities while controlling for student background and other factors) as measured by increased student confidence, enjoyment, and interest in mathematics, Calculus I grade, and persistence onto Calculus II (Sonnert and Sadler 2015).

The goal of the second phase was to identify characteristics of the different Calculus I programs identified as having a relatively more successful Calculus I program. During case study site visits we interviewed students, instructors, and administrators, observed classes and recitation sections, and collected any relevant documents, such as homework and exams. Due to the design of this study, no such data is available from the non-selected universities.

The five selected universities varied in characteristics related to institution type, enrollment, the structure and class size of Calculus I sections, and the type of faculty employed to teach Calculus I. These five universities included private, public, technical, small (less than 20,000 undergraduate population) and large (more than 20,000 undergraduate population) universities. The enrollment of undergraduate students ranged from approximately 3000 to over 25,000. Also important to note was that the structure of the Calculus I course and class size varied. Three of the universities convened small class sizes of about 30 students, while the others had large lecture sections of about 200 students accompanied by smaller recitation sections. Additionally, two of these universities used the Hughes-Hallett et al. (2009) textbook in all Calculus I classes, and the others used Stewart (2007), Rogawski (2011), and Edwards and Penney (1998). For an analysis of the content covered among these five universities see Johnson et al. (2014). The diversity of institution size and population and class size enables us to comment on what is similar and different regarding homework across the different institutions.

Data Collection and Analysis

We draw on both quantitative survey data and the qualitative case study data from the CSPCC study. Specifically, end-of-term survey data from 2000 students and 204 instructors were analyzed as well as student focus group interviews at the five case study sites. Of the 2000 students, 580 came from a selected case study universities. There were 204 instructors who completed the end of term survey, 46 of which came from a selected university. A multiphase mixed methods approach was taken to analyze the quantitative and qualitative data (Creswell 2002). In particular, the case study data from the selected universities were first used to understand the nature of the homework assignments implemented at each of the case study sites. Subsequently, descriptive analyses were conducted on both student and instructor responses to understand the nature of the homework at selected and non-selected universities. Homework and exam problems pertaining to related rates from the five selected universities were also examined in order to better understand the nature of the problems assigned at these institutions within a specific mathematical context. Additional case study data (including class documents, instructor interviews, and classroom observations) from the selected universities were then used to triangulate findings.

In examining the case study data we conducted inductive thematic analysis on focus group interviews. Thematic analysis is a qualitative approach that involves “identifying, analyzing and reporting patterns (themes) within data. It minimally organizes and describes the data set in rich detail. However, frequently it goes further than this, and interprets various aspects of the research topic” (Braun and Clarke 2006, p.79). Inductive thematic analysis is a bottom-up approach where the themes are data-driven, though are not developed in an epistemological vacuum. Thus, these themes emerged from the data, but were also informed by our reading of the relevant literature and our previous experiences with the data.

We conducted a total of 12 student focus groups across the five case study universities. After coding the interview transcripts for excerpts related to homework, we distinguished between factual references to homework and personal reflections on homework. Fifty of the 68 excerpts were of the latter type. We then conducted a more fine-grained analysis on these excerpts, resulting in 10 codes that captured positive and negative statements about an aspect of homework. These 10 codes were then grouped according to three themes: structure, content, and feedback. Following this thematic analysis, we conducted descriptive quantitative analysis on the student and instructor survey data. More specifically, we conducted t-tests and Chi-squared analyses to compare responses from instructors at the five selected case study universities (N ~ 158) and their students (N ~ 1410) to instructors from all other PhD-granting universities (referred to as the non-selected universities) (N ~ 46) and their students (N ~ 590). These comparative analyses enabled us to explore the relationships between the aspects identified in answering the first research question to student success in a quantitative manner. In our presentation of these quantitative results, we provide examples of related excerpts from interviews with students regarding the various aspects of the nature of homework identified in answering the first research question. These excerpts were selected as vivid, representative, and compelling examples of each theme. As previously mentioned, due to the design of the CSPCC study, no such data is available from the non-selected universities.

While we make no causal claims, the five case study universities were selected due to higher student success compared to comparable universities, where student success was measured by increased student confidence, enjoyment, and interest in mathematics, Calculus I grade, and persistence onto Calculus II. Thus, in comparing selected universities to non-selected universities, we can begin to connect specific aspects of the nature of homework to specific aspects of student success in Calculus I. Further studies are needed to more clearly explore the relationships we identify.

Results

To begin we present the results of our first research question by characterizing the nature of homework at the selected PhD-granting universities with demonstrated successful Calculus I programs. We then address our second research question by first quantitatively comparing the nature of homework at selected universities to non-selected universities. Lastly, we complement and triangulate these quantitative comparisons with qualitative analyses of student focus group interviews and collected classroom data, including observations and assigned homework.

The Nature of the Homework at Selected Universities

The inductive thematic analysis of the case study data resulted in the creation of 10 codes, each with possible positive and negative expressions, as shown in Table 1. Through thematic analysis, these 10 codes were grouped into three salient features related to the nature of homework: structure, content, and feedback. Structure refers to the format and delivery mode of the homework: the frequency with which it was assigned, how much homework was assigned, the level of coordination across instructors, whether it was online or written, and whether it was group or individual. Content refers to the nature of the problems or tasks on the homework, whether they emphasize or assess procedural or conceptual knowledge. We draw on Hiebert and Lefevre’s (1986) definitions of procedural and conceptual knowledge, where procedural knowledge is understood as fluency with symbols, rules, algorithms, and procedures and conceptual knowledge is characterized as “knowledge that is rich in relationships” (p. 4). Feedback refers to dialogue between the instructor and the student (potentially via automated online homework response systems) that informs the student about the quality of his or her work. Black and Wiliam (1998) argue that to be considered feedback the goal of this dialogue must be to improve learning and thus feedback is viewed as a defining component of formative assessments. Moreover, while grades have a summative aspect, they can also be formative in so much that they function to improve learning. We refer to these three components (structure, content, and feedback) as a homework system.

Table 1 Codes and themes related to the nature of homework
Full size table

After identifying the three main themes of a homework system, we then used these themes to compare the homework systems at each of the five selected universities. This comparison is shown in Fig. 2, where the rows indicate each institution and the columns specify the extent to which particular aspects of the structure, content, and feedback are uniform across instructors. The system of coordination (or uniformity across sections) was identified as a key feature of selected successful programs in a related study (Rasmussen and Ellis 2015). We use a dotted circle or a completely filled in circle to indicate that a particular aspect exists, where a dotted circle indicates that only some instructors implemented this aspect, and a completely filled in circle indicates that this component is implemented uniformly across all instructors.

Fig. 2
figure 2

Homework systems at selected universities

Full size image

For example, Private Technical University (PTU) had an online procedural component that every Calculus I instructor assigned, as indicated by the solid circles in the online column under structure and the procedural column under content. Along with the uniform component, we also saw that some individual instructors at PTU assigned additional written conceptual homework, as indicated by the circles with several dots in the written column under structure and the conceptual column under content. There were no instructors at PTU that assigned group homework or gave a homework quiz, represented by an empty circle in the group column under structure and the HW quiz column under feedback.

In addition to illustrating the homework systems at PhD-granting universities with successful Calculus I programs, Fig. 2 highlights the commonalities among the homework systems. All universities incorporated a combination of procedural and conceptual problem solving in their assignments, and all provided some form of feedback to the students. However, the structure of the different homework components, whether it be written, online, or how they gave feedback, varied among the five selected universities.

Using Brousseau’s (1997) language, the homework system (structure, content, and feedback) functions as an important component of the milieu in which students interact with the mathematics. The homework system allows students to complete tasks, receive feedback, and avail themselves of certain resources to complete the tasks. For example, a homework system may create an environment where students can receive instant feedback by completing online homework and have access to resources such as hints provided by the online homework provider. Viewing the homework system as an important component of the milieu allows us to focus on the interactions between the homework system, the teacher, and the knowledge at stake for the course as well as the external environment that the interactions take place within.

We next direct our attention to how the homework systems vary across the five selected universities. Understanding different homework systems provides insight into how instruction can affect student success. We draw on the instructional triangle used by Herbst and Chazan (2012) to understand how the interactions between students, the teacher, and the content relate to instruction, and further to student success. Within this environment we view the teacher as anyone in a position to alter or create the homework system. This may include the course instructor, as well as recitation leader, or Course Coordinator. We view the knowledge at stake as the mathematical content covered within the course that is specified in the syllabus, textbook, or course objectives. In Herbst and Chazan’s (2012) elaboration of the instructional triangle, they discuss the milieu as a “counterpart environment that provides feedback to the student” and includes the goals students are working towards and the resources with which the students are operating (p. 607). While homework systems may be viewed as a resource to students, they can also be viewed as something beyond a resource, especially within the undergraduate context. The homework system is an especially important component of the milieu within the undergraduate mathematical environment, necessitated by the shifted didactical contract. Students are expected to spend more time struggling with the material and constructing their own knowledge outside of class, and the structure, content, and feedback of the homework are important components of how students do these things.

We focus on the relationships that exist between the homework system (as part of the milieu), the teacher, and the knowledge at stake, which all function within the larger environment of the department and institution. For example, the teacher uses the knowledge at stake to alter and create the homework system. The teacher is responsible for choosing the method of homework whether it is online or written based. Additionally, she decides what aspects of content to emphasize through the problems assigned and what content to cover throughout the course. As the students complete the homework, both the student and the teacher receive feedback, which affects future instruction. The teacher can look at the students’ progress and accuracy of the homework and assess the students’ learning of that certain topic and respond by altering lesson plans or changing the pace of the course, therefore changing the homework system. The students are also provided feedback from the teacher when their homework is returned in the form of a grade and/or receive written comments about certain problems. Students can also be given feedback within the homework system through the online program marking the problems correct or incorrect. Depending on the institution and corresponding homework system, these relationships exist to varying degrees.

Differences Between Selected vs. Non-selected Universities

After identifying the salient features of the homework systems at selected universities through the thematic analysis of the case study data, we investigated how the homework systems at non-selected universities compared to the selected universities by drawing on the survey data. In the following sections, we compare the responses from students or instructors at the selected universities to those at the non-selected universities along each of the three components of a homework system. We complement these descriptive analyses with illustrative and representative quotations from interviews with students at the selected universities.

Structure

Through the qualitative analyses, we attended to the mode of delivery of the homework as the key component of structure (online, written, and/or group). In the quantitative analyses, we also have reports of how often homework was assigned and/or collected. As shown in Table 2, there were significant differences between student reports of the structure of homework at selected versus non-selected universities. Compared to students at non-selected universities, (a) students at selected universities report that assignments were assigned and collected more frequently, (b) that group projects were assigned more frequently, and (c) that online homework was more likely to be used to complete homework assignments.

Table 2 Comparison of structure of assignments and assessments
Full size table

These findings indicate that students at selected universities are being offered more opportunities to interact with the knowledge at stake and are often being held more accountable for practicing the course material outside of the classroom. Thus, these homework systems have developed in accordance to an altered didactical contract, where the students are expected to spend more time outside of class working with the material, but also are provided supports in order to succeed when doing so. Students at selected universities also indicated a higher use of online homework, which contributes to a greater amount of instant feedback within the milieu of the homework system. As discussed by Halcrow and Dunnigan (2012), the greater the amount of time spent outside of class thinking and practicing the material as well as the ability to gain instant feedback and persist in a problem until the correct solution is found reflected an increase in students’ confidence. Therefore, the increased existence of homework and feedback provided at selected universities appears to provide the practice and guidance needed for students to gain confidence in mathematics and may have an indirect affect on their increased persistence to Calculus II.

During the focus group interviews, students at selected universities often reflected on the structure of the homework. Seventy-four percent of 50 reflective student comments were related to structure, and 62 % of these were positive statements. The most commonly discussed aspect of structure was online homework, followed closely by group homework, and then the existence of homework. The negative statements regarding online homework were almost uniformly about the logistics of entering solutions into the online system. A few students also disliked the procedural nature of online homework. For instance, one student from LPU1 disliked the online homework because “it doesn’t really have a lot of application on it. It’s just cranking out formulas.” However, many students also had positive experiences with the online component, and appreciated the ability to try a problem multiple times until they got it correct.

The negative statements regarding group homework were also logistically oriented, citing the difficulty of finding a time when all group members could meet and the high expectations relating to the presentation of the homework and the content. Students appreciated that the group homework exposed them to “how different students answer the problems in different ways…Different ways to help you learn the problems better”, and that the group homework offers “another way to do math outside of class” that is “really interactive and if you don’t understand it someone else is going to teach you how to do it.”

Students also made statements (100 % positive) regarding the existence of homework, and how it helped them stay on top of the material and gave them multiple entry points into the content.

LEPU Student: Throughout the entire week we have to at least think about calculus, at least once a day. So it’s not like a lot of workload, so it’s still really manageable, it’s just it’s always there so I feel like that helps.

LPrU Student: I would say probably I learn more when I’m doing my homework when, because he’ll do a couple of examples but then there’s so many more still. Then there’s a broader range of them in the homework, so base what you learn in class and like you realize you’re going to use the same steps, but you really have to think about what, I don’t know. You have to think differently every time.

The student statements help to illustrate why some specific structures of homework systems may lead to student success. The combination of having multiple opportunities to interact with the material with supports available allows students to benefit from the extra time they are spending outside of class grappling with the content.

Content

The content of the homework can be thought of as a source of knowledge at stake for students to develop. The balance between procedures and concepts is often an issue for instructors, both in terms of what is valued and what one has time for (Johnson et al. 2014). All five of the selected universities assigned both procedural and conceptual questions on their homework. Interestingly, we do not find the same balance at non-selected universities. Table 3 shows that students at selected universities reported having to explain their thinking more on homework problems and were assigned more word problems than students at non-selected universities. Table 3 also shows that instructors reported assigning more homework that focused on graphical interpretation, complex or unfamiliar word problems, and proofs or justifications than instructors at non-selected institutions.

Table 3 Comparison of content of assignments and assessments
Full size table

Table 3 also shows that instructors at both selected and non-selected universities report that they assigned homework involving standard word problems. Selected universities assigned standard word problems that were complemented by complex or unfamiliar word problems more so than non-selected universities. White and Mesa (2014) also found this to be true when they studied the types of assignments at one of the selected AS-granting 2-year colleges. They saw instructors at this one selected institution overall assigning simple procedure problems 54 % of the time, having the rest of the problems either be complex procedure problems or rich tasks.

In order to understand more concretely and triangulate findings about the content of assessments at selected universities we looked in detail at all problems involving related rates that were part of assignments collected during the case study. We chose related rates over a different content area (such the chain rule) because this topic can be rich in content, often presents challenges for students, and allows for both a more conceptual or more procedural treatment (Cartledge and Sasser 1981; Martin 2000; Piccolo and Code 2013). Our goal in looking to the actual problems assigned at the selected universities was to understand how reported emphasis in the content was manifested in actual problems.

The ladder problem is a very common word problem involving related rates, and we saw versions of this classic problem assigned or discussed during the case study visits at multiple selected universities:

A ladder 10 ft long rests against a vertical wall. If the bottom of the ladder slides away from the wall at a rate of 1 ft per second, how fast is the top of the ladder sliding down the wall when the bottom of the ladder is 6 ft from the wall?

Along with the standard ladder problem, instructors at selected universities also assigned more complex related rates word problems that allowed students to apply the concepts they learned in the ladder problem to a novel, more complicated problem, such as the following:

The lookout on board a ship spots an unknown ship on the open seas and immediately alerts Chris, the ship’s captain. The unknown ship is currently 4 miles south and 3 miles east of Captain Chris’s ship, and it is at that moment sailing due west at 20 mph. Captain Chris’s ship, meanwhile, is at that moment sailing due north at 16 mph. (a) Is the distance between Captain Chris’s ship and the unknown ship increasing or decreasing? How fast? (Distance is measured along a straight line joining the two ships.) (b) How does your answer change if Captain Chris’s ship is going 14 mph instead of 16 mph?

The students at the selected universities spoke about their assignments as being challenging and more than the routine “plug-and-chug” problems they had seen in previous mathematics courses. None of the students overtly discussed the procedural nature of the content, except when in comparison to the conceptual problems. In these cases, the statements were coded as predominantly about the conceptual problems. For instance, one student from LPrU noted that his experience with mathematics in high school was about applying algorithms, while in his current Calculus I course he was expected to “think- it’s much deeper.”

LPrU student: Well because generally for me math has always been nice because it’s very like, you see this situation, ‘apply this, get this answer.’ Then all of a sudden with a proof, you have to like think all of a sudden. It’s not something I’ve been trained to do in high school, was to like actually have to think and break down and say, ‘Theorem 1, theorem 2 and theorem 3 make theorem 1 become theorem 3. And then also you can use theorem 2.’ You have to think, it’s like much more deeper.

In addition to working on more novel and/or complex problems, the content of the homework at selected universities went beyond students’ previous experiences with mathematics homework in other ways. As seen in Table 3, many students were asked to explain their thinking on their homework, which was not something many students were accustomed to.

LPU2 student: And she asks that, with each problem we write a little explanation of what we did. So if we say we took the derivative and set it equal to zero to solve for a max or a min, she wants us to explain how and why we did it that way. So it’s more than just the plug-and-chug.

Some of the students expressed that being challenged and being able to complete the assignments successfully increased their expectations of the kinds of problems they were capable of completing and understanding in mathematics.

LPU2 student: I think it’s increased [my confidence], because like I’ll look and sometimes I’ll look at a problem and be like holy [moly], I have no idea what I’m doing here, but then once I can figure, actually figure out what I’m doing, it’s like ‘wow I was able to do that’. I just had to look at it in a different way to figure it out, so it kind of boosts your expectation of what you can do next.

Only one student mentioned that she did not like the conceptual nature of the homework, because the conceptual problems “are definitely kind of a pain on the homework” but she did appreciate working on them with her teaching assistant during recitation.

Feedback

Feedback on homework emerged as a salient feature of the homework systems at selected universities. The feedback given on the homework is a critical way the teacher can interact with the student and the student can interact with the content. When giving feedback, especially at the undergraduate level, it is implicitly understood as part of the didactical contract that students will use this feedback to ask and answer questions such as “Where am I going? Where am I now?; and How can I get there from here?” (Stiggins and Chappuis 2005, p. 4).

Students were asked to report on a number of aspects of the nature of the feedback on their homework, both written and online. As shown in Table 4, students from selected universities report that their assignments are more frequently completed and graded online and are more frequently returned with helpful feedback/comments. These results indicate that students from selected universities receive more feedback on both online and written homework than students at non-selected universities.

Table 4 Student reports of the nature of homework feedback
Full size table

During the site visits at the selected universities, we identified seven statements about feedback on homework, with six of these being negative. One thing that students liked about online homework was the ability to know right away if their response was correct. However, some students were frustrated that they were “not really getting feedback. It’s just right or wrong.” For some students, this was an indicator to seek additional help to understand what they got wrong.

PTU student: With [the online homework] you immediately know if you’re wrong, and sometimes you can look up a hint to see what you’re doing wrong. So that will help if you don’t understand a certain tool, you can definitely look it up and understand better, especially if you’re doing it wrong, you don’t want to keep doing it wrong.

One instructor at PTU saw this as a chance to take advantage of students working together in the dormitories and viewed their online homework as an extra resource for studying, as opposed to solely a venue for assessment.

PTU instructor: But yet on the other hand, with this [online homework] stuff that we do, you know that there’s a group of students in a dorm right now… all doing their homework together. And I think as faculty what we have to do is just accept that is happening, so we have to structure our course in such a way where we can actually use that in a good way. We can say, ‘Let’s view it as this is their study time.’

The previous excerpt exemplifies the shift in the didactical contract between education at the K-12 level and education at the undergraduate level, and how the homework systems can be used to complement this shift. If undergraduate students are expected to spend more time constructing knowledge outside of class compared to during high school, then the nature of the homework must support them in doing so. One key component of this appears to be providing responsive feedback to students as they grapple with the material outside of the classroom, with or without their peers, complemented by providing resources to get additional support.

Taken together, the above findings point to a significant shift in the didactical contract surrounding who is responsible for grappling with difficult material and where this takes place. The homework systems at the selected universities manifest a response to this shift by (1) providing multiple opportunities for students to interact with the content outside of class, (2) providing feedback as a way for the teacher to interact with the students and the content while students work outside of class, and (3) expecting students to struggle with more complex content and explain their thinking related to this content. We posit that these features are one reason why students at the selected universities experienced an increase in their mathematical confidence, realizing that they are capable of succeeding with more difficult material.

Conclusion

In this report we investigated the nature of Calculus I homework at five PhD-granting universities identified as having more successful Calculus I programs compared to comparable institutions. We drew on case study site visits at these five universities in order to gain an understanding of characteristics of these programs that potentially contributed to their success. One such characteristic was the nature of homework. Based on our analysis of the case study data, we identified the content, the structure, and the feedback of the homework as salient aspects of the homework systems. In order to better understand how the nature of Calculus I homework may be related to student success, we (1) conducted comparative analyses of the reported content, structure, and feedback by students and instructors at selected versus non-selected universities, and (2) analyzed student focus group interviews from the case studies to better understand the connection between these aspects of homework and student success. By viewing these analyses through the lens of the instructional triangle (Herbst and Chazan 2012) and the didactical contract (Brousseau 1997), we situate the homework system within the broader context of the instructional environment. This allowed us to focus on how relationships between students, the instructor, and the content interact with the homework system, and how all of these interactions are governed by the didactical contract between the students and the instructor.

The quantitative analyses indicate that compared to non-selected universities, homework and group projects at selected universities were assigned more frequently, online homework systems were used more frequently, this homework was more frequently graded and returned with feedback, and the content focused on more novel, cognitively demanding tasks. Moreover, by analyzing the student focus groups, we see a much more nuanced relationship between specific aspects of Calculus I homework and increased student confidence in mathematics and persistence onto Calculus II. As we articulate next, these findings lead to a number of conjectures about the relationships between the nature of homework at selected universities and student success.

At the selected universities we saw that the online homework had two components that students felt contributed to their growth in confidence throughout the course. The students received instant feedback and were then given multiple or unlimited chances to complete the problems correctly. Halcrow and Dunnigan (2012) explain that there are high achievers who complete their homework and spend outside class time studying regardless of the rewards given by the instructor, but not all students can be classified as high achievers. They also found that both high achieving and low achieving students were more motivated to complete the homework when online homework was the main source of homework. We saw in our study that many students reported that they completed their online homework and used it as a study resource outside of class. They were able to complete many different problems multiple times, which motivated them to spend time outside of class going through the homework and receiving the feedback and using that to prepare for the next class or the next exam.

Both online and written homework contributed to students increase in confidence in mathematics and changed their perception of their ability to learn, which itself has likely affects on their persistence onto Calculus II (Ellis et al. 2014). Online homework, which allows students to attempt the problems multiple times, helped students gain confidence because it showed them that they could persist through a problem and eventually succeed and get the correct answer. Along with increasing their confidence in mathematics, students also changed their beliefs about their ability to learn. Dweck (2008) delineates between a fixed mind set and a growth mind set. A fixed mindset is characterized by a belief that there is not much one can do to affect their intelligence level, whereas a growth mindset is characterized by the belief that one’s intelligence can be significantly changed. Dweck found that mindsets can predict math/science achievement over time and interventions that change mindsets can boost achievement and reduce achievement discrepancies.

Students that may have entered the course with a fixed mindset were given multiple chances to correct their mistakes and see that they were capable of eventually getting the correct answer. This may have encouraged them from a fixed mindset, thinking they would never be able to solve the problems, to a growth mindset, where they believed they could practice until they succeeded.

At some of the selected universities, written homework was also implemented to provide students with more conceptual and application problems. Instructors believed that assigning written homework was important because of the types of problems the students had to work through, but because it also allowed students to practice organizing their work and explain their thinking. Morrel (2007) found that assigning written homework in his class benefited students in many ways, asserting that, “my students gain mathematical confidence by tackling these more difficult problems, and it keeps the class more interesting for those who had calculus in high school” (p. 383). We also found that at the selected universities, Calculus I students attributed their increase in confidence to their ability to work through difficult problems together and being able to explain their mathematical thinking.

These findings regarding the role that the homework system plays in student success in college calculus is embedded within a larger context. In our large national study, we identified seven features of calculus programs at PhD-granting universities as supporting student success. These seven features are: the collection (and use) of local data, support for teaching and active learning, graduate student teaching preparation, adaptive placement strategies, learning resources, coordination, and rigorous courses (Rasmussen et al. 2014). These seven features do not exist in isolation, but rather work together at the successful universities to create a successful calculus program. Homework systems are a component of rigorous courses, and thus cannot be thought of as the sole contributing factor to students’ success.

Our findings also provide evidence for the applicability of the instructional triangle at the undergraduate level. At the K-12 level, the majority of the interactions between student, teacher, and content occur within the classroom. However, at the undergraduate level, many of these interactions take place outside of the classroom. Thus, the homework system plays a heightened role in undergraduate mathematics because it acts as the milieu for these interactions to occur both inside and outside the classroom. Our findings also have direct implications for the classroom. As an undergraduate instructor, the homework system is an especially important resource for extending and furthering the learning that takes place during lecture. One must be purposeful about how to utilize online and/or written homework as a medium for students to practice skills and grapple with concepts, while providing feedback for the successful (and more enjoyable) development of both.