The importance of affective learning goals: lessons learned from the design and evolution of a geoscience course for preservice primary teachers

The challenges of teaching science to preservice elementary teachers include an overall negative attitude about science that translates to future teachers who do not teach science confidently and/or teach it superficially and hurriedly compared to other subjects. College-level science courses for preservice teachers have an opportunity to reverse this trend and help create teachers who are knowledgeable about science and enjoy teaching it. In this paper, we discuss the hybrid model of an online course and a hands-on, inquiry-based lab that was developed, implemented, and evolved over eight years. The lab activities, in particular, were developed to address both content and affective goals. These goals were to provide students with a solid foundation in Earth and Space Science content and to make them comfortable teaching science. An assessment of these goals shows that this course increased preservice teachers’ level of confidence to teach these topics, and students generally found the course to be enjoyable and the content to be useful to their future careers. More work needs to be done to assess the content knowledge of this group of students, highlighting the need for developing a validated instrument that covers the breadth of Earth and Space Science content included in the NGSS for elementary grades.


Introduction: the challenge of designing a science course for preservice primary teachers
In 2009, Margaret Avard made a startling discovery about elementary school science teachers: many had low confidence to teach science, and, as K-12 students themselves, had already started acquiring the attitudes of ambivalence or dislike toward learning science (Avard 2009). In high school, some revealed a propensity to avoid science by taking only the minimum required number of courses, and, as students, they were reporting experiencing these courses as difficult, boring, or frustrating (Bleicher and Lindgren 2005). By the time they reached the upper undergraduate ranks of preservice elementary teachers, a significant number could no longer explain why science is an important part of the curriculum that they would soon teach (e.g., Bulunuz and Jarrett 2010;Cervato and Kerton 2017). Not surprisingly, studies confirm that it is the science courses that are often taught superficially or hurriedly in elementary grades (Dorph et al. 2011;Howitt 2007).
In science courses designed for future elementary education teachers, most emphasis is on the mastery of cognitive learning outcomes, preferably developed through active learning experiences. However, the summary above confirms the prevalence of powerful affective dispositions that act to undercut enthusiasm and confidence. These dispositions subvert cognitive learning and even professional practice.
Therefore, when we were asked to provide a science course for future elementary teachers, we decided to work to develop better affective dispositions along with the prescribed cognitive outcomes. When students develop jaundiced attitudes toward learning science, these often come from their experiences in the laboratory portion of science courses. Thus, we chose the laboratory portion of the course as our focal point for redesign in ways that encouraged self-reflection and built more constructive affect.
The eight years over which we worked with this unique course occurred coevally with others' relevant work that was establishing new understanding about links between science literacy, affective feelings of competence, direct measures of cognitive competence, self-efficacy, and self-regulation. Only recently did this other work become available, but it allows us to interpret our results in light of this new understanding.
Because this involved a project without benefit of precedents, we now find ourselves in the state described in a Rod Stewart song line: "I wish that I knew what I know now." However, this state yielded more valuable insights than the usual "suggestions for further work." Therefore, we expended some effort to use the experiences that advanced our own learning to inform readers how we would redesign our course and the study of it, if we were newly initiating a similar project now.

Background
To promote greater understanding of science and the use of hands-on teaching techniques, encourage future teachers to engage more with science material, as well as meet our State's science requirements for preservice teachers, a hybrid introductory science course (online lecture and hands-on lab) was launched in fall 2011 at a large research-intensive public institution in the U.S. Midwest (Cervato et al. 2013). The course was created in response to changes in the state-mandated requirements for licensure that moved from nine credits of science content, chosen by the individual student from a list of entry-level science courses, and three credits of science methods, to three credits each of physical, Earth and Space, and life sciences plus three credits of science methods.
Social cognitive theory says that student performance is influenced by students' selfregulating beliefs that influence and are influenced by motivation; these beliefs control both cognitive and affective learning (Bandura 2001). Thus, understanding motivation and other social factors is as important as understanding cognitive factors (Black and Deci 2000). Self-determination theory suggests that motivation varies by the degree to which it is autonomous or controlled (Black and Deci 2000). Intrinsic motivation is related to autonomy, and an instructor who provides autonomy support acknowledges the student's perspective and feelings. In this learning environment, the instructor provides students with information and opportunities to make choices, rather than using demands (Deci and Ryan 1985;Black and Deci 2000). This has been shown to lead to better conceptual learning (Grolnick and Ryan 1987) and more positive affect (i.e., moods and feelings) in children (Ryan and Grolnick 1986). On the other hand, controlled behaviors by the instructor lead to extrinsic motivation, like a focus on grades rather than learning (Black and Deci 2000).
Active learning strategies that engage students in the content are a way to provide autonomy support, have been shown to result in a higher level of learning (e.g., Handelsman et al. 2004;Arthurs and Templeton 2009;Moss and Cervato 2016;Ryker and McConnell 2017), and a better understanding of the nature of science (National Research Council 2000). Middle-and high-school students enrolled in inquiry labs taught by instructors using reformed teaching techniques and active learning achieved greater knowledge gains (Blanchard et al. 2010). Bransford and Donovan (2005) showed that preservice teachers who engaged in inquiry-based instruction are more likely to implement reformed teaching practices in their future classrooms than their peers who did not. This paper narrates the process of designing and implementing a hybrid Earth and Space Science (E&SS) course for preservice elementary teachers in order to address both content and affective learning goals. It describes the evolution of the curriculum over eight years in response to student feedback, summative evaluation, and growing student numbers and the lessons learned through the process, or "what we wish we had known when we started and done while we were teaching it." We conclude by describing the limitations of the evaluation of our outcomes and making recommendations for instructors who embark on the demanding process of designing a course for this group of students and researchers who aim to assess content learning in an E&SS course.

Development and implementation phase
Prior to the creation of this course in 2011, the State of Iowa allowed preservice teachers to complete nine credits of science courses to be chosen from a broad list. As a result, most students chose courses that were perceived to be 'easier,' like biology and geology, and avoided 'harder' courses, like chemistry and physics. For example, of the 92 elementary education majors enrolled in a science course in fall 2010, 56 were enrolled in physical or environmental geology, 14 in astronomy, 12 in physics, 8 in meteorology, and only 2 in chemistry (Compton personal communication 2018).
In 2010 new State licensure requirements requested that preservice elementary teachers' science content include three credits each in physical sciences (physics and chemistry), life sciences, and Earth and Space Sciences. Since no one course existed at our university that covered the required E&SS content, a new twocredit online course and one-credit lab were created and offered fall and spring semesters starting in fall 2011 (Cervato et al. 2013). The state standards guided the content selection for the course, and the initial course design and creation process took place over one year. The underlying theme of the course was to make science enjoyable for the students with the thought that preservice teachers who like science, are familiar with science, and are unafraid of science will be much more likely to enthusiastically teach science content to their future students.
We used a backward design approach for both course and lab, starting from the projected learning outcomes and building a course that equally involved content learning and motivation following the social-cognitive theory (e.g., Black and Deci 2000). The online course covered the content material using interactive assignments, videos, readings, and online simulations. The instructors collaborated with a team of three instructional designers to adopt and adapt best practices in online learning and content delivery. The choice to deliver the course online rather than face to face was dictated by practical reasons: the two designated instructors, tenured faculty in geology and astronomy with a 45% teaching, 45% research, and 10% service appointment, were already carrying a full teaching load and believed that after the initial significant time investment in designing the course, an online format would be more manageable and flexible, allowing them to offer it both semesters to better accommodate students' schedules.
The online course consisted of 14 weekly modules administered through a course management system, with assigned readings (one to two textbook chapters per week) and an open book quiz. Four proctored online exams covering content from 3-4 modules were completed every four weeks. There was no cumulative final exam. To engage the students more deeply with the content and to emphasize the relevance of E&SS, we designed a semester-long project that involved writing a letter to the U.S. President, with multiple drafts and feedback opportunities (4 semesters, Cervato and Kerton 2017;Cervato et al. 2013) and then transitioned to a semester-long project culminating in a teaching demonstration addressing common E&SS misconceptions (2 semesters). Students were also required to complete seven assignments using Google Earth that allowed them to explore geological features and work on simple quantitative exercises.
The online course was complemented by a 1-h and 50-min laboratory that met once a week and was taught by graduate teaching assistants (TAs). Following the social-cognitive theoretical framework, the main learning goals of course and associated lab were to (1) give future elementary teachers a sufficiently solid foundation in E&SS content and (2) make the students comfortable teaching this content to K-5 students. We intentionally identified one cognitive and one affective goal because of our understanding of the importance of motivation to achieve learning, especially for students with a negative, or skeptical, disposition toward science (Cantrell et al. 2003;Howitt 2007).
Lab activities used an inquiry-based approach and modeled active science teaching practices through cooperative learning. We deliberately chose to use inquiry and peer learning as it has been shown to be more effective than a traditional 'cookbookstyle' lab (e.g., Bransford et al. 2000;Arthurs and Templeton 2009;Gray 2017). We also intentionally chose to avoid using a lab manual: lab manuals commonly used in geology and astronomy and available when the lab was designed, only included confirmation experiments, and no attempt at engaging students in inquiry, unlike biology, chemistry, and physics lab manuals (Buck et al. 2008).
Our implementation of inquiry-based learning required students to be involved in the discovery process (Weaver et al. 2008) and engaged them in many of the activities and thinking processes of scientists (NRC 2000). The National Research Council outlines five essential components of inquiry, and we followed these steps in our lab whenever possible (as in the examples given after each component): (1) students engage in scientifically oriented questions (How do we know if the current layout of continents has remained the same throughout Earth's history?); (2) students give priority to evidence in responding to questions (study the location of glacial deposits across various geologic time periods and on different continents); (3) students formulate explanations from evidence (hypothesize that continents were connected at specific times in the past); (4) students connect explanations to scientific knowledge (the theory of plate tectonics provides the mechanisms and describes the processes that explain the findings); and (5) students communicate and justify their explanations (groups share and compare results with other groups) (NRC 2000).
Designed around the principle of student-centered learning, the lab included activities, both guided and inquiry based, designed to foster hands-on learning while maintaining open lines of communication. In this type of learning, the teacher is primarily a facilitator rather than a lecturer, supporting students' autonomy and including both content knowledge and development of higher-order thinking skills (Avard 2009;Black and Deci 2000). Since 2014, hands-on activities were complemented by lecture tutorials (Kortz et al. 2008) using a workbook that students were required to purchase (Kortz and Smay 2014). Selected tutorials were completed in class or as homework.
This inquiry-based teaching approach differs from the one traditionally used in science labs. This means that it is unlikely that our graduate TAs experienced it in their own undergraduate labs and courses. For this reason, TAs for the lab course needed to be comfortable assuming a facilitator role and avoid lecturing (Ryker and McConnell 2014). While more recent lab manuals include some level of inquiry (Ryker and McConnell 2017), this lab curriculum is one of the first examples of an E&SS lab largely founded on inquiry. The highly flexible and modular approach, including multiple short mini-labs and the broad range of content, allowed each lab component to be extracted and adapted to different contexts (Supplementary Materials). This made it easy to add a new activity by replacing one that did not work as well, following the focus on flexibility and adaptability in the design.
We did not use a validated instrument to assess content learning since, to our knowledge, there is no such instrument for the breadth of content covered in the course. Learning assessment of the online course consisted of proctored online exams (40% of final grade), weekly quizzes administered through the course management system (40%), a semester-long project (10%), and Google Earth-based assignments as described above (10%). The grade for the lab was based on the scores of three quizzes with multiple-choice questions, practical problems, and short answers (30%); homework (20%); and lab assignments and attendance (50%). While the quiz questions were originally designed to match State Core Curriculum standards, to broaden the applicability of this course material to other institutions, and to confirm that our content encourages the thinking processes of scientists, we have paired Next-Generation Science Standards with appropriate questions from each quiz (the paired quiz questions are available on request).
Concurrent enrollment in the online course and lab was required for the elementary education majors. The initial enrollment in the course was 48 students and for the lab they were divided into two sections of 24 students each. Although, as we detail in the next section, the total enrollment changed significantly over the time period the course was offered (2011-2019, Table 1), the characteristics of the student population remained essentially unchanged. The majority of students were female (85-90%) and from the Midwest. Racial, ethnic, and gender diversity were also very low. This low level of diversity is representative of the overall student population at the institution (in 2017-2018, 13.7% of domestic undergraduate students enrolled were minorities), and the high percentage of female students is typical for

Evolution phase
The curriculum of both course and lab initially evolved based on feedback from students and TAs. However, the most significant push to innovate the course came from the growth in enrollment: growing student numbers at the institution resulted in a larger population of students wanting to become elementary teachers. The required course, designed to enroll primarily freshmen and sophomores, started enrolling more and more seniors who needed the course to graduate. The addition of one additional lab section, bringing the maximum enrollment to 72 students each semester, between fall 2014 and spring 2015, was insufficient to keep up with the demand (Table 1).
To accommodate the request from the School of Education to find a solution to the enrollment increase, the instructors and TAs developed a plan that would allow 36 students to be enrolled in each lab section by renovating a large office space adjacent to the lab classroom to create a new lab space with shared projection capabilities. Starting in fall 2015, the course and lab enrollment increased to a maximum of 132 students in four lab sections each semester (Table 1). The new space allowed us to keep larger learning and modeling apparatuses, like a stream table and an Augmented Reality Sandbox, inside the new room, and use them more effectively in the lab.
This enrollment growth required significant adjustments to the lab and course curriculum. The semester-long project was dropped from the course because the instructors could not accommodate the time commitment needed to give meaningful feedback to multiple drafts and grade more than twice as many essays as in the first years. To allow a single TA to engage two groups of students in two separate rooms, we created mini-lab activities, 15-20-min hands-on activities that allowed teams of four students to rotate among them and move between the two rooms. As before, students worked in groups of four and made extensive use of the think-pair-share collaborative learning strategy (Lyman 1987).
Between fall 2015 and spring 2017 several factors remained unaltered in the course and lab, allowing us to assess the direct impact of the lab on students' content learning and attitude toward the course: same TAs, same number of sections offered at the same time, roughly same total enrollment and percentage of freshmen/sophomore vs. juniors/seniors, no changes to the online course, and a defined curriculum fine-tuned from semester to semester in response to student and TA feedback. Following these four semesters of increased enrollment, numbers returned to 24 students per section but the adjusted curriculum was retained.
Below we share the results of the assessment of the two main learning goals (1) to provide students with a solid foundation in E&SS content and (2) to make students comfortable teaching science, with a focus on the fall 2015 to spring 2017 offerings. This study was reviewed by our Institutional Review Board and deemed exempt (IRB 02-048).

E&SS content learning
Final grades are the one imperfect tool commonly used to assess students' performance in a course. With no independent validated assessment instrument available, we were not able to assess the cognitive learning outcome. Thus we limit this discussion to course performance as assessed with final grades.
Students' performance on exams, quizzes, and the semester-long project (when offered) led to a weighted total that was converted to a letter grade for the online course using a traditional scale that did not change throughout the implementation phase. The lab grade was based on three quizzes, lab assignments, and homework. Students received one grade each for the lab and the course. Average grades varied between C+ and B− in the course and B+ to A− in the lab.
The lab quizzes included multiple assessment methods designed to assess content learning and included collaborative components that reflected the environment created in the lab. Since the graded quizzes were returned to the students with feedback, as is commonly the case for lab assignments, and no copies were made, we have no way to study more in depth potential misconceptions that surfaced in the students' responses. In retrospect, we could have kept copies of students' labs for future reference.
With no independent validated assessment of E&SS learning covering the breadth of the course and with the potential biases that contribute to a letter grade (e.g., proficiency in taking multiple-choice exams, reliance on search engines to find the answers to the quiz questions rather than the reading, the possibility that TAs would 'teach to the test', prior content knowledge), we are unable to give a reliable and independent assessment of the first course outcome. Thus, we encourage groups involved in the creation of E&SS content for Next-Generation Science Standards (NGSS) to develop validated content assessment instruments that could be used to assess content learning in courses for preservice teachers.

Science teaching self-efficacy and affective response
To assess the second course outcome, we administered for three years (2013)(2014)(2015)(2016) at the beginning and end of the course a validated science teaching self-efficacy survey designed for this cohort of students (STEBI-B, Enochs and Riggs 1990;Riggs and Enochs 1990). The results of this multi-year study show a consistent increase in the students' personal science teaching efficacy (PSTE) scores between the beginning and end of the semester, comparable to other studies of small enrollment face-to-face inquiry-based courses (Gray 2017). The comparison of different curriculum components of the online course showed that the semester-long project had no significant impact on student learning and suggested that the lab was largely responsible for the growth of positive affect in our students (Cervato and Kerton 2017).
An additional measure of the students' affective response to the course is provided by the responses to the anonymous course evaluations. At the end of each semester students receive automated emails giving them access to a course evaluation. We focus on the evaluations of the lab because the weekly lab sessions and interactions with the TA provided the students with better insight into the pedagogical approach of the course. Evaluations are anonymous and instructor specific, so each teaching assistant received a separate summary report for each lab section. To encourage submission of the evaluations, every student who submitted the evaluation received a small amount of extra credit. The average response rate was 82% across all semesters.
The course evaluations consist of twelve questions/statements rated on a Likert scale from 1 (poor/low) to 5 (excellent/high) and two open-ended questions. We analyzed the summative responses to three questions: (1) What is your overall rating of the lab? (2) How much do you feel you have learned in this course? and (3) How useful were the in-class activities in helping you learn? The two open-ended questions asked the students what they liked best and least in the course, respectively. On average, each semester students submitted for each TA 34 comments on what they liked best and 13 on what they liked least. The two supervising faculty coded jointly the responses to these two questions during four semesters with no changes in instructor and content (lab and course) and the largest enrollment (fall 2015-spring 2017) based on their reference to the content, the pedagogy, or the instructor.
We examined trends in the average student responses to the three course evaluation questions (course rating, amount learned, and usefulness of in-class content) over the four semesters. Figure 1 shows that the average rating for each question, combining results from all sections, has increased or remained steady each semester, with the increase being statistically significant with a medium effect size (Table 2). We explain the increase in scores with the progressive adjustment of students and TA to the two-classroom setup and lab format changes made to accommodate the larger than usual number of students in each lab section.
The course and amount-learned ratings are closely matched (Fig. 1). However, the rating for the usefulness of the in-class material is consistently higher, suggesting that the effort in creating the diverse lab curriculum was worthwhile and that students appreciated it. This is confirmed by the coded responses to the question about what the students liked the best in the lab: 71% of the positive comments were about the pedagogy, specifically mentioning group work, fun labs, diversity of activities, engagement, adaptability of the content to their own classroom, and the minilabs. The positive effect of the TA appears in 17% of the comments and 11% make specific reference to the content. A sampling of these positive comments include "I liked that no class was ever the same" (spring 2017); "TA really encourages us to love science" (fall 2014); "All labs I feel will be very helpful and applicable to my future classroom" (fall 2015); "The hands-on activities we do in lab make something I am not particularly interested in more tolerable" (fall 2015); "Our TA made science fun, a subject which I normally stay away from!" (fall 2015); "I actually understood science in this class!" (fall 2015); "I liked the experiments and the group work" (fall 2015); and "Thanks for a great year. I almost changed my major to Geology." (spring 2015).
The comments on what the students liked least in the lab were fewer. Of those, 48% were about pedagogy, all referring to the homework or the duration of the lab, not the collaborative learning or inquiry style. One-third of the negative comments were about the TA and 16% about the content.
While we do not have the ability to test the longer-term outcomes of our second goal by following our students after they graduated and assess their classroom practices, we speculate that their perception of the amount they have learned and usefulness of the learning material are important first-level proxies for their future classroom choices: if they found the class and lab useful and if they had a higher science teaching self-efficacy at the end of the course than they did when they started it, they might be more likely to be comfortable teaching the content.

Retrospective analysis of course design
Our assessment of the affective course goal, making our students comfortable teaching E&SS content to K-5 students, by measuring the students' self-efficacy to teach science, suggests that we successfully achieved that goal (Cervato and Kerton 2017). However, we cannot state with confidence if we met our content knowledge goal. When we set out to design this course, we did what many instructors do: create learning outcomes that focus on the content of the course. As scientists and educators with more than a decade of teaching experience, but no experts in pedagogy, we decided to keep it simple and agreed that the course would be successful if our students had a 'sufficiently solid foundation on Earth and Space Science content.' However, we did not think thoroughly at what that actually meant, or how we could effectively assess if the course successfully achieved it.
Since this course and lab were developed with a specific student population in mind, we could not use other physical geology labs as a control group. Students enrolled in physical geology labs come from a wide range of majors, including science programs, and would be a poor analog to our homogenous cohort of education majors. The curriculum and pedagogy of the course/lab described here also differed significantly from the ones used in the physical geology labs taught by geology graduate TAs selected based on their schedule and not their teaching experience as was the case with the TAs for this course. Without a control group, we cannot say if our students would have achieved a similar level of content learning with a different kind of instruction.
Without a viable control group, grades are an inadequate measure of content learning, evaluating instead the students' level of success in the course. It was not until we set out to carefully evaluate the cognitive outcomes after eight years that we realized that the content goal was too vague. Is a B− average evidence of a 'sufficiently solid foundation'? A letter grade is an artifact of the grading scale that we selected, the points assigned to each assignment, and the weighting of each component: it is not an independent measure of content learning.
Without a pretest, we also had no way to determine what our students already knew about E&SS and what conceptions they had developed in grade school. Concept inventories administered at the beginning and end of courses are often used to quantitatively assess content learning. Although we were familiar with the Geoscience Concept Inventory (GCI, Libarkin and Anderson 2005), its focus on plate tectonics and geologic time covered just three of the 14 modules of this course. The questions in the Astronomy Diagnostic Test (Zeilik 2002) only covered one additional module. So in our partial defense, it was the lack of suitable validated content assessment instruments that stopped us from developing a robust content learning assessment plan for the course. If we were to design this course today, we would probably rephrase the first outcome to 'provide course content that would increase the scientific literacy of future elementary teachers' and assess it using the Science Literacy Concept Inventory (SLCI) instrument (Nuhfer et al. 2016(Nuhfer et al. , 2017, an instrument that did not exist when this course was created. The other shortcoming of our assessment is based on the assumptions we made on our TAs. Although the assessment of metacognitive responses of our students suggest that our choices had a positive impact on our students' motivation, we did not conduct observations of our TAs or attempted to assess how they succeeded at teaching inquiry labs as Ryker and McConnell (2014) did.
There were other factors that affected the development and implementation of the course, and the students' perception of it: the hybrid nature of it, curricular changes derived from growth in enrollment, and technical challenges due to institutional decisions to change course management platform were challenges both for the instructors and the students. However, the curriculum that resulted from all of these combinations of factors, particularly the lab one, is innovative and led to the growth in science teaching self-efficacy of our students.

Limitations and recommendations for future work
There are limitations to this study. Our results are limited to the time that the students were enrolled in the lab, and our conclusions have not been confirmed by a longitudinal study of effective teaching practices of our students once they became teachers.
Since the course is no longer offered, we are unable to continue this study on E&SS curriculum development. However, we hope that our groundwork may be useful to other instructors and researchers who can learn from our experience and improve upon it.
An additional limitation of our metacognitive assessment is the low diversity of our student population and how our results are restricted to the specific researchintensive institution where the course was taught. While our results are based on responses and scores of close to 1200 students, a group that is inherently diverse in terms of engagement, level of commitment, and achievement, we cannot generalize these results and assume that they are representative of the overall population of preservice elementary students. Implementations of this curriculum at other institution types and in other parts of the country are needed to confirm, or disprove, our results.

Conclusion
Self-efficacy growth and ratings from student evaluations from eight years of a hybrid online and lab course indicate that, while we cannot assess the E&SS content that was learned by students, we can be certain that our affective goal was met-students were more comfortable with science content and more confident in teaching science concepts after taking this course. Given the assumption that if preservice teachers like science more, they are more likely to teach it, the long-term potential for increasing science skills in elementary teachers and students is large.
The impact that the students who took this course over the eight years (~ 1200, Table 1), assuming a 60% 6-year graduation rate (The National Center for Education Statistics 2018), means that ~ 700 of them have become, or will soon become, elementary teachers. The number of children that they will influence over a 40 + years career is staggering. Given the shortage of STEM majors (PCAST 2012), anything that can be done to encourage these teachers to like science, become familiar with science, and be unafraid of science is well worth the investment. This curriculum provides a model that can be a starting point for implementation and further research.