Fostering Self-Reflection and Meaningful Learning: Earth Science Professional Development for Middle School Science Teachers

Abstract

This paper describes the analysis of teachers’ journal reflections during an inquiry-based professional development program. As a part of their learning experience, participants reflected on what they learned and how they learned. Progress of subject matter and pedagogical content knowledge was assessed though surveys and pre- and posttests. We found that teachers have difficulties reflecting on their learning and posing meaningful questions. The teachers who could describe how they reasoned from evidence to understand a concept had the highest learning gains. In contrast those teachers who seldom or never described learning a concept by reasoning from evidence showed the smallest learning gains. This analysis suggests that learning to reflect on one’s learning should be an integral part of teachers’ professional development experiences.

Appendices

Appendix A: Representative survey items to determine a participant’s orientation toward science teaching

Representative survey item
Based on your own views of what is important in teaching science, rate the following statements using the following scale: (1) not at all important, (2) of little importance, (3) quite important, (4) very important
Example expository [E] and an inquiry [I] survey item:
[E] Asking students’ questions to hold them accountable for knowing the facts
[I] Encouraging students to ask questions

Appendix B: Pre- and posttest sample questions

Test section	Topic of focus	Sample test question
1	Teachers’ knowledge about the FOSS Earth History science curriculum	Directions: Use the photograph of Chuar Butte (page 18 of the Earth History Resource Book) to answer the following questions:
		Which kinds of rocks tend to form the cliffs, and slopes of Chuar Butte? And why?
2	Teachers’ knowledge about students understanding of earth science topics	Listed below are the three main types of weathering processes. Beneath each type, make a list of concepts you think your students will need to learn in order to have an in-depth understanding of this topic
3	Teachers’ knowledge of instructional strategies for teaching the FOSS Earth History science curriculum	Describe what instructional strategies you would use in order to facilitate student learning. Be sure to include any learning materials, and or pedagogical skills that would aid students in developing an understanding of specific concepts or relationships

Appendix C: Rubric for scoring open-ended test questions

Point value	Description
4	The response demonstrates understanding of pedagogical content knowledge (PCK) for teaching science or a topic-specific concept beyond the Mastery level
3	Mastery level—Response demonstrates a fundamental understanding of PCK for teaching science or a topic-specific concept that is being addressed in question
2	The response has essentially correct elements; only minor mistakes or minor information is left out
1	The response contains related information but significant information has been left out
0	No response is given, or information does not apply

1. Modified from FOSS Earth History module p. 387 (http://www.fossweb.com/)

Appendix D: Equations of normalized gain

Symbol	Definition and formula
g The single-student normalized gain	Defined: g = % Gain/% Gain max Formula: g = (% posttest − % pretest)/(100 − % pretest)
〈g〉 The class (course) average normalized gain	Defined: The actual average gain, %〈Gain〉, divided by the maximum possible actual average gain, %〈Gain〉max where %〈posttest〉 and %〈pretest〉 are the final (posttest) and initial (pretest) class percentage averages Formula: 〈g〉 = %〈Gain〉/%〈Gain〉 max 〈g〉 = (% 〈posttest〉 − % 〈pretest〉)/(100 − % 〈pretest〉)

1. Modified from Hake (2001)

Appendix E: Coding for question #1: “What Did You Learn?”

Code indication	Description and example
Vocabulary	A teacher defines a geologic term
	Example: “I learned that hardness is the resistance of a given mineral to scratching”
Concept	A teacher describes or mentions a concept, idea, relationship or limitations of these
	Example: “I learned that the hardness of quartz is much greater than calcite”
Science kill (Topic-specific)	A teacher describes or mentions skills
	Example: “I learned how to do a hardness test”
Teaching skill (Topic-specific)	Teacher describes or mentions a method or topic specific techniques for teaching Earth science
	Example: “I learned how to use fieldtrips (weathering rate of tombstones in the local cemetery) to help my students understand how acid rain effects the local environment”
I learned about…	A teacher does not say explicitly what it was that she/he learned
	Example: “I learned about terminology used to describe mineral properties”

1. Modified from Etkina (2000); May and Etkina (2002)

Appendix F: Coding for question #2: “How Did You Learn It?”

Code indication	Description and example
Learned from authority	A teacher describes learning a concept by instructor, textbook, handouts, and or reference material (FOSS Earth Science CD)
	Example: “I learned through reading assorted compiled texts”
Learned by doing	A teacher describes learning a concept or process by performing or practicing it
	Example: “I learned about the contour map activity by actually doing it”
Collaborative learning	A teacher describes learning a concept or process through discussion, activity or investigation with a small group of peers
	Example: “I learned by sharing (in a small group) different ways of explaining topographic maps and how important these skills are especially for the standardized tests”
Learned by reasoning from evidence	A teacher describes learning a process or concept by performing or practicing it while reasoning from evidence
	Example: “I learned this by performing this hands-on activity with dilute HCL. If the rock began to bubble than it contained calcium carbonate which is a mineral”
Learned by applicability	A teacher describes learning a concept by relating it to an everyday experience or activity
	Example: “I learned the information by actually relating what I have been doing with my students to the discussion we had in class. Example: “I learned about erosion by making observations on a trip around the campus”

1. Modified from Etkina 2000; May and Etkina 2002

Appendix G: Coding (by topic) for question #3: “What Concepts Remain Unclear?”

Code category	Description and example
Concept	The student is concerned with the meaning of a concept or the extension of the concept into a different area and tries to make sense out of it
	Example: “I am confused about the concept of uplift”
Application	The student tries to relate the concepts to her everyday experience
	Example: “How to determine the elevation along a contour line on a NJ map?”
Knowing	The student is interested in how a particular piece of knowledge was constructed by scientists
	Example: “How do scientists know the age of rocks?”
Pedagogy	The student (teacher) needs clarification on how to teach the concept, or process to students
	Example: “How do I explain crossbedding to my students?”

1. Modified from Harper et al. (2003)

Appendix H: Coding (by difficulty) for question #3: “What Concepts Remain Unclear?”

Level of difficulty	Definition/example from participants’ journal	Similar to Bloom’s taxonomy
Minimal level	Questions that are seeking factual knowledge	Knowledge
	Example: “What is crossbedding?”
Low level	Questions that are seeking comprehension of what was previously learned	Comprehension
	Example: “As I tried to determine what was the sequence of plateau, river, canyon, some ideas are not completely clear”
Medium level	Questions that are seeking a better understanding of the data, processes, or skills needed to complete a task or problem (application). Or, questions that are seeking to distinguish, classify, and relate assumptions hypotheses, and evidence (analysis)	Application and analysis
	Example: “How do I relate the contour mapping activity to elevation differences in the Grand Canyon?”
High level	Questions are those seeking to synthesis ideas into a problem or idea that is new (synthesis)	Synthesis and evaluation
	Question that is related to the assessment or critique of a specific standard, topic or science processes (evaluation)
	Example: “How does this (classifying rocks) help us understand the history of the area that the rock came from?”

1. Note: To assess the questions based on their level of difficulty four codes (Minimal, Low, Medium, High) were adopted from Etkina (2000) and Harper et al. (2003). They fit well with Bloom’s taxonomy of the cognitive domain (Bloom et al. 1956)