Teaching Students to Grasp Complexity in Biology Education Using a “Body of Evidence” Approach

Abstract

Understanding how ecosystems scientists characterize causal dynamics in ecosystems and the approaches that they use to build disciplinary knowledge should inform curriculum in biology. Research shows that they view these relationships within a complex matrix of interactions, subject to inherent and often irreducible variability. This makes experimentation involving manipulation of variables not always possible or the best approach. In these situations, they use a “Body of Evidence” (BOE) approach. It entails integrating results of multiple approaches, measuring and describing variability, conducting experiments in context, taking advantage of natural experiments, thinking across levels, and considering the limits to generalizability.

We conducted a case study to assess the impact of teaching seventh graders a “Body of Evidence Approach” infused into a computer simulation-based ecosystems curriculum by contrasting the understanding of two classes, one with and one without the BOE components. We found shifts in how BOE students viewed the importance of evidence and focused on constructing a compelling body of evidence in support of each claim. However, they constructed less of the causal explanation than students without the BOE focus. This suggests that students can learn a BOE approach, but that more time may be needed to fulfil these broader curriculum goals.

Appendix

9.1.1 Overview of the Plus BOE Curriculum

Overview:
Day One: Essential question of the day: How can I get to know an ecosystem through exploration? Students begin exploring EcoXPT and focus on getting to know the layout of the world, what organisms live there (both micro- and macroscopic) and how the field guide tool works. They are introduced to a thinking move called Deep Seeing	Unlocked: Camera Field Guide Submarine Notebook	Locked: Data View Calendar Water Tools Weather Tool Population Tool Atom Tracker Concept Map Lab (includes scientists)
Day Two: Essential question of the day: How might things change in an ecosystem over time? Students continue exploring EcoXPT and focus on traveling over time and seeing what can be learned on different days. They may also start collecting water quality measurements and gathering data for those measurements across time. The weather tool, population tool and Data View are also unlocked on the second day and some students will find them and use them. They will be more formally introduced on Day Three	Unlocked (in addition to what was unlocked on previous days) Calendar Water Tools (Weather Tool) (Population Tool) (Data View)
Day Three: Essential question of the day: How can I collect evidence to help me figure out what’s going on? Sometime during Day Two and Three, students will have found the fish die-off. If they have not yet found it by the beginning of Day Three, they are guided to exploring the date of July 28. They focus on their initial hypotheses about what may have happened and begin collecting evidence in support of their hypotheses. They are introduced to the move of Evidence Seeking. As they collect pieces of information, or evidence for what might be happening in the world, they are able to collect evidence in relation to each claim. The opening PPT draws their attention to the Population Tool, Data View and Weather Tool
Day Four: Essential question of the day: How can I look for patterns that suggest what might be going on? Students continue seeking evidence in support of their ideas about what happened to the fish. They are introduced to the move of Pattern Seeking as they explore patterns in the data that suggest what might be going on
Day Five: Essential question of the day: How can I start to connect the information that I’m gathering? Students continue seeking evidence in support of their ideas about what happened to the fish. They are introduced to a Concept Mapping Tool that will help them to make possible connections and seek evidence for each claim represented in their concept map	Unlocked: Concept Map
Day Six: Essential question of the day: How can I use experiments to answer the questions that I have about what’s going on? Students continue seeking evidence in support of their ideas about what happened to the fish and exploring patterns in the data that suggest what might be going on. Once they have discovered patterns between algae, bacteria and the fish die off, typically on Days Five or Six, they are introduced to the differences between correlation and causation and the Analyzing Causality Thinking Move. The “Lab Building” and related tools are unlocked so that they can begin to conduct experiments to confront some problems in reasoning only from patterns and will begin to see how it is important to explore the mechanisms behind the patterns. The Atom Tracker Tool appears on the Tool Bar but is not discussed until Day Seven	Unlocked: Lab (includes:  Lab building  Tracers  Mescosm  And related Scientist NPCs) (Atom Tracker)
Day Seven: Essential question of the day: How can I continue to use experiments to test my claims, collect evidence and build causal connections? Students focus on asking questions about what might be going on in the ecosystem and on studying through experimentation and other forms of investigation about what might be happening. They continue working with the Evidence Seeking and Analyzing Causality moves to hypothesize about what might have happened in the world. The Atom Tracker is introduced	(Atom Tracker)
Day Eight: Essential question of the day: How can I think about what parts of my explanation seem incomplete and what else I need to fill those gaps? Students step back and reflect upon what they do and do not know and to focus on getting the information that they need to really understand what is going on. As part of a class discussion, they consider the difference between seeing patterns and determining causality. They continue to refine their questions and to make sure that they have evidence to back up their claims
Day Nine: Essential question of the day: How can I use multiple pieces of evidence and multiple types of evidence to further develop my explanations about what’s going on? Session Nine introduces the Body of Evidence Approach. Students learn from the PPT and the BOE Thinking Move how the BOE approach requires using multiple pieces of evidence and multiple types of evidence and how this can help them to evaluate the overall strength of each claim and to consider the level of certainty or uncertainty that is possible for each claim. Students evaluate two Bodies of Evidence and then evaluate their own explanations to see how they can further collect evidence to support their growing claims	*Remind students to talk to new NPC- Dr. Aziza Al Dahan
Day Ten: Essential question of the day: How can I construct a scientific explanation about what’s going on? This session picks up where Day Nine left off as students continue piecing through their explanations. They continue conducting experiments and using the evidence from their experiments to understand, as fully as possible, what is going on in the ecosystem. They are introduced to the “Constructing Explanations” Thinking Move. It is used along with the Concept Mapping Tool to support them in making sense of the “big picture” as they put all of their clues together
Day Eleven: Essential question of the day: How can I think about the values and limits of different types of evidence? Students transition from building their concept maps to finishing compiling their evidence and preparing to present their work to others. Students focus on building the fullest explanation that they can with their concept maps. As they are working, the teacher circulates and helps them to find gaps in their explanation. They use confirming and disconfirming evidence to support their explanation. With help from the visual cues/codes in the concept maps, they reflect on the kinds of evidence that they are using (patterns, textual information from the field guide, testimony from characters and outcomes from experimental studies) and figure out if there may be information that is missing from their explanation
Day Twelve: Essential question of the day: How can I communicate my findings about what’s going on? For the first third of class, students continue preparing their concept maps to present to the class. They make sure that all of their evidence is listed and that there are no gaps in their explanations. They include confirming and disconfirming evidence in their concept maps. The teacher then stops them and asks them to carefully review their evidence and concept maps. Then the computers are put away and for the rest of class, students write up an individual essay explaining what they think happened to the fish
Day Thirteen: Essential question of the day: How can I communicate my findings about what’s going on? Students share their findings for what happened at the pond. They are charged with listening carefully to each other’s presentations and to help their classmates discover what is well-supported in their arguments and where evidence for claims may be missing. If conducted as a whole class discussion, it is facilitated so that all of the students are able to contribute aspects of the complex causal scenario underlying what happened in the ecosystem. The session underscores that a good explanation is a well-supported, well-reasoned one in which the mechanisms for the causal connections are explained
Day Fourteen: Essential question of the day: How can I reflect on my experience in EcoXPT? This is a day of reflection on the big lessons from EcoXPT. It is not about the explanation that they came up with but about the messages that they learned about science, ecosystems science and coming up with an explanation. Students have an opportunity to reflect upon their own ideas and then the class has a discussion about it

9.1.2 Experimentation Tools in EcoXPT

Experimentation Tools in EcoXPT
	The Tolerance Tanks display three virtual fish tanks, each with a different type of fish and allow students to test any of seven factors to see if different levels of those factors would directly kill each type
	The Comparison Tanks display two virtual fish tanks within a 3D lab environment. Each tank has an associated shelf of objects: a fan, a fish, a plant, or acid. Students choose to fill each tank with either pond or tap water and select up to one (or “none”) objects to place in each tank. Once the tanks are set up, students can “run” the experiment and use the water measurement tools to see the results
	The Mesocosm Tool allows students to investigate how real-world contextualization interacts with the behavior of the variables that they combine in the pool. They consider how changing temperature, levels of nitrates, etc. interact over time. They configure up to four pools with up to two factors each. Once the pools are set up, student can “run” the experiment and use the water measurement tools to see the results
	The Tracer Tool allows students to understand the movement of matter in the environment. They can test how the spatial lay-out and topography play a role in the process. They can choose to place tracers of different colors in different places. The tool allows then to understand how the spatial terrain interacts with the movement of materials
	Buoy Sensor Data is collected over time in the pond. Students can access this data to understand changes in the pond over time that ultimately, they will realize, are relevant to understanding what happened to the fish. They access the buoy data by talking to a scientist at the edge of the pond (Dr. Hsieh) who has a tablet that enables them to access the information.

1. Note: Reprinted with permission from EcoXPT Teacher’s Guide and Resource Materials available at: https://ecolearn.gse.harvard.edu/projects/ecoxpt

9.1.3 EcoXPT Thinking Move Posters Including a Body of Evidence Approach

9.1.4 Script for Body of Evidence Approach Thinking Move Video

• Building a Body of Evidence Thinking Move:

• Wow, how cool is it that we get to use the experiments in the lab now?! Experiments can help test whether a pattern is actually a causal relationship. This evidence is useful because it helps us construct causal claims about what’s going on in the world.

• But we can’t always conduct an experiment. Here are some examples:

• Imagine you wanted to study the impacts of fires on forest ecosystems. You wouldn’t burn the forest down just to see the impacts. That would harm the ecosystem and the organisms that inhabit it! OR imagine you wanted to increase the CO₂ in the forest to find out what the long-term impacts are. This experiment might hurt the organisms and could also take many years to conduct.

When they can’t conduct an experiment, scientists use something called the Body of Evidence Approach. A Body of Evidence Approach is when scientists look for multiple pieces of evidence and many different types of evidence in order to support their claim. Gathering multiple pieces and types of evidence from different sources reduces the uncertainty of the results.

But remember, a Body of Evidence Approach can be used even when we can conduct an experiment! Experimental results are just one of the many types of evidence that we can use to support our claims.

Remember that there are many types of evidence that we can collect in EcoXPT. Consider talking to people and other scientists, as well as using your observations, data and information you’ve collected from opportunistic experiments. Doing this will also help fill in some of the gaps you may have in your explanation!

In EcoXPT, use a Body of Evidence Approach, just like ecosystems scientists do. Be sure to use multiple types of evidence to support your claims. You can make sure that you are doing this by checking the evidence for the links in your Concept Map. Check to see that you’re using multiple pieces of evidence and evidence from different sources, by clicking on the arrow between factors you’ve used to build connections.

When you’re using the Building a Body of Evidence Thinking Move, remember to:

• Use multiple pieces of evidence to support each claim.

• Use multiple types of evidence.

• Evaluate the overall strength of the evidence for each claim.

• Consider the level of certainty or uncertainty that is possible for each claim.

9.1.5 Body of Evidence Worksheet

9.1.6 Thinking About Different Types of Evidence Worksheet (Both Classes)

9.1.7 Supporting Materials for Body of Evidence Thinking Move

Thinking Moves Scientists Use	Try this:	Ask:
Building a Body of Evidence Instead of focusing mainly on discrete pieces of evidence, scientists consider what the collection of evidence suggests in order to support a causal claim. They gather multiple pieces and forms of evidence. They evaluate the strength and weaknesses of the collection of evidence. They consider their level of certainty and uncertainty about the claim based upon what the collection of evidence can support.	It is not always possible to conduct an experiment to test for causality. However, if the collection of evidence is varied (especially if it includes natural contrasts or opportunistic experiments), extensive and highly suggestive of causality, a causal claim may be warranted. Make sure that you consider the body of evidence through the same questions as you would for “Evidence-Seeking” above. Include information about the strengths and weaknesses of your body of evidence in your explanation. Include information about your level of certainty and uncertainty, as scientists do, when offering a causal explanation.	Have I included multiple and diverse pieces of evidence (including data from observations, patterns), experiments (including natural contrasts and opportunistic experiments) and trustworthy sources? Have I evaluated the body of evidence carefully (as per the “Evidence-Seeking” guidelines above)? Have I included information about the strengths and weaknesses of my body of evidence in my explanation? Have I included information about my level of certainty and uncertainty for specific claims, as well as what claims the body of evidence supports, in my explanation?

Accompanying Teacher Pedagogical Moves to Support Student Thinking Moves:
Building a Body of Evidence Approach:
Help students to realize ways that there are different kinds of information and that some are more useful in determining causality than others.  Help students to evaluate the trustworthiness of claims by considering whether claims appear to predict outcomes. For instance, if a claim states that adding phosphates and nitrates should increase algae levels, is that what happens when they do?  Help students think about other cases that are hard to test but the overwhelming evidence points in a certain direction. For instance, it is difficult to link behaviors like smoking to cancer but over the years, a body of evidence supported the finding of a causal relationship.  Help students to think about instances that are hard to test, such as processes that take a long time to reveal outcomes or where there are many possible interacting causes. These are often cases when a Body of Evidence Approach is helpful.

1. Note: Reprinted with permission from EcoXPT Teacher’s Guide and Resource Materials available at: https://ecolearn.gse.harvard.edu/projects/ecoxpt

9.1.8 Learning from Opportunistic Experiments

9.1.8.1 Discussion Sheet

Experimentation is easier to conduct in some disciplines than others. Ecosystems scientists do conduct small scale experiments in a lab, but when they want to understand changes in the broader environment, they need to rely on a variety of approaches. One of these approaches is using “Opportunistic Experiments” or “Natural Experiments.” They involve studying changes that happened either through natural processes or unintentionally by humans or other animals.

“Opportunistic Experimentation” or “Natural Experiments” are often used in cases where an intentional experiment would cause harm or would be unethical, for instance to an ecosystem or a population of people. For example, if you wanted to know if chemicals are harmful to a pond, scientists wouldn’t fill one pond with the chemical and compare it to another pond without it. But if a chemical spill releases the chemical into a pond, they could study it and compare it to other ponds. Similarly, if scientists what to know the impact of environment on children, they can study identical twins but they can’t send one twin to live in a different environment. However, if they find twins who were somehow separated at birth, they can study their differences.

In EcoXPT on Lesson Day 9, there is a scientist by a small woodland pond and she is studying what happened to the pond such that it turns bright green. She discovers that a farm worker moved a manure pile such that the runoff began entering the pond and explains this to the students.

Consider the following questions:

• 1. Did any of you meet a scientist on Day 9 at a small woodland pond that had turned green? If so, what did you learn from her?

• 2. What do you think opportunistic experiments are? Why are they so important in ecosystems science?

• 3. Ecosystems scientists adopt a “do no harm” approach. Do you think this means that they never conduct an experiment in which an organism dies? Are there any instances in which this might be justified?

• 4. What other examples of opportunistic experiments or natural experiments can you think of? Make a list together as a class.

Note: Reprinted with permission from EcoXPT Teacher’s Guide and Resource Materials available at: https://ecolearn.gse.harvard.edu/projects/ecoxpt

9.1.9 Uncertainty and Constructing a Best Explanation

9.1.9.1 Discussion Sheet

Scientists aim to construct the very best explanation that they can with the available evidence. Often it is not possible to definitively know the “right answer.” Therefore, it is important that scientists talk about uncertainty and the sources of scientific uncertainty in their work.

The focus is a little different than talking about your personal certainty or uncertainty. There are always things that we as people don’t know. Scientific uncertainty is more about what we do or do not have the data to support and even if it is possible to know something.

Scientific uncertainty is especially important when we are constructing explanations about the past. Think about fossil evidence, for example. We can use what is left behind to create the best story about what happened but since we can’t travel backwards to the time of the dinosaurs we will never know for sure. Even for more recent events that we did witness, there are often different perspectives and different sources of data on what happened. Recall the last time you had a disagreement with a friend. You probably both give a different explanation.

Even when you are present to observe something happening, there can be uncertainty about what happened in between the times you are there. For instance, in EcoXPT, you only visit the pond during the day and so it is hard to know what happens when you are not there. So when you take measurements, you have the day to day data but you don’t have the data points in between. When you collect data, you are guessing that there is a straight line between the data points, but you cannot be certain.

Sometimes new information causes scientists to revise their explanations. Revising explanations is part of how science works. An explanation can be the best one for a certain period of time and then new evidence might suggest an even better explanation. Even so, the old explanation may have been very helpful in the meanwhile.

It is common to hear scientists:

• 1. …express uncertainty. (The data suggests that it might be due to this cause but we still have further questions about other possible causes.)

• 2. …talk about how much certainty they have in a set of findings. (We have a lot of certainty in these findings because we have seen this outcome so often.)

• 3. …express certainty at some levels of analysis of a problem and not at others. (We know how this chemical behaves in a lab but we don’t know what happens over time in the broader ecosystem.)

• 4. …talk about certainty in some contexts but say that it is not generalizable to other contexts. (We know that these findings are reproducible in these contexts but in other contexts with changes in variables such as temperature, moisture levels, etc. they may not be reproducible.)

Consider the following questions:

• 1. In what ways is scientific uncertainty similar to and different from personal uncertainty? (Think of examples when you didn’t know something because you didn’t have the information yet but it was knowable. Think of examples of when you didn’t know something because it was unknowable.)

• 2. What are some instances when scientists might talk about uncertainty?

• 3. What does it mean to give the best possible explanation?

• 4. What are some places in EcoXPT where there are sources of uncertainty?

Note: Reprinted with permission from EcoXPT Teacher’s Guide and Resource Materials available at: https://ecolearn.gse.harvard.edu/projects/ecoxpt