Comparing Simulation Sequencing in a Chemistry Online-Supported Project-Based Learning Unit

Abstract

Numerous studies illustrate the value of simulations or project-based learning approaches to enhance the learning of science. Simulations can help students connect across macroscopic, microscopic, and symbolic representations of scientific phenomena, while project-based learning can provide a meaningful narrative and activities for students to engage in. Despite such studies, less is known about simulation sequencing within project-based learning and how such sequencing may influence conceptual understanding. This study investigates such simulation sequencing within an online-supported project-based learning unit that focuses on a drought narrative to teach solution chemistry concepts for 10th graders (n = 72). Specifically, we compare two treatments with students randomly assigned to using a simulation before (planning treatment; n = 37) or after (reflection treatment; n = 35) the design of desalination units. This quasi-experimental study includes a pre/post conceptual test, embedded conceptual measures, a student questionnaire, a teacher interview, and classroom observations. Results show significant pre/post gains for all students on solution chemistry concepts. Findings show no overall difference between the two treatments. Student and teacher comments, and classroom observations illustrate affordances and constraints for teaching solution chemistry concepts through a technology-supported project and point to future directions for iteration. We discuss the implications of this work for future technology-supported projects and science education.

Appendices

Appendix A: Curriculum

Students first completed an introduction section, “Water Supply and Demand”, that placed the project in the context of the California drought as the use of contexts is more effective at the start of instruction (Bennet et al., 2005; Bulte et al., 2006; See Fig. 2). Students shared their perspectives and current ideas/experiences about the drought.

Fig. 2

Context for the Desalination Project

The second section, “Understanding Desalination”, required students to engage with a simulation that focused on explaining aspects of heat transfer and concentration connected to desalination. This section included an embedded PhET simulation (Fig. 3 ) that links evaporation with molarity, a key concept for desalination (https://phet.colorado.edu/sims/html/concentration/latest/concentration_en.html). Students followed the KI framework with this simulation where their initial ideas were elicited, they added to those ideas through engaging with the simulation, distinguished their initial and new ideas, and reflected on their new understanding. This section was placed in the second last section of the project for the reflection treatment (Sect. 6 in the Reflection Treatment).

Fig. 3

PhET Simulation Embedded in WISE

The third section, “Design your Desalinator”, asked students to plan out their desalination unit design. This section took advantage of a WISE draw tool that allows students to illustrate what their design would look like and explain some of the reasoning for their designs. The teacher introduced this as a competition for students in that the group with the most freshwater would be the winner and would get extra credit. The teacher also matched the salinity of the ocean with the saltwater to make the unit more authentic. Further, the teacher used a blue food coloring to ensure students obtained water free from the food coloring.

The fourth section, “Build and Test Your Desalinator”, allowed students to start building their desalination units with students building a wide variety of desalination units (Fig. 4). Students took pictures of their designs and uploaded them to WISE. Students also added data they collected based on the internal temperature of their desalination units from outside testing.

Fig. 4

Students Planning and Designing their Desalination Units

The fifth and sixth sections (“Design a Revised Desalinator” and “Test a Revised Desalinator”) guided students to refine their desalination unit by changing one thing in their desalination unit and testing it again. The final seventh section asked students about their final thoughts on how a desalination unit works and how desalination on a large-scale could be helpful for areas affected by drought.

Bennett, J., Gräsel, C., Parchmann, I., & Waddington, D. (2005). Context-based and conventional approaches to teaching chemistry: Comparing teachers’ views. International Journal of Science Education, 27(13), 1521-1547. https://doi.org/10.1080/09500690500153808

Bulte, A. M. W., Westbroek, H. B., de Jong, O., & Pilot, A. (2006). A research approach to designing chemistry education using authentic practices as contexts. International Journal of Science Education, 28(9), 1063-1086. https://doi.org/10.1080/09500690600702520

Appendix B: Pre/Post and Embedded Items

Item 1: Car on Cold Day (Adopted from McBride et al., 2016 )

On a cold day, Akbar walks to his car that is parked in the sun and has not been driven for a week.

Question 1: Predict the temperature of the air inside the car:

a) Colder than the outside air;

b) Warmer than the outside air;

c) Exactly the same as the outside air

Question 2: Explain your answer.

KI Rubrics

Main ideas:

• Energy comes from the sun

• Energy from sun moves by radiation/ light/ waves/ inside the solar oven

• Energy from the sun/solar radiation/light transforms/changes into/turns into/becomes heat OR VICE VERSA

• Warm air/heat is trapped/blocked by windows

KI Score	KI Level	Example
1	Off task	I don't know, I guessed.
2	Non-normative ideas: Heat comes from the sun Metal/dark material attracts heat Vague explanations	Akbar's temperature inside of the car is exactly the same as the outside temperature due to the air circulating.
3	Partial ideas: including at least one main idea	Since it hasn't been driven in a week, it is safe to assume the heater hasn't run in the car for at least that time. The day is also cold, making the temperature in the car cold, too. The sun probably won't have that much power, too, because it isn't warming Earth up much.
4	Normative ideas: including at least two main ideas	It will feel warmer inside the car because the heat from the sun is being transferred to the particles inside of the car, however, since the windows are all closed and the car has not been used, the hot air is trapped inside of the car, making it seem more stuffy and hot inside the car than outside.
5	Disciplinary links: including at least three main ideas	The heat is trapped within the car. The car has direct sunlight and due to conduction within the car and radiation from the sun, the temperature inside the car will be higher than outside.

Items 2: Car on Hot Day Item (Adopted from McBride et al., 2016 )

Laura wants to make sure her car does not get too hot while it is sitting outside during the summer. What do you think is most important to help Laura keep her car cool?

Question 1: What options are important to keep the car cool while it sits in the sun?

• a) Using a light-colored paint on the outside

• b) Using a dark-colored paint on the outside

• c) Using light-colored fabric on the inside of the car

• d) Using dark-colored fabric on the inside of the car

Question 2: Explain your choice.

Question 3: Laura is also going to use a sunshade on her windshield. One side of the sunshade is very shiny, and the other side is not shiny. Which way should the shiny part face?

• a) Toward the inside of the car

• b) Out of the car

KI Rubrics

Main ideas:

• Light colors will reflect the sunlight (reflect heat: non-normative) AND reduce the heat/temperature

• Dark colors will absorb heat/solar radiation (attract, capture, trap: non-normative) AND increase the heat/temperature

• Solar radiation converts to heat inside the car OR warm air is trapped

KI Score	KI Level	Example
1	Off task	I don't know, I guessed.
2	Non-normative ideas	The outside doesn't actually matter so then the fabric inside the car depends on how hot or cold it is, so if it is a dark color fabric is it is going to be hot. The sun is reflecting so if it is towards the car it is going to be extremely hot and if it is facing out it is going to be cooler than the shiny facing towards the inside of the car.
3	Partial ideas: including at least one main idea	If Laura chooses light-colored paint on the outside, her car will attract less heat. Dark colors such as black attract more solar radiation, because of the fact that it is black. The same goes for the inside of her car and with the color of her fabric. Since the shiny part is similar to aluminum foil, it will help keep Laura's car cool. It will only attract more solar radiation because, but because it is reflective the light will bounce back but not toward the car since it is facing away. This will protect the car from the heat coming into the car and will help the car be cool.
4	Normative ideas: including at least two main ideas	Darker colors such as black absorb the suns light and heat. If you have a darker colored car that is left out in the sun it is going to absorb all of the heat. This will cause all of the air inside of the heat up as well. The light-colored interior will also help to keep it cool. The light colors do not absorb as much heat and energy as dark colors do. The shiny metallic side of the sunshade is a reflector. This means that when the sun hits the shade its light will bounce right off. This will deflect the suns heat and energy away from the car keeping it cool. If you would face the reflector side toward the car it would push the sun rays towards the car heating it up
5	Disciplinary links: including three main ideas	When things are black the color absorbs wavelengths of light and turns them into heat and the color white reflects wavelengths of light. The shiny part should be faced outside because if its out then the shade will act as a reflector and will reflect the sun's light/heat away from the car.

Item 3: Desalinator Shape (Energy; Adapted from McBride et al., 2016 )

A student named David compared two desalinators made from boxes with the same amount of space inside. One is skinny and tall, while the other is wide and short. Which desalinator would heat up faster? David made this claim:

“I think the skinny and tall desalinator will heat up faster than the wide and short desalinator because the space is smaller, so it will let less energy leave the desalinator so more evaporation can occur.”

Question 1: Explain why David's claim is correct or incorrect using the evidence you collected from the model. Be sure to discuss how the movement of energy causes one desalinator to heat up faster than the other.

KI Rubrics for Desalinator Shape (Energy)

Main ideas:

• One that is wide and short has more area or space to heat up faster/let in more solar radiation (light)/absorb more energy

• One that is tall and thin/has less area/a smaller opening to heat up slower/let in less solar radiation (light)/absorb less energy

• More energy heats up faster OR Less energy heats up slower

• Solar radiation changes into/becomes heat energy

KI Score	KI Level	Example
1	Off task	I don’t know.
2	Non-normative ideas	David's claim is incorrect because the wide and short desalintator did take more time to rise in temperature but it absorbed the most heat.
3	Partial ideas: including at least one main idea	David's claim is incorrect because the evidence collected from the model shows that the wide box heated up at a faster rate and continued at the rate. The faster the molecules meant the faster the boxes heated up.
4	Normative ideas: including at least three ideas	David's claim is incorrect because a wide and short desalinator allows solar radiation and infrared radiation to move around more quickly, and also generates heat more quickly than in one that is tall and skinny.
5	Disciplinary links: including four main ideas	David's claim is incorrect because the tall and skinny box reached 30 degrees Celsius in one minute and thirty seconds while the wide and short box reached 37.3 degrees Celsius in one minute and thirty seconds. The short and wide box got hot faster because it had more surface area. This greater surface area allowed the box to get hot faster because it got hit with more sunlight giving it more energy.

Item 4: Desalinator Shape (Desalination; Adapted from McBride et al., 2016 )

Question 2: Will having a “skinny and tall” or a “wide and short” desalinator be better for evaporation? Explain your answer.

KI Rubrics for Desalinator Shape (Desalination)

KI Score	KI Level	Example
1	Off task	I don’t know.
2	Non-normative ideas	Having a skinny and tall desalinator would be better because there is less surface area.
3	Partial ideas: only mention wide or short and that it heats up faster/absorb heat without explaining why	Having a wide and short desalinator will be better for evaporation because of its ability of being able to absorb heat.
4	Normative ideas: discuss solar radiation/energy/sun rays heats up faster OR greater surface area/wider area heats up faster	Wide and short desalinator will be better for evaporation because there will be more surface area of the plastic wrap on the unit to collect the water.
5	Disciplinary links: discuss solar radiation/energy/sun rays heats up faster AND greater surface area/wider area heats up faster	Having a wide and short desalinator will be better for evaporation for several reasons. One is because the desalinator is able to absorb most of the heat energy from the solar radiation. Two is because there is more surface area so more sunlight will be able to hit the desalinator. Three is because the more sunlight there is, the quicker the increase in temperature and the more water would be evaporated.

Item 5: Heat Saltwater (Drawing)

What happens when we heat saltwater?

Question 1: Using the stamp tool on the left, show the position of the particles in a saltwater solution before and after they are heated.

KI Rubrics for Heat Saltwater (Drawing)

Representations:

• Show water particles and salt particles distributed amongst each other (Before Heating)

• Show salt particles at the bottom of the container (After Heating) and has the same number of salt particles before and after heating.

• Show water particles gone, some left, and/or towards the top of the container (After Heating)

KI Score	KI Level	Example
1	Non-normative ideas: incorrect representations	Show the loss of salt particles in the ‘after heating’ diagram Show separation of salt and water in the ‘before heating’ diagram Show the same amount of water particles in both diagrams
2	Partial ideas: including one representation	Have at least 1 of the representations noted
3	Normative ideas: including two representations	Have at least 2 of the representations noted
4	Disciplinary links: including three representations	Have all three representations noted

Item 6: Heat Saltwater (Explanation)

Question 2: Include a description of your diagram, explaining what happens to saltwater when it is heated.

KI Rubrics for Heat Saltwater (Explanation)

Main ideas:

• Salt particles are distributed amongst the water molecules initially (with some sitting at the bottom)

• The water molecules evaporate from heating

• The salt particles remain in the container after heating

• The concentration increases

• Increasing temperature increases solubility of salt

KI Score	KI Level	Example
1	Off task	I don’t know.
2	Non-normative ideas	Salt evaporates/Less salt No water is lost Salt does not dissolve Molecules are packed more closely together after heating Molecules are spread out before heating Less salt in the container because it dissolves from heating
3	Partial ideas: including at least one main idea	The water is being evaporated so it goes up.
4	Normative ideas: including at least three main ideas	Before the saltwater solution is heated, the salt is at the bottom of the solution and it is mixed in with the water. After being heated, salt particles are still present. It still remains at the bottom of the solution. The water evaporates, separating the salt from the water.
5	Disciplinary links: including at least 4 main ideas	The water molecules before the heat is applied, are sitting in the container bouncing around with the salt. After the heating occurs, the water molecules evaporate, and the salt remains, and the concentration increases.

Item 7: Distinguish (Drawing)

What’s the difference between salt (s) and salt (aq)?

Question 1: Using the stamp tool on the left, draw the structure of salt as a solid and of salt in an aqueous solution.

KI Rubrics for Distinguish (Drawing)

Representations:

• Show the formula unit of NaCl (s).

• NaCl (s)—Show the Na and Cl particles stacked in a repeating pattern.

• NaCl (aq)—Ions are represented as opposed to the formula unit.

• NaCl (aq)—Show Na⁺ and Cl⁻ ions surrounded by water molecules.

KI Score	KI Level	Example
1	Non-normative ideas	Show no coherent pattern in the organization of Na and Cl ions, not even the formula unit Include water in the NaCl (s) representation
2	Partial ideas	Have at least 2 representations listed
3	Normative ideas	Have at least 3 representations listed
4	Disciplinary links	Have 4 representations listed

Item 8: Distinguish (Explanation)

Question 2: Include a description of your diagram, explaining any differences between NaCl (s) and NaCl (aq).

KI Rubrics for Distinguish (Explanation)

Main ideas:

• NaCl (aq) has NaCl dissolved/mixes and surrounded by water.

• NaCl (s) is in a crystal/compacted/block structure with no water.

KI Score	KI Level	Example
1	Off-task	I don’t know.
2	Non-normative ideas	NaCl (s) and NaCl (aq) both combine to form sodium chloride. NaCl(S) has two chlorines bonded too one sodium while NaCl(aq) has two sodiums and two chlorines. The salt as a solid starts off together as one whole molecule in an aqueous solution. Then goes off and splits up between NaCl (s) and NaCl (aq).
3	Partial ideas	In the first diagram the salt is in a solid where the molecules are close together, in the solution the molecules are spread out.
4	Normative ideas	NaCl (s) stays a solid, so you can see the particles, but NaCl (aq), dissolves in water when put in a aqueous solution. The difference is that you cannot see the salt once it is put in water, due to it dissolving.
5	Disciplinary links	On the left box, you can see salt as a solid form without any water, it is compacted together because it is a solid. On the right box you can see salt mixed with water, and this makes the salt move all over the place not being a solid anymore, but instead dissolved in the water.

Item 9: Concentration (Choice; Adopted from Adadan & Savasci, 2012 )

Figure 5 shows a 1 M solution of sugar dissolved in water. The dark triangles in the magnification circle represent sugar molecules. In order to simplify the diagram, the water molecules have not been shown. Figure 2 shows the solution after one fourth of the solution is poured out.

Fig. 5

1 M solution of sugar dissolved in water.

Question 1: Which of the diagrams below best represents the magnification circle after one fourth of the 1 M sugar solution is poured out? (Binary Coding: 1—Correct Diagram Selection (a); 0—Incorrect Diagram Selection (b or c)).

Item 10: Concentration (Explanation)

Question 2: Explain your choice.

KI Rubrics for Concentration (Explanation)

KI Score	KI Level	Example
1	Off-task	I don’t know, I guessed.
2	Non-normative ideas: choose (b) or (c) and give wrong explanations	It would be (b) because the sugar molecules would start dissolving so there would be less than before.
3	Partial ideas: choose (a) but give wrong or vague explanations	Although 1/4 of the solution was poured out, the concentration didn't not increase or decrease since there was not addition or subtraction of the solute.
4	Normative ideas: Choose (a) and know that the concentration of the solution does not change	The sugar molecules stay the same in Fig. 3 because the amount of the sugar does not change since it is dissolved into the solution. The concentration stays the same.

Item 11: Concentration (Drawing; Adopted from Devetak et al., 2009 )

The diagrams show two solutions in Beaker A and Beaker B. In Beaker A, the solution is double the amount in beaker B. Use the stamp tool on the left to represent the solute. Water molecules are omitted for clarity.

Question 3: Create 3 frames using the 'New Frame + ' tool on the right. (Binary Coding: 1—Correct Diagram Representation; 0—Incorrect Diagram Representation).

• In Frame 1, represent the solute such that the concentration in Beaker A is the same as in Beaker B.

• In Frame 2, represent the solute such that concentration in Beaker A is half the concentration in Beaker B.

• In Frame 3, represent the solute such that the concentration in Beaker A is one third the concentration in Beaker B.

Representations:

• Frame 1: 2n solute particles represented in Beaker A and n solute particles represented in Beaker B.

• Frame 2: n solute particles represented in Beaker A and n solute particles represented in Beaker B.

• Frame 3: n solute particles represented in Beaker A and 1.5n solute particles represented in B, where n is 2 or greater.

Concentration Item (Embedded)

Question 1: Make a prediction! What happens to the mass of solute in the solution when there is increased evaporation? Explain your choice.

KI Rubrics for embedded Concentration Item 1

KI Score	KI Level	Example
1	Off task	I don’t know.
2	Non-normative ideas	The mass of the solute increase/decreases with wrong explanations. The mass of the solute stays the same, but the explanations do not match.
3	Partial ideas	The mass of the solute stays the same but without explanations.
4	Normative ideas	Salts are not decomposed by evaporation, so the mass of the solute stays the same.
5	Disciplinary links	The salt particles require far more energy to decompose due to stronger forces (ionic bonds) so they are left behind, which means that the mass of the solute stays the same.

Question 2: Make another prediction! What happens to the concentration of solution when there is increased evaporation? Explain your choice.

KI Rubrics for embedded Concentration Item 2

KI Score	KI Level	Example
1	Off task	I don’t know.
2	Non-normative ideas	The concentration of the solution stays the same/decrease. The concentration of the solution increases but the explanations do not match.
3	Partial ideas	The concentration of the solution increases but without explanations.
4	Normative ideas	Salts are not decomposed by evaporation, so the mass of the solute stays the same. The mass of solvent (water) decreases because of evaporation and the mass of solute stays the same, which means there is the same mass of solute dissolved in less water, so the concentration of the solution increases.
5	Disciplinary links	The water evaporated because the particles have enough energy on the surface to break intermolecular forces (hydrogen bonding) and escape as a gas. The salt particles require far more energy to decompose due to stronger forces (ionic bonds) so they are left behind. There is the same amount of salt dissolved in less water so the concentration increases.

Drawing Item (Embedded)

Question: What happens when we heat saltwater? Using the stamp tool on the left, show the position of the particles in a saltwater solution before and after they are heated. Include a description of your diagram, explaining what happens to saltwater when it is heated.

KI Rubrics

Representations:

• Show water particles and salt particles distributed amongst each other (Before Heating)

• Show salt particles at the bottom of the container (After Heating) and has the same number of salt particles before and after heating

• Show water particles gone, some left, and/or towards the top of the container (After Heating)

KI Score	KI Level	Example
1	Non-normative ideas: incorrect representations	Show the loss of salt particles in the ‘after heating’ diagram Show separation of salt and water in the ‘before heating’ diagram Show the same amount of water particles in both diagrams
2	Partial ideas: including one representation	Have at least 1 of the representations noted
3	Normative ideas: including two representations	Have at least 2 of the representations noted
4	Disciplinary links: including three representations	Have all three representations noted

Desalination Item (Initial Ideas and Final Ideas; Embedded)

Question: Explain how you think a desalinator works. Use scientific concepts to help you explain.

KI Rubrics

Main ideas:

• Energy comes from the sun (Radiation/Sunlight)

• Water is evaporated and condensed/separated

• Salt is not evaporated/left behind

KI Score	KI level	Examples
1	Off task	I don’t know.
2	Non-normative ideas: Heat comes from the sun (Instead of radiation). Salt is evaporated (instead of water).	My thoughts on how a desalinator works is that when the sun heats up the solute, the with that it evaporates up, but with some kind of conduction, it falls down to ad open and transfers to water. So basically, it evaporates then transfers. A desalinator works by using heat from the sun and making saltwater turn into freshwater.
3	Partial ideas: Water is evaporated OR Water is condensed (NOT BOTH)	The water can get heated either through convection or conduction. The radiation from the sun heats up the air inside the unit, then through convection the water is heated by the air. The water can also be heated through conduction by the water being hit by direct sunlight.
4	Normative ideas: including two complete main ideas	A desalinator works by evaporating the water from the salt, then condensing the water vapor and putting it into another container. Saltwater is heated causing the water to evaporate leaving the salt behind. This way we get freshwater.
5	Disciplinary links: including three complete main ideas	A desalinator uses the sun's solar energy to separate fresh water from saltwater. The sun's energy needs to be absorbed though to heat up the saltwater so that is why the box, aluminum, paper, and plastic wrap are needed. Once the energy is absorbed it tries to escape through evaporation and the fresh water evaporates to the surface, separating itself from the saltwater.

Item Explanations

The Heat Transfer concept was examined by Desalinator Shape (Desalination) Item in the pretest and posttest. Students were asked to explain ideas about the ideal shape of a desalination unit in terms of function with ideas of heat transfer (radiation, conduction, convection) expected to be addressed. Similarly, the students were also asked twice in the main project to explain how a desalination unit works (Desalination (Initial Ideas) Embedded Item and Desalination (Final Ideas) Embedded Item).

The Solubility concept was targeted with a draw step for students to show the positions of particles in a saltwater solution before and after it is heated (Heat Saltwater (Drawing) Item) in the pretest and in the posttest. This question was also asked in the project (Drawing Item (Embedded)). It was expected to show an understanding of saltwater solubility at the molecular level.

The Concentration concept was examined by the Concentration Item in the pre/post-tests and the project. The question on pre/post-tests required students to select and explain the appropriate microscopic representation of particles when one fourth of a one molar sugar solution is poured out (The concentration doesn’t change; Concentration (Choice and Explanation) Item). The embedded concentration question in the project asked students to explain the change of solute mass and solution concentration when there was increased evaporation (Concentration Item (Embedded)).

Appendix C: Results for Quantitative analysis

Table 5 Inter-rater reliability coefficients and agreement percentage for pre/post-test items.

Table 6 Inter-rater reliability coefficients and agreement percentage for embedded items.

Cronbach's Alpha is one of the most important and pervasive statistics in research involving test construction and use (Cortina, 1993, p. 98), to report the internal consistency and quality of conceptual items within the pretests and posttests. The Cronbach’s alpha for pretest and posttest overall was 0.782. We further sorted the 11 items within the pre/post-tests into three concept categories which were the same as the concepts we targeted by the embedded items. The concept of heat transfer and solubility contained four items respectively, leaving three items for the concept of concentration. The Cronbach’s alpha for the conception of heat transfer was 0.509, for the conception of solubility was 0.777, and for the conception of concentration was 0.561. We interpret these values as illustrating fairly high consistency in whole items for conceptions of solution chemistry in the context of desalination, but being impacted by the number of items in the three sub-categories of heat transfer, solubility, and concentration (Taber, 2018). Additional open-response items may improve the Cronbach’s alpha, but would potentially increase testing fatigue. We list all the items (see Appendix B) so they can be reviewed for judgements of face equivalence (Taber, 2018). Tables 5, 6, 7, 8, 9, 10, 11, 12 and 13.

Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98–104. http://dx.doi.org/10.1037/0021-9010.78.1.98

Taber, K. (2018). The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Research in Science Education, 48(6), 1273-1296. https://doi.org/10.1007/s11165-016-9602-2

Table 7 Results for Individual Items between pretests and posttests.

Table 8 Overall scores for pretests and posttests by gender.

Students for different genders gained significantly within each treatment, however, there were no differences between treatments for female students (F (1, 56) = 0.143, p = 0.707) and for male students (F (1, 12) = 1.737, p = 0.212).

Table 9 Overall scores for pretests and posttests by class period.

Students for different class periods gained significantly within each treatment, however, there were no differences between treatments for class period 1 (F (1, 13) = 1.052, p = 0.324), for class period 2 (F (1, 17) = 0.845, p = 0.371), for class period 3 (F (1, 16) = 0.390, p = 0.541), and for class period 4 (F (1, 18) = 0.507, p = 0.486).

Table 10 Overall scores for pretests and posttests by prior knowledge.

For prior knowledge, we used a mean split to identify LPK and HPK. The mean for the pretest was 26.67, so students scoring 26 or less were categorized as LPK while students with 27 or higher were categorized as HPK. Students for different prior knowledge groups gained significantly within each treatment, however, there were no differences between treatments for the LPK group (F (1, 36) = 0.709, p = 0.405) and between treatments for the HPK group (F (1, 32) = 0.278, p = 0.602).

Table 11 Results for embedded Desalination Item by treatment.

Appendix D: Representative students’ responses

Affordances

Theme 1: Theory into (Repeated) Practice

“I like the online simulations that we do and videos we watch because they help us understand what we're learning a lot more and in more detail at our own pace.”

Female Student 251670

“I like doing projects because it gives me a hands-on experience and it challenges me into really understanding what is going on.”

Female Student 251698

“Hands-on learning activities with less "teacher gives instructions that you must follow exactly or have a failing grade" gives students (like me) a better understanding for the lesson taught. It also allows us to learn to deal with frustration and success from the multiple trials that must take place.”

Female Student 251574

“Doing projects challenges you as a student and I like those challenges because it tests your capabilities of accomplishing and finishing the project.”

Female Student 251580

“I like doing projects because they show how what we have learned during class may be applied to the real world. Also, hands-on projects help me remember topics as I am able to see the ideas in action.”

Male Student 251,675.

“I like doing projects in chemistry class because they allow me to take the concepts that we learn and apply them to real life situations.”

Male Student 251,585.

“I like using technology because it is helpful when doing a project. If I need help with a question or need to do research technology is very helpful. Through using it I can better understand the assignment, which improves my overall project.”

Male Student 251,570.

Theme 2: Meaningful Learning

“I feel that doing projects is hands on and I like learning in that way. I tend to retain more knowledge and be more engaged.”

Female Student 251693

“In chemistry class I like doing projects because it allows me to challenge myself and to also be creative with my ideas. It is different from just sitting in class and learning about something, and it is definitely more hands-on.”

Female Student 251581

“It's a fun new way to learn instead of learning straight out of a book and take test.”

Female Student 251571

“I like doing projects because it’s like a hands-on thing. I learn better if I’m doing it so when we do projects, I feel like I get a better understanding on what we are learning.”

Female Student 251699

“Projects makes things easier to learn since we're testing things ourselves and it makes chemistry more interesting.”

Female Student 251670

“I feel that doing projects is hands on and I like learning in that way. I tend to retain more knowledge and be more engaged.”

Female Student 251693

“With technology, more information and knowledge can be gained. It also allows for the project work to be done faster and it is kept safe in one spot.”

Female Student 251698

“I like using technology because it’s easier to organize the stuff and get a better understand of it.”

Female Student 251676

“I feel that doing project enables me to learn better, instead of just taking notes and reading out of book which is boring to me and I cannot learn like that.”

Male Student 251674

Constraints

Theme 3: Pen and Paper Ease of Use

“Using technology can be very difficult for some people because they might not have internet access or sometimes the questions are a bit confusing. Also, at times you can't view all the questions.”

Female Student 251,571.

“I neither like or dislike using technology because sometimes it can have so complications rather than using pencil and paper, but it allows me to gather information faster.”

Male Student 251,585.

“I don't mind using technology, but sometimes in the classroom it can be frustrating or difficult to work with, mostly due to problems with devices.”

Male Student 251,563.

Table 12 Agreement Percentage for Initial Codes.

Table 13 Theme Identification Progress.