Improving and Assessing Computational Thinking in Maker Activities: the Integration with Physics and Engineering Learning

Abstract

Computational thinking (CT) is believed to be a critical factor to facilitate STEM learning, and a vital learning objective itself. Therefore, researchers are continuing to explore effective ways to improve and assess it. Makerspaces feature various hands-on activities, which can attract students with diverse interests from different backgrounds. If well designed, scaffolded maker activities have the potential to improve students’ CT skills and STEM learning. In this study, we explore ways to improve and assess physics and engineering integrated CT skills through developing maker activities and assessments, which are applicable in both informal and formal educational settings. Our paper presents our work on improving and assessing CT in maker activities with two primary goals. First, it introduces the maker activities and instruments we developed to improve and assess CT that are integrated in physics and engineering learning. Second, it presents the students’ CT skill and disposition change from pretest to posttest in two summer academies with CT enhanced maker activities, which was respectively led by after school educators and formal educators in a public library.

Appendix

Appendix 1. Internal CT integrated achievement tests

4. Arduino/Programming

Appendix 2. Items on self-report survey

Computational thinking disposition: Please indicate to what degree you agree with each of the following statement about your attitudes toward problem solving. (Options: Strongly disagree, Disagree, Neutral, Agree, Strongly agree)

• 1. I have high confidence in dealing with complex problems.

• 2. I can handle complex problems if I keep trying.

• 3. I can solve complex problems if I have enough time.

• 4. I have high persistence in working with difficult problems.

• 5. I try as hard as I can when I face difficult problems.

• 6. I am willing to spend extra time and effort when solving difficult problems.

• 7. I have a great ability to deal with open ended problems (problems that do not have one single solution).

• 8. I am able to handle the problems that do not have one single answer.

• 9. I have a great tolerance of ambiguity (uncertainty and clarity) during problem solving.

• 10. I have a great ability to communicate and work with others to achieve a common goal or solution.

• 11. I am a great team player when working in teams.

• 12. I can work in teams productively.

Frequency of using computational thinking skills. Please indicate the frequency of your taking each of the following actions when solving problems. (Options: Almost never, About 1/4 of the time, About 1/2 of the time, About 3/4 of the time, Almost all the time)

• 1. I break down a complex problem or system into smaller parts that are more manageable and easier to understand.

• 2. I organize resources to work out a possible solution through the most efficient and effective steps.

• 3. When facing a complex problem (phenomena), I gather the general characteristics and filter out unimportant details to solve the problem.

• 4. I find the similarities or patterns within and among problems to find a solution.

• 5. I reduce complexity and find the main idea through models.

• 6. To solve many problems I develop step-by-step solutions or rules to follow.

• 7. After solving a problem, I think about how the solution could be improved.

• 8. After figuring out a solution, I examine whether the solution properly solved the problem and if it did so efficiently.

• 9. I compare the pros and cons of multiple possible solutions to a problem and pick the best solution.

Self-evaluation of maker activity related knowledge and CT skills. Please check your CURRENT knowledge about each of the following topics on a scale of 0 (No knowledge at all) to 10 (Very knowledgeable).

• 1. Electric circuits

• 2. E-textiles

• 3. Makey Makeys

• 4. Arduinos

• 5. Breadboard

• 6. Makerspace activities

• 7. Overall computational thinking

• 8. Computational Thinking (CT): Decomposition

• 9. CT: Abstraction

• 10. CT: Algorithm design

• 11. CT: Pattern generalization

Appendix 3. Appendix C. Scoring rubrics for CT achievement test

Part A. Background knowledge

1 What do you know about electric circuits? How do they work? Where might you see them in your life? (8 points)

	What do you know? How does it work?	Where did you see?
0	Nothing, I do not know, or completely wrong	Nothing, I do not know, or completely wrong
1	One correct	One example
2	Two correct	Two example
3	Three correct	Three example
4	Four or above correct	Multiple or general correct, or two or above example etc.

Possible answers:

• Parallel

• Series

• How parallel circuit work

• How series circuit work

• circuit transfer electricity/conductive path

• power source or provide power

• complete loop

• Energy transfer

• battery negative and positive

• simple circuits and complex circuits

• explain simple

• explain complex

Part A 2. What do you know about Makey Makeys? How do they work? (4 points)

	What do you know? How does it work?
0	Nothing, I do not know, or completely wrong
1	One correct
2	Two correct
3	Three correct
4	Four or above correct

Possible answers:

• board connected to a computer/power source

• use alligator/wire/circuits

• ground connection

• arrow connection /e.g., up, down, left, right, space, and click

• control a program/work as keys

• Use fruits

• use body

• Any conductive material (worth 2 points)

Part A 3. What do you know about Arduinos? How do they work? (4 points)

Points	What do you know? How does it work?
0	Nothing, I do not know, or completely wrong
1	One correct
2	Two correct
3	Three correct
4	Four or above correct

Possible answers:

• Uno board/hardware connect with a computer

• Breadboard for compartments/circuits

• Code in computer/use C++

• Computer control the board by program/software

• Work without connecting to the computer/Like a mini computer /microcontroller

• Device

• Sense light

• LED glow

• Speakers

• Buzzers

Part A 4. What is computational thinking? Provide several specific computational thinking skills and explain each. (8 points)

Points	What is CT? CT components	Explanation of each component
0	Nothing, I do not know, or completely wrong	Nothing, I do not know, or completely wrong
1	One correct	One correct
2	Two correct	Two correct
3	Three correct or above	Three correct or above
4	Four or above correct	Four or above correct

Part A 5. Please give example(s) for how you use computational thinking in life and/or study. (4 points)

Points	How do you use computational thinking in life and/or study?
0	Nothing, I do not know, or completely wrong
1	One correct
2	Two correct
3	Three correct
4	Four or above correct

Part B. Maker activity questions

1. Please design and draw a circuit diagram for each of the following scenarios. When you draw the circuit diagram, feel free to use the provided symbols to make your drawing easier if you’d like

Part B 1(1) Design and draw a circuit with one light bulb and one switch, the light bulb lights up. (6 points)

	Symbols	Function
0	Nothing, I do not know	Nothing
1	Something but incorrect, e.g., just draw a light bulb symbol without anything else.	Working but not appropriate, wrong design, e.g., use a series circuit
2	drawing, not use symbols correctly, use mixed drawing and symbol, e.g., use both light picture and light symbols; use the LED light but in the wrong direction; use a battery symbol but put it in the wrong way.	Working partially appropriate, correct design, but not completely right. e.g., use a parallel circuit but without a switch.
3	Correctly use symbols,	Working completely appropriate, e.g., use a parallel circuit with a switch in the main route.

Part B 1(2) A student builds a circuit described in (1), but the light bulb does not light up. What would you suggest him to do to fix his circuit? (4 points)

Points	Criterion
0	Nothing, I do not know, or completely wrong
1	One correct
2	Two correct
3	Three correct
4	Four or above correct

Part B 1 (3) Make a circuit which contains 2 light bulbs and one switch. The switch controls both light bulbs. If either light bulb breaks, the other light bulb will not light up. (6 points)

	Symbols	Function
0	Nothing, I do not know	Nothing
1	Something but incorrect, e.g., just draw a light bulb symbol without anything else.	Working but not appropriate, wrong design, e.g., use a series circuit
2	Drawing, not use symbols correctly, use mixed drawing and symbol, e.g., use both light picture and light symbols; use the LED light but in the wrong direction; use a battery symbol but put it in the wrong way.	Working partially appropriate, correct design, but not completely right. e.g., use a parallel circuit but without a switch.
3	Correctly use symbols,	Working completely appropriate, e.g., use a parallel circuit with a switch in the main route.

Part B 1 (4) Make a circuit which contains 2 light bulbs and one switch. The switch controls both light bulbs. If either of the light bulbs breaks, the other light bulb will still light up. (6 points)

	Symbols	Function
0	Nothing, I do not know	Nothing
1	Something but incorrect, e.g., just draw a light bulb symbol without anything else.	Working but not appropriate, wrong design, e.g., use a series circuit
2	drawing, not use symbols correctly, use mixed drawing and symbol, e.g., use both light picture and light symbols; use the LED light but in the wrong direction; use a battery symbol but put it in the wrong way.	Working partially appropriate, correct design, but not completely right. e.g., use a parallel circuit but without a switch.
3	Correctly use symbols,	Working completely appropriate, e.g., use a parallel circuit with a switch in the main route.

Part B. 2 (1) E-textiles: A fashion designer wants to decorate a hat by sewing 5 LED lights that require a 3 V battery. Draw a diagram to reflect your design. You may use the following symbols or any other ones to make your drawing easier if you’d like. (8 points)

	E-textile components	Step
0	Nothing	Nothing
1	One component	Mentioned one of the four major steps
2	Two components	Mentioned two of the major steps
3	Three components	Mentioned three of the major steps
4	Four components and more	Mentioned four or more steps

B2(2) Please list necessary steps to make the E-Texile above, so that the fashion designer can follow your instruction to finish the decoration. (6 points)

	Symbols	Function
0	Nothing, I do not know	Nothing
1	Something but incorrect, e.g., just draw a light bulb symbol without anything else.	Working but not appropriate, wrong design, e.g., use a series circuit
2	drawing, not use symbols correctly, use mixed drawing and symbol, e.g., use both light picture and light symbols; use the LED light but in the wrong direction; use a battery symbol but put it in the wrong way. .	Working partially appropriate, correct design, but not completely right. e.g., use a parallel circuit but without a switch.
3	Correctly use symbols.	Working completely appropriate, e.g., use a parallel circuit with a switch in the main route.

3. Makey Makey. Miguel sets up a Makey Makey as pictured below. It is connected to a computer and all the connections work well. Miguel wants to use the fruits to trigger the space key on his computer, what should he do? (2 points)

Score	Step
0	Nothing is mentioned or something wrong
1	Touch or move banana 5, or talking about holding ground.
2	Connect banana 5 with orange

B4. Please draw a circuit diagram for it. Feel free to use the following symbols to make your drawing easier. (6 points)

	Symbols	Function
0	Nothing, I do not know	Nothing
1	Something but incorrect, e.g., just draw a light bulb symbol without anything else.	Working but not appropriate, wrong design, e.g., use a series circuit
2	drawing, not use symbols correctly, use mixed drawing and symbol, e.g., use both light picture and light symbols; use the LED light but in the wrong direction; use a battery symbol but put it in the wrong way.	Working partially appropriate, correct design, but not completely right, e.g., use a parallel circuit but without a switch.
3	Correctly use symbols.	Working completely appropriate, e.g., use a parallel circuit with a switch in the main route.

6. Arduino/Programming

Look at the lines of code from the “Blink” sketch for Arduino. (4 points)

	13	Twice the other one
0	Neither blank has 13.	Did not figure out the twice relationship, or simply double 1000 to 2000 or divide 1000 to 500
1	One blank has 13	Used the twice relationship but in the wrong way.
2	Both blanks have 13	Use the twice relationship correctly.