Advertisement

Concept learning by direct current design challenges in secondary education

  • Dave H. J. van Breukelen
  • Marc J. de Vries
  • Frank A. Schure
Open Access
Article

Abstract

This paper presents a mixed methods study in which 77 students and 3 teachers took part, that investigated the practice of Learning by Design (LBD). The study is part of a series of studies, funded by the Netherlands Organisation for Scientific Research, that aims to improve student learning, teaching skills and teacher training. LBD uses the context of design challenges to learn, among other things, science. Previous research showed that this approach to subject integration is quite successful but provides little profit regarding scientific concept learning. Perhaps, when the process of concept learning is better understood, LBD is a suitable method for integration. Through pre- and post-exams we measured, like others, a medium gain in the mastery of scientific concepts. Qualitative data revealed important focus-related issues that impede concept learning. As a result, mainly implicit learning of loose facts and incomplete concepts occurs. More transparency of the learning situation and a stronger focus on underlying concepts should make concept learning more explicit and coherent.

Keywords

Learning by Design Physics Technology Concept learning Electric circuits 

Introduction

Science and technology play an important and increasing role in our modern world. However, international studies, e.g. ROSE (Sjöberg and Schreiner 2010), indicate this is not followed by an increasing interest in and understanding of science and technology among juveniles. To counter that, more meaningful and motivating teaching methods based on interdisciplinary teaching are necessary (Lustig et al. 2009; Osborne and Dillon 2008). In response to this the integration of science, technology, engineering and mathematics (STEM) has become a main topic within educational systems (Rennie et al. 2012) where (designing) technology, due to its wide contexts and informative activities, has the means of becoming the catalyst for integration (Clark and Ernst 2007).

Based on Roth (2001) technology is described as the entire set of activities that leads initial vague ideas through construction and testing of prototypes to a final model. This model, including all the knowledge and skills its creation entails, solves a problem or improves a pre-existing solution. On this basis, the potential of teaching science through designing technology is that the design task provides the context for applying science knowledge and science concepts provide content needed for design realisation. Many attempts to respond to this strong interplay of science and technology appear to be unsuccessful (Lustig et al. 2009; Osborne and Dillon 2008). Nevertheless, reasonably successful approaches like “Learning by Design” (LBD; Kolodner 2002b) show that proper integration can bring significantly better collaboration skills, meta-cognitive skills (e.g. checking work) and science and technology skills (e.g. fair testing). An essential prerequisite for success, however, is a teacher’s deep understanding of the design process. Science teachers, for example, often may not be able to adapt their science teaching methods to activities that involve designing technology (Sidawi 2009; Wendell 2008). In view of this it is obvious that technology and design teachers have to take a leading role. Therefore, they need know that LBD suffers from the fact students do not learn scientific concepts better within LBD (Kolodner 2002b; Kolodner et al. 2003a, b) which is also the main topic of this study that gains insight into why concept learning is limited and how to fortify it.

Importance for technology curricula

The introduction demonstrates technology and design education are directive for LBD and STEM education in general. However, technology education curricula should also benefit from STEM research like, for example, this LBD study. This is based on the Standards for Technological Literacy (International Technology Education Association 2007) where design activities are regarded as a core process of technology education. All too often, however, design is used as instructional strategy where product realisation has the emphasis, often by using trial-and-error as strategy (Burghardt and Hacker 2004). To tackle this problem students have to notice that conceptual knowledge and design processes cannot be divorced (Jones 1997). The goal is to produce students with a more conceptual understanding of design technology (International Technology Education Association 2007). For example, students should focus on concepts behind design realisations such as properties of materials, construction techniques and knowledge of electric circuits, where the latter concerns this study. Therefore, the ITEEA (formerly ITEA) emphasizes design challenges that rely on math and science knowledge to improve design performance. Against this background we need to say that also technology has its own network of conceptual knowledge. It is just because of the focus of this study that this conceptual framework has less attention. In practice this framework is just as important as the scientific knowledge domain. To conclude, knowledge about (the interplay of) concept learning and design processes, both important technological learning objectives, will strengthen technology curricula.

Foundations of LBD

LBD is a project-based inquiry approach where students learn, among skills and practices, scientific content through achieving design challenges (Kolodner 2002b). LBD is based on two educational pedagogies. First, problem-based learning (PBL): a cognitive apprenticeship approach that stimulates learning by collaboration, solving real-world problems and reflection (Norman and Schmidt 1992). Second, case-based reasoning (CBR): a constructivist model of learning that refers to solving new problems by adapting old solutions or interpreting new situations in light of similar situations (Kolodner et al. 1996). Thus, combining both pedagogies learning becomes problem-based, collaborative, reflective, context-related and task-related (doing and knowing). All fundamental elements in solving real-world, design-related problems. Therefore, design is a suitable context for learning, where science provides a part of the content needed for success (Sidawi 2009).

Figure 1 shows LBD is based on two cycles of activities: design and investigation. We will give a short description of how both cycles interplay. A more detailed description can be found in Kolodner et al. (2003a). To achieve a design challenge students have to explore design-related skills and concepts they need to learn/know. By carrying out investigations they learn those things in order to apply them during prototyping and eventually in the final design. Investigation of this application may lead to other things they need to learn and investigation starts again. Thus, students learn concepts and skills (science- and technology-based) that are needed for success by identifying a need to learn them, trying them out, questioning their handling and thinking, and acting again (iteration). That is how the practice of science (investigation) and technology (design) constantly interact. But, students will not necessarily identify all aspects they have to learn and not all insights will be applied properly. Therefore, teacher-guided rituals (poster session, pin-up session and gallery walk) take place for sharing experiences and ideas among design groups. This is more or less similar to how engineers engage with peers and clients (Kolodner et al. 2003b). Complemented by whole-class discussions students are assisted in understanding design-related principles and science. Table 3 shows how LBD activities take place in practice.
Fig. 1

Learning by design’s cycles. Note: Reprinted from Kolodner (2002b, p. 339)

Because concept learning has the main focus in this study it is necessary to illuminate how LBD aims for this. Generally, we can say LBD points toward a constructivist mode of education, in which one learns by extracting wisdom from experiences (Kolodner et al. 1996). It fits together with the conceptual change model that learning is a process of personal construction and that students, in an appropriate environment, will construct a more scientific framework of knowledge if they notice scientific conceptions are superior to their pre-task conceptions (Abdul Gafoor and Akhilesh 2013; Cobern 1994). For this, LBD challenges deliberately address cognitive conflicts where students’ existing ideas are no longer sufficient for succeeding. In compliance with Nussbaum and Novick (1982), and Cosgrove and Osborne (1985), LBD contains four main elements for conceptual change: (a) exploration of students’ preconceptions: preliminary phase, (b) sharpening student awareness of own and other’s framework: focus phase, (c) investigation and explanation of the conceptual conflict: challenging phase, (d) accommodation of the new (context-free) conceptual model: application phase. Furthermore, according to literature on learning, e.g. Brandsford, Brown, Donovan, and Pellegrino (2003), LBD contains several elements that (should) promote concept learning: collaboration, reflection, contextual learning, applying what is learned, learning from failures and iteration, and connecting skills, practices and concepts.

Previous LBD research and objective of this study

From 1999 till 2003 over 3500 American middle school students (ages 12–14; grades 6–8) took part in studies that compared achievements of LBD classes to non-LBD classes (Kolodner et al. 2003a, b). Results show that LBD students learn scientific concepts as well or slightly better (not significant) than comparison students with respect to knowledge transfer (mastery outside the design context). However, LBD students performed significantly better at collaboration skills, metacognitive skills (e.g. checking work, reflection) and science skills (e.g. fair testing, using prior knowledge). So it seems LBD makes students more skilful but does not care for better concept learning. This is supported by review of literature on design-based science teaching (Sidawi 2009).

This is notable because, according to the previous section, LDB theoretically provides a sound basis for concept learning. Thus, what factors impede concept learning? The results of previous LBD studies were based on a set of validated performance tasks and multiple choice tests conducted before and after the learning intervention. A detailed analysis of the LBD practice itself had less attention despite the fact it could provide more insight into the process of concept learning. Therefore, this will be the main objective of this study.

To gain insight into a hypothesis that states why concept learning is limited and to provide the study with important points of interest, literature upon design-based learning is helpful. Nearly all design-based science approaches are complex because many objects of integration (e.g. skills, practices, attitudes and content) are combined and remain under-exposed (Berlin and White 1994). Various studies give similar focus-related explanations for this. For example, expert designers focus on content because skills, practices and activities are familiar. Novices mainly focus on process-related issues, needed for success, in which content is largely overlooked (Popovic 2004). Wendell (2008) states that scientific content may not emerge because students focus on doing. For this, they try to avoid unknown content areas, because of complexity and diversity of hands-on activities that mainly dominate the process. Students rather rely on prior knowledge and assumptions (trial and error). Thus, a lack of focus on content and a dominant process focus might cause limitations in concept learning. Therefore, this study investigates where students (senior general education) focus on during LBD and, more specific, how and when scientific content is addressed and what students learn from it. Eventually, implications can be deduced for better concept learning and further research.

Methods

For this study the methodology of design-based mixed methods research was chosen. Beside quantitative data about learned science, qualitative data is necessary to investigate the learning process by a thorough analysis of events. The study took place in the second grade of senior general education (havo). 77 students (age 13–14; 33 female; 44 male), spread over 3 adjacent classrooms, were involved accompanied by 3 teachers. All students and teachers had prior experiences regarding characteristic LBD components, but the students had no specific prior knowledge with respect to the scientific design-related content.

Design of the LBD challenge

The LBD task “Back to the Nineties” was related to the physics domain “direct current electric circuits” and design groups (3 students per group, randomly chosen) were challenged to build a battery-operated dance pad that let them use their feet to sound a buzzer or flash lights. The dance pad had to consist of four operating floor pads and one main power switch. The entire activity took 5–6 class periods of 100 min and was guided by an instructive presentation and a student’s and teacher’s guide. To accomplish the task, design specifications were formulated, shown in Table 1, that stimulated the use of underlying science (A till D) and the process of decision-making and creative thinking (E till G).
Table 1

Design specifications and materials

Design specifications

A. The readily available push button serves as main power switch

B. One self-designed floor pad, recognizable by a circular form, should flash a light by stepping on it

C. One self-designed floor pad, recognizable by a triangular form, should flash a second light by stepping on it

D. To sound the buzzer you have to push two self-designed floor pads, cross and rectangular shaped, simultaneously (with two feet)

E. It is not allowed to use more design materials than available

F. The dance pad consists of one piece and can be used frequently without failure

G. The dance pad has a nice design (an eye candy) and is easy to use

Material

Quantity

Material

Quantity

1.5-V AA battery

2

Push button

1

AA battery holder

1

Light bulb

2

Aluminium foil

1 roll

Light bulb holder

2

Cardboard (4 colours)

1 sheet (50 × 70 cm) per colour

Buzzer

1

Tape (one- and double-sided)

1 roll

Electrical wire

500 cm

Regarding specifications A till D, the most fundamental (scientific) design principles concerned proper wiring (combining series and parallel parts) and a proper use of conducting and insulating materials for floor pad creation. Figure 2 shows an example of a design outcome and wiring. To investigate and design electric circuits students used real experiments and an interactive simulation (PhET™ DC-circuit construction kit). Beside proper circuit creation, the design challenge sets for more scientific objectives. Table 2 shows all objectives and their initial appearance.
Fig. 2

Design outcome and wiring

Table 2

Scientific objectives and initial appearance

DC electric circuits objectives

Appearance

1. Students can describe properties of direct current: (A) Conservation of current: current will not be consumed in a circuit; (B) Current can be seen, based on an educational model, as a substance for energy transportation

The interactive simulation shows current flow and enables current measurement

Real experimentation enables students to measure current flow

2. Applying the fact that a battery is an energy source and the driving force behind current flow. Beside a closed circuit this force is a prerequisite for a functional circuit

The effect of a power supply and circuit switching is explored during experimentation

Dance pad operation is based on circuit switching

3. Knowing the effect of series and parallel components on current flow (through a battery): parallel components increase and series components decrease current flow

Similar to objective 1

4. Recognizing and designing series, parallel and combined circuits and, with respect to this, identifying and describing circuit operation

Operation is based on proper wiring. Students have to meet design specifications A till D

Wiring can be studied by experimentation

5. Students know that conductors and insulators influence current flow: conductors enable current flow while insulators impede current flow

Students have to design floor pads by combining conducting and insulating materials (design specifications B till D)

6. Students know that circuits (in daily life) have a purpose in converting an input in an output (action)

The dance pad is a daily life example of a system based on an electric circuit

Objectives adapted from Kennisbasis natuurwetenschappen en technologie voor de onderbouw (pp. 41–44), by Oorschot et al. (2014)

Furthermore, Table 3 shows which LBD stages and activities took place to guide the process and to help students to understand underlying concepts and phenomena. In addition, a few modifications, listed below, were implemented to enrich the original LBD approach. These modifications mainly concern the usage of modern learning resources.
Table 3

Stages and activities

Stages (time)

Activitiesa

Final productsb

1. Introducing the Design Challenge and Context (20–30 min)

Introduction of context, design challenge, activities, organisation, learning sources, time schedules, materials, objectives, etc

 

2. Understanding the Challenge, Messing About, Whiteboarding (50–60 min)

Exploration of the challenge, context and objectives (G)

Writing down ideas, (research) questions and hypotheses (G): what to do and learn?

Whiteboarding: sharing results; feedback session (C)

Design diary stage 2

• Flip chart for whiteboarding (G)

3. Investigate & Explore, Poster Session (120–180 min)

Formulate and distribute (scientific) research questions (C)

Discussion “fair test rules of thumb” (C)

Design and conduct experiments, collect data, conclude (G)

Presentation of results: poster session; feedback session (C)

Discussion about results and fair testing: redoing/adjustments (C/G)

Design diary stage 3

• Final research questions (C)

• Fair test rules of thumb (C)

• Laboratory notebook (G)

Experiment poster (G)

4. Establishing Design Rules of Thumb (20–30 min)

Determination of design rules using experiment results (C)

Focus on the science content involved: use of science vocabulary and concepts (C)

Design diary stage 4

• Design rules of thumb (C)

5. Design Planning, Pin-Up Session (80–90 min)

Devise, share and discuss design solutions: divergent thinking (G)

Poster: provisional design solution (G)

Pin-up session (posters): feedback session (C)

Adjusting the provisional design solution (G)

Redoing until satisfied: final design solution (C/G)

Design diary stage 5

• Design posters (G)

• Design sketch (G)

6. Construct & Test, Analyse & Explain, Gallery Walk (120–180 min)

Prototyping and design realisation (G)

Testing the design: realization of design specifications (G)

Gallery walk: determine shortcomings; feedback/reflection (C)

Adjustments of the design rules and design solutions (C/G)

Design diary stage 6

• Prototype design (G)

7. Iterative Redesign (60–120 min)

Iteration of previous steps depending on decisions made (C/G)

Improving the design (G)

Final discussion about design solutions and scientific concepts (C)

Design diary stage 7

• Final design solution (G)

• Final reflection (individual)

C = class activity or product; G = design group activity or product

aAvailable resources: ELE, smartphones, laptops, tablets, Microsoft Office® software, interactive simulation, internet access, materials and tools for design realisation, materials for conducting experiments

bDesign diary (ELE-archived): reflections, feedback, process descriptions and pictures/movies. Bulleted lists are stage-specific

  • A fully equipped (online) electronic learning environment (ELE) with guidance for each design stage, background materials regarding skills and practices and space to collect (requested) writings, pictures, sketches, simulations, etc.

  • The possibility of using tablets, laptops and smartphones to build a digital design diary and to access digital resources like internet and simulation software.

  • The obligation to build virtual simulations in addition to real experiments, based on Finkelstein et al. (2005).

Framework for learning

Because the students’ focus and the way they use and learn science from that is the main topic of this study, literature was studied to become more informed about elements related to concept learning. This resulted in three important, closely connected, elements that were helpful in collecting and analysing qualitative data. According to Horton (2006), as shown in Table 4, a learning activity has three essential types of interaction that should contribute learning. Within these interactions five important intertwined activities can be specified. Maybe not surprisingly, all elements in Table 4 are to a greater or lesser extent part of the LBD approach.
Table 4

Learning-related interactions and elements

Type of interaction

Learning-related elements

1. Student (to Student) interaction

(A) Collaboration: sharing information enriches the individual learning process and fortifies knowledge building (Parkinson 2001; Roth 1995). Sketching and drawing helps students to externalise and share their ideas and it allows peers to review ideas (Popovic 2004; Roth 2001). The presence of the construction materials and tools, that are necessary for design creation, stimulates peer discussion about scientific concepts (Murphy and Hennessy 2001; Roth 2001)

(B) Reflection: reflecting on knowledge, skills, practices, attitudes and received feedback makes students more aware of doing and thinking and stimulates to maintain strengths or to make adjustments. Student collaboration provides input for reflection (Roth 1995)

2. Student to teacher interaction

(C) Teacher and peer feedback: providing feedback and receiving peer and expert feedback is invaluable for teaching and learning. Constructive feedback, also important for self-reflection, provides insight into doing and thinking and reveals students’ strengths and weaknesses (Kolodner 2002a; Kolodner et al. 2003b). Constructive feedback is relevant, goal-directed, well timed, behaviour-focused, collaborative, factual and respectful (Wiggins 2012) and focuses on knowledge, skills, practices and attitudes

(D) Explicit teaching: students often solve problems intuitively by using their awareness and foreknowledge (Hennessy and McCormick 1994; Roth 1995). Students rarely solve problems in a strategic way by using (scientific) domain-related knowledge (Parkinson 2001). Also new insights are rarely linked to underlying concepts. All of this results in trial-and-error behaviour (Popovic 2004). To prevent this, teachers should help students making strategic decisions and knowledge domain connections (Kolodner et al. 2003b; McCormick 1997). By doing this, processes and contents become explicit (Hennessy and McCormick 1994)

3. Student to content interaction

(E) Process-related issues: First, mistakes are an important learning source and provide information about students’ (mis)conceptions, so mistakes must not be corrected prematurely, but should be provided by feedback (Kolodner et al. 2003b). Second, experiencing different contexts in which the same concepts occur fortifies learning, because students’ knowledge is always context-related and not directly related to decontextualized knowledge domains. Through de- and recontextualisation, complemented by explication, understanding is supported (Brandsford et al. 2003; Fortus et al. 2004; Johnson 1997; Parkinson 2001). Third, time pressure impedes learning, because students do not take ownership of the learning process (Murphy and Hennessy 2001). Encouraging students using positive and constructive feedback is to be preferred. Fourth, to incentivise the learning process sufficient control of the classroom management and organisation is needed (e.g. through clear instructions and high-quality learning materials) (Bruinsma 2003). However, it is very important teachers know when to intervene and when to hold back: sensitive assistance (Murphy and Hennessy 2001)

Data collection

To get informed about the (advancement of) students’ mastery of content knowledge pre- and post-exams (multiple-choice) were used. The same exam was used for pre- and post-testing and a control group (N = 26), not taught the task-related content, was used to rule out knowledge absorbing from taking the test. Questions were based on validated multiple-choice tests that proved to uncover students’ (mis)conceptions (Engelhardt and Beichner 2004; Licht and Snoek 1986; Niedderer and Goldberg 1993). The exam consisted of 20, objective-linked, questions. Each objective was served by pairs of similar conceptual and contextual questions to investigate differences in de- and recontextualisation (transfer). Figure 3 shows two examples of paired questions.
Fig. 3

Examples of paired conceptual and contextual questions. Note: Questions are paired horizontally. Question numbers and objectives are corresponding to Tables 2 and 6

During the challenge direct non-participant observations took place to investigate students’ and teachers’ behaviours and actions. The event- and scan-based observations mainly focused on occurrence, frequency and (indirectly) absence of events. The observations were guided by observation forms to respond to the simultaneous occurrence or close temporal proximity of events. These forms included a list of behaviours and events, grouped by the learning-related key elements mentioned in Table 4, with space for describing the observation in detail.

During the learning task sound recordings were made of teacher instructions, teacher-student interaction, collaboration between students within design groups and class activities. Sound recordings provide authentic data (regarding the content of explication, reflection and feedback) and express students’ thoughts and use of science vocabulary. Especially, because students were encouraged to think aloud.

Questionnaires (mostly open-ended) were used to ask students to reflect on the learning process. Questions were based on the STARR-method that provides a framework for proper reflection (Verhagen 2011). Especially, students were asked to express their opinion on learning outcomes, disturbing elements and activities that stimulated learning. Questioning took place after the learning intervention and included all students.

For deeper understanding of students’ answers, retrospective interviews took place at the end. Stimulated-recall techniques were used to investigate the extent to which students used science consciously; according to literature a rich source of data (Popovic 2004; Rennie et al. 2012; Roth 2001). In preparation for this, student products were studied to become informed of used science and successfulness of design outcomes. Visible scientific elements were noted and served as stimulus during interviews. Sixteen students, the number data occurred to be saturated (Mason 2010), and all teachers were interviewed.

Analysis

The results of the pre- and post-exam scores will be represented by the total number of correct answers among all students and corresponding fractions. This is performed per question, for contextual and conceptual questions separately and for all questions. The fractions will be used to calculate the gain index 〈g〉. The latter is defined as the ratio of the actual average gain (%post − %pre) to the maximum possible average gain (100 − %pre) (Hake 1998). A paired samples t test and a Wilcoxon signed-rank test are used to determine the difference between pre- and post-scores. Both tests were used because frequency analysis showed the data was only approximately normally distributed. The internal consistency of the exam was tested by calculating Cronbach’s alpha for the items within the different objectives, resulting in an average correlation. Finally, a factor analysis was used to test the (amount of) assumed objectives the exam is based on.

For the qualitative data we derived guidelines for analysis from methodological literature (Boeije 2005; Trochim 2006). Table 5 gives an overview of the qualitative data collection and a brief description of the analysis.
Table 5

Overview of qualitative data and analysis

Qualitative data collection

Implementation

Analysis

Observations

Non-participant expert observations (event- and scan-based) guided by observation forms based on learning-related elements in Table 4

Grouping and categorising observations using labels equal to learning-related issues in Table 4

Sound recordings

Recordings of teacher instructions, student–teacher interaction and student–student interaction

Similar to observation analysis

Counting relative usage of scientific, design-related terms during student collaboration

Product analysis

(1) Examining successfulness of design outcomes (two experts) by scoring per design specification based on 3 categories. (2) Examining the underlying science students used for creating their products

(1) Calculating weighted Cohen’s Kappa and the percentage of successes

(2) Simply writing down which science, according to Table 2, is visible in products

Questionnaires

Students had to reflect on the learning process: learning outcomes, disturbing elements and activities that stimulated learning

Categorising and labelling, similar to observation analysis, students’ answers per question

Interviews

Students: retrospective semi-structured interviews to deepen questionnaire answers. Complemented by stimulated-recall techniques to check the extent to which students used science for the design outcome consciously

Teachers: semi-structured interviews to investigate their opinion regarding learning outcomes, disturbing elements and activities that stimulated learning

Similar to questionnaire analysis

Determining, by studying students’ reasoning, if underlying science is understood and used consciously for design realisation

Because observation forms were based on learning-related elements mentioned in Table 4, those elements were also used as label (code) for (re)grouping the observations (A: collaboration, B: reflection, C: feedback, D: explication, E: process-related). A sixth label was added (F: miscellaneous) for observations that seemed hard to define. In addition, also the type of interaction, according to Table 4, was noted. In the context of methodological triangulation we used the same labelling method for analysing sound recordings, questionnaires and student interviews. Where, in the case of questionnaires and student interviews, labelling took place per question. Sound recordings were first broken down into relevant fragments, after whereupon labelling started. Next, per data collection the data was first sorted by type of interaction, in order to specify the initial focus. Second, the data was sorted by learning-related element(s) to gain insight in the learning process. This resulted in sub-categories of common content, where each sub-category was accompanied by a short description. At this stage the two researchers, guaranteeing reliability by peer debriefing, compared their findings until agreement was reached. According to literature inter-rater agreement can be determined by dividing the number of agreements by the sum of agreements and disagreements (Bijou et al. 1968). In our case (NA = 54, ND = 8) inter-rater agreement was 0.87 which is sufficient.

Furthermore, sound recordings and interviews were analysed to get informed about an increase in learned science. First, sound recordings of student collaboration were used to investigate possible changes in verbal use of scientific terms during the process. For this, the usage of 13 predefined scientific, design-related terms was counted for different stages. Second, student products were examined by simply writing down the used science that was visible in products. Then, student interviews made clear, by looking at the quality of scientific reasoning/underpinning, whether this science was understood and used consciously.

Also, design realisations were reviewed per design specification by two experts using three categories (successful, partially successful and unsuccessful). The percentage of successful scores indicates how successful students faced the design challenge. By calculating the linear weighted Cohen’s Kappa inter-rater agreement was established.

For the qualitative part we ensured validity and reliability in several ways (Hake 2004; Niedderer and Goldberg 1993). First, all methods of qualitative data collection and analysis were based on scientific literature in order to guarantee test-validity resulting in well-founded results. Second, due to coding, peer debriefing and member checking a coherent and explicit chain of analysing and reasoning was provided. Third, we used direct investigation techniques in a real-world educational setting to avoid restricted experimental settings that may cause quasi-valid results because important impacts are ruled out. Fourth, methodological and investigator triangulation is used to check results and to interpret findings.

Results

Students’ achievements

Table 6 shows the pre- and post-exam results (experimental and control group) listed by objective. Cronbach’s alpha, for each individual objective, indicates that the questions have sufficient internal consistency. Regarding objective 1–6 (α1 = 0.80, α2 = 0.83, α3 = 0.71, α4 = 0.79, α5 = 0.66, α6 = 0.76) we find an overall alpha of 0.76.
Table 6

Results pre- and post-exam: experimental group (N = 77) and control group (N = 26)

Question information

Pre-exam

Post-exam

Difference

No.

Obj.a

Type

Score

Fraction

Score

Fraction

Abs.

GainInd.b

Gainc

Exp.

Cont.

Exp.

Cont.

Exp.

Cont.

Exp.

Cont.

Exp.

Cont.

Exp.

Cont.

Exp.

1

1

CC

26

9

0.33

0.35

53

10

0.68

0.38

27

1

0.52

0.06

Medium

6

1

CC

11

3

0.14

0.12

39

3

0.50

0.12

28

0

0.42

0.00

Medium

5

1

CT

12

2

0.15

0.08

55

2

0.71

0.08

43

0

0.65

0.00

Medium

9

1

CT

4

2

0.05

0.08

45

4

0.58

0.15

41

2

0.55

0.08

Medium

12

2

CC

21

7

0.27

0.27

53

8

0.68

0.31

32

1

0.56

0.05

Medium

17

2

CC

22

8

0.28

0.31

26

7

0.33

0.27

4

−1

0.07

−0.13

Low

15

2

CT

22

6

0.28

0.23

32

8

0.41

0.31

10

2

0.18

0.10

Low

20

2

CT

23

6

0.30

0.23

41

6

0.53

0.23

18

0

0.33

0.00

Medium

2

3

CC

46

11

0.59

0.42

39

13

0.50

0.50

−7

2

−0.15

0.13

None

7

3

CC

29

10

0.37

0.38

59

9

0.76

0.35

30

−1

0.61

−0.10

Medium

13

3

CT

31

9

0.40

0.35

35

7

0.45

0.27

4

−2

0.09

−0.22

Low

19

3

CT

27

7

0.35

0.27

52

8

0.67

0.31

25

1

0.49

0.05

Medium

10

4

CC

24

10

0.31

0.38

41

12

0.53

0.46

17

2

0.32

0.13

Medium

11

4

CC

22

9

0.28

0.35

40

7

0.51

0.27

18

−2

0.32

−0.22

Medium

3

4

CT

17

7

0.22

0.27

40

7

0.51

0.27

23

0

0.38

0.00

Medium

18

4

CT

48

12

0.62

0.46

57

11

0.73

0.42

9

1

0.30

−0.08

Medium

16

5

CC

22

5

0.28

0.19

23

7

0.30

0.27

1

2

0.02

0.10

Low

8

5

CT

25

7

0.32

0.27

29

6

0.37

0.23

4

−1

0.08

−0.14

Low

4

6

CC

46

11

0.59

0.42

59

10

0.76

0.38

13

−1

0.41

−0.09

Medium

14

6

CT

28

8

0.36

0.31

48

10

0.62

0.38

20

2

0.40

0.11

Medium

  

CC

269

83

0.35

0.32

432

86

0.56

0.33

163

3

0.33

0.02

Medium

  

CT

237

66

0.31

0.25

434

69

0.56

0.27

197

3

0.37

0.02

Medium

  

Total

506

149

0.33

0.29

866

155

0.56

0.30

360

6

0.35

0.02

Medium

CC = conceptual question, CT = contextual question

aObjective numbers according to Table 2

bGain-index: 〈g〉 = (%post − %pre)/(100 − %pre); in case of regression of the score: (%post − %pre)/(%pre)

cHigh gain: 〈g〉 ≥ 0.70, Medium gain: 0.70 > 〈g〉 ≥ 0.30, Low gain: 〈g〉 < 0.30 (Hake 1998)

A principal component analysis suggests, according to Kaiser’s criteria (eigenvalue > 1), 7 factors are present. However, screen plot analysis indicates, based on linear coinciding, the data should be analysed for 6 factors. Studying the (rotated) component matrix 17 test items (questions) across the components match the distribution of the questions across the objectives which gives a 85 % match.

The control group, used to determine a possible learning effect from completing the test, showed now average gain (%pre = 29; %post = 30). For the experimental group, a paired samples t test indicates the overall gain is significant, t(76) = −18.18; p < 0.001. This is confirmed by the Wilcoxon signed-rank test that gives the same p value. Unless, the experimental group made significant progress, substantially more gain could be possible because the overall gain is just enough to be called “medium” (Hake 1998). Compared to the gains found in different physics (-related) course studies, also including LBD, this gain is comparable (Churukian 2002; Coletta and Phillips 2005; Hake 1998; Kolodner 2002b).

The exam results regarding objective 1, 4, 5 and 6 are consistent, where objective 5 barely shows any gain, and for objectives 2–3 an anomaly is shown. Analysing the questions with no or low gain two things are noteworthy. First, question 2 shows a slight decline because students used the concept for parallel current behaviour, that was mainly important, in a series circuit. Second, the other questions also appealed to concepts that were barely exposed during the challenge (potential difference and resistance). For example, objective 5 (resistance and current flow) was addressed by the increasing amount of components students had to add to their design. However, a correlation with changing current was not investigated. Objective 1 and 4 (highest gains) were appealed strongly during the challenge. Thus, unravelling the requested design is important to predict learning outcomes, to set objectives and to notice possible shortcomings. Finally, differences between CT- and CC-questions were not found.

Table 7 shows how students’ final designs were scored by two experts. For these results the linear weighted Kappa κw is 0.70 (lower limit = 0.60; upper limit = 0.79), so inter rater agreement can be specified as good or substantial. The average relative amount of successes (successful) based on all specifications and both experts is 73 %. For the specifications based on proper science (A–D) this percentage is even 84 %, what implies that a medium gain according to learned science was sufficient for proper design realisation.
Table 7

Assessment of 25 design outcomes by two experts

Design specification according to Table 1

Successful

Partially successful

Unsuccessful

Expert 1

Expert 2

Expert 1

Expert 2

Expert 1

Expert 2

A

22

23

2

2

1

0

B

20

20

4

4

1

1

C

21

20

3

5

1

0

D

21

21

4

3

0

1

E

24

24

1

1

0

0

F

11

10

12

15

2

0

G

9

11

13

9

3

5

Despite the fact students performed reasonably well and students’ talking, as illustrated by Fig. 4, showed increasingly more scientific terms, interviews made clear, shown in Table 8, students lacked proper scientific reasoning. This is supported by the observation students continuously tend to apply trial and error to complete tasks. Summarizing, according to the knowledge dimensions of Bloom’s taxonomy, scientific insights are used as isolated facts and explicit interrelationships that enable them to function together remain underexposed (Krathwohl 2002).
Fig. 4

Relative usage of scientific and design-related terms. Note: Sound-recordings of 5 design groups were analysed by counting the usage of 13 predefined design- and objective-related terms. This was done for student discussion (40 min per group) during the early exploration stage and for one of the final stages (stage 6). For stage 3 a total amount of 417 terms were counted compared to 741 terms for stage 6. For both stages the relative distribution of used terms is shown by a chart. Beside the fact the total amount of used terms increased, the initial relatively large difference in usage decreased and certain terms became more favourite (e.g., current, circuit and parallel). The terms resistance, voltage and series stayed less favourite and were never addressed explicitly during the task

Table 8

Two examples of scientific reasoning by different students

Example 1 (interview 3)

Example 2 (interview 8)

Q. Why did you use a parallel circuit to build the dance pad?

Q. How your self-designed floor pad works?

A. Because the laptop told me to do

A. By stepping on it

Q. Why didn’t you choose a series circuit?

Q. Why does it result in, for example, flashing a light?

A. Then the dance pad will not work

A. Because we used aluminium foil

Q. Why?

Q. Why is this foil so special for your design?

A. Everything goes on and off at the same time

A. Because it lets current through

Q. Why is it that parallel circuits don’t do this?

Q. Does a normal switch functions like your floor pad?

A. Because then the parts don’t have the same wire

A. No, a normal switch contains no aluminium foil

Q. Is there a difference in the amount of current that flows through series or parallel circuits?

Q. How do you call things that can easily let current through?

A. No, it’s the same battery and current always comes back to the battery

A. Conducting

Stimulus Example 1: electric circuit

Stimulus Example 2: designed switch

Open image in new window

Open image in new window

Students’ focus

Because the students’ focus is one of the main topics of this study, students were questioned about learning outcomes. Table 9 shows experienced learning outcomes were mainly task- and product-related. Only 9 % of all replies were related to a better mastery of electricity concepts. According to questionnaires and interviews concepts were seen as a tool for designing a dance pad and the latter was, maybe logically, qualified as the ultimate goal of the challenge. This also explains that the virtual simulation was a successful tool for circuit creation, but circuit operation was not sufficiently understood. Overall, as suggested earlier, our novice design students indeed focused on process-related issues needed for success.
Table 9

Learning outcomes

Learning outcomes: what have you learned from the challenge?a

Fractionb

Knowing how to design (a dance pad) and the practice of designing (a dance pad)

0.33

The practice of creating electric circuits (for a dance pad)

0.18

Creating posters for class discussion

0.13

Proper use of construction materials and tools

0.10

(Better) mastery of electric current concepts

0.09

Knowing how to collaborate with students

0.07

Learning outcomes other than mentioned above (e.g. seeking for information, usage of ICT for educational purposes, presentation techniques, engineering concepts)

0.10

aDescriptions are revised to make categorisation possible

bRelative distribution of all replies gathered through questionnaires and interviews

Learning-related interactions

To investigate science-related learning incitements, students were asked to rate activities based on Table 4. Rating took place, as shown by the results in Table 10, using a five point Likert scale (very poor, poor, fair, good, very good).
Table 10

Learning incitements

Activities

To what degree did the activities listed below help you learn about electricity?

− − [1]

− [2]

o [3]

+ [4]

++ [5]

N

Average

Modus

% (>o)

Student (to student) interaction

25

70

83

39

14

231

2.8

3

23

Suggestions/advice from peers

6

22

29

16

4

77

2.9

3

26

Reviewing own thinking/doing

17

29

16

12

3

77

2.4

2

19

Searching for information

2

19

38

11

7

77

3.0

3

23

Student to teacher interaction

11

23

56

86

55

231

3.7

4

61

Suggestions/advices from teacher

3

9

19

26

20

77

3.7

4

60

Teacher-guided class sessions

5

5

20

28

19

77

3.7

4

61

Teacher-guided science talking

3

9

17

32

16

77

3.6

4

62

Student to content interaction

10

22

62

66

71

231

3.7

5

59

Circuit simulation (software)

0

3

21

23

30

77

4.0

5

69

Real circuit experimentation

8

14

19

18

18

77

3.3

3

47

Creating products/design(parts)

2

5

22

25

23

77

3.8

4

62

Student (to student) interaction

Table 10 shows student (to student) interaction was, according to students, least helpful to learn about electricity. Especially, self-reflection is not appreciated as a useful learning activity. Collaboration with peers scored a better rating (fair), where spontaneous collaboration was mainly triggered by the presence of the construction materials/tools and the making of sketches and drawings. This was established by counting different triggers for student–student discussion based on observation forms (Table 11).
Table 11

Observed triggers for student–student discussion

Trigger for student–student discussiona

Fractionb

Design-related activities and the presence of materials and tools

0.26

Prescribed by the learning task

0.21

The making of sketches and drawings

0.19

Teacher stimulated discussions

0.16

Scientific experimentation (real experiments and simulation software)

0.10

Other triggers (e.g., information seeking, spontaneous discussions, non-task-related triggers)

0.08

aDescriptions are revised to make categorisation possible

bThe fraction of triggers for student–student discussion based on relative distribution of observed events (observation forms)

Regarding information seeking, interviews made clear that gathered information is not shared spontaneously among peers. The major reason for this was, also demonstrated by interviews, the inability of students to properly estimate the value of the information. Furthermore, enthusiastic, highly involved students tended to dominate collaboration or, in case of no effect, to act alone in the future. This in order to finish a task as quickly as possible and experience a sense of accomplishment.

Student to teacher and content interaction

These interactions are rated equally, where circuit simulation gets the highest score regarding learning about electricity. However, interviews made clear, as mentioned before, students need considerable assistance to explicate scientific insights and design decisions adequately. For this, the teacher seems to be important. 12 of 16 interviewed students mentioned the teacher as the most reliable and important source for this kind of reasoning and fellow students were seen as incompetent doing this. Nevertheless, all teachers described the (guidance of the) LBD-task as intensive, time-consuming, complex and a real challenge for students and teachers. Especially, the process of sensitive assistance, mentioned before, seemed to be difficult. Reasons for this were mainly time constraints and the tendency to be too helpful. Students’ reactions were more or less similar and included the complexity of the design diary and complete challenge, mainly due to the extent and openness, experienced time constraints and, sometimes, the low intensity of relentless senses of accomplishments. Students often mentioned to find it difficult to stay focused and to make up their mind, but nearly 72 % also mentioned they became more motivated than usual, what also was noted by teachers. Finally, nearly one fifth of the students indicated it would be desirable to enrich the challenge by adding non-dance pad-related tasks or content. Table 12 provides an overview of the most important criticism expressed by students and teachers.
Table 12

Criticism expressed by students (questionnaires) and teachers (interviews)

Students’ criticisma

Fractionb

1. Having lack of time or experiencing time constraints

0.30

2. The complexity of the task due to diversity, complexity and openness

0.20

3. One-sided focus on the dance pad (over a long period)

0.18

4. Difficult to stay focused and concentrated (tumultuous learning environment and task duration)

0.11

5. The dependency on ICT quality (wireless network connection, hardware and software)

0.09

6. The teacher providing advice and guidance instead of answers and confirmation

0.06

Teachers’ criticism: examples of teachers’ pronouncementsc

Corresponding students’ criticism

“We also ran into time constraints and this led to some amount of stress to get everything done”

1

“Some students had problems to keep on track. […] The learning task is quite complex and appeals to many skills. […] I had to appeal to the utmost of my abilities”

2, 4

“Students often lacked concrete input from me […]. They asked for answers and confirmation […]. It was obvious they often hackled the uncertainty about their progress”

2, 6

“Often I found it difficult to give proper feedback or information. […] I want to help students but I don’t want to impede their learning process by giving too less, too much or wrong information”

2, 6

“It was very busy and noisy in the classroom. A few children asked me to create some rest”

4

“Students described administrative operations as time-consuming, disturbing and confusing. This was strengthened by the fact that internet access was often a problem”

1, 2, 5

“Two groups could not finish their design […] and they were not amused […]. They had just too little time and lost a lot of time due to filling in the design diary”

1

“The network access was frustrating. Also some laptops refused to access the simulation software”

5

“[…] so more teaching or learning activities are necessary to cover the learning content”

3

“[…] and then he asked me whether the dance pad was the only topic or something else was coming on. […] Students often have problems to stay focused when a task is complex or time-consuming”

1, 2, 3, 4

aDescriptions are revised to make categorisation possible

bRelative distribution of students’ criticism based on the total amount of criticism mentioned in questionnaires

cPronouncements teachers made during interviews (translated from Dutch to English)

Discussion and implications

By studying the practice of LBD and less emphasis on pre- and post-testing, this study reveals why concept learning has its limitations, despite the fact LBD theoretically provides a rich learning environment. It clarifies why the found average medium gain (0.35 gain-index) stayed relatively low and offers room for improvement. For example, a previous survey of pre-/post-test data for 62 introductory physics courses, based on interactive engagement (IE) methods, showed gain-indices up to 0.60 (Hake 1998). Those IE-methods are, similar to LBD, designed to promote conceptual understanding through heads- and hands-on activities contributed by peer feedback and discussion and intensive teacher guidance (Hake 1998). A main difference between those IE-methods and LBD is the amount and extensiveness of objects of integration, where LBD seems to be more diverse: teachers and students defined the LBD challenge as complex, mainly due to the extent and openness. Where time-constraints, the malfunctioning of the virtual simulation and network connection, and a disturbing emphasis on the (extensive) design diary were additional negative elements. Thus, the complexity and extensiveness forced students to focus on doing the right things and delivering requested products. Therefore, in accordance with the hypothesis stated before, students were indeed strongly product- and process-focused (What to do and what to deliver?) and qualified scientific content (What to learn?) as tools they needed for success.

The science students learned and used for producing their design mainly became available from activities that strongly determined a successful completion of the challenge. First, the virtual simulation that provided insight in electrical wiring and, second, teacher-driven activities (e.g. student–teacher interaction and teacher-driven class discussions) when concepts were discussed explicitly. Therefore, the more concepts directly determined a successful design outcome the better the concepts were understood. An important fact that also was indicated by Jones (1997) for technological concepts. The students’ strong focus on acting and delivering successful products, according to students the main goal, suppresses the fact that those processes (can/must) increase their concept-related knowledge. This resulted in the fact that concepts, certainly when they were poorly design-related, were badly or only partially understood. This lack of focus on scientific objectives and associated concepts caused the learning of isolated facts that stayed, more or less, implicit. Students used more scientific terms and symbols and designed proper electric circuits, but did not achieve a deeper conceptual understanding. Thus, students learned incomplete concepts, just enough for design-implementation, and too little explicit interrelationships that is essential to master the knowledge domain (Brandsford et al. 2003; Wiggins and McTighe 2006).

This problem of incidental, implicit, informal or unintentional learning was also found in other non-LBD studies (Baskett 1993; Kerka 2000; Marsick and Watkins 2001; Rogers 1997). For example, our findings correspond to the important design-related issue stated in the run-up to the presentation of this study: design is seen as an instructional strategy where product realisation has the emphasis and more (underlying) conceptual understanding is necessary to improve design performance and conceptual understanding. Therefore, the results of this study can be more broadly understood. The practice of design offers a rich learning environment but an overall reinforcement of conceptual awareness is required.

Possibilities for improvement and further research

According to the previous, there are mainly two (interrelated) problems for which solutions will need to be found: (1) reducing the complexity of the challenge without diluting the potentially rich learning environment; (2) a stronger focus on domain specific objectives and related (scientific) concepts, where important interrelationships become explicit.

In general, a detailed analysis of concepts (technology and science), crucial for succeeding, is necessary: when they (have to) emerge and how they are related. This also makes clear which concepts are poorly task-related and need to be addressed otherwise (e.g. demonstrations, lectures, further readings, experiments, etc.). Other previous studies give some more important insights. To discuss and explicate concepts students used for their products and during their collaboration the technique of guided discussion may be helpful (Brandsford et al. 2003). This teacher-led discussion technique encourages students to share (scientific) insights and develop a deeper understanding. To emphasize en explicate the important role of concepts for design purposes (elements of) informed design (Burghardt and Hacker 2004) might be interesting. This strategy aims for thoughtful design decisions based on scientific and mathematical concepts without reverting to trial and error, the tendency the students in our study had. Furthermore, applying explicit instruction (Archer and Hughes 2011) and the use of scaffolding strategies (Bamberger and Cahill 2013) are interesting. Both strategies help to facilitate students’ understanding and overseeing of the learning process. Students are guided through the learning process with clear instructions, proceeding in small steps, checking for understanding and achieving active and successful participation by all students. This focus on successful participation could also respond to the problem we encountered: students (within design groups) were sometimes not equally involved.

To conclude, LBD activities are very teacher-dependent, whether due to content-related choices and otherwise, because of sensitive assistance (guidance). Maybe not surprising, because the teacher plays a significant role in the succeeding of learning activities (Bamberger and Cahill 2013; Van der Veen and Van der Wal 2012). Thus, it will be valuable to study (the interplay of) concept learning and teacher handling in detail to distract important clues for appropriate teacher behaviour.

References

  1. Abdul Gafoor, K., & Akhilesh, P. T. (2013). Strategies for facilitating conceptual change in school physics. Researches and Innovations in Education, 3(1), 34–42.Google Scholar
  2. Archer, A. L., & Hughes, C. A. (2011). Exploring the foundations of explicit instruction. In K. R. Harris & S. Graham (Eds.), Explicit instruction: Effective and efficient teaching (pp. 1–21). New York: The Guilford Press.Google Scholar
  3. Bamberger, Y. M., & Cahill, C. S. (2013). Teaching design in middle-school: Instructors’ concerns and scaffolding strategies. Journal of Science Education and Technology, 22(2), 171–185.CrossRefGoogle Scholar
  4. Baskett, H. K. M. (1993). Workplace factors which enhance self-directed learning. Paper presented at the seventh international symposium on self-directed learning, West Palm Beach, FL.Google Scholar
  5. Berlin, D. F., & White, A. L. (1994). The Berlin-White integrated science and mathematics model. School Science and Mathematics, 94(1), 2–4.CrossRefGoogle Scholar
  6. Bijou, S. W., Peterson, R. F., & Ault, M. H. (1968). A method to integrate descriptive and experimental filed studies at the level of data and empirical concepts. Journal of Applied Behavior Analysis, 1(2), 175–191.CrossRefGoogle Scholar
  7. Boeije, H. (2005). Analyseren in kwalitatief onderzoek. Amsterdam: Boom Lemma Uitgevers.Google Scholar
  8. Brandsford, J. D., Brown, A. L., Donovan, M. S., & Pellegrino, J. W. (2003). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.Google Scholar
  9. Bruinsma, M. (2003). Effectiveness of higher education: Factors that determine outcomes of university education. Doctoral dissertation, Rijksuniversiteit Groningen, Groningen.Google Scholar
  10. Burghardt, M., & Hacker, M. (2004). Informed design: A contemporary approach to design pedagogy as the core process in technology. Technology Teacher, 64(1), 6–8.Google Scholar
  11. Churukian, A. D. (2002). Interactive engagement in an introductary university physics course: Learning gains and perceptions. Doctor of Philosophy Dissertation, Kansas State University, Manhattan, KS.Google Scholar
  12. Clark, A., & Ernst, J. (2007). A model for the integration of science, technology, engineering and mathematics. The Technology Teacher, 66(4), 24–26.Google Scholar
  13. Cobern, W. W. (1994). Worldview theory and conceptual change in science education. Paper presented at the National Association for Research in Science Teaching, Anaheim.Google Scholar
  14. Coletta, V. P., & Phillips, J. A. (2005). Interpreting FCI scores: Normalized gain, preinstruction scores, and scientific reasoning ability. American Journal of Physics, 73(12), 1172–1182.CrossRefGoogle Scholar
  15. Cosgrove, M., & Osborne, R. (1985). Lesson frameworks for changing childrens ideas. In R. Osborne & P. Freybergs (Eds.), Learning in science: The implications of childrens science. London: Heinemann.Google Scholar
  16. Engelhardt, P. V., & Beichner, R. J. (2004). Students’ understanding of direct current resistive electrical circuits. American Journal of Physics, 72(1), 98–115.CrossRefGoogle Scholar
  17. Finkelstein, N. D., Adams, W. K., Keller, C. J., Kohl, P. B., Perkins, K. K., Podolefsky, N. S., & Reid, S. (2005). When learning about the real world is better done virtually: A study of substituting computer simulations for laboratory equipment. Physics Education Research, 1(1), 8.Google Scholar
  18. Fortus, D., Dershimer, R. C., Krajcik, J., Marx, R. W., & Mamlok-Naaman, R. (2004). Design-based science and student learning. Journal of Research in Science Teaching, 41(10), 1081–1110.CrossRefGoogle Scholar
  19. Hake, R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64–74.CrossRefGoogle Scholar
  20. Hake, R. (2004). Design-based research: A primer for physics-education researchers. American Journal of Physics, 1–35. http://www.physics.indiana.edu/~hake.
  21. Hennessy, S., & McCormick, R. (1994). The general problem-solving capability process in technology education. Myth or reality? In F. Banks (Ed.), Teaching technology (pp. 94–108). London: Routledge.Google Scholar
  22. Horton, W. K. (2006). E-learning by design. San Francisco, CA: Wiley.Google Scholar
  23. International Technology Education Association. (2007). Standards for technological literacy: Content for the study of technology (3rd ed.). Virginia: Reston.Google Scholar
  24. Johnson, S. (1997). Learning technological concepts and developing intellectual skills. International Journal of Technology and Design Education, 7, 161–180.CrossRefGoogle Scholar
  25. Jones, A. (1997). Recent research in learning technological concepts and processes. International Journal of Technology and Design Education, 7, 83–96.CrossRefGoogle Scholar
  26. Kerka, S. (2000). Incidental learning. Trends and Issues Alert, 4. Retrieved from http://eric.ed.gov/?id=ED446234.
  27. Kolodner, J. L. (2002a). Facilitating the learning of design practices: Lessons learned from an inquiry into science education. Journal of Industrial Teacher Education, 39(3), 9–40.Google Scholar
  28. Kolodner, J. L. (2002b). Learning by design: Iterations of design challenges for better learning of science skills. Bulletin of the Japanese Cognitive Science Society, 9(3), 338–350.Google Scholar
  29. Kolodner, J. L., Camp, P. J., Crismond, D., Fasse, B., Gray, J., Holbrook, J., & Ryan, M. (2003a). Problem-based learning meets case-based reasoning in the middle-school science classroom: Putting learning by design into practice. The Journal of the Learning Sciences, 12(4), 495–547.CrossRefGoogle Scholar
  30. Kolodner, J. L., Gray, J. T., & Fasse, B. B. (2003). Promoting transfer through case-based reasoning: Rituals and practices in learning by design classrooms. Cognitive Science Quarterly, 3(2), 1–28.Google Scholar
  31. Kolodner, J. L., Hmelo, C., & Narayanan, N. (1996). Problem-based learning meets case-based reasoning. The Journal of the Learning Sciences, 12(4), 495–547.CrossRefGoogle Scholar
  32. Krathwohl, D. R. (2002). A revision of bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212–218.CrossRefGoogle Scholar
  33. Licht, P., & Snoek, M. (1986). Electriciteit in de onderbouw: Een inventarisatie van begrips- en redeneerproblemen bij leerlingen. NVON Maandblad, 11(11), 32–36.Google Scholar
  34. Lustig, F., West, E., Martinez, B., Staszel, M., Borgato, M. T., Iosub, I., & Weber-Hüttenhoff, U. (2009). Experiences and results from the European project ‘Integrated Subject Science Understanding in Europe’. Paper presented at the ESERA conference, Istanbul.Google Scholar
  35. Marsick, V. J., & Watkins, K. E. (2001). Informal and incidental learning. New Directions for Adult and Continuing Education, 2001(89), 25–34.CrossRefGoogle Scholar
  36. Mason, M. (2010). Sample size and saturation in PhD studies using qualitative interviews. FQS: Forum Qualitative Social Research, 11(3), Art. 8. http://nbnresolving.de/urn:nbn:de:0114-fqs100387.
  37. McCormick, R. (1997). Conceptual and procedural knowledge. International Journal of Technology and Design Education, 7, 141–159.CrossRefGoogle Scholar
  38. Murphy, P., & Hennessy, S. (2001). Realising the potential—And lost opportunities—For peer collaboration in a D&T setting. International Journal of Technology and Design Education, 11, 203–237.CrossRefGoogle Scholar
  39. Niedderer, H., & Goldberg, F. (1993). Qualitative interpretation of a learning process in electric circuits. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, Atlanta.Google Scholar
  40. Norman, G. R., & Schmidt, H. G. (1992). The psychological basis of problem based learning: A review of the evidence. Academic Medicine, 67(9), 557–565.Google Scholar
  41. Nussbaum, J., & Novick, S. (1982). Alternative frameworks, conceptual conflict and accommodation: Toward a principled teaching strategy. Instructional Science, 11(3), 183–200.CrossRefGoogle Scholar
  42. Oorschot, F., Ottevanger, W., Spek, W., Boerwinkel, D. J., Eijkelhof, H., de Vries, M., et al. (2014) Enschede: SLO (Nationaal Expertisecentrum Leerplanontwikkeling).Google Scholar
  43. Osborne, J., & Dillon, J. (2008). Science education in Europe: Critical reflections (p. 32). London: The Nuffield Foundation.Google Scholar
  44. Parkinson, E. (2001). Teacher knowledge and understanding of design and technology for children in the 3–11 age group: A study focussing on aspects of structures. Journal of Technology Education, 13(1), 44–55.Google Scholar
  45. Popovic, V. (2004). Expertise development in product design-strategic and domain-specific knowledge connection. Design Studies, 25(5), 527–545.CrossRefGoogle Scholar
  46. Rennie, L., Venville, G., & Wallace, J. (2012). Integrating science, technology, engineering and mathematics. New York: Routledge.Google Scholar
  47. Rogers, A. (1997). Learning: Can we change the discourse? Adults Learning, 8(5), 116–117.Google Scholar
  48. Roth, W.-M. (1995). Inventors, copycats, and everyone else: The emergence of shared resources and practices as defining aspects of classroom communities. Science Education, 79, 475–502.CrossRefGoogle Scholar
  49. Roth, W.-M. (2001). Learning science through technological design. Journal of Research in Science Teaching, 38(7), 768–790.CrossRefGoogle Scholar
  50. Sidawi, M. (2009). Teaching science through designing technology. International Journal of Technology and Desing Education, 19(3), 269–287.CrossRefGoogle Scholar
  51. Sjöberg, S., & Schreiner, C. (2010). The ROSE project: An overview and key findings. http://roseproject.no/network/countries/norway/eng/nor-Sjoberg-Schreiner-overview-2010.pdf. Accessed 10 Nov 2015.
  52. Trochim, W. (2006). Web center for social research methods 2013, from http://www.socialresearchmethods.net/.
  53. Van der Veen, T., & Van der Wal, J. (2012). Van leertheorie naar onderwijspraktijk. Groningen: Noordhoff Uitgevers B.V.Google Scholar
  54. Verhagen, P. (2011). Reflectie met de STARR-methode Kwaliteit met beleid. Bussum: Coutinho.Google Scholar
  55. Wendell, K. B. (2008). The theoretical and empirical basis for design-based science instruction for children. Unpublished Qualifying Paper. Tufts University.Google Scholar
  56. Wiggins, G. (2012). 7 keys to effective feedback. Educational Leadership, 70(1), 10–16.Google Scholar
  57. Wiggins, G., & McTighe, J. (2006). Understanding by design. Upper Saddle River, NJ: Pearson Education Inc.Google Scholar

Copyright information

© The Author(s) 2016

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Dave H. J. van Breukelen
    • 1
  • Marc J. de Vries
    • 2
  • Frank A. Schure
    • 1
  1. 1.Fontys University of Applied Sciences for Teacher Education SittardSittardThe Netherlands
  2. 2.Delft University of Technology, Science Education and CommunicationDelftThe Netherlands

Personalised recommendations