Theoretical Framework

Education for Sustainable Development

Within the last two decades, the global community has acknowledged the need to ensure sustainable development of the world to meet the “needs of the present without compromising the ability of future generations to meet their own needs” (World Commission on Environment and Development (WCED) 1987, p. 43). Environmental threats and increasing social injustice require thoughtful decisions and actions on both a political and an individual level. As a consequence, non-sustainable courses of action must be identified when deciding for a course of action. Options that do not meet standards that are based on societal norms should be excluded. An international consensus on the characteristics of sustainable development was reached at the UN Conference on Environment and Development (UNCED) and in its final action plan, Agenda 21 (UNCED 1992). However, these goals can only be reached through interdisciplinary approaches that integrate ecological, social and economic aspects (Bourn 2005; Eilam and Trop 2011; Herremans and Reid 2002; Marcinkowski 2010; Sauvé 1996, 2005; Scott and Gough 2003; UNCED 1992). One vital aspect of promoting sustainable development is education (UNCED 1992). Education should aim to foster societal development according to norms negotiated by the global community, i.e., the United Nations (de Haan et al. 2008). However, education in general, and science and environmental education in particular, should not promote a specific point of view or certain courses of action; instead, it should strengthen students’ autonomy and empower them to make thoughtful decisions (Aikenhead 1985; de Haan et al. 2008; Hodson 2003; Hungerford 2010; Jickling 1992; McConnell 1982; Pedretti 2003; Ratcliffe and Grace 2003; Solomon and Aikenhead 1994).

Within the broader framework of science, technology, society and environment (STSE) education, which embraces education for sustainable development (Pedretti 2003), the quality of the students’ decisions can be enhanced through dealing with socioscientific issues (ibid.; Eggert and Bögeholz 2006; Eggert et al. 2010; Gresch et al. 2011; Ratcliffe and Grace 2003). Sadler (2004) defines these types of issues as complex, open-ended and contentious problems that lack simple and straightforward solutions. Moreover, scientific evidence, ethical implications and inherent values must be integrated to deal with socioscientific issues adequately (Aikenhead 1985; Bögeholz and Barkmann 2003; Eggert and Bögeholz 2006; Kolstø 2001; Ratcliffe and Grace 2003; Sadler 2004; Sauvé 2005; Zeidler et al. 2005). Consequently, decision-making regarding socioscientific issues has been incorporated into national standards worldwide (American Association for the Advancement of Science (AAAS) 1993; Kultusministerkonferenz (KMK) 2005; National Research Council (NRC) 1996; Qualifications and Curriculum Authority (QCA) 2004). Moreover, education for sustainable development as a field of socioscientific decision-making has been integrated into the national standards of many countries, including England and Germany (KMK 2005; QCA 2004).

Decision-Making in Science Education

In the present study, decision-making competence refers to the ability to systematically evaluate possible courses of action in factually and ethically complex situations related to sustainable development and systematically make a final decision. Moreover, students are expected to reflect on their own decision-making processes and those of others (Bögeholz 2007; Bögeholz and Barkmann 2003; Eggert and Bögeholz 2006, 2010). This concept of decision-making competence is based on the general definition of competence as a domain-specific cognitive ability to solve specific problems (Hartig and Klieme 2006; cf. Weinert 2001a, b), which was proposed by the Organisation for Economic Co-operation and Development (OECD).

Several intervention studies in the field of science education have focused on improving the quality of students’ decision-making processes (Eggert et al. 2010; Grace 2009; Gresch et al. 2011; Jiménez-Aleixandre and Pereiro-Muñoz 2002; Ratcliffe 1997; Roberts et al. 1997 (Science Education for Public Understanding Program (SEPUP)); Seethaler and Linn 2004; Siegel 2006). In the science and sustainability curriculum that was part of SEPUP, Siegel (2006) successfully trained students to connect specific evidence to their arguments to make trade-offs. Ratcliffe (1997) provided her students with a guideline to structure the decision-making process. Although the students considered this guideline helpful, they did not weigh the advantages and disadvantages of their options in a systematic way. Using this guideline in small-group discussions about biological conservation issues, Grace (2009) found that the students’ level of personal reasoning could be improved within a short time. In the field of education for sustainable development, Eggert et al. (2010) trained students in cooperative learning settings to trade off several courses of action. A general increase in decision-making competence was found, although additional metacognitive training in a second training group did not further improve the students’ decision-making processes.

Although several interventions have been conducted to enhance students’ decision-making processes, no study has examined whether reflectively applying several decision-making strategies fosters decision-making competence. Because such strategies seem promising for systematically structuring the decision-making process, they were chosen for the present study.

Decision-Making Strategies

Different decision-making tasks related to sustainable development require different decision-making strategies to evaluate possible courses of action because some decisions involve non-sustainable and, thus, unacceptable options with regard to ecological, social or economic standards while others may demand a full trade-off of all evidence. These standards may be set at an individual or societal level, i.e., based on personal values or societal norms, respectively.

Behavioral decision research aims to describe the strategies people use to make a decision. In certain decision-making situations, decision-makers exclude options if one or several characteristics do not reach a required standard. Some criteria may be considered knockout criteria, for which unacceptable characteristics cannot be outweighed by other attributes. Jungermann et al. (2005) and Payne et al. (1998) describe this choice rule as a non-compensatory strategy. One example is the elimination-by-aspects rule (Tversky 1972). If the characteristics of the most important criteria do not reach the threshold level to fulfill a certain socially or individually required standard, the options are eliminated. For other decisions, the decision-maker may weigh all available information before reaching a conclusion. Hence, in this compensatory strategy, all advantages and disadvantages are evaluated in a full trade-off (Jungermann et al. 2005; Payne et al. 1998).

Beach (1990) and Beach and Mitchell (1978) state that decision-making frequently involves both compensatory and non-compensatory approaches. During a screening phase, unacceptable options are eliminated in the first step, and the remaining options are compared in a full trade-off (mixed strategy).

Metadecision and Self-Regulated Learning

The application of a decision-making strategy is a cognitive process whereas the selection of a decision-making strategy requires metacognitive skills to determine which strategy fits best. Beach (1990) refers to this concept as metadecision, which involves considering the features of a decision-making problem, the environment of the decision and the characteristics of the decision-maker. From a normative, educational perspective, students should be enabled to detect non-sustainable options based on societal norms regarding the framework of sustainable development and apply a non-compensatory strategy to such tasks. On the other hand, students should also be capable of identifying decision-making situations with equally legitimate options (i.e., options without knockout criteria based on societal norms) and employing a compensatory strategy (Eggert and Bögeholz 2006; Gresch et al. 2011). Consequently, science and environmental education should foster the ability to distinguish between different types of decision-making situations to apply an appropriate decision-making strategy.

Zimmerman’s model of self-regulated learning provides a suitable framework to foster these metadecision skills because it integrates cognitive, metacognitive and motivational aspects to describe how self-regulated learners deal with new tasks (Zimmerman 2000). Before beginning a task, learners should analyze the task in the forethought phase to select an appropriate strategy; they should observe and control the procedure during the performance phase; and finally, they should self-reflect on their own performance and draw conclusions for future tasks in the reflection phase.

To resolve socioscientific issues related to sustainable development, these concepts of metadecision and self-regulated learning can be combined to construct a training program that focuses on the reflective application of decision-making strategies (Gresch et al. 2011). In the forethought phase, the learner analyzes the decision-making task in terms of the sustainability of the options. Can the benefits and drawbacks of different options be compensated for, or should a non-compensatory strategy be applied to exclude non-sustainable options if knockout criteria exist based on societal norms? Do the task’s characteristics require a combination of both strategies? In this metadecision activity, a decision-making strategy should be selected. During the performance phase, the learner observes the correct application of the strategy and, finally, reflects on the outcome and whether the decision-making strategy was appropriate for the presented task.

Intervention Study on Decision-Making Strategies

Outline of the Research Project

To investigate the effects of a reflected use of decision-making strategies on decision-making competence, a computer-based intervention study was conducted. Two distinct perspectives were selected to describe the outcomes and mechanisms of the intervention: a summative evaluation to demonstrate the effects using a pre-post-follow-up control-group design and an in-depth analysis of qualitative data collected during the intervention. This paper focuses on the analyses of process-related data, and therefore, the results of the summative evaluation reported by Gresch et al. (2011) will only be reviewed briefly before deriving our research questions.

Description of the Intervention

In the intervention, which consists of two 45-minute sessions, students resolved different socioscientific issues pertaining to sustainable development (Gresch et al. 2011). All students were assigned to one of two training groups or the control group when they began the program.

In the first session, all students were introduced to one crucial principle of sustainable development: the interrelatedness of ecological, social and economic facets. Subsequently, the students were confronted with three decision-making tasks, which the students in the training groups solved by applying a non-compensatory, a compensatory and a mixed strategy. Each context was selected and modified to match one of the three strategies (see Table 1). The central features of the strategy and the way to apply it were introduced. The control group received additional ecological information instead of strategic training.

Table 1 Structure of the training program: contexts chosen to demonstrate the decision-making strategies
Full size table

In the second session, which took place within a week of the first session, the students in the training groups were asked to select the strategy they considered to fit best by taking the framework of sustainable development into account. The first training group selected the strategy directly whereas the second training group had to conduct a prior task analysis and explain why they considered this strategy to fit best. They were encouraged to reflect on whether knockout criteria based on societal norms were inherent in the given task. The control group again was given additional information instead of strategic training, as in the first session.

All contexts were chosen and designed so that one strategy fitted the best in terms of the societal norms of the framework of sustainable development (see Table 1). The features of all courses of action were discussed with the research group and optimized in several consecutive steps to improve the strategy’s fit with the decision-making task. The presentation of the contexts included short descriptions of the decision-making situation and each option, followed a table containing an overview of all the options’ characteristics. In each task, three or four options were presented with regard to four or five criteria.

  1. 1.

    Example for the compensatory decision-making strategy- Apples for the school cafeteria: The first context of the second session was a consumer decision. The students were asked to select one variety of apples that should be sold in the school’s cafeteria (cf. Barfod-Werner et al. 2008). Characteristics such as the price (1.80 to 2.49 Euro/kg), the taste (sweet, slightly sour, juicy), the length of the transportation route (short to intermediate; all within Europe) and whether the apples were organically grown were presented.

    Because all these features were considered acceptable according to the normative framework of sustainable development, the compensatory strategy was expected to provide the best fit by allowing a complete trade-off between all advantages and disadvantages.

  2. 2.

    Example for the mixed decision-making strategy- Shipyard production site: In the second decision-making task, possible production sites of a shipyard had to be examined from a political point of view (cf. Mühlenhoff 2009). One option would have a strong negative impact on the quality of the adjacent river; hence, these effects on the limnological ecosystem are considered unacceptable in terms of ecological standards. To pursue this course of action, the river would have to be dammed to launch the container ships safely. Consequently, the water temperature would increase in the summer and the oxygen concentration would fall below a level that could support animal life, e.g., endangered fish species. Furthermore, the implications for the workforce in a region with high unemployment rates and the productivity of the enterprise had to be taken into account.

    Thus, one option had unacceptable characteristics based on societal norms (ecological standards) whereas the features of the remaining options should be compared in a full trade-off. Hence, the mixed strategy was expected to fit best.

  3. 3.

    Example for the non-compensatory decision-making strategy- Purchase of a bed: In this task, four beds for the student’s new apartment after he or she left school were presented as options. Upon closer examination, most of the beds either consisted of wood from primeval forests, in which endangered animal species such as the Siberian tiger live, or were built in regions with extremely low wages, where social standards are nearly non-existent. Moreover, disproportionately long transportation routes were necessary to deliver one bed from Indonesia across the globe to Europe.

    Hence, several knockout criteria based on societal norms were apparent (wood from primeval forests, working conditions, long transportation), which would suggest the use of the elimination-by-aspects rule as one example of a non-compensatory strategy.

Review of the Summative Evaluation of the Presented Intervention Study

To determine the effects of the intervention (Gresch et al. 2011), decision-making competence was assessed at three measurement times using an open-ended questionnaire developed by Eggert and Bögeholz (2010). The analyses revealed that the quality of the students’ decisions on the follow-up test administered three months after the intervention were significantly more elaborate in the training groups compared to the control group. However, the students who received a strategic training tended to overuse non-compensatory strategies in the post-test. Further analyses showed that their ability to reflect on the decision-making processes of others improved between the pre- and post-tests. The long-term effects on reflection were less evident.

Research Questions

Although the summative evaluation of the intervention has shown promising results with regard to gains in decision-making competence, it is still unclear in which way students who are familiar with different decision-making strategies use metadecision skills to transfer these strategies to new tasks by selecting a strategy suitable for the given task. Moreover, the mechanisms of the intervention should be investigated by looking at the processes that take place in the course of the training. The qualitative data that was collected while the students of the second training group worked on the decision-making tasks are considered valuable for this purpose.

First, it is essential to investigate the explanations for selecting a decision-making strategy as one central aspect of metadecision and to determine the quality of these explanations. Hence, the first research question is as follows:

  1. 1.1

    How do students explain their use of different decision-making strategies?

Categories of explanations are generated, and levels of quality are described. These quality levels of the explanations should be validated. From a theoretical point of view, the level of explanations for selecting a decision-making strategy should be related to decision-making competence. Hence, we expect that students with higher levels of explanations also yield higher scores in the summative decision-making competence assessment conducted by Gresch et al. (2011; cf. Eggert and Bögeholz 2010). Consequently, the next research question is as follows:

  1. 1.2

    Are the quality levels of the explanations valid? In which ways are they related to decision-making competence?

From these levels of metadecision explanations, inferences should be made about the students’ performance in the transfer phase of the decision-making training and the quality of the learning tasks used to train the students to deal systematically with socioscientific issues.

The construction of the learning environment was based on one premise: The characteristics of the task should induce the selection of one specific decision-making strategy (Eggert and Bögeholz 2006; Gresch et al. 2011). Options that were unacceptable in terms of the societal norms stated in the sustainable development framework should be detected and excluded using a non-compensatory or mixed strategy whereas equally legitimate options should be compared in a full trade-off. Hence, the next research question is as follows:

  1. 2.1

    To what extent did the students select the decision-making strategies that best fit the decision-making tasks according to societal norms?

The next step was to unravel whether selecting the best-fitting strategy depends on the student’s metadecision performance. Regarding the fit of a decision-making task with one specific strategy, we supposed that those students who selected the expected strategy had conducted a thorough task analysis whereas those who chose a different strategy conducted superficial task analyses:

  1. 2.2

    Did those students who selected the strategy that fits best according to societal norms elaborate more on explanations than those who selected a different strategy?

The results of the summative assessment showed that the students who were familiar with the application of decision-making strategies tended to overuse the non-compensatory strategy during the post-test administration of the decision-making questionnaire (F (373,2) = 2.51, p < .10, η² = .013).Footnote 1 In another study in science education, Hong and Chang (2004) found that students most frequently applied the non-compensatory strategy. They suggested that the non-compensatory strategy was chosen because it is less cognitively demanding; i.e., it allows the user to consider less evidence compared with a complete trade-off. Therefore, the next research focus is as follows:

  1. 2.3

    Is the use of the non-compensatory decision-making strategy associated with a lower cognitive load?

Methods

Sample

A total of 386 students from 25 biology classes (Grades 11 to 13) in Northern Germany participated in the entire intervention and were randomly assigned to one of two training groups or a control group (training group 1: n = 126; training group 2: n = 137; control group: n = 123). For 120 students in the second training group, who conducted an initial task analysis before selecting a decision-making strategy, complete process-related data sets with explanations for their decision-making strategy selection are available. The mean age of this subsample was 17.3 years, and 58% were girls.

To investigate research questions 1.1, 1.2 and 2.2, process-related data from training group 2 is used because only this group reported explanations regarding the selection of a decision-making strategy. For research questions 2.1 and 2.3, data from both training groups is analyzed.

Recollection of Data and Data Analysis

The metadecision statements for the task analysis were entered online by the students in the second training group during the second session of the computer-based intervention. After each of the three decision-making situations was presented, the students were asked to respond to the following:

Note in bullet points the task characteristics that provide information about which decision-making strategy best fits this situation. Explain briefly why these task characteristics make one decision-making strategy particularly appropriate. Finally, click on a button to select the strategy that you consider to fit best to this task.

The data were imported using software for qualitative content analyses (MAXQDA) and coded using a deductive-inductive approach (Mayring 2008). In the first step, categories were derived from behavioral decision-making research (Eggert and Bögeholz 2006; Jungermann et al. 2005; Payne et al. 1998). Next, sub-categories and different quality levels were developed according to the data to further differentiate between different types of responses. For each sub-category, examples (anchors) and definitions were generated.

A second rater coded all text passages to further refine the categories and sub-categories, the selection of examples and the definitions. All data were then independently coded by the two raters based on this final scoring rubric. The interrater reliability (Cohen’s kappa (percentage of agreement)) was good (κ = .72 (94%)). In a final step, all differing scores were discussed to reach a consensus before the statistical analyses were performed.

To determine the validity of the quality levels of the explanations (see research question 1.2), the scores that students yielded in a decision-making questionnaire (Eggert and Bögeholz 2010) were used. This test was administered directly before the intervention (Gresch et al. 2011).

Moreover, data regarding the frequencies of the selected decision-making strategies were collected in the course of the intervention.

Assessment of Cognitive Load

Cognitive load is a concept that consists of two aspects: mental load, which is induced by the task structure and other instructional parameters, and mental effort, i.e., the capacity that a person allocates to the task (Paas 1992). The higher the mental load of a task, i.e., the more difficult it is, the more mental effort it requires. In other studies, mental effort has been assessed effectively in problem-solving tasks and computer-based trainings (Paas 1992; Paas and van Merriënboer 1994) to provide insights about the quality of instructions. Consequently, instructions should have a medium level of cognitive load to be efficient. In the present study, the cognitive load construct is used to determine which decision-making strategy produces more cognitive load. After each task of the learning program, the students were asked to self-report the level of mental effort expended during the preceding task on a symmetrical seven-level Likert scale (−3/+3).

Results

Research Question 1.1: How Do the Students Explain Their Use of Different Decision-Making Strategies?

The students had been asked to explain which decision-making strategy would be most appropriate to resolve the issue and, hence, to provide metadecision statements. In the qualitative content analysis, all students’ responses were assigned to two major categories: explanations for the use of parts of a compensatory and a non-compensatory strategy. Because the mixed strategy is a combination of both, explanations were scored separately for each component. Furthermore, we distinguished between three levels of explanations. At the lowest level (Score: 0), the students made no reference to the strategy they selected. At a basic level (Score: 1), the students explained their selection in terms of strategic considerations; however, this explanation was not linked to specific task characteristics. At the highest level (Score: 2), the students based their decision-making strategy selection on both strategic aspects and task characteristics. Both aspects were clearly linked.

The students at the lowest level (Score: 0) merely described the presented options, chose one course of action without reference to the decision-making strategy or stated which criteria they considered important. Moreover, those students who simply mentioned the name of a strategy without offering any explanation yielded Score 0.

“I decide for the 380 Euro bed from Finland because it seems best to me. Furthermore, it is made from birch wood and is hence from nature.” (Task 3, purchase of a bed; UserID 429)

“For such a topic, the compensatory strategy is best.” (Task 2, shipyard production site; UserID 465)

“Mixed strategy: I’ll take Granny Smith.” (Task 1, apples for the school cafeteria; UserID 468)

“It’s important to preserve jobs!” (Task 2, shipyard production site; UserID 531)

For the more elaborate responses (Scores 1 and 2), subcategories were developed and refined using an inductive approach, according to the investigated data (see Table 2):

Table 2 Scoring rubric with anchor examples: explanations for the selection of a decision-making strategy
Full size table

The explanations for compensatory and non-compensatory decision-making showed two clear patterns in each category. In the first subcategory, which was the most frequently occurring one (see Table 3), the students’ responses referred to whether a full trade-off was necessary and/or whether knockout criteria existed in the decision-making situation. At Level 1 (Score: 1), only a general statement was made about whether the task permitted a trade-off, whereas at level 2 (Score: 2) it was explicitly stated, between which criteria of the decision-making situation a trade-off was possible, or which criteria were considered knockout criteria. In the second subcategory, the students explained their decision-making strategy selection by stating that the relevance of the task criteria influenced their selection. Regarding non-compensatory decision-making, the students identified as knockout criteria those factors they considered important; hence, they referred to a hierarchy of values. On the other hand, those students who considered all criteria equally important tended to use compensatory strategies to weigh all available evidence.

Table 3 Relative frequencies of the categories and subcategories of explanations for the selection of a decision-making strategy
Full size table

On average across all three contexts, more than 10% (sum of responses scored 1 and 2) of the students explained their choice of a compensatory or non-compensatory decision-making strategy by referring to a hierarchy of values (see Table 3).

Other subcategories had lower frequencies. Some students explained their choice of a strategy by referring to the differences between single attributes inherent in the decision-making situation. Students justified selecting the compensatory strategy by stating that the differences between the characteristics of two options were rather small (6.4% of those who selected the compensatory strategy responded this way):

“I choose the compensatory decision-making strategy because the price difference is not too big, and the rest should be weighed up.” (Task 1, apples for the school cafeteria; UserID 621)

On the other hand, students reported selecting the non-compensatory strategy because of perceived large differences between different options (1.8% of those who selected the non-compensatory strategy made statements to this effect):

“The differing prices are particularly conspicuous and could result in an exclusion principle.” (Task 1, apples for the school cafeteria; UserID 261)

The students presented explicit thresholds justifying the use of a non-compensatory strategy only in regard to financial criteria. A few students stated that options that were unacceptable in financial terms should be excluded.

“First, the price is important, because only 400 Euros are at my disposal. Higher prices can be excluded.” (Task 3, purchase of a bed; UserID 540)

For other criteria, no ecological or social standards were stated as a threshold.

Research Question 1.2: Are the Quality Levels of the Explanations Valid? In Which Way are they Related to Decision-Making Competence?

To validate the quality levels of the explanations, a total score was calculated, indicating the overall performance in the explanations of the selected decision-making strategies. The scores of each task were added, and the median score was used to divide the students into one group with high-quality explanations and another with low-quality explanations. Furthermore, the median of the decision-making questionnaire scores (Eggert and Bögeholz 2010) was determined,Footnote 2 and the students were divided into those who showed a high level of decision-making competence and those with a lower level. Chi-square analyses revealed that the students who had higher-quality explanations also reached a higher level of decision-making competence, according to the questionnaire: χ 2 = 5.6, df = 1, p < .05. Hence, the level of explanations and decision-making competence are connected. This connection supports the validity of the identified quality levels.

Research Question 2.1: To What Extent Did the Students Select the Decision-Making Strategies That Best Fit the Decision-Making Tasks According to Societal Norms?

Figure 1 shows the relative frequencies of the selected strategies. Generally, the a priori expected decision-making strategy was among the one most frequently chosen by the students in the training groups: 43% selected the compensatory strategy for Task 1 (apples for the school cafeteria). For Task 2 (shipyard production site), 39% selected the mixed strategy, which was only the second-most selected strategy. A total of 55% selected the non-compensatory strategy for Task 3 (purchase of a bed). Although a large proportion of the students chose a strategy other than the one we expected, most students were able to detect options that were non-sustainable based on societal norms. For the second and third tasks, approximately 80% selected the non-compensatory or mixed strategy and, hence, identified at least one knockout criterion presented in the decision-making task. However, for the first task, in which a full trade-off with a compensatory strategy was expected to fit best, many students excluded options based on personal values.

Fig. 1
figure 1

Decision-making strategies selected in the learning program by the students in both training groups (n = 263)

Full size image

Research Question 2.2: Did Those Students Who Selected the Strategy That Fits Best, According to Societal Norms, Elaborate More on Their Explanations than Those Who Selected a Different Strategy?

Figure 2 shows the relative frequencies of the different levels of explanations among those who chose a compensatory strategy and those who applied a non-compensatory strategy. Contrary to our expectations, the level of the explanation was not higher for those students who selected the strategy that fits best according to societal norms (see Table 4 for a schematic overview of the expected and observed results). Instead, these data demonstrate that the levels of the students’ explanations show a similar pattern for each context. The students’ explanations of why they would use a compensatory decision-making strategy were more frequently made on a purely strategic level (Score: 1) whereas the explanations regarding non-compensatory aspects were based on strategic considerations linked to specific task characteristics (Score: 2). 2 × 2 (Score 1 versus Score 2; selection of a compensatory versus a non-compensatory strategy) chi-square analyses were conducted to analyze whether this difference was statistically significant (see Table 4).

Fig. 2
figure 2

Levels of explanations for selecting a strategy (separated according to whether the compensatory or the non-compensatory strategy was selected; n = 120). Note. Score 0: No reference to the selected strategy. Score 1: Reference to strategic aspects. Score 2: Reference to the interrelation between strategic aspects and specific task characteristics

Full size image
Table 4 Expected and observed patterns for the quality of metadecision explanations according to different selected decision-making strategies
Full size table

For the first task (the compensatory decision-making strategy example, apples for the school cafeteria), it could be shown that the students who selected the compensatory strategy provided explanations at a basic level (Score: 1) whereas those who selected the non-compensatory strategy provided higher-level explanations (Score: 2): χ 2 = 9.9, df = 1, p < .01. For Task 2 (the mixed strategy example, shipyard production site), the pattern was similar: χ 2 = 8.6, df = 1, p < .01. For the third task (the non-compensatory strategy example, purchase of a bed), however, the students who chose the compensatory strategy offered fewer strategic explanations in general whereas those who selected a non-compensatory strategy offered Level 1 explanations more often than in the preceding tasks. Thus, no differences in the level of explanation could be found between the students who applied a compensatory strategy in Task 3 and those who selected a non-compensatory strategy: χ 2 = 1.0, df = 1, p > .05.

These analyses reveal that in the first two tasks, the pattern of responses was quite similar, even though different strategies were expected to fit best according to societal norms (the compensatory strategy in the first task and the mixed strategy in the second task). The third task presented a comparable pattern, although the level of the explanation was generally lower than in the preceding tasks: it included more statements that did not refer to the selection of a strategy, and more explanations were made on a basic level than the higher level, at which strategic considerations were linked to specific task characteristics. Hence, the hypothesis that those students who selected the a priori expected strategy would also offer higher-level explanations for their selection had to be rejected.

Research Question 2.3: Is the Use of the Non-Compensatory Decision-Making Strategy Associated with Lower Cognitive Load?

To investigate this research question, three types of results were examined. First, verbal data from the students’ explanations about their selection of a decision-making strategy were considered. Second, analyses of the levels of explanations were conducted to examine whether the proportion of students who did not provide an elaborate explanation for their strategy choice (Score: 0) was higher for those who used a non-compensatory strategy than for those who used a compensatory approach. Third, we tested whether the application of non-compensatory strategies caused less cognitive load than the application of compensatory strategies and, hence, is less complex.

Some students justified selecting the non-compensatory strategy by stating that this would be the easiest and fastest way to reach a decision because of the reduced number of options to consider:

“I have chosen the non-compensatory strategy because it works faster than the mixed strategy and, thus, comes faster to a conclusion.” (Task 1, apples for the school cafeteria; User ID 594)

However, only 3% of the students per task explained their strategy selection by referring explicitly to reduced complexity.

Across all three tasks, the average percentage of the students who used a compensatory strategy but did not provide an explanation for doing so (Score: 0) was 28.5% whereas the average percentage of students with a 0 score was higher among those who applied a non-compensatory strategy (34.1%; cf. Fig. 2 for the relative frequencies separately for each task). However, this difference was not significant. Chi-square analyses were conducted separately for each task, contrasting inadequate explanations (Score: 0) with elaborate explanations (Score: 1 or 2) differentiated according to whether the compensatory or the non-compensatory strategy was chosen. As a result, a score of 0 was not significantly more frequent for those who used the non-compensatory strategy than those who used the compensatory strategy for any of the tasks.

In addition, we investigated whether the use of non-compensatory strategies was related to low cognitive load. Therefore, the mean self-reported cognitive load values, which were measured after each task during the learning program, are reported in Table 5. The results of t-tests revealed that in the first two tasks, the perceived cognitive load did not differ according to the strategy applied. In the third task, however, the students who selected the non-compensatory strategy perceived significantly less cognitive load than the students who chose the compensatory strategy (t(143) = −2.30, p < .05, d = .42, r = .19).Footnote 3

Table 5 Self-reported cognitive load after applying the decision-making strategy (means and standard deviations; seven-level Likert scale; −3/+3)
Full size table

Discussion and Implications for Science and Environmental Education

In this study, we investigated how students made decisions on socioscientific issues pertaining to sustainable development. The computer-based intervention enabled students to differentiate between sustainable and non-sustainable options through training that focused on the reflective application of three decision-making strategies. In a transfer phase, the students had to use metadecision skills to select an appropriate strategy for resolving the socioscientific issue based on the characteristics of the decision-making situation. The levels of metadecision explanations, which were developed through qualitative content analyses, could be validated by measures of decision-making competence (Eggert and Bögeholz 2010; Gresch et al. 2011). Moreover, the high interrater reliability suggests clearly identifiable categories.

From these categorizations, it could be shown that the majority of the students scrutinized the task to analyze whether knockout criteria existed or whether a full trade-off was possible. Consequently, the intervention addresses crucial requirements of the German science education standards: systematic decision-making processes and the identification of non-sustainable options through the analysis of relevant criteria (Eggert and Bögeholz 2006; KMK 2005). The levels described to which extent strategic considerations regarding the selection of a decision-making strategy were linked with specific task characteristics. The highest score required the students to reflect on the application of the decision-making strategies in a more elaborate way and process the task information more deeply. This graduation of levels according to the degree of contextualization is in line with the more general definition of competence as a cognitive ability to master particular, contextualized problems effectively (Hartig and Klieme 2006; cf. Weinert 2001a, b). On the whole, it is compelling that approximately three-quarters of the students offered reflective explanations on a strategic level to explain their strategy choice. We suggest that these elaborate metadecision considerations are an important component of thoughtful decisions (Eggert and Bögeholz 2006; Gresch et al. 2011). Similarly, Kuhn (1999) and Means and Voss (1996) consider metastatements to be part of high-quality argumentation as a way to structure an argument. Moreover, the application of decision-making strategies combined with metadecision considerations allow for a free choice of a course of action, while still triggering deeper reflection upon the inherent task characteristics. This is supported through the findings of Gresch et al. (2011), who found that the application of decision-making strategies combined with metadecision aids enhanced the students’ perceived autonomy. Consequently, the intervention study addresses one central requirement of science education, the empowerment to make autonomous and reflected decisions rather than adopting particular viewpoints (Aikenhead 1985; Hodson 2003; Hungerford 2010; Jickling 1992; McConnell 1982; Pedretti 2003; Ratcliffe and Grace 2003; Solomon and Aikenhead 1994).

When designing the learning environment of this study, all decision-making contexts were selected and modified so that each decision-making task required one specific decision-making strategy to adequately resolve the issue according to societal norms regarding sustainable development. However, despite a validation process in which the research group members discussed the fit of the decision-making context with the specific decision-making strategy and optimized it in several consecutive steps, only 39 to 55% of the participating students chose the expected strategy in each of the three tasks of the transfer phase. When evaluating these results, one should consider that these data were not collected after the intervention as part of the assessment, but during the learning process. A compelling result is that in both tasks with inherent knockout criteria based on societal norms (Task 2, the mixed strategy example (shipyard production site), and Task 3, the non-compensatory strategy example (purchase of a bed)) approximately 80% of the students identified at least one of the knockout criteria, i.e., they excluded options using a non-compensatory or mixed strategy. This result suggests that the students considered inherent norms and values, one requirement for the resolution of socioscientific issues (Jiménez-Aleixandre and Pereiro-Muñoz 2002; Kolstø 2001; Ratcliffe and Grace 2003; Sadler 2004; Zeidler et al. 2005). However, from these data, it could not be distinguished between an exclusion of options based on societal norms or personal values. Many students excluded courses of action in the task without knockout criteria (Task 1, the compensatory strategy example (apples for the school cafeteria)) for reasons that were not based on societal norms. To investigate the reasons that the students did not select the expected strategy, two hypotheses were tested: first, selecting the strategy that fits best according to societal norms is associated with more elaborate metadecision explanations, whereas other strategies are justified in a less elaborate way; and second, the non-compensatory strategy was selected more often because it causes less cognitive load.

Contrary to our expectations, the students who selected the most adequate strategy did not provide higher level explanations than those who chose other strategies. Students who selected a compensatory strategy offered explanations regarding strategic aspects without linking them to specific task characteristics whereas students who selected a non-compensatory strategy offered explanations that were more often linked to the decision-making task. This result suggests that the students found it easier to identify concrete knockout criteria than to illustrate why the disadvantages of one criterion can be compensated for by the advantages of another in a full trade-off. This outcome is plausible because it is less demanding to identify one aspect than to simultaneously take several aspects into account to describe why a trade-off between them would be appropriate.

Gresch et al. (2011) found that students who were familiar with decision-making strategies tended to overuse non-compensatory approaches. One possible explanation for this finding is that the non-compensatory decision-making strategy was associated with a lower cognitive load because it requires less evidence to be considered before reaching a conclusion than a compensatory strategy does (Hong and Chang 2004). The analyses of the process-related written data in the present study and the measurements of cognitive load revealed that for some students, the non-compensatory strategy is indeed a way to reduce the complexity of the decision-making situation, particularly in the last task of the intervention. This difference may also be a result of the effect of the task’s position at the end of the program, when students may have attempted to finish the program more quickly. Nevertheless, the large majority of the students dealt with the socioscientific issues in an elaborate and reflective way.

In conclusion, the analysis of the process-related data from the intervention study revealed that the students did not randomly select a strategy in the transfer phase of the intervention, but did indeed exert elaborate and reflective metadecision skills. Because the students generally dealt adequately with the intervention tasks, two aspects should be considered when interpreting the finding that the students selected different strategies than expected. First, a closer examination of the presented contexts is necessary to draw conclusions for developing decision-making tasks of future trainings. All tasks consisted of three or four options, which were described according to four or five relevant criteria. Generally, the cognitive load was considered moderate for all tasks. This finding suggests that the complexity level was adequate for the trained students and that the quality was suitable to engage the students in the training. Another aspect is the presentation of the knockout criteria. When the contexts were selected and modified during the learning program development, options were considered non-sustainable if they had knockout criteria based on societal norms, such as devastating ecological, social or economic impacts. For the decision-making tasks with inherent knockout criteria, most students identified at least one of these characteristics, which suggests that these knockout criteria were presented in a way that allowed the students to recognize them. However, for the first task, an example of compensatory decision-making in which the students had to select a variety of apples for the school’s cafeteria, many students selected the non-compensatory or mixed strategy. Because no societal norms demanded an exclusion of options in this task, personal criteria, such as the price of an apple or a strict avoidance of fruits that were not locally grown, prompted the selection of non-compensatory decision-making strategies. Hence, the individual decision-makers had different thresholds for eliminating options. We propose that consumer decisions that are routine and everyday decisions are not optimal for introducing the application of decision-making strategies because thresholds determined by societal norms interfere with personal knockout criteria. Furthermore, consumer decisions are not ideal for assessing decision-making competence when considering the application of decision-making strategies. However, we recommend dealing with consumer decisions in a subsequent step in class to further differentiate between societal norms and personal values.

This leads to the second, more theoretical, implication, the balance between the students’ autonomy when resolving socioscientific issues and the need, from a normative educational perspective, to confront students with the framework of sustainable development as the outcome of an international political consensus. In science and environmental education, it is widely acknowledged that teachers should not promote a particular point of view to avoid indoctrination (de Haan et al. 2008; Hodson 2003; Hungerford 2010; Jickling 1992; Ratcliffe and Grace 2003). Instead, education should enable students to make elaborate decisions. In the present study, the autonomous selection of a decision-making strategy and a course of action allowed the students to express their procedural knowledge and their standpoint in an elaborate and independent way. Gresch et al. (2011) found that this combined teaching of decision-making strategies and metadecision activities not only enhanced the level of decision-making competence but also increased the students’ perceived autonomy when dealing with socioscientific issues related to sustainable development. Furthermore, we observed that despite this autonomy, the majority of the students detected non-sustainable options based on societal norms in the transfer phase of the training. In addition, personal values, which are highly dependent on the decision-maker’s attitudes, prior individual experiences and routines, were considered. This result supports the claim of Beach (1990) and Beach and Mitchell (1978) that the selection of a decision-making strategy depends on both the characteristics of the decision-making task, which involved both non-sustainable and sustainable options in this study, and on the decision-maker’s characteristics, because the perception of the task characteristics is a subjective process. Consequently, an evaluation of the quality of students’ decision-making processes should involve an analysis of the metadecision explanations to gain further insights. Nevertheless, the concept of the fit of a strategy with the type of task is useful when designing learning environments due to normative educational considerations. Strategic training in decision-making becomes meaningful only if the contexts are selected in a way that makes the application of the decision-making strategy plausible to the majority of the students. On the other hand, if societal norms and a multitude of personal values interfere, it provides a good opportunity to reflect on the development of societal norms. Moreover, it is vital from a normative educational viewpoint that students are able to reflect upon the norms that were negotiated and accepted by the majority of the world’s societies to ensure sustainable development because this stimulates self-reflection (de Haan et al. 2008). Central efforts to yield a global consensus, such as Agenda 21 of the United Nations (UNCED 1992) and subsequent global conferences, are important points of reference for classroom activities.

The present study has revealed that thresholds are only considered explicitly for financial criteria. Consequently, ecological standards, e.g., minimum requirements for the water quality of limnological or marine ecosystems, or social standards, such as working conditions and social security, should be discussed in class. Such negotiations can be used to make the societal norm development process more transparent. This determination of thresholds could then be combined with strategic considerations to examine which options should be excluded in a non-compensatory approach because they do not reach the minimum thresholds. Consequently, the use of decision-making strategies to resolve socioscientific issues related to sustainable development offers opportunities to reflect on the distinction between societal norms and personal values and improve perspective-taking abilities. Therefore, future research should further elaborate on the development of suitable methods and appropriately framed decision-making tasks to stimulate this reflection process. Moreover, we suggest that the distinction between societal norms and personal values should not only be part of learning environments and curricula but also be assessed as a component of decision-making competence. Analyses of metadecision statements are one promising approach for this purpose.