The Role of Value Deliberation to Improve Stakeholder Participation in Issues of Water Governance

Abstract

Stakeholder participation is a requirement for environmental decision-making in the European Union. Despite this, numerous instances can be seen in water governance in which stakeholders feel undervalued and unheard, thereby creating unfavourable procedural outcomes, resistance and conflict. In this article, we propose that a process of early-stage deliberation constructed around the values of the stakeholders involved can reduce, and even prevent such conflicts. We suggest that if values that stakeholders perceive as relevant can be identified and discussed as part of the deliberation process then (1) stakeholder preferences can change, and (2) participants can develop a mutual understanding of each other’s values and perspectives. To explore these propositions, facilitated workshops were conducted at two Dutch water institutes, based around the topics of land subsidence and the pharmaceutical contamination of water systems. Participants deliberated on values that they considered relevant. The results suggest that mutual understanding of stakeholders’ perspectives increases as a result of value-based deliberation.

Introduction

Stakeholder participation is widely recognised as a central component of environmental decision-making. This is reflected in the Aarhus Convention¹ and subsequent environmental legislation, with both the European Water Framework Directive (Directive 2000/60/DC) and the European Floods Directive (Directive 2007/60/EC) placing similarly strong emphasis on the role of the stakeholder in the water management process. In its most general sense, stakeholder participation refers to a process that facilitates the inclusion of those involved in, affected by, knowledgeable of, or having expertise or experience relevant to the issues at stake (Van Asselt and Rijkens-Klomp 2002). This process can range from simple information provision to independent, public-led decision-making (Mostert 2003). Benefits commonly associated with stakeholder participation include better use of the available knowledge and experiences of different stakeholders; increased public acceptance, through more transparent decision-making processes; and reduced litigation, delays, and inefficiencies in outcome implementation (Ruiz-Villaverde and García-Rubio 2017; Papadopoulos and Warin 2007). Despite this, and despite the involvement of stakeholders in water management-related decision making being something that is both required and frequently practised (WMO 2009; Huitema et al. 2009), no standardised means for facilitating stakeholder participation exists. Nonetheless, numerous examples of failed or poorly implemented stakeholder participation can be seen across the literature (Reed 2008). Stakeholder participation may fail, for instance, if stakeholders are left subjectively unheard, leading to feelings of resistance and conflict; if they are involved at too late a stage in the decision-making process; or if the temporal demands placed upon them are such that they are unable to commit themselves to the process (Hommes et al. 2009; Reed 2008; Pahl-Wostl 2002; Andersson et al. 2008).

Stakeholder participation is considered especially important in situations with complex, unstructured problems – described previously as ‘wicked’ problems (Rittel and Webber 1973) (‘messy’ Ackhoff1974 or ‘ill-structured’ problems Dunn1988). Wicked problems often lack a straightforward answer because of incomplete, contradictory, or dynamic components that are often difficult to address directly (Brunner et al. 2005). They are characterised by ambiguity regarding problem definition, uncertainty involving the causality of relationships between problems and potential solutions, and disagreement surrounding important normative elements (e.g. values, norms or objectives) (Hisschemöller and Hoppe 1995). Many water management problems are ‘wicked’ because they comprise complex human and natural systems (Kolkman et al. 2005) often involving different governmental spheres (OECD 2011) and multiple stakeholders using the water system (Geldof 2001). Typical examples are problems relating to pharmaceuticals in the water system (Kasprzyk-Hordern et al. 2008), land subsidence as a result of groundwater extractions (Van den Born et al. 2016), and pluvial flooding during heavy rainfall (Farrelly and Brown 2011).

Given the (wicked) nature of these problems, the challenge for participatory methods lies in being able to interweave process-related elements with situation-specific content, thereby developing a shared understanding between stakeholders regarding the problem at hand (Edelenbos et al. 2003; Koppenjan and Klijn 2004). It has been argued that participatory methods can, and should, actively stimulate participant’s understanding of one another’s values and interests – as opposed to being solely directed towards consensus finding (Karpowitz and Mansbridge 2005). Stakeholders are thereby better able to recognise the moral merit of perspectives that are different from their own (Gutmann and Thompson 2009).

Despite this growing recognition that an exploration of differing values, as well as interests, can be of significance to the deliberative process, and that participatory methods can be highly useful in recognising the (moral) merit of other people’s perspectives, little attention has been given to methods that systematically address stakeholders’ values, or their differing interpretation of these values. Empirical research from the energy sector suggests that moral and social values have a large impact on the general societal acceptance of energy projects (Gross 2007; Wustenhagen et al. 2007). A growing number of scholars are therefore calling for more comprehensive integration of moral values throughout the design of energy projects (Kostyk and Herkert 2012; Demski et al. 2015). Although less systematically studied, empirical evidence from the water domain suggests a similar positive effect in adopting a value-based approach over an interest-based one (Briggs et al. 2005; Glenna 2010; Doorn 2016).

‘Values’ are used in this article as the basis for describing stakeholder participation. This approach develops on that of Cheng and Fleischmann (2010) in treating values as underlying determinants of what people think is important in life. The concept of values is used to form a basis for the deliberation process – defined here as a group-based process of participation, social exchange, reflection, and learning – in which participants have the opportunity to reflect upon, form, express, and debate their viewpoints, values and beliefs (Kenter et al. 2016a). We focus on value deliberation in the early stage of the decision-making process in which direct stakeholders discuss possible solutions in order to ascertain existing and prevailing attitudes and values surrounding the problem in question. This is with the aim of understanding the role that value identification and discussion can play in this early stages of environmental decision-making, building on the ideas of Karpowitz and Mansbridge (2005), Gutmann and Thompson (2009), Doorn (2009), and Glenna (2010).

This article is structured as follows: Section 2 details the objectives of the research and the propositions to be tested. In Section 3, value elicitation, participatory deliberation, and the process of data collection are discussed. Next, Section 4 describes the results of the data collection at, respectively, a Dutch water research institute and a Dutch water board – allowing for analysis of the propositions. Following this is a discussion of the results in Section 5 and a section detailing related work. Finally, in Section 7 we present our conclusions and introduce possibilities for future research.

Objectives

Our aim is to use value deliberation to achieve stakeholder mutual understanding that will ultimately lead to better-supported water governance. This is achieved by integrating the values of the stakeholders involved in the process. Stakeholders can include policymakers, concerned citizens and business owners. Values that appear to be relevant to stakeholders should be expressed consistently over the course of the process (Gregory et al. 2005) to allow for good water governance. We aim to answer the following research question:

To what extent can the discussion of values impact the deliberation process concerning water governance issues?

In order to answer this question, we analyse two propositions constructed to define the extent of the impact of value discussions. First, we want to know whether the discussion of values affects the ranking order of alternative values. This leads to our first proposition:

P1: Making values part of the deliberation process may change participant’s preferences with respect to the alternatives.

This proposition is analysed by comparing participant’s ranking of prefered values at the beginning of the deliberation to a ranking performed towards the end of the deliberation. In Section 3.2 this is explained further.

Second, whether a participant changes their preferences does not infer that their understanding of the preferences has changed. Increased understanding could lead to a change in preference ranking, or equally result in confirmation of the earlier ranked order of preferences (Barabas 2004). In order to test for a post-deliberation impact other than change in individual preferences, we investigate whether there is a change in participant understanding of other people’s perspectives:

P2: The discussion of values during the deliberation process leads to an increased understanding of other perspectives.

We use the term ‘understanding’ here to describe an acquired acceptance of another viewpoint with a certain degree of sympathy, not the act of cognitive understanding. Proposition 2 is analysed through a post-workshop evaluation completed by participants, including a short survey.

Methodology

There has been ample research into methods that increase participation in social learning (Hommes et al. 2009; Pahl-Wostl 2002; Reed 2008; Cuppen 2012a; Renn et al. 1997), yet specific methods concerning value deliberation are scarce.

Value Elicitation

The main rationale for deliberating values instead of discussing interests is that value discussions have been shown to be able to transcend the inevitable interest-based disagreements amongst stakeholders (Glenna 2010; Doorn 2016), and instead become a dialogue around other people’s perspectives. If participants understand each other’s viewpoints and perceptions, they can be more willing to work towards a common solution (Habermas 1995). To reach this mutual understanding, participants have to be willing to reflect on their values, assumptions and larger social context. In addition, they must try to understand arguments from other perspectives through respectfully listening to each other (Dahlberg 2001).

Before one can investigate an appreciation or understanding of other people’s values, the values themselves must first be identified. One potential approach to this is that researchers select the values they think are worthwhile investigating (Kenter et al. 2016b). ‘Values’ can be an abstract concept for participants, so having researchers preselect a defined set of values can make the concept clearer and more practical. The clear downside is that this makes the deliberation biased towards values chosen by the researchers.

Another approach, therefore, is to give participants total autonomy to identify which values they think are relevant. This could be facilitated either by imposing a definition of value, or by leaving it up to participants to dictate what they consider to be a value.

Participatory Deliberation

We took inspiration from the Delphi methodology, a much used and investigated data-gathering method designed to elicit consensus amongst experts. This method is further able to determine a range of alternatives around a given discussion topic, and can also be used to delineate the underlying assumptions they are based on (Hsu and Sandford 2007). The Delphi method provides measurable outcomes and a clear structure: questionnaires are sent to preselected experts, their responses are summarised, and summaries are sent back to the experts together along with a second questionnaire comprising roughly the same questions. This is commonly done through completing two rounds of the process, but more can be performed if required. In the last round, the experts often rank the ideas generated in the questionnaires (Landeta 2006; Van de Ven and Delbecq 1974).

Using the iterative structure of the Delphi method, we organised two workshops based around two discrete (wicked) water governance problems. Each workshop contained participants with differing levels of expertise. The process is depicted in Fig. 1. Data were collected via rankings, group evaluation and a survey.

Fig. 1

Value deliberation process

The deliberation starts with a formulation of alternatives that could solve the problem. Some of the alternatives may be pre-determined because they are ‘standard’ or currently investigated approaches to the problem. Additional or new alternatives can be introduced (Renn et al. 1993). For each alternative, pro and con arguments are collected.

Participants then rank all alternatives on a ranking form – from most preferable, ranked number one, to least preferable, ranked bottom – to establish a baseline preference measurement. Using the Borda count for the aggregation of individual rankings, any changes in personal ranking preferences can be tracked whilst remaining anonymous, ensuring that the outcome is unbiased towards individual voters (Young 1988).

The rankings are collected to calculate the aggregated score for each alternative. The alternative that scores highest on the aggregate level is not necessarily the most preferred alternative for participants individually. Therefore the individual rankings are considered as well, to have a complete overview of the changes in ranking behaviour. This information is used to analyse Proposition 1.

Values are then identified by asking participants what values they think are relevant per alternative, and if they can identify overarching values common across alternatives. The participants then reflect on why these values are important to them. Sharing these reflections forms the core of the deliberation (McCrum et al. 2009). Alternatives are then ranked again, after which point the differences between the two rankings (or a lack thereof) becomes visible and is discussed. These outcomes are used to analyse proposition 2. Furthermore, participants are asked if the process gave them any additional insights or taught them anything new by way of a short survey. The answers to this are used as a second unit of analysis for proposition 2.

Data Collection

Data collection took place in the spring of 2017 over two workshops. In workshop 1, four researchers facilitated and documented a value deliberation process at a Dutch water research institute. There were six participants, all connected to the water research institute, deliberating on the topic land subsidence.

In workshop 2, three researchers facilitated and documented a value deliberation process at the Dutch water board Waterschap de Dommel. Ten participants from the water board deliberated on the topic pharmaceuticals in the water system.

Results

In this section, we describe the results of the two workshops. In both workshops, participants contributed two additional alternatives to the list of pre-determined alternatives that served as a starting point.

Workshop 1 on Land Subsidence

For the measurement of impact and analysis of the propositions, the arguments are an aid rather than a key outcome. A complete overview of the arguments related to each alternative can be found in Appendix B, Table 1. The values that were identified are also listed in Table 1.

Rankings

Aggregate rankings for the two rounds are depicted in Fig. 2.

Fig. 2

Aggregate rankings of Workshop 1

The chart shows that, on aggregate, option C was ranked first, then option B, and so on. The order of the aggregated preferences did not change in the second round (after the discussion).

The differences between the two rounds appear minor on an aggregate level, yet all participants changed their order of preferences in the second round, and one changed their most preferred alternative (see Appendix B).

Group Evaluation

During the evaluation, all participants explained why they had changed their order of preferences. Reasons given include that ‘the urgency of the problem became clearer because of the value discussion’ and ‘too-invasive changes/alternatives became less important because of the discussion’. In summary, participants stated that the value discussion resulted in a better understanding of the problem and of the alternatives.

Discussion

The discussion on values, guided by the questions ‘What values did you write down?’, ‘Why?’, ‘Does everyone agree?’, and ‘Why not?’, led to discussion of the values of cultural history, governance, safety, landscape innovation and landscape disappearance.

Survey

In the survey (see Appendix A) participants were asked if their ideas had changed after the value discussion. Some of the responses given to this included: ‘Yes, I have more understanding of the alternative and the impact of the alternative’, and ‘My ideas have become richer, more complete’. Another question asked was if the process taught them something new, or allowed for different insights to be gained. Participants responded, ‘Not something new, but new points of view [from which] to approach the problem’; ‘Maybe not content-wise, but it sharpens the mind’; and ‘Different types of values [became apparent], that did not cross my mind before but which are actually very important’. The answers to these questions show that participants generally experienced a change after the value discussion, allowing them to better understand others’ – as well as their own – perspectives.

Workshop 2 on Pharmaceuticals in the Water System

For this workshop, the complete overview of alternatives, arguments and values per argument can be found in Appendix C, Table 2.

Rankings

The aggregate rankings of this workshop are depicted in Fig. 3. This shows that in the first round option A was most preferred, followed by the alternatives C, D, B, F and E. However, the aggregated preferences changed after the second round, with C ranked first, followed by D, A, E, B and F.

Fig. 3

Aggregate rankings for workshop 2

One participant did not change anything, yet all others did. In addition, six of the nine participants changed their most preferred alternative in the second ranking (see Appendix C).

Group Evaluation

During the evaluation, participants expressed their reasoning behind changing or maintaining their order of preferences, with explanations such as: ‘In the first round I reflected from the water-authority perspective; in the second I reflected on the values for society’; ‘First I thought: what is the most practical to do? But then I realised that we are really going the wrong way – something really needs to change’; and ‘There is no wrong or right. It feels very good to discuss it in such way.’

Discussion

The discussion on values expanded into a discussion concerning personal values, societal values and organisational values. Values that were discussed were the concept of ‘Gaia’ (the theory that sees the earth is a complex, living organism), priorities, survival, own responsibility, and safety. Participants told personal stories about the future of their children and the lives of their elderly parents to express what these values meant to them. This made the discussion per value longer than in workshop 1, and more personal. Participants stated that the realisation that these issues (around pharmaceuticals in the water system) had become personal was only apparent after the value discussion had occurred.

Survey

To the question of whether participants’ ideas had changed after the value discussion, answers included: ‘No, but I could understand my thoughts after the discussion about values better’, and ‘Yes, [my ideas changed as a result of] thinking more about society’s point of view’.

To the question of whether the deliberative process gave them different insights, participants’ stated, ‘Yes, [in] making explicit that people have values and discussing them’, and ‘More understanding of differences [allowed for] insights in the complexity of the issue’.

Most participants expressed that they had gained new insights because of the deliberation (five out of six in Workshop 1, and eight out of ten in Workshop 2).

Discussion

Our aim was to develop a better understanding of the role of value discussions in deliberative processes. In this section, we first reflect on the measurability of the deliberation process and on how values were identified and discussed. This is followed by a discussion of the propositions and research question.

How We Measured Value Deliberation

Deliberation took place in an environment conducive to open discussion; in which participants felt free to speak up and to reflect on their values. Measurements were performed using a ranking process whereby the preferences of participants are quantified and become easily comparable.

In workshop 1, the value discussion did not result in large changes in aggregate rankings. The discussion was vivid, but participants tended to express a similar set of moral convictions, or adopt a similar ‘worldview’. This led to rankings that were fairly homogenous. All participants, however, changed their rankings between first and second rounds.

The aggregate rankings from Workshop 2, however, exhibited substantial differences. Even though all participants were working for the water authority, they displayed highly different ideas regarding the discussed values. The value ‘Gaia’ drew particularly intense discussion, developing into an exchange of personal stories to support each participant’s values. This discussion resulted in changes in the rankings on an aggregated level.

In both workshops, new alternatives were suggested and brought to the table, allowing the discussion to adopt a more comprehensive view of the topics. The participants became aware of their own conceptions of values and of those of other participants. Stories regarding values became both factually rooted and highly personal, and allowed participants to understand each other better. In the evaluative discussions for both workshops, participants stated that they had obtained a better understanding of the problem, the alternatives, and what is at stake with respect to the values relevant to all stakeholders.

Identifying values was not a goal in itself, but rather an instrument to support a deliberative process that invites participants to reflect on the topic in more abstract terms. The aim was to transcend discussion on the level of interests, which was achieved.

The moderator did not set rules detailing what constitutes a value and what does not. Not being strict regarding this definition created room for an open-ended identification of values. This resulted in values that could be actual sentences (e.g. ‘We are a societal player, so we are part of the issue’), worries (e.g. bad for economy), or opposing concepts presented under one label (e.g. landscape innovation and landscape disappearance).

In both workshops, many different values were identified by participants (an overview of the values can be found in Tables 1 and 2 in the Appendices) and the value elicitation evolved organically; there was a sense of openness in which no input was considered ‘wrong’.

During the Workshop 2 discussion on pharmaceuticals in the water system, participants described that their perspective on the topic seemed to have shifted to a personal one, even though all participants were present in the role of water authority representatives. If value discussion can allow participants to temporarily relax or step aside from their official role – simply by virtue of becoming focused or engaged in the topic under discussion – this could create a way of approaching the complex social process whereby stakeholders have to meet in often less harmonious settings. It could open up new discussions and challenges between stakeholders; ones that were not conceivable before. This is reflected in the previously posited idea that diversity – and open-mindedness in perspectives and opinions – can be considered a prerequisite for proper deliberation in which participants learn from each other and reflect on each other’s perspectives (Barabas 2004; Cuppen 2012b). In addition, opening up on a personal level during the workshop can facilitate increased feelings of trust amongst participants (Kenter et al. 2016a).

In Workshop 2, the differences between the individual rankings increased in the second round. This scenario could be seen as expected, as earlier research has described how deliberation can bring pre-existing differences to the surface, diverging the perspectives of participants rather than converging them (Shapiro 2002). If participants’ expectations are well facilitated during the deliberation, such divergence is not necessarily a problem; deliberation in an early stage is meant to allow participants to understand each other better and to develop an awareness of the different concepts present.

Through conducting two separate, small-scale studies – in which external researchers assume the role of facilitators – group dynamics could be observed and interactions could be documented. The experimental, qualitative, and small-scale nature of this study does not allow for statistical analysis. However, if the same methodology were performed on a larger scale, the rank orders obtained through the Borda count allow for calculation of Kemeny-Snell distance (Kemeny and Snell 1972) to evaluate whether the rankings of a deliberation have become more similar in a second measurement.

Proposition Testing and Answering Research Question

We analysed two propositions to answer our research question. The first proposition was evidenced in Workshop 1 whereby all participants changed their preferences. In Workshop 2, the changes in preferences were even more apparent: six of the nine participants changed their most preferred alternative after the value discussion. These results confirm that making values explicit and subject to group discussion can change a participant’s preference of the alternatives.

The second proposition was confirmed in both workshops. In Workshop 1, the group evaluation reported a clarification of the problem and the alternatives; the survey, in turn, reported a broadening in understanding. One participant described specifically an increased understanding of other perspectives: ‘I appreciated the change-over between the technologies and the values. Hearing values from others has an effect that your own scope is broadened; you get empathy for others. For example, if you hear others talk about the speed of the transition, this gives new insights that adjust your own understanding and opinion. You cannot attack someone on their values. Hearing the values of another participant influences your own opinion.’

In Workshop 2, one of the reasons that the understanding of participants had changed was that that they were able to reflect on the values from differing perspectives: their professional perspective as a representative of the water board; a societal perspective; and as a private citizen, reflecting on the personal consequences that the issue encompasses. In particular, the differing views and discussion arising with respect to the value ‘Gaia’ reflected thoughts that were not expressed before by the participants.

This leaves us to finally to consider our original research question: To what extent can the discussion of values impact the deliberation process concerning water governance issues? Based on the analysis of the two workshops we can see that deliberation of values changes both participants’ preferences and increases the understanding of other perspectives. This finding is a promising first step to improve the deliberation and subsequently decision-making processes in water governance.

Related Work

Research based both in environmental governance and innovative democracy has yielded studies describing the significance of deliberation. In a study on methods of enhancing social learning, researchers organised deliberative workshops allowing land managers to share and deliberate their reflections on climate change (McCrum et al. 2009). These workshops stimulated social learning, but neither mutual understanding nor values were taken into account. Kenter et al. (2016b) discuss how to combine value deliberation with group-based decision-making in managing ecosystems. The goal of these studies was to deliberate on monetary values through various workshops in which information is elicited and values become explicit. No deliberation took place, however, concerning stakeholders’ moral values.

Deliberative democracy is a process by which to involve citizens in decision-making processes for policies (Mouffe 1999; Gastil and Levine 2005; Gutmann and Thompson 2009). The role of values in these deliberations is only mentioned sparsely, but – when discussed – is presented as a crucial aspect (Karpowitz and Mansbridge 2005). Nonetheless, no systematic attempts to realise this are discussed in the deliberative democracy literature.

Additionally, the concept of perspective-taking in relation to decision-making is investigated within the field of social psychology (Galinsky et al. 2014). The study discusses how perspective-taking is able to influence decision-makers, and seems to prevent them from clashing with others in the process. However, the results are laboratory-based, with students role-playing the decision-making process. In addition, the experiments did not focus on moral values.

Conclusions and Further Research

Numerous examples of issues of water governance can be characterised as wicked problems, with corresponding difficulties in stakeholder participation. The goal of this study was to increase participants’ understanding of other participants’ perspectives – through discussing values instead of interests. We explored the role of value discussions as part of stakeholder participation for water governance by facilitating two workshops. The first workshop was a deliberation on land subsidence; the second concerned pharmaceuticals in the water system. In both workshops we facilitated the identification of relevant values. This resulted in a rich palette of values that were reflected upon through vivid discussions. For all participants, deliberating on values was a new way of reflecting on the topic. Most participants stated that their understanding of both the topic and of perspectives other than their own had increased.

The two workshops validated Proposition 1: that value deliberation changes a participant’s preferences. In Workshop 1, all participants changed their rankings; in Workshop 2, all but one participant changed their ranking.

Proposition 2 – that value deliberation leads to an increased understanding of other perspectives – was confirmed in both workshops, making value deliberation a promising approach to incorporate into policy-making processes in general, and water governance policies in particular.

One of the limitations in this research is the lack of diversity in stakeholders, as all participants in both workshops were colleagues. In future research, the group of participants should represent the diversity of stakeholders inherent in such wicked problems. We expect this to affect the dynamics during the workshop. A new proposition could be to test whether a greater diversity in perspectives among participants increases the impact of the deliberation.

A challenge in deliberating with more diverse stakeholders is that trust among the participants is not implicit and pre-existing, yet it is necessary that they work collaboratively towards a shared goal (Focht and Trachtenberg 2005). This does not mean that the participants should have similar ideas on a topic. However, it does require that there is an environment of mutual respect in which participants feel safe to share their ideas and perceptions, even though these may diverge or conflict (McDonough and Cedrone 2000).

Since this study was the first to measure the impact of value deliberation on water governance issues, we constructed a small-scale study using two cases. This scale provided enough room to collect participants’ experiences in a qualitative way, which not only resulted in measuring the impact, but also lead to an understanding of why there was an impact.

The intuitive next step of this study would be to scale-up in order to perform a statistical analysis on the impact of value discussions. The Borda count allows for statistical analysis when large amounts of deliberations are held, all using the same method and tools, and in the same context. A tool would need to be devised so that the data can be collected by multiple facilitators simultaneously and in a uniform way. This can be achieved by developing a data collection tool and by training facilitators in the use of the method and the tool. The number of people participating per workshop should remain within the current range – under ten – or meaningful discussions in which all members of the group can participate would become difficult.

Appendices

Appendix A: Survey

The survey as distributed at the end of the workshops. Since both workshops were held in Dutch, the original survey was also composed in Dutch. This is the English translation.

Process

• 1. Can you describe what you think of the process?

• 2. Did you think the process was clear?

• 3. Did you think the process was useful?

• 4. Did your ideas change after discussing the values? Can you explain why (or why not)?

• 5. Did the process teach you something new? Give different insights?

Role in the process

• 6. What is your job title?

• 7. What is your experience with the topic on a scale from 1 to 7, where 1 stands for no experience at all, and 7 for a lot of experience?

• 8. Do you have a direct stake in the final decision that will be made?

Appendix B: Outcomes workshop 1

Outcomes ranking per participant for workshop 1: ranking 1 – ranking 2

• 1. CDBAE – CBADE

• 2. CDBAE – CBADE

• 3. CDBAE – CDEAB

• 4. CABED – CBADE

• 5. CBAED – BCAED

• 6. CBAED – CBADE

Table 1 Alternatives, arguments and values of Workshop 1 on Land subsidence

The overview of alternatives, arguments and values of workshop 1 are listed in Table 1. Arguments and values are written down in a random order. The values that were discussed are marked in bold.

The values that were mentioned as overarching, not related to one specific alternative are: agriculture, economy, solidarity, safety, ease, democratic values, we are a knowledge economy, making money with agriculture is so last century, agricultural economics, local assignment, national assignment, intragenerational justice, intergenerational justice, future resistance, the pollutant pays.

Appendix C: Outcomes of workshop 2

Outcomes ranking per participant for workshop 2: ranking 1 – ranking 2

• 1. AFBECD – CEFABD

• 2. ACBDFE – CEDABF

• 3. DBFCEA – DCEFBA

• 4. ADCBFE – DACBFE

• 5. FCABED – FCABED

• 6. ADCBEF – DACFBE

• 7. ACDBEF – DCBAEF

• 8. ACFEDB – AEBCFD

• 9. ACDBEF – DECABF

Table 2 Alternatives, arguments and values of Workshop 2 on Pharmaceuticals in the water

The overview of alternatives, arguments and values of workshop 2 are listed in Table 2. Arguments and values are per alternative written down in a random order. Values that were discussed are marked in bold.

The values that were mentioned as overarching: a healthy society, future of clean and healthy environment, protect water for future generations, prevention is better than healing, awareness, sustainability, quality of life, survival.