1 Introduction

Water management is becoming increasingly complex (Pahl-Wostl 2007). It requires not only knowledge about the functioning of water systems, but it also involves many different and often conflicting interests and stakeholders. Solutions for coping with this complexity are often sought either in more research or in more stakeholder involvement, and sometimes in both. Prime examples of the latter are the European Water Framework Directive (2000/60/EC) and Floods Directive (2007/60/EC), which require different technical analyses as well as active participation of all interested parties (Howarth 2009; Dieperink et al. 2012).

While research and stakeholder involvement are usually treated as separate activities, it is increasingly claimed that they should not and that researchers should collaborate with authorities and other stakeholders. Collaborative research would be the best way for dealing with different perspectives, informing decision-making and preventing controversies (Folke et al. 2005; Funtowicz and Ravetz 1993; Gibbons et al. 1994; Maasen and Weingart 2005; Pahl-Wostl 2007). It would "allow for exchanges, co-evolution, and joint construction of knowledge with the aim of enriching decision-making” (Van den Hove 2007). Relations and trust between the actors involved may improve (e.g., Collins and Ison 2010), the credibility of the research results and their relevance for policymaking may increase, and as a result of this appreciation of the research results by authorities and other stakeholders may improve (Lavis et al. 2004; Sarewitz and Pielke 2007; Henriksen et al. 2009). By enhancing reflection on their own and others’ perspectives, collaborative research may also lead to better understanding among the stakeholders of each other’s perspective (Habermas 1984; Slavin 1996; Croke et al. 2007; Dimadama and Zikos 2010).

And finally, collaborative research may promote cognitive learning (Gray 1989; Busenberg 1999; Boonstra 2004; Hisschemöller 2005; Stringer et al. 2006; Croke et al. 2007; Muro and Jeffrey 2008; Collins and Ison 2010; Dimadama and Zikos 2010). As a result of the exchange of insights, the perspectives of the actors involved may change. They may gain new knowledge about the system to be managed and new ideas about management strategies (Croke et al. 2007; Collins and Ison 2010), but they may also develop new perspectives on the problems at hand and the goals to be achieved (cf. Argyris 2004). This change of perspectives may result in a better alignment of the perspectives of the actors involved and hence to more consensus (Muro and Jeffrey 2008).

Despite of the many alleged benefits of collaborative research, little empirical research exists (Raadgever 2009). The purpose of this article is to increase the limited body of empirical research. It aims to test the hypotheses that collaborative research leads to cognitive learning, and to improve insight in the conditions under which cognitive learning is more or less likely to happen. To reach these aims, we conducted two case studies in the field of water management, one on future flood management in the Lower Rhine basin, and one on groundwater management in and around the city of Delft. We assessed whether the perspectives of the actors involved changed over time, using a repeated Q sorting questionnaire (Pelletier et al. 1999), focusing on the uptake of research results (cf. Croke et al. 2007) and the development (or not) of consensus (cf. Muro and Jeffrey 2008). Moreover, we inventoried the literature for factors that may influence cognitive learning from collaborative research and assessed whether these factors were present and whether they could explain the cognitive learning that occurred. For this purpose we observed relevant processes and conducted workshop evaluations and interviews.

In the remainder of this article, we first elaborate the concept of cognitive learning and the factors that may influence cognitive learning from collaborative research. We then introduce the methodology that we used to assess cognitive learning and potential factors of influence in the cases. Next, we describe the cases and present and analyze the case study results. We end our article with a discussion section, conclusions and recommendations.

2 Cognitive Learning and Influencing Factors

Cognitive learning can be defined as the mental process that results in changes in the perspectives of stakeholders. Perspectives in turn can be defined as more or less consistent and enduring mental representations of a specific issue and the position of the individual related to this issue as seen by this individual (cf. "mental models" in Doyle and Ford 1998; Kolkman et al. 2005). They may concern problems, values and interests, goals, and preferred management strategies.

Cognitive learning is influenced by the information that an individual is exposed to and individual selection and interpretation of this information (Dewulf et al. 2004; Beratan 2007; Carton 2007). Argyris (2004) distinguishes two levels of learning: single loop and double loop learning. Single loop learning can be summarized as ‘learning to do things better’ – learning which strategies to implement to solve a given problem –, whereas double loop learning is ‘learning to do better things’ – learning about which problems to solve and which goals to reach. Single-loop learning is relatively easy, but double loop learning requires a reconstruction of the person's perspective.

In collaborative research not only the researchers, but all participants may introduce new information, co-construct information, and influence reconstruction of others’ perspectives. Table 2 identifies specific factors that may influence learning from collaborative research. The first category of factors is related primarily to the researchers and their activities and predominantly influences the uptake of research results. Uptake is more likely when the exchanged knowledge is seen as relevant for the issue at stake (Cash et al. 2003; Brugnach et al. 2007; Collins and Ison 2010). In addition, the scientific quality and credibility of the knowledge influence learning (Weiss 1977; Cash et al. 2003; Brugnach et al. 2007; Croke et al. 2007; Henriksen et al. 2009). Credibility depends to a large extent on the type and authority of the researchers involved. Moreover, the perceived relevance and quality of specific research is important. These may be influenced by other research on the issue at stake, which may support or contradict the results of collaborative research and may make the results new or not.

The second category of factors is related to the policy process and the stakeholders involved in this process. First, cognitive learning is more likely when the issue at stake is seen as urgent and requiring new knowledge. Secondly, the specific stakes of and the relations between the stakeholders are important (Dillenbourg et al. 1996; Olsson and Andersson 2007). For example, individuals are most likely to learn from knowledge that promotes their values and interests (Pelz 1978; Fischer 1995; Beyer 1997; Sabatier 1998; Olsson and Andersson 2007). They may try to manipulate research in order to achieve such results, or ignore research results that conflict with their interests (Croke et al. 2007).

The third category of factors is related primarily to the cooperation between researchers and stakeholders involved in the policy process. An important precondition for collaborative learning is that the researchers and other stakeholders are sufficiently motivated and are equipped to collaborate and learn from each other (Slavin 1996; Mostert et al. 2007). Moreover, mutual understanding and trust are necessary to be able to deal with mutual differences (Mostert and Raadgever 2008). These factors may be the result from past experiences, but to some extent they may also develop during the collaborative process. Intermediaries may play a positive role in this respect, such as researchers intervening in policy settings and organizations bridging the boundary between science and policy, such as governmental research institutes (Guston 2001; Cash et al. 2003).

Furthermore, learning is more likely when researchers and other stakeholders have an equal input in and influence on the process (cf. Habermas 1984; Dillenbourg et al. 1996; Muro and Jeffrey 2008). Ideally, researchers and policymakers should jointly decide on the purpose of the research, the questions to be studied, the methods to be used, the expected results, and the form in which they will be presented (Busenberg 1999; Olsson and Andersson 2007). They should also be flexible to modify the research process, the policy process and the collaboration process when needed (Lavis et al. 2003; Olsson and Andersson 2007). When researchers drive the process, other stakeholders may be less motivated to collaborate and learn (cf. Collins and Ison 2010). Also hierarchical relations between the involved actors may form a barrier for learning (Dimadama and Zikos 2010).

In addition, the intensity of the collaborative process is important. Learning is more likely in long-term collaborative processes; when the number of stakeholders involved is low; when many meetings are organized of sufficient length; and when methods are used that allow for intensive interaction and exchange of knowledge and perspectives (Saarikoski 2000; Croke et al. 2007; Mostert et al. 2007; Muro and Jeffrey 2008). Simple, interactive and transparent communication tools may support the discussion and integration of perspectives (van Boxtel et al. 2000; Borowski and Hare 2006; Olsson and Andersson 2007). Presentation of research results should be attractive and clear and tailored to the needs and possibilities of the intended users (Lavis et al. 2003), using for instance appealing metaphors and visuals (Dahinden et al. 2000). Critical reflection by researchers and other stakeholders on the assumptions and uncertainties may be crucial for developing a sense of the reliability of the research and required improvements (Lavis et al. 2003; Brugnach et al. 2007; Croke et al. 2007; Olsson and Andersson 2007).

The fourth and final category of factors influencing learning are external factors. Examples include natural disasters and media coverage.

3 Case Study Methodology

3.1 Measuring Cognitive Learning

Although the number of case studies that assess learning from collaboration is growing (e.g., Daniels and Walker 1996; Saarikoski 2000; Steyaert and Jiggins 2007), the methods used have important limitations. Learning is usually assessed using only indirect measures, such as participants’ perceptions of their own learning, process observations, or a combination of both. However, learning is mostly diffuse, indirect and intuitive. Individuals are therefore only partly aware of their own learning and the factors that influence this (Weiss 1977; Beratan 2007) and participants’ perceptions of their own learning may be incomplete and biased and lack detail (cf. Innes and Booher 1999). Process observations can only capture fragments of perspectives and changes therein.

To improve insight into learning in water management, we conducted two case studies in which we measured cognitive learning – our dependent variable – more directly. We used five indicators for cognitive learning (Table 1). Three of these – changes in perspectives (CP), learning from research results (LR), and development of consensus (DC) – were assessed in a semi-quantitative way, using Q methodology (Stephenson 1953; Brown 1980; McKeown and Thomas 1988; Durning and Brown 2007). The other two – stated learning from each other (SLP) and from research results (SLR) – were assessed using evaluations forms and interviews.

Table 1 Indicators of the dependent variable (cognitive learning)
Full size table

Q methodology involved the preparation of a set of statements about (expected) problems, management strategies and the goals to be achieved, using literature study and interviews. Next, stakeholders involved in collaborative research were asked to rank these statements according to how much they agreed or disagreed with each statement (see Raadgever et al. 2008). The resulting “Q sorts” represent the perspectives of individual actors at a certain moment in time. To assess changes, we asked the stakeholders to conduct the Q sorting both before and after the collaborative process (cf. Pelletier et al. 1999; Niemeyer 2004). Most performed the ex ante Q sorting before the first collaborative meeting, but stakeholders who became involved later completed their ex ante Q sort later. After the process, we asked the respondents to repeat the Q sorting. In addition, we asked stakeholders who had not participated in the collaborative research to perform the repeated Q sorting. They functioned as a control group. For the control group, we invited professionals and representatives of organized interest groups with an interest in the issue at stake. In total, we received 27 complete responses in the Lower Rhine case and 14 in the Delft case.

It is common in Q methodology to use factor analysis to identify shared perspectives that correlate with multiple individual Q sorts (as in Raadgever et al. 2008). However, in this paper we are interested primarily in the individual perspectives themselves and the changes therein because many of the factors that may influence cognitive learning are related to individuals. To assess the extent to which each individual’s perspective changed (CP), we used common statistical analysis (for more details: Online Resource 1). In addition, we assessed whether the respondents’ perspectives changed in the direction of the model results that were presented during the collaborative process (LR). For this purpose, we focused on changes in perspectives related to a small number of Q sorting statements that were either clearly supported or clearly rejected by the model results. Furthermore, we assessed whether cognitive learning of individual stakeholders resulted in an increased correlation between the perspectives of multiple stakeholders, as a measure of development of consensus (DC).

3.2 Explaining Cognitive Learning

In order to explore the conditions under which cognitive learning is more or less likely, we first carefully assessed the factors that may influence cognitive learning according to literature (the independent variables). For this we used Q sorts concerning specific statements, workshop evaluations, interviews and qualitative process descriptions. We analyzed the participants’ Q sorts concerning specific statements to learn about their technical knowledge, values and interests. The workshop evaluations, filled in by the participants, were not only used to evaluate the workshops, but also for eliciting participants’ perceptions of the research results and how they were presented. Ex ante interviews helped us to understand the issues at stake, their urgency and the stakeholders involved. The interviews after the process helped to gain insight into the relative influence of the different factors on cognitive learning, including the influence of processes that we could not observe. By combining the results of the formal analysis with personal observations and information from informers, we developed a qualitative description of the processes at hand. Table 2 gives an overview of which methods were used to assess which factors (for more information: Raadgever 2009).

Table 2 Independent variables explaining cognitive learning from collaborative research
Full size table

Secondly, we analyzed the relation between the dependent and independent variables in an exploratory and qualitative way. The limited number of case studies and the large number of factors did not allow for statistical analysis of the relation between the dependent and independent variables.

4 Case Studies

The two cases studied were future flood management in the Lower Rhine basin and groundwater management in Delft. Whereas the Rhine case was an international one and involved the development of a long-term vision, the Delft case was more local and concerned a more pressing issue. The Rhine case was conducted first, and the Delft case was used to verify and enrich the insights obtained in the first case. The main events in the cases and our involvement have been summarized in Table 3.

Table 3 Main events in the collaborative research process in each case
Full size table

4.1 Case 1: Future Flood Management in the Lower Rhine Basin

In 1997, a German-Dutch Working Group on Flood Management (WGFM) had been set up for the Lower Rhine basin. In this group, policymakers from several governmental levels in The Netherlands and the German federal state of Northrhine-Westphalia exchanged information and executed joint projects. Moreover, in September 2005 a research project started, the ACER project, which aimed to develop an integrated Rhine model for quantifying the effects of long term autonomous developments and management strategies on the frequency and magnitude of floods and droughts (http://ivm5.ivm.vu.nl/adaptation/project/acer). It was recognized that ACER could potentially support the work of the WGFM and cooperation developed.

In 2005, we got into contact with both the WGFM and ACER to find out whether the Lower Rhine could be a case study in our research. In the end, it was decided to organize and analyze three collaborative scenario workshops. The workshops took place between September 2006 and April 2008 and were facilitated by professional facilitators involved in the ACER project. The WGFM co-organized the first workshop, but decided that the following workshops would not be official WGFM workshops. Still, individual WGFM members could and did participate in them. The less official character of the second and third workshop made it easier to involve non-WGFM members, such as NGOs, consultants and policymakers from upstream German federal states.

The main goals of the workshops were to jointly develop scenarios, management strategies and success criteria for future flood management in the Lower Rhine basin until 2050. As the issue at stake was a long term issue, the perceived urgency was limited. The scenarios, strategies and success criteria developed at the first and second workshop provided input for the rainfall-runoff model that was being developed in the ACER project. The preliminary results of this model were then presented at the third workshop. The results suggested that climate change will increase peak discharges, but that flooding in Germany will strongly decrease discharges downstream. Retention measures could reduce discharges even further.

4.2 Case 2: Groundwater Management in and Around Delft

The second case was an already ongoing collaborative research process. The trigger for the process was the announcement of the industrial company DSM that they intended to substantially reduce groundwater abstraction in the city of Delft (the Netherlands). In response, the municipality of Delft ordered a consortium of research institutes and consultants to perform a qualitative Quickscan study of the expected effects and possible management strategies. Based on the results of the Quickscan, policymakers from three local and regional authorities and researchers from several research institutes jointly started a quantitative follow-up study. The results were urgently needed to support decision-making.

The three authorities formed the steering group of the research project. The steering group met every few weeks with the project leader from the leading research institute to discuss progress, direct the research, and prepare communications with their political superiors, other governments, DSM and NGOs. We attended all steering group meetings that took place between January 2007 and December 2008. In addition, we attended three workshops that were organized in this period in order to involve a broader range of policymakers and NGOs. The main goals of the workshops were to present (preliminary) research results, to get feedback on the results, and to jointly develop management strategies.

In the research a broad range of models was used to assess the effects of reducing the abstraction and the costs and benefits of alternative management strategies. The main finding was that abstraction cannot be substantially reduced without problems. When abstraction would be reduced, groundwater levels would rise, causing flooding of cellars. Moreover, levees might become unstable and the ground level would change to different degrees, causing damage to buildings. Flooding could be prevented by constructing drains. In addition, intensive monitoring would be required to signal the effects on levees and buildings before damage would occur.

5 Results and Analysis

5.1 Cognitive Learning

Table 4 gives an overview of the cognitive learning that occurred in the two cases. In the Lower Rhine case, we distinguish between 16 respondents who attended at least one workshop (the participants) and 11 respondents who were not connected to the scenario study at all (the control group). In the Delft case, we distinguish between three groups: 1) three respondents who attended the steering group meetings, 2) nine respondents who attended only one or more workshops and 3) two respondents who did not attend any of the meetings.

Table 4 Cognitive learning by specific groups of respondents (group average values)
Full size table

The results confirm that stakeholder perspectives on environmental management issues change over time (cf. Pelletier et al. 1999; Driessen and Glasbergen 2002). The perspectives of 29 of the 41 respondents in total changed with statistical significance (CP, one-tailed z-test with p < 0.025, see Online Resource 1). A possible explanation for the non-significant change in the perspectives of the other respondents are the relatively short period of time over which change was assessed, varying from one to 20 months. Table 4 shows that in the Delft case the level of change was on average higher than in the Lower Rhine case, but stated learning from others’ perspectives (SLP) was lower. The perspectives of the workshop participants changed most, followed by the perspectives of the steering group members and the non-participants. In the Lower Rhine case, the level of change among participants and non-participants was similar.

Analysis of the topics of change revealed that the respondents changed their perspectives about water management problems, strategies and goals. Yet, the average level of learning from the model results (LR) was limited. The members of the steering group in the Delft case learned most from the research results. The level of learning from the research results as stated by the participants (SLR) was also highest in the Delft case, both among steering group members and workshop participants.

Table 4 also shows a significant increase in consensus among the members of the steering group in the Delft case (DC, two-tailed t-test with p < 0.05). In the Lower Rhine case consensus between the participants increased as well, but not significantly. Consensus between the workshop participants in the Delft case and the non-participants in the Lower Rhine case actually decreased significantly.

5.2 Factors Influencing Cognitive Learning

Because of the limited number of cases – two – and the large number of factors, we can only give a qualitative explanation of the cognitive learning that occurred. We will first try to explain the differences between the two cases and then the differences between groups and individuals within the cases.

5.2.1 Explaining Differences Between the Cases

A major difference between the two cases is that in the Delft case perspectives changed more than in the Lower Rhine case. This may be explained, first, by the fact that the issue at stake emerged more recently. Consequently, the stakeholders had less time to develop their perspectives before the process commenced and may have been more open to learning. Second, in the Delft case, research was expected to have a direct impact on decision-making. This may have increased the motivation to learn, but it may also have politicized the process. Some of the stakeholders involved in the workshops considered their influence to be small and were afraid that the steering group would use the research strategically to support their own position in the political debate. This appears to have limited open exchange and the motivation to learn. A comparable complication was present in the Lower Rhine case, where some of the policymakers involved feared that the collaborative research process would upset transboundary politics.

Another difference between the two cases is that the participants in the collaborative research in the Delft case reported higher levels of learning from the model results than in the Lower Rhine case. This can be related to the perceived urgency of the issues. Secondly, the research in the Delft case had a more applied nature and was performed by applied research institutes, whereas the research in the Lower Rhine case was more fundamental and was performed mostly by academics. This seems to have resulted in a better adjustment to the demand for knowledge in the Delft case than in the Lower Rhine case. Thirdly, recent experiences with collaboration between the steering group members and researchers in the Delft case appears to have increased mutual trust and understanding, as well as the motivation and capability to collaborate and learn. In contrast, collaboration in the Lower Rhine case had to be built up from scratch. The policymakers were not used to participating in processes that were initiated and funded by researchers. Developing the required understanding and trust for such a process appeared difficult, even though professional facilitators and a governmental research institute acted as intermediaries.

5.2.2 Explaining Differences Between Groups Within the Cases

The results in Table 4 suggest that the extent of change in perspectives cannot clearly be related to the intensity of participation in the collaborative research process, but the direction of change can. The members of the steering group in the Delft case learned most from the research results and developed most consensus. This can be explained by the strong willingness and ability of the steering group members to collaborate and learn from the research and the subsequent strong involvement in the research and intensive exchange of perspectives. During the regular steering group meetings, the researchers presented preliminary model results, the policymakers gave feedback, and both were flexible enough to adjust the collaborative research where needed. The authorities in the steering group contributed equally to financing the collaborative research and had an equal say in the steering group meetings. The workshop participants in the Delft case, on the other hand, had much less opportunity for active involvement in, and influence on, the research process. This can explain that they did not significantly learn from the model results.

The increase in consensus between the steering group members appears to be related to their mutual trust and understanding. The decrease in consensus among workshop participants appears to be related to the limited opportunity for discussing perspectives during the workshops. Workshop observations showed that the workshops were dominated by researchers and most other participants contributed little. Furthermore, the politicization of the process appears to have limited the development of consensus.

In the Lower Rhine case, it is quite surprising that the perspectives in the control group changed more in the direction of the model results than those of the participants. The participants also stated in interviews and evaluations to have learned little from the model results. This may be explained by the fact that during the workshops limited attention was paid to the research results and by the limited novelty of the results. Many members of the control group were researchers, who stated to have learned from other research, conferences, media attention and informal discussions. The average correlation between the perspectives of the workshop participants did increase, but not significantly. Yet, compared to the decrease in consensus in the control group, the insignificant increase among participants indicates that attending workshops may enhance the development of consensus to some degree.

5.2.3 Explaining Differences Between Individuals

Finally, some remarkable differences in learning occurred between individual participants. In both cases, cognitive learning appeared to be biased by the participants’ individual interests. Not surprisingly, many learned predominantly from knowledge that was relevant for their own tasks and responsibilities. Furthermore, according to the final interviews, prior technical knowledge of the participants influenced their learning in two opposite ways: it helped them to understand the model results, but it also made them more critical about the modeling. The workshop evaluations revealed that most participants thought the model results were of good quality and presented in a clear and attractive way. Only two participants considered the research and one the presentations to be of low quality. These participants learned little from the model results and their perspectives actually changed in the opposite direction.

5.3 Analysis

Our empirical findings confirm that stakeholders learn about water management issues on relatively short time scales (one to 20 months). Moreover, they suggest that stakeholder learn (CP) more when the issue at stake emerged only recently and is considered to be urgent. Yet, especially in the case of urgent issues, political sensitivities may form a barrier for collaboration and learning. Learning from model results (LR) appears to be strongly related to the motivation and capability to collaborate and learn. These in turn appear to be positively correlated with the involvement of applied researchers (as opposed to academics), a strong urgency of the issue at stake, research results that fill in perceived knowledge gaps, and mutual trust and understanding between the researchers and other stakeholders involved. Moreover, learning from research results occurs only when collaboration between researchers and other stakeholders is intensive and when the participants have an equal contribution to and influence on the research process. Intensive collaboration offers far more opportunities for developing mutual trust and understanding, tailoring the presentation of research results to the needs and background knowledge of the stakeholders, for reflection on assumptions, methods and uncertainties, and for incorporating changes.

The development of consensus (DC) appears to be mainly related to the trust and understanding between the stakeholders involved. Differences in values and interests and misunderstandings may limit the development of consensus. Intensive discussion of these differences and developing mutual understanding appears to reduce these limitations and promote the development of consensus. Such discussions can take place in the collaborative research process and/or in parallel, more politically oriented processes.

6 Discussion

Our study shows that repeated Q sorting in combination with common statistical analysis can be used for assessing individual cognitive learning. The applied methodology is a valuable improvement on earlier attempts to assess learning, as it measures learning in a more objective and direct way. In addition, the methodology can be used for assessing factors that influence learning, both at the individual level and at the level of groups.

The methodology used did not allow for assessing changes in technical knowledge in much detail, of for assessing increased understanding of others’ perspectives that did not result in changes in one’s own perspective on the issue at stake. This may explain why the participants reported more learning than was visible in the results of the Q sorts. Furthermore, the final interviews revealed that a Q sort depends on the interpretation of the Q sorting statements and on the emphasis that the individual wants to make at the time of sorting. In addition, respondents may start to sort the statements less accurately when the sorting takes more time than expected. To prevent these problems, the number of Q sorting statements should be kept low (maximally 40). Moreover, interviews can be held with the respondents to complement and help interpret the results of the Q sort and filter out measuring errors.

By applying a broad range of methods in addition to Q sorting, we were able to assess the independent variables related to the research process, the policy process, the collaboration process and external factors in much detail. We could also obtain information at the individual level, such as the respondents’ technical knowledge, values and interests, and their perceptions of the research.

That being said, our research still has an exploratory character. Major sources of uncertainty are the large number of factors that may influence cognitive learning, the limited number of cases, the limited number of respondents in some of the analyzed groups, and the unobservable processes in-between the collaborative meetings. For explaining cognitive learning, we still had to rely partly on statements from the participants that we interviewed after the process, which is problematic because cognitive learning is largely an unconscious process. Consequently, we could not establish unambiguous causal relations between cognitive learning and influential factors, but only “grounded speculations”. To address this issue, we recommend additional case studies in which many potential influential factors are kept constant, for example by assessing learning from a single collaborative event instead of learning over the course of years. Additional case studies can also increase the range of contexts covered. This is important because the Delft case and the Lower Rhine case have already shown that in different contexts different factors may promote or limit learning.

A specific limitation to our findings is the limited number of Q sorting responses from the researchers involved. The final interviews revealed that many researchers aimed to produce knowledge, and not so much to learn from other stakeholders. Considering this role, they perceived it as unnecessary to fill out the repeated Q sorting. This suggests a limited willingness to learn from the other stakeholders. Yet, several researchers stated that on specific topics they did learn from the other stakeholders.

In the case studies, we focused on cognitive learning and collaboration. It is not possible to generalize the conclusions to other forms of learning, such as social learning, which includes the development of new capabilities and new collective behavior (Mostert et al. 2007; Muro and Jeffrey 2008) or policy learning, which includes changes in institutions such as policy, law and organizations (Bennett and Howlett 1992; Sanderson 2002). Yet, individual perspectives are also important in these more encompassing forms of learning. Moreover, collaboration plays a very big role in these forms. Hence, while not proven in any sense, the conclusions of the analysis are certainly of relevance for social learning and policy learning.

7 Conclusions and Recommendations

The complexity of water management requires continuous cognitive learning about the issues at stake. Our cases suggest that such learning about water management problems, goals and measures is quite common. About 70 % of the analyzed stakeholder perspectives changed significantly over a period of one up to 20 months. Most learning occurred in the case where the issue at stake emerged only recently and required a quick solution.

According to scientific literature, collaborative research – research in which researchers and other stakeholders participate - may promote both learning from model results and the development of consensus. Our findings confirm that collaborative research enhances such learning, but only under specific conditions. In contrast to some of the literature, we found that only intensive collaboration in which all participants have an equal contribution to and influence on the research, significantly enhances the uptake of model results and the development of consensus (cf. Saarikoski 2000).

We also found that there may be many obstacles for collaborative research and learning that are beyond the direct control of the stakeholders involved. Collaborative learning appears only feasible when all involved stakeholders are highly motivated to collaborate and learn (cf. Croke et al. 2007) and willing to devote all the necessary time and other resources to the collaborative process (cf. Huxham and Vangen 2005). Yet, individual stakeholders have little influence on the willingness to collaborate and learn of others. Developing enthusiasm is particularly difficult when the issue at stake is not very urgent. Moreover, developing mutual trust and understanding may take a long time, in particular when collaborative relations have to be built up from scratch. In addition, political motives may prevent open discussion and limit cognitive learning.

Based on these findings, we recommend to organize collaborative research only when all involved stakeholders are highly motivated to learn from each other about the issue at stake. Hence, first a proper stakeholder analysis should be conducted. All stakeholders should perceive that their participation will be an opportunity for them to fulfill some of their own interests and that they can benefit from participation (e.g., Ridder et al. 2005). In order to keep the participants motivated throughout the process, we recommend that all stakeholders involved should have equal input in and influence on the research process. If these conditions can be met, we recommend to organize an intensive collaborative research process, including many meetings and intensive discussion of perspectives.