Introduction

Transdisciplinary (TD) research is often promoted as the key to solving complex societal problems, and is typically applied when normative problems arise in which valuation and decision-making are major issues. Consequently, TD approaches have increasingly been promoted in the field of land-use research. However, in a comprehensive literature review, Zscheischler and Rogga (2015) found that the theoretical discourse about transdisciplinarity in land-use science is far more advanced than its implementation in practice, as Brandt et al. (2013) also noted for the TD research in sustainability science. No consistent definition of transdisciplinarity is, though, available even in the theoretical literature, but three characteristics of TD research can be identified: (1) the research seeks to contribute to solving a problem in practice; (2) researchers from different disciplines and practitioners collaborate; and (3) the methodology is developed during the research process and in a problem-specific way (Wickson et al. 2006; Lang et al. 2012; Zscheischler and Rogga 2015). This definition is unfortunately limited as the focus is only on the process with little consideration of the results or effects of this kind of research (see e.g. Klein 2010; Lang et al. 2012). Moreover, the TD literature takes a very science-oriented perspective with little mention of the views of practitioners (Polk 2014; Zscheischler and Rogga 2015).

As large environmental research programmes require in their funding schemes the integration of different scientific disciplines or practitioners, an increasing body of literature has emphasised the evaluation of the procedures and the effects of joint knowledge production between scientists of different disciplines and/or between scientists and practitioners (e.g. Rossi et al. 2003; Stokols et al. 2003; Bergmann et al. 2005; Enengel et al. 2012; De Jong et al. 2016). A number of challenges come along with these types of cooperation and their evaluation. For a start, participants from different backgrounds tend to have different expectations for the joint project and, consequently, follow different objectives, use different criteria to assess the project’s success, and are interested in different results and products (Klein 2008; Edelenbos et al. 2011). As a consequence, researchers tend to collaborate with colleagues from similar disciplines, even if they are supposed to work in interdisciplinary team-science projects (Stokols et al. 2008). Another important challenge is the fact that both social and cognitive factors influence the collaboration. It has been found that intensive communication between the project participants and opportunities for informal exchange support mutual understanding. Having a variety of products and involving all parties in writing the project proposal from the beginning enhance the chances that a collaborative project will have a satisfactory outcome for all participants (Klein 2008; Gaziulusoy et al. 2016; De Jong et al. 2016; Hessels et al. 2014). Several studies (e.g. Blackstock and Carter 2007; Talwar et al. 2011; Lang et al. 2012) have noted that the practitioners are often insufficiently interested, committed, or willing to participate in TD projects, due to or resulting in unbalanced problem ownership. TD research has, thus, as a prerequisite to ensure good team building and communicating, so that a feeling of joint problem ownership can be established or sustained among the project participants.

What specific kind of knowledge can be created in TD research projects? Summarizing the results of different evaluation studies of TD projects, it can be concluded that the most important impact is in influencing the project participants’ ways of thinking. Here, we use Fleck’s (1979) concept of thought-styles (Cohen and Schnelle 1986; Pohl 2011), which says that each scientific discipline and each field of policy and practice has developed its particular culture of thinking, i.e. its specific thought-style, according to specific norms, values and attitudes. We adopt this concept for the analysis of our TD process because it explains a variety of potentials and challenges in communication and cooperation between experts from different working fields.

Bringing together different thought-styles and having them interact supports societal debate and reflection and enhances the participants’ capacity for decision-making (Walter et al. 2007; Pohl 2011; Wiek et al. 2014; Polk 2014; Hegger and Dieperink 2015; Luederitz et al. 2016). Capacity building in decision-making occurs in TD research according to Pohl (2011) because the participants collectively enhance their own thought-styles when interacting with different thought-styles, by broadening or changing their understanding of an issue and developing new ways of handling the issue in their own real worlds. Such increases in decision-making capacity are not necessarily an immediate consequence of participatory events, but are induced by a number of intermediate effects, since the exchange of different thought-styles can influence the knowledge, attitudes and behaviour of the participants (Walter et al. 2007; Wiek et al. 2014; Hegger and Dieperink 2015). The authors mention examples such as ‘generating attention and enthusiasm’, ‘community identification’, and ‘trust in others’. These intermediate effects are also seen as merits, added values or social impacts of TD processes (ibid.).

In this paper, we assessed the effects of a TD research programme on the participants’ ways of thinking. The research programme had produced scenarios and steering approaches for the future of urban and landscape development in four case study regions in Switzerland. This empirical study is intended to help fill the gap in research on the interaction of different thought-styles in TD projects. We do not present a comprehensive evaluation of all aspects of a TD research process, but rather focus on the participants’ evaluations of how the TD process in this research programme influenced their ways of thinking. Our TD research followed four objectives derived of from Pohl (2011), Lang et al. (2012) and Polk (2014):

  1. 1.

    To generate a feeling of joint problem ownership among the project participants.

  2. 2.

    To facilitate the interaction between stakeholders with different thought-styles, i.e. between researchers of different disciplines, or practitioners with different professional backgrounds, as well as between scientists and practitioners.

  3. 3.

    To enable the project participants to link abstract (scientific) with case-specific (practical) knowledge.

  4. 4.

    To encourage the project participants to incorporate the shared knowledge in their real-world situations.

The evaluation included: (1) checking how far our objectives for the TD research had been fulfilled; (2) assessing the intermediate effects of the TD process on the participants’ ways of thinking; and (3) identifying obstacles and challenges in the TD process. We chose a combination of quantitative and qualitative approaches for the evaluation at the end of the research programme using standardised questionnaires on the one hand, and structured and open discussions as well as our own observations on the other hand.

Materials and methods

Objectives and structure of the research programme

The research programme ‘Room for People and Nature’ of the Swiss Federal Research Institute WSL ran from 2011 to 2016 and focused on the future landscape development in regions undergoing strong urbanisation pressure. Four rural regions in the sphere of influence of the city-region of Zurich (Switzerland) were selected for the case studies. The regions struggle with the problem of how to develop their villages while safeguarding specific landscape features of the regions, which are the basis for the regions’ high quality of life. The core objectives of the research programme were to create and evaluate scenarios for the future development of the landscapes in the case study regions, and to derive steering approaches for sustainable landscape development there.

The programme consisted of four natural and social science research projects and the synthesis project, in which the findings of the research projects were compiled and interpreted in the context of the case study regions. The approach was only transdisciplinary in the synthesis project, which was co-designed by researchers and practitioners. The scientists selected the main research questions of the other four projects, but the practitioners gave specific inputs to make the projects relevant for the case study regions. This structure enabled the scientists in the individual projects to fulfil the requirements of scientific rigour in their specific communities, and at the same time to cooperate effectively in the TD parts of the research programme.

Participants and their involvement in TD cooperation

Three groups participated in the study: 19 scientists, 10 funding partners (practitioners) and approx. 35 local stakeholders in the case study regions. All the scientists were from the Federal Research Institute WSL but with different research fields: landscape ecology, biodiversity, social sciences of human–landscape relations, and governance studies. The programme leader and two scientists experienced in TD research were responsible for the programme synthesis and facilitated the TD process. The other scientists took part in the TD process as participants of workshops and meetings.

The funding partners were representatives of cantonal and national spatial planning and environment protection authorities. The case study regions were located in the cantons, which provided funding. The funding partners were involved in the programme design, implementation and evaluation. TD collaboration between the funding partners and the scientists was mainly in managing aspects of the programme.

The local stakeholders, who participated in each case study region included local politicians, such as mayors or municipal council members as well as representatives of NGOs, such as nature, landscape or bird protection associations. They were mostly key figures in local politics concerned with urban and landscape development. The municipal politicians were responsible for the zoning and development plans of their municipalities, while the NGOs were particularly interested in contributing to good management of the open landscape. The local stakeholders participated in two workshops in each case study region, and the drop-out rate was surprisingly small. In the first workshop series, they developed a common vision about the future development of the landscape in their specific region. To support the workshops, real-time visualisations were produced (Tobias et al. 2016a; see also Fig. 1). In the second workshop series, the stakeholders developed steering approaches for sustainable landscape development. Some of the funding partners also participated in the workshops with the local stakeholders and thus experienced an additional TD element of the research programme.

Fig. 1
figure 1

Examples of the visuals showing visions of future landscape development in the case study regions. The visuals were created during the workshops by scientific illustrators guided by the local stakeholders

Full size image

TD work was carried out at three levels adapted from Stauffacher et al. (2008) and Schneider and Buser (2018): (1) information, where the researchers informed the practitioners; (2) consultation, where the researchers obtained information from the practitioners and vice-versa; and (3) coproduction, where the researchers and the practitioners or different practitioners together developed new knowledge (Fig. 2).

Fig. 2
figure 2

The TD elements of the research programme assigned to the three levels of TD cooperation. The box colours indicate which agents were mainly involved in the events, with two groups represented in each case. All events were organised by the scientists responsible for the research programme synthesis

Full size image

The TD events can be divided into the typical phases of a TD process according to Lang et al. (2012): (1) problem framing and team building; (2) co-creation of solution-oriented transferable knowledge; and (3) (re-)integration and application of the created knowledge. The researchers’ and the funding partners’ joint elaboration of the programme’s objectives and research questions was the key element in problem framing and team building (Fig. 2). The explorative, guided face-to-face, interviews in the case study regions were the basis for the region-specific problem framing and team building between the programme leader and key local stakeholders.

Typical elements of the phase of co-creation of knowledge were the workshops in the study regions. At these workshops, the three scientists responsible for the programme synthesis facilitated the reflection- and decision-making process for the local stakeholders and provided information from the research projects. The local stakeholders actually generated new knowledge together, taking into account research findings.

The process of co-creating knowledge between the researchers and the funding partners was less direct because the researchers worked on their research projects separately and informed the funding partners about their progress. The funding partners provided the researchers with information about the features of the case study regions and the needs of the local stakeholders. Knowledge exchange between these groups of participants took place during field trips to the case study regions, the yearly meetings, the presentation of the visuals, and the conference on the results of the individual research projects (Fig. 2). The field trips were rather unusual settings for the meetings of the funding partners and the researchers. We organised bus trips through the case study regions with stops at certain places where selected local stakeholders informed the researchers and funding partners about the urban and landscape development in their regions. After the bus trips, the official meetings between the scientists and the funding partners took place in meeting rooms at local restaurants in the case study regions. The field trips offered the opportunity for a more informal exchange between the researchers and the funding partners, and for them to have direct contact with some local stakeholders.

The events of the final phase of integration and application of the created knowledge were the evaluation of the research programme by the funding partners and researchers, and the information events in the case study regions for the local stakeholders. In this phase, the scientists and funding partners also jointly produced a leaflet (Tobias et al. 2016b), which synthesized the most important issues for practice based on the findings from the research projects and the knowledge generated at the workshops with the local stakeholders. Further products of the research programme were a poster visualising the key points of the leaflet, a synthesis report addressing the funding partners and other practitioners (Tobias 2015), six articles in magazines for practitioners, and eight scientific papers for the different scientific communities.

Evaluation procedure for the TD process

We asked each group of participants (scientists, funding partners and local stakeholders) to evaluate the TD process of the research programme at two different times: the local stakeholders filled in a standardised questionnaire immediately after the second workshop in their specific case study regions, while the researchers and funding partners completed an online questionnaire after the last joint meeting.

The qualitative evaluation was conducted with the researchers and funding partners at the last joint meeting using a simplified version of the ‘most significant Change’ (MSC) method according to Davies and Dart (2005). In the MSC method, the participants tell stories about moments, actions or processes that induced a significant change in their way of thinking or doing, and a panel of selected stakeholders or staff selects the most significant of these change stories. At our evaluation workshop, we asked the researchers and funding partners to write down on a sheet of paper short sentences or key words about what they personally experienced as most important steps or moments that advanced or influenced the research programme significantly. Afterwards, the participants exchanged their experiences in small groups of three or four people, including both researchers and funding partners. In the subsequent plenary discussion, the groups exchanged their consolidated experiences. The plenary discussion was tape-recorded and transcribed. We analysed the discussion transcript and the participants’ notes in a qualitative content analysis (Schreier 2013; Mayring 2014), and compared our results to findings from the literature. The MSC method was used to obtain information about intermediate effects and challenges of this TD research on the participants’ ways of thinking. Our emphasis was on the events and the levels of TD cooperation shown in Fig. 2. Moreover, we looked for evidence about the achievement of the objectives of TD research described in the introduction. Our own observations were also fed into the evaluation.

The questionnaires were completed by 26 local stakeholders, but not all respondents answered every question. 12 researchers and 4 funding partners participated in the online survey. For the MSC method, 7 researchers and 5 funding partners were present. As the samples were so small, we did not perform detailed statistical analyses and only ascertained the distributions of the answers to the statements in the questionnaires.

Evaluation framework

From the literature, we derived four objectives for our TD research (cf. “Introduction” and Table 1) and defined specific criteria for each objective as indicators of success. These then provided the bases for the questions in the questionnaires for the programme participants (Table 1).

Table 1 Objectives of our TD research derived from the literature and the specific criteria we defined as the basis for the standardised questionnaires
Full size table

For each specific criterion, we formulated one or two statements the interviewees had to respond to on a scale of 1–5 (1 = disagree; 5 = fully agree), with separate standardised questionnaires for each group, the researchers, funding partners and local stakeholders. All questionnaires had the same structure, but the individual questions were formulated to take into account the specific perspective of each participant group, resulting in different questionnaires for each group. The single statements for each participant group are listed in the electronic supplementary material S1.

Results

Questionnaire results

Objective 1

The first objective was to generate a feeling of joint problem ownership among the project participants. In general, it seems to have been at least partially met (Fig. 3). The median values for the non-academics, i.e. the local stakeholders and the funding partners, were slightly more positive than those of the researchers, particularly if they were asked about the whole group as joint problem owners (Fig. 3b). The questionnaire also included the question whether the respondents had missed the inclusion of important stakeholder groups. Although most answers were negative, some participants mentioned representatives from agriculture, followed by representatives of industry and trade, investors and landowners, should have been included.

Fig. 3
figure 3

TD objective 1: to generate a feeling of joint problem ownership among the project participants. Research programme participants’ assessments of specific statements. Median values: funding partners 4 (a) and 4.5 (b); researchers 4 (a, b); local stakeholders 4 (a) and 5 (b). N number of respondents

Full size image

Objective 2

The second objective was to facilitate the interaction between stakeholders with different thought-styles, i.e. between researchers from different disciplines, or practitioners with different professional backgrounds, as well as between scientists and practitioners. How much respondents valued the interaction with other thought-styles differed between the participants (Fig. 4). According to the median values, the local stakeholders were those who thought they had benefited most from the exchange with other thought-styles, i.e. from discussions with local representatives of other disciplines or policy sectors (Fig. 4a). They were also the most likely to say they had developed joint products during the workshops in the case study regions (Fig. 4b). The funding partners tended to think they had developed new perceptions and ideas through exchanging with the researchers’ thought-styles (median of 4 in Fig. 4a), but were less enthusiastic about the development of joint products (median of 3 in Fig. 4b). The researchers were the most sceptical. They felt they had benefited somewhat from the exchange with the funding partners (median of 4 in Fig. 4a), but they were critical about the development of joint products (median of 2.5 in Fig. 4b).

Fig. 4
figure 4

TD objective 2: to facilitate the interaction between stakeholders with different thought-styles. Research programme participants’ assessments of specific statements. Median values: funding partners 4 (a) and 3 (b); researchers 4 (a) and 2.5 (b); local stakeholders 4.5 (a) and 4 (b). N number of respondents

Full size image

Objective 3

The third objective was to enable the project participants to link abstract (scientific) with case-specific (practical) knowledge. The responses to the questions concerning this objective were rather evenly distributed across the groups (Fig. 5). This indicates that the results depend more on the individual perception of each participant than on differences between the participant groups. The results for the funding partners may suggest more consensus (Fig. 5b), but clustered around the indifferent level. Only four of the funding partners, however, answered the questionnaire. According to the median values, the local stakeholders agreed most with both statements, whereas the researchers were as indifferent as the funding partners.

Fig. 5
figure 5

TD objective 3: to enable the project participants to link abstract (scientific) with case-specific (practical) knowledge. Research programme participants’ assessments of specific statements. Median values: funding partners 3.5 (a) and 3 (b); researchers 3.5 (a) and 3 (b); local stakeholders 4 (a, b). N number of respondents

Full size image

Objective 4

The fourth objective was to encourage the project participants to incorporate the shared knowledge in their real-world situations. The local stakeholders were far more positive than the other two groups in terms of their motivation to disseminate the new knowledge in their working environment (Fig. 6a), and in terms of ideas considering new cooperations (Fig. 6c), but they were largely indifferent about introducing new approaches at work (Fig. 6b). Conversely, the researchers and the funding partners were more positive about obtaining ideas for new approaches or activities in their working environments (Fig. 6b) than about feeling motivated to disseminate the knowledge or form new cooperations.

Fig. 6
figure 6

TD objective 4: to encourage the project participants to incorporate the shared knowledge in their real world situations. Research programme participants’ assessments of specific statements. Median values: funding partners 3.5 (a), 4 (b) and 2.5 (c); researchers 3 (a), 4 (b) and 3 (c); local stakeholders 4 (a), 3 (b) and 4 (c). N number of respondents

Full size image

The respondents who tended to agree with the statement in Fig. 6c were additionally asked to indicate with whom they would like to establish a new or intensified cooperation. The local stakeholders mentioned NGOs such as nature protection organisations, local trade associations or local initiatives for sustainable regional development. They further listed regional governmental organisations such as commissions for agriculture, regional planning, and landscape protection. The funding partners mentioned authorities within their own or other political departments like the cantonal transport office. The researchers thought of looking for more scientific contacts with other national and international universities, and for more contacts with the authorities responsible for recreation infrastructure.

Summary of the notes and discussion on the most significant changes

In both the written notes and the plenary discussion, direct exchange between research and practice in the case study regions was cited as the most valuable. The funding partners who attended the workshops with the local stakeholders in the case study regions said they personally found them the most valuable during the research programme. In the discussion, they explained that, thanks to the scientific context, these workshops provided conditions for building up mutual trust, where the local stakeholders from the different interest groups could freely discuss issues without being in the context of municipal politics. Several funding partners greatly appreciated the way the researchers managed to organise meetings where local politics played little role.

The field trips to the case study regions also provided opportunities for science–practice interaction, which all participants appreciated. Both the funding partners and the researchers were enthusiastic about the field trips and found the discussions on-site with representatives of the case study regions very fruitful. Some of the participants mentioned that the field trips were opportunities for team-building between researchers and funding partners. The participants appreciated the way the dialogue between research and practice had been implemented, and said it had fostered mutual learning and trust.

The visuals were seen as playing important role in knowledge exchange. The researchers perceived them as a kind of report about the workshops with the local stakeholders. The funding partners considered them to be both the bases for the cooperation among the local stakeholders as well as the products of the cooperation. All said the visuals served as bridging elements between research and practice. Both the researchers and the funding partners remarked in their written notes that co-creating the leaflet as a joint product was also an important experience.

The researchers and the funding partners reported in retrospect how important the framing phase had been for the research programme. They mentioned, in particular, the workshop on defining the research questions as a crucial event, albeit most of them were not really satisfied with it. The funding partners generally felt they had missed the opportunity to actively design the research questions. Some of them were of the opinion that the direct benefits of the research for practice were necessarily limited because the research topics and methods were too heterogeneous with an unbalanced mix of detailed and strategic perspectives. They thought it was difficult for the practitioners to define their research needs. The researchers, in turn, expressed their disappointment that the funding partners had provided them with too little tangible information about their specific needs.

Several participants complained that not all interest groups from practice had been present at all the meetings. The funding partners felt that the cantonal authorities had been over-represented, whereas there were too few representatives of the federal and municipal offices even though they had also contributed to financing the research programme. They interpreted this absence as lack of commitment. The researchers, on the other hand, tended to miss representatives from agriculture and the private economy. One practitioner, however, countered that he had experienced the discussions to be freer and more open than at events where the private economy was strongly represented. The researchers said they were pleasantly surprised by the strong commitment of the practitioners who participated regularly in the research programme.

Finally, at the end of the plenary discussion and in their written notes, the participants indicated some lessons learned from developing shared knowledge. They felt having found colleagues from other real worlds, what convinced them about the relevance of urban and landscape development and of their own contributions to this issue. The funding partners said the discussions and the documents (leaflet and synthesis report) provided them with substantial arguments for their own work. They encouraged the researchers to continue with this kind of landscape research in other regions. Some researchers reported that they had discovered connections between their research fields and problems in practice that they had not been aware of before, e.g. “landscape research incorporates planning”.

The statements indicate a number of intermediate effects of this TD research that induced changes in the participants’ ways of thinking. Similar changes have been reported in the literature. They are compiled in Table 2 and discussed in “Discussion”.

Table 2 Participants’ statements in the written notes and the discussion about the most significant changes (1st column), and our interpretation in terms of the intermediate effects of TD (TD) research (2nd column) compared to findings from the literature (3rd column) with the corresponding references (4th column)
Full size table

Initial effects of the research programme in the case study regions

In addition to the feedback in the written questionnaires, the local stakeholders commented informally on the workshops in the case study regions over food and drink at the end of the event. They particularly appreciated the composition of experts and the holistic way of discussing the topic of urban and landscape development. They enjoyed the joint meetings with local politicians, cantonal authorities and representatives of landscape, nature and bird protection associations. They liked being able to discuss freely outside a clear political context and the way the workshops were embedded in a scientific programme, which, they felt, made the workshop results more relevant. Some participants said they had told others at work about the new ideas they had encountered at the workshops.

After the second workshop series, two municipal mayors invited the authors to the retreat sessions of their municipal councils in two case study regions. We were asked to repeat our presentations of the workshops as an input to the subsequent discussions. With this external input, the mayors wanted to encourage their councils to adopt a more holistic and open-minded approach to the issues. After the end of the research programme, we were also invited to give a talk at a public event organised by the local trade association in a case study region.

Finally, we were surprised about the enthusiasm of the local stakeholders at the closing information events in the case study regions, which had originally been planned as a way to thank the participants for having attended the two workshops, and to promote the leaflet and the poster as the key products for practice. Almost all workshop participants made sure that they could come to the information events and some of them even brought a colleague. In two case study regions, we also invited the local press at the suggestion of some stakeholders. We were particularly surprised how motivated the participants were to spread the new ideas among other key players in the case study regions. This became apparent during the discussions, where they almost mandated one another to promote the lessons learned about landscape development among their expert colleagues.

Discussion

Achievement of TD objectives

All participant groups, the local stakeholders, the funding partners, and the researchers were rather positive about having reached the objective of achieving a feeling of joint problem ownership (Fig. 3). In the course of the regular meetings, the teams created an atmosphere of mutual trust, and the great commitment of the practitioners sometimes even surprised the researchers (cf. “Summary of the notes and discussion on the most significant changes”). We also, however, faced the challenge of unbalanced problem ownership that other researchers have found (Blackstock and Carter 2007; Talwar et al. 2011; Lang et al. 2012). The participants were more bothered by the lack of commitment or discontinuous participation of a few project participants than by the fact that not all interest groups could be invited. Making a financial commitment is obviously not sufficient to ensure participation in a TD process. The local stakeholders and the funding partners from the cantons where the case study regions are located may have developed a strong feeling of joint problem ownership because the research was on a concrete issue in the region. These regions did not, however, come directly under the responsibility of the funding partners from the national offices, which may be why the national officers only attended a few of the joint meetings. For the others, a certain feeling of joint problem ownership was probably already around at the beginning of the cooperation due to the actual research topic, and this feeling was reinforced during the TD process.

The meetings were rather successful in facilitating the interaction of different thought-styles, which helped, in particular, to induce new perceptions and ideas among the participants (Fig. 4). The findings indicate the effectiveness of the different levels of TD cooperation (Fig. 2). TD events involving the coproduction of knowledge or at least intensive consultation with two-way communication appear to be necessary to trigger the interaction of thought-styles. In the questionnaire results (Fig. 4), the local stakeholders, who had attended the workshops in the case study regions, were more positive than the other participant groups. Conversely, at the ‘most significant change’ workshop, the funding partners and researchers said that the field trips to the case study regions were what had made most impression on them. The field trips enabled the researchers and funding partners to exchange with local stakeholders from the case study regions. For the researchers, this was an interaction between science and practice. For the funding partners, it was an exchange between different governmental levels.

The most striking effect the interaction of thought-styles had for all participants seems to have been becoming aware of people in other real worlds committed to the same issues, i.e. the awareness of joint problem ownership. This made the participants look at the issue from new perspectives, as one researcher at the evaluation workshop put it: “landscape research incorporates planning” (cf. “Summary of the notes and discussion on the most significant changes”). His views shifted from a large landscape ecological perspective to include a spatial planning perspective.

Considering the third objective, the participants did not really feel enabled to link scientific and practical knowledge (Fig. 5). The local stakeholders made the most attempts to integrate the experiences they had with the scientists into their own real worlds (Fig. 5a). This became evident when, for example, they invited us to attend their municipal council retreats. Some funding partners also mentioned that the scientific knowledge documented in the leaflet and the synthesis report supported their reasoning in their practical work (“ Summary of the notes and discussion on the most significant changes”). The researchers’ responses to the questionnaire varied most. At the final workshop, some researchers said they would have liked more direct exchange with the local stakeholders.

The participants’ motivation to incorporate the shared knowledge in their real worlds (objective 4) became particularly clear in the statements and actions of the local stakeholders after the workshops (“Initial effects of the research programme in the case study regions”), and in the statements of the researchers and funding partners at the evaluation workshop (“Summary of the notes and discussion on the most significant changes”). The awareness of joint problem owners with other thought-styles encouraged the local stakeholders to work more closely together with other interest groups on their particular political tasks. Conversely, the feeling of having found colleagues from other real worlds encouraged the researchers and funding partners to develop new ideas for their work in their own real worlds.

Intermediate effects and challenges of the TD process

The biggest success factors in this research programme were the field trips to the case study regions and the workshops there because they offered opportunities to experience the real worlds of other stakeholder groups and to experience an “agora”-like context for science and practice. Participants at these events were enthusiastic and able to build up a mutual trust (Table 2).

The effects of the workshops in the case study regions were similar to those in sustainability transition experiments described by Luederitz et al. (2016). As in such experiments, the participants enhanced their decision-making capacities, particularly for incorporating the new knowledge in their own real worlds, what became evident at the final information events when they mandated one another to spread the lessons learned. The use of visuals may have additionally triggered the participants’ confidence and commitment, as participants are particularly committed in transition experiments trying novel practices (Wittmayer ànd Schäpke 2014; Luederitz et al. 2016).

Coproduction of knowledge appears to be particularly effective if joint products are generated that can be used as boundary objects by different stakeholder groups. In our case, the visuals played the role of boundary objects in two ways: (1) as boundary objects for the panels of local stakeholders at the workshops in the case study regions; and (2) as bridging elements for the researchers and funding partners with the local stakeholders. The visuals were ideal boundary objects between the different thought-styles because they met the prerequisites Carlile (2002) identified for boundary objects in organisational development: (1) enable a shared language; (2) serve as a means to learn about the differences and dependencies across the boundaries between different thought-styles; and (3) facilitate the process of conjoint knowledge transformation. Briers and Chua (2001) further noted that visionary boundary objects like joint principles or rules of the game were more efficient than joint data repositories. As our visuals showed ideas and scenarios for future landscape development, they also had the characteristics of visionary boundary objects.

In the research programme, the existence or coproduction of boundary objects, together with opportunities for generating enthusiasm and trust, fostered among the participants the feeling of joint problem ownership. Together, they developed a feeling of companionship and belonging to the research, which Walter et al. (2007) also found to be an important intermediate effect of TD research. This feeling of companionship increased the participants’ awareness of the relevance of urban and landscape development and, in turn, they acknowledged the necessary legitimacy of the results of their TD cooperation. Di Iacovo et al. (2016) made similar observations in a TD social farming project, while Lang et al. (2012) identified a lack of legitimacy of TD findings as one of the main impediments for implementing shared knowledge in the real world. Reed et al. (2014) also stress the importance of including stakeholders with decision-making powers and trusted individuals as knowledge brokers. In our case, the legitimacy of the joint products was not only supported by the positions of the participants in municipal politics or as representing national or cantonal authorities, but also by the exchange of science and practice.

The researchers and funding partners mentioned the project-framing workshop as one of the key challenges of the research programme (cf. “Summary of the notes and discussion on the most significant changes”; Table 2). We assume that, at the time when the workshop took place, both the researchers and the practitioners were unable to cope with the way of thinking of the other agent-group and experienced the workshop as kind of “cold booting”. At this time, all participants saw the problem from their own specific perspectives and were, therefore, not able to capture its complexity. Although they had the opportunity to interact with different thought-styles, they were, at this early stage of the research programme, not able to come to terms with the impressions from the other thought-styles. This is again a typical challenge in TD research, as Wiek (2007), Zeischler et al. (2014) and Di Iacovo et al. (2016) have observed in other TD projects as well. At the beginning of most TD projects, the project leaders and participants put too little emphasis on the very different obligations of scientists and practitioners. This often results in a predominance of scientific considerations (Wuelser and Pohl 2016).

The research programme’s structure may have supported an initially more researcher-dominated situation, as it consisted of several individual, not TD research projects contributing to a TD synthesis (cf. “Objectives and structure of the research programme”). The project framing workshop was actually intended not only to promote the research ideas of the individual projects to the practitioners, but also to collaboratively develop the research questions of the programme synthesis and modifications of the individual projects. This structure of the research programme was not just a challenge during the framing phase, as some researchers also complained that they had received very little information about the work in the case study regions. Therefore, the field trips to the case study regions were crucial events for the scientists to experience the context and the real worlds of the beneficiaries of their research. In addition, the researchers appreciated the visuals as bridging elements to the case study regions. On the other hand, the structure of the research programme allowed us to work parallel in four case study regions without overloading the researchers schedules, and it enabled a variety of publications. In contrast to other TD projects (Hegger and Dieperink 2015), this research programme did not result in fewer scientific publications than five individual projects would have done. Simultaneously, we could benefit from opportunities for coproduction and published more articles for practice than five single projects.

All participants were enthusiastic about the lessons they had learned for their own real worlds. The process of the discussion indicated that it was not only a matter of providing useful information but also being able to share enthusiasm and take pleasure in the learning process. Wiek et al. (2014) also observed in participatory sustainability research that enthusiasm among the project participants supported building their decision-making capacities. The motivation to incorporate the shared knowledge into their real worlds is closely connected with the feeling of joint problem ownership. The awareness that agents in other real worlds are committed to the same issue was very motivating for all participants. In addition, adopting the perspective of the other stakeholders on the joint problem enhanced the relevance of the issue and their own contributions.

Our findings confirm the generation of hybrid spaces and communities as others have found in TD projects (Polk 2014; Hegger and Dieperink 2015). The workshops with the local stakeholders in the case study regions were, in particular, appreciated as opportunities for discussion in a neutral environment removed from local politics and disciplinary science, but at the same time embedded in a practical and scientific context. The funding partners and researchers had similar experiences during the field trips to the case study regions and in the workshops for defining the research questions and evaluating the programme. The respective groups, i.e. the local stakeholders as well as the funding partners and researchers, built mixed teams, which developed new TD knowledge. These hybrid spaces were most important for the interaction of thought-styles and for trust building. As Polk (2014) points out, they were both a prerequisite and a product of the TD research. Although the participants were motivated to make use of the personal lessons they learned, these will not necessarily be directly implemented in either the urban and landscape development in the case study regions or in new research projects. The jointly created knowledge needs “translation” or transformation to make it compatible with existing institutional structures and decision-making processes in the specific scientific or practical realms (Wiek et al. 2012; Polk 2014).

Limitations

Our evaluation procedure has some serious limitations. First, the stakeholder groups differed in their sizes and were all rather small. This is why we conducted no statistical analyses and relied on combining the quantitative and qualitative data for our interpretations. Second, we conducted the evaluation immediately after the end of the workshops or the research programme to reach the maximum number of people possible. However, at this stage in the research programme only the first few effects on the participants’ real worlds had become apparent (cf. “Initial effects of the research programme in the case study regions”). At a later point in time, however, it is not necessarily easier to identify the influence of the TD process because many additional parameters can play a role in the participants’ real worlds. In addition, the participants might not be able to recall the situations, which Wiek et al. (2014) found to be a core challenge in evaluating TD projects. Given these limitations, we compared our results to findings in the literature for the interpretation (Table 2).

Conclusions

In our evaluation, we combined a quantitative approach using targeted questions in questionnaires with a qualitative approach involving a simplified version of the ‘most significant change’ method, together with our own observations. The two approaches complement each other. The questionnaire results gave an overall picture, although the number of participants was small. The qualitative evaluation revealed highlights, key challenges, and the atmosphere during the TD process. In particular, it allowed us to make recommendations for successfully organising TD processes. The exchange between different thought-styles should be in two-way communication and preferably at the TD level of coproduction. Opportunities to experience the real worlds of others, e.g. on field trips, are a good way to foster participants’ enthusiasm for the project and build up mutual trust. These qualities (enthusiasm and mutual trust) appear to strongly support the interaction of different thought-styles. In addition, boundary objects, such as our visuals, are very useful to facilitate the interaction of thought-styles. Boundary objects may be given as bridging elements between the different thought-styles or developed together as joint products.

In our case, the TD process was successful in the four aspects Belcher et al. (2016) identified for assessing the quality of TD research: (1) relevance, (2) credibility, (3) legitimacy, and (4) effectiveness. (1) The participants became aware of additional problem owners (“colleagues”) from other thought-styles and consequently were more convinced about the relevance of the issue. (2) The practitioners, in particular, found the results of the process credible because they considered the researchers to be representatives of intelligence and objectivity. (3) The interaction of different thought-styles helped the participants adopt a broader view on the issue of urban and landscape development. These wider perspectives, together with a feeling of companionship, enhanced the legitimacy of the issues and (4) motivated the participants to incorporate the shared knowledge into their own real worlds.

The evaluation also revealed the importance of hybrid spaces and communities, as Polk (2014) and Hegger and Dieperink (2015) have observed in other TD projects. In such independent spaces, participants can experience enthusiasm and trust and together produce new knowledge. This new TD knowledge includes: (1) the awareness of problem owners with other thought-styles; (2) a mutual feeling of companionship; (3) a broadened view on the issue; (4) increased legitimacy of the issue; and (5) participants motivated to tackle the issue with new approaches or with new cooperations. However, this TD knowledge tends to be tacit knowledge, and it will always be difficult to prove what changes in the participants’ real worlds it induces. In our case, the local stakeholders made some attempts to apply and spread the new knowledge in their real worlds, e.g. bringing it into the process of regional landscape development policies, and they invited us to follow-up presentations to reach further decision makers. But there is still a long way to go before visible changes in their land-use plans become apparent.