Introduction

A growing sense of urgency in addressing sustainability challenges is leading to increased motivation for environmental scientists to justify the societal influence of their research (Backstrand 2003; van Kerkhoff and Lebel 2006). Scientists are increasingly asked to consider not only the scientific credibility and adequacy of their work, but also its salience to the needs of the public and its legitimacy among stakeholders beyond their scientific peers (Cash et al. 2003). Stakeholder engagement is often conceptualized as communication of research results after the project is complete (Green et al. 2009), although the value of engaging with stakeholders during the knowledge production process is increasingly recognized (Callon 1999; Phillipson et al. 2012).

Coinciding with efforts to strengthen linkages between knowledge and action in environmental research (Kates et al. 2001; Stephens and Graham 2008), is the promotion of transdisciplinarity, an approach characterized by partnerships that cross boundaries among fields of research and modes of inquiry and between academic and non-academic actors (Kates 2002; Wainwright 2010). Most definitions of transdisciplinarity articulate an explicit incorporation of knowledge and goals of stakeholders that includes processes of mutual learning between science and society (Scholz et al. 2000). As increasing numbers of large transdisciplinary research projects involving engagement with stakeholders (Stephens and Graham 2010) are encouraged and supported, multiple anticipated and unanticipated challenges and opportunities for learning are emerging (Romsdahl and Pyke 2009).

With regard to climate change research, a “usability gap” has been identified; fostering new types of interactions between researchers and potential “users” of knowledge has been suggested as a way to narrow the gap between what scientists understand as useful and what decision-makers consider usable (Lemos et al. 2012). As research institutions and scientists respond to the call to generate usable research, many questions and uncertainties related to effective strategies for stakeholder engagement are emerging. These questions include: How should communication be structured? And, what approaches maximize mutual understanding and appreciation for different kinds of knowledge (Cash et al. 2003)?

A diversity of approaches to involving non-academic stakeholders in academic research has emerged. These approaches involve different types of researcher–stakeholder interactions and different potential for co-production of knowledge (Stephens et al. 2008; Voinov and Bousquet 2010; Palmer 2012). Participatory research has been defined as any research that integrates stakeholder knowledge into the research process (Blackstock et al. 2007). Participatory Action Research is one evolving approach with a long history that entails researchers’ and stakeholders’ active engagement as members of a knowledge production collective focused on effecting social change (Greenwood et al. 1993; Kindon et al. 2008; Smith et al. 2010). Related approaches include knowledge co-operatives and competency groups involving researchers and local communities (Phillipson et al. 2012). Another approach, increasingly applied in environmental problem solving, is group-based system dynamics modeling, variations of which include collaborative, participatory, and mediated modeling (van den Belt 2004; Antunes et al. 2006; Gaddis et al. 2007; Prell 2007; Becu et al. 2008). The process helps stakeholders understand connections and causal relationships between aspects of the human–ecological system being studied and has been applied in contexts of watershed planning and managing habitat for endangered species, for example (Beall and Ford 2010, Beall et al. 2011).

Among these multiple different approaches to stakeholder engagement, some research has reflected on the quality and type of interactions between researchers and stakeholders (Becu et al. 2008; Reed et al. 2009; Romsdahl and Pyke 2009; McNie 2012; Clavisi et al. 2013; Gourmelon et al. 2013). Within this work on researcher–stakeholder interactions, a limited amount of attention has been paid to researchers’ perceptions of the value of the stakeholder interactions (Reed et al. 2009; Romsdahl and Pyke 2009).

Models have been referred to as “boundary objects” as they enable joint collaborative knowledge production by experts and decision-makers (Cash et al. 2003), and they have potential to provide “useful” tools that can play a translational role in communicating knowledge to stakeholders. Model development involves determining types and structure of both input and output information, and stakeholder engagement during, rather than after, the model design process has potential to enhance the salience and legitimacy of the model (Phillipson et al. 2012).

In the case of a process-based earth system model, development begins with a mechanistic representation of parts of the system as they change over time and then integrates human dimensions of the system after other model components have been linked (Ford 1999; Bernholdt et al. 2005). The proposed stakeholder engagement approach for the process-based regional earth system modeling project in this study includes iterative meetings with diverse stakeholders who provide insights, guidance and feedback to the modeling team.

As stakeholder engagement in earth systems modeling becomes more frequently expected, understanding the range of researchers’ perceptions of stakeholders and attitudes about the value of their engagement could help facilitate productive interactions. This paper reports on a study of scientists’ perspectives on stakeholder engagement in the first year of a large and complex 5-year integrated modeling project addressing climate impacts on water and nutrient cycling in the Columbia River Basin. The modelers involved in this project are aware of plans for stakeholder engagement, but during the first year the mechanisms for this engagement are still evolving. Analysis of surveys and interviews conducted with researchers provides insights about breadth and variation of perspectives on the value of engaging with stakeholders. The paper begins with a review of research on stakeholder engagement and environmental modeling and background on the BioEarth project. Methods are then described, followed by a discussion of the results, and a concluding assessment of the implications of this study for transdisciplinary research projects that seek to address sustainability challenges by integrating technical academic modeling with stakeholder engagement.

Background

Environmental Modeling

Modeling is a unique mode of inquiry. It has been suggested by some philosophers of science that modeling as a form of knowledge production is qualitatively different from experimental science, although the two modes of study often work in conjunction with one another (Frigg and Hartmann 2012). There are several forms of scientific models including: metaphors that explain abstract principles with concrete imagery, systems models which visually and/or mathematically represent system interactions, and process models which simulate change over space and time (Frigg and Hartmann 2012). While models need not be characterized as “fictions”, scientific models should be approached with caution and awareness of inherent limitations, uncertainties, and simplifications (Frigg and Hartman 2012, Sterman 2002).

Earth systems models are a subset of process-based environmental models that use high-powered computing to simulate atmospheric, hydrological, and terrestrial processes over space and time and incorporate feedbacks among interrelated systems (Bernholdt et al. 2005). Since the late 1980s, earth system models have been employed to assess potential impacts of anthropogenic climate change on crop productivity and various other specific activities critical to society, but there is a recognized need to further develop earth systems models to enable better prediction of risks, to inform adaptation strategies for managing risk and to reflect growing understanding of climate–nutrient–crop dynamics (Rotter et al. 2011). A new generation of sophisticated earth system models employ economic modeling in conjunction with atmospheric and terrestrial process modeling, improving the ability of models to represent policy alternatives (Shackley and Deanwood 2003; Prinn 2012). To promote decision-making relevance, it is acknowledged that additional experimental testing of model outputs and enhanced regional specificity are needed (Rotter et al. 2011).

Transdisciplinarity, Sustainability Science, and Large Collaborative Projects

As the complexity and interconnections among human–environment systems are recognized, funding agencies have been increasingly promoting and encouraging large collaborative research projects that are “transdisciplinary”, crossing disciplinary and professional boundaries to integrate multiple different kinds of knowledge. The high degree of uncertainty associated with future environmental changes and societal responses to those changes results in a multiplicity of perspectives on how to adapt to and prepare for environmental change. While this breadth of perspectives allows for a diversity of strategies and priorities to be proposed and considered, this breadth also creates challenges for researchers seeking to integrate different perspectives in their work (Bucchi and Neresini 2008). There is a widely acknowledged need within academic institutions to build capacity to network and exchange information with representatives from government, industry, special interest groups and communities (Backstrand 2003; Cross and Smith 2007). The trending interest in science-based stakeholder dialogues has been partly driven by researchers themselves, but also brought about to a great extent by funding agencies and the general public’s demand for greater accountability in science (Welp et al. 2006). To date, stakeholder engagement initiatives do not seem to be achieving their full potential of simultaneously informing research processes and improving decision support tools available to stakeholders (Wynne 1994; Holmes and Clark 2008; Voinov and Gaddis 2008). Additional attention to effective stakeholder interactions in environmental research is fundamental to narrowing the gap between useful and useable knowledge (Lemos et al. 2012).

Within large transdisciplinary projects, communication challenges among researchers from diverse disciplines must be acknowledged alongside the challenges associated with effective communication between researchers and non-academic stakeholders. Previous research has demonstrated that effectiveness of collaborative research involving academics from different disciplines is greatly enhanced by the development of shared concerns and objectives to motivate the effort, and the cultivation of an atmosphere of openness to new approaches and innovative modes of problem solving (Lélé and Norgaard 2005). In addition, improved understanding of the attitudes, assumptions, and objectives of the researchers from different backgrounds may help in the effective design and structuring of stakeholder engagement efforts.

Among recent changes in science toward increasingly collaborative and interdisciplinary approaches to research, calls for new ways of conceptualizing stakeholder engagement have encouraged actionable science with its focus on stakeholders’ needs and interests (Palmer 2012). In addition, sustainability science, an emerging academic area that internalizes the link between knowledge and action and is defined by the problems it addresses rather than by the disciplines it employs, is expanding and influencing perceptions of the value of engagement in research (Kates et al. 2001; Clark 2007; Kajikawa 2008).

Within environmental model-based research, which has a goal of understanding complex environmental change and human–earth systems feedback processes, a paradox exists. To adequately represent the diversity and complexity of the dynamics of the earth’s systems the research must involve people with diverse expertise. Yet as the complexity of model outputs increases, the pool of individuals who can interpret those findings decreases. A high level of technical expertise is necessary to understand complex models and to promote the application or usability of that research to decision-making.

Stakeholder Engagement

A stakeholder is generally defined as a person or a group who has an interest in an issue, policy, company, or other entity (Welp et al. 2006). The concept originates from business and management literature where a distinction is made between shareholders, or those who own the company, and stakeholders, those individuals or groups who are impacted by business activities or can influence the business environment (Welp et al. 2006). In part because there are many ways to define who the stakeholders are for any given research process, there are many possible varieties of stakeholder participation in science (Bucchi and Neresini 2008). Potential roles of stakeholder in research are varied and can include anything from identifying research questions; sharing values, preferences, expectations and perceptions of risk; providing quantitative data or local expertise; commenting on research concepts, drafts and results; learning from the research process; and/or integrating research findings into decision-making processes.

Cooperative extension programs designed to connect university research with agriculture have a long history of effective stakeholder engagement work connecting local decision-makers and resource managers with academic research (Bull et al. 2004). A current debate within the extension system focuses on whether the traditional communication strategies and programmatic mission of university extension services are viable in the modern context (Kalambokidis 2004; McDowell 2004; Franz and Cox 2012). Many factors have contributed to a diminished role of these extension programs including decreased funding, the changing landscape of American livelihoods, and new modes of communication and information exchange than render some of extension’s traditional knowledge development and information sharing methods outdated (Kalambokidis 2004; Franz and Cox 2012).

Study of the perceptions and attitudes surrounding stakeholder engagement processes in current cutting-edge environmental science is in part an effort to re-envision traditional extension methodologies used by land-grant universities to bridge the academic and public and private decision-making spheres. Given the sustainability challenges facing the world, it is appropriate that academic institutions cultivate new opportunities to influence society by enhancing the quality of interactions with industry, government, and the non-profit sector (Probst et al. 2003). The possible mechanisms for enhanced linkages between academia and decision-makers at different levels are varied, but include, for example, engagement in policy-making, non-formal education, community development and planning, and technology assistance (Probst et al. 2003; McDowell 2004). Engagement of stakeholders external to academia is fundamental to many conceptions of transdisciplinarity (Scholz et al. 2000). As funding agencies and research bodies increasingly seek transdisciplinary approaches and require stakeholder engagement, learning from and expanding upon the approaches pioneered by extension services could be increasingly valuable.

The majority of case studies of stakeholder engagement in environmental research focus on decision-making processes and highlight the role of stakeholders in interpreting scientific results and making resource management decisions on the basis of data and models (Grimble and Wellard 1997; Cash et al. 2003; Hare et al. 2003; Shackley and Deanwood 2003; McNie et al. 2007). Under the rubric of “stakeholder engagement” a smaller body of work explores the role of non-academic stakeholders in framing research questions and actively participating in the development of new scientific and technical knowledge (Grimble and Wellard 1997; Hare et al. 2003; Welp et al. 2006; McNie et al. 2007; Prell et al. 2007; Voinov and Gaddis 2008). A recent evaluation of the US National Oceanic and Atmospheric Administration’s Regional Integrated Sciences and Assessments programs, which are designed to produce useful information about climate for decision support, considers multiple aspects as critical to program effectiveness including identifying stakeholders’ information needs, translating and communicating knowledge, situating social capital and building users’ capacity to interpret and apply research findings, and establishing a flexible organization with strong leadership (McNie 2012).

There are few published accounts of researchers’ reflections on stakeholder participation processes, yet several case-study reports focusing on stakeholder engagement in environmental research contain some discussion of scientists’ attitudes about the process (Becu et al. 2008; Gardner et al. 2009; Prell et al. 2009; Reed et al. 2009; Romsdahl and Pyke 2009). It is clear that taking a participatory approach to model development provides both refreshing new perspectives as well as some frustrations for scientists (Reed et al. 2009; Romsdahl and Pyke 2009). Beginning a research project without knowing exactly what parameters will be modeled is clearly a novel experience for many scientists.

BioEarth

BioEarth is a large collaborative 5-year project funded by the US Department of Agriculture, National Institute of Food and Agriculture (2011-67003-30346). This project aims to develop an earth systems model that addresses climate change impacts on agriculture and forestry. The research will investigate climatic and anthropogenic impacts on nutrient cycling, water resources and air quality in the Columbia River basin and the U.S. Pacific Northwest. BioEarth is among a new generation of large environmental change research projects that is transdisciplinary and integrates stakeholder engagement as a key aspect of the proposed research plan (Godin and Gingras 2000; Cummings and Kiesler 2005).

The BioEarth research team comprises individuals from the disciplines of atmospheric sciences, biogeochemistry, agricultural sciences, hydrology, aquatic chemistry, economics, and environmental communication. These researchers are arranged within five working groups: modeling, cyberinfrastructure, economics, ecology, and communication. The communication working group is tasked with developing mechanisms for interactive communication between model developers and practitioners throughout the project, including workshops, meetings, and a virtual Internet forum. A related objective of the communications working group is to analyze the perceptions and understandings of stakeholders and scientists throughout the research process using surveys and interviews to track the evolution of perceptions of the stakeholder engagement process and of the utility and relevance of the model to decision-making. This paper reports on the results of one component of this participatory action research conducted by the communication group researchers.

The initial BioEarth project proposal described a plan for bi-directional communication to enable stakeholders to influence the research questions that are addressed within the model development process. A series of advisory workshops are to be conducted throughout the 5-year project with stakeholders from the agriculture and forestry sectors of the Pacific Northwest. With facilitation from experienced extension faculty on the communication team, project modelers will engage directly with stakeholders from a diverse array of government and industry groups in discussions of the model development process. Enhancing the relevance and utility of the BioEarth model within the forestry and agricultural sectors is an objective of these interactions between modelers and stakeholders.

As is typical of large transdisciplinary research projects conducted at universities, the BioEarth research team was assembled based on previously established working relationships among PIs and brought in individuals from other institutions and disciplines based on their known areas of research expertise. The collaboratively written proposal was tailored for a joint National Science Foundation–US Department of Agriculture regional earth systems modeling funding opportunity. Stakeholder engagement and associated communication research was a critical and substantial part of the funded research proposal. Project communication is facilitated through four mechanisms: working groups meet regularly; monthly integration meetings provide an opportunity for cross-working group communication; the full research team of PIs and graduate students (from four different universities and two government research institutions) meets twice a year to share progress and make decisions about overall project direction; and an all-project email list-serve is used to update researchers on project progress.

This research focused on understanding BioEarth researchers’ initial perceptions of stakeholder engagement. It will inform the design of information exchange mechanisms between researchers and stakeholders and will assist in anticipating communication challenges and preparing engagement strategies. While the research presented here focuses on understanding researchers’ perceptions of stakeholder engagement during the initial phase of the project subsequent research will assess the changing perceptions of both stakeholders and researchers throughout the duration of the project. The results of this research may contribute to preventing potential stumbling blocks associated with environmental science research that emphasizes transdisciplinary collaboration and a sustainability solutions orientation.

Methods

To assess researchers’ perceptions of stakeholder engagement in this collaborative earth systems modeling project, a participatory action research approach was developed and carried out by the communication team. This study involved a brief questionnaire and a 30-4y semi-structured interviewFootnote 1 which was conducted with each of the 18 Principal Investigators (PIs) of the BioEarth project. The questionnaire, administered online, was used to obtain baseline information about BioEarth researchers’ previous experiences and attitudes related to stakeholder engagement. Five simple Likert-scale (ranking) questions allowed for responses to be tabulated numerically and represented graphically. The questionnaire asked the PIs to self-report their frequency of interacting with stakeholders, level of satisfaction with previous stakeholder interactions, and perceptions about the importance of engaging various stakeholder groups at various phases of the project.

Following the questionnaire, interviews were scheduled with each of the PIs to obtain more in-depth information about perceptions of stakeholder engagement. During the interviews, each PI was asked to describe how they envisioned successful research outcomes for the project and how they perceived potential challenges, particularly challenges related to communication and stakeholder engagement.

Analysis of the interview transcripts emphasized understanding the range of researchers’ perspectives. Coding of the interview transcripts was accomplished through the use of QSR International’s NVivo 8.0™ qualitative software. The coding scheme was developed to assess PIs’ perspectives on a series of overlapping topics including project challenges, communication pathways, goals, expectations, utility, novelty, stakeholder definitions, stakeholder buy-in, and timing. Within the coded text, three key themes emerged: goals and expectations, definitions of stakeholders, and project challenges. Analysis included clustering of similar PI responses related to these themes.

Results and Discussion

This section describes the professional diversity within the BioEarth research team, and then reports the survey results, followed by a discussion of the interview results.

BioEarth Research Team Demographics

Among the 18 PIs, a diversity of professional roles are represented (Table 1). PIs have varying levels of previous experience with integrated biogeochemical modeling and stakeholder engagement. The BioEarth research initiative is composed of four working groups (some PIs are part of more than one working group): modeling (ten individuals); cyberinfrastructure (three individuals); economics (three individuals); ecology (three individuals); communications (four individuals). Three of the PIs are assistant or associate professors at partner universities other than WSU, and two of the PIs are affiliated with government research laboratories not based at universities. The remaining 11 PIs are WSU professors: 8 are associate, assistant, or research professors; 3 of the WSU-based PIs are full professors and/or department heads (Table 1).

Table 1 BioEarth Principal Investigators’ professional backgrounds
Full size table

Five of the PIs are women and 13 are men. Although gender was not explicitly considered in the analysis of researchers’ perceptions in this study, the role of gender in interdisciplinary research, collaborative science, and stakeholder engagement is deserving of additional study. Recent research on the learning styles, work preferences and career behaviors associated with interdisciplinary research explores whether women may be more drawn to interdisciplinary research than men (Rhoten and Pfirman 2007). Future analysis could focus on the gender dynamics within the BioEarth project.

Survey Results

The survey provided quantitative data about four topics: (1) researchers’ frequency of interaction with stakeholders in their previous work; (2) researchers’ satisfaction with previous stakeholder interactions; (3) assessments of the value of engaging various kinds of stakeholders; and, (4) assessments of the potential for successful stakeholder engagement at different phases of the project.

BioEarth PIs have varying levels of experience with stakeholder engagement; half of the 18 PIs reported occasionally working with stakeholders on other projects, 5 members of the research team have previously worked with stakeholders frequently or always, and 3 members of the research team reported rare or no previous work with stakeholders (Fig. 1a).

Fig. 1
figure 1

Previous experiences with stakeholder engagement from survey responses: a frequency of researchers’ previous interactions with stakeholders; b researchers’ level of reported satisfaction in their interactions with stakeholders in previous research projects

Full size image

Reported satisfaction about previous stakeholder interactions emphasized the middle ground; neither dissatisfaction nor having expectations exceeded were reported. Ten PIs reported that they were mostly satisfied, five reported that they felt neutral, neither satisfied nor dissatisfied with the process, and two responded that previous experiences with stakeholder engagement had been satisfactory (Fig. 1b). One PI did not answer the question due to having no prior experiences with stakeholder engagement.

The survey results demonstrate variation in perceptions among PIs of the importance of different stakeholder groups to the overall success of the project (Fig. 2). When each PI was asked to rank the level of importance (on a scale from 1–5 with 1 being low importance and 5 being high) of five specific types of predefined stakeholder groups (academia, advocacy, public, government and industry), the academic stakeholders were deemed the most important. Eleven of the 18 PIs assigned the academic stakeholders an importance score of 5. Government and industry stakeholders were judged to be of relatively high importance for project success as well; the majority of PIs assigned a score of 4 or 5 to these categories. Advocacy groups such as NGOs and the general public were viewed by many BioEarth PIs as not important to the success of the project. These two stakeholder categories also show the greatest range in importance values assigned, indicating divergent views within the research team.

Fig. 2
figure 2

Perceived importance of engaging various stakeholder groups. Co-PIs were asked to assign an importance score to each of five categories of potential stakeholder groups. A score of “5” represents critical importance to overall success of the project and a score of “1” represents no importance. The size of each circle represents the number of PIs who selected a given importance score for each stakeholder group

Full size image

When PIs were asked to assess the importance of stakeholder engagement at different phases within the 5-year project, survey results show a general consensus that engagement in the middle years and at the end of the project was deemed most important for overall project success (Fig. 3). Perceptions about the value of stakeholder engagement in the early phases of model development (year 1) were highly variable with a relatively even distribution of perceptions for each value score from 1 to 5.

Fig. 3
figure 3

Perceived importance of stakeholder engagement at different phases of the project. Survey respondents were asked to assign each of three discrete phases of the project an importance score with“5” representing critical importance to overall success of the project and “1” representing no importance. The size of each circle represents the number of PIs who selected a given importance score

Full size image

Interview Results

The results of the interviews reveal considerable variation in BioEarth researchers’ definitions of project success, perceptions of stakeholder identities, and assumptions of likely project challenges (Fig. 4). To represent the range of PIs’ responses, circles sized to represent the number of PIs in each emergent cluster have been placed on a spectrum (Fig. 4). Each individual PI’s response was assigned to a cluster based on comparative content analysis of the transcripts. With regard to project success, responses were distributed on a spectrum with one side representing definitions focused solely on technical capabilities and contributions to scientific knowledge and the other side representing definitions focused on the effective utilization of the model to informing management decisions (Fig. 4a). With regard to the question of who are the stakeholders for this project, responses were distributed on a spectrum with one side representing a narrow definition focused on academic stakeholders and the other side representing broader definitions that included more general audiences (Fig. 4b). With regard to the project’s primary challenges, responses were distributed on a spectrum with one side representing responses focused on technical model integration issues and the other side representing challenges associated with communication and stakeholder engagement. Ten of the 18 researchers highlighted challenges in both of these areas (Fig. 4c).

Fig. 4
figure 4

Continuum of researcher perceptions on 3 key thematic questions: a What defines a successful outcome for the project? b What groups are stakeholders? and c What are primary challenges of the project? The circle size represents the number of PIs whose interview transcripts revealed answers clustering in specific places on the continuum

Full size image

Trends in the placement of each PI’s perceptions on the three continuums displayed in Fig. 4 are apparent, such that respondents on the left of one of these continuums are likely to have a similar leftward placement on the two other continuums. PIs with a vision of project success focused on increasing technical modeling capabilities were likely to define members of academia as primary project stakeholders and focus their discussion of challenges on technical issues of model integration. PIs who related project success to model application by stakeholders in a decision-making capacity tended to define stakeholders more broadly and focus primarily on challenges associated with communication.

Table 2 represents each co-PI’s level of experience and satisfaction with stakeholder engagement in previous research projects as well as the clustering of their interview responses along the three continuums in Fig. 4. Responses in clusters A and B are shaded dark gray representing the left side of the continuum in Fig. 4, cluster C is shaded medium gray representing the middle of the continuum, and clusters D and E have light shading representing the right side of the continuum. Responses of 6 of the 18 PIs (PI No. 2, 3, 14, 16, and 17) are represented by the same level of shading in all three continuums, and only one PI’s (No. 15) responses are on the left of one continuum, the middle of another and the right of another. The remaining 11 PIs responses have the same shading on two of the three continuums. Findings suggest that researchers in the modeling working group tend to focus on technical outcomes when considering project success while the communication researchers prioritize decision-making utility for project success. Those with prior experience with stakeholders tend to be clustered in groups D and E, the left side of the spectrum, for most categories.

Table 2 PIs’ Previous Experience with Stakeholder Engagement and Position on 3 Continuums
Full size table

3a Diversity of Visions of Project Success within the Research Team

Perceptions among BioEarth PIs of what would constitute success of the project have been divided into five categories along a continuum of those who prioritize the technical and scientific contributions on one end and those who prioritize the influence of the model on management decisions on the other (Fig. 4a). Cluster A represents 5 of the 18 PIs involved in the project who were strongly focused on benchmarks of success related to technical capabilities of the model, 4 PIs (cluster B) were focused in this direction but less strongly, 3 had a balance between these two viewpoints (cluster C), 4 prioritized influence on management decisions to some degree (cluster D), and only 2 PIs (cluster E) felt strongly that project success depended entirely on the ability of the model to influence management and policy decisions (Fig. 4a).Footnote 2

The following interview excerpts exemplify the viewpoint that project success will be defined by new scientific understanding and enhanced technical capabilities without the expectation that the model will be relevant to non-academic stakeholders:

We’re developing a research framework… the model is not going to answer very specific questions. It’s not going to tell you what’s going to happen in the future (PI_13).

The number of people who will actually be able to use the model is a pretty small number (PI_5).

The PIs whose definition of success focused on technical integration of model components were generally interested in the model as a tool which would be refined in future research initiatives and modified for application in other settings. This group of respondents generally did not expect the model to contribute significantly to stakeholders’ knowledge about regional climatic, ecological, or economic conditions nor did they expect the model to play an important role in discussions of natural resources policy. The quotation above (PI_13) also highlights concern regarding perceptions of how a regional-scale model can be useful to stakeholders making decisions at a local level. While regional-scale information can inform decision-making at multiple levels, this researcher is demonstrating recognition that decision-makers may not be in a position to effectively utilize regional-scale information.

The six PIs in clusters D and E on continuum 4a focused their description of project success on the vision that stakeholders would use the model to support decision-making. The PIs who considered stakeholder participation and application of the model as critical benchmarks of success do not have homogenous views of who the project stakeholders are and at what point in the research process stakeholder engagement is most valuable. These researchers, who were clustered toward the end of the continuum focused on “informing management decisions”, did all mention the need for simplification and clear communication of elements of the research in order for it to be accessible to non-specialists.

Regardless of how PIs defined a successful outcome for the BioEarth project, all participants recognized that this integrated regional modeling effort represents new and exciting research and that this project is likely to lead to other more sophisticated models in the future.

3b Concepts of stakeholder identities

In addition to the diversity of perceptions about what constitutes project success, a spectrum of perspectives about how broadly “stakeholders” should be defined in the context of the BioEarth project was evident (Fig. 4b). Four PIs (cluster A) expressed a narrow definition of project stakeholders; these participants identified academia as the sphere in which knowledge generated by BioEarth would be relevant. Six PIs (cluster B) expected that the research would be of interest to a small circle of academics and decision-makers within government agencies already familiar with earth systems modeling. Four PIs (cluster C) had an intermediate viewpoint of how broadly project stakeholders should be defined, mentioning that industry, government and academia were the probable participants in the stakeholder engagement process. Three PIs (cluster D) broadened their list of potential stakeholders to include anyone who makes land management decisions in the region and NGOs interested in natural resources policy. At the far right end of the continuum, one PI (cluster E) defined potential stakeholders very broadly, suggesting that anyone living in the region of study should be regarded as a potential stakeholder.

Six participants mentioned that academics and non-academics outside the project are two distinct groups that require different strategies for engagement and information sharing. Individual PIs characterize and prioritize the involvement of these two groups (‘academic stakeholders’ and ‘other stakeholders’) differently, as expressed in the following quote:

It would be nice if we could engage the bigger academic community perhaps by organizing a workshop of groups doing integrated earth system models… And then a second group of stakeholders would be people that can make use of the insights that will eventually fall out of the models that we develop (PI_11).

Four PIs expressed a broad definition of stakeholders by suggesting multiple different groups who could be involved in the stakeholder engagement process. These PIs with the broadest perceptions of relevant stakeholders are the same individuals who expressed the expectation that research carried out in BioEarth would lead to policy changes or aid in resource management decisions. These PIs were also among those who ranked all the suggested categories of stakeholders at an importance score of three or greater in the quantitative survey (see Fig. 2). One participant said that if the project focused only on academic stakeholders and was not shared with groups outside of the scientific community, then project goals would not be achieved. The PIs with broad definitions of stakeholders tend to have some degree of a social sciences orientation in their own research. This suggests that the PIs whose own research does not integrate a social science orientation, i.e., those focused explicitly on the technical details of the modeling of the physical sciences, have very different expectations for stakeholder engagement.

When asked to identify key stakeholders for BioEarth, only three PIs explicitly mentioned the role of representatives from agriculture and forestry. Later in the interviews, when participants were questioned specifically about how agriculture and forestry sector representatives might be able to utilize the model, 13 PIs explained facets of the research that may be applicable. Three PIs stated that making the model applicable to industry was not necessarily feasible or necessary (although this was a clearly stated goal in the project proposal that was successfully funded). Two PIs deferred from answering the question about the agriculture and forestry industry; one because of a lack of familiarity with the needs and priorities of those industries, and the other PI noted that it remains uncertain what kind of results the integrated model will yield, and explained that the specificity and certainty of research findings will determine their applicability. Of the 13 PIs who described potential utility of the model to industries in the region, 5 participants included caveats in their response to clarify that only some portion of the model would be useful to stakeholders from these groups; concern was expressed about how stakeholders without technical scientific training would use detailed information from the model.

3c Perceptions of project challenges

When asked about major challenges of the project, two distinct categories of challenges are evident in the responses: (1) challenges of communicating the scientific information to non-specialists and (2) technical challenges of model integration. At a basic level, both challenges arise from the need for a common language between people with different expertise and from the time and effort required to learn from others and share information. The challenges identified are interconnected and were referenced at multiple different phases in the interviews, i.e., not only when participants were specifically prompted to discuss the major challenges of the project. Three PIs (cluster A) focused the majority of their discussion of project challenges on technical issues, ten PIs (cluster C) discussed both categories of project challenge with equal frequency, and five individuals (cluster E) focused heavily on communication-related challenges (Fig. 4c)Footnote 3.

One researcher with a technical challenge focus (cluster A) noted:

There’s a challenge in actually providing really good information on future regional climate. The models today have a challenging time just predicting sort of global changes in climate and the earth system, and it gets more difficult when you get down to the regional level (PI_4).

Model integration is also a significant technical hurdle, as mentioned by this researcher (cluster C):

There are the climate models and surface hydrology and economic models and they are usually defined over space and time, I think the biggest challenge is going to be that, working out space and time, how you break up space and time in a model. Which is difficult anyways, but then if you have to have multiple models and integrate them, and have them talk to each other and they all have different space and time elements, then it really becomes difficult (PI_2).

The question of how to go about engaging stakeholders and justifying the utility of the project to groups outside of the research team was mentioned frequently, most often among members of the research team who expected a model development process that would contribute to natural resource decision-making and engage stakeholders in a meaningful way. Five researchers (cluster E) defined the project’s primary challenges as communication with stakeholders and managing expectations. One PI from this cluster noted:

Stakeholders want to know ‘how many inches of water will be in this reservoir on this year?’ and it’s really difficult to provide that really specific information that they want with the accuracy that they want, so communication is huge (PI_10).

PIs grouped in cluster E on the “project challenges” continuum expressed awareness that facilitating stakeholder engagement in the research process will require sensitivity to conflicts of interest among different stakeholder groups:

We are often faced with the need to balance one group’s wants and desires with the desires of other people. That can often be uncomfortable, and it can often lead to a lot of misunderstanding (PI_7).

Additional difficulty, cost, time and effort associated with a large transdisciplinary collaborative endeavor were mentioned frequently when PIs were asked about project challenges. The challenges of collaboration and communication with other scientists are similar to and connected to the challenges of communication with stakeholders. One participant (cluster C), stated:

In a collaborative project like this you just can’t have investigators out doing their own thing. The outcomes are going to be much greater than the sum of any individual efforts that we could do and that’s because of the synthesis we can provide by collaboration (PI_18).

But this potential for development of a model with capabilities and societal relevance that are greater than the sum of the parts of component models comes with a cost and additional challenges, as noted by another PI (cluster C):

Everybody’s busy, everybody has a lot of meetings to attend, lot of other demands on their time. But I think the fact that we strive for having frequent connections, is extremely important. Otherwise, we’ll end up just doing ‘island things’, each one of us is doing something and the integration maybe will happen in a rush (PI_13).

Conclusions

As environmental research continues to shift toward use-inspired, socially engaged, transdisciplinarity, there is potential to improve understanding of researchers’ experiences and expectations of stakeholder engagement. Developing an understanding of researchers’ diverse perspectives will enhance the research-practice landscape. While this analysis focuses on researchers from a single earth system modeling project, BioEarth is representative of an increasingly common type of large, collaborative environmental research project that integrates researchers from multiple disciplinary backgrounds and has articulated high expectations for stakeholder engagement. This study of researchers’ perceptions reveals a lack of consensus among PIs involved in the same project regarding the value, type, timing, and expected outcomes of stakeholder engagement—highlighting the social complexities of these emerging types of research projects.

The heterogeneity of researchers’ perceptions of who the key stakeholders are and how they might interact with the researchers over the course of the project represents more variety than might be expected given the description of stakeholder engagement in the original project proposal. Some PIs retain the conventional research paradigm in which scientists communicate their results to stakeholders after the research has been conducted. These PIs do not tend to recognize value in stakeholders providing input early on in the project. Some PIs view the integrated regional earth system model that is being developed as a tool for decision-making, others describe the model as ‘preliminary’ and consider the central objective of the project to be addressing purely academic questions and building technical modeling capacity. The fact that roughly one quarter of the PIs in this research project consider their primary stakeholders to be other academics suggests that the goals of transdisciplinarity, with its emphasis on connecting to knowledge outside the traditional academic disciplines, are not equally prioritized and acknowledged by all members of the research team.

While surveys and interviews conducted in the first year of BioEarth demonstrate that a common vision of stakeholder engagement does not yet exist in this project, communication among the PIs in full-team meetings since the surveys and interviews has provided opportunities to move closer toward shared stakeholder engagement objectives. And as the project moves beyond its first year, a set of stakeholders with knowledge about the regional decision-making context have been identified and will participate in a series of stakeholder advisory meetings with project PIs. Findings from this assessment of PI’s perceptions have informed the planning of these stakeholder advisory workshops. In an effort to facilitate dialogue and interactions among stakeholders and researchers, workshops will include a series of open-ended discussion prompts and digital-response multiple choice questions. Questions and discussion prompts have been developed with input from multiple PIs representing the different working groups.

Communication about earth systems model development poses specific challenges in stakeholder engagement due to model complexity and uncertainty (Rotter et al. 2011). Scientists who have expertise in developing computer-based earth system models often have less experience working with stakeholders outside academia than do economists, experimental environmental scientists, and academics with extension appointments. PIs with a social science background reported higher expectations for impactful stakeholder engagement than did those whose backgrounds were more exclusively technical. The tendency for modelers to value academic stakeholders’ participation most highly is not surprising because most of their professional interactions are among other academics, and their professional communication skills are developed primarily with a focus on communicating with other academics. In contrast, experimental agricultural scientists may consider agriculture industry representatives as important stakeholders because they may have communicated with these actors in previous research.

PIs with previous exposure to stakeholder engagement processes in which environmental and economic science were communicated and applied tend to have more broad and inclusive perceptions of relevant stakeholders and to place a higher value on stakeholder engagement early in the research process as opposed to thinking of engagement solely as communicating results after project completion. Given the diversity of scientists’ perspectives on stakeholder engagement, a priority that emerges from the results of this study is the strengthening of peer-learning among the collaborating scientists; i.e., valuable learning experiences occur when PIs with prior experience interacting with stakeholders work closely with PIs who have less experience and a more technical modeling orientation. This suggests that effective stakeholder engagement is dependent on effective transdisciplinarity communication among the research team.

The potential for transdisciplinary sustainability science research projects to improve the understanding of regional human–environment interactions and inform decisions is large, but overcoming multiple challenges related to communication across disciplinary divides and between academics and non-academic stakeholders involves learning new approaches to research and communication. These new approaches include: working in the early phase of research question development to identify stakeholders’ information needs; developing shared vocabularies and new forums for translating and communicating knowledge; and working closely with stakeholder groups to increase organizational capacity to apply research findings. Environmental scientists who are willing and able to learn new approaches to integrating stakeholder knowledge into their work are likely to be more successful as science that is socially salient and actionable is increasingly valued. The analysis presented here may contribute to overcoming potential communication barriers by encouraging modelers to place a higher value on working with stakeholders to develop decision-relevant outputs.