Introduction

Identifying patterns of transdisciplinary research practice

Planetary sustainability rests on the twin conceptual ideals of remaining within specific environmental boundaries, like biospheric integrity, climate limits and freshwater use, and providing social foundations for a safe and just space for humanity to thrive (Rockström et al. 2009; Raworth 2012). Key approaches to tackling these global sustainability challenges include interdisciplinarity—a means of knowledge creation that integrates information, data, methods, perspectives, concepts and/or theories from multiple disciplines (Kates et al. 2001; National Academies Press 2004) and transdisciplinarity—involving people outside of academia to fully understand pressing social, ecological, and economic problems and co-create feasible solutions (Carew and Wikson 2010; Lang et al. 2012; Scholz and Steiner 2015).

Scholars who pursue transdisciplinary work face challenges in demonstrating evidence of research productivity and practice that differ from traditional practices of science. Significant social capital often needs to be invested in building project teams and relationships with communities to carry out collaborative research (Stokols et al. 2008). When compared to traditional, curiosity-driven science projects, publication outputs in the early stages of highly collaborative work have been found to lag behind, which can pose a risk especially to early career scientists (Hall et al. 2012; Fischer et al. 2012) based on conventional academic models.

In this article, we demonstrate how social network analysis (SNA) can be used to provide empirical insight into the detection of patterns and practices associated with transdisciplinarity. In many respects, a transdisciplinary project is an exercise in creating new knowledge through co-production. It can also be conceived as a network comprised of a variety of actors of different types—community members, industry, government representatives, academic researchers, students, non-profit organizations and so on. Each brings their own values, views, experiences and knowledge to bear on framing, designing and implementing the project. As such, a variety of network tools and measures can be used to evaluate social structure and the knowledge exchange patterns supported by those structures. Key SNA metrics can provide insight into who is participating, whether and how integration and collaboration are occurring, the relative degrees of network stability and fragility, and how the network is structured to achieve its goals. In this article, we illustrate how SNA was used with the example of the Delta Dialogue Network (DDN)—a multi-year transdisciplinary project that involved work with Indigenous and non-Indigenous communities from three inland deltas across Canada to promote knowledge co-production and address regional-scale sustainability problems.

Defining, evaluating and detecting transdisciplinary knowledge patterns

Transdisciplinarity has several working definitions; we define transdisciplinarity as a partnership among participants within and outside the academy—one that moves beyond academic walls and disciplines to work with those most affected by the problems that will be addressed (Defila and DiGiulio 1999; Wickson et al. 2006; Lang et al. 2012; Scholz and Steiner 2015; Belcher et al. 2016). There are three key distinguishing characteristics of transdisciplinary research relative to more conventional disciplinary science and these include: (1) collaborative problem identification because those affected by the problem and the disciplines needed to understand the problem are all involved in the problem definition; (2) an explicit normative focus as solution oriented work, which inherently involves the identification of preferred values; and (3) the promise of increased saliency, legitimacy, credibility and effectiveness of proposed solutions (Carew and Wickson 2010; Lang et al. 2012; Belcher et al. 2016).

The challenges of transdisciplinary work lie in respecting, balancing, bridging, reconciling and/or sometimes integrating differing knowledge systems, values and processes among disciplines and with partner communities, which translate into potentially differing assumptions about what constitutes effective interaction and credible knowledge generation (Wickson et al. 2006; Carew and Wickson 2010; Edelenbos et al. 2011; Belcher et al. 2016; Steelman et al. 2015). This would entail a network of actors who combines learning and doing, as well as creating original knowledge-based products and outcomes in appropriate formats (Tengö et al. 2014). Hallmarks of this new type of knowledge co-production pattern would include indicators of collaboration and integration among disciplines and beyond disciplines, a more “democratic research practice” as well as evidence of shared decision-making related to problem definition and solutions (Lang et al. 2012; Belcher et al. 2016; Steelman et al. 2015).

An additional challenge for transdisciplinary practice is the lack of quality assessment criteria (Durose et al. 2018; Belcher et al. 2016). While more conventional disciplinary research has evolved relatively clear expectations for what constitutes quality (e.g. peer review processes, publication productivity, citations, journal impact factors), equivalent criteria, metrics, processes and other sources of evaluation are less agreed upon when it comes to transdisciplinary research (Belcher et al. 2016; Ascher et al. 2010). A significant challenge for transdisciplinary research is its inclusion of partners outside of the academy and how their involvement, expectations and contributions can be appropriately measured and evaluated (Belcher et al. 2016). This includes the creation of processes and outputs that are most relevant to non-academic partners, which typically move beyond peer reviewed publications and conference presentations.

To date, there is little consensus on how transdisciplinary processes and outcomes should be evaluated (Wickson et al. 2006; Klein 2008; Brandt et al. 2013; Durose et al. 2018). Several approaches have been recommended, yet their units of analysis for comparability differ. For instance, Wickson et al. (2006) suggest introspective criteria for individuals within a project while also providing a quality framework to allow comparability across projects. Others identify criteria such as relevance, credibility, legitimacy and effectiveness, which can be applied throughout the research process to the final outcome, so that transdisciplinary contributions can be assessed (Belcher et al. 2016). These criteria also suggest that the unit of analysis is the project. The checklists (Defila and Di Giulio 1999) and survey instruments (Mâsse et al. 2008) derived from specific projects identify both process and outcome criteria, suggesting the focus is on both individuals and groups within a project. Others have devised quantitative measures such as author credit algorithms (Xu et al. 2015) for assessing contributions to complex projects, targeting an analysis of individuals.

An additional, yet underexplored approach to evaluating transdisciplinarity work is through the lens of social networks. We focus on SNA because it allows researchers to measure and, in some cases, visually represent individual, group, or organizational relationships, relational structure, and their patterns. While analyzing social networks has helped provide insight into a variety of environmental and natural resource-based problems (Bodin and Crona 2009; Bodin 2017), it has been under-utilized to understand transdisciplinary research. Since transdisciplinarity seeks to engage multiple partners to inform problems, co-create solutions, and diffuse new knowledge more effectively, SNA is well suited to capture some of the dynamics of research with communities outside of academia (Brandt et al. 2013). Hence, understanding social networks may help us assess the roles of individuals and groups as well as their patterns of interaction in a transdisciplinary research initiative. In doing so, these performance metrics can help begin to provide norms for what constitutes “success” in transdisciplinary research.

Social network indicators for identifying transdisciplinary research patterns

Our proposed approach focused on elements of structural functionality that we would expect to see in a viable transdisciplinary network. Four key characteristics are derived from the literature. These include evidence of: (1) diversity; (2) collaboration and integration; (3) network stability; and (4) a balance between efficiency and power sharing, as detailed in Table 1.

Table 1 Transdisciplinary social network criteria
Full size table

In thinking about transdisciplinary research in the context of the DDN, we would first want to consider the diversity of the network in terms of number of members—network size, and members’ affiliation (i.e. academic, non-academic)—as part of the network composition. We see diversity as a necessary, but insufficient condition for enabling collaborations in a transdisciplinary research setting.

Second, we would want to see evidence of integration and collaboration (Carew and Wickson 2010; Reed et al. 2010; Edelenbos et al. 2011) that could help enhance communication and overcome the significant barriers to transdisciplinary practice. There is no clear-cut value or unique measure that indicates if a network is integrative or collaborative. However, identifying and comparing a variety of indicators provides evidence about relative levels of integration and collaboration. For this we would want to consider the cohesion of the network in terms of density, average path length, diameter and network clustering (Prell 2012).

Density is a measure of cohesion, calculated as the proportion of all possible ties that are present within a network. Density measures the degree to which members of a network create ties between each other. The degree to which actors are connected to each other or how inclusive the network can inform about the extent and speed at which information or other resources can spread among members, and the extent to which individuals have high levels of social capital or social constraint (Durkheim 1964; Coleman 1988; Scott 2017). Density scores range from 0 to 1,Footnote 1 where 0 indicates that the network is fully disconnected, and a score of 1 indicates that all actors are connected directly to each other. Density should make comparison across groups of different sizes possible, since it accounts for network size. However, caution needs to be used when comparing networks with large differences in size (Prell 2012; Scott 2017). This is because densities are always lower in larger networks, since it becomes increasingly hard for individuals to connect as the network size increases. The negative association between network size and density measures needs to be taken into consideration when trying to understand network connectedness.

The average number of connections each collaborator has is typically measured. However, a more robust way of understanding the connections within a network is to use the median and the interquartile range (IQR), since the distribution of ties an individual has is typically not normal. Since cohesion is relative to network size, we also use average path length and diameter to understand a network’s cohesion. The diameter of a network represents the longest path in the network, which connects the two most distant members. The smaller the values for these indicators, the closer or the more socially proximal members of the network are, hence the more cohesive the network as a whole (Prell 2012).

Another indicator of cohesion is clustering. A social network characterized by high levels of clustering when compared to a random network, and with relatively short paths between individuals, can be considered a “small world” (Travers and Milgram 1967; Watts and Strogatz 1998; Borgatti et al. 2013) which provides an enhanced opportunity for information or innovations to spread faster among the members of the network (Fleming and Waguespack 2007; Klenk et al. 2010). Carrington et al. (2005) uses the clustering coefficient as a measure of decision-making collaboration in groups. Collaborative groups were found to have higher clustering coefficients (Carrington et al. 2005). The overall network clustering coefficient represents the degree to which members of a network form tight groups, which are characterized by high density of interactions (Borgatti et al. 2013). Two other metrics can be used to assess cohesion. These include homophily and network density by group. Homophily measures the extent to which network members collaborate with the same others, and network density by group provides information about the extent to which members interact within their respective groups of or outside of them.

Third, we would want to see a stable network. This would entail observing multiple individuals with high centrality scores. This could indicate that the network is not fragile or dependent on one or two key individuals. We would want to see evidence of boundary spanners (Edelenbos and van Meerkerk 2015) playing a role in building connective capacity across the network to facilitate learning. Consequently, we would look for cut-off points occupied by key bridging actors who exhibit high betweenness centrality. This would indicate the network is at risk if something happened to these individuals. We would look for clustering indicators that convey the relative stability of the network in terms of connection (Hanneman and Riddle 2011). Betweenness centrality is a network indicator developed by Freeman (1979) that measures the frequency with which an individual is positioned along the shortest path between two other individuals in the network (Borgatti et al. 2013).

Finally, we would want to see a network structure that represented the need to balance power and decision sharing with efficiency. To avoid power asymmetries (Tengo et al. 2014), we would want to see a relatively decentralized network structure or a horizontal network as opposed to a hierarchical or highly centralized network. But we would also want to see some level of centralization to maintain an ability to function. Someone needs to lead the network and provide direction. Degree centralization can provide an indicator of the relative level of hierarchy and whether collaborators are more vertically or horizontally interacting within the network. Indicators of whether the network is dense or closed versus highly centralized provide insight about the relative efficiency of network structure as well as where power is distributed. Networks that have high centralization scores tend to have clear separation between the core and periphery. In their research focused on the social network of researchers on the Amundsen–Scott South Pole Station, Johnson et al. (2003) found that a core-periphery structure supported a better functioning of the group and enhanced consensus among team members. A network with a core-periphery structure has core members connected to each other and members located in the periphery of the network connected only to individuals located within the core (Borgatti et al. 2013).

In the context of transdisciplinary, problem-based research, a number of recent analyses of network structures support the assertion that there is an emergent spectrum of relational patterns that are influenced by factors including the phase of development of the research fields’ involved, inclusion of social scientists, maturation of the field, international campaigns such as UN thematic years, institutionalization of the network, the activities of founding members, and of key individuals in the network. Three phases have been identified in the establishment of social networks: exploratory, establishment, and development. Using co-authorship, collaborative work, and relationship data, Stewart et al. (2017) found that across the three phases of a research network on polar tourism, the network changed from a small, highly fragmented structure (122 researchers, 44 ties) to one double the size (279, 194). Over this time, the network exhibited decreasing density (0.006–0.005), with the exception of a small centralized five-member principal component (with betweenness centrality scores of 181.699–486.367). They concluded that collaboration was only moderately increased even though network membership had doubled, and that top contributors focused on recurring collaborations instead of expanding into new partnerships. In their analysis of the publications of polar bear researchers and partners over a 40-year period, Egunyu et al. (2018) demonstrated that among the 245 social and natural scientists, 10 researchers (7 natural, 3 social scientists) were vital to the non-fragmentation of the network (i.e., they are cut-off points). The network density was 0.021, and betweenness centrality scores of the 10 cut-off points ranged from 1029.893 to 17,799.443. Factions in the network were not discipline-based; instead, recurring collaborations among small groups containing both social and natural scientists occurred as in the polar tourism network. Finally, Hauck et al. (2016) used participatory network analysis among 72 stakeholders in agricultural biodiversity and land-use governance at local and regional scales in Germany. They concluded that institutionalization of ties (through money transfer, dependency, or regulating/policing an actor’s work) provided stability to network members. They also found that local actors (i.e. farmers) had higher betweenness scores and greater influence on all the network members as a whole. These relational patterns can be invoked by the researchers engaged in the work to demonstrate progress in long-term transdisciplinary sustainability science projects. These indicators can act as alternative metrics ahead of publication or other more conventional outputs to demonstrate the development of a robust network as the precursor to achieving additional transdisciplinary sustainability science goals.

Materials and methods

The unit of analysis for this research is the network of multiple actors involved in various projects taking place in Canadian inland deltas, which were part of the DDN. The DDN was established in 2014 to complement on-going research in three inland deltas across Canada. Beginning in 2011, researchers at the University of Saskatchewan engaged in research with Indigenous and non-Indigenous communities and partners in inland deltas, including the Peace-Athabasca Delta in northern Alberta, Slave River and Slave River Delta in the Northwest Territories and Saskatchewan River Delta on the Saskatchewan and Manitoba border (Fig. 1). This work built on existing partnerships in these deltas including: (1) the Slave River and Delta Partnership (SRDP), a collaborative research and monitoring network made up of indigenous, territorial and federal government partners and northern academic institutions focused on answering key community-driven questions about the health of the Slave River and Delta (NWT Water Stewardship); (2) the Peace-Athabasca Delta Ecological Monitoring Program (PADEMP) in the Peace-Athabasca Delta Region, a collaborative research and monitoring group comprised of Indigenous, territorial and federal government partners and non-government organizations, undertaking work on the Peace-Athabasca Delta (Peace-Athabasca Delta Ecological Monitoring Program); and (3) work by unaffiliated individuals in Cumberland House, Saskatchewan. Upstream development, including mining and agricultural activity, climate change and dams have cumulatively affected the predominantly indigenous communities who live in the deltas and continue to derive their livelihoods from the land. Numerous projects, some initiated as early as 2011 and before DDN was established, had been taking place in each delta.

Fig. 1
figure 1

Three Canadian inland deltas within the western Canadian provinces and territories

Full size image

In 2014, the DDN was established to intentionally bridge across the three inland deltas and their communities, building off existing partnerships and research initiatives in each delta. In 2011, researchers and government representatives began to engage with communities in the Slave River and Delta in the Northwest Territories, as well as the Peace Athabasca Delta in Alberta to build partnerships as precursors to work with the people in these regions on projects related to river and fish health. In 2012, community outreach to fishermen, trappers and outfitters began in the Saskatchewan River Delta in Saskatchewan, while work in the Slave River and Delta and Peace Athabasca Delta continued. The proposal for the DDN emerged in 2013 as a means to connect these existing, but independent, efforts and catalyze new projects across the three deltas. Ultimately funded in 2014, the DDN sought ways to bring the individuals involved in the multiple and continually developing and evolving projects. DDN had its own management group made up of community representatives from the three deltas who oversaw the work as well as a research advisory group comprised of researchers and community representatives. The DDN consisted of Indigenous government representatives, community members, the existing groups who provided insight into research needs and protocols in their delta and guided development of knowledge mobilization products and a team of interdisciplinary scholars including faculty and students, federal and territorial government staff. Four new research projects were initiated across the three deltas and efforts were started to build or strengthen existing collaborations with community members and researchers across the deltas. These projects included (1) efforts to document diverse perceptions of losses and gains from dam construction in the Saskatchewan River Delta; (2) the documentation of intergenerational perspectives, including Indigenous youth and elders, to identify more robust paths to socio-ecological resilience in the Saskatchewan River Delta; (3) understanding the barriers and opportunities to more effective knowledge-sharing strategies in the Slave River and Delta; and (4) the barriers and drivers that influenced the adoption of Aboriginal Navigation Flows in the Peace Athabasca Delta. During this same time, the other projects that had been initiated prior to DDN continued, and in some cases, new projects were also developed.

Data were collected from 16 key academic researchers, who were involved in the DDN. In an effort to capture the totality of the research happening in the deltas, each academic researcher was asked to record all collaborators and key outputs from the work in which they were involved between 2011 and 2017. Researchers were asked to recall these activities, and these data were triangulated with records of the activities that were kept as part of project management. Key items of outputs included conventional academic products (e.g. theses/dissertations, academic papers/articles, solo art exhibits, conference presentations, reports), as well as more explicitly transdisciplinary products (e.g. community presentations, community meetings, community engaged artistic exhibits, field visits, web pages, blogs, videos), management meetings and media hits. The types of output together with dates and locations of each activity were also identified. These data constituted the basis of developing the analytical network of interactions.

The analytical networks underpinning the analysis, were formed initially by linking DDN members to outputs or activities they participated in, as reported by academic researchers. An individual-event network is called an affiliation or a two-mode network. The DDN affiliation network of researchers connected to events was then transformed into a co-affiliation network—collaborator to collaborator, or a one-mode network. In the co-affiliation network, a node represents collaborators and the edges represent ties between collaborators—an edge or tie between two individuals indicates that the two were co-involved in at least one research output. This transformation was done using UCINET’s routine “Affiliations (2 mode to 1 mode)”, in which the minimums method was selected (Borgatti et al. 2013). The analysis focuses on the DDN network, which is analyzed at two distinct points in time. The 2014 network consists of all DDN members active at that time in the network (N = 63), whereas the 2017 network represents the DDN three years later, when it comprised of 218 members.

Valued datasets, including the strength of ties were used for the calculation of individuals’ centrality scores, core/periphery estimations, and clustering coefficients. These valued data were transformed into binary datasets for the calculation of centrality scores, centralization, density, average path, and diameter. Connections between any two actors can be characterized as valued, if the frequency or strength of interaction between the two individuals is measured or accounted for. Connections are referred to as binary, if a link exists between two actors.

Results

Evidence of diversity

In 2014, the DDN had 63 members—8% students, 40% faculty, 35% community members and 17% other organizations (including territorial and federal governments), co-producing 50 research outputs. By 2017 the DDN had reached a size of 218 members—15% students, 19% faculty, 41% community members, 1% management staff, and 24% other organizations, with over 300 research outputs produced (Fig. 2a, b). These results suggest that DDN maintained its diversity and saw a rebalancing of proportional representation over time.

Fig. 2
figure 2

a Delta dialogue network in 2014. b Delta dialogue network in 2017

Full size image

Evidence of integration and collaboration

To assess integration and collaboration, we examined the overall cohesion of the network. In 2014, the DDN was a sparse network, with approximately 13% of ties present in the network among its 63 collaborators (see Table 2). However, the network had areas of low and high density, as indicated by the overall clustering coefficient (57%). This meant that collaborators were active in relatively dense “local” neighborhoods. On average, each collaborator had eight connections (SD = 5.82), to seven other members (IQR = 11–5), if we look at the median number of connections for each DDN member. In this early stage of the network’s development, collaborators were “close” to one another, as indicated by the short average path length of 1.96. In 2014, DDN members were at a social distance of two degrees of separation, meaning that any member was only two steps removed from any other member in the network.

Table 2 Integration and collaboration measurements for 2014 and 2017 networks
Full size table

By 2017, the DDN had changed, with approximately 20% of ties present in the network among its 218 collaborators. However, the network increased in clumpiness, as indicated by the overall clustering coefficient. This meant that collaborators were active in highly dense local neighborhoods. On average, each collaborator had 43 connections (SD = 42.31) and was connected to 14 other members (IQR = 89–7), as indicated by the median.

Over time, the DDN evolved into a network characterized as a “small-world” with a high clustering coefficient and short distances between collaborators. The change in clustering coefficient indicated how much an individual’s collaborators were likely to collaborate with each other, and it represented the extent to which two collaborators co-participated in research outputs. The network was integrated, in the sense that all members were connected in one large component. The DDN in 2017 had a high overall weighted clustering coefficient of 0.84, compared to 0.57 in 2014. The clustering coefficient suggests that collaboration across the network increased over time.

Throughout the years, collaborators were less inclined to form connections only within their respective groups—academic and non-academic. While in 2014, there was more of a tendency for people to interact within their own group (Yule’s Q = 0.52), this tendency diminished in 2017 (Yule’s Q = 0.20). This was further tested using the following homophily hypothesis: the group a collaborator belongs to—academic or non-academic-affects who interacts with whom in the DDN. Results showed a very weak correlation of 0.08 (p value < 0.01), which did not indicate support for the homophily hypothesis. This is indicative of a relative level of balance among in-group and out-group ties for academic and non-academic participants. This is desirable from the standpoint that a transdisciplinary project would want to promote collaboration among respective groups over time.

When evaluating interactions between students, faculty, community members and other organizations, we noted similar trends in the homophily indicator (Yule’s Q2014 = 0.48 and Yule’s Q2017 = 0.42). We statistically tested the homophily hypothesis: the group a collaborator belongs to—student, faculty, community member, management and other organization—affects who interacts with whom in the DDN. Results showed a weak correlation of 0.17 (p value < 0.01), which did not indicate support for the homophily hypothesis. These findings suggest there were tendencies for people similar to each other to affiliate in the earlier days of the DDN, but this tendency diminished over time. In the latter years of the DDN, members moved outside of their respective groups (students, faculty, community and other organizations) for collaborations. This is a desirable outcome from a transdisciplinary standpoint because we would want to see mixing among different groups of participants.

When investigating network density by group, we hypothesized that non-academic members were less likely to engage in collaborations, than were academic members. Network density by group revealed that density scores were actually higher for non-academics than the academic collaborators. The network density scores among academic collaborators was 0.14 (1470 ties), while for non-academic collaborators it was 0.30 (4010 ties). This suggests there was more interaction within the non-academic collaborators in outputs than within the academic collaborators.

The level of engagement of any collaborator within the network can be assessed using degree centrality. This indicator represents the number of ties an individual has. To evaluate whether academic and non-academic members were engaged or active to differing degrees, we used a two-sample t test.Footnote 2 We specifically assessed if the mean degree centrality of non-academic collaborators was lower than that of academic collaborators. The difference between groups was found to be statistically significant (p value < 0.01); however, the non-academic group was the one that engaged most out of the two groups, with an average degree of approximately 52, compared to 34 for academic collaborators. One explanation for this was that the DDN placed greater emphasis on the non-academic outputs rather than academic publications in the early stage of its existence. This encouraged greater interaction among non-academics. In many respects, this is a desirable outcome, because it demonstrates a priority on community and other organizational outputs. This also provides an ethically justifiable rationale for not prioritizing academic publications. In all likelihood, if we were to repeat this exercise in the future, there would be more academic publications and interaction, which would shift the scores and the implications for the hypothesis.

Evidence of network stability

Findings indicate that in 2017 the DDN was a relatively stable network with some potential weaknesses (Table 3). For stability, a large network of collaborators is likely to require multiple leaders. There are various typologies of leaders in networks, however for the DDN, brokers were important in determining the flow of information. Their main function was to facilitate knowledge mobilization across the network. To attain network stability, it is preferable that distinct typologies of individuals are present. In 2017, the DDN had several members who had high betweenness centrality scores including members coded as 201, 205, 207, 206, 131, 304, 202, 228, 200, 203, 400, 101, 630, 232, 233, 234, 354, and 356. These were the primary individuals who made connections across the network. The stability of the network would be at risk if these individuals left.

Table 3 Network stability measurements for 2014 and 2017 Networks
Full size table

To assess for vulnerabilities or instability in the network, we looked for cut-off points, i.e.: points where, if these collaborators decided to leave the DDN, it would cause a disintegration of the network into multiple disconnected subgroups. Referring to Table 3, members coded as 202, 205, 228, 400, 630 were identified as cut-off points in the DDN in 2017. We also looked to vulnerabilities in brokers with similar structural equivalents, which could mitigate these potential risks. Potential risks are present and could be mitigated to some degree by the presence of other collaborators in the network with similar roles.

Brokers and cut-off points who have redundant structural equivalences can protect against the departure of another participant with similar ties. Two members are considered to be structurally equivalent if they have similar relationships to other members. We calculated the structural equivalents for all cut-off points and key brokers. We noted that there were certain key individuals who acted as equivalents for each other. For example, 202’s equivalents are 400, 131, 207 and 630; however, these members were themselves either cut-off points or important brokers. However, this vulnerability was diminished by other members, which could potentially mirror this members’ function in the network and include: 204, 200, 234, and 305. The structural equivalence analysis suggests that the network was relatively stable, given that there were some redundancies that protected against vulnerabilities of key members departing.

In the early stages of the DDN formation, the most influential collaborators were faculty members (205, 201, 206, 207, 218) or members of an organization (630), as indicated in Fig. 3. In 2017, influential individuals included students, members of other organizations and community partners (201, 207, 202, 205, 200, 203, 101, 131, 630, 400, 206, and 228). In the DDN, there were approximately 30 collaborators with high scores of betweenness centrality (ranging from 5.56 to 42.62 in 2014 and 1.77 to 16.52 in 2017). This illustrates that the DDN relied on multiple individuals, who were also representative of different groups (6 students, 12 faculty members, 1 management staff, 7 community members and 4 collaborators from other organizations). This demonstrates that the network grew in stability over time as it diversified the participants who were viewed as most influential.

Fig. 3
figure 3

Total Freeman Degree Centrality Scores of Collaborators in DDN 2014 (left) and 2017 (right)

Full size image

Evidence of efficiency and power sharing

In 2017, the DDN could be characterized as a moderately hierarchical structure, where collaborators interacted vertically in the network, with many individuals connecting to the same few others. As indicated in Table 4, degree centralization was 54%. This can be compared to the DDN in 2014, when degree centralization was 41% and the DDN was less hierarchical. At that time, DDN was a more sparse and clustered structure. Overall, the DDN could be characterized as having a core-periphery structure (r = 0.78). Values approaching 1.0 are more hierarchical. Ideally, we would like to see a mixture of collaborators in the core with representation from all areas to illustrate the potential for power sharing. We see this in 2017 compared to 2014. Based on this dichotomous core-periphery model, 13 collaborators were identified in the core out of 218 collaborators and we saw a mixture of academics and other organizations. In 2014, the DDN did not fit as well the core-periphery model structure (r = 0.56). The core’s composition in 2014 was mostly composed of faculty members (5) and few collaborators from other organizations (3). These findings suggest that the structure along with the composition of the core indicates a model for power sharing. The model does lean toward greater hierarchy and this suggests a vulnerability toward a concentration of power.

Table 4 Efficiency and power sharing measurements for 2014 and 2017 networks
Full size table

The degree assortativity coefficient provides information regarding the extent to which highly engaged collaborators form ties with other highly engaged members of the network. The degree of engagement was calculated using degree centrality scores. The coefficient values ranged between -1 and 1. Positive values indicate that individuals with high degree centrality scores interact with each other; negative values indicate that individuals with high degree centrality scores interact with individuals with lower degree centrality scores (Newman 2010). In core-periphery structures, we expect to see assortative mixing by degree among collaborators (Newman 2010). In 2014, there was no evidence of strong patterns in terms of degree mixing, as indicated by the coefficient’s value of -0.05. With time, the DDN shifted towards an increase in highly engaged collaborators mixing with others like them, however, even in 2017 the correlation remained weak (r = 0.10). Again, we see evidence of mixing among participants and a structure that reinforces that mixing.

Discussion and conclusions

Transdisciplinary work is characterized by collaboration with diverse partners who possess varied knowledge systems and epistemologies. A premium is placed on communication and coordination of effort that seeks to address traditional power imbalances between academic researchers and research partners (Kates et al. 2001; Crona and Parker 2012). Transdisciplinary sustainability scientists are also often focused on how to co-create usable knowledge in partnership with the communities with whom we work. This action imperative is often lacking in more traditional disciplinary academic work, which is characterized by different patterns of norms, rules, behaviors, and cultural practices that are driven by incentives to publish in venues among peer experts and enhance prestige indicators among peer groups and societies. Accordingly, those patterns for transdisciplinary scholars are likely to be different from those engaging in traditional disciplinary, multidisciplinary or even interdisciplinary research. But we do not have good methods or metrics to help make the case for evidence of research productivity in these emerging fields (Durose et al. 2018).

Our analysis revealed that the DDN did well on diversity indicators and evidence of integration and collaboration. Indicators for network stability and efficient power sharing suggest these are areas for continued attention. In our case study, diversity in the network grew over time with 63 members and 50 research products in 2014–218 members with 312 research products in 2017. In comparison to other transdisciplinary networks evaluated using peer-reviewed authorship and citation counts (Stewart et al. 2017; Dahlander and McFarland 2013), the DDN showed rapid movement through phases of exploration, establishment via collaboration and output creation.

The network became more integrated over time. In 2014, the network was sparse with areas of high and low density. By 2017, the network had more areas of higher density with integration demonstrated through great connectivity among members. In contrast, polar tourism and polar bear research networks were characterized by low or decreasing density. Further evidence suggested that DDN partners were participating outside their own groups.

While the network was also identified as stable in 2017, it was vulnerable given key cut off points and brokers. There was some redundancy in the network, but if key individuals left, then many ties would be broken resulting in future fragility in the network. Cut-off points pose vulnerabilities and have been observed in other transdisciplinary contexts as well (Stewart et al. 2017; Egunyu et al. 2018).

The structure of DDN in 2017 was more hierarchical than in 2014 and best described as a core-periphery structure. While not troublesome in its own right, these trends need to be watched carefully so that power sharing approximates the ideals of transdisciplinarity, in which decision-making is intended to be more equitable than traditional academic research. Interestingly, none of the core-periphery indicators indicate community partners as central; this could be influenced by a number of factors, including researcher-specific data contributions to this paper or a structure that did not result in equal power-sharing opportunities for communities. Other network structures in transdisciplinary contexts have been characterized as fragmented (Stewart et al. 2017); hierarchical (Egunyu et al. 2018), and egalitarian (Hauck et al. 2016). More research is needed to tease this out in greater depth, not only in this case, but in SNA more broadly.

The methodology and results presented in this article illustrate not only how transdisciplinary work may be evaluated, but what kinds of evaluative criteria are most applicable given the different intentions for transdisciplinary work. Research like this presented here begins to move us toward new norms that could be invoked as performance metrics for transdisciplinary sustainability science scholars. We have proposed and demonstrated how key criteria distinguish transdisciplinary work from more conventional academic research. These include integration and collaboration, the need for a stable social network over time and a functional structure that balances efficiency with power sharing. We suggest using SNA and these criteria as tools to illustrate whether knowledge co-production effort is effective. These short- and medium-term indicators provide evidence for capacity and network building that may serve as the steps toward more meaningful societal impact, which other authors have pointed to as important aspects for advancing the evaluation of transdisciplinary work (Belcher et al. 2016).

Key limitations in our approach include that we consulted only the researchers who were involved with the outputs. Had we solicited input from our community and other partners, we might have included more outputs of which we are not aware. A more robust research design would be more inclusive of those partners and avoid this sample bias. Additionally, we do not have an ideal, normalized or standardized model of transdisciplinary practice as expressed through SNA statistics against which to compare. The best we could do in this project was compare where we were in 2014–2017, and look to bibliometric evaluations and one example of participatory network analysis in other contexts. More research into social networks associated with transdisciplinary practice could begin to reveal different models and insights into the relative strengths and weaknesses associated with how social networks support transdisciplinarity.

Notwithstanding these limitations, assessment of early adopters of transdisciplinary research practice is important because it can inform new approaches and initiatives. The guiding ideas behind transdisciplinarity are that problem-oriented work in collaboration with partners will lead to more durable solutions to the significant sustainability challenges we face (Carew and Wickson 2010; Lang et al. 2012; Scholz and Steiner 2015). Knowledge co-production is a very different model for knowledge creation compared to conventional academic scholarship, but evaluating transdisciplinary work is challenging. Significant time must be invested in building the relationships to enable appropriate problem definition, collaborative research and progress toward solutions (Kates et al. 2001; Crona and Parker 2012). The process of exploration, establishment and development in building relationships in a network of collaborators for inter- and transdisciplinary problems as described by Stewart et al. (2017) is evident in this as in other contexts. While effective solutions would be the definitive indicator of success, “solutions” may take a long time to emerge and the concept may mean different things to different people involved. Intermediate indicators are needed as signposts to signal whether partners are on the right path. Identifying shorter- and medium-term outcomes and indicators can be used as proxies in the march toward longer term, sustainable solutions.

At a time when community-based research is becoming more popular in universities, what kind of alternative measurements can be valued as evidence of research? Evidence suggests that interdisciplinary and transdisciplinary projects initially result in fewer peer-reviewed publication in early stages, but more publications in the longer term (Hall et al. 2012). In the case of DDN, greater priority was placed initially on serving community members through research products that met their needs, while peer reviewed publications were not initially prioritized. For example, the DDN co-created a traveling art exhibit that documented change in the deltas that circulated among communities and government offices, developed policy briefs, and established community forums before turning to peer-reviewed academic outlets (Steelman et al. 2019). These patterns of publication run the risk of penalizing early career faculty, who need classical publications to demonstrate productivity. In all likelihood, the total number of peer reviewed publications coming out of the DDN work will increase over time, but intermediate indicators and rewards for community-oriented products are needed as incentives for faculty so as not to put them at career risk.

There is also well-deserved skepticism about whether such work leads to more meaningful impact. We suggest that in the case of the DDN, there has been evidence of change. Consider the following: the work of the DDN is associated with the creation of a newly formed Delta Stewardship Committee in the Saskatchewan River Delta (Patrick et al. 2018), that is leading in multiple, successful community-led funding application to federal and provincial agencies to do restoration and education work in the delta. Publications incorporated traditional knowledge into research reporting, as well as led to joint publications with Métis council members and other SRDP members in the Slave River and Delta (Das et al. 2015; Baldwin et al. 2018; Mantyka-Pringle et al. 2017). This evidence has provided data for continued advocacy related to concerns about ice cover progression, fish distribution, fish monitoring and the cumulative stressors in these ecosystems. In the Saskatchewan River Delta, two co-created and co-written articles contributed to locally-developed curricula in the Charlebois Community School (Andrews et al. 2016; Andrews et al. 2019).

Collecting SNA metrics over several years of transdisciplinary collaboration may provide evidence for how we reach the long term proposition of sustainable change. Without effort to create diverse, collaborative, structurally sound networks, the work of sustainability science will remain challenging. The metrics presented here help make visible the social processes that undergird the efforts to create outputs that serve as the building blocks for meaningful sustainability impacts.

Our research helped bridge a gap in the evaluation literature on transdisciplinary work that currently focuses either on processes or outcomes and individuals and groups (Wickson et al. 2006; Klein 2008; Brandt et al. 2013). While consensus on how transdisciplinary works should be evaluated will continue to evolve, we are contributing to the discussion by proposing new units of analysis—research outputs, individuals and whole social networks—that can allow for comparison across different kinds of transdisciplinary projects. Our work builds on the process-based approaches typified by Lang et al. (2012), Wickson et al. (2006), and Scholz and Steiner (2015) who seek to identify design principles. Our design principles—integration and collaboration, the need for a stable social network over time, and a structure that balances efficiency with power sharing—are functional and diagnostic. Selected metrics helped determine whether the network could be more or less integrated and collaborative, stable or democratic. Process to outcome based criteria, as advanced by Belcher et al. (2016), such as relevance, credibility, legitimacy and effectiveness, are dependent on the perspectives of those involved in the transdisciplinary practice so that transdisciplinary contributions can be assessed. Our approach—arguably less subjective—takes as given the research outputs and seeks to identify social structures that support their production. An important caveat is that SNA focuses on the structure of these relationships but remains silent on the quality of them. Our employment of SNA illustrates how progress can be made in how knowledge is jointly produced, how it can be improved based on those findings and how this kind of capacity building over time may provide evidence of the interim steps needed for more sustainability impacts. In light of pressing collective needs to address sustainability, harnessing the ability of academics to work in partnership with those outside our communities is but one way for us to place science in the broader service of society.