Introduction

Adaptive management has been recognized as a promising approach to inform and iterate decision-making and promote learning to achieve conservation and natural resource management objectives under uncertainty in diverse contexts [1••, 2]. Although there are many ways in which adaptive management has been interpreted [3•, 4•], a common thread among disparate approaches emphasizes learning by doing rather than managing systems as if they were static or so unpredictably dynamic that they preclude the possibility to learn. Theoretical expositions and frameworks for adaptive natural resource management (henceforth, adaptive management) are quite common, whereas documented examples showing successful implementation of adaptive management and learning through multiple cycles of decision-making have remained uncommon [3•, 5]. Proposed factors limiting implementation of adaptive management based on literature reviews are many and include level of organizational and financial support for the approach [6, 7], appropriate application of the approach [3•], concise and vetted problem definition [2], stakeholder involvement in tailoring adaptive management plans to local situations [5, 8], acceptance of the approach by practitioners [9], communication between scientists and managers [3•, 4•], and balance between planning and real-world action [10]. This leads to the question: Which factors are most limiting for successful implementation of adaptive management and how can they be overcome?

Viewing the adaptive management process through the lens of quantitative decision support (QDS) offers a way to examine and decompose this question. We define QDS as an approach to inform management choices by specifying ultimate management objectives, identifying at least two possible management strategies, and comparing the strategies using a predictive modeling procedure that includes at least one quantitative variable to represent the objectives. This definition complements definitions of decision support systems in the field of forest management (e.g., [11, 12]), which have emphasized all-in-one computer-based systems that usually require quantitative inputs. QDS need not be entirely computer-based and would therefore be suitable for application in regions of the world where computer resources are limited. We propose that QDS offers a transparent means to inform adaptive forest management (AFM) and thereby improve learning and achievement of management objectives relative to approaches that do not use QDS. The parameter values for the predictive modeling may be all theoretical, all empirical, or some combination of these as long as they are quantified. To increase the chances that QDS will inform real-world decision making, stakeholders and end users should be involved throughout the development process [13••].

QDS can be used to inform implementation of forest management, but published examples of cases are rare [6, 14, 15••]. One possible explanation for this rarity is the lack of documentation rather than a low rate of true uptake of QDS for AFM, and certainly better documentation of use vs. nonuse is needed. The reasons for the lack of uptake of QDS overlap with the reasons for the lack of implementation of adaptive management. Lack of sufficient involvement by stakeholders and end users is a commonly cited reason for the rare uptake of QDS in natural resource management (e.g., [13••, 16]). Developing an indicator of quality of adaptive management among tasks of QDS development and implementation would provide a means to learn about drivers of successful implementation, but metrics to construct such an indicator are currently lacking.

Recently, multimetric indicators have been developed to quantify impacts of management activities on ecosystem functions and services [17,18,19], for assessing participatory development of decision support systems [14] and for evaluating collaborative planning approaches to support natural resource management [20]. Using multimetric indices allows for comprehensively and consistently characterizing elements in a complex system, as component indices can be examined individually and in aggregate [19]. Investigating a total index score can account for simultaneous yet minor shifts in multiple subcomponents that would otherwise go unnoticed if examined individually. On the other hand, the quality of a system as a whole may remain unchanged while the quality of subcomponents shift in contrasting ways that cancel each other out. Such component dynamics could be crucial for improving understanding and decision-making in natural resource management. Therefore, examining both total scores and subcomponent scores of a multimetric index is important to obtain a comprehensive understanding of a multipart system. Developing and using a QDS to inform natural resource management is a highly complex and multidimensional endeavor [13••, 21, 22••], which warrants such a multimetric index approach to examining quality of the components and entirety of the QDS development and application.

Forests offer a particularly interesting study system for developing indicators of QDS quality for adaptive management and formal learning. Forests are managed at multiple spatial scales (e.g., stand, forest, and landscape level; [23]) and the majority is publically owned, but forests are becoming increasingly owned by private individuals [24]. Management of forests is challenged due to many sources of uncertainty and long turnover rates and time lags of woody productivity [25, 26]. Forest management is also a diverse subject that we define as any decision-making process that affects conditions in a forest. The conditions could be abiotic (e.g., temperature, precipitation, fire), biotic (e.g., tree growth, animal population demographics), or a combination of these [27]. Forest management actions can also be diverse (e.g., timber harvest, tree planting, protection against disturbance by recreationists, providing artificial nest cavities). Correspondingly, there are many types of QDS, which range from more academically driven dynamic optimization tools (e.g., [28]) to graphical interfaces that managers can readily learn and use on their own (e.g., Heureka; [29]). For reviews of decision support systems available to inform forest management, see forest DSS Community of Practice [30] and Segura et al. [31]. Existing reviews have pointed out prospects for QDS and formal learning to be useful to inform forest management in the face of uncertainties about climate change [23, 32]. Although the topic of learning through actual implementation of adaptive management has been reviewed [5], a comparative assessment of the quality of decision support to inform adaptive management is needed for a better understanding of the mechanisms limiting the application of adaptive management.

Our overarching objective is to identify the key gaps in the development and application of QDS that allow for the implementation of AFM to improve learning and achievement of management objectives in a changing world. Toward this end, we develop a multimetric index of quality of applications of QDS to inform AFM. We then conduct a structured literature review to evaluate them according to the selected attributes and the multimetrics index. Next, we utilize the index and component metrics to examine temporal and geographic variation in the use of particular methods and approaches, to enhance understanding about any changes or patterns in the comprehensiveness and application of QDS to inform AFM. The geographic analysis enables us to identify broadscale gaps in the application of QDS and AFM. We then discuss the evolution of the interface between QDS and the application of AFM, along with complementary cultures of decision support and forest management that could learn from each other in terms of best practices. We use regression models to examine a priori hypotheses, and we use an exploratory cluster analysis to generate ideas for future research on adaptive management that complements the vast existing literature.

Methods

We hypothesized that the quality of case studies using quantitative decision support to inform adaptive forest management has varied across time (temporal hypothesis), among global geographies (geographic hypothesis), and among and within tasks of QDS development and application (task hypothesis) or has been constant across time, space, and tasks (null hypothesis). To examine these hypotheses, we identified relevant case studies in the literature and developed a method for scoring their quality.

Case Study Selection

To ensure relevant case studies were included in the analysis and that a suitable number of papers were selected, we used two rounds of literature search within the Web of Science database on 10 August 2017. In designing the selection process, we followed the principles of systematic literature review [33, 34], with some simplifications to keep the process feasible. In the first round of searching, our goal was to identify a series of keywords that reflected our main objective and assess the number of matching papers for multiple combinations of keywords. Toward this end, we conducted a series of exploratory keyword searches with increasing specificity (Online Resource 1). For example, adaptive forest* management yielded 1429 matches, whereas refining these with AND “decision support” yielded 54 papers. We aimed for a close fit to our main objective while keeping the search as broad as possible to allow for any application of adaptive forest management, not limited to any particular definitions of adaptive management [3•, 4•]. During the first round of searches, we found that there were over 1400 papers that appeared to focus on adaptive forest management, and of these, < 300 focused on modeling, < 200 focused on learning or dynamic optimization, < 60 focused on decision support, and < 50 focused on Bayesian updating.

In the second round of searching, we developed a final set of search keywords (Table 1), which were chosen based on our main objective, the first round of searches, and deliberation among coauthors. Our goal in this final search was to identify approximately 200 papers, which would be a feasible number of papers for the coauthors to review. We also had a goal of selecting at least 30 case studies, and we believed this would be possible from a set of 200 papers identified via the keyword search. When combining the final set of keywords with an OR operator, 188 papers were identified. Of these matching papers, nine were classified as review papers in the Web of Science database and were removed from our list.

Table 1 Final search criteria used to identify relevant papers for the literature review on quantitative decision support to inform adaptive forest management. Asterisks (*) indicate wildcard characters, quotes (“) surround exact phrases, and numbers at the end of each row indicate the number of papers matching that set of criteria in the Web of Science database on 16 December 2017. Unless otherwise noted with italics, search terms were applied to the titles, abstracts, and keywords within the database
Full size table

We then read the title and abstract from each of the 179 candidate papers identified from the final set of search keywords to determine if it would be selected for inclusion in the analysis. In particular, we determined whether each candidate paper included a case study that developed and described QDS in a forest management context (for definitions of QDS and forest management, see “Introduction”). In some cases, we read the candidate paper if we could not determine its relevance with confidence based on reading the title and abstract alone. To examine the recent trends, we included papers published between 2005 and 2015 in the final analysis.

Attributes and Scoring Selected Case Studies

To score the quality of QDS to inform AFM in each selected case study, we identified 23 attributes comprising three main tasks of adaptive management, including conceptual setup (n = 5), modeling (n = 10), and application (n = 8; Table 2). We further distinguished these into subtasks, including definition of objectives and management options (setup); specifying uncertainty, prediction, and optimization (modeling); and stakeholder involvement along with practice and learning (application). We recognize that some subtasks can be done simultaneously (e.g., stakeholder involvement and defining objectives), but each subtask on its own represents an important aspect of adaptive management. We are aware of the many definitions of adaptive management (e.g., [7, 10, 42,42,44]), which to a large degree represent the inherent flexibility in how the approach is applied in a given context [10] but also owing to fundamental differences in approaches [4•]. We have chosen the tiered system of tasks and subtasks to account for these diverse approaches while facilitating our analysis and to help ensure that our framework could be extended to other domains beyond forest management with varying levels of complexity. The three main tasks we identified are largely consistent across approaches, whereas the subtasks within modeling are more specific to a decision-theoretic school of thought [4•]. In addition to attributes for scoring, we also classified each selected case study according to the location and climatic zone (Table 2). These classifications were not used for the scoring, but rather to examine the geographic hypothesis by comparing scores among locations and climatic zones.

Table 2 Attribute descriptions and scoring schemes for a multimetric index applied to case studies using quantitative decision support (QDS) to inform adaptive forest management. Descriptions of points assigned to the attributes (numbered) within subtasks (lettered) and tasks (Roman numerals) of adaptive management are provided, and wherever none of the conditions were met the score was set to 0
Full size table

For each attribute describing a task of adaptive management, we assigned a score from 0 to 10 representing low to high quality for that attribute (Table 2). Assignments of quality were subjective, and we used our collective judgment and logic in designing the scoring criteria. For example, stakeholder input on the objective function was assigned 5 points if stakeholders (or literature produced by stakeholders) were consulted on the structure or parameterization. If stakeholders were consulted on both structure and parameterization, then 10 points were assigned. We reasoned that structure and parameters of the objective function are equally important and additive in contributing to this criterion.

The scores were readily assigned based on simple attributes except the quality of recommendations, which was scored according to the level of conciseness of the text and graphics representing the suggested course of action given the QDS inputs. For example, a case study received a higher score if it presented a clear graph showing how predicted outcomes (in terms of the objectives) change among management options. We also assigned a higher score for quality of recommendation if there was a clear one to two sentence summary of how the results could be used to inform decision-making. Such concise descriptions would be interpretable by less technical stakeholders and decision-makers, and the quality of the recommendation is an important indicator of successful knowledge transfer between science in practice in the context of QDS and AFM.

Recognizing the importance of balanced emphasis among attributes of tasks and the tasks themselves, we assumed equal weight among attributes within tasks and equal weight among tasks when computing scores for tasks and total scores. In particular, we summed attribute scores within each task, and each sum was standardized on a scale from 0 to 10. Likewise, we summed task scores and computed a total score that was then standardized on the 0 to 10 scale.

Statistical Analysis

We used linear regression to examine our hypotheses about spatiotemporal variation and differences among tasks and subtasks regarding quality of QDS to inform adaptive forest management, after confirming that the model assumptions were met (i.e., normally distributed residuals and homoscedasticity). Each model included the index of quality as a response variable, along with one of the following predictor variables: year (continuous), continent, task, subtask, or attribute. We used an alpha level of 0.05 for determining statistical significance and Tukey’s honest significant difference test to conduct pairwise contrasts of means. We also conducted k-means cluster analysis to explore similarities within groups of case studies across scores for subtasks of adaptive management. To determine the number of clusters for this analysis, we examined the frequency of optimal numbers of clusters across 30 independent methods, setting the maximum possible number of clusters to 10 to ensure feasible interpretation [45].We used program R for all statistical analysis [46].

Results

Based on a structured literature review, we selected 34 case studies, each of which used QDS to inform adaptive forest management and was published between 2005 and 2015 (Online Resources 2 and 3). Across these case studies, there were 25 unique lead authors and 1–5 case studies published per year. Two of the 34 case studies reported that QDS was used to inform real-world AFM [15••, 47••]. The majority of case studies (n = 21, 62%) modeled a single group of ecosystem services (ESs), and more case studies modeled two or more groups of ESs (n = 8, 24%) than those that modeled none (n = 6, 18%). Provisioning services were modeled by the majority of case studies (n = 24, 71%), followed by regulating (n = 14, 41%) and cultural services (n = 1, 3%). The most frequently modeled group within provisioning services was “biomass” (n = 23), and the most frequently modeled group within regulating services was “lifecycle, habitat, genetic” (n = 9; Table 3). A cultural service (i.e., recreation access) was only modeled in one case study [55].

Table 3 Numbers of case studies where ecosystem services (ESs) were modeled to inform adaptive forest management 2005–2015.  ES classification is based on CICES [35] version 4.3
Full size table

Temporal Trends

Based on the attributes of the selected case studies, the overall score of AFM declined by 35% from 4.5 ± 0.7 to 2.9 ± 0.7 between 2005 and 2015 (Fig. 1). The score of the modeling task also declined during this period by a similar amount from 4.8 ± 0.7 to 3.3 ± 0.8. We detected no other statistically significant temporal trends in scores for the remaining tasks nor subtasks of adaptive management, although the parameter for the year effect on score was negative for all subtasks. Notably, the number of case studies published per year (range 1 to 5) neither increased nor decreased significantly. Scores for prediction and management options had the greatest interannual variability (0 to 9), compared to the other subtasks (range of ca. 5). We also found that five of six case studies with an overall score ≥ 5 modeled multiple ecosystem services and used stochastic dynamic programming to evaluate management options.

Fig. 1
figure 1

Changes in scores of overall quality of quantitative decision support (a) and quality of prediction (b) to inform adaptive forest management in 34 case studies published 2005–2015. The solid line is the best fit to the data based on a linear regression, and dashed lines represent the 95% confidence limits

Full size image

Geographic Patterns

Case studies were either globally relevant (n = 6) or focused on forest management in one of the following four continents: Asia (n = 2), Australia (n = 7), Europe (n = 9), or North America (n = 10). Place-specific case studies each covered one (n = 22), two (n = 2) or three (n = 1) of seven climatic zones, including temperate (n = 20) followed by cold (n = 5), arid-steppe (n = 2), and tropical-rainforest zones (n = 1; Online Resource 4). With the exception of two papers focused on China and Costa Rica, respectively, place-specific case studies focused on forest management in developed countries (Online Resource 4). All place-specific case studies focused on forest management within a single country, except for one that focused on forests in parts of the USA and Canada [55].

The overall score of adaptive management did not vary significantly among the three continents having ≥ 7 case studies (i.e., Australia, Europe, and North America). When examining the raw average scores, none of the average scores for a given geography exceeded those of all other geographies across all subtasks (Fig. 2). Subtask scores for Asia were less than or similar to those of the other continents, except for the management options task. Australia had similar or greater scores for each subtask, except for optimization. Compared to other geographies, case studies in Australia and North America had higher scores for prediction, stakeholder involvement, and practice. Scores were most similar for these two continents. Globally relevant case studies had a higher optimization score compared to place-specific case studies.

Fig. 2
figure 2

Scores for 34 case studies (published 2005–2015) using quantitative decision support to inform adaptive management by subtask among focal continents (or global focus)

Full size image

Variation within and among Tasks

When comparing scores among the three tasks, the application task (stakeholder involvement and practice) score (1.8 ± 0.7) was nearly 25% less than those of the modeling or setup tasks (4.0 ± 0.7 or 4.7 ± 0.7, respectively). When comparing subtasks, the scores for objectives, options, and predictions exceeded those for uncertainty, stakeholder, and implement (Fig. 3). Except for uncertainty, which had a lower score, the subtask scores were balanced between modeling (predictions and optimization) and the conceptual setup tasks (objectives and options).

Fig. 3
figure 3

Scores for quality of case studies using quantitative decision support to inform adaptive forest management by subtask in each of the 34 case studies published 2005–2015. Dots are averages, whiskers are 95% confidence intervals, and differing letters between means indicate a significant difference

Full size image

Average scores for individual attributes ranged from 0.3 to 7.4, and we detected significant pairwise contrasts between attributes within each subtask except for uncertainty and stakeholder (Fig. 4). Within the objectives subtask, the average score for temporal horizon exceeded that of ES classes (attributes 3 vs. 1). Considering the options subtask, the average score for management intervals was greater than that of stand level (attributes 5 vs. 4). In the predictions subtask, the average score for model visualize (attribute 11) was lower than both stochastic and parameterize (attributes 10 and 12), and there was no significant difference between these higher scoring attributes. Within the practice subtask, the average score for management recommendation (attribute 23) exceeded that of the remaining attributes (20 through 22), which included applied QDS, applied model update, and demonstrated model update.

Fig. 4
figure 4

Scores for quality of attributes within 7 subtasks (bold x-axis labels) and 3 tasks (gray boxes) of quantitative decision support to inform adaptive forest management in each of the 34 case studies published 2005–2015. Dots are averages, whiskers are 95% confidence intervals, and differing letters between means within a subtask indicate a significant pairwise difference

Full size image

Cluster Analysis

The majority of indices indicated that two clusters are optimal when considering the variation in scores among subtasks (Online Resource 5). Average overall score did not differ significantly between the two clusters, but differences were observed when comparing the mean scores for subtasks in the exploratory analysis (Fig. 5). Compared to cluster 2, cluster 1 had higher scores for stakeholder (6.0 ± 1.1 vs. 0.5 ± 0.8) and prediction (9.2 ± 1.4 vs. 4.5 ± 1.1) but a lower score for optimize (2.6 ± 1.4 vs. 5.1 ± 1.1).

Fig. 5
figure 5

Scores for quality of quantitative decision support to inform adaptive forest management by subtask in each of the 34 case studies published 2005–2015. Radar graph depicts average scores for each cluster identified by cluster analysis, average scores across all case studies, and scores for individual case studies with the highest and lowest scores across all attributes

Full size image

Discussion

We developed a multimetric index of quality of applications of quantitative decision support to inform adaptive management and applied this through an intensive structured literature review of case studies on adaptive forest management. The index provides not only a total score for a given case study but also scores for each of the main tasks, subtasks, and attributes of developing and applying QDS to inform AFM. This way the index measures the multiple dimensions of success of specific applications and can reveal important contrasts and synergies among these tasks as they contribute to quality of the application as a whole. For example, AFM could be implemented based on a poorly developed QDS, and therefore, the effort could be considered successful at implementation but having a low potential for learning (due to the poorly developed QDS). Our multimetric index is designed to capture such contrasts within a given case study. We therefore argue it is important to consider task-specific scores in addition to total scores to have a more complete picture of QDS for informing adaptive management.

Temporal Trends

We discovered that over the past decade, overall quality of QDS to inform AFM declined by 33% from 4.5 to 3.0 of 10, which supports the temporal hypothesis. Quality of modeling (especially optimization and prediction) associated with QDS for AFM has also declined during this period, while metrics related to uncertainty have remained low (< 5 of 10). Declines in quality of modeling within QDS for AFM have not been balanced with improvements in other tasks, for which there are at least three possible explanations. First, the decline may be related to reduced per capita funding for developing QDS for AFM. This hypothesis stems from observations of stable to declining funding available for environmental research in general within the USA [56, 57], along with an increasing number of publications in the field of adaptive management [4•]. As a result of this decoupling, environmental research institutions and researchers may have shrunken the time and resources they devote toward application-oriented modeling activities. Another hypothesis is that the per capita funding for research supporting adaptive management has remained stable, but researchers in this area are developing models that are increasingly complex but less comprehensive when it comes to developing QDS to inform AFM. A third hypothesis is that there may be a growing publication bias toward publishing novel tools and approaches, which has pushed researchers to publish early iterations of QDS rather than invest in demonstrating actual implementation.

Whatever the mechanism, our findings indicate that contemporary QDS to inform AFM is less intensive and more qualitative than their predecessors. We argue that this comes at a cost of less transparent decision-making for natural resource management and undermines the ability of researchers to contribute a critical task for adaptive management. Despite a large interest in adaptive management, as evidenced by many review papers on the subject (10 published 2006–2014; cited in the first paragraph of the “Introduction”), we hypothesize that research funding priorities have drifted away from comprehensive modeling needed to inform adaptive management. Such a shift could have been spurred by a belief that rare implementation of adaptive management is the fault of the approach itself rather than how QDS is developed to inform it [10]. As such, the underlying mechanism for this hypothesized decline in per capita funding may be that the number of such researchers in this field has increased faster than total funding for such efforts. We are unaware of any peer-reviewed study that has examined trends in funding and researchers in the field of environmental science (including QDS and AFM), but there is some supporting evidence from gray literature on recent trends in funding for environmental science in the USA [56, 57]. We are unaware of any database that tracks funding nor numbers of researchers in the realm of QDS or AFM, although this would be a useful tool for future research.

Variation Within and Among Tasks

Ecosystem services relevant to management objectives being modeled in the selected case studies lacked diversity, as the strong majority of case studies have focused on biomass as a provisioning service. Case studies focusing on regulating and especially cultural services are needed to demonstrate the potential of QDS to inform AFM addressing a broader suite of ESs, which is relevant for the United Nations Sustainable Development Goals [37].

We confirmed the expectation that there has been a substantial lack of stakeholder engagement during QDS development tasks and on-ground implementation of adaptive forest management. Aligning with the reviews of adaptive natural resource management [3•, 5], reported on-ground application of QDS to inform AFM was rare (2 out of the 34 case studies). We were therefore unable to examine relationships between successful implementation and other tasks such as stakeholder involvement. The question of which factors are most limiting for successful implementation of adaptive forest management remains an important area of future work, and examining patterns in scores within our multimetric index helps to generate hypotheses about the limiting factors. For example, the spatial scale of management options considered by the reviewed case studies was often broader than stand level, which could limit the real-world application of QDS. Furthermore, many case studies lacked a visualization of the structure of the modeling framework(s) embedded within the QDS. Successful uptake of QDS by forest managers likely depends on their understanding of it, which would be supported by clear diagrams illustrating the logic and key parameters.

Our finding that stakeholder involvement and practice occurs in a minority of decision support applications is supported by previous literature reviews (e.g., [5]). Of the 108 case studies on adaptive comanagement reviewed by Plummer et al. [8], only 17 were focused on forest management. They found that stakeholder participation was the most commonly cited factor contributing to the potential success of adaptive comanagement, whereas this was one of the less commonly cited factors contributing to actual success. They also reported that conflict of interest among participating stakeholders was the most commonly cited factor leading to failures of adaptive comanagement. When asked about suggested guidelines on developing decision support systems to inform forest management, stakeholder involvement was one of the most frequently voted suggestions according to experts as part of another study [58]. Although caution must be taken when applying adaptive management in contexts with stakeholder conflicts, resolution of conflicts was the most commonly cited outcome of these processes. They also found that learning was the most common emergent property of adaptive comanagement. These findings highlight the importance of future research that compares and evaluates approaches to stakeholder involvement in adaptive management with respect to outcomes for learning and management objectives.

On a more positive note, the quality of considered management options, predictive modeling, and optimization in QDS has been high in many cases. There has also been a general balance between the quality of the conceptual setup task and modeling task of QDS to support AFM. Although examples of implementation were lacking, several (n = 5) of the selected case studies demonstrated theoretical updating of model parameters for adaptive management. Such illustrations can facilitate the use of QDS to inform actual implementation of AFM.

Geographic Variation

Our ability to evaluate the geographic hypothesis was limited, as none of the reviewed case studies were in developing regions. This finding is consistent with a literature review on adaptive comanagement, which included case studies that incorporated learning by multiple actors collaborating on implementing adaptive management in an environmental context [8]. In contrast with our sample that only included one case study from Asia, published case studies on adaptive comanagement across all ecosystem types (published between years 2000 and 2010) were relatively common in Asia, some were in Africa and South America, and a small minority was in Australia. Further work is needed to understand and address geographic gaps of published case studies using QDS to inform AFM.

Based on our analysis, applications of QDS to inform AFM in Australia and North America were more successful with practice compared to those in other continents. By contrast, case studies from Asia were strong in defining management options. This could provide a basis for intercontinental working groups to exchange knowledge and support improved applications of QDS to inform implementation of AFM.

Cluster Analysis

Our exploratory cluster analysis revealed two clusters of case studies. One cluster performed relatively well on optimization, but it performed relatively poorly on prediction and stakeholder involvement. This cluster fits well under the decision-analytic school of adaptive management, with a greater emphasis on modeling compared to application and practice. Despite their differences in quality of subtasks, the two clusters were similar in terms of overall index of quality. An emergent hypothesis, then, is that resources may be limiting the merging of these schools of adaptive management (sensu [1]) to enable both high-quality optimization and high-quality application to overcome the current ceiling of moderate-quality QDS to inform AFM. Additional resources could allow for a new school of adaptive management that mitigates these tradeoffs in a more balanced approach that improves quality of all subtasks. Our scoring method provides a transparent way of monitoring existing and emerging schools of adaptive management.

Bayesian Decision Networks

We argue that Bayesian decision networks (i.e., Bayesian belief networks containing at least one decision node and one utility node) [15••] are useful yet underused for QDS. A Bayesian decision network (BDN) is a parameterized influence diagram (i.e., visual representation of a predictive model that links management objectives to decision alternatives and possibly factors beyond direct control of managers), allowing for explicit representation of uncertainty in the relationships between factors that are usually represented by conditional probability tables [15••]. The expanded use of BDNs could form the basis of an emerging school of thought that would bring together the more application-focused resilience school with the more optimization-focused decision-theoretic school of adaptive management. We postulate that harmonizing these two schools of thought could help to slow or reverse the declining quality of QDS to inform AFM. Another advantage is that BDNs can be designed in a way that is intuitive for nontechnical stakeholders to visualize and understand how their inputs are used to form the structure and content of QDS that informs decision-making ([59]; B.J. Mattsson, unpublished data).

Despite the known advantages of this tool, only one of the 32 case studies explicitly used a BDN as part of QDS [15••]. As with any computer-based decision support method, using a BDN carries the risk of a management team getting bogged down in planning and computing rather than on-ground action and learning. BDNs do, however, provide a user-friendly and customizable platform for planning and learning in the context of adaptive management that can be applied to many possible contexts within and beyond forests [15••, 60, 61]. There are many applications that users can use to construct simple BDNs free of charge (e.g., Netica, Norsys Software Corp, Vancouver, British Columbia, Canada; http://www.norsys.com). While having the ability to serve on its own as QDS, a BDN can also be embedded in more complex decision support systems that include a geographic information system or that offer multicriteria decision analysis tools (e.g., Ecosystem Management Decision Support; http://emds.mountain-viewgroup.com/explore). A BDN can also serve as a starting point for dynamic algorithms to inform iterative decision-making [6, 62]. Increasing interest and demand should lead to a community of practice that will allow for the development of open-source BDNs that are freely available for more widespread and advanced use (see for example [38]). Four case studies used a Bayesian belief network (i.e., BDN without decision or utility nodes) [6, 47••, 59, 63], and 10 presented an influence diagram to represent a stochastic optimization (e.g., [64]). Extending these existing applications to specifying a BDN would require a rather small additional time investment, with a distinct advantage of providing a more transparent means of linking QDS to real-world decision-making.

Conclusion

Our study represents the first comprehensive assessment of the quality of QDS to inform adaptive management, and our framework is extensible such that it can be applied to decision contexts outside of forest management. We have also demonstrated for the first time how cluster analysis and radar charts can be useful for exploring patterns according to the seven subtasks of QDS development and application for adaptive management. We found these techniques to be useful for generating new hypotheses about how the multidimensional quality of QDS for AFM has evolved and might continue to vary over time and space. This approach would be useful to inform future research agendas to ensure that key knowledge gaps are being filled, for example addressing how QDS can better integrate stakeholder input and be implemented on the ground as part of AFM. An expanded literature review would enable sufficient sample sizes to formally evaluate these hypotheses. This further work is important to quantify relationships between scores for actual implementation and tasks related to QDS development, which will provide insight into the usefulness of the index to examine drivers of successful implementation of AFM. To minimize bias, scoring of attributes should be conducted independently by multiple experts and then discussed before assigning a final score. Another area of future research is a controlled experiment where adaptive management teams are assigned restrictions as to the levels of emphasis on particular subtasks (e.g., stakeholder involvement and predictive modeling). We encourage any research that critically examines what works and what does not work in the field of QDS and adaptive management, so that we can more quickly learn and achieve conservation and natural resource management goals.