Introduction

Building from relationships between pattern and process, landscape sustainability science is a place-based means to understand and improve the dynamic interactions between ecosystem services and human well-being under changing conditions and uncertainties (Pearson and McAlpine 2010; Musacchio 2013; Wu 2013). Focusing on the sustainability of a landscape requires assessing how processes interact over time so as not to compromise the benefits they provide. Whereas there is strong consensus regarding the need for sustainability, there is little agreement about what method or metrics should be used to measure it. A literature review of sustainability indicators for agriculture identified 294 unique metrics (Rasmussen et al. 2017). Hundreds of additional metrics have been proposed to measure sustainability associated with specific processes and commodities (e.g., Ashok et al. 2017; Genovese et al. 2017; Carlsson et al. 2017; Smith et al. 2017; Wheaton and Kulshreshtha 2017) and applied to more than 460 different products (EcoLabel Index 2018; Yokessa and Marette 2019). While Vörösmarty et al. (2018) caution that businesses are criticized for “green washing” their sustainability ratings unless a standard method is adopted, attempts to standardize approaches for measuring landscape sustainability have been frustrated by the diversity of contexts and issues and sometimes result in laundry lists of guidelines rather than specific methods that enable consistent measurement and unambiguous results. Identifying effective practices to meet long-term objectives is a key challenge in developing an approach that guides practical decisions about landscapes (Williams and Brown 2012; Opdam et al. 2018). Changing environmental, social, and economic conditions make this effort especially difficult. Addressing this challenge requires multiple disciplines and systems integration (Jordan and Davis 2015)—including simultaneous incorporation of human and natural components, quantification of underlying effects and feedbacks, and consideration of multiple spatial and temporal scales (Miller et al. 2014; Liu et al. 2015).

Our goal in this paper is to provide a structured approach that is useful in quantifying progress toward landscape sustainability and helping users advance toward shared goals. This approach is an evaluation that identifies essential features of the landscape and what needs to be done to maintain its distinctive qualities. The assessment approach is meant to be adaptable to diverse situations, each with unique constraints, opportunities, and stakeholders with distinct values and priorities (Johnson et al. 2003; Ness et al. 2007; Sydorovych and Wossink 2008; Cebrián-Piquerasa et al. 2017). The proposed approach synthesizes insights from literature and our own experiences with resource management in landscapes where agricultural production is a dominant factor. This approach evolved through decades of trial and error in programs designed to measure and achieve specific goals. We provide examples that underscore the difficulties that practitioners encounter when they attempt to assess landscape sustainability. Our ultimate objective in sharing this approach is to support a community of practice that shapes “living” landscapes to maximize potential benefits while minimizing negative impacts (Palmer 2012; Dale et al. 2015; Muneepeerakul and Castillo-Chavez 2015; Vörösmarty et al. 2018; Opdam et al. 2018).

A six-step process for assessing progress toward landscape sustainability

The landscape sustainability assessment process is triggered by a need to evaluate a production system, product, technology, investment, policy, or other discreet intervention (herein called the activity) designed to meet specified societal needs or maintain environmental benefits. Indeed, determining for whom or for what sustainability is intended is a key question that must be addressed for the concept to be implemented (Wiens 2013). An interested party, herein called the coordinator, initiates the assessment and engages relevant stakeholders in the landscape sustainability assessment process within specified context and goals (e.g., Cebrián-Piquerasa et al. 2017; Efroymson et al. 2013; Petersen et al. 2018). The coordinator may be a government official, scientist, or a representative from industry, local community, or a non-governmental organization (NGO). The coordinator organizes and oversees the assessment process and, to be successful, should be trusted by stakeholders (Beardsley 2011) or, at minimum, be considered an “honest broker” (Varner 2014). The coordinator drafts the sustainability assessment objectives and context, which establish the spatial and temporal boundaries of the process and identify major stakeholders. These objectives and context may be modified during the assessment process, for learning about the system occurs as the “living” evaluation is deployed.

Each of the six steps of the landscape sustainability assessment approach requires actions and decisions illustrated by the decision-making cycle shown in Fig. 1 and detailed in the following subsections. While several steps can be initiated simultaneously, finalization of later steps depends on completion of the prior steps. Iterations within and among the steps are common, as each step is influenced by new information. And the entire circuit may be reinitiated as goals are adjusted in light of changing conditions, priorities, and available information toward agile decision making. The decision-making cycle is underlain by five essential themes discussed in a final section of this manuscript: stakeholder engagement, timely and effective communication, transparency and trust, monitoring, and continual improvement (Box 1).

Fig. 1
figure 1

Six steps and cross-cutting insights for assessing progress toward sustainability goals

Full size image

Define scope

Defining the objectives for an assessment requires clarity about the plans and goals for the activity, the landscape assessment process and stakeholders and their aims. The temporal and spatial extent (i.e., system boundaries for the assessment) are largely determined by the location and the extent of costs and benefits coming from the activity. Identifying boundaries can be difficult in practice, particularly if the area supports a production system (such as agriculture) and products or wastes are exported beyond the area of production (e.g., see Galik and Abt 2016). A telecoupling framework has been proposed by Liu et al. (2013) for considering causal connections or correlations between environmental phenomena that occur across large distances. Understanding past trends, current conditions, and potential future conditions for the system is essential, as is identifying expected decisions that will be most relevant to the assessment (Efroymson et al. 2013; Petersen et al. 2018).

The context of the assessment process strongly affects the choice, measurement, and interpretation of indicators and other measurements that provide information about potential or realized effects of human activities on environmental, social, or economic phenomena of concern (Heink and Kowarik 2010; Efroymson et al. 2013). Other context considerations include the specific purpose of the assessment, the distribution system(s) associated with any production systems in the landscape, policy influences, stakeholders’ values, baseline attributes, information availability, spatial and temporal scales of interest, material and energy flows through the system, and the ecosystem services that might be affected (Efroymson et al. 2013; Dale et al. 2018a; Eichler Inwood et al. 2018). The land area of focus may be the region that supports the cultivation and refining of specific products or the area expected to be affected by a proposed new production system or intervention. Given water’s essential role in supporting life on earth, watersheds are useful units of landscape analysis because effluent characteristics at the outlet reflect the integrated effects of management decisions made throughout the contributing land area (e.g., Ice 2011; Williams and Brown 2012). Throughout this initial step of defining the scope, there is a need to engage stakeholders in ways that help them understand how activities within a local landscape interact with and form part of global phenomena that affect and are affected by that landscape (Opdam et al. 2018).

Prioritize indicators

Appropriate indicators are needed to assess environmental, social, and economic aspects of the activity. Determining indicators of impending critical transitions between states is one of the major challenges in the science of sustainability (Levin 2013). Indicators can be selected to achieve multiple goals such as (1) providing useful information that addresses priorities of stakeholders and decision makers, (2) capturing relevant costs and benefits across the entire landscape, (3) facilitating measurement, or (4) supporting corrective actions (Cairns et al. 1993; Dale et al. 2015; Iacovidou et al. 2017).

A challenge faced when selecting indicators is to balance the desire to be comprehensive and the need to keep the process doable within limited constraints of time and budget while addressing inputs from diverse interests. The selected indicators should avoid redundancy and represent a minimum set of metrics required to address key concerns of stakeholders (e.g., Rasmussen et al. 2017). By selecting a few indicators that are sufficient to meet specific goals, the assessment process can avoid time and costs that would be incurred by adopting a longer list of indicators. However, oversimplification by having too few metrics can omit significant effects or situations that make it difficult to interpret and validate the information obtained (Landres et al. 1988).

Selecting indicators using an established framework (Fig. 2) (1) contributes to clarifying and building consensus around stakeholders’ goals for a desired future outcome, (2) ensures that important stakeholder concerns are considered in the indicator-selection process, (3) identifies a suite of indicators that is appropriate for the context of the analysis, (4) yields a high amount of information relative to the cost of obtaining relevant data, and (5) leads to higher likelihood of being applied and monitored over time (Dale et al. 2015; Garrick et al. 2017). Measurement is facilitated by selecting indicators that are practical, reliable, and provide timely information (Dale and Beyeler 2001).

Fig. 2
figure 2

Adapted from Dale et al. (2015)

Framework for selecting sustainability indicators. Components of the indicator selection process are shown in dark orange. Light-colored shapes are aspects of the assessment process that use the indicators or support their selection.

Full size image

To facilitate the process of selecting indicators, a checklist of categories of indicators was developed to reflect potential environmental (McBride et al. 2011), socioeconomic (Dale et al. 2013), and management (Eichler Inwood 2018) implications for landscapes (Table 1). However, some assessments may demand unique indicators that are not considered in generic lists and standards (Efroymson et al. 2013). For example, where land tenure and equity are key priorities for sustainability assessment (e.g., Bailis and Baka 2011; Anseeuw et al. 2012), corresponding indicators should be considered (FAO 2014).

Table 1 Checklist of environmental, social and economic, and management indicators for resource management in sustainable landscapes building from McBride et al. (2011), Dale et al. (2013), and Eichler Inwood (2018). Ideally the suite of indicators should represent key information about structure, function, and composition, but knowledge about any one of these features may provide information about other aspects of the system (Dale and Beyeler 2001). It is often easier to measure structural features that convey information about the composition or functioning of the system (Lindenmayer et al. 2000); so here we focus on indicators pertaining to structure
Full size table

Establish targets

A target is a future state indicating progress toward specific goals. Targets for each indicator should be clearly specified with input from stakeholders to describe a desired future condition (Dale et al. 2016; Hunter et al. 2017), and it is important for targets to represent the interests of future generations as well as near-term stakeholder goals (Dale et al. 2013). The time horizon for targets should consider when information is required to guide decisions so that timely corrective measures can be adopted if targets are not being met. Target values for indicators are established based on existing goals, local or regional planning, sustainable development goals, values in the literature, expert opinion, scientific analysis of the activity or similar systems, regulations such as threshold values for water or air quality, or the results of consensus-building among informed stakeholders (Moldan et al. 2012) (e.g., see examples described by Tarter et al. 2016).

Two or more scenarios are needed to explore how future conditions play out based on assumed decisions, policies, and environmental, social, and economic conditions. For example, a renewable energy project may define a scenario of increased energy contributions from solar, wind, and/or biomass and compare the potential social, environmental, and economic conditions to those that might result from a reference scenario defined by current conditions extended across the same time frame. Deciding what management options are considered and compared under each scenario should relate to stakeholder concerns as well as what is possible (Parish et al. 2017). Assumptions underlying scenarios should be documented and based on published sources, to the extent possible, to facilitate interpretation of results and replication of assessment parameters.

Targets for each indicator relate to the baseline conditions and scenarios being considered. Baseline information for each indicator provides a starting point from which to compare other information and can be derived from historical information, scientific literature, or measurements made before the assessment is implemented (when possible). Historical information is used to document recent trends and intra- or inter-annual variability and cycles, which may otherwise confound interpretation of indicator values.

Determine indicator values

Once a set of indicators and scenarios are selected and defined, indicator values need to be quantified for each scenario to allow stakeholders to compare the alternatives. Indicator values can be obtained through empirical measures or surveys of similar situations, model projections, expert opinion, or a combination of these sources. It is helpful to inform stakeholders about the means of collecting indicator values, and engage them in the process when possible, so that the results are transparent and understood by all concerned.

Reviews of diverse projects aiming for sustainable outcomes confirm that one of the most common challenges is timely access to reliable and useful information to guide decisions (e.g., Williams and Brown 2012; CMP 2013; Tarter et al. 2016; Eichler Inwood et al. 2018). When indicator values are estimated by models, it is important to use best available data for calibration (i.e., to adjust parameters to maximize agreement between observed data and the model’s predictions) and validation (i.e., to compare model projections to independent data sets). Whenever future conditions are projected or counterfactual conditions are simulated via models, variability in estimated values and uncertainties in the underlying data and modeling should be documented (e.g., ASTM 2017).

Analyze trends and tradeoffs

Trends in selected indicators should consider linkages between the performance of the activity and affected variables. Documenting cause and effect via experiments and models is challenging (De Boeck et al. 2015). Correctly assigning attribution to certain practices takes time, knowledge, and resources (Efroymson et al. 2016). Methods to test the weight of evidence using different tools for causal analysis help distinguish the share of effects that is caused by the activity being assessed versus effects that are more reasonably attributed to other factors (Efroymson et al. 2016). Applying this process over time ultimately helps stakeholders to understand cause-and-effect relationships, which are at the core of most sustainability debates, and to identify improved practices.

Evaluating trade-offs involves the comparison of changes in indicator values under different scenarios (Fisher et al. 2011; King et al. 2013; Villa et al. 2014; Parish et al. 2016). Techniques used to evaluate trade-offs include optimization approaches and algorithms for scenario comparison of indicator values. No matter what method is used, it is essential to share complete information on all indicators, potential trade-offs, and any underlying assumptions.

Managing trade-offs between potential benefits and negative impacts is challenging but necessary (Chapin et al. 2011; Kanter et al. 2018). Analysis of indicators under each scenario identifies environmental, social, and economic costs and benefits. Ultimately, choices must be made about what result is more desirable (e.g., greater profit or cleaner water). Ideally the evaluation process identifies costs and benefits across diverse metrics so that an informed decision can be made. Some tradeoffs may be avoided. For example, “win/win” options avoid trade-offs by strategic crop placement across a landscape to produce multiple benefits simultaneously (Parish et al. 2012) such as improving water quality while increasing habitat suitable for pollinators (Graham et al. 2017).

Identify good practices

One goal of the iterative process (Fig. 1) is to identify and promote “good practices” for the activity being assessed. We prefer to call them good or better practices, rather than “best management practices (BMPs),” because what is “best” may transform over time as conditions change. The assessment should lead to identification and implementation of better practices for producing goods and managing the overall system in which the goods are produced while recognizing that good practices depend on a specific set of conditions and stakeholders at a given point in time. Recommended practices related to agricultural and forestry activities support better water quality, soil quality, and crop production. For example, good practices for harvesting wood products from forests include those that minimize soil disturbance and water quality impacts (Cristan et al. 2016). FAO (2012, 2018a, b) provides examples of good environmental practices for agricultural and forest production pertaining to soil, water, biodiversity, and climate-change mitigation, as well as the social and economic benefits to income, availability of inputs, and access to energy. While many industries have developed BMPs (Ice 2011; Seuring and Muller 2008), their further development is an active area of research (Youngs and Somerville 2014).

Examples of success, failure, and lessons learned are important to support a culture of continual learning and to better quantify risks as they become apparent (e.g., Seuring and Muller 2008; Lindenfeld et al. 2014). Good practices also consider preparation and early warning systems to reduce impacts from extreme events (drought, flood, fire, disease, economic downturns, political upheaval, etc.) (Dale et al. 2016). Development and implementation of good practices are supported by transparent and participatory monitoring as well as by the ability to adjust targets, interventions, and regulations in response to new knowledge (Dale et al. 2016).

Insights from experience

We have used this decision-making cycle (Fig. 1) to assess progress toward landscape sustainability goals for bioenergy (Case 1; Fig. 3) and row-crop agriculture (Case 2; Fig. 4). We suggest that this method has broad applicability for decision makers evaluating a variety of alternative pathways for production of food, fiber, feed, energy, and other goods required by human society as well as for practitioners interested in assessing progress toward established goals.

Fig. 3
figure 3

By providing incentives to harvest a portion of residues from high-yielding fields that are used to produce biofuel, U.S. bioenergy production in Iowa (a) facilitates no-till corn production that reduces soil erosion and promotes the use of perennial and annual cover crops, which further reduce erosion and enhance soil carbon and other qualities. Another alternative (b) illustrates that erosion is reduced by cover crop mix (on the left) compared to a soybean monoculture (on the right) (photos by authors, September 2017)

Full size image
Fig. 4
figure 4

Large-scale agriculture in the Yaqui Valley of Mexico (a) shortly after conventional tillage and planting and (b) mature wheat. See Case 2 for summary of Yaqui Valley case study. Photographs by authors, March 2017

Full size image

Case 1: Example of applying the landscape assessment approach to cellulosic-based bioenergy in the United States

Landscape sustainability is a major concern for bioenergy production. Multiple studies suggest that cellulosic feedstocks provide more sustainable and ethical energy options to society than fossil fuels but should be produced under designs that support ecosystem services as well as social and economic benefits (Robertson et al. 2008; Haughton et al. 2009; Buyx and Tait 2011; Dale et al. 2014) (Fig. 3). The environmental advantages are most evident when wastes and residues that would otherwise rot or be burned are used for bioenergy (Dale et al. 2014). Social benefits include enhanced food security, increasing rural employment opportunities, expanding the tax base that supports community services, and increased fuel efficiency with reduced greenhouse gas emissions and improved public health (Dale et al. 2014). Below is a description of the application of the six steps of assessing progress toward landscape sustainability (Fig. 1) to cellulosic bioenergy production.

  1. (1)

    Define scope: The purpose of bioenergy sustainability assessment is to evaluate costs and benefits of bioenergy production and use compared to an alternative energy, typically fossil fuel. The spatial extent for bioenergy production focuses on the areas where biomass is grown or collected as well as where the energy is produced and used. Time steps are annual, and temporal horizons are on the order of years to decades depending on the indicators selected. The context or reference system refers to conditions without bioenergy production, which are often typical agriculture or forestry practices (e.g., Parish et al. 2017; Koponen et al. 2018).

  2. (2)

    Prioritize indicators: A checklist of indicators in twelve categories was developed to reflect potential environmental (McBride et al. 2011) and socioeconomic (Dale et al. 2013) effects across the bioenergy production and use supply chain (Table 1). Nineteen environmental and sixteen socioeconomic categories of indicators provide a basis to quantify and evaluate effects of bioenergy systems across many regions where they are being deployed. Indicators relating to management performance are also useful. However, choosing the most appropriate indicators for a particular situation requires consideration of the specific project and stakeholder input (Efroymson et al. 2013; Dale et al. 2016). For instance, stakeholders in an Iowa project did not prioritize air quality indicators (Dale et al. 2018a), and a case study of east Tennessee switchgrass production identified the need to add “profit variability” as an indicator since switchgrass was less susceptible to volatility in yields and market prices than products of conventional agriculture (Parish et al. 2016).

  3. (3)

    Establish targets: Targets are reasonably achievable values for each indicator affected by bioenergy production that would result in enhanced environmental, social, or economic conditions. For bioenergy systems, production targets were determined by the amount of biomass required by the refinery (e.g., Parish et al. 2012). Targets for stream nutrient concentration were set by using existing regulations or (in their absence) by potential thresholds of stream eutrophication (Dodds 2007). Baseline refers to conditions for each indicator that occur when bioenergy is not produced. Since bioenergy production often happens where agricultural or forest activities are ongoing, the baseline values are typically quite different from “natural” conditions.

  4. (4)

    Determine indicator values: Documenting changes in indicator values over time or space when bioenergy is produced as compared to no bioenergy is challenging because production is limited. Information has been obtained from monitoring programs of the U.S. Department of Agriculture, field studies, and through comparison of modeled alternative scenarios (e.g., see Parish et al. 2012, 2016; Landis et al. 2018).

  5. (5)

    Analyze trends and tradeoffs: Trends differ for each context. For example, when the bioenergy feedstock is a waste that otherwise might contribute to environmental degradation, benefits accrue. Use of perennial crops may improve ecosystem services (Cacho et al. 2018). When feedstock production occurs in rural areas where jobs are in decline, the bioenergy industry provides a boon to the economy and social structure. There are trade-offs among different scenarios in terms of overall area affected and the characteristics of ecosystems (e.g., Costanza et al. 2017), for landscape effects on wildlife habitat varies by species (e.g., Tarr et al. 2017). Displacement of carbon-intensive fossil fuels helps keep carbon in the ground and therefore reduce carbon emissions. When use of the biomaterial for energy is a small fraction compared to other uses, the effects are difficult to discern. A causal analysis approach and simulation models are useful in considering attribution of effects associated with bioenergy systems (Efroymson et al. 2016). Spider diagrams are a common means to display potential tradeoffs (e.g., Landis et al. 2018). We used a combination of techniques to visualize potential tradeoffs between profit and water quality resulting from different switchgrass planting configurations within a watershed (e.g., Parish et al. 2012). In that East Tennessee context, switchgrass production for bioenergy improved environmental and social sustainability conditions without adverse economic impacts, which enhanced landscape sustainability; however, achieving these benefits requires a strong bioenergy market (Parish et al. 2016).

  6. (6)

    Identify good practices: Results confirm the benefits of using waste material when possible, following approved good management practices (e.g., Ice 2011; Williams and Brown 2012), conducting systematic monitoring and evaluation of potential effects to inform continual improvement of management practices, and relying on a network of transparent training and monitoring support to ensure accountability rather than a single program or metric.

Case 2: Example of applying the landscape assessment approach to integrated row crop agriculture in western Mexico

Evaluation of the agriculture systems in the Yaqui Valley of Mexico first focused on factors that affect yield (e.g., Luers et al. 2003). However, consideration of landscape sustainability in the region requires a broader perspective (Matson 2012; Matson et al. 2016). Application of the approach described in this paper (Fig. 1) to the systems that predominate production in the Yaqui Valley surrounding Obregon, Mexico (Fig. 4), requires consideration of the environmental, social, and economic implications of large-scale farming (Eichler Inwood 2018). The application of the six steps of assessing progress toward landscape sustainability to large-scale agriculture in western Mexico is described below.

  1. (1)

    Define scope: The purpose of the assessment is to determine the environmental, social, and economic implications of irrigated agriculture for the broader Yaqui River watershed. Major sustainability questions revolve around future implications of water management and effects on environmental and social conditions. The spatial extent is the area of the Yaqui Valley that is irrigated for row crops (predominantly wheat) and spans approximately 225,000 ha. Time steps of primary agricultural management decisions and data availability on most indicators are annual or by cropping season.

  2. (2)

    Prioritize indicators: Indicators were selected from an initial checklist with input from stakeholders. Stakeholders included representatives from agriculture industries, commodity organizations, farmers’ unions, local environmental research faculty, international researchers, farm owners, irrigation managers, plant pest and disease control specialists, agricultural outreach agents, and community members. The checklist proposed for sustainability indicators (Table 1) required modifications to reflect local contextual issues and availability of information. The 25 indicators that were selected are those deemed to be useful to diverse groups, technically effective for the location, practical in terms of obtaining indicator values, and sufficient but not exhaustive. These indicators were determined by ranking candidate indicators through a process of compiling expert opinions after consultation with several stakeholder groups (Eichler Inwood 2018). Seasonal availability of reservoir irrigation water and water quality were identified as top priorities in this desert region (Eichler Inwood 2018). Stakeholders also prioritize crop diversification and seed varieties that reduce vulnerabilities to extreme events and market fluctuations that are inherent under current practices where most land is planted in a single variety of wheat. In addition, stakeholders are interested in indicators that guide agricultural management practices toward higher profits, lower water demand, lower pesticide exposure, and reduced wastes. Indicators such as energy security and social acceptance were rated by local stakeholders as being less important than measures such as conservation of water and soil, nutrient management, resilience to extreme events, transparency, and equity.

  3. (3)

    Establish targets: Targets are being developed in consultation with stakeholders (Fig. 5) based on their perspectives on desired future conditions and review of current conditions and past trends that may suggest impending changes (Eichler Inwood 2018) as well as theories of change that identify strategies for meeting goals (Mayne and Johnson 2015; CIMMYT 2017). The reference conditions are those associated with prior cultivation practices or “business as usual” where about 80% of irrigated land is used for durum wheat (Triticum turgidum subsp. Durum) managed as a single-season monoculture.

    Fig. 5
    figure 5

    Stakeholders were actively engaged in the evaluation of agriculture systems in the Yaqui Valley of Mexico by using interactive posters (Dale and Kline 2017). Stakeholder engagement is critical to each stage of the sustainability assessment process. Photograph by authors, March 2017

    Full size image
  4. (4)

    Determine indicator values: Of the 25 prioritized indicators, information is readily available for only 7 of them: soil quality (area of at-risk soils), productivity (measured by the Normalized Difference Vegetation Index, NDVI), biodiversity (protected area), vulnerability (government indices related to climate change), poverty (government index), transparency (Corruption Perceptions Index), and economic implications of crop diversity (Shannon diversity index by area and value of crops) (Eichler Inwood 2018). Baseline data for selected indicators are derived from information collected by local researchers or prior studies (e.g., Matson 2012; Matson et al. 2016) and models (e.g., for greenhouse gas emissions). The indicator values used are annual or seasonal data from 2015 to 2017 and earlier trend data when available. Data were accessed through archived national and international sources such as census data, agricultural records, and public satellite imagery (Eichler Inwood 2018).

  5. (5)

    Analyze trends and tradeoffs: Recent trends are summarized by Eichler Inwood (2018). Whereas the region is highly productive, it is at risk of yield decline in areas with low soil quality. About half of the area contains the more desirable soils for agriculture, but the rest of the area is at risk of compaction or salinization. Areas under conservation agriculture practices have better physical properties, moisture retention capacity, and soil biota (Govaerts et al. 2006; Verhulst et al. 2011; Fuentes et al. 2012; Rivers et al. 2016). Peak productivity for winter crops in the Yaqui Valley tends to be in late-February to mid-March. There are no habitat conservation areas within this agricultural valley, and hence native diversity is low; however planting of perennial crops could create more favorable habitat for some species (Asbjornsen et al. 2014). Transparency (as measured by perception of corruption) in the state has declined recently. Crop diversity has not changed in recent years and remains lower than that of a similarly productive climate in Southern California. Risk of drought and pest susceptibility related to low crop diversity are prevailing concerns.

  6. (6)

    Identify good practices: The analyses suggest that there is an opportunity for improvement of soils through conservation agriculture techniques especially for soils at risk of compaction or salinization (Eichler Inwood 2018). The exchanges with stakeholders identified several promising practices including drip irrigation systems, improved management of nitrogen fertilizer and pesticides, and alternative production systems involving small animals, greenhouses, aquaculture, and planting new crop varieties. For example, farmers in the valley are experimenting with over 65 different crops ranging from walnuts to asparagus (Asparagus officinalis). Although new technologies, crops, and practices help achieve goals, this example also highlights the important roles played by policy, regulations, field-level monitoring, and enforcement of local regulations. If these aspects of governance do not provide the incentives needed to conserve water and avoid water contamination, then technical assistance to promote new crops and technologies may be for naught. Farmers are likely to do what is easiest and continue the same practices that they already know and consider less risky. However, making crop insurance available may encourage farmers to increase cropping diversity and thereby reduce the economic risk associated with crop loss.

Overarching themes

Our experiences in developing and testing this approach have led to the identification of several overarching and inter-connected themes (shown in the center of Fig. 1) that are critical for success when applying each step of the process, namely stakeholder engagement, transparency and trust, communication, monitoring, and continual improvement. Each of these themes is described below. We also identify some common pitfalls encountered when implementing the process and provide recommendations for minimizing costs and delays.

Stakeholder engagement

Stakeholder involvement (Beier et al. 2016; Djenontin and Meadow 2018; Petersen et al. 2018) underlies all aspects of this landscape sustainability assessment approach and helps build local capacities. Stakeholders should be a part of every step, for they form part of the context and are the key to bridging the “practitioner-researcher divide,” which deters adoption of sustainability approaches (Rasmussen et al. 2017). Early and ongoing stakeholder engagement in defining desired future conditions, prioritizing practical indicators, and setting realistic goals determines success or failure of both the assessment and progress toward landscape sustainability (e.g., see Tarter et al. 2016; Rasmussen et al. 2017). Stakeholders also play critical roles in defining priorities based on the landscape context and in implementing good management practices (Johnson et al. 2003; Ness et al. 2007; Sydorovych and Wossink 2008; Cebrián-Piquerasa et al. 2017). For example, when asked to prioritize indicators, stakeholder often communicate their values and insights, as well as provide examples of how to measure progress, thereby helping to identify pertinent indicators (e.g., see Dale et al. 2018a). Therefore, the coordinator should take the time required for stakeholder engagement in planning and implementation of the assessment.

However, diverse perspectives from stakeholders also make it difficult to reach consensus on clear goals and a manageable number of indicators. From the outset of an assessment, it is often evident that some stakeholders have conflicting goals for the outcome. Developing consensus on well-defined and focused goals is critical because goals that are ambiguous or too broad result in the measurement of variables that are misleading or inaccurate, overly broad scopes, or targeting too many objectives (Tarter et al. 2016). It may be easier to reach consensus about what stakeholders don’t want to happen, i.e., undesired future scenarios and changes to be avoided, rather than specific targets to be achieved (Zenner 2016). Stakeholder ownership of the process and results is instrumental in supporting long-term efforts to identify improved practices based on the signals provided by indicators.

Because participation of stakeholders results in a process that is not as neat and simple as idealized in Fig. 1, our recommendation is to work on several of the steps concurrently but to complete them in the order shown. Unexpected detours, hurdles, and delays are the norm and represent an important part of co-learning and capacity building (e.g., Petersen et al. 2018; Williams and Brown 2012) in adaptive management. Nevertheless, it is advisable to revisit steps if new information or changing circumstances affect the activity, especially when such practices increase trust and local ownership of the process.

Transparency and trust

Transparency refers to sharing of all aspects of the assessment process including the methods used to establish goals and objectives, indicators, baselines, and targets as well as the information obtained (Cash et al. 2003). The assessment process requires trust founded on transparency including who is behind the activity and why, what kind of information is collected, and how that information is vetted, interpreted, and communicated. The inclusion of all key stakeholders is an essential part of transparency. Too often, native people, absentee owners, low income owners, renters, minorities, and those who are affected but are distant from the activity are not included in the evaluation. It may be useful to find multiple avenues of engaging stakeholders (e.g. through open discussions or focus groups, as well as semi-anonymous or closed surveys) so that literacy or language barriers can be addressed and community hierarchies or security concerns do not override expression of conflicting landscape goals.

Trust is a measure of confidence in the reliability and truthfulness of information and in the ability for the process to achieve desired results. Trust is essential for the process to fulfill its purpose in guiding decisions toward improved conditions. If stakeholders do not trust the process, the results have little value. Being open and transparent builds trust, but, particularly for outsiders, it is difficult to establish trust within assessment time frames. Hence it is useful to engage leaders and coordinators who have earned local respect.

Applying the framework (Fig. 1) at the inception provides an explicit commitment to transparency that increases legitimacy and helps build supportive constituencies for subsequent steps in the activity’s development. Up-front investments to engage stakeholders pay off in the long run, for developing trust and good working relationships provides a pathway to leveraging the resources of other participates (Petersen et al. 2018). We recommend that the process be built on honesty and commitment to listening and sharing.

Communication

Transparent communication about available resources, developing consensus with stakeholders around a “doable” scope of work, and developing practical timelines with sufficient flexibility or a cushion that allows for unforeseen delays are important for success and reduce unrealistic expectations. Effective and timely communicating of information that helps guide decisions is key to making scientific assessments relevant to pending decisions and changes in policy (Lindenfeld et al. 2014, Kanter et al. 2018). Monitoring data and analyses should be shared in a timely fashion so that stakeholder decisions, actions, and opinions are based on current information (Palmer et al. 2005). To minimize misunderstandings, it is critical to ensure that terminology is explicitly defined and accepted by local stakeholders.

For landscapes that support production systems (such as agriculture and forestry), the iterative process in Fig. 1 entails sharing information about how progress toward established goals can be enhanced (or deterred) at all points along a value chain [e.g., procurements and supply chains, training and education, value proposition, outreach, quality control, and management systems as discussed in International Organization for Standardization (ISO) management system standards 14,001 and 90,001 (ISO 2018)]. New information and new contextual conditions result in new questions about how to better achieve goals, thereby continuing the assessment cycle.

The diversity of stakeholders in any one landscape means that diverse communication tools should be adopted. New technologies that provide information and feedback via the web or mobile applications have transformed the speed and ease with which information is shared (Nghiem and Carrasco 2016). However, it is challenging to authenticate the validity of information. While the internet provides an effective means to relay information to some groups, it does not work for all. Surveys and citizen science are effective ways to collect information as well as share information about the assessment. Often it is necessary for the assessment coordinator to reach out to individual stakeholders to ensure that all relevant groups are involved in the assessment process (e.g., Dale et al. 2018a).

Because of the underlying complexity of sustainability and its many dimensions, visualization is an essential part of communication. Visualization involves translating complex information about the landscape into a graph, diagram, or images to facilitate understanding. Tufte (1983) notes that visualizations should provide the data and encourage the viewer to think about the meaning of the information rather than about methods used to collect it or the graphic design. Effective visualization transforms large amounts of information in a manner that allows users to capture the big picture or zoom into different levels of detail (Tufte 1983; Dale and Kline 2017). Visualization tools may use a combination of graphical, statistical, and verbal descriptions. Integrating information for effective visualization of indicator trends is necessary for stakeholders to better understand progress toward established goals and any potential tradeoffs and synergies.

We recommend that procedures for data collection, monitoring, and sharing of data and analytical results should be clearly documented early in the process and understood by all concerned (Dale et al. 2016). Special attention needs to be paid to data aggregation, any procedure by which information is expressed in a summary form for purposes such as reporting or analysis. As data are aggregated, the risks for errors, misinterpretation, omission, or misrepresentation increase. For example, if data on environmental qualities are aggregated, progress in improving one aspect, such as air quality, could mask a critical decline in quality for another aspect, such as water or biodiversity. Or improvements in one watershed could mask deteriorating water qualities in another area. Consolidated scores based on off-setting values is described as a “weak” method for measuring effects compared to “strong” approaches where each indicator is assessed and reported individually (Pearce et al. 1994; Rickels et al. 2016). Mathematically valid approaches for aggregation provide criteria for selection of functions to combine information, which are often necessary when collating data for a landscape sustainability assessment (Pollesch and Dale 2015).

Monitoring

Ongoing data collection programs are another challenging but essential cross-cutting theme. Decisions about what data to obtain should be guided by identifying information that is relevant, timely, and at an appropriate scale. Monitoring data are gathered by governmental agencies, citizen science, civil society organizations, or industry. While government measurement programs do not always focus on stakeholders’ issues, they are often the best option available for gathering consistent and statistically valid data [e.g., the U.S. Department of Agriculture’s Forest Inventory and Analysis (https://www.fia.fs.fed.us), the Conservation Effects Assessment Project (https://www.nrcs.usda.gov/wps/portal/nrcs/main/national/technical/nra/ceap/), or the Cropland National Assessment (https://www.nrcs.usda.gov/wps/portal/nrcs/detail/national/technical/nra/ceap/na/?cid=nrcs143_014144)]. Information collected by citizens groups or non-governmental organizations is valuable and typically focuses on a particular concern, place, and time. Such snapshots can be accurate and important as reference points, particularly if data collection and documentation are guided by professionals (Tredick et al. 2017). Reports that arise from photographs, anecdotes, or the internet should be validated and confirmed by authoritative sources. Industries affected by or affecting system conditions often collect their own information as required by permits or for their edification—but these data are not always available to or understandable by the public. All sources of monitoring information are useful, but each source must be interpreted carefully by considering the purpose, protocols, time, place, sampling methods, etc. to determine representativeness and statistical validity.

While the tendency is to seek out as much information as possible and to identify all ideal indicators, our recommendation is to use available data when possible and to focus on a few key measures that reveal trends or portend changes in conditions of highest priority to stakeholders. We find it useful to start with defined categories and checklists of potential indicators that reflect environmental, social, and economic concerns (e.g., Table 1). This approach allows stakeholders to question the terms and their meaning and often results in a lively discussion and reevaluation of indicators and their definition. The future health of ecosystems and human well-being should be considered (Collins et al. 2011), and, as conditions change, checklists must be modified to reflect stakeholder concerns about environmental, social, and economic outcomes. Indicators that are not priorities for stakeholders should be considered for removal (e.g., Dale et al. 2018a), and context-specific priorities might necessitate the addition of one or more indicators to the initial check list (e.g., Case 2).

Continual improvement

A focus on continual improvement is recommended, for it helps an activity adapt to ever-changing economic, political, social, and environmental conditions, including extreme weather events and disturbance regimes (Dale et al. 2018b). There is no “silver bullet” to achieve landscape sustainability goals but rather many small steps that move an activity closer to goals over time. Thus, continual improvement is commonly adopted by standards and certification schemes (e.g., GBEP 2011; ISO 2015; RSB 2016; ASTM 2017). Adaptive management that promotes continual improvement is a widely recognized means to promote more sustainable resource management (Williams and Brown 2012).

Systematic monitoring to determine progress toward indicator targets and analysis to relate changes to activity interventions may result in discarding some indicators that are not helpful or revising how indicators are measured and interpreted based on new understanding. Historical data are limited, and knowledge of the future is uncertain, but as new information is gathered, new challenges and opportunities for improvement arise. The assessment process should be updated based on new understanding about the costs and benefits of available options.

Assessment developers need to allocate sufficient resources for science-based analysis to foster learning with and among stakeholders (Petersen et al. 2018). Efforts to guide decisions toward continual improvement will be fruitful only if people understand why changes in practices are necessary for the achievement of desirable outcomes.

Conclusions

Implementing this process to assess progress toward landscape sustainability entails significant resources (e.g., staff time, money, and multi-disciplinary skill sets that are difficult to assemble and maintain—including a team capable of effective outreach and who are perceived as being honest brokers). Each assessment involves unique opportunities and constraints. Deployment requires time for discussion, comprehension, and evaluation of goals, indicators, changing circumstances, tradeoffs, and ways to improve practices. To achieve established goals with limited resources requires creativity, partnerships, and identifying local champions who are respected and enthusiastic about those goals. With good planning and local collaborators, we find it is possible to engage stakeholders in defining goals, indicators, and targets. For example, low-cost efforts with high returns include taking advantage of existing meetings and extension agent activities, employing simple tools such as interactive posters, literature reviews, and outreach to partners to collect available data. Combining these inputs with strategic interviews to fill gaps in representation, we have been able to generate preliminary quantitative and qualitative results that could be refined with further stakeholder input.

Our experience is that engaging in this assessment approach supports adoption of better landscape management practices via adaptive management and a process involving transdisciplinary research. Viewing landscape sustainability assessment as an ongoing process emphasizes local social and institutional mechanisms to identify and adapt good practices to continually improve outcomes over time. Stakeholder engagement in, and ultimate ownership of, the process is more important than a specific result or indicator value. While idealized goals may or may not be achieved, the assessment approach described in this manuscript provides a means to build capacities to assess environmental, social, and economic patterns and processes that are important to local stakeholders. Flexibility and a willingness to respond to input by adjusting assessment plans and goals, patience on the part of all parties, and a long-term commitment to the iterative process are also important qualities for successful assessment involving stakeholders.

Just as sustainability is not a final state to be achieved, the assessment process does not provide a final answer that assures a landscape is sustainable. Experience suggests that ongoing monitoring and evaluation yield new insights and, sometimes, unexpected information. Working through this landscape assessment approach leads to discoveries and new questions that may require reevaluation of goals or stepping back to restart the assessment. This process supports the role of researchers and scientists to provide training, data, and analytical support to local stakeholders. The ultimate goal is to be facilitators, helping to transparently compare alternatives, identify good practices, and guide subsequent decisions toward ever-improving outcomes.