Abstract
Decision-support tools (DSTs) synthesize complex information to assist environmental managers in the decision-making process. Here, we review DSTs applied in the Baltic Sea area, to investigate how well the ecosystem approach is reflected in them, how different environmental problems are covered, and how well the tools meet the needs of the end users. The DSTs were evaluated based on (i) a set of performance criteria, (ii) information on end user preferences, (iii) how end users had been involved in tool development, and (iv) what experiences developers/hosts had on the use of the tools. We found that DSTs frequently addressed management needs related to eutrophication, biodiversity loss, or contaminant pollution. The majority of the DSTs addressed human activities, their pressures, or environmental status changes, but they seldom provided solutions for a complete ecosystem approach. In general, the DSTs were scientifically documented and transparent, but confidence in the outputs was poorly communicated. End user preferences were, apart from the shortcomings in communicating uncertainty, well accounted for in the DSTs. Although end users were commonly consulted during the DST development phase, they were not usually part of the development team. Answers from developers/hosts indicate that DSTs are not applied to their full potential. Deeper involvement of end users in the development phase could potentially increase the value and impact of DSTs. As a way forward, we propose streamlining the outputs of specific DSTs, so that they can be combined to a holistic insight of the consequences of management actions and serve the ecosystem approach in a better manner.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Coastal and marine ecosystems around the globe provide a range of services supporting the social and economic well-being of communities (Turner and Schaafsma 2015). The increasing use of marine resources leads to growing pressures and impacts on the marine environment, compromising the sustainable provisioning of ecosystem services (Airoldi and Beck 2007). In addition to sea-based activities, activities on land, e.g., agriculture and industry, also alter the condition of the marine environment through enhanced nutrient and pollutant runoff (HELCOM 2018a). Managing the use of resources, while sustaining ecological integrity, is a key challenge in marine environment management. This has led to the development of ecosystem-based management, striving to ensure that marine ecosystems are well functioning and able to support sustainable delivery of ecosystem goods and services (Elliott 2011; Borja et al. 2016).
The DAPSI(W)R(M) concept (Elliott et al. 2017), later in this paper referred to as DAPSIWRM, provides a general structuring and holistic framework for use in environmental management. The concept has evolved over many years, rooting back to the PSR framework that was proposed by Rapport and Friend (1979), further adapted by several international organizations (OECD 1993; EPA 1994; UNEP 1994) and expanded to DPSIR (EEA 1999), before stepwise transforming into its current format (Elliott et al. 2017).
The DAPSIWRM concept describes links and interactions between drivers (defining the needs), activities (human activities to fill the needs), pressures (caused by the activities), state changes (how the pressures affect the environment), impacts on welfare (how society is impacted), and response using measures (management actions). The ability of the DAPSIWRM framework to link natural and societal information facilitates integrated marine management while incorporating the ecosystem approach (Elliott et al. 2017). Ecosystem-based management relies on information of various components from the marine environment and on the sources and magnitude of pressures that impact marine ecosystem services. Consequently, vast amounts of information need to be synthesized to make well-informed decisions (Elliott 2013; Scharin et al. 2016).
Decision-support tools (DSTs) combine, process and condense large amounts of information, assisting the manager in the decision-making process. Decision makers at local, national, and intergovernmental level need this information to, for example, evaluate the current status of the marine environment, and estimate the most cost-efficient measures that are needed to restore impaired status (Balana et al. 2011) or to facilitate spatial planning (e.g., Bagstad et al. 2013, Stelzenmüller et al. 2013). DSTs allow a structured process where alternative options can be compared, improving the transparency of the decision-making process (Nicholson et al. 2020; Sullivan 2004; Ward 2007). Technically, DSTs are expected to produce outcomes, as realistic as possible, that answer a specific set of questions; for the eutrophication problem, as an example, questions such as: what is the eutrophication status of a sea area? How much nutrient pressure can it tolerate? What are the benefits of nutrient abatement? What are the most cost-efficient management actions?
The marine environment is protected by a large number of international and national legislation and policies (Boyes and Elliott 2014; Reusch et al. 2018). In the Baltic Sea, environmental problems and subsequent socio-economic impacts are addressed through regional, European and global policies and agreements, directed toward sustainable management of the Baltic Sea. The HELCOM Baltic Sea Action Plan (BSAP) is an example of an international regional agreement. On the European level, many environmental EU directives (Marine Strategy Framework Directive (MSFD), Water Framework Directive (WFD), Habitats Directive, Waste Framework Directive, Maritime Spatial Planning Directive) as well as sectoral policies (e.g., Integrated Maritime Policy, Common Fisheries Policy, Common Agricultural Policy) and strategies (Strategy on Adaptation to Climate Change, Biodiversity Strategy) exist. The implementation of these initiatives challenges the coastal states, that strive, both individually and collectively, to assess the potential outcomes of choices and decisions regarding the management of human actions and their impact on the environment. These decisions are guided by the policy requirements but they need to be based on first-hand information from the environment and the society. This is where DSTs are essential. The DSTs developed and applied in the Baltic Sea and its drainage basin range from quantitative assessment tools to model-based or operational systems linked to databases, but also include tools that handle more descriptive, nonquantitative information.
Here, we provide an overview of DSTs developed for application in environmental management of the Baltic Sea and its drainage area. First, we made an inventory, where the DSTs are categorized according to the DAPSIWRM framework and the environmental problems they address. Particularly, the role of the DSTs within the DAPSIWRM context, in terms of how well its components and linkages are covered, was investigated. Second, we developed a framework to assess DST performance and to understand their strengths and weaknesses. Third, we assessed the DSTs against the end user needs. Finally, we investigated the development and maintenance of the DSTs, addressing questions such as how end users had been involved in the tool development and how well the sustainability of the DSTs were ensured.
Material and Methods
Definition of DST
DSTs can be defined very broadly to include any tool that condenses complex information into an easier understandable format to be applied in decision-making (Sullivan 2004). Such tools include, for example quantitative assessment tools, model-based management tools and operational systems linked to databases. To structure our overview and to have a common understanding of the term “DST”, with focus on marine environmental management in the Baltic Sea, a definition for use in the BONUS DESTONY projectFootnote 1 was elaborated and a set of five definition criteria (DC) were formulated (Table 1). Ideally, a DESTONY DST should fulfill all these criteria, but in practice, they turned out to be very demanding. To include a broader range of DSTs in the inventory, fulfilling all DC was not considered as a requirement. However, this implies that we do not cover all tools (e.g., models) fulfilling few of the DC. In the performance analyses, we included only DSTs fulfilling more than four of the DC, to ensure comparability.
Inventory
As a basis for the inventory of available DSTs intended for the Baltic Sea and its drainage area, environmental management problems, where decision-making is needed, were identified (e.g., eutrophication, contaminant pollution, loss of biodiversity, etc.). Responsibility for reviewing the availability of DST within these problem areas was then distributed among the project participants based on specific expertise. As participants in the project consortium have been widely involved in both developing and using DSTs, there was a good understanding of the availability of DST already from the start. In addition, web searches were performed to ensure that other tools were not overlooked. For problem areas where there was no expertise in the project consortium (e.g., nonindigenous species, underwater noise, litter), interviews with experts in the field were made in order to identify DSTs. Ongoing and finished BONUS projectsFootnote 2 were also contacted to scan for recently developed tools or tools still under development. The list of identified tools was sent as part of the questionnaire to end users and stakeholders and they were asked to add potential DSTs missed.
Information about the DSTs and their performance were collected using a common template. The template consisted of three parts: (1) general information including problems, policies and DAPSIWRM components addressed, platform used, type of input data needed, description of outputs, a short description and information on developer/host, and where to find the tool as well as links to specific examples where the tool had been used; (2) DC (see Table 1); and (3) fifteen performance criteria (PC) (see Table 2). The latter was described both in words and with a 1–5 scale to further evaluate the tools. The filled templates (excluding the performance scoring) were sent to the developers or hosts of the DSTs for cross-checking and commenting, and the templates were updated accordingly. However, responses from hosts were received for only two thirds of the DTSs.
Performance Criteria (PC)
A set of fifteen PC was defined to evaluate in greater detail how the tools work and how they differ from each other (see Table 2). The PC mirror the scientific foundation of the tools, how well they fit into Baltic Sea management as well as their user-friendliness, reflecting characteristics such as quality and relevance, that can be ranked according to general performance, and descriptive qualities that need to be considered case-specifically in the ranking. The aim was not to rank the tools based on the PC but rather to shed light on the applicability and transparency of the tools. For each criterion, a five-step scale was developed to reflect how well the tools respond to the criteria. The scale is categorical, but with increasing relevance. Generally, the value of 1 refers to “not fulfilling the criterion” whereas 5 refers to “fully complying with the criterion”. The scale is defined separately for each criterion.
End User Involvement and Use of DSTs
As the filled DST templates were sent to the hosts for cross-checking, a questionnaire regarding the development of the tool and involvement of end users was attached. This questionnaire aimed at gaining understanding of why and how the tool was originally developed, if end users were involved in the development, and if the hosts were satisfied with how the tool had been applied in environmental management. The questionnaire was structured according to five themes, each with a set of questions/statements to be answered with yes or no: (1) initiation of tool development; including questions on what motivated the development of the tool, (2) defining end users; to find out if and at what stage in the development phase end users were identified, (3) end-user involvement; with questions on how end users were involved in the development process, (4) maintaining and updating; to find out if and by whom the tool is maintained and updated, and (5) use of the tool; asking if the host is satisfied with the way the tool has been used. The full questionnaire can be found in Online Resource 1. The survey was answered by 27 DST hosts out of 42.
Based on the results from the questionnaire on end-user involvement in development and use of DSTs, we examined if there are performance differences between (a) tools involving end users in the development phase, (b) tools that have been developed as a direct response to a management need, (c) tools where developers/hosts are satisfied with how tools have been used, and (d) tools where this has not been the case.
End User Preferences
An online questionnaire was created to assess end users’ level of knowledge about DSTs, their own use of the tools and their preferences for the predefined PC (not including the scoring categories). End users were asked for their perception of importance of the DST PC into four categories: (1) very important, (2) important, (3) not so important, (4) not important at all. The questionnaire was distributed using the online software JotForm and made available during March 2019 (the questionnaire can be found in Online Resource 2). Identified end users of DSTs included representatives of national, regional, and local administrations, international organizations and institutions (e.g., HELCOM, VASAB, ICES, EU), nongovernmental organizations (NGOs), as well as researchers represented in national or international working groups dealing with management issues related to the environment of the Baltic Sea and its drainage basin. In total, 811 email invitations were sent to potential DST end users in all Baltic Sea countries. In addition, an invitation to participate was launched on the webpage and social media pages (Twitter and Facebook) of the Stockholm University Baltic Sea Center.
In total, 108 responses were received. Number of responses varied strongly between countries. 45% of all participants indicated to be working in Sweden, 19% in Finland, and 9% in Germany. Lithuania and Estonia were represented by 6%, Denmark and Latvia by 4%, and Poland and Russia by 2%. Three participants (3%) stated that they are working in multiple countries and one EU representative from Ireland participated. Around half of the participants (53%) were representatives of public administration, 38% from research institutes and universities, and 8% from NGOs. Others included participants from consultancies and private companies and made up 5%. Participants had diverse thematic backgrounds, such as eutrophication (55%), marine spatial planning (MSP) (38%), marine habitats (36%), nature protection (32%), marine litter (28%), hazardous substances (26%), fisheries, coastal zone management (19% each), and coastal protection (32%). Additional thematic fields were mentioned by 20% of the participants and included underwater noise, acidification, invasive species, biodiversity, environmental economics, and social sciences.
Results
Inventory
A total of 42 DSTs were identified as being used in the Baltic Sea and drainage basin (Table 3). Assessments of the DSTs can be found in the DESTONY DST catalogFootnote 3. Only 12 tools fulfilled all five DESTONY DST DC, and four of the DC were fulfilled by 14 tools. All 42 tools fulfilled the DC #3 (supports decision-making related to degradation of the aquatic environment). DC #2 (the tool can be accessed and operated on internet) turned out to be decisive as it was fulfilled by only ~40% of the tools and none of the tools with less than four fulfilled DC succeeded on this criterion. Most of the tools were directly available, i.e., found on the internet, but they usually lacked a web interface and required downloading and installation of a software or downloading a script or code to be run in e.g., R, Matlab or other software requiring licenses (e.g., GAMS). Approximately 25% of the DSTs were not directly available for end users, but only accessible through contact with the host. The criterion on interactivity, i.e., that the end user can modify inputs and/or settings of the tool, was fulfilled for about 80% of the tools. Examples of DSTs not fulfilling this criterion were tools in which the host feed data into a model and the end user only inspects the model outputs for a given area and/or time period, i.e., the required input from the end user is only to choose an area and/or a time period relevant for the specific problem setting.
DSTs were found to respond to all segments of the DAPSIWRM framework and covered a broad range of environmental problems (Figs 1 and 2). Nine tools were specific to only one segment in the framework; of these, all but one treated state changes. Tools covering several segments often focused on the links between activities, pressures, and state changes. Drivers were only addressed by three DSTs and these were also the only ones to cover the whole DAPSIWRM framework. The three DSTs covering the full framework were of different types (model, stakeholder tool, and planning tool) and addressed different issues (biodiversity and conservation, impact evaluation, and sea-area use). All DAPSIWRM segments, apart from drivers, could be covered using a single DST for problems related to eutrophication, whereas for problems related to contaminants, several tools were needed to cover the same segments. No tools dealing with cumulative effects of pressures, fishery management, noise, nonindigenous species, or hydrography were found to address issues related to impacts on welfare or responses as measures.
Most of the DSTs were different types of assessment tools (17 tools) and models (16 tools), but also planning tools (4 tools) and tools for stakeholders (5 tools) were identified. Both models and assessment tools covered a wide range of environmental problems and could be used to assess environmental status or pressures. Also societal impacts and measures, including cost-effectiveness, could be evaluated with these tools. Planning tools mainly addressed sea-area use, i.e., MSP. Stakeholder tools, i.e., the tool type for interactions with stakeholders, covered a variety of problem settings including fisheries, eutrophication, conservation, and impact evaluation.
DST Performance
DSTs fulfilling at least four of the DC were analyzed in more detail regarding their performance (DC listed in Table 1, PC scoring described in Table 2 and illustrated in Fig. 3). A clear majority of these DSTs scored high (5 or 4) in PC4 management relevance to the Baltic Sea (96% of the DSTs), PC1 scientifically documented (81% of the DSTs), and PC3 transparency (77% of the DSTs). In contrast, the DSTs scored low (1 or 2) in PC6 temporal limitations (69% of the DSTs), PC8 data dependencies (62% of the DSTs), and PC7 confidence (58% of the DSTs).
Some general pattern could be observed regarding how the DSTs covered the DAPSIWRM framework (Fig. 4). DSTs covering less than three DAPSIWRM segments, generally had higher scores in PC7 confidence and PC15 time effort, than tools covering more than three DAPSIWRM segments. Vice versa, DSTs covering more than three DAPSIWRM segments applied more complex methodologies (PC2) and were thematically broader (PC11) than DST that inspected the DAPSIWRM framework more narrowly. Tools inspecting activities (n = 11), pressures (n = 16), or state changes (n = 23) had high management relevance (PC4), but where typically sensitive to the type of input data (PC8) and were thematically narrower (PC11) than tools exploring the other DAPSIWRM segments. DSTs addressing impacts on welfare (n = 8) and responses using measures (n = 6) were generally thematically broad and flexible to different policy issues (PC11).
Comparing DSTs developed for different problem areas, some general patterns could be observed, although the number of tools per problem area was low (Fig. 5). Opportunities to examine temporal changes (PC6) were best implemented in tools related to eutrophication (n = 7) and contaminant pollution (n = 3). DSTs addressing eutrophication were generally also well compatible with available input data and the outputs were operationally used (PC13). Confidence (PC7) was best documented in DSTs dealing with biodiversity issues (n = 5). Tools addressing impact evaluation (n = 3) and sea area use (n = 3) generally had a broad thematic scope (PC11), entailing several policy issues. Transparency (PC3) and management relevance (PC4) was generally high in DSTs in all problem areas.
Through the eyes of an end user, the most important features of a DST included PC3 transparency, PC7 confidence, PC4 management relevance to the Baltic Sea, and PC15 time effort, which 93%, 86%, 83%, and 82% of end users, respectively, rated as very important or important (Fig. 6). PC15 Time effort was especially valued by end users working in administration. PC deemed unimportant by end users were PC11 thematic broadness, PC10 transferability, and PC12 DAPSIWRM component broadness, for which 67%, 60%, and 56% of end users, respectively, responded not important at all or not so important.
Responses from Hosts on Tool Development
Approximately two thirds of the DST hosts answered the survey related to the development and current use of the tools (Fig. 7). According to the answers, two thirds of the tools were initiated as a response to a management need. End users had a strong role in the initial tool development process for half of the DSTs. Once developed, the DSTs are maintained and updated to a varying degree as 60% of the developed DSTs had allocated funding for maintaining and updating the tools. A quarter of the tools are not actively updated or maintained. More than a third of the tools are reported not to be used for environmental management as intended by the tool hosts/developers.
The involvement of end users in DST development varied between different problem areas (Table 4). Tools addressing eutrophication-related problems were normally developed in close cooperation with end users, and in half of the DSTs end users were part of the development team. DSTs concentrating on impact evaluations and cumulative effects also had good contact with end users. For tools addressing questions related to biodiversity and conservation as well as contaminants, end users were not well integrated into the development process. Although contact with end users was maintained when developing tools related to fishery management, nonindigenous species and underwater noise, the end users were not directly involved in the development teams.
To evaluate if (i) DSTs with end-user involvement in development, (ii) DSTs developed in response to a need, or (iii) DSTs where hosts were satisfied with the tool use perform better than where this has not been the case, we chose the three PC that end users valued the most, namely PC3 transparency, PC4 management relevance to the Baltic Sea, and PC7 confidence. However, only small differences were found in the median values between the groups.
Discussion
The DSTs in our inventory often focused on specific environmental problems, and seldom provided solutions for a complete ecosystem approach or integrated management of marine environment. The problems addressed in the tools mirror the main challenges in the Baltic Sea (HELCOM 2018c; Reusch et al. 2018). The most addressed problem in the existing DSTs, eutrophication, as well as the frequently addressed biodiversity and contaminant issues are defined as priority areas in the HELCOM BSAP. Also, other marine environmental policies relevant for the Baltic Sea, such as the EU WFD and the MSFD, focus on these environmental issues. Tools addressing aspects relevant for the Maritime Spatial Planning Directive, e.g., trade-offs between human activities and conservation, were also well represented. Relatively new aspects from a marine management point-of-view, such as underwater noise and litter, were on the other hand incompletely represented in the inventory.
Compliance with the Ecosystem Approch
One of our main interests was to investigate how the DSTs fit into the DAPSIWRM framework, as the ecosystem approach currently is the aim in marine management (Atkins et al. 2011; Elliott et al. 2017; Hyytiäinen et al. 2014; Reusch et al. 2018; Scharin et al. 2016). As DSTs most often are designed to deliver answers to specific question settings (Sullivan 2004), it was not expected that many tools would address the whole DAPSIWRM framework. This was also the case, as only three tools were found to consider all segments. Most often, three or less segments of the DAPSIWRM framework could be addressed with a single tool, with a dominant focus on the activities–pressures–state changes segments or links between them. DSTs were found to address this part of the DAPSIWRM framework in almost all considered environmental problem areas. The links between activities and pressures are well-defined (e.g., HELCOM 2018b), and DSTs focusing on single pressures and DSTs estimating cumulative effects were found in the inventory. Dynamic models assessing pressure-related environmental changes and tools mapping pressures spatially were also found. Some of these DSTs covered the entire Baltic Sea, while others were directed toward supporting management at nation level. DSTs estimating effects of cumulative pressures were based on similar principles and all had a spatial approach. State assessment tools were often indicator-based, providing systems for integration and aggregation of indicator data.
It is evident that tools addressing the impacts on welfare (the socio-economic perspective) and linking to management responses are clearly underrepresented compared to those concentrating on the environmental and ecological impacts of human activities and the pressures they cause. A relatively large number of cost-effectiveness models have been developed and used to assess nutrient abatement in the Baltic Sea (e.g., Ahlvik et al. 2014; Elofsson 2010; Hasler et al. 2014; Hasler et al. 2019; Hyytiäinen et al. 2014), but the use of these tools to support choices between policy options are relatively rare. Some models are used in cost–benefit analysis frameworks that could qualify as DSTs (Hyytiäinen et al. 2014; Scharin et al. 2016), but those made for the Baltic Sea have not been developed into DSTs. Also, Pınarbaşı et al. (2017) found that DSTs used in MSP mostly focused on environmental problems and seldom incorporated socio-economic aspects, which was also pointed out as a weakness by end users of MSP tools (Pınarbaşı et al. 2019).
As DSTs addressing the impacts on welfare–responses using measures–segments of the DAPSIWRM framework are so strongly underrepresented, we may conclude that the ecosystem approach is not very well expressed by the currently available DSTs. The fact that few tools addressing impacts on welfare were operational online, underpins the need for further development of this aspect. Mee et al. (2015) describe challenges of linking social and ecological components in the ecosystem approach, identifying an institutional gap between the disciplines and e.g., differences in interpretation of impacts, while Hasler et al. (2016) and Scharin et al. (2016) discuss problems linking marine data and economic models, but also suggest solutions. Bateman et al. (2013) have, among others, demonstrated that this is possible for terrestrial ecosystems, providing guidelines for how an ecosystem services mapping exercise can be made and used for decision support. Although the DAPSIWRM framework clarifies such epistemic discrepancies (Elliott et al. 2017), solutions for better linkage of environmental and socio-economic aspects are not yet mirrored in the available DSTs. Tools that did estimate the impact on welfare included a number of integrated indicator-based assessment methods, which were not exclusively focused on environmental aspects, but also considered economic aspects as well as contributions to social well-being or welfare. These included sustainability assessments methods, cost–benefit analysis tools, ecosystem services assessment methods, as well as tools that assess public or stakeholder concerns and preferences. DSTs evaluating responses and measures linked back to the other segments in the DAPSIWRM framework. Most frequently, measures were linked to state changes, but also activities, pressures, and impacts on welfare were often addressed, allowing for a more holistic evaluation of the effectivity of measures.
Drivers, setting the requirements for management, are formed by basic human needs (food, space, recreation, etc.), but few of the existing DSTs actually identified such drivers. Many of the DSTs in our inventory responded to specific question settings, where the societal drivers might be clear although not directly included in the tool. In comparison to the tools used in management, those directed toward citizens to raise awareness on the impact of their consumption, such as carbon footprint or nutrient pollution calculators (not included in the inventory), often take a starting point from the basic needs of people such as housing, food, and transportation. Citizens can, based on the drivers, evaluate how their consumption choices impact their welfare. Including clearer links to the drivers in management tools could help in identifying alternative solutions to fulfill the needs, thus facilitating the assessment of impacts on welfare. Consequently, this would facilitate appliance of the ecosystem approach.
Performance of DSTs and End User Preferences
In general, the DSTs included in our inventory were well documented and transparent. DSTs are most often science-driven (Bolman et al. 2018), meaning that transparency and repeatability (i.e., that methodology is sufficiently described), is of high standard in the tools. Transparency was also, together with confidence, one of the most important properties end users pointed out. Transparency and confidence are important factors in the decision-making process, as managers may need to defend their decision choices to stakeholders (Verweij and van Densen 2010). Confidence estimates can facilitate the decision-making, whereas transparency makes it easier to explain the background and defend the decision choice.
Confidence was one of the PC scoring lowest among the tools in our study. Challenges in estimating the uncertainty is often a reason for not communicating uncertainty in DST outputs (Borja et al. 2016; van Beest et al. 2020). As measures can be expensive to carry out, managers want to be confident in any decision taken. In environmental management, reducing the uncertainty in available information can prove cost-efficient (Nygård et al. 2016). van Beest et al. (2020) discuss possibilities for improving communication of uncertainty in decision-support tools, arguing that in order to successfully manage the environment, sources of uncertainty need to be quantified, incorporated, and communicated throughout the tool development and decision-making process.
Broadness of tools, both in ability to inspect several policy issues and segments in the DAPSIWRM framework, was generally not considered important by end users. In fact, this was also reflected in the existing DSTs: they did not perform well in the criterion. As mentioned, DSTs should provide answers to specific questions (Sullivan 2004) and this is obviously also something end users value. The challenges in applying the ecosystem approach in a single DST discussed earlier in combination with the wish for specific DSTs imply that instead of one universal tool, a toolbox of DSTs seem to be the most realistic way to implement the ecosystem approach in marine management. An optimal way forward would be that the DSTs in this toolbox had the ability to utilize the outputs from one tool as inputs into the next one. Such a toolbox would maintain transparency, but potentially impede estimates of uncertainty sources to be retained throughout the management chain.
Although the spatial dimension is well incorporated in the DSTs, the examination of temporal aspects is usually restricted. The DSTs are most often temporally fixed, e.g., to the current situation or to a certain assessment period or scenario, not incorporating temporal trends or projections into future. A similar lack in DSTs used in MSP was also identified by Pınarbaşı et al. (2017). DSTs are usually also dependent on a certain type of data input, not always flexible to fully incorporate e.g., available monitoring data or operationally produced data products such as indicator data.
Host Experiences
A marked number of the DST hosts were of the opinion that their tool had not been applied by the management community to the extent expected and/or could not say that they were satisfied with the way the end users applied their tool. Part of the hosts had the impression that their tool was applied broadly, although not for decision-making. Put together, this suggests that the existing DSTs are not applied to their full potential. Furthermore, it seems that even though many end users are well informed, there is an information gap between the supply/development and use/demand of DSTs.
One way of increasing the knowledge, experience, and commitment of end users to DSTs is through involvement already during the development phase. End user involvement in DST development is important for improving user-friendliness and ability to account for issues related to the decision-making process (Pınarbaşı et al. 2017; Bolman et al. 2018). End user involvement is also needed to improve inclusion of existing regulatory and legal demands (Pınarbaşı et al. 2019). We suggest that a common feature to a successful DST is early involvement of end users, since this promotes end user commitment and ensures a more accurate implementation of their needs. Most of the DSTs in the Baltic Sea area were developed in response to a need, and half of the DSTs were actually initiated by end users. End users, however, had been part of the development team in less than a quarter of the cases. Involvement through interviews and questionnaires was more common, but to secure that DSTs are user-friendly (also for non-scientists) and increase their use, closer involvement of end users is recommended.
Concluding Remarks
In summary, the currently available DSTs address the major environmental problems in the Baltic Sea. To facilitate the implementation of the ecosystem approach, we suggest that attempts to combine the outputs of existing tools in further development of DSTs could lead to more holistic insight of the consequences of management actions. We recommend closer involvement of stakeholders and end users in the DST development phase, in order to improve the tool’s usability and thus, increase the value and impact of the DSTs in environmental management.
Data Availability
The collected material on the Decision-Support Tools is available in the DESTONY Database: http://nest.su.se/bonus_dst/.
References
Ahlvik L, Ekholm P, Hyytiäinen K, Pitkänen H (2014) An economic–ecological model to evaluate impacts of nutrient abatement in the Baltic Sea. Environ Model Softw 55:164–175
Airoldi L, Beck MW (2007) Loss, status and trends for coastal marine habitats of Europe. Oceanogr Mar Biol: Annu Rev 45:345–405
Andersen JH, Murray C, Larsen MM, Green N, Høgåsen T, Dahlgren E, Garnaga-Budré G, Gustavson K, Haarich M, Kallenbach EMF, Mannio J, Strand J, Korpinen S (2016) Development and testing of a prototype tool for integrated assessment of chemical status in marine environments. Environ Monit Assess 188:115. https://doi.org/10.1007/s10661-016-5121-x
Aroviita J, Mitikka S, Vienonen S (eds) (2019) Pintavesien tilan luokittelu ja arviointiperusteet vesienhoidon kolmannella kaudella. Suomen ympäristökeskuksen raportteja 37:2019. Finnish Environment Institute SYKE, Helsinki. https://helda.helsinki.fi/handle/10138/306745
Atkins JP, Burdon D, Elliott M, Gregory AJ (2011) Management of the marine environment: integrating ecosystem services and societal benefits with the DPSIR framework in a systems approach. Mar Pollut Bull 62:215–226
Baartmans R, van Tongeren W, van der Vlies J, Ullrich S, Mattila T, Palm Cousins A, Belhaj M, Munthe J, Pacyna J, Sundseth K (2009) DSS handbook SOCOPSE. SOCOPSE Project Report D4.1. University of Southampton, UK. https://www.researchgate.net/publication/279257247. Accessed 30 Apr 2020
Bagstad KJ, Semmens DJ, Waage S, Winthrop R (2013) A comparative assessment of decision-support tools for ecosystem services quantification and valuation. Ecosyst Serv 5:27–39. https://doi.org/10.1016/j.ecoser.2013.07.004
Balana BB, Vinten A, Slee B (2011) A review on cost-effectiveness analysis of agri-environmental measures related to the EU WFD: Key issues, methods, and applications. Ecol Econ 70:1021–1031
Bateman IJ, Harwood AR, Mace GM, Watson RT, Abson DJ, Andrews B, Binner A, Crowe A, Day BH, Dugdale S, Fezzi C, Foden J, Hadley D, Haines-Young R, Hulme M, Kontoleon A, Lovett AA, Munday P, Pascual U, Paterson J, Perino G, Sen A, Siriwardena G, van Soest D, Termansen M (2013) Bringing ecosystem services into economic decision-making: land use in the United Kingdom. Science 341(6141):45–50. https://doi.org/10.1126/science.1234379
Berg T, Murray C, Carstensen J, Andersen JH (2019) NEAT—nested environmental status assessment tool manual—version 1.4. http://www.devotes-project.eu/wp-content/uploads/2019/06/NEAT-manual-v1.4.pdf. Accessed 30 Apr 2020
Bolman B, Jak RG, van Hoof L (2018) Unravelling the myth—the use of decisions support systems in marine management. Mar Policy 87:241–247. https://doi.org/10.1016/j.marpol.2017.10.027
BONUS BASMATI Project. User guide, Baltic Explorer v 1.0.0. Aalborg University, Denmark. http://balticexplorer.eu/static/umap/BalticExplorerUserGuide.pdf. Accessed 30 Apr 2020
Borja A, Elliott M, Andersen JH, Berg T, Carstensen J, Halpern BS, Heiskanen A-S, Korpinen S, Lowndes JSS, Martin G, Rodriguez-Ezpeleta N (2016) Overview of integrative assessment of marine systems: the ecosystem approach in practice. Front Mar Sci 3:20. https://doi.org/10.3389/fmars.2016.00020
Boyes SJ, Elliott M (2014) Marine legislation—the ultimate ‘horrendogram’: international law, European Directives & National Implementation. Mar Pollut Bull 86:39–47
Burchard H, Bolding K (2002) GETM, a General Estuarine Transport Model: scientic documentation. Technical Report EUR 20253 EN. European Commission, Ispra
Czajkowski M, Zandersen M, Aslam U, Angelidis I, Becker T, Budziński W, Zagórska K (2018) Recreational value of the Baltic Sea: a spatially explicit site choice model accounting for environmental conditions, Working papers 2018-11, Faculty of Economic Sciences, University of Warsaw. https://ideas.repec.org/p/war/wpaper/2018-11.html. Accessed 30 Apr 2020
EEA (1999) Environmental indicators: typology and overview. European Environment Agency, Copenhagen
Eigaard OR, Bastardie F, Hintzen N, Buhl-Mortensen L, Buhl-Mortensen P, Catarino R, Dinesen GE, Egekvist J, Fock HO, Geitner K, Gerritsen HD, González MM, Jonsson P, Kavadas S, Laffargue P, Lundy M, Gonzalez-Mirelis G, Nielsen JR, Papadopoulou N, Posen PE, Pulcinella J, Russo T, Sala A, Silva C, Smith CJ, Vanelslander B, Rijnsdorp AD (2016) Benthic impact of fisheries in European waters: the distribution and intensity of bottom trawling. ICES J Mar Sci 74:847–865. https://doi.org/10.1093/icesjms/fsw194
Elliott M (2011) Marine science and management means tackling exogenic unmanaged pressures and endogenic managed pressures—a numbered guide. Mar Pollut Bull 62:651–655
Elliott M (2013) The 10-tenets for integrated, successful and sustainable marine management. Mar Pollut Bull 74(1):1–5
Elliott M, Burdon D, Atkins JP, Borja A, Cormier R, de Jonge VN, Turner RK (2017) “And DPSIR begat DAPSI(W)R(M)!”—a unifying framework for marine environmental management. Mar Poll Bull 118(1–2):27–40
Elofsson K (2010) Cost-effectiveness of the Baltic Sea Action Plan. Mar Policy 34(5):1043–1050
EPA (1994) A conceptual framework to support the development and use of environmental information. EPA 230-R-94-012. Environmental Statistics and Information Division, Office of Policy, Planning and Evaluation, USEPA, Washington DC
Fleming-Lehtinen V, Andersen JH, Carstensen J, Lysiak-Pastuszak E, Murray C, Pyhälä M, Laamanen M (2015) Recent developments in assessment methodology reveal an expanding eutrophication problem area in the Baltic Sea. Ecol Indic 48:380–388. https://doi.org/10.1016/j.ecolind.2014.08.022
Fyhr F, Nikolopoulos A (2016) The BIAS soundscape planning tool for underwater continuous low frequency sound—user guide. AquaBiota Water Research, Stockholm. https://biasproject.files.wordpress.com/2017/12/bias_soundscapeplanningtool_userguide_201712.pdf. Accessed 30 Apr 2020
Hansen HS (2019) Cumulative impact of societal activities on marine ecosystems and their services. In: Misra S, Gervasi O, Murgante B, Stankova E, Korkhov V, Torre C, Taniar AMACD, Apduhan BO, Tarantino E (eds) Computational science and its applications—ICCSA 2019—19th International Conference Proceedings, Part III, lecture notes in Computer Science (LNCS), vol 11621. Springer, Saint Petersburg, Russia, pp 577–590. https://doi.org/10.1007/978-3-030-24302-9_41
Hasler B, Ahtiainen H, Hasselström L, Heiskanen A-S, Soutukorva Å, Martinsen L (2016) Marine ecosystem services: marine ecosystem services in Nordic marine waters. Nordic Council of Ministers, Copenhagen
Hasler B, Hansen LB, Andersen HE, Termansen M (2019) Cost-effective abatement of non-point source nitrogen emissions: The effects of uncertainty in retention. J Environ Manag 246:909–919. https://doi.org/10.1016/j.jenvman.2019.05.140
Hasler B, Smart JCR, Fonnesbech-Wulff A, Andersen HE, Thodsen H, Blicher-Mathiesen G, Smedberg E, Göke C, Czajkowski M, Was A, Elofsson K, Humborg C, Wolfsberg A, Wulff F (2014) Hydro-economic modelling of cost-effective transboundary water quality management in the Baltic Sea. Water Resour Econ 5:1–23. https://doi.org/10.1016/j.wre.2014.05.001
HELCOM (2015) HELCOM eutrophication assessment manual. Helsinki Commission - HELCOM, Helsinki. https://helcom.fi/media/publications/Eutrophication-assessment-manual.pdf. Accessed 30 Apr 2020
HELCOM (2018a) The sixth pollution load compilation (PLC-6). Helsinki Commission - HELCOM, Helsinki. https://helcom.fi/media/publications/PLC-6-Executive-Summary.pdf. Accessed 24 Apr 2020
HELCOM (2018b) Thematic assessment of cumulative impacts on the Baltic Sea 2011–2016. Baltic Sea Environment Proceedings No. 159. Helsinki Commission - HELCOM, Helsinki
HELCOM (2018c) State of the Baltic Sea—second HELCOM holistic assessment 2011–2016. Baltic Sea Environment Proceedings No. 155. Helsinki Commission - HELCOM, Helsinki
Huttunen I, Huttunen M, Piirainen V, Korppoo M, Lepistö A, Räike A, Tattari S, Vehviläinen B (2016) A national scale nutrient loading model for Finnish watersheds—VEMALA. Environ Model Assess 21(1):83–109. https://doi.org/10.1007/s10666-015-9470-6
Hyytiäinen K, Blyh K, Hasler B, Ahlvik L, Ahtiainen H, Artell J, Ericsdotter S (2014) Environmental economic research as a tool in the protection of the Baltic Sea: costs and benefits of reducing eutrophication. TemaNord 504, Nordic Council of Ministers, Copenhagen. https://doi.org/10.6027/TN2014-504
Inácio M, Schernewski G, Nazemtseva Y, Baltranaitė E, Friedland R, Benz J (2018) Ecosystem services provision today and in the past: a comparative study in two Baltic lagoons. Ecol Res 33:1255–1274. https://doi.org/10.1007/s11284-018-1643-8
Karnauskaitė D, Schernewski G, Schumacher J, Grunert R, Povilanskas R (2018) Assessing coastal management case studies around Europe using an indicator based tool. J Coast Conserv 22(3):549–570
Korpinen S, Meidinger M, Laamanen M (2013) Cumulative impacts on seabed habitats: an indicator for assessments of good environmental status. Mar Pollut Bull 74:311–319. https://doi.org/10.1016/j.marpolbul.2013.06.036
Korpinen S, Meski L, Andersen JH, Laamanen M (2012) Human pressures and their potential impact on the Baltic Sea ecosystem. Ecol Indic 15:105–114. https://doi.org/10.1016/j.ecolind.2011.09.023
Kreins P, Gömann H, Herrmann S, Kunkel R, Wendland F (2007) Integrated agricultural and hydrological modeling within an intensive livestock region. In: Erickson JD, Messner F, Ring I (eds) Ecological economics of sustainable watershed management (advances in the economics of environmental resources, vol. 7). Emerald Group Publishing Limited, Bingley, pp 113–142. https://doi.org/10.1016/S1569-3740(07)07006-X
In: Lindegarth M, Carstensen J, Drakare S, Johnson RK, Nyström Sandman A, Söderpalm A, Wikström SA (eds) (2016) Ecological assessment of Swedish water bodies; development, harmonisation and integration of biological indicators. Final report of the research programme WATERS. Deliverable 1.1-4, WATERS report no 2016:10, Havsmiljöinstitutet, Sweden
Loh J, Green RE, Ricketts T, Lamoreux JF, Jenkins M et al. (2005) The Living Planet Index: using species population time series to track trends in biodiversity. Philos Trans R Soc Lond B 360:289–295
MareFrame project. MareFrame decision support framework: co-creating ecosystem-based fisheries management Solutions. https://mareframe.github.io/dsf/DST_Guidelines.pdf. Accessed 30 Apr 2020
MARMONI project. Marmoni Biodiversity Assessment Tool. www.sea.ee/marmoni/Marmoni%20Biodiversity%20assessment%20tool%20-%20Documentation.pdf. Accessed 30 Apr 2020
Mee L, Cooper P, Kannen A, Gilbert AJ, O’Higgins T (2015) Sustaining Europe’s seas as coupled social-ecological systems. Ecol Soc 20(1):1. https://doi.org/10.5751/ES-07143-200101
Menegon S, Sarretta A, Depellegrin D, Farella G, Venier C, Barbanti A (2018) Tools4MSP: an open source software package to support Maritime Spatial Planning. PeerJ Comput Sci 4:e165. https://doi.org/10.7717/peerj-cs.165
Moilanen A, Pouzols FM, Meller L, Veach V, Arponen A, Leppänen J, Kujala H (2014) Zonation—spatial conservation planning methods and software. Version 4. User manual. C-BIG Conservation Biology Informatics Group, Department of Biosciences, University of Helsinki, Finland. https://github.com/cbig/zonation-core/releases/download/4.0.0/zonation_manual_v4_0.pdf. Accessed 30 Apr 2020
Neset T-S, Wilk J (2018) Toolkit for use of visualization to facilitate social learning. Deliverable 4.6. BONUS MIRACLE Project Report. Linköping University, Sweden. http://www.bonus-miracle.eu/wp-content/uploads/2018/09/call2014-73_D4.6_0.1.pdf. Accessed 30 Apr 2020
Neumann T, Fennel W, Kremp C (2002) Experimental simulations with an ecosystem model of the Baltic Sea: a nutrient load reduction experiment. Glob Biogeochem Cycles 16(3):1033–1054
Nicholson F, Krogshave Laursen R, Cassidy R, Farrow L, Tendler L, Williams J, Surdyk N, Velthof G (2020) How can decision support tools help reduce nitrate and pesticide pollution from agriculture? A literature review and practical insights from the EU FAIRWAY project. Water 12:768. https://doi.org/10.3390/w12030768
Nygård H, Murray C, Andersen JH, Martin G, Torn K, Korpinen S (2018) BEAT 3.0—a tool for integrated biodiversity assessments. J Open Res Softw 6:19. https://doi.org/10.5334/jors.226
Nygård H, Oinonen S, Hällfors HA, Lehtiniemi M, Rantajärvi E, Uusitalo L (2016) Price vs. value of marine monitoring. Front Mar Sci 3:205. https://doi.org/10.3389/fmars.2016.00205
OECD (1993) OECD core set of indicators for environmental performance reviews. A synthesis report by the Group on the State of the Environment. OECD, Paris
Oltmans J, Bohlen J-M, Escher S, Schwarz M, Licht O (2019) Final report: applying a tested procedure for the identification of potential emerging chemical risks in the food chain to the substances registered under REACH—REACH 2. External scientific report. OC/EFSA/SCER/2016/01-CT1. https://efsa.onlinelibrary.wiley.com/doi/pdf/10.2903/sp.efsa.2019.EN-1597
Pınarbaşı K, Galparsoro I, Borja A, Stelzenmüller V, Ehler CN, Gimpel A (2017) Decision support tools in marine spatial planning: present applications, gaps and future perspectives. Mar Policy 83:83–91. https://doi.org/10.1016/j.marpol.2017.05.031
Pınarbaşı K, Galparsoro I, Borja A (2019) End users’ perspective on decision support tools in marine spatial planning. Mar Policy 108:103658. https://doi.org/10.1016/j.marpol.2019.103658
Rapport D, Friend A (1979) Towards a comprehensive framework for environmental statistics: a stress-response approach. Statistics Canada catalogue 11-510. Minister of Supply and Services Canada, Ottawa
Reusch TBH, Dierking J, Andersson H, Bonsdorff E, Carstensen C, Casini M, Czajkowski M, Hasler B, Hinsby K, Hyytiainen K, Johannesson K, Jomaa S, Jormalainen V, Kuosa H, Kurland S, Laikre L, MacKenzie B, Maronski P, Melzner F, Oesterwind D, Ojaveer H, Refsgaard JC, Sandström A, Schwarz G, Tonderski K, Winder M, Zandersen M (2018) The Baltic Sea: a time machine for the global future ocean? Sci Adv 5:eaar8195. https://doi.org/10.1126/sciadv.aar8195
Ruiz M, Sethuraman S (2015) BALSAM Risk Assessment Tool. User guide. Helsinki Commission - HELCOM, Helsinki. http://jointbwmexemptions.org/ballast_water_RA/apex/r/balwat/100/files/static/v1Y/BALSAM_Tool_User_Guide.pdf. Accessed 30 Apr 2020
Scharin H, Ericsdotter S, Elliott M, Turner K, Niiranen S, Blenckner T, Hyytiäinen K, Ahlvik L, Ahtiainen H, Artell J, Hasselström L, Söderqvist T, Rockström J (2016) Processes for the sustainable stewardship of marine environments. Ecol Econ 128:55–67
Schumacher J, Schernewski G, Bielecka M, Loizides MI, Loizidou XI (2018) Methodologies to support coastal management—a stakeholder preference and planning tool and its application. Mar Policy 94:150–157
Sharp R, Tallis HT, Ricketts T, Guerry AD, Wood SA, Chaplin-Kramer R, Nelson E, Ennaanay D, Wolny S, Olwero N, Vigerstol K, Pennington D, Mendoza G, Aukema J, Foster J, Forrest J, Cameron D, Arkema K, Lonsdorf E, Kennedy C, Verutes G, Kim CK, Guannel G, Papenfus M, Toft J, Marsik M, Bernhardt J, Griffin R, Glowinski K, Chaumont N, Perelman A, Lacayo M, Mandle L, Hamel P, Vogl AL, Rogers L, Bierbower W, Denu D, Douglass J (2020) InVEST 3.8.0.post63+ug.g2fd0aa1 user’s guide. The Natural Capital Project, Stanford University, University of Minnesota, The Nature Conservancy and World Wildlife Fund. Stanford, California. http://releases.naturalcapitalproject.org/invest-userguide/latest/. Accessed 30 Apr 2020
Stelzenmüller V, Lee J, South A, Foden J, Rogers SI (2013) Practical tools to support marine spatial planning: a review and some prototype tools. Mar Policy 38:214–227. https://doi.org/10.1016/j.marpol.2012.05.038
Stock A (2016) Open source software for mapping human impacts on marine ecosystems with an additive model. J Open Res Softw 4(1):e21. https://doi.org/10.5334/jors.88
Støttrup JG, Dinesen GE, Schumacher J, Gillgren C, Inácio M, Schernewski G (2019) The systems approach framework for collaborative, science-based management of complex systems. J Coast Conserv 23:881–898
Sullivan T (2004) Evaluating environmental decision support tools. Brookhaven National Laboratory, United States, 10.2172/15016504
Swedish Agency for Marine and Water Management (2017) Symphony—a tool fro ecosystem-based marine spatial planning. Swedish Agency for Marine and Water Management, Gothenburg, Sweden. https://www.havochvatten.se/en/swam/eu-international/marine-spatial-planning/symphony-a-tool-for-ecosystem-based-marine-spatial-planning.html. Accessed 30 Apr 2020
Turner RK, Schaafsma M (eds) (2015) Coastal zones ecosystem services: from science to values and decision making. Studies in Ecological Economics 9, Springer International Publications, Switzerland
Undeman E, Gustafsson BG, Humborg C, MacLachan MS (2015) Application of a novel modeling tool with multistressor functionality to support management of organic contaminants in the Baltic Sea. AMBIO 44:498–506. https://doi.org/10.1007/s13280-015-0668-2
UNEP (1994) World Environment Outlook: brainstorming session. ENEP/EAMR. 94-5. UNEP, Environment Assessment Programme, Nairobi
van Beest F, Nygård H, Fleming V, Carstensen J (2020) On the uncertainty and confidence in decision support tools (DSTs) with insights from the Baltic Sea ecosystem. AMBIO. https://doi.org/10.1007/s13280-020-01385-x
Venohr M, Hirt U, Hofmann J, Opitz D, Gericke A, Wetzig A, Natho S, Neumann F, Hürdler J, Matranga M, Mahnkopf J, Gadegast M, Behrendt H (2011) Modelling of nutrient emissions in river systems—MONERIS—methods and background. Int Rev Hydrobiol 96(5):435–483
Verweij MC, van Densen WLT (2010) Differences in causal reasoning about resource dynamics and consequences for the participatory debate on North Sea fisheries. Mar Policy 34:1144–1155. https://doi.org/10.1016/j.marpol.2010.03.014
Wania F, Persson J, Di Guardo A, McLachlan MS (2000) The POPCYCLING-Baltic Model a non-steady state multicompartment mass balance model of the fate of persistent organic pollutants in the Baltic Sea Environment. NILU OR 10/2000. https://www.utsc.utoronto.ca/labs/wania/wp-content/uploads/sites/4/2014/02/NILU_OR_10_2000.pdf. Accessed 30 Apr 2020
Ward FA (2007) Decision support for water policy: a review of economic concepts and tools. Water Policy 9(1):1–31. https://doi.org/10.2166/wp.2006.053
Wulff F, Sokolov A, Savchuk O (2013) Nest—a decision support system for management of the Baltic Sea. A user manual. Technical Report No. 10, Baltic Nest Institute, Stockholm University, Sweden. https://www.su.se/polopoly_fs/1.417210.1544612031!/menu/standard/file/TR%2010%20-%20Nest%20user%20manual%20MINDRE.pdf. Accessed 30 Apr 2020
Acknowledgements
We wish to thank the experts involved in filling DST templates and the DST developers/hosts for checking the templates and providing information on the development of the tools and their experiences in usage of the tools. End users and stakeholders are also thanked for their input on their preferences on DST characteristics. Open access funding provided by Finnish Environment Institute (SYKE).
Funding
The study received funding from BONUS (Art. 185), funded jointly by the EU, the Academy of Finland, Innovation Fund Denmark and the Swedish Research Council FORMAS. J.S. was supported by the Doctorate Study program in Ecology and Environmental Sciences, Klaipeda University.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Nygård, H., van Beest, F.M., Bergqvist, L. et al. Decision-Support Tools Used in the Baltic Sea Area: Performance and End-User Preferences. Environmental Management 66, 1024–1038 (2020). https://doi.org/10.1007/s00267-020-01356-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00267-020-01356-8