Introduction

Our coastal zones are facing serious problems of habitat destruction, water contamination, coastal erosion and resource depletion also suffer from serious socio-economic and cultural problems such as weakening of the social fabric, marginalization, unemployment and destruction of property by erosion. Given the coast’s critical value and its potential, these problems must be solved (European Parliament and the Council 2002). Both Brundtland Report and Agenda 21 identified the need for a sustainable management of the coastal zone. Integrated Coastal Zone Management (ICZM) seeks to develop an integrated model for sustainable development that is based on finding points of convergence among environmental, socio-economic, cultural and governance factors (Diedrich et al. 2010; Maelfait et al. 2006). To promote sustainable development of coastal zones, the European Commission adopted the document “Integrated Coastal Zone Management: A Strategy for Europe” (COM/2000/547) in 2000, as well as, the “Recommendation concerning the implementation of Integrated Coastal Zone Management in Europe” (2002/413/EC) in 2002. ICZM is defined as “a continuous process with the general aim of implementing sustainable development in coastal zones through optimal management of human activities in this area to improve the state of the coastal environment and maintain its diversity” (European Commission 1999). Until 2010, the European Union (EU) Member States reported their progress in implementing ICZM, and this resulted in a proposal for a Directive establishing a framework for maritime spatial planning and integrated coastal management, adopted by the European Commission (EC) in 2013 (European Commission 2013). However, these activities can not conceal that policy development and practical implementation of ICZM which is a slow process in Europe.

In order to promote ICZM and to stimulate its practical implementation on local level, the EC created an online database called OURCOAST. Over 350 ICZM best practice case studies are documented, to ensure that lessons learned from experiences and practices are shared and available to improve coastal management practices. However, concrete criteria for “best practice” are missing and an evaluation of the single case studies did not take place. It is commonly observed that success and implementation process of ICZM measures are not evaluated. For example, Pendle (2013) recommend that key performance indices of sustainability should be developed, included in predictions and thereafter monitored to provide evidence that measures meet economic, social and environmental sustainability goals.

The measurement of the effectiveness of a management system requires performance measures that have easily comparable goals (Maccarrone et al. 2014; Ehler 2003; Henocque 2003). Indicators are commonly used, because they provide a simplified view of complex phenomena, quantify information, and make it comparable. They can also be used to simplify the extent to which the objectives for the management programmes are being achieved (Malfait et al. 2006). Many coastal indicator sets have been developed for different purposes (Olsen 2003, Pickaver et al. 2004, Breton 2006, Hoffmann 2009, Ballinger et al. 2010; Cummins and McKenna 2010; Azuz-Adeath et al. 2015). An example, is the “Handbook for Measuring the Progress and Outcomes of Integrated Coastal and Ocean Management” published by UNESCO-IOC (Belfiore et al. 2006).

The project DEDUCE developed a list of 27 indicators which were simplified and streamlined into 15 coastal sustainability indicators (Gvilava et al. 2015; DEDUCE Consortium 2007). Project SUSTAIN provided a set of indicators designed to measure sustainable development in coastal areas on a local and regional level (Schernewski et al. 2014). Most of the existing guidelines for ICZM underline the importance of indicators to monitor states and developments in coastal zones and to assess performance (Belfiore et al. 2003; Hoffmann 2009; Gallagher 2010; Meiner 2010; Burbridge 1997; Scottish Executive Central Research Unit 2001). However, in practice the indicator systems are hardly applied. The reasons are the lack of a guided stepwise process and lack of supporting tools that enable an easy and relatively fast application process. More important is that many indicator systems are kept general and do not meet the practical demand or the concrete objectives of an application.

A retrospective assessment of ICZM best practice cases not only requires a tailor-made indicator system, but also a separation of two aspects: a) it has to be possible to measure changes in the state of sustainability before and after the measure implementation considering the categories of sustainability (economics, environmental quality and social well-being); b) it has to measure the ICZM process from issue identification, via planning towards implementation. The indicators can be used as supporting tool for System Approach Framework (SAF) evaluations. SAF is a holistic research approach which addresses the general problem of assessing complex systems for conversion to sustainable development. The best available scientific knowledge shall be mobilized to support deliberative decision-making processes aimed at improving the sustainability of coastal systems by implementing suitable ICZM policies (Hopkins et al. 2011; Newton 2012).

Our objectives are: (1) to present a tailor-made set of indicators suitable for the assessment of ICZM best practice case studies as well as for measuring the quality of the ICZM process and changes in the state of sustainability; (2) to provide a stepwise application methodology and a user-friendly, spread-sheet tool for easy application that builds upon previous approaches; (3) to apply the indicator set to 18 contrasting study sites; (4) to identify strengths and weaknesses of different ICZM case studies; (5) to provide a critical analysis of different application methods by comparing in-depth assessments with a fast screening on 3 German case studies; (6) to analyse the role of different evaluators and their perception, background and required time for applications; and (7) to critically discuss benefits and limitations of our indicator set as well as the consequences of data aggregation in the assessment tool.

Methodology

The stepwise approach for indicator development has been adopted, to cover different coastal themes and consider the coastal interests and concerns at local, regional and national scales. Based on previous projects (DEDUCE, IOC, SUSTAIN, QualityCoast and Progress indicators by Pickaver et al.) we provide a set of indicators that can be applied at different scales of ICZM initiatives or programmes around Europe. The research methodology is divided into five steps (Fig. 1):

Fig. 1
figure 1

An indicator set research methodological steps

Development of tailor-made indicators

Step 1: To make use of the already existing initiatives on sustainability and ICZM progress indicators a review of more than 300 indicators has been undertaken. Indicators have been chosen to cover 3 categories of sustainability (environmental quality, economics, social well-being), moreover, additional governance (process) indicators were included to measure the ICZM process. The first screening of indicators showed that some of indicators are the same or similar in the different methodologies. Step 2: 97 indicators were initially selected using thematic and technical criteria. The thematic criteria: (a) indicators are covering the main coastal issues themes based on European Commission OURCOAST database (adaptation to risk, sustainable use of resources, sustainable economic growth), including indicators related to (b) sustainability and (c) ICZM progress. Technical criteria: (d) ability to be scored, (e) data availability and (f) quality of data sets. Step 3: Selected indicators based on thematic and technical criteria were the basis for creating online checklist. The main aim was to check indicators’ suitability to measure changes. This process required an expert consultation. The question to experts on each indicator was: “Is this indicator suitable to measure changes in the state of sustainability before and after implementation of ICZM initiative?” All 97 indicators were included in this list and answers such as ‘no effect’, ‘some effect’, ‘strong effect’ were inserted. Four coastal management experts filled in the online checklist, the results were collected and analysed. It showed that many indicators chosen were too general to reflect single measures and were not suitable to reflect changes. Step 4: The desk-review and discussion with scientist group provided a good overview of which indicators cover various issues of the coastal zone. The discussion of the proposed indicators led to a reduction of the number of indicators and a rewording of their definition. Additionally, some tailor-made indicators covering coastal issues were suggested. They were included in the new indicator set. In total, 45 tailor-made indicators were selected to reflect changes in the state of sustainability. The indicators are the basis of the assessment and scoring tool, implemented in Microsoft Excel.

Indicator-based ICZM best-practice evaluation tool

The tool is divided in 4 categories Fig. 2: environmental quality, economics, social well-being and governance (process) and it can be used as supporting generalized spreadsheet tool for SAF evaluations.

Fig. 2
figure 2

Indicator-based ICZM best-practice evaluation tool capability to support SAF evaluations based on 4 categories

Indicators of Environmental quality category have been selected to demonstrate the status and changes of sustainable environmental practices. In the tool there are 13 environmental quality indicators which cover many issues of coastal zones such as air, water and soil pollution, coastal erosion, flooding, climate change, biodiversity and land use. Economic indicators represent the accomplishment of sustainable coastal economy. Many issues have been identified which are important for the economic contribution of sustainability, for example the economic stability and resilience, economic diversification, employment opportunities, green economy, investments in climate change mitigation and adaptation. In this category there are 9 indicators. Social well-being indicators, 9 in this category, have been chosen to promote social justice and equal opportunities for all members of society. Issues of social well-being have been selected as quality of life, educational opportunities, and conservation of cultural heritage, crime prevention and safety. Governance (process) indicators have been selected to show to what extent the objectives of integrated management initiative have been practically achieved. There are14 indicators of this category. The tool consists of 6 worksheets. The first worksheet introduces the tool and provides guidelines to the user, additionally, there is space (highlighted) to provide the title, map and pictures of the study site. The next four worksheets are divided into categories of indicators which are compiled in the tables. Please see an example of the tool in Fig. 3.

Fig. 3
figure 3

Excerpt from the tool of social well-being category

The tables are divided into columns representing: indicator, description, scoring ranges, indicator total score and comments. Two different scoring ranges by Likert scale were developed to give a quantitative score for each indicator. The first scale has seven ranges for scoring the sustainability indicators (environmental quality, economics, and social well-being). As shown in the Fig. 3, indicators need to be scored on scale from a minimum value “-3” to maximum value “3”. The scores indicate a degree of contribution to sustainability: 3 (strong effects), 2 (considerable effects) and 1 (weak effects). The minus values show negative effects and other values show positive effects, with the exception of “0”, which indicates no contribution to sustainability, i.e., no changes after ICZM measure implementation. The second scoring scale was created for governance (process) indicators. The scale is divided in 5 scoring ranges, ranging from 0 to 4. The 0 value shows that objectives of the indicator were not implemented (no, not at all), other scoring ranges represent the degree of implementation and are divided into: 1 (yes, slightly), 2 (yes, moderately), 3 (yes), 4 (yes, fully). All scores are indicated by the scoring bar under the scoring ranges. The total indicator score will be automatically calculated in the ‘Indicator score’ column and in the ‘Final assessment’ worksheet. The tool is user-friendly and it is easy to understand and to follow the instructions given in the introduction. Aditionally, in the last column of the table the specifications for each indicator need to be filled out by identifying problems occurred by scoring indicators, linking the data sources or adding other additional comments. The comments are important for understanding the coastal systems and later used for interpreting the results. The last worksheet named ‘Final assessment’ shows all calculated results: indicators overall score and the number of the used indicator in the assessment, in forms of tables and graphics. Furthermore, the evaluator of the application should take into account the time used for the application, which is included in the ‘Final assessment’ sheet. The full list of the indicators can be found in the Annex I.

An indicator set application process

An indicator set application process was based on two steps methodology, which was adopted from European INTERREG-IVC-Project SUSTAIN (SUSTAIN partnership 2012). Firstly, the evaluator collected the data related to the indicators. Secondly, the indicators were scored based upon the data using a Likert scale. Based on the score of each indicator, the average was automatically calculated and aggregated in the respective categories. If no data was available, the ‘X’ was inserted in the cell and the indicator was excluded from the calculations. Scores were calculated automatically and showed in the final assessment sheet. The average is shown in decimal numbers from −3 to 3 in the sustainability categories and from 0 to 4 in governance category. Furthermore, the time that the application consumed was filled in the ‘Final assessment’ sheet.

The indicator set was applied to diffrenet case studies using two methods: a fast screening and an analysis in-depth. Whereas the time for an in-depth analysis of one study is unlimited the given time frame for the evaluation of one study via screening is restricted to an exact time limit of a maximum of 12–16 h. Furthermore, we included time restriction in order to find out if a fast screening method by non-experts of study site will be comparable as in-depth evaluation and if the available time resources are restricting. The indicator set was applied in-depth to seven different case studies of Lithuania, Finland and Germany by experts of the study sites. Additionally, in order to compare the two methods, the three applications were repeated using fast screening method for three German cases by non-expert evaluator. Applications for coastal management case study of Abbott’s Hall using the fast screening method were done by 4 evaluators with different backgrounds. The experts conducting in-depth analysis had various data available, due to their long involvement into the respective studies, ranging from different literature, online resources, observations in site, interviews and talks with locals and other specialists. The search of data by fast screening method is regulated by the evaluator and was mostly based on personal knowledge, opinion, theoretical considerations, information publicly available in OURCOAST database and other internet sources. The evaluators had to complete the evaluation on their own and were not allowed to talk to each other, neither about the meaning of a specific indicator nor about anything in regard to the content of the best practice in order to achieve results best comparable. Furthermore, they were not allowed to share their data amongst each other. Questions that arose during the application, especially concerning the understanding of the indicators, had to be solved self-dependently. This way, the comprehensibleness of indicators is tested as well as the possibility to transfer this tool to an application on local levels and by non-scientists. Applications were carried out using the same available data and with the time restrictions without contact to site experts.

There is no overall definition of a best practice or what charactersitics have to be fulfilled to account for one but we are presenting different key elements that have to be fulfilled in order to account for a best practice. The criteria are based on the indicators of sustainability, the ICZM and SAF principles. A case study can be characterised as best practice if it has a broad overall perspective and merges environmental, economic and social aspects, if its measures collaborate with natural systems while respecting their capability and when it is seminal and promising, not just in specific case but also for other regions. Furthermore, an improvement of the state of sustainability in all categories has to be achieved and the protection of the coastal and marine environment should be enacted. The provision of systematic solution, which is responding to the practical needs of the area is important as well as the involvement of all parties concerned. It should mobilise the best available knowledge, which is supporting deliberative decision making process, support and involve relevant administrative bodies. If basically all criteria are fulfilled, the respective case study can be described as a best practice example. However, it is still possible that implemented measures are best practice. This has to be further monitored and evaluated.

Study sites

Seven study sites covering different coastal themes and policy objectives in Germany (Geltinger Birk, Timmendorf, Markgrafenheide), Lithuania (Rusne, Klaipeda) and Finland (Southwest and Western Finland), were selected for analysis in-depth. The objective was to assess if studies are considered as best practice examples (see Fig. 4).

Fig. 4
figure 4

Study sites of in-depth analysis: (a) Geltinger Birk, (b) Timmendorf, (c) Markgrafenheide, (d) Klaipeda, (e) Rusne, (f) Western Finland, (g) Southwest Finland

Additionally, 14 studies from Germany, Sweden, Netherlands, the United Kingdom, Denmark and Ireland were selected for fast screening analysis in order to have an international comparison of different ICZM best practice examples (see Fig. 5). The established criteria leading to these choices were: the best practice describes a measure implemented in coastal areas; the spatial scale; a broad variety in key approaches given, in order to test the tool in different cases to ensure it is widely applicable; sufficient data availability; especially for those studies conducted in countries where German or English is not the first language. The variety of cases led us to test suitabilty of indicators. Study sites used for fast screening method are described by themes, key approaches, objectives and main measures and summarized in a table available in the Annex II.

Fig. 5
figure 5

Study sites selected for fast screening method: (1) Geltinger Birk, (2) Timmendorf, (3) Markgrafenheide, (4) Gotland, (5) Ystad, (6) Køge Bay, (7) Tryggelev Nor, (8) Odense, (9) Rotterdam, (10) Perkpolder, (11) Coastline: Weybourne to Lowestoft, (12) Abbott’s Hall, (13) Horsey Islnad, (14) Inch Beach, Co. Kerry

Additionally, three German case studies (Geltinger Birk, Markgrafenheide and Timmendorf) were chosen to compare the in-depth analysis and fast screening, with the focus on the differences of the evaluations. Moreover, a comparison of the different assessments was made on Abbot‘s Hall (see Fig. 5, number 12), which was evaluated 4 times by different evaluators.

Coastal nature restoration and flooding in Geltinger Birk, Germany

The nature reserve Geltinger Birk is located at the entrance of the Flensburg fjord, in the federal state of Schleswig-Holstein. Geltinger Birk is a coastal wetland with lagoons separated by sandy spits from the Baltic Sea and it covers an area of 10 km2 at the northern German Baltic coast. A combined coastal realignment and nature protection scheme was implemented in this study site. Two major scenarios were developed, either to upgrade the old dyke at costs of 10 million euros or to create a realigned protected coastline with a new shortened dyke with costs of 6 million euros (including costs for land purchase). The second scenario included a re-wetting with a water level 1 m below sea level and the restoration of temporary lagoons. In 1993 the parliament of the federal state of Schleswig-Holstein agreed on the second scenario and demanded local stakeholder involvement in the implementation process. As a consequence, 6 scenarios for the future integrated development of the Birk were developed and commercial agriculture was determined. This process took another 20 years to fully implement the measure and the management concept. In 2013, the implementation of the measure was completed and the first controlled large scale test flooding took place. During the implementation agriculture, a farm and houses, owned by permanent residents, had to be given up. The old dyke was decommissioned and a new one was built to protect the village Falshöft. Problems with local acceptance, public participation and information seriously hampered the project. Climate change, the associated sea level rise and increasing costs for coastal protection, favour combined coastal realignment and wetland restoration measures but require a much faster implementation. The study site of Geltinger Birk is an example of the successful technical implementation from a nature protection and coastal engineering point of view ((Hofstede, personal communication, April 22, 2016).

Fig. 6
figure 6

Results of evaluation in-depth on category level

Fig. 7
figure 7

Fast screening evaluation results of Abbot’s Hall by evaluators with different background

Fig. 8
figure 8

Results comparison of fast screening (grey triangle; light yellow bar) and analysis in-depth (black triangle; dark yellow bar) evaluations

Coastal protection & realignment and the role of public participation in Markgrafenheide, Germany

The study covers a coastal protection and realignment measure in the village of Markgrafenheide, located in the rural part of the city of Rostock. Markgrafenheide has a population of around 560 inhabitants and the local economy depends mostly on tourism. Markgrafenheide and the adjacent nature reserve Hütelmoor have a total outer coastline of around 6 km, with a long history of coastal protection measures. Yet, the village had been further at risk of inland flooding from the ‘Breitling’, a wide opening of the lower Warnow river and was one of the priority cases of coastal protection in the state of Mecklenburg-Western Pomerania. Therefore, plannings for a complex protection scheme, initiated by the State Agency for Environment and Nature, started in the 1990s and a ring-dyke was constructed around the village and completed in 2006. Hütelmoor area was drained until 2008 and used for extensive agriculture. Within the scope of the realignment measure, the pumping station was abandoned and drainage of the area stopped leading to a rise in water levels and partly inundation of meadows (Weisner and Schernewski 2013). Despite being considered an outstanding example from a coastal protection and nature conservation point of view, the measure triggered a long-lasting controversial debate. Stakeholder groups affected by the coastal protection and realignment group include the local populations, different regional and state authorities and agencies, tourism businesses and visitors.

Public participation in Integrated Flood Risk Management in Timmendorf, Germany

Timmendorfer Strand and Scharbeutz are two renowned coastal holiday resorts located on the Baltic Sea coast of Germany. With about 1.3 million overnight stays per year, the local economy depends strongly on tourist activities. At the same time, almost 6000 inhabitants live in coastal flood-prone areas and are at risk from extreme storm surges. The main flood defence is the natural beach-ridge with heights of about 2.5 to 4.0 m above mean sea level. Climate change and sea-level rise called for a new coastal defence strategy in this study site. The municipalities are responsible for flood defence. Coastal defence authorities have, in the past, pointed out the hazard and proposed technical solutions. A solution of building a sea wall on the beach was met with great scepticism by the local community, which strongly depends upon the beach as the main tourist attraction. To overcome this deadlock situation, in 1999, the municipalities and coastal defence authorities agreed upon a new and participative procedure to develop an integrated flood defence solution. As a starting point, a public meeting with about 65 persons was organised, followed by another five more meetings in working groups and a final public meeting. The major objective was to carry out a sensitivity analysis in a participatory process, which included characterisation of the system with appropriate variables, definition of the effects (direction and strength) of the system variables upon each other, definition and simulation of different scenarios focusing on the problem/action and development of recommendations as to which coastal protection measures should be implemented in the region. The objectives were reached within the given time-frame. In a participative process, this strategy was successfully developed by local stakeholders, municipalities and coastal defence authorities (Hofstede and Schernewski 2005).

Integrated shoreline management for a large harbour city and an adjacent seaside resort, Lithuania

The study site focuses on integrated shoreline management of the continental Baltic Sea coast of Lithuania stretching over 45 km from the Klaipeda Strait to the border with Latvia. In 2001, a pilot programme was undertaken to stabilize recreational beaches of Palanga seaside resort, located north of the Klaipeda seaport. These beaches were particularly vulnerable facing increased storm frequency and mismanagement in the previous decades. The nearshore and beach nourishment was aimed to mitigate former human intervention, which caused the deficit of sediments, without compromising the naturalness of the coastal zone of Lithuania. The coastal nourishment is regularly continued since 2001. Nowadays it is a core element of the national integrated shoreline management programme of Lithuania. It balances the necessity to maintain the seaside beaches of Klaipeda and Palanga as the main recreational amenities for seaside tourism, and the need to compensate the affected sectors, primarily the foreshore fisheries (Povilanskas 2010).

Restoration of important habitats through sustainable agricultural practices, Rusne (Lithuania)

This study site focuses on integration of dairy farming on alluvial grasslands with nature conservation of valuable bird habitats. Rusne Island is widely known as the bird paradise. The fertile sediments facilitate the development of prosperous alluvial grasslands, which support a rich biodiversity. However, in the beginning of the 90s, almost all grasslands were abandoned. They became overgrown with scrub and reeds, unsuitable as feeding and breeding habitat for most of the birds but still used for roof making. The lack of funds, initiatives and capacity of the local farmers impeded a rapid and successful land reform. Low agriculture activity was followed by degradation of grasslands as habitats for the rare and endangered species. The main goals of the initiatives taken was to encourage environmentally sound and sustainable agriculture by management of the abandoned grasslands on Rusne Island. The dual purpose was to improve the local economy and make the grasslands more suitable for breeding and migratory birds. Other objectives aimed to promote environmental/ecological education within the local population and to develop ecotourism. Rusne Island is an example of how traditional farming practices stimulate the local economy and support conservation goals through good management of the land. This case shows that sustainable agricultural practices can embrace nature conservation and improve the local community economy. It shows that for an action to be successful at the local level, a strong local partner working closely with all stakeholders is an advantage (Pickaver 2010).

ICZM in the Bothnian Sea, Western Finland

The general objective of the Bothnian Sea ICZM was to apply an integrated spatial planning and management approach for the coastal zone of the northeast part of the Baltic Sea, in Finland (situated between the Aland and the Kvarken archipelagos). This planning approach, which was new for Finland, was integrated into the regional spatial planning system and was the responsibility of the governing and regional development bodies in the Satakunta and Ostrabothnia regions. The initial project launched in 2009, was later split into three parallel continuous projects, inspired by the National ICZM Strategy for Finland (2006), the Baltic Sea Action Plan (2008) and the EU Water Framework Directive. It was a consistent integrated spatial planning effort on the regional and municipal levels implemented in parallel in both regions, with specific attention to the Kvarken archipelago as a special pilot area. The main sectors addressed were fisheries and aquaculture, tourism and recreational facilities, coastal forestry, nature conservation and aquatic environment protection.

Coastal management measure for Southwest Finland

The project objectives were to develop and implement innovative and effective measures for ICZM in Southwest Finland. The project enabled the drawing up of a comprehensive ICZM strategy for Southwest Finland. It addressed the issues and constraints of preserving the regional lifestyle and economic activities while at the same time tackling the need for environmental protection of the HELCOM hot-spot area of the Archipelago Sea. This included participatory planning with all relevant stakeholders involved, and created a framework for decision-making to ensure that the special circumstances of the area were taken into consideration. It featured the extensive inclusion of regional stakeholders and the general public to ensure a shared understanding of integrated coastal zone management. The stakeholder forum “Pro Skärgårdshavet” comprised between 60 to 70 stakeholders, from NGOs to industrial companies. A specific pilot project, to demonstrate coastal zone management in an operational context was established around the Uusikaupunki city.

Results and discussion

Critical evaluation of the tool: suitability and limitations of the indicator set

Environmental quality indicators for application were used based on environmental themes to measure progress in achieving environmental objectives (Fig. 6). The indicators “4, 5, 6, 7, 11, 13” – indicators reflected by many measures; “8, 9” – indicators covered specific coastal theme and they were affected by some study sites. The remain indicators were more complicated to understand and posed some difficulties when applying them to the study sites. The 1st environmental quality indicator raised the question about what can be considered as pollution. In some cases, it was understandable as emission and in other cases implied as increased sand accumulation, due to conducted measures, as a part of this indicator. A short explanation for the 3rd indicator of is needed because it was not clear what is meant by ecological status of water, furthermore, the other part of this indicator is related to chemical status of water. The 10th indicator in this category was rarely answered because of misunderstanding the definition of energy efficiency and there were some difficulties interpreting the given data and to score the indicator. A short explanation of this indicator would probably be helpful especially, if the tool is applied by evaluators with different background or non-scientists. The 12th indicator proved to be hard to assess due to data availability and led to high subjectivity when giving the score.

Fig. 9
figure 9

Indicators application results of Environmental quality category. The scores indicate a degree of contribution to sustainability of environmental quality. Scale from −3/dark red = strong negative effects; −2/light red = considerable negative effects; −1/orange = weak negative effects; 0/yellow = no effect; 1/light green = weak positive effects; 2/lime green = considerable effects; 3/green = strong positive effects

Another problem is the broadness of some indicators. Those enumerating two or more characteristics combined with the linking word and are challenging to score. As an example, for the indicator ‘1. The best practice reduces waste, prevents air, water and soil pollution and stimulates material reuse and recycles’, several issues are addressed and one has to scale the importance of them respectively, in order to estimate the appropriate score. If the best practice, for example, reduces waste and furthermore builds a new recycling plant, which is in turn increasing air pollution, which score would be acceptable? Or if only 2 of the issues asked are addressed and there is either no data on the others or no change observable, is it still reasonable to give the highest score? For some indicators, a solution could be to address only one objective or split the indicator in two.

Overall, environmental quality indicators were helpful for evaluating environmental progress towards sustainable development and useful to indicate ICZM measures effect in respect to environment.

Economic category, turned out to be incomplete because not all 9 indicators were successful in tracking changes in respect to economics (Fig. 7). Application results showed that three out of nine economic indicators (“1. Effects financial policies and instruments to support economic stability and resilience; 2. Increases economic diversification; 7. Increases investment in innovation for green economy”) were not affected by many measures with respect to thematic coverage. The indicators which raised many questions were 4th and 9th in since it was unclear what are these indicators exactly asking for: further investments in the respective study or further investments in prospective other studies. In this case, both definitions are reasonable. For instance, when maintenance work is reduced, further investments are unnecessary. A positive score could still be entered, but this is totally up to the evaluator and highly increases subjectivity. The 7th indicator was hardly applied because evaluators had to assume, based on given data, whether or not an increase in green economy is observable. Evaluators commented that short a explanation is needed. The 5th indicator ‘The best practice promotes infrastructure development and increases environmental friendly transport’ has two objectives. Even though they influence one another, infrastructure development can be promoted without an increase in environmental friendly transport and vice versa, so splitting indicator in two would be reasonable. Furthermore, is the issue of environmental friendly transport also included in the 11th indicator of environmental quality category. The indicators that were well understandable and easy to apply but also had some shortcomings were: 3, 6 – indicators were reflected by many measures, 8 important showing changes in respect to sustainable agriculture and fisheries.

Fig. 10
figure 10

Indicators application results of Economic category. The scores indicate a degree of contribution to sustainability of economic category. Scale from −3/dark red = strong negative effects; −2/light red = considerable negative effects; −1/orange = weak negative effects; 0/yellow = no effect; 1/light green = weak positive effects; 2/lime green = considerable effects; 3/green = strong positive effects

Social well-being indicators helped to reflect the changes of this category by using 9 indicators in all analysed study sites. The 1st and 2nd indicators scoring strongly differed between evaluations because of different understanding of what is social justice, equal opportunities and quality of life. The different understanding of indicator objectives caused subjectivity in its weighting. The 3rd indicator covered to many objectives in one and increased subjectivity by giving the score. The indicators that were well understandable and easy to apply were: 4, 5, 7, 8. The 6th indicator was hardly applied to case studies because it has specific objectives. Language and interpretation also needs to be considered, if a clear communication and a common message wants to be achieved. For example, the 9th indicator of ‘The best practice reduces vulnerability of people to climate change and promotes comprehensive risk based assessment and prioritised action in area’. It is, first of all, very long and to some evaluators, it was not clear what exactly is meant by ‘comprehen-sive risk based assessment’, or what ‘prioritised action’ should include. For the social well-being indicators often the problem occurs, that information about the process is considered. For instance, if the best practice promotes communication and cooperation between citizens and local authorities. The communication during the planning and implementation should be considered in governance category. A more explicit formulation could prevent this doubling. Furthermore, indicators are quite wordy and can lead to different interpretations affecting the scores. Some of the indicators are slightly repetitive and the same data can be addressed for different indicators. Some evaluators are only paying attention to specific keywords, others may see the bigger picture of it and interpret data differently. If the wrong conclusions are drawn, the evaluation will be affected negatively leading to falsified results. Furthermore, in some cases the indicators are asking for a wide range of changes, this is the case of the 5th indicator which asks if the best practice protects, monitors, and safeguards local resident access to natural, historical, archaeological, religious, spiritual, and cultural sites. Here, a highly subjective evaluation can be expected. The question one should ask himself is if it is necessary that all these points have to be addressed.Strong positive effects can therefore still occur but are difficult to evaluate without a site inspection and only based on literature (Fig.  8).

Fig. 11
figure 11

Indicators application results of Social Well-being category. The scores indicate a degree of contribution to sustainability of social well-being. Scale from −3/dark red = strong negative effects; −2/light red = considerable negative effects; −1/orange = weak negative effects; 0/yellow = no effect; 1/light green = weak positive effects; 2/lime green = considerable effects; 3/green = strong positive effects

Governance indicators were complete and demonstrated the ICZM process from starting to build a management team, to evaluation of implemented measure success. The process of planning and implementing the measure is following the Systems Approach Framework. There is a small degree of subjectivity but usually it only slightly modifies the score. For instance, some of the process indicators ask for alternative scenarious but in some cases no alternatives were suggested or discussed, as the one implemented was the only suitable measure, therefore the best solution. Scoring the indicator with ‘0’, for ‘No, not at all’, would unjustifiably downscale the effectiveness of the study. The bigger differences can be attributed to the lack of data. The 12th indicator ‘The concept was implemented and accepted by the public’ could be splited in two because it covers two diffrenet objectives. Yet, the last indicator, which asks if the success of the measure was evaluated, is sometimes hard to assess since some reports only summarise that it was a big success but do not show specific results or ranked evaluations. To use the indicators optimally, the cross-linking of different indicators is needed and the relationship with the governance of the coast needs to be demonstrated (Maelfait et al. 2006) (Fig. 9).

Fig. 12
figure 12

Governance (process) indicators application results. The scores evaluate management processes following the SAF procedure steps. Scale ranges from: 0/yellow = no, not at all; 1/light green = yes, slightly; 2/lime green = yes, moderately; 3/green = yes; 4/dark green = yes, fully

In some study sites applications, sustainability indicators have high scores because they can be very important in specific ICZM context and cover different themes. In the prevailing literature, controversial discussions about sustainability indicators are observed. The main critics is the reductionism or over-simplification and the subjectivity when applying indicators (Singh et al. 2009). Others even think, that sustainability and the progress to achieve it cannot be effectively assessed using indicators in general, due to the complexity and equivocality of the concept itself (Mori and Christodoulou 2012, Reed et al. 2006). Even though the indicators have been carefully worded in order to make them understandable and applicable by anybody with interest in the topic, some are verbalised somewhat incomprehensibly. As stated by O’Mahony (2009), the semantic of indicator sets is decisive.

Consequences of score aggregation

The sustainability appears on three levels, so this methodology is based on triangle of sustainability and its categories are connected to one another. Therefore, the different categories are weighed equally, in order to adjust this indicator set towards an equal status. The categories of tool have different number of indicators that are included in the calculation and affects the category level.

Comparing evaluation in-depth and the fast screen analysis results strongly differ (Fig. 10). When analysing these results, it showed that when expert indicated ‘No change’, the evaluator could not find any data and scored indicator as ‘No data’. It is assumed, that when no change of a particular state occurred, there is no data available or is very challenging to find. It was easy to score respective indicators for an expert of particular study. Yet, as the fast screening method evaluator did not have a specific or detailed knowledge about the case study he indicated that no data was available. This led to strong discrepancies since the indicator scored with an ‘X’ (no data) will be excluded from the summation, whereas ‘0’ (no change) reduced the final score of the respective category.

Similar results were noticed and significant differences occured when four evaluators scored the same indicators (Fig. 11). For example, the application of Abbot’s Hall showed that 10th indicator in environmental quality, the 6th indicator in social well-being and the 9th indicator in the governance categories were scored as no change by three evaluators while the other evaluator could not find data on that indicator. Between these two answering options the high subjectivity came out. Some evaluators assumed that no change has occurred because nothing related was found during the data research. The indicators which were specified with ‘No data’ in fast screening results of German cases were used to ‘No change’ by following expert results. On the other hand, indicators which are not aggregated but simply listed are often considered to be too complex in practice (Sébastien & Bauler, 2013).

Additionally, to avoid significant differences in the results, a solution could be the splitting of indicators into core and optional, this idea is supported by Reed et al. (2006) in wich is stated that indicators are rarely of equal importance and they could be differentiated through the division into core and optional indicators or the selection of weights. That would equalise all categories but also complicate the comparability between different case studies as not all case studies would assess the same indicators. According to O’Mahony (2009) is very important that indicator sets should be seen as dynamic as the environment they are trying to represent. When working with the fast screening method, the division into core and optional indicators however, would complicate both the evaluation itself and the comparison of different evaluations. A recommendation depends on how the tool is supposed to be used. Further testing however, is necessary to verify this theory.

The importance of the evaluator, perception and time-frame

The qualitative assessment refers to a more critical evaluation. Indicators applications can be carry out by experts of ICZM measures, scientists or young scientists and non-scientists but experts can bring better knowledge and perception of the study site. Experts of ICZM measures showed relatively fast application process even when there was no fixed time for applying indicator set for applications in-depth.

Throughout our applications, we identified several aspects that influence the evaluator to give a particular indicator score. A difference in the understanding of indicators can be seen from the comparison of the in-depth assessment with the fast screening method assessment (Fig. 10), as well as from the comparison of the different evaluations on Abbott’s Hall (Fig. 11). Overall, sustainability indicators are quite wordy and can lead to different interpretations and variations in the scores. Personal interpretation is allowed, due to the formulation of the indicators and can have a small impact as well. This was particularly visible in the applications conducted on Abbott’s Hall. Almost the same data was used and according to the results the assessments vary between the different application results. For some indicators, similar comments were given but the scores varied. The 2nd indicator in Economics, for example shows scores from 0 to 3 and the 12th indicator shows strong variations since the environmental awareness is prone to personal perception. Whether or not the evaluator is an optimist or rather pessimistic, in his or her overall worldview, has an influence on some of the indicators.

For the Governance category, there is a small degree of subjectivity but this usually modifies the score by only about one point and do not have high effect on results. Bigger differences can be attributed to the lack of data. Yet, the last indicator, which asks if the success of the measure was evaluated, is hard to assess sometimes since some reports only summarise that it was a big success but do not show specific results or ranked evaluations.

Subjectivity can occur based on two different reasons. First one is connected, in general, to, the different understanding of the indicators whereas the second is connected to, how the data is then interpreted based on this understanding. As the indicators are scored on a Likert Scale, they are already prone to subjectivity. The question is if the rating is well-founded, based on personal perception or a composition of these two.

The academic or professional background affected the selection of scores. If he has, for example, a background in natural science, he will most likely be much more critical in the environmental category when compared to an evaluator having a background in business management more concerned with the economic category. In previous appraisals of coastal sustainability and ICZM implementation, these variations in the perspectives of evaluators have been noticed. For this reason, Pickaver et al. (2004) and Breton (2006) suggested to build groups of evaluators with different backgrounds to conduct the assessments, rather than a single person. This would furthermore increase the potential in data processing and lead to more significant results. According to Schernewski et al. (2014) this resulted in a stabilisation of the results and reduced subjectivity. Results also depend on the evaluators expertise in a specific study site. Even though participatory approaches are setting the context for assessment of sustainability at local scales, the role of an expert, who is leading the indicator evaluation and dissemination, is however very important, as stated by Reed et al. (2006).

Another reason for variations in the assessments is presumably resulting from differences in language skills. Whereas one evaluator has a very good command of English and is able to understand the data and the indicators, another evaluator might have some problems in understanding the given information. There might be a relation, between language skills and the time consumed for one evaluation. The evaluator with a very good command in English only took 5 h to complete the assessment, whereas the third evaluator, needed more than 13 h. This was already observed in previous assessments of various indicator sets, for example by Schernewski et al. (2014).

As the different applications are effected by the evaluators’ perception, personal perspective and back-ground, the results are not easily reproducible. Previous assessments for coastal sustainability and ICZM implementation have already been affected by this, which is, why this point is taken into consideration in the development of various indicator sets (Pickaver et al. 2004).

When comparing the screened evaluations of the three German case studies with the ones done by an expert applying the in-depth method differences occur (Fig. 10). The results of different applications using the two methods are slightly comparable, although their outcome do not have have the same quality. Further testing of this method is needed. An important factor for the tool is the availability of time to ensure the reproducibility and the application on a local level. The time frame of 12 to 16 h for a screened evaluation is not always enough to gather all the information needed to assess the best practices thoroughly. In some cases, the evaluators were able to answer all indicators but in most instances information was lacking. This is leading to problems in the aggregation and the possibility of comparison. However, in most cases, an extension of the time frame would not solve this problem. Some data is not obtainable for a screened evaluation, because an inspection of the site or the consultation of an expert, who participated in the process, would be necessary to successfully evaluate the respective indicator. From the different applications, conducted in a short screening, it becomes clear that a general overview can be obtained in a short time, yet to achieve reliable results either various applications or an expert-led evaluation should be performed. Schönwald (2014) points out, that time is decisive to attain detailed and informative applications. A comparison of different indicator evaluations is thus not constructive. A holistic approach should be braced to record an overall state of the sustainability. Working with indicators can support the self-assessment, help to recognise problems, to focus on objectives and to estimate bearings of decisions. The use of indicators increases the internal exchange between stakeholders and stimulates to an active partaking in formulating the surrounding conditions. This can lead towards a sustainable development (Schönwald, 2014).

Disparities also arose in the areas where the indicators were chosen to be applied, particularly between the expert and the screened evaluations. For instance, for the site Timmendorfer Strand, the expert only considered the beach area, whereas the fast screening evaluator also included the municipality, which lies behind the dunes. Moreover, what influenced the results notably are the differences in definition of what was the state before the implementation and what has been achieved.

The example of the site Timmendorfer Strand where the fast screening method evaluator estimated the improvement of the 4th indicator of environmental quality category to be considerably positive since the population of old trees has been preserved, when discussed with the expert later on this is not considered as an improvement because the old trees have been there before. The expert and the screener had variable starting points in all three case studies. The expert who conducted the in-depth analysis of the three German case studies was completely familiar with the areas Timmendorfer Strand, Markgrafenheide and Geltinger Birk already from the time before any implementation was carried out and thus, had a profound knowledge of the sites. In contrast to this, the screener had only some information available which was based on different states and times. As this information is not congruent to those the expert had access to, differences in the evaluations and the final results are visible. Especially for long-term studies, it is very important to have the previous state and the objectives in mind while evaluating. Besides this, benchmarks are very important to ensure the reproducibility. This was already criticised by Schönwald (2014) that each evaluator should have thresholds from which a development is to be evaluated and what are the objectives, simulating the maximum permissible value.

Strenght and weaknesses of ICZM study sites

The evaluation covered the full cycle of ICZM and indicators measured the extent to which an action was sustainable, as well as, revealed to which extent targets and objectives towards sustainable development have been met (Fig. 12). Results are represented graphically using three axis – an axis for each component of improved sustainability indicating positive changes (weak, considerable, strong). Moreover, the governance (process) bar showes to what extent ICZM was implemented. An indicator-based tool was helpful to evaluate if a study can be defined as best practice. According to the results (analysis in-depth and fast screening), a case study was characterised as best practice if it had a broad overall perspective and merged economic, social and environmental issues, if its measures collaborated with natural systems while respecting their capability and when it was seminal and promising. Furthermore, an improvement of the state of sustainability in all categories have been achieved and the protection of the coastal and marine environment was enacted in all study sites. The provision of a systematic solution, which was responding to the practical needs of the area was important as well as the involvement of all parties concerned. It mobilised the best available knowledge, which was supporting deliberative decision making processes, supported and involved relevant administrative bodies. The study sites application results have different characteristics and some study sites have more impact on a social level, some have a greater impact of environment and, others indicating greater investment in economy. The relations between these categories have to be contemplated as they influence one another. For example, in Finland a cultural tradition is to build holiday houses along the coast but it affects coastal management plans and conservation of habitats and species.

The case study of Geltinger Birk showed considerable positive effects on sustainability. The implementation process itself had some shortcomings. It improved the local state of sustainability and especially had positive impacts on environmental quality. Due to the realignment of the coast, which included the building new dyke, the nearby village is protected from flooding. The re-wetting of the area enhanced the development of biodiversity and the establishment of flora and fauna, which were formerly domiciled here. The measure promoted local economy and increased the low-impact tourism, as well as, had considerable positive effects on social well-being.

Markgrafenheide showed considerable positive effects on development towards sustainability. The aspects that stand out the most are: the prevention, protection and mitigation of floods, the support of natural habitats, biodiversity and their quality. The realignment of the coast as a new approach towards a more sustainable development in this area. According to the evaluation, the process of implementing the measure had several shortcomings, however, critics on the process were made, as the inclusion of the local community and stakeholder groups failed with the start of the project and not all concerns had been considered. Yet, the measure can be evaluated as a success restrospectively because it is an outstanding example of coastal protection and nature conservation.

The implementation of the ICZM initiative on Timmendorfer Strand is estimated as very high with 92%. With the exception of the management team, which was not fully build to lead the planning, the measure was successful implemented according to the guidelines of the Systems Approach Framework. Thus, a successful flood protection system was established after argumentative and strong public participation into the planning and implementation process. The issue was chosen driven by ecological, social and economic needs and targets were set. The possible consequences on these, were identified and a strategy was developed regarding how to assess these effects. All stakeholders were determined and, included into the process and impacts on them were assessed. This lead to an accepted and successful implementation, following SAF leading to an overall success. The results of process showed that good implementation process could have weak positive effects on sustainability. Due to the flood protection walls, the inhabitants living in the área, as well as, their tangible values, are protected. Economic stability and resilience is enhanced and changes in the foreshore area are improving the management of coastal erosion, which is leading to reduced vulnerability to climate change impacts. Climate change is the most significant challenge to achieving sustainable development.

During the analysis of the various case studies (analysis in-depth and fast screening) two keywords arose frequently: ‘public participation’ and ‘development strategy’. The success of an integrated coastal zone management process can be narrowed down to the effective operation between the public and authorities. Looking at the case studies that were evaluated, especially those where public participation was emphasised and where all kind of stakeholder groups were involved from the very beginning of the planning process, were successful. This is one of the main components of the SAF, where the processes planning show good approaches and can yield in successful implementation. For example, in the beginning of the Geltinger Birk process stakeholders and the public were not involved leading to problems and hampering the project. The acceptence of the scheme was very low as not all scenarious were put into account as the affected groups and persons felt excluded. The proper application of the SAF would have avoided this problem. The presented tool could be used to support planning process as well as to help analysing stakeholder participation and their preferences.

The time-frame for evaluation and comparison of results showed high importance. If one evaluation shows a development observed over 5 years and the other considered 20 years of development, the reproducibility of data is insufficient. Initially the study of Timmendorf promoted the communication and cooperation between citizens and local authorities but as the whole process took over 20 years before the implementation, these assets were lost until then. Each case study has to be evaluated separately, with regards to its previous state and the objectives that were predetermined.

All analysed studies have been conducted on a rather small scale, focussing on municipalities and shorter coastal strips. Therefore, based on this research, a distinct statement about the applicability of this tool on different spatial scales is possible. However, as already been mentioned by Schernewski et al. (2014), based on previous applications of the SUSTAIN indicator set, there is the risk that the spatial coverage of an indicator might go beyond the level of municipalities and results then, rather reflect the regional conditions.

Another important factor for the subsequent evaluation of the assessments are the given comments especially for comparison reasons. Without comments, the given scores are neither comprehensible nor comparable, which decreases the relevance of the evaluation and the tool. Moreover, what has already been stated by O’Mahony (2009) is that data is not always collected with sustainability in mind and a reliance on surrogate data may tamper the results.

As the principles of ICZM have been stipulated only recently, a lot of initiatives are not finished yet. The effects on environment, social well-being, economy as well as the lessons learned are not measurable as development has to take place first, especially if the changes in the environment are only visible and reliably analysable in the long term. An estimation is therefore, not reasonable. On the other hand, there is not enough data available as often, final reports, are concluding all affected parameters whereas progress reports discuss only parts of the effects. A tendency can be estimated using the tool, but a conformable evaluation should be conducted after completion of the measures.

Due to the novelty of ICZM, long-term studies are rare. For this work, only the initiative in Køge Bay (see Fig. 5, number 6), which was planned in the 1930s and implemented in the 1970s, was evaluated as a long-term study where a long-term development is observable. However, the data availability for this case is limited. Online resources are rare and data is available in Danish only. This applies as well for the initiative in Perkpolder, which is documented in Dutch. The documentation is not enough for a distinct application of the indicator tool. Moreover, the data search in all cases was the most time consuming part. A common database with sufficient and high quality information is a goal in ICZM, as well as for the SAF (Hopkins et al. 2011). A first attempt is the OURCOAST website which holds an ICZM database, containing summaries describing 350 cases where ICZM principles and tools that have been applied. For a decent evaluation the availability of data is therefore decisive.

Conclusions

The indicator set was helpful to evaluate if a case study can be defined as best practice. It was characterised as a best practice if it had a broad overall perspective and merged economic, social and environmental issues. Summarizing results of analysis we conclude that indicator-based tool applications can tell us things such as: to what exent ICZM initiative objectives have been met; what progress the measure has made towards sustainability; to which extent targets have been met.

Results of the fast-screening assessments show, that this method serves to attain a good overview and a first impression of a study site’s state towards sustainable development. Fast-screening results compared with the in-depth method differences occurred. However, both methods results showed improved sustainability and achieved positive changes in respect to sustainable development of specific case studies.

The evaluator as an expert brought a better knowledge on study area. However, subjectivity in the understanding of data and indicators, and the personal perception are influencing results. The availability of time and data is decisive for a successful and reliable evaluation. For the fast-screening method, both are limited and this resulted in differences in both comparisons conducted in this research. Besides this, the evaluator background and his command in the demanded language are influencing the assessment. The idea of building groups of evaluators, thus stabilising the results and minimise subjectivity, should be deepened and the method then further tested.

Indicators should simplify and communicate complex information, but some indicators are incomprehensible and they complicating the application process. A revision of the indicators is needed. They can be tailored to the needs of the strategic goals of ICZM initiatives. To counteract the broadness and to achieve dependable results, some indicators should be splited. A division in core and optional indicators could contribute to appropriate weighting and make their use more flexible. However, the tool only indicates that something has happened but there is not proof and results can not tell us: why and how changes occur and what solutions should be undertaken but it could help for coastal management decision making.