1 Introduction

To be effective and timely, adaptation measures to a heating climate need to be based on state-of-the-art-knowledge about the current and future impacts of climate change. With high-resolution climate data sets becoming widely available, climate change adaptation policy can be supported with detailed local climate information. Climate indicators can be used to translate this information for specific sectors like health, agriculture, and biodiversity. This is usually achieved by linking climate data with the sectors’ needs via critical thresholds or statistical relations (e.g. Van Den Besselaar et al. 2014, Juckes et al. 2016).

Maps are a powerful way to visualise and communicate scientific information (Card 1999, Cairo 2013, Ware 2013), and to bridge the gap between climate models and adaptation policy (McNie 2012), but also to mislead and misinform (McKendry and Machlis 2009). Therefore, making local climate information usable for adaptation planners requires close and iterative interaction between scientists and involved stakeholders (Shaw et al. 2009, Mastrandrea et al. 2010, Dilling and Lemos 2011, Lorenz et al. 2015, Meadow et al. 2015, Prutsch et al. 2018).

Recently, an increasing body of literature has been dedicated to this issue. Some authors give recommendations about the appropriate use of colours or on displaying uncertainties in maps on a theoretical basis (e.g. Kaye et al. 2012, Grainger et al. 2016); others conducted surveys to test (objective or perceived) understanding of graphics with potential users (e.g. Becken et al. 2015, McMahon et al. 2015, Retchless and Brewer 2016). All the listed papers call for additional empirical studies that explore user involvement into the visualisation design process. One considerable research gap persists in empirically linking objective understanding of climate change maps with subjective aspects like aesthetics and their usefulness as communication tool. Also, while map comprehension has often been tested as the ability to read map data points, there are deeper cognitive levels that come into play when reading and interpreting maps (Claaßen 1997).

Starting in 2009, a participatory process between governmental and non-governmental actors (mainly scientists) took place to develop a national adaptation strategy for Austria (Prutsch et al. 2018). This led to the adoption of the first (BMLFUW 2012a and 2012b) and second (BMNT 2017a and 2017b) version of the Austrian National Adaptation Strategy (NAS). Subsequently, Austrian federal states adopted their own federal adaptation strategies, or integrated them into existing strategies for climate change mitigation (Environment Agency Austria 2018). In order to base these strategies on state-of-the-art knowledge about climate impacts, local and sector-specific climate information was required. The project CLIMAMAP (Climate Change Impact Maps for Austrian Regions), funded by the Austrian Climate Research Programme (ACRP), answered this call with a transdisciplinary research team. The project was set up as a co-design effortFootnote 1 to provide regional adaptation planners with a decision support tool based on state-of-the-art climate data that is tailored to their specific needs and priorities. A stakeholder network consisting of representatives from federal administration was set up to define the theoretical framework of the research and prioritise climate change impacts most relevant for their region. This network was also in charge of nominating potential users of the maps that were invited to a workshop series. The aim of the workshops was to test the users’ understanding, aesthetic preference and perceived usefulness of different map designs with a questionnaire. Refined map designs based on the results of the workshops were further tested in a survey with experts in climate change communication. The resulting maps and data were published via the Climate Change Centre Austria’s (CCCA) Data Centre (Becsi and Laimighofer 2018).

The present paper focusses on the methods and results of the stakeholder involvement process (workshops and expert survey) to provide new insights for user-centred design approaches. It addresses the research gaps mentioned above by examining two main questions:

  1. 1.

    What are the links between objective map comprehension, aesthetic preference and perceived usefulness of climate maps as a communication tool?

  2. 2.

    What role does cognitive complexity play for overall map comprehension?

The following sections are structured in a way to easily guide readers through the two-step stakeholder involvement process. The methods and results of each step are presented consecutively: First, the methods and results of the workshops, then the methods and results of the expert survey. All results are then discussed and concluded in the final part of the paper.

2 Methods: workshops with practitioners

2.1 Participants

Participants of the workshop series were nominated by the projects’ main network of stakeholders. This network consisted primarily of the Austrian climate mitigation representatives (so-called Klimaschutzbeauftragte) of the Austrian federal state administrations. Their agenda is network-building, consulting policymakers on climate change and raising public awareness on the issue. Since Austria is geographically diverse, the federal states face different local impacts of climate change. Therefore, it was important to invite regional stakeholders with their specific needs and expertise. The group consisted of experts from the federal state administrations, consultants to local-level policymakers and mayors. The demographics of the participants are found in Online Resource 2.

2.2 Map designs

Twelve map designs were developed based on different combinations of three map design features: (1) portrayal of indicator values on the map as absolute values of the climatological mean or anomalies, (2) complexity of the map design (i.e. number of climate scenario maps included) and (3) handling of uncertainties. The role of different colour schemes was not tested in this study; instead, fixed colour schemes from the tool ‘colorbrewer’ (Brewer and Harrower, 2016) were used for each indicator type according to recommendations by Kaye et al. (2012, p. 249). Table 1 shows all twelve map designs used in the workshops and their respective combination of design features. The maps are found in Online Resource 4.

Table 1 Map designs used in the practitioners’ workshops, combined from three different design features. Designs with orange background (D1–D6) were used for maps showing temperature-related indicators, designs with blue background (D7–D12) for maps with precipitation-related indicators
Full size table

One map always showed the current climate normal of 1981–2010 (World Meteorological Organization 2017); the other maps depicted a varying number of climate scenarios. Indicators were always portrayed as absolute values of the climatological mean for the current climate, but were varied between absolute values and anomalies for the climate scenarios. In the ‘complexity’ design category, the designs varied between one climate scenario map (timeframe 2071–2100, RCP8.5) and six climate scenario maps (timeframes 2016–2045, 2036–2065 and 2071–2100, each under RCP4.5 and RCP8.5). The design feature ‘uncertainties’ describes whether uncertainties were displayed (1) on a separate map sheet, (2) on a separate map on the same map sheet or (3) as a bivariate map showing both variables on the same map. Two map designs (D6 and D12) represented uncertainties not as geographically explicit information, but as time series and frequency distribution graph. The map designs were divided into two groups (orange and blue shadings in Table 1): designs D1–D6 showed temperature-related indicators (e.g. heat days, vegetation period) and D7–D12 showed precipitation-related indicators (e.g. days with heavy precipitation, precipitation intensity). The two groups were created to avoid cheating: Neighbours’ maps looked completely different from each other since different colour schemes were used for the two indicator types. The different map features were distributed nearly equally over the two groups, and each participant completed questionnaires for all twelve map designs in randomised sequence. Therefore, a comparison of learning effects between the two groups could be carried out (see section 2.5).

2.3 Questionnaire design

Workshop participants were asked to fill out questionnaires and give feedback on the twelve initial map designs. The questionnaire was structured into three parts that relate to the first research question of this paper:

  1. 1.

    Which map designs are comprehended best?

  2. 2.

    Which map designs are most aesthetically appealing?

  3. 3.

    Which map designs are most useful for communicating climate change impacts?

In the first part of the questionnaire, participants anonymously filled in socio-demographic information (age, gender, highest education and pre-existing experience with maps) and rated the twelve map designs according to their aesthetic preference. While Retchless and Brewer (2016) tested aesthetic preference after comprehension of the map contents, the present study started with aesthetic preference to capture first impressions of the maps, without focussing on the map contents first.

The second and main part of the questionnaire addressed the comprehension of the maps. It consisted of closed-ended single- and multiple-choice questions that required participants to work with the map contents. Claaßen (1997) describes map competences as a cognitive progression from reading a map to understanding its relation to the real world, and further to interpret the map and predict future development. Inspired by his theoretical frame, three progressively more demanding classes of understanding the map, called cognitive levels of comprehension (CLCs), were distinguished:

  1. (a)

    Reading: Respondents were asked to extract information from a given location in a single map or from the explanatory texts.

    Example: How many days with heavy precipitation occurred in the town of Klosterneuburg on average per year in the timeframe 1981–2010?

  2. (b)

    Understanding: Respondents had to combine different pieces of information from one or more maps and/or the explanatory texts.

    Example: (Map showing absolute values of the climatological mean for future climate projection) Do you see a change in the average annual number of heat days per 30-year period between 1981–2010 and 2036–2065? If so, by how much has it changed?

  3. (c)

    Interpreting: Respondents had to validate or refute statements or facts based on the map contents. In this class, not all relevant information was provided on the maps, but the map contents needed to be combined with pre-existing knowledge.Footnote 2

    Example: Which of the following statements about the certainty of the information shown in the future climate maps is correct?

A standard set of questions was developed and adapted to the different designs and contents of the twelve maps. The same question can be linked to different CLC classes for different map designs. For instance, a question of CLC class (a) for a map showing absolute indicator values can switch to CLC class (b) for a map showing anomalies to a reference situation. In the first case, the information can be read directly from the map. In the second case, it has to be combined from two separate maps. Since a similar set of questions was used for each map design, the number of CLC classes differed. In total, this part contained 51 questions (4–5 per map design). A total of 30 questions were related to CLC class a, 15 questions to CLC class b and 6 questions to CLC class c. This skewed distribution is owed to the fact that CLC classes were only introduced ex post after designing the questionnaire for the workshops. In the expert survey, the concept was considered from the start, and the CLC levels were more equally distributed across the questions (see section 3.3).

2.4 Types of measurement

Two different types of measurement were used to evaluate the questionnaire. Map comprehension was measured as the percentage of a maximum of possible score points per answer. Each single-choice question was worth one score point; for each multiple-choice question, a total of one point was credited if all answers were chosen correctly. Otherwise, the score of multiple-choice questions was calculated as follows:

$$ \mathrm{Score}=\frac{\mathrm{number}\ \mathrm{of}\ \mathrm{correct}\ \mathrm{choices}}{\mathrm{total}\ \mathrm{number}\ \mathrm{of}\ \mathrm{choices}} $$

This calculation method tends to overestimate map comprehension, because it weights randomly selected answers the same as intentionally selected answers. To minimise this effect, each question offered the choice ‘this information/statement cannot be obtained from /validated with the map’.

Aesthetic preference and perceived usefulness are subjective ratings and were measured by asking participants to distribute a total of three points to any of the twelve map designs. In the first part of the questionnaire, the points should be assigned according to aesthetic preference. In the third part, the preferences were allocated according to the perceived usefulness of the maps as a communication tool. Map designs were ranked in each category by the total number of preference points they received.

2.5 Workshop procedure

Workshops were held at three different locations in Austria (St. Pölten, Klagenfurt and Innsbruck) to minimise travelling time for local participants. At the workshops, participants received the questionnaire in two sessions, with a short break in between. To avoid sequence effects, half of the group started with map designs D1–D6 (temperature related indicators), the other half with D7–D12 (precipitation related indicators). After the break, they received the opposing set of map designs and questionnaires. The maps in each set were randomly shuffled to further minimise sequence effects and to avoid cheating.

In addition to filling out the questionnaire, the participants were asked to give verbal feedback on the maps. This moderated discussion addressed two guiding questions:

  1. 1.

    How would you assess the comprehensibility of the map designs?

  2. 2.

    From your perspective, are the map designs useful for communicating climate change impacts to decision-makers? If yes, why?

3 Results: workshops with practitioners

3.1 Which map designs are comprehended best?

A total of 26 participants attended the three workshops. An overview of the socio-demographic statistics can be found in Online Resource 2. Figure 1 compares the results for the twelve map designs presented in section 2.2. In the category comprehension of the map contents, most map designs scored between 70 and 80%. D12, D7 and D2 scored highest, with differences of 3.7 and 0.8 percentage points between them, respectively. Although characteristics like map complexity or indicator portrayal (absolute values vs. anomalies) differ between the three best performing map designs (highlighted in Fig. 1a), they share one common feature: Uncertainties are displayed on a separate map, not overlaid on the climate scenario map. Map design D4 performed lowest with a score of 52%, more than eleven percent lower than the next lowest map design. D4 shows indicator values in the climate scenario map as anomalies from the current climate. One of the test questions that were classified as CLC class understanding was answered correctly by only two respondents out of 26.

Fig. 1
figure 1

Evaluation of practitioners’ responses (n = 26) to twelve varying map designs. Comprehension of map contents (a) is indicated by the mean score of all respondents for each design variation. Error bars show the standard deviation of scores for each map design. Aesthetic preference (b) and usefulness of the maps as communication tool (c) are quantified as the total number of preference points awarded to each design variation. The three highest scoring map designs are highlighted in each bar plot with dark blue colour. In the category usefulness (c), D4, D6 and D9 scored the same number of points. Therefore, four bars are highlighted

Full size image

3.2 Which map designs are most aesthetically appealing?

Assigning points according to aesthetic preference was the first task of the questionnaire. Design D12 was perceived as most aesthetically appealing, followed by D6 and D11. Map designs D6 and D12 used different approaches to represent uncertainties, showing graphical representations like time series (D6) and distribution curves (D12), whereas the other design variations portrayed geographically distributed uncertainties in the form of maps. It is likely that most preference points were allocated to those two map designs because they contrasted the others visually.

3.3 Which map designs are most useful for communicating climate change impacts?

The last part of the questionnaire asked participants to rate the usefulness of the different design variations as communication tools. Here, design D12 disappears from the list of highest ranked designs, scoring the third lowest number of preference points. Its representation of uncertainties as frequency distribution curves of anomalies from different climate models for three future time slices was considered too complex to communicate to decision-makers. Design D6 scored second highest (together with D4 and D9), confirming that uncertainties are also found useful for communication purposes when depicted as (not geographically explicit) time series. Design D11 was ranked highest in the category usefulness as communication tool. It comprises of two maps: the situation under current climate conditions and one future climate scenario (RCP8.5 for the timeframe 2071–2100), portraying the climate impact indicator in absolute values of the climatological mean (as opposed to anomalies from the current climate). In the category map comprehension, D11 scored only in the lowest quartile, showing uncertainties as overlaid symbol layer on the climate scenario map.

3.4 Summary of main workshop results

Table 2 provides an overview of the quantitative workshop results for all three questionnaire parts. It summarises the similarities and differences between the best ranked maps by showing how often each design feature occurs within the three highest scoring maps in each category (highlighted in Fig. 1). The total sum in the rightmost column shows the design choice’s number of occurrences over all three categories, providing a measure for its overall performance.

Table 2 Occurrence of different map design features within the three highest-scoring map designs of each category (coloured differently), as identified from practitioners’ questionnaire responses
Full size table

Reduced complexity is a common feature of the map designs that were best understood, preferred aesthetically and found most useful for communicating climate change impacts. The picture is less clear for the other map features. Particularly, the portrayal of uncertainties yielded mixed results that required additional examination in the subsequent expert survey.

3.5 Further analysis of map comprehension

An evaluation of the score distribution between CLCs is presented in Fig. 2. The median of scores of the CLC class understanding is 23 points lower than the median of the class reading and 8.3 points lower than that of the class interpreting. This means that the participants performed better in interpreting the maps to support or refute statements than in combining separate pieces of information to answer more complex questions than reading.

Fig. 2
figure 2

Distribution of participants’ scores according to three CLCs (n = 26)

Full size image

In addition to the different levels of map comprehension, learning effects over the course of the testing process were analysed. Each participant’s average score within the two groups of indicators (T, D1–D6; P, D7–D12) was calculated. Figure 3 visualises how scores changed between the two map sets, depending on the type of indicator the participants started with. In the first part of the workshop, temperature indicators scored slightly better than precipitation indicators. Participants starting with T improved their average score by 12.3% in the second part. This difference in performance was found statistically significant by a Wilcoxon signed rank test (p = 0.01). Participants starting with P showed a weaker learning effect, improving their mean score by 1.1 points (no statistical significance, p = 0.38). On average, scores of the map set received in the second part improved by 7.4 percent compared with the map set in the first part, no matter which indicator type participants started with. This difference was found statistically significant (p = 0.01) and is mainly attributed to the learning effect of the group that started with temperature indicators.

Fig. 3
figure 3

Score changes between the first and second map set that participants worked on during the workshops. Average scores per group for each participant are shown as coloured points: red are participants that started with a T indicator (n = 15), blue started with P (n = 11). Dashed trend lines illustrate the mean learning effect of each group (red = T to P, blue = P to T)

Full size image

3.6 Qualitative results

Statements were recorded during the workshops’ moderated feedback discussions and grouped into five categories: (1) map contents (e.g. complexity, informative value), (2) colour use, (3) size and descriptiveness of text and labels, (4) uncertainties and (5) the usefulness for communication. Table 3 summarises the qualitative feedback from the workshops. Statements that were brought up multiple times were merged. Marks in the table are recommendations considered in the subsequently revised map designs, statements in agreement with results from the quantitative evaluation, as well as statements contradicting other statements.

Table 3 Qualitative feedback from workshop participants on different aspects of the map designs
Full size table

Plenty of the statements given by the workshop participants are in agreement with the results from the quantitative analysis (marked with * in Table 3). Some feedback was in direct contradiction to statements made by other participants (marked with ** in Table 3). For example, using a symbol layer as overlay on the climate scenario map was seen as the best way to portray uncertainties by some. Others stated that a combination of graphical uncertainty representation and a climate scenario map is best, while a third group said that this form of uncertainty illustration is too complex. This discussion is also reflected by the equivocal results for the illustration of uncertainties in Table 2. Another case of dissonance is found regarding the complexity of the maps. While a number of participants stated that less complex maps are easier to understand and therefore better for communication, there were also statements that showing different emission scenarios on the same map is crucial for communicating society’s potential for emission reduction. This contradiction was addressed by the final map design presented in section 6.

4 Methods: expert survey

4.1 Participants

The expert survey was carried out as a second step in the stakeholder involvement process during a congress on climate change communication (K3, https://k3-klimakongress.org/). The aim was to optimise the maps with input from experts on climate change communication and to further examine the inconclusive results of the first step. The demographic statistics of the experts’ user group is appended in Online Resource 2.

4.2 Revised map designs

Results of the workshop series clearly suggested reducing complexity of the maps. Results were less clear on the portrayal of indicators as absolute values or anomalies, and on the handling of uncertainties (see Table 2). This was also reflected in some contradictory statements from the qualitative analysis.

Based on the findings of the workshops, revised map designs were developed. Each revised design featured one map showing the current climate and one map showing a climate projection of the distant future (2071–2100) under emission pathway RCP8.5. Other feedback suggestions like less nuanced colour schemes and increased font size were also taken into account for the revised designs. To focus only on the inconclusive results from the practitioners’ workshops, the number of design variations was reduced from twelve to four. The revised designs only differed in two aspects: portrayal of indicators (absolute values or anomalies) and uncertainties (separate map sheet or overlaid symbol layer on the climate scenario map). Another reason for using fewer map designs was the fact that the expert survey was carried out on a stand at the conference venue, and the time for completing the survey was limited.

The expert survey at the K3 conference in September 2017 involved the following four revised map designs:

  • RD1) absolute values, uncertainties as additional symbol layer on the climate scenario map

  • RD2) absolute values, uncertainties on a separate map

  • RD3) anomalies, uncertainties as additional symbol layer on the climate scenario map

  • RD4) anomalies, uncertainties on a separate map

The maps are available in Online Resource 5.

4.3 Design of questionnaire

The questionnaire for the expert survey was structured the same way as in the workshops. First, participants were asked to rate the map designs according to their aesthetic preference, then to complete test questions in the map comprehension section and finally to assign points for the usefulness of the maps as communication tool. Due to time restrictions, participants of the survey only worked on two different map designs. In total, 20 questions pertained to CLC class reading, 15 to understanding and 25 to interpreting. A translated expert survey questionnaire is available in Online Resource 1.

4.4 Survey procedure

For each of the four revised map designs, one map showing a temperature-related indicator (T) and one map showing a precipitation-related indicator (P) were produced, resulting in a total of eight maps. The printed maps were labelled and mounted on pinboards that surrounded the survey stand where participants received the questionnaire. The map comprehension section of the questionnaire asked participants to test two different map designs, one showing a T indicator and one showing a P indicator. The questionnaires were not randomly shuffled but contained fixed pairs of map designs (RD1 and RD4, RD2 and RD3) in order to engage every participant with all different map design features. Participants were asked to adhere to the sequence of map designs indicated by the questionnaire. To avoid crowding at the pinboards and potentially cheating, questionnaires were handed out in a certain sequence. After finishing the questionnaire, participants were asked to return it to the survey stand.

5 Results: expert survey

Online Resource 2 shows the demographics of the expert surveys’ participants. Figure 4a compares the scores of the four revised designs for the map comprehension section of the questionnaire. Both map designs with uncertainties on a separate map (RD2 and RD4) were understood better than those portraying uncertainties with a bivariate mapping technique. The design RD2 scored the highest in the category map comprehension, featuring absolute values of the indicators’ climatological mean and portraying uncertainties separately. In Fig. 4b, the total sum of aesthetic preference points attributed to the map designs is shown. Maps with uncertainties portrayed on a separate map were also found more aesthetically appealing, although in this category, anomalies were ranked higher than absolute values (RD4 > RD2, RD3 > RD1). In the categories map comprehension and usefulness for communication, absolute indicator values scored better than anomalies (RD2 > RD4, RD1 > RD3).

Fig. 4
figure 4

Evaluation of expert survey carried out at K3 conference in 2017 (n = 68).

Full size image
  1. (a)

    Mean score (in percent) of all respondents according to map comprehension. Error bars show the standard deviation of all scores for each revised map design

  2. (b)

    Points attributed according to aesthetic preference

  3. (c)

    Points attributed according to the perceived usefulness for communication

The points attributed in the category usefulness for communication (Fig. 4c) result in the same ranking order as the scores in the map comprehension section (Fig. 4a). The perceived usefulness of the maps as communication tool matched the understanding of map contents very well in this survey.

Figure 5 shows the results of the analysis of CLCs for the expert survey. The pattern resembles Fig. 2, with similar median scores (0.8, 0.5 and 0.72, respectively) for the three CLC levels.

Fig. 5
figure 5

Distribution of experts’ scores according to three CLCs (n = 68)

Full size image

Learning effects were analysed by comparing the scores of the two map designs that were tested by the experts. While workshop participants completed six maps per group, experts only tested one map per indicator type. The learning effects are therefore less distinct than those of the practitioners’ workshop series. On average, scores of the expert survey group that started with T decreased by 1.6 percent (not significant with a p value of 0.71, analysed with a paired t test). In comparison, workshop participants that started with T maps showed a significant increase in scores (shown in red in Fig. 3). The group that started with P in the expert survey showed a slight increase of approx. 2.5% (not significant, p value of 0.47). This score change is comparable with the learning effect of the practitioners’ subgroup that started with the P maps.

6 Discussion

Co-design of communication tools is a key piece in the puzzle towards more informed adaptation to existing and future climate impacts. The information to support the political decision-making process needs to be practical, easy to comprehend and to communicate to others, credible, and based on the needs and experiences of those who use it (Lorenz et al. 2015, Grainger et al. 2016, Grothmann et al. 2017, Prutsch et al. 2018). Climate change impact maps were co-designed and tested with practitioners of local adaptation planning and with experts in the field of climate change communication.

The principal research question of the study was the link between objective map comprehension, perceived aesthetics and usefulness for communication. Some authors have examined this link (Lorenz et al. 2015, McMahon et al. 2015, Taylor et al. 2015, Grainger et al. 2016, Retchless and Brewer 2016) and have concluded that visualisations that are aesthetically appealing differ from those that are comprehended best or that are suitable for communication. In the first part of this study’s stakeholder process, only three map designs out of twelve were among the top three rated designs in two categories, and none were present in all three categories. This certainly supports the results of the existing literature. However, there was a high consistency between these three categories in the expert survey. Maps that showed uncertainties separately and not as overlay on the thematic layer performed better in all three categories.Footnote 3 The ranking order of tested map designs is even the same for comprehension and usefulness as a communication tool. This may be an effect of a more focussed approach in the expert survey, where only two map design features were combined into four map designs instead of twelve combinations of four different design features in the first part. Because of this reduction of variables, fewer variations of rank orders were possible. Another reason could be that experts on climate change communication were more used to the tasks of the survey and better at evaluating the maps’ potential to communicate climate information.

The most distinct common feature of map designs that were best understood, useful and aesthetically pleasing in the practitioners’ workshops was reduced complexity (cf. Dransch et al. 2010). In this study, reduced complexity meant a small number of maps (at least two) on the same map sheet compared with up to 13 single maps on the same map sheet. The task to orient oneself on the map sheet and to find the relevant information in a cluster of complex maps was decisive for answering the questions correctly. With less complex maps, orientation was achieved more easily. In addition, many participants worked as consultants for regional or local policymakers where time to review complex scientific visualisations is often limited and decisions are based on the message that is most effectively communicated in that short time (Grainger et al. 2016). Those circumstances explain why less complex map designs were also preferred aesthetically and as a communication tool. On the other hand, some statements from the discussions at the workshops suggested that a certain complexity is desired and necessary. For example, showing different emission pathways may help to communicate the interaction between mitigation and adaptation to climate impacts. Even the design of different versions of the maps for experts and decision-makers was called for. Still, it is important to maintain a fine balance between ease of comprehension and scientific scrutiny that demands communicating ‘[...] our ignorance alongside our knowledge’ (McInerny et al. 2014).

However, uncertainty still has a very ambivalent role in the field of visual climate communication (cf. Dransch et al. 2010, McMahon et al. 2015, Johannsen et al. 2018). While some map users find it more comprehensive and honest to inform about uncertainties, others interpret this as incompetence and dishonesty (Johnson and Slovic 1995). Often, uncertainties are ignored in political decisions (Padilla et al. 2015). This ambiguity is reflected clearly in the results of the quantitative and qualitative parts of the workshops (Tables 2 and 3). The expert survey provided a more consistent picture: Map designs with separate uncertainties were understood better, preferred aesthetically and found more communicable than those with uncertainties overlaid on the map variable (Fig. 4). This supports the results of Kaye et al. (2012) and Retchless and Brewer (2016). Both concluded that separate maps are more comprehensible for tasks that require reading a single piece of information from a map but hamper the understanding of the correlation between a climate variable and its associated uncertainty. The projects’ final map design described in section 6 features a set of three separate maps for each climate impact indicator. Map users can select the desired scope of information themselves, ranging from a single clear message to a more complex one involving several uncertainties and a climatic development over time. Ultimately, this choice depends on the users’ communication skills and the experiences and needs of their target audience (MacEachren et al. 2004).

Although ample literature about visualisation techniques in climate maps exists (e.g. Kaye et al. 2012, Retchless and Brewer 2016, Schneider and Nocke 2018), up until now, it focussed on visual variables such as colour or shape. For the first time, this study also empirically compared using absolute values of indicators’ climatological mean with anomalies from a reference climate. While anomalies generally scored better over all three categories in the workshop series, the opposite is true for the expert survey. Here, the map designs showing absolute values (RD1 and RD2) outperformed the other two map designs in the categories map comprehension and usefulness for communication. However, the differences in scores between absolute values and anomalies were smaller than that of overlaid vs. separate uncertainties. Considering also the qualitative feedback from the workshops, no reliable evidence was found that map users preferred one option over the other. Evidently, tasks requiring participants to read data from a single map were solved better than combining separate pieces of information; so, it depends on the map users’ objective to rather communicate changes or future states. This objective should be defined beforehand with the target audience in mind to optimise map comprehension (Sedlmair et al. 2012). Moreover, the decision to use absolute values or anomalies in maps depends on the complexity of the portrayed indicator more than the meteorological variable(s) associated with it. Typical ‘peak over threshold’ indicators like heat days or number of days with heavy precipitation may be easier to communicate with absolute values, while more complex indicators with abstract units (e.g. heating degree days, distribution quantiles) are better understood as anomalies from a known state.

Map comprehension was differentiated into three classes of cognitive complexity. In both steps of the stakeholder involvement process, the CLC class understanding scored significantly lower than the other two classes. The progression of map competences described by Claaßen (1997) lists reading, spatialising (relating the map to reality), interpreting and developing hypotheses and prognoses from the map as competences in ascending order of cognitive complexity. Thus, it would be expected that questions of the understanding type scored higher compared with the interpreting type, where participants were asked to validate or refute statements based on the maps’ contents. Typically, tasks of the understanding type involved calculating the difference or sum of two map values at a given location. One reason for this unexpected outcome could be that understanding tasks were mistaken for reading tasks by participants, thus answering understanding type questions systematically wrong. Answer options to those questions often included the indicator value shown on one of the maps, which led participants to think that the single value instead of a sum or difference was the correct answer. On the other hand, questions of CLC class interpreting listed distinct statements as answer choices, making it easier to guess the correct answer without really interpreting it from the maps. This methodical shortfall should be considered in future research that evaluates map competences.

Learning effects were analysed by comparing the participants’ scores for the first and second map (expert survey) or map set (practitioner workshops). The groups were differentiated by the type of indicator (temperature or precipitation-related) shown on the maps. Overall, map comprehension increased with the time participants was engaged with the maps. This overall increase is more pronounced in the results of the practitioner workshops, where participants completed six maps in each session and had much more time to learn the layout of the maps and the mode of the test questions. In comparison, experts tested only one map for each indicator type and had lower overall learning rates. The expert group that started with a temperature map scored even lower with their second map. This finding is in line with Retchless and Brewer (2016), who found that the ability to rank sections of a map correctly increased with the time participants took to complete the task.

By aggregating the scores over each indicator type, it was possible to isolate the effect that the type of indicator had on map comprehension. For the workshops, the total learning effect is driven primarily by the group that started with temperature-related indicators. It seems that they had an easier entry point into the survey, already scoring slightly better in the first half than participants that started with precipitation-related indicators. A possible explanation is that in everyday life, people are more often confronted with temperature maps (e.g. weather forecast) and the spatial context of temperature is more meaningful to them than that of precipitation patterns. In addition, maps of temperature-related indices show familiar patterns (like alpine mountains and valleys, cities and basins) that may be not as apparent on precipitation maps. Orientation on a temperature map is therefore achieved more intuitively, which is especially helpful for complex map designs. With the experience from the first half of the survey, the group that started with temperature indices had a learning advantage for the second half. In the expert survey, the learning curve is much flatter and shows only a minor gain for participants that started with precipitation-related indicators. Since only less complex maps were tested in the expert survey, orientation did not play as big of a role as in the workshops. Also, it can be assumed that experts on climate change communication have more experience in dealing with scientific data visualisations.

7 Conclusions

This study examined whether there are design principles that support both the aesthetics, usefulness for communication and also comprehensibility of climate change impact maps. Ample literature on map design principles has been published (e.g. Brewer 1996, McKendry and Machlis 2009, Kaye et al. 2012) and most of it concludes that there is no optimal visualisation for every aspect of communicating climate impacts to adaptation policymakers (Lorenz et al. 2015, Grainger et al. 2016). However, some new insights were gained in a two-step survey with stakeholders from regional and local adaptation policy and climate communication experts. The findings suggest that climate change impact maps should not be cluttered with too much visual information. Including information about uncertainties is important for scientific credibility, for showing multi-model spread and the interrelation between mitigation and adaptation. However, map comprehension is higher when uncertainties are portrayed separately (Kaye et al. 2012, Retchless and Brewer 2016).

Maps comparing multi-model climate scenarios with the current climatic situation are inherently complex, especially for users with no background in climate modelling. To gain deeper insight into the understanding of such complex visualisations, this study is distinguished between different levels of cognitive comprehension. Tasks that require reading values from map locations consistently scored highest, while combining information from different maps resulted in lower comprehension scores. Map makers can aid comprehension by providing guidelines that explain the difference between absolute values and anomalies and demonstrate the effective use of both depending on the communication goal. Examples of correct (or even incorrect) interpretation of the map contents can help users understand the strengths and limitations of such maps for adaptation planning.

The final map-set of the project CLIMAMAP takes up most qualitative and quantitative results of the stakeholder involvement process and provides consultants and policymakers with a tool that is adaptive to a range of needs. The map-set comprises of eleven climate impact indicators (available in Online Resource 3) on regional scale (Austrian federal states). A less complex map sheet presents absolute values of the indicators’ climatological mean, and a more complex one shows the climatological development over the 21st century for two different emission scenarios. The set includes an uncertainty measure for the underlying model ensemble. The maps and corresponding styles were also published as GIS-data so they can be easily integrated into existing online GIS-systems (see recommendations by McInerny et al. 2014). Accompanying guidelines explain the map contents and list some considerations for map users. All material was made publicly available via the Climate Change Centre Austria’s (CCCA) Data Centre (Becsi and Laimighofer 2018). An example of the final maps and direct URLs to additional materials can be found in Online Resource 6.

The co-design of climate change impact maps yielded important insights into the needs of adaptation planners. Future research should identify the maps’ real-world application and evaluate their effectiveness in informing adaptation policy. Furthermore, an empirical evaluation of the recommendations and design principles from this study could yield important knowledge to further support the science-policy-interface in adaptation planning.