1 Introduction

1.1 What is Meant by Citizen Science?

The topic of Citizen Science (CS) has received increasing attention in recent years. The basic idea of this research approach is the interaction of scientific and civil society actors to jointly address scientific questions (Bonn et al. 2016). From the perspective of institutionally integrated researchers and experts, Citizen Science offers the potential to communicate approaches, methods and findings beyond the specialist public and to place the acquisition of data on a broader basis through participatory approaches. A common orientation of Citizen Science is the broad and decentralized collection of data and information (Pettibone et al. 2017). This is done, for example, by collecting and analyzing observations of native flora and fauna, changes in the environment, or developing social perspectives on sustainability issues. With the help of interested Citizen Scientists, many more places can be studied at many more times than the resources of a small full-time research team would allow.

From the perspective of the citizens involved, Citizen Science offers the opportunity to participate in scientific processes, to contribute their own questions, to exchange ideas with other interested parties, to learn new things, and to help shape the basis for decisions. The range of topics is diverse. The central platform for Citizen Science in Germany (www.buergerschaffenwissen.de, Fig. 1) currently features around 120 Citizen Science projects, ranging from nature and environmental topics to technology research and social issues. There is often a direct call to participate in research, with digital tools such as apps and reporting platforms playing a central role.

Fig. 1
figure 1

Website of Bürgerschaffenwissen.de with a selection of projects, including “Experiencing biodiversity” (Artenvielfalt erleben) https://www.buergerschaffenwissen.de/index.php/projekte (5. March 2021)

Full size image

The topic of Citizen Science is viewed critically if citizens are exclusively assigned the role of free data collectors (Lave 2012). In this case, the idea of Citizen Science would be instrumentalized and hobby and amateur researchers would be denied the opportunity to experience scientific working methods. Moreover, science would not do itself any favors by unilaterally reducing Citizen Science to the free acquisition of data, if this would encourage the undermining of science funding.

Accordingly, basic concepts and funding guidelines for Citizen Science call for a broad-based involvement of Citizen Scientists that goes beyond the collection of data (ESCA 2015; Bonn et al. 2016). Instead, participants should be given the opportunity to contribute their own questions, learn about methods and participate in the evaluation and use of results.

1.2 Quality and Scope of Crowdsourced Collected Data

The size and scope of Citizen Science projects can vary greatly. On the one hand, there are often regionally based initiatives, but there are also large-scale initiatives for the investigation of questions from the immediate living environment. At the same time, digital tools and communication channels make it possible for many participants to work together, gather information and link the findings of different participants, regardless of location and time.

Regardless of the popularity and reach of specific Citizen Science projects, they usually face the challenge of unsystematic data collection (Wahl et al. 2014). For example, information about the local flora and fauna is collected when and where volunteers have the time and inclination to do so. This may be more likely to happen on weekends in local recreation areas than at consistent intervals at specific points in urban areas. In contrast to standardized measurements, Citizen Science research often involves unsystematically collected data that cannot be compared with at least standardized and regularly repeated monitoring surveys.

Initiatives with explorative questions without claims of generalization have it easier to emphasize the context dependency of their findings. However, when it comes to the area-based recording of biodiversity, crowdsourced information at first glance resembles data that has been measured in a highly standardized manner. However, they do not necessarily meet the high demands of standardization. Challenges with regard to data quality and generalizability exist, for example, in guiding Citizen Scientists to record information as precisely as possible (Ratnieks et al. 2016). However, the meaningfulness and scope of the results must always be reflected against the background of selective participation (in terms of content, space and time), among other things because only bird species known are (or can be) reported.

With regard to the quality of reported data, good training of the recorders as well as algorithm-based plausibility checks and evaluations of statistical outliers by experts help in the first instance (Jacobs and Resch 2013). Furthermore, it can be helpful for the data quality if users can classify their own contribution to a data collection and thus gain an awareness of the significance of their observation. When using and interpreting observation data, users must be made aware of the origin of the data basis and the corresponding scope. This applies in particular to frequently used cartographic visualizations. It is important to differentiate that data gaps or empty fields on distribution maps can indicate either selective participation or the absence of a studied species. So far, this differentiation has hardly been possible for most maps created in the context of Citizen Science projects (Fig. 2).

Fig. 2
figure 2

The presentation of all observations of Bewick's Swans between October 2020 and March 2021 allows an impression of the distribution of the population in autumn and winter. However, in order to be able to interpret the visualized data, especially data gaps, more precisely, important information is missing, such as information on migration behaviour but also on the reporting behaviour of birds as a whole (https://www.ornitho.de/; 18. March 2021)

Full size image

2 The Research Project „Artenvielfalt erleben“

We have taken up these considerations as part of the research project “Experiencing biodiversity. How nature research on your own doorstep benefits from interactive web maps” and translated into a practical solution proposal Footnote 1. The research project aimed to make the collaboratively collected data of the bird observation portal ornitho.de accessible to the public with the help of web maps and by taking data interpretation into account. Ornitho.de is Germany’s largest platform for reporting bird observations. Since 2011, 32,000 registered users have reported their observations to the portal, which reached the 50 million entry mark in 2020. For the general public, the data has so far only been displayed on static maps of Germany. Only particularly active observers had access to the database for their own queries and analyses.

Based on this initial situation, the two primary project goals were (1) to involve the community of data-collecting Citizen Scientists in the sense of a comprehensive definition of Citizen Science (see above) in the further development of ornitho.de by developing interactive web maps. On this basis (2) to design a web map together with the ornithologists that makes the huge data treasure available to a broad public in the form of visual and scalable evaluations, the “Ornitho-Regioportal”. Both regionalization and visualization are intended to motivate Citizen Scientists to report more regularly and thoughtfully, and in particular to prepare observation lists (Moser et al. 2020).

To achieve these goals, we implemented several elements of a user-centered design approach (e.g., Roth et al. 2015). For example, we actively involved the ornitho community in the development in several workshops (Moser et al. 2020). In addition to content-related and functional requirements for a web map, we discussed different visualization variants together. Thereby, it was necessary to balance between the participants’ experiential knowledge, i.e., known forms of cartographic visualizations, the cartographic-technical demands, but also the most diverse protection demands on the part of the participants as well as on the part of nature conservation (see e.g., Moser 2018). The result were different conditions that had to be considered in the design of the visualization solutions. Based on the previous experience of the Citizen Scientists with raster maps and the necessary protection of the data of both the reporters and sensitive bird species, it was already decided at the first workshop that the data would be presented as a raster representation, which, however, should have a much better resolution than previous services.

Furthermore, accompanying research was conceived to support the development of the web maps with regard to usability. In contrast to basic research, the research and investigation questions were strongly related to the respective project phase and also depended on the developments in the project. The goal was to use empirical research to find solutions for certain parts of the visualizations that could best be understood and interpreted by the target group, the large group of dedicated birdwatchers.

3 Visualization Options of CS-Based Datasets

3.1 Cartographic Visualization of Data Gaps

Numerous topics in the context of Citizen Science have spatial aspects and generate georeferenced data accordingly. These can be locations of animal and plant species, information on the condition of water bodies or soil types, hotspots of environmental pollution or danger spots in road traffic. The data is collected via smartphone apps with directly recorded locations or via browser-based input masks that query the discovery or reporting locations. These are always data sets that, due to their collaborative creation, can be defined as neither complete nor certain.

The representation and consideration of uncertainty in the visualization of data has been intensively discussed in cartography for many years. Since any spatial datasets can have uncertainties, not only visualization possibilities are in the focus of research (e.g., Kinkeldey and Schiewe 2012) but also impacts on decision-making processes and trust in results if the (in)certainty of data is ignored (Zhang and Goodchild 2002; Kinkeldey et al. 2017). In addition to appropriate typologies, the development of visual methods for representing uncertainty has been a focus of interest (MacEachren et al. 2005; Kinkeldey and Schiewe 2012, among others). Digital and web applications, in particular, can promote the representation of uncertainty through a variety of visualization options. Approaches to uncertainty representation have been designed and also evaluated for various projects. According to Kinkeldey et al. (2014), five approaches to uncertainty visualization can be distinguished:

  • Explicit or implicit: uncertainties are represented directly or indirectly by combining several visualizations showing different possible outcomes.

  • Intrinsic or extrinsic: existing objects are, for example, changed in their color value to represent uncertainty or new objects are introduced on the map for this purpose

  • Visually integrated or separated: uncertainty representation may or may not be perceptually separated from the data

  • Simultaneous or side by side/consecutive: data and uncertainty representation are displayed together in one view or separately, side by side or consecutively

  • Static or dynamic: it is a static map or interactive or animated visualizations.

A crucial factor for further development was the easy readability and interpretability of the data also for population groups that are less intensively engaged with the data and its contents.

3.2 Examples from the Biodiversity Domain

Interactive or static web maps are often used to evaluate and visualize jointly collected spatial data. However, it can be observed that especially—but not only—projects organized by citizens continue to struggle to deal with the uncertainty of their data base and, above all, to make it transparent. This may be due to a lack of contact with visualization experts, a lack of programming skills, or even a lack of resources. On the other hand, we did not find a lack of interest in or disregard for the problems of uncertain data and their evaluation during our interviews. People who are intensively involved with the data are aware of the challenges of evaluation and therefore use maps only as one element of evaluation among several. However, this becomes critical with the claim and goal of also making the jointly collected data available to a broader public, as envisioned by the basic CS idea of joint work on scientific questions (Bonn et al. 2016). In this case, it can no longer be assumed that appropriate background knowledge is available or that visualizations are classified and critically scrutinized. At the same time, the designed visualizations should be as simple, understandable and intuitively usable as possible.

Biodiversity data are often presented either on a raster basis in the form of points (Fig. 3) or raster areas, on an administrative level such as counties (Fig. 4) or as point data of individual observations (Fig. 5). The type of presentation depends, among other things, on conservation interests, be it species protection or protection of the reporters. For this reason, the point-based and precisely localizable representation of individual observations is often avoided, especially in the case of highly endangered species, preferring a grid representation of various scales. The representation of data on the basis of administrative boundaries, i.e., in the form of an area cartogram, can be easily implemented, but in our opinion does not meet the objectives of such a map. Especially large administrative units distort the perception, especially if observations are only made in small areas, but are transferred to a larger area (cf. Fig. 4).

Fig. 3
figure 3

Section of a grid-based distribution map of the House Sparrow for the year 2020 (TK25). In addition to the information on whether the House Sparrow was observed and reported as a species in a grid cell, the colour indicates the number of reported observations. However, empty grid cells can hardly be interpreted, as the data situation there is completely unclear. The additional note above the map that the representation is based on the casual observations and not on systematically collected data is considered positive. This map display is only available for registered users (https://www.ornitho.de/; 18. March 2021)

Full size image
Fig. 4
figure 4

Distribution map of the Blue Tit based on the administrative units from the project “Stunde der Gartenvögel”. The darker the individual districts are shown, the more birds were reported per garden. However, the map does not show how many gardens were observed per unit, nor what area of the total municipal area these gardens correspond to (https://www.nabu.de/tiere-und-pflanzen/aktionen-und-projekte/stunde-der-gartenvoegel/ergebnisse/15767.html; 6. November 2020)

Full size image
Fig. 5
figure 5

Distribution map of the Long-tailed Tit as individual observation points (light blue dots) in the Senckenberg Species Database Main-Kinzig-Kreis. In the case of endangered species, this type of representation is not recommended, as, for example, nests can be recognised and plundered (https://rmo.senckenberg.de/search/search.php; 6. November 2020)

Full size image

Other projects, such as the Atlas of Living Australia, offer extensive display options including colors and display methods that can be changed by the user. Figure 6, for example, shows two different animal species together, using different methods (point display, grid display). Such portals must be described as expert versions, since on the one hand scientific genus names are used and on the other hand the conditions, differences and possible interpretations of the display options are hardly explained.

Fig. 6
figure 6

Comparative representation of two species in the “Atlas of Living Australia”. Apart from the use of subject-specific species names, the map contains no indication of whether a comparison of these species is advisable or not, i.e., whether the selected species interact with each other, are dependent on each other or possibly displace each other. This form of map portal is particularly suitable for experts with the corresponding prior knowledge (https://spatial.ala.org.au/; 5. March 2021)

Full size image

For some applications, heat maps are also used to visualize point clusters (Fig. 7) or interpolations. Like in the present example, areas requiring protection can be visually highlighted. However, for each heat map and especially for collaboratively collected data, the underlying database should be known and verifiable. Other graphical representations of spatial data are also used, depending on the application. However, most of these representations are difficult to read and can only be interpreted reliably with extensive prior knowledge.

Fig. 7
figure 7

Forest nature conservation in the Spessart: “Heatmap” of the focal points “old forests” and species. Yellow to orange-red areas show the highest density of forest bird species, supported here by the dotted representation of observation data. In this case, areas with a high need for protection become visible https://www.lbv.de/naturschutz/standpunkte/wald-und-forstwirtschaft/spessart/; 5. March 2021)

Full size image

All exemplary maps visualize observation data of voluntarily engaged Citizen Scientists. The locations as well as the density of findings depend significantly on the activity of the participants, both in the sense of the spatial and temporal distribution of observation data as well as in the sense of the content level. In addition to the factors that flora observation objects usually move around and that observations vary in intensity spatially and temporally due to their voluntary nature, it must also be considered that reports can only be submitted for species that can be recognized and named. Even though there are now numerous identification apps, observation lists on which, for example, all detected bird species/birds in a certain area during a certain time are recorded are rather rare. Such data differs from standardized measurements in that locations without reports do not provide any information about whether the respective species does not occur there, or whether no observers were active there. This fact has to be taken into account when interpreting the maps. In order to differentiate between data gaps and the absence of certain species in an area, it is helpful (and a task of cartographic design) to be able to include information about the activity areas of the reporters.

3.3 Support of Data Interpretation in the Ornitho-Regioportal

The ornithologists involved in the development process (see 2) discussed various cartographic elements that can support the interpretation of the data. In addition to the essential innovation of displaying the reporting activity (see 3.4), several switchable background layers were integrated. In addition to standard maps such as OSM and satellite imagery, layers on land cover, precipitation, air temperature and relief, but also administrative boundaries, water bodies and bird sanctuaries were integrated. These layers contain information, either individually or in combination, which enables trained ornithologists in particular to better classify the observation data displayed.

Furthermore, there were detailed discussions about the display of further contents of the observation data as well as about further display methods. The main focus was on the presentation of a reporter’s own observations. Proposals were developed for this, but these were initially put on hold in order to design a functional portal for the broad interested public within the framework of the project. For the participating community in particular, however, the precise presentation of one's own reports has a great added value: on the one hand, for the motivation to recognize one’s own contribution, but on the other hand, also with regard to the quality of data collection, in order to be able to assess and reflect on the value of one's own reports. In addition, the observation data offer a variety of possibilities for cartographic presentation. These include information such as the number of different species per grid, the number of observers per grid, the maximum number of individuals per grid. Such information can support interpretations, but it can also lead to misleading conclusions if used incorrectly, as randomness, multiple observations by different people and other effects have to be taken into account. Before implementation, an evaluation must be carried out to determine the extent to which the presentation of such content may also affect conservation interests, e.g., if the number of species and/or observers can implicitly be used to draw conclusions about protected species. For a similar reason, other methods of representation such as heat maps or interpolations were not used. The former was considered for the representation of species density and implemented on a test basis. However, due to the unsystematic collection of the underlying data, we had to conclude that the heat map showed less “hotspots of species diversity” than “hotspots of observation intensity”, since in areas with a high observation density, more species are apparently reported overall. While such a presentation also brings new insights, especially for experts; in the context of a portal for the general public, however, it must be weighed more carefully which impressions the map's image causes, which in most cases are difficult to catch and classify through additional explanations.

3.4 Visualization of Reporting Activity in the Ornitho-Regioportal

The innovation in the development of the map-based analysis platform within the framework of the research project lies in the implementation of a map layer that depicts the reporting activity of the participants. This reporting activity represents all reports of all types over a certain period of time on a grid basis (for all scale levels TK50 to half-minute fields). It thus provides information about the site-specific activity of the reporters, but not about the occurrence of a species. Reporting activity is represented by transparent shades of blue (Fig. 8). The occurrence of the selected species (Kingfisher in the example) is indicated by the red dots also grid-based.

Fig. 8
figure 8

Reporting activity (blue grid fields) and species occurrences (red dots) in the ornitho-Regioportal

Full size image

From the combination of the occurrence and reporting activity layers, the reliability of the display of the species occurrence in a grid field can now be concluded. If the reporting activity is high (dark blue grid field), it is very likely that the actual occurrence of the selected species in this grid field and in the given period is shown. In the case of low reporting activity (light blue grid), on the other hand, a missing red dot may mean that the selected species is not present there or that the selected species occurs there but has not yet been reported due to the small number of observers. This representation identifies data gaps in unsystematic surveys. This should enable users to reflect on the fact of selective data collection when interpreting the maps and to indicate “white spots” on the map accordingly (cf. Fig. 9).

Fig. 9
figure 9

Reporting activity as a tool for interpretation: Distribution of Robin observations in the south-west of Münster for the years 2018 to 2020 based on the half-minute fields. From the upper map, it could be concluded that Robins mainly occur in settlement areas. By adding the reporting activity, it becomes clear that the birds mainly “occur” where there are also many observation reports. In this way, the reporting activity can contribute to a better interpretation of the distribution pattern shown

Full size image

One advantage of the grid-based representation of the reporting activity and the bird data is that, in addition to providing the information, it also takes data protection into account. For example, it would have been inconceivable to display the precise data of the observations together with the time stamp, as this would have made it possible to analyze movement data and possibly even draw conclusions about individual Citizen Scientists. The protection of rare or sensitive bird species can also be guaranteed far better with a grid-based representation than is possible with other, mainly point-based forms of visualization. For example, the level of detail can also be varied depending on the protection status of bird species by limiting the zoomability. In addition, the point-based representation would suggest an accuracy that does not apply to migrating animals.

The depiction of reporting activity as the “sum of all observations of all bird species in the relevant grid field in the selected time period” (Moser et al. 2020: 35) is one way to support users in interpreting observation data. With regard to the five approaches to visualizing uncertainty by Kinkeldey et al. (2014), the representation of reporting activity can be classified as explicit and extrinsic, as uncertainty is directly represented in the form of the new map layer “Meldeaktivität”. Furthermore, it is visually separated from the occurrence information and can be shown and hidden simultaneously and interactively.

4 Empirical Evaluation of the Presentation of Data Gaps

4.1 Gradation of Reporting Activity

In the actual implementation of the representation of reporting activity, the question arose as to a meaningful number of gradations. Since there has been little experience with similar types of representations, this question could not be answered based on prior knowledge or literature, so we conducted empirical studies on this issue. This experiment investigated whether a three-level or five-level gradation of reporting activity is better suited to help the relevant target group interpret the dissemination data. We differentiated between individuals registered with ornitho.de who report less than ten observations on a monthly average and individuals who report ten or more observations on a monthly average and thus also have access to the database search. The background of this distinction was our assumption that persons with more reports are also more experienced in the use of ornithological data and their visualization. That experience has an influence on the use of available information has been shown in numerous studies (e.g., Egan and Schwartz 1979; Soloway and Ehrlich 1984). Thus, it was shown that experts coped worse with more supporting information, while less experienced persons benefited from supporting information (“Expertise Reversal Effect”; Kalyuga 2007). With regard to the abundance of information, on the other hand, it could be shown that experts are better able to organize this information and to integrate it into a coherent overall picture, so that they were able to cope with more information than persons with less prior knowledge (Chase and Simon 1973). Since the five-level classification of reporting activity provides additional information but no further support, we hypothesized that more experienced users would cope better with the five-level version than less experienced users. The latter should perform better with the reduced three-level representation than with the five-level representation. We tested the extent to which this works from the user's perspective and which classification provides the best support in two studies.

4.1.1 Methods

The two representations differed only in the number of gradations of reporting activity. In the five-level representation, the levels consisted of “none (0 reports),” “low (1–40 reports),” “moderate (> 41 reports),” “high (201–1000 reports)” and “very high (> 1000 reports)”. The three-level map did not distinguish between the last three levels, so “moderate” included all values greater than 40. The level thresholds were determined by comparing empirical data from ornitho.de with the Breeding Bird Atlas (“Brutvogelatlas” Gedeon et al. 2014) by calculating how many reports are needed to record a certain proportion of bird species on average (for derivation of the class thresholds, see Moser et al. 2020: 36 f.). Quartiles were defined as threshold limits for the five-level map. For TK25 quartiles, values were obtained that, when rounded, resulted in the above-mentioned limits.

The participants were given two different tasks. In the first task, they were asked to evaluate the truth of statements on the basis of the maps. They had four different answer options: “true,” “not true,” “information not available on map,” and “don't know” (example in Fig. 10). All participants were presented with three maps, each with five statements, which they were asked to rate. In a second task, three grid fields were to be marked on each of two maps, in which a given bird species could most likely be observed or not observed. The corresponding information was to be read from the maps (see Fig. 11).

Fig. 10
figure 10

Example of a map with a statement from task 1 of the representation study. Participants should use the map to evaluate whether the statement is true. In this example, the correct answer is 'True' („Stimmt“)

Full size image
Fig. 11
figure 11

The first of the two maps from task 2 of the representation study. The participants could click on individual grid fields to mark them

Full size image

A total of N = 173 individuals (128 male, 45 female) with an average age of M = 51.78 years (range: 18 a–83 a) participated in the study, two-thirds of whom reported having at least a college degree. 84 individuals contributed an average of less than 10 reports per month and were classified as low activity, while 89 individuals with at least 10 reports per month were classified as very active. One half of each group was presented with the three-level maps and the other half of the groups with the five-level maps, so that both presentation variants could be tested with both more active and less active individuals.

4.1.2 Results

An analysis of the proportion of correct responses (Table 1) showed that neither membership in one of the user groups (\({\chi }^{2}\) (1) = 0.03; p = 0.862) nor presentation (\({\chi }^{2}\) (1) = 0.95; p = 0.331) had an isolated influence. We also found no evidence of an interaction between these two factors, \({\chi }^{2}\) (1) = 0.47; p = 0.495. Thus, both variants turned out to be equally suitable for mapping and interpreting bird species dispersal and reporting activity for both user groups. However, we found an unexpected difference between the statements. Participants had greater difficulty with statements when information relevant to their response was not mapped (M = 38.4% correct responses, SD = 25.2%). In comparison, statements that could be unambiguously assessed as “true” or “not true” resulted in M = 77.9% correct responses (SD = 20.2%), \({\chi }^{2}\) (1) = 9.86; p = 0.002). It should be noted here that participants were not asked to know information that was not depicted but were simply asked to indicate when the required information was not depicted.

Table 1 Proportion of correct responses for tasks 1 and 2 of the representation study, broken down by experimental condition
Full size table

In the second task, in which grid fields were to be marked, there were also no differences between the gradationsFootnote 2 (both |z|≤ 0.81; both p ≥ 0.420). Also, more active participants did not perform better than less active participants (both |z|≤ 1.02; both p ≥ 0.307). Similarly, there was no interaction between user group and representation (both |z|≤ 0.42; both p ≥ 0.673). In the subjective evaluation of the tasks in terms of mental and temporal demand, performance, effort and frustration (recorded in the NASA TLXFootnote 3; Hart & Staveland 1988), there were also no influences of representation and user group (all F ≤ 1.50; all p ≥ 0.218).

4.1.3 Interpreting

The study made clear that both representations were equally difficult to interpret and that there was no difference between more active and less active reporters. Thus, the hypothesis of dependence between activity and number of gradations could be rejected. Since the representation of the reporting activity can basically be seen as supporting information for the interpretation, it could be that the active participants did not need this information and therefore did not pay attention to it. This could be a reason why they did not benefit from the differentiated representation. On the other hand, the proportion of correct answers was rather low in both groups. Thus, it remains open why there were no differences between more active and less active Citizen Scientists and why both representations seem to be equally suitable.

The only difference was that the answer “information not available on map” was more difficult than the specific answers “true” or “not true”. For individual statements, the rate of correct answers dropped to about 10%, which was even below guessing level. In a second study, we therefore tested the hypothesis that additional instructions explaining the representation could facilitate the processing of the tasks. We selected a video tutorial as the format of the instructions. On the one hand, support services should be offered in a low-threshold and entertaining way due to the open character of the online platform. On the other hand, against the background of research, we assumed that with the help of a videotutorial comprehension could be promoted (Brecht 2012; van der Meij and van der Meij 2014). In the study, we systematically varied whether or not participants were shown a tutorial prior to completing the task.

4.2 Video Tutorial Support

4.2.1 Methods

The tutorial was implemented as a video. It lasted 3:24 min and consisted of a screen recording with an accompanying explanatory audio track. The tutorial focused on species occurrence, i.e., whether a particular bird species was reported in the specified time period and reporting activity, i.e., how often bird observations were generally reported in this area in the specified time period. Both the presentation of the information and its meaning were explained. Participants were also informed that a missing dark red dot marking the presence of a bird species in a grid field does not automatically indicate that the bird species under consideration was not present there during the relevant period.

The tasks were identical to those previously completed by the participants in the representation study. A total of N = 90 individuals (71 male, 18 female, 1 no information) with a mean age of M = 52.52 years (range: 21a–80 a) participated in this second study. Also in this group, over 60 percent of respondents reported having at least a college degree. Since based on the results of the representation study the average reporting frequency had no influence on the correctness of the responses, we decided not to differentiate between more active and less active ornitho reporters. For reasons of practicability, we therefore only invited more active ornitho reporters to participate.

4.2.2 Results

Similar to the representation study, the assessment of the 15 statements showed no difference in correctness between the group with a tutorial and the group without a tutorial \({\chi }^{2}\) (1) = 0.29; p = 0.590 (Table 2). As in the representation study, there was a difference in the proportion of correct responses between questions that could be answered unambiguously with “True” or “Not true” and those for which the relevant information was not presented, so that “Information not available on map” was the correct response option. The former were answered correctly in M = 80.0% (SD = 17.8%) of cases, the latter only in M = 40.4% (SD = 24.3%) of cases.

Table 2 Proportion of correct responses for tasks 1 and 2 of the tutorial study, broken down by experimental condition
Full size table

However, in the first map of the second task (see Fig. 11), in which the participants were asked to mark grid squares on the map themselves, there was an effect of the tutorial, \({\chi }^{2}\) (1) = 5.69; p = 0.017. The group that had previously seen the tutorial marked grid squares more often correctly than the group without tutorial (see Table 2). However, this difference was no longer significantly pronounced in the second map of this type, which asked to mark the grid patches where a particular bird species was most likely to be found, \({\chi }^{2}\) (1) = 0.43; p = 0.512, although in both groups the task was solved correctly very frequently, with over 90% correct answers. Another result that suggests at least a small advantage of the group with tutorial are the data on subjective stress, collected by the NASA-TLX (Table 3). Thus, the participants with tutorial indicated a significantly lower mental demand (F(1, 80) = 6.14; p = 0.015) and a lower necessary effort (F(1, 74) = 4.42; p = 0.039) than the participants* without tutorial, while time demand (F(1, 84) = 3.93; p = 0.051), estimation on own performance (F(1, 79) = 1.47; p = 0.230) and frustration (F(1, 79) = 1.66; p = 0.201) were reported at a comparable level. Thus, in summary, the tutorial showed an advantage on the first map of Task 2 and partly on the subjective evaluation of the task, whereas on answering the questions of Task 1 and part of the subjective evaluation, the tutorial did not make a difference.

Table 3 Mean rating in the NASA-TLX of the tutorial study, divided according to experimental conditions
Full size table

4.2.3 Interpreting

In this second study, the difficulties associated with the correct response “Information not available on map” were also evident. Obviously, the subjects found it particularly difficult to identify where the limits of validity of the mapped data lay. Contrary to our hypothesis, we were not able to significantly reduce these difficulties even with the tutorial, although care was taken to explain all the information presented. Therefore, in future tutorials and supporting materials, special care should be taken to explain not only the content, but also where the limits of validity of the data depicted lie.

4.3 Discussion

In order to make the visualization as comprehensible as possible for many target groups, the gradation by means of different numbers of levels and a tutorial as a support service were empirically investigated. The first study examined the influence of the number of gradation levels used to visualize the data density on comprehension: no measurable effect was found. Rather, participants had difficulty recognizing when the information presented was insufficient to make a specific point. This difficulty could not be remedied by additional instruction in the form of a video tutorial, as we were able to show in the second study. This finding could be an artifact of the study situation. Since participants were presented with a map for each statement, their expectation might have been that the information presented on the map was also suitable for answering the question. This could be counteracted by explicitly pointing out before the start of the tasks that not all required information is always available. On the other hand, this finding could show how difficult it is for users to recognize the limits of the validity of the information presented. This should be taken into account when data is made available in the context of Citizen Science projects or when Citizen Scientists are involved in the evaluation and interpretation of data. Thus, explicit examples could be used to clarify which statements can or cannot be made based on the available and presented data. In the tutorial study, we were able to show that an interpretation aid in the form of a video improved performance on at least one of the tasks. Respondents also indicated that they found the tasks less mentally demanding and that they had to exert less effort. These results provide at least some indication that additional interpretive aids can reduce the difficulty of understanding maps that combine multiple layers of information.

The results of the representation study also indicate that the activity of the reporters had no influence on whether the maps could be correctly assessed. However, this also means that the active participation of a person in reporting observations does not indicate their expertise in interpreting the observation data. This insight is also important for coordinators of Citizen Science projects when it comes to evaluating contributions in terms of their reliability. Hence, a good reporting performance is not the same as a good performance in evaluation and interpretation, which is why separate quality and review criteria should also be developed for evaluation and interpretation. Similar to already available trainings for reporters, trainings for evaluators should be offered as well. Due to the novelty of our map presentation, it remains to be seen and to be investigated in further studies whether a more intensive engagement with the maps will build up an expertise among those Citizen Scientists who are more engaged with the data provided. In any case, we were unable to show that more active reporting behavior automatically led to a more correct interpretation of the dispersal data.

Another aspect of reporting activity representation that has not been addressed here so far is the highlighting of “under-monitored” areas. This creates transparency so that Citizen Scientists can decide for themselves, based on this data, when to monitor and where. In this way, this information could also contribute to making under-observed areas more interesting for observers and thus also ensure a more even coverage of the area with observations and, above all, with reports. This more even coverage would allow for spatial comparisons and interpretation of changes in a species population over longer periods of time. Since the two studies reported here did not investigate reporting behavior, but rather the interpretation of the data presented, no statement can be made in the context of this paper as to whether this additional layer of information can also influence observation and reporting behavior. For this, further studies are needed that observe the reporting behavior and ask Citizen Scientists about the criteria on which their selection of observation areas is based.

5 Conclusion

This article emphasizes in the first part the importance of presenting not only dissemination data but also reporting activity in the context of Citizen Science projects. The presentation of the data basis is particularly important for a correct interpretation, since, on the one hand, the data collection by Citizen Scientists is often spatially unsystematic and, on the other hand, Citizen Scientists do not necessarily have an academic-scientific training regarding the handling of data. In the paper, possibilities of visualizing the data density were discussed and the approach used in the project “Artenvielfalt erleben” was presented, including two empirical studies.

This article shows that the publication and visualization of data collected in Citizen Science projects alone is not sufficient to generate surplus value for the Citizen Scientists involved and to bring science closer to them. Rather, it shows that efforts must be made to adequately present and explain the data base. Thus, it is especially important to communicate where the limits of the validity of the collected data lie. It is important to convey to interested persons that a missing presence report (in our example the missing red dot in a grid field on the map) does not mean that the bird species is not common there, but it can just as well be that there were simply no reporters around who could have observed the bird species. This interpretation can be facilitated in the case of dissemination maps by the additional representation of the reporting activity. In this layer all observations of all species are summarized. Similar representations could provide additional value in other subject areas as well, and whenever large data sets with many observations are available and when the full information on data density is not apparent from the data presented itself. Based on the way, the reporting activity is obtained, it is also clear that this approach can only be used when multiple pieces of information are collected, such as the distribution of all bird species in our example, which when combined can provide an indication of data density. Thus, the approach presented here is most suitable for projects that have large amounts of data and whose analysis usually only considers individual subcategories of these data, such as the distribution of individual species.

In summary, the involvement of Citizen Scientists beyond the collection of data, as is often called for (e.g., ESCA 2015; Bonn et al. 2016), requires additional support services in the form of interpretation aids and expanded representation options. The unsystematic collection of data in particular means that they are not self-explanatory and can be used by anyone at any time. Using the example of the dissemination of bird species, a map-based visualization option was presented, and its comprehensibility was examined in two empirical studies. It turned out that the participants found it particularly difficult to recognize the limits of the validity of the map representations. For Citizen Science projects this means that help for the interpretation of the data should be offered in order to enable the participants to examine the data independently with regard to their own questions and to process the results accordingly. Hence, it seems worthwhile for similar projects to take up the approach presented here for the visualization of data gaps and to pay even more attention to the development of suitable interpretation aids.