Abstract
Citizen Science initiatives often collect the data in an unsystematic way and dependent on the voluntary participation of interested citizens. The visualization of such data must present this incompleteness in a way that is understandable and comprehensible even to scientific laypersons if the correct conclusions are to be drawn. Using the project “Experiencing Biodiversity” as an example, this article shows how these requirements can be taken into account in the development of an online portal. Thereby, we discuss how data gaps can be pointed out by means of maps in order to be able to better interpret the significance of crowd-based data. We use empirical user studies to show the extent to which this was successful in the “Experiencing Biodiversity” project and the role of additional interpretation aids. Finally, we discuss how the findings of this project could be transferred to other use cases.
Zusammenfassung
Citizen Science-Initiativen sammeln Daten häufig unsystematisch und anhängig von der freiwilligen Beteiligung interessierter Bürgerinnen und Bürger. Die Visualisierung solcher Daten muss diese Lückenhaftigkeit auch für wissenschaftliche Laien verständlich und nachvollziehbar darstellen, wenn die korrekten Schlüsse gezogen werden sollen. Der Beitrag stellt beispielhaft anhand des Projekts „Artenvielfalt erleben“ dar, wie bei der Entwicklung eines Online-Portals diesen Anforderungen Rechnung getragen werden kann. Dabei wird erörtert, wie Datenlücken anhand von Karten aufgezeigt werden können, um die Aussagekraft crowdbasiert gewonnener Daten besser einordnen zu können. Inwieweit dies im Projekt „Artenvielfalt erleben“ gelang und welche Rolle zusätzliche Interpretationshilfen spielen, wird anhand empirischer Nutzerstudien gezeigt. Schließlich wird diskutiert, wie die Erkenntnisse dieses Projekts auf andere Anwendungsfälle übertragen werden könnten.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
1.1 What is Meant by Citizen Science?
The topic of Citizen Science (CS) has received increasing attention in recent years. The basic idea of this research approach is the interaction of scientific and civil society actors to jointly address scientific questions (Bonn et al. 2016). From the perspective of institutionally integrated researchers and experts, Citizen Science offers the potential to communicate approaches, methods and findings beyond the specialist public and to place the acquisition of data on a broader basis through participatory approaches. A common orientation of Citizen Science is the broad and decentralized collection of data and information (Pettibone et al. 2017). This is done, for example, by collecting and analyzing observations of native flora and fauna, changes in the environment, or developing social perspectives on sustainability issues. With the help of interested Citizen Scientists, many more places can be studied at many more times than the resources of a small full-time research team would allow.
From the perspective of the citizens involved, Citizen Science offers the opportunity to participate in scientific processes, to contribute their own questions, to exchange ideas with other interested parties, to learn new things, and to help shape the basis for decisions. The range of topics is diverse. The central platform for Citizen Science in Germany (www.buergerschaffenwissen.de, Fig. 1) currently features around 120 Citizen Science projects, ranging from nature and environmental topics to technology research and social issues. There is often a direct call to participate in research, with digital tools such as apps and reporting platforms playing a central role.
Website of Bürgerschaffenwissen.de with a selection of projects, including “Experiencing biodiversity” (Artenvielfalt erleben) https://www.buergerschaffenwissen.de/index.php/projekte (5. March 2021)
The topic of Citizen Science is viewed critically if citizens are exclusively assigned the role of free data collectors (Lave 2012). In this case, the idea of Citizen Science would be instrumentalized and hobby and amateur researchers would be denied the opportunity to experience scientific working methods. Moreover, science would not do itself any favors by unilaterally reducing Citizen Science to the free acquisition of data, if this would encourage the undermining of science funding.
Accordingly, basic concepts and funding guidelines for Citizen Science call for a broad-based involvement of Citizen Scientists that goes beyond the collection of data (ESCA 2015; Bonn et al. 2016). Instead, participants should be given the opportunity to contribute their own questions, learn about methods and participate in the evaluation and use of results.
1.2 Quality and Scope of Crowdsourced Collected Data
The size and scope of Citizen Science projects can vary greatly. On the one hand, there are often regionally based initiatives, but there are also large-scale initiatives for the investigation of questions from the immediate living environment. At the same time, digital tools and communication channels make it possible for many participants to work together, gather information and link the findings of different participants, regardless of location and time.
Regardless of the popularity and reach of specific Citizen Science projects, they usually face the challenge of unsystematic data collection (Wahl et al. 2014). For example, information about the local flora and fauna is collected when and where volunteers have the time and inclination to do so. This may be more likely to happen on weekends in local recreation areas than at consistent intervals at specific points in urban areas. In contrast to standardized measurements, Citizen Science research often involves unsystematically collected data that cannot be compared with at least standardized and regularly repeated monitoring surveys.
Initiatives with explorative questions without claims of generalization have it easier to emphasize the context dependency of their findings. However, when it comes to the area-based recording of biodiversity, crowdsourced information at first glance resembles data that has been measured in a highly standardized manner. However, they do not necessarily meet the high demands of standardization. Challenges with regard to data quality and generalizability exist, for example, in guiding Citizen Scientists to record information as precisely as possible (Ratnieks et al. 2016). However, the meaningfulness and scope of the results must always be reflected against the background of selective participation (in terms of content, space and time), among other things because only bird species known are (or can be) reported.
With regard to the quality of reported data, good training of the recorders as well as algorithm-based plausibility checks and evaluations of statistical outliers by experts help in the first instance (Jacobs and Resch 2013). Furthermore, it can be helpful for the data quality if users can classify their own contribution to a data collection and thus gain an awareness of the significance of their observation. When using and interpreting observation data, users must be made aware of the origin of the data basis and the corresponding scope. This applies in particular to frequently used cartographic visualizations. It is important to differentiate that data gaps or empty fields on distribution maps can indicate either selective participation or the absence of a studied species. So far, this differentiation has hardly been possible for most maps created in the context of Citizen Science projects (Fig. 2).
The presentation of all observations of Bewick's Swans between October 2020 and March 2021 allows an impression of the distribution of the population in autumn and winter. However, in order to be able to interpret the visualized data, especially data gaps, more precisely, important information is missing, such as information on migration behaviour but also on the reporting behaviour of birds as a whole (https://www.ornitho.de/; 18. March 2021)
2 The Research Project „Artenvielfalt erleben“
We have taken up these considerations as part of the research project “Experiencing biodiversity. How nature research on your own doorstep benefits from interactive web maps” and translated into a practical solution proposal Footnote 1. The research project aimed to make the collaboratively collected data of the bird observation portal ornitho.de accessible to the public with the help of web maps and by taking data interpretation into account. Ornitho.de is Germany’s largest platform for reporting bird observations. Since 2011, 32,000 registered users have reported their observations to the portal, which reached the 50 million entry mark in 2020. For the general public, the data has so far only been displayed on static maps of Germany. Only particularly active observers had access to the database for their own queries and analyses.
Based on this initial situation, the two primary project goals were (1) to involve the community of data-collecting Citizen Scientists in the sense of a comprehensive definition of Citizen Science (see above) in the further development of ornitho.de by developing interactive web maps. On this basis (2) to design a web map together with the ornithologists that makes the huge data treasure available to a broad public in the form of visual and scalable evaluations, the “Ornitho-Regioportal”. Both regionalization and visualization are intended to motivate Citizen Scientists to report more regularly and thoughtfully, and in particular to prepare observation lists (Moser et al. 2020).
To achieve these goals, we implemented several elements of a user-centered design approach (e.g., Roth et al. 2015). For example, we actively involved the ornitho community in the development in several workshops (Moser et al. 2020). In addition to content-related and functional requirements for a web map, we discussed different visualization variants together. Thereby, it was necessary to balance between the participants’ experiential knowledge, i.e., known forms of cartographic visualizations, the cartographic-technical demands, but also the most diverse protection demands on the part of the participants as well as on the part of nature conservation (see e.g., Moser 2018). The result were different conditions that had to be considered in the design of the visualization solutions. Based on the previous experience of the Citizen Scientists with raster maps and the necessary protection of the data of both the reporters and sensitive bird species, it was already decided at the first workshop that the data would be presented as a raster representation, which, however, should have a much better resolution than previous services.
Furthermore, accompanying research was conceived to support the development of the web maps with regard to usability. In contrast to basic research, the research and investigation questions were strongly related to the respective project phase and also depended on the developments in the project. The goal was to use empirical research to find solutions for certain parts of the visualizations that could best be understood and interpreted by the target group, the large group of dedicated birdwatchers.
3 Visualization Options of CS-Based Datasets
3.1 Cartographic Visualization of Data Gaps
Numerous topics in the context of Citizen Science have spatial aspects and generate georeferenced data accordingly. These can be locations of animal and plant species, information on the condition of water bodies or soil types, hotspots of environmental pollution or danger spots in road traffic. The data is collected via smartphone apps with directly recorded locations or via browser-based input masks that query the discovery or reporting locations. These are always data sets that, due to their collaborative creation, can be defined as neither complete nor certain.
The representation and consideration of uncertainty in the visualization of data has been intensively discussed in cartography for many years. Since any spatial datasets can have uncertainties, not only visualization possibilities are in the focus of research (e.g., Kinkeldey and Schiewe 2012) but also impacts on decision-making processes and trust in results if the (in)certainty of data is ignored (Zhang and Goodchild 2002; Kinkeldey et al. 2017). In addition to appropriate typologies, the development of visual methods for representing uncertainty has been a focus of interest (MacEachren et al. 2005; Kinkeldey and Schiewe 2012, among others). Digital and web applications, in particular, can promote the representation of uncertainty through a variety of visualization options. Approaches to uncertainty representation have been designed and also evaluated for various projects. According to Kinkeldey et al. (2014), five approaches to uncertainty visualization can be distinguished:
-
Explicit or implicit: uncertainties are represented directly or indirectly by combining several visualizations showing different possible outcomes.
-
Intrinsic or extrinsic: existing objects are, for example, changed in their color value to represent uncertainty or new objects are introduced on the map for this purpose
-
Visually integrated or separated: uncertainty representation may or may not be perceptually separated from the data
-
Simultaneous or side by side/consecutive: data and uncertainty representation are displayed together in one view or separately, side by side or consecutively
-
Static or dynamic: it is a static map or interactive or animated visualizations.
A crucial factor for further development was the easy readability and interpretability of the data also for population groups that are less intensively engaged with the data and its contents.
3.2 Examples from the Biodiversity Domain
Interactive or static web maps are often used to evaluate and visualize jointly collected spatial data. However, it can be observed that especially—but not only—projects organized by citizens continue to struggle to deal with the uncertainty of their data base and, above all, to make it transparent. This may be due to a lack of contact with visualization experts, a lack of programming skills, or even a lack of resources. On the other hand, we did not find a lack of interest in or disregard for the problems of uncertain data and their evaluation during our interviews. People who are intensively involved with the data are aware of the challenges of evaluation and therefore use maps only as one element of evaluation among several. However, this becomes critical with the claim and goal of also making the jointly collected data available to a broader public, as envisioned by the basic CS idea of joint work on scientific questions (Bonn et al. 2016). In this case, it can no longer be assumed that appropriate background knowledge is available or that visualizations are classified and critically scrutinized. At the same time, the designed visualizations should be as simple, understandable and intuitively usable as possible.
Biodiversity data are often presented either on a raster basis in the form of points (Fig. 3) or raster areas, on an administrative level such as counties (Fig. 4) or as point data of individual observations (Fig. 5). The type of presentation depends, among other things, on conservation interests, be it species protection or protection of the reporters. For this reason, the point-based and precisely localizable representation of individual observations is often avoided, especially in the case of highly endangered species, preferring a grid representation of various scales. The representation of data on the basis of administrative boundaries, i.e., in the form of an area cartogram, can be easily implemented, but in our opinion does not meet the objectives of such a map. Especially large administrative units distort the perception, especially if observations are only made in small areas, but are transferred to a larger area (cf. Fig. 4).
Section of a grid-based distribution map of the House Sparrow for the year 2020 (TK25). In addition to the information on whether the House Sparrow was observed and reported as a species in a grid cell, the colour indicates the number of reported observations. However, empty grid cells can hardly be interpreted, as the data situation there is completely unclear. The additional note above the map that the representation is based on the casual observations and not on systematically collected data is considered positive. This map display is only available for registered users (https://www.ornitho.de/; 18. March 2021)
Distribution map of the Blue Tit based on the administrative units from the project “Stunde der Gartenvögel”. The darker the individual districts are shown, the more birds were reported per garden. However, the map does not show how many gardens were observed per unit, nor what area of the total municipal area these gardens correspond to (https://www.nabu.de/tiere-und-pflanzen/aktionen-und-projekte/stunde-der-gartenvoegel/ergebnisse/15767.html; 6. November 2020)
Distribution map of the Long-tailed Tit as individual observation points (light blue dots) in the Senckenberg Species Database Main-Kinzig-Kreis. In the case of endangered species, this type of representation is not recommended, as, for example, nests can be recognised and plundered (https://rmo.senckenberg.de/search/search.php; 6. November 2020)
Other projects, such as the Atlas of Living Australia, offer extensive display options including colors and display methods that can be changed by the user. Figure 6, for example, shows two different animal species together, using different methods (point display, grid display). Such portals must be described as expert versions, since on the one hand scientific genus names are used and on the other hand the conditions, differences and possible interpretations of the display options are hardly explained.
Comparative representation of two species in the “Atlas of Living Australia”. Apart from the use of subject-specific species names, the map contains no indication of whether a comparison of these species is advisable or not, i.e., whether the selected species interact with each other, are dependent on each other or possibly displace each other. This form of map portal is particularly suitable for experts with the corresponding prior knowledge (https://spatial.ala.org.au/; 5. March 2021)
For some applications, heat maps are also used to visualize point clusters (Fig. 7) or interpolations. Like in the present example, areas requiring protection can be visually highlighted. However, for each heat map and especially for collaboratively collected data, the underlying database should be known and verifiable. Other graphical representations of spatial data are also used, depending on the application. However, most of these representations are difficult to read and can only be interpreted reliably with extensive prior knowledge.
Forest nature conservation in the Spessart: “Heatmap” of the focal points “old forests” and species. Yellow to orange-red areas show the highest density of forest bird species, supported here by the dotted representation of observation data. In this case, areas with a high need for protection become visible https://www.lbv.de/naturschutz/standpunkte/wald-und-forstwirtschaft/spessart/; 5. March 2021)
All exemplary maps visualize observation data of voluntarily engaged Citizen Scientists. The locations as well as the density of findings depend significantly on the activity of the participants, both in the sense of the spatial and temporal distribution of observation data as well as in the sense of the content level. In addition to the factors that flora observation objects usually move around and that observations vary in intensity spatially and temporally due to their voluntary nature, it must also be considered that reports can only be submitted for species that can be recognized and named. Even though there are now numerous identification apps, observation lists on which, for example, all detected bird species/birds in a certain area during a certain time are recorded are rather rare. Such data differs from standardized measurements in that locations without reports do not provide any information about whether the respective species does not occur there, or whether no observers were active there. This fact has to be taken into account when interpreting the maps. In order to differentiate between data gaps and the absence of certain species in an area, it is helpful (and a task of cartographic design) to be able to include information about the activity areas of the reporters.
3.3 Support of Data Interpretation in the Ornitho-Regioportal
The ornithologists involved in the development process (see 2) discussed various cartographic elements that can support the interpretation of the data. In addition to the essential innovation of displaying the reporting activity (see 3.4), several switchable background layers were integrated. In addition to standard maps such as OSM and satellite imagery, layers on land cover, precipitation, air temperature and relief, but also administrative boundaries, water bodies and bird sanctuaries were integrated. These layers contain information, either individually or in combination, which enables trained ornithologists in particular to better classify the observation data displayed.
Furthermore, there were detailed discussions about the display of further contents of the observation data as well as about further display methods. The main focus was on the presentation of a reporter’s own observations. Proposals were developed for this, but these were initially put on hold in order to design a functional portal for the broad interested public within the framework of the project. For the participating community in particular, however, the precise presentation of one's own reports has a great added value: on the one hand, for the motivation to recognize one’s own contribution, but on the other hand, also with regard to the quality of data collection, in order to be able to assess and reflect on the value of one's own reports. In addition, the observation data offer a variety of possibilities for cartographic presentation. These include information such as the number of different species per grid, the number of observers per grid, the maximum number of individuals per grid. Such information can support interpretations, but it can also lead to misleading conclusions if used incorrectly, as randomness, multiple observations by different people and other effects have to be taken into account. Before implementation, an evaluation must be carried out to determine the extent to which the presentation of such content may also affect conservation interests, e.g., if the number of species and/or observers can implicitly be used to draw conclusions about protected species. For a similar reason, other methods of representation such as heat maps or interpolations were not used. The former was considered for the representation of species density and implemented on a test basis. However, due to the unsystematic collection of the underlying data, we had to conclude that the heat map showed less “hotspots of species diversity” than “hotspots of observation intensity”, since in areas with a high observation density, more species are apparently reported overall. While such a presentation also brings new insights, especially for experts; in the context of a portal for the general public, however, it must be weighed more carefully which impressions the map's image causes, which in most cases are difficult to catch and classify through additional explanations.
3.4 Visualization of Reporting Activity in the Ornitho-Regioportal
The innovation in the development of the map-based analysis platform within the framework of the research project lies in the implementation of a map layer that depicts the reporting activity of the participants. This reporting activity represents all reports of all types over a certain period of time on a grid basis (for all scale levels TK50 to half-minute fields). It thus provides information about the site-specific activity of the reporters, but not about the occurrence of a species. Reporting activity is represented by transparent shades of blue (Fig. 8). The occurrence of the selected species (Kingfisher in the example) is indicated by the red dots also grid-based.
From the combination of the occurrence and reporting activity layers, the reliability of the display of the species occurrence in a grid field can now be concluded. If the reporting activity is high (dark blue grid field), it is very likely that the actual occurrence of the selected species in this grid field and in the given period is shown. In the case of low reporting activity (light blue grid), on the other hand, a missing red dot may mean that the selected species is not present there or that the selected species occurs there but has not yet been reported due to the small number of observers. This representation identifies data gaps in unsystematic surveys. This should enable users to reflect on the fact of selective data collection when interpreting the maps and to indicate “white spots” on the map accordingly (cf. Fig. 9).
Reporting activity as a tool for interpretation: Distribution of Robin observations in the south-west of Münster for the years 2018 to 2020 based on the half-minute fields. From the upper map, it could be concluded that Robins mainly occur in settlement areas. By adding the reporting activity, it becomes clear that the birds mainly “occur” where there are also many observation reports. In this way, the reporting activity can contribute to a better interpretation of the distribution pattern shown
One advantage of the grid-based representation of the reporting activity and the bird data is that, in addition to providing the information, it also takes data protection into account. For example, it would have been inconceivable to display the precise data of the observations together with the time stamp, as this would have made it possible to analyze movement data and possibly even draw conclusions about individual Citizen Scientists. The protection of rare or sensitive bird species can also be guaranteed far better with a grid-based representation than is possible with other, mainly point-based forms of visualization. For example, the level of detail can also be varied depending on the protection status of bird species by limiting the zoomability. In addition, the point-based representation would suggest an accuracy that does not apply to migrating animals.
The depiction of reporting activity as the “sum of all observations of all bird species in the relevant grid field in the selected time period” (Moser et al. 2020: 35) is one way to support users in interpreting observation data. With regard to the five approaches to visualizing uncertainty by Kinkeldey et al. (2014), the representation of reporting activity can be classified as explicit and extrinsic, as uncertainty is directly represented in the form of the new map layer “Meldeaktivität”. Furthermore, it is visually separated from the occurrence information and can be shown and hidden simultaneously and interactively.
4 Empirical Evaluation of the Presentation of Data Gaps
4.1 Gradation of Reporting Activity
In the actual implementation of the representation of reporting activity, the question arose as to a meaningful number of gradations. Since there has been little experience with similar types of representations, this question could not be answered based on prior knowledge or literature, so we conducted empirical studies on this issue. This experiment investigated whether a three-level or five-level gradation of reporting activity is better suited to help the relevant target group interpret the dissemination data. We differentiated between individuals registered with ornitho.de who report less than ten observations on a monthly average and individuals who report ten or more observations on a monthly average and thus also have access to the database search. The background of this distinction was our assumption that persons with more reports are also more experienced in the use of ornithological data and their visualization. That experience has an influence on the use of available information has been shown in numerous studies (e.g., Egan and Schwartz 1979; Soloway and Ehrlich 1984). Thus, it was shown that experts coped worse with more supporting information, while less experienced persons benefited from supporting information (“Expertise Reversal Effect”; Kalyuga 2007). With regard to the abundance of information, on the other hand, it could be shown that experts are better able to organize this information and to integrate it into a coherent overall picture, so that they were able to cope with more information than persons with less prior knowledge (Chase and Simon 1973). Since the five-level classification of reporting activity provides additional information but no further support, we hypothesized that more experienced users would cope better with the five-level version than less experienced users. The latter should perform better with the reduced three-level representation than with the five-level representation. We tested the extent to which this works from the user's perspective and which classification provides the best support in two studies.
4.1.1 Methods
The two representations differed only in the number of gradations of reporting activity. In the five-level representation, the levels consisted of “none (0 reports),” “low (1–40 reports),” “moderate (> 41 reports),” “high (201–1000 reports)” and “very high (> 1000 reports)”. The three-level map did not distinguish between the last three levels, so “moderate” included all values greater than 40. The level thresholds were determined by comparing empirical data from ornitho.de with the Breeding Bird Atlas (“Brutvogelatlas” Gedeon et al. 2014) by calculating how many reports are needed to record a certain proportion of bird species on average (for derivation of the class thresholds, see Moser et al. 2020: 36 f.). Quartiles were defined as threshold limits for the five-level map. For TK25 quartiles, values were obtained that, when rounded, resulted in the above-mentioned limits.
The participants were given two different tasks. In the first task, they were asked to evaluate the truth of statements on the basis of the maps. They had four different answer options: “true,” “not true,” “information not available on map,” and “don't know” (example in Fig. 10). All participants were presented with three maps, each with five statements, which they were asked to rate. In a second task, three grid fields were to be marked on each of two maps, in which a given bird species could most likely be observed or not observed. The corresponding information was to be read from the maps (see Fig. 11).
A total of N = 173 individuals (128 male, 45 female) with an average age of M = 51.78 years (range: 18 a–83 a) participated in the study, two-thirds of whom reported having at least a college degree. 84 individuals contributed an average of less than 10 reports per month and were classified as low activity, while 89 individuals with at least 10 reports per month were classified as very active. One half of each group was presented with the three-level maps and the other half of the groups with the five-level maps, so that both presentation variants could be tested with both more active and less active individuals.
4.1.2 Results
An analysis of the proportion of correct responses (Table 1) showed that neither membership in one of the user groups (\({\chi }^{2}\) (1) = 0.03; p = 0.862) nor presentation (\({\chi }^{2}\) (1) = 0.95; p = 0.331) had an isolated influence. We also found no evidence of an interaction between these two factors, \({\chi }^{2}\) (1) = 0.47; p = 0.495. Thus, both variants turned out to be equally suitable for mapping and interpreting bird species dispersal and reporting activity for both user groups. However, we found an unexpected difference between the statements. Participants had greater difficulty with statements when information relevant to their response was not mapped (M = 38.4% correct responses, SD = 25.2%). In comparison, statements that could be unambiguously assessed as “true” or “not true” resulted in M = 77.9% correct responses (SD = 20.2%), \({\chi }^{2}\) (1) = 9.86; p = 0.002). It should be noted here that participants were not asked to know information that was not depicted but were simply asked to indicate when the required information was not depicted.
In the second task, in which grid fields were to be marked, there were also no differences between the gradationsFootnote 2 (both |z|≤ 0.81; both p ≥ 0.420). Also, more active participants did not perform better than less active participants (both |z|≤ 1.02; both p ≥ 0.307). Similarly, there was no interaction between user group and representation (both |z|≤ 0.42; both p ≥ 0.673). In the subjective evaluation of the tasks in terms of mental and temporal demand, performance, effort and frustration (recorded in the NASA TLXFootnote 3; Hart & Staveland 1988), there were also no influences of representation and user group (all F ≤ 1.50; all p ≥ 0.218).
4.1.3 Interpreting
The study made clear that both representations were equally difficult to interpret and that there was no difference between more active and less active reporters. Thus, the hypothesis of dependence between activity and number of gradations could be rejected. Since the representation of the reporting activity can basically be seen as supporting information for the interpretation, it could be that the active participants did not need this information and therefore did not pay attention to it. This could be a reason why they did not benefit from the differentiated representation. On the other hand, the proportion of correct answers was rather low in both groups. Thus, it remains open why there were no differences between more active and less active Citizen Scientists and why both representations seem to be equally suitable.
The only difference was that the answer “information not available on map” was more difficult than the specific answers “true” or “not true”. For individual statements, the rate of correct answers dropped to about 10%, which was even below guessing level. In a second study, we therefore tested the hypothesis that additional instructions explaining the representation could facilitate the processing of the tasks. We selected a video tutorial as the format of the instructions. On the one hand, support services should be offered in a low-threshold and entertaining way due to the open character of the online platform. On the other hand, against the background of research, we assumed that with the help of a videotutorial comprehension could be promoted (Brecht 2012; van der Meij and van der Meij 2014). In the study, we systematically varied whether or not participants were shown a tutorial prior to completing the task.
4.2 Video Tutorial Support
4.2.1 Methods
The tutorial was implemented as a video. It lasted 3:24 min and consisted of a screen recording with an accompanying explanatory audio track. The tutorial focused on species occurrence, i.e., whether a particular bird species was reported in the specified time period and reporting activity, i.e., how often bird observations were generally reported in this area in the specified time period. Both the presentation of the information and its meaning were explained. Participants were also informed that a missing dark red dot marking the presence of a bird species in a grid field does not automatically indicate that the bird species under consideration was not present there during the relevant period.
The tasks were identical to those previously completed by the participants in the representation study. A total of N = 90 individuals (71 male, 18 female, 1 no information) with a mean age of M = 52.52 years (range: 21a–80 a) participated in this second study. Also in this group, over 60 percent of respondents reported having at least a college degree. Since based on the results of the representation study the average reporting frequency had no influence on the correctness of the responses, we decided not to differentiate between more active and less active ornitho reporters. For reasons of practicability, we therefore only invited more active ornitho reporters to participate.
4.2.2 Results
Similar to the representation study, the assessment of the 15 statements showed no difference in correctness between the group with a tutorial and the group without a tutorial \({\chi }^{2}\) (1) = 0.29; p = 0.590 (Table 2). As in the representation study, there was a difference in the proportion of correct responses between questions that could be answered unambiguously with “True” or “Not true” and those for which the relevant information was not presented, so that “Information not available on map” was the correct response option. The former were answered correctly in M = 80.0% (SD = 17.8%) of cases, the latter only in M = 40.4% (SD = 24.3%) of cases.
However, in the first map of the second task (see Fig. 11), in which the participants were asked to mark grid squares on the map themselves, there was an effect of the tutorial, \({\chi }^{2}\) (1) = 5.69; p = 0.017. The group that had previously seen the tutorial marked grid squares more often correctly than the group without tutorial (see Table 2). However, this difference was no longer significantly pronounced in the second map of this type, which asked to mark the grid patches where a particular bird species was most likely to be found, \({\chi }^{2}\) (1) = 0.43; p = 0.512, although in both groups the task was solved correctly very frequently, with over 90% correct answers. Another result that suggests at least a small advantage of the group with tutorial are the data on subjective stress, collected by the NASA-TLX (Table 3). Thus, the participants with tutorial indicated a significantly lower mental demand (F(1, 80) = 6.14; p = 0.015) and a lower necessary effort (F(1, 74) = 4.42; p = 0.039) than the participants* without tutorial, while time demand (F(1, 84) = 3.93; p = 0.051), estimation on own performance (F(1, 79) = 1.47; p = 0.230) and frustration (F(1, 79) = 1.66; p = 0.201) were reported at a comparable level. Thus, in summary, the tutorial showed an advantage on the first map of Task 2 and partly on the subjective evaluation of the task, whereas on answering the questions of Task 1 and part of the subjective evaluation, the tutorial did not make a difference.
4.2.3 Interpreting
In this second study, the difficulties associated with the correct response “Information not available on map” were also evident. Obviously, the subjects found it particularly difficult to identify where the limits of validity of the mapped data lay. Contrary to our hypothesis, we were not able to significantly reduce these difficulties even with the tutorial, although care was taken to explain all the information presented. Therefore, in future tutorials and supporting materials, special care should be taken to explain not only the content, but also where the limits of validity of the data depicted lie.
4.3 Discussion
In order to make the visualization as comprehensible as possible for many target groups, the gradation by means of different numbers of levels and a tutorial as a support service were empirically investigated. The first study examined the influence of the number of gradation levels used to visualize the data density on comprehension: no measurable effect was found. Rather, participants had difficulty recognizing when the information presented was insufficient to make a specific point. This difficulty could not be remedied by additional instruction in the form of a video tutorial, as we were able to show in the second study. This finding could be an artifact of the study situation. Since participants were presented with a map for each statement, their expectation might have been that the information presented on the map was also suitable for answering the question. This could be counteracted by explicitly pointing out before the start of the tasks that not all required information is always available. On the other hand, this finding could show how difficult it is for users to recognize the limits of the validity of the information presented. This should be taken into account when data is made available in the context of Citizen Science projects or when Citizen Scientists are involved in the evaluation and interpretation of data. Thus, explicit examples could be used to clarify which statements can or cannot be made based on the available and presented data. In the tutorial study, we were able to show that an interpretation aid in the form of a video improved performance on at least one of the tasks. Respondents also indicated that they found the tasks less mentally demanding and that they had to exert less effort. These results provide at least some indication that additional interpretive aids can reduce the difficulty of understanding maps that combine multiple layers of information.
The results of the representation study also indicate that the activity of the reporters had no influence on whether the maps could be correctly assessed. However, this also means that the active participation of a person in reporting observations does not indicate their expertise in interpreting the observation data. This insight is also important for coordinators of Citizen Science projects when it comes to evaluating contributions in terms of their reliability. Hence, a good reporting performance is not the same as a good performance in evaluation and interpretation, which is why separate quality and review criteria should also be developed for evaluation and interpretation. Similar to already available trainings for reporters, trainings for evaluators should be offered as well. Due to the novelty of our map presentation, it remains to be seen and to be investigated in further studies whether a more intensive engagement with the maps will build up an expertise among those Citizen Scientists who are more engaged with the data provided. In any case, we were unable to show that more active reporting behavior automatically led to a more correct interpretation of the dispersal data.
Another aspect of reporting activity representation that has not been addressed here so far is the highlighting of “under-monitored” areas. This creates transparency so that Citizen Scientists can decide for themselves, based on this data, when to monitor and where. In this way, this information could also contribute to making under-observed areas more interesting for observers and thus also ensure a more even coverage of the area with observations and, above all, with reports. This more even coverage would allow for spatial comparisons and interpretation of changes in a species population over longer periods of time. Since the two studies reported here did not investigate reporting behavior, but rather the interpretation of the data presented, no statement can be made in the context of this paper as to whether this additional layer of information can also influence observation and reporting behavior. For this, further studies are needed that observe the reporting behavior and ask Citizen Scientists about the criteria on which their selection of observation areas is based.
5 Conclusion
This article emphasizes in the first part the importance of presenting not only dissemination data but also reporting activity in the context of Citizen Science projects. The presentation of the data basis is particularly important for a correct interpretation, since, on the one hand, the data collection by Citizen Scientists is often spatially unsystematic and, on the other hand, Citizen Scientists do not necessarily have an academic-scientific training regarding the handling of data. In the paper, possibilities of visualizing the data density were discussed and the approach used in the project “Artenvielfalt erleben” was presented, including two empirical studies.
This article shows that the publication and visualization of data collected in Citizen Science projects alone is not sufficient to generate surplus value for the Citizen Scientists involved and to bring science closer to them. Rather, it shows that efforts must be made to adequately present and explain the data base. Thus, it is especially important to communicate where the limits of the validity of the collected data lie. It is important to convey to interested persons that a missing presence report (in our example the missing red dot in a grid field on the map) does not mean that the bird species is not common there, but it can just as well be that there were simply no reporters around who could have observed the bird species. This interpretation can be facilitated in the case of dissemination maps by the additional representation of the reporting activity. In this layer all observations of all species are summarized. Similar representations could provide additional value in other subject areas as well, and whenever large data sets with many observations are available and when the full information on data density is not apparent from the data presented itself. Based on the way, the reporting activity is obtained, it is also clear that this approach can only be used when multiple pieces of information are collected, such as the distribution of all bird species in our example, which when combined can provide an indication of data density. Thus, the approach presented here is most suitable for projects that have large amounts of data and whose analysis usually only considers individual subcategories of these data, such as the distribution of individual species.
In summary, the involvement of Citizen Scientists beyond the collection of data, as is often called for (e.g., ESCA 2015; Bonn et al. 2016), requires additional support services in the form of interpretation aids and expanded representation options. The unsystematic collection of data in particular means that they are not self-explanatory and can be used by anyone at any time. Using the example of the dissemination of bird species, a map-based visualization option was presented, and its comprehensibility was examined in two empirical studies. It turned out that the participants found it particularly difficult to recognize the limits of the validity of the map representations. For Citizen Science projects this means that help for the interpretation of the data should be offered in order to enable the participants to examine the data independently with regard to their own questions and to process the results accordingly. Hence, it seems worthwhile for similar projects to take up the approach presented here for the visualization of data gaps and to pay even more attention to the development of suitable interpretation aids.
Availability of Data and Material
The datasets and analysis scripts supporting the conclusions of this article are available in the Zenodo repository (https://doi.org/10.5281/zenodo.5159305; https://doi.org/10.5281/zenodo.5159312).
Code Availability
The program code is available in the GitHub repository (https://github.com/ifl-geovis/ornitho-Regioportal).
Notes
The project was carried out in cooperation with the Leibniz Institute for Regional Geography, the Leibniz Institute for Knowledge Media and the Dachverband Deutscher Avifaunisten (umbrella organisation of German avifaunists) from January 2018 to December 2020 and was funded by the Federal Ministry of Education and Research in the priority area of Citizen Science.
The two maps in the second task were evaluated in separate logistic regressions.
The NASA Task Load Index (NASA-TLX; Hart & Staveland, 1988) is a widely used, subjective, multidimensional assessment instrument to capture perceived task workload.
References
Bonn A, Richter A, Vohland K, Pettibone L, Brandt M, Feldmann R, Goebel C, Grefe C, Hecker S, Hennen L, Hofer H, Kiefer S, Klotz S, Kluttig T, Krause J, Küsel K, Liedtke C, Mahla A, Neumeier V, Premke-Kraus M, Rillig MC, Röller O, Schäffler L, Schmalzbauer B, Schneidewind U, Schumann A, Settele J, Tochtermann K, Tockner K, Vogel J, Volkmann W, von Unger H, Walter D, Weisskopf M, Wirth C, Witt T, Wolst D, Ziegler D (2016) Grünbuch Citizen Science Strategie 2020 für Deutschland. Helmholtz-Zentrum für Umweltforschung (UFZ), Deutsches Zentrum für integrative Biodiversitätsforschung (iDiv) Halle-Jena-Leipzig, Leipzig, Museum für Naturkunde Berlin, Leibniz-Institut für Evolutions- und Biodiversitätsforschung (MfN), Berlin-Brandenburgisches Institut für Biodiversitätsforschung (BBIB), Berlin
Brecht HD (2012) Learning from online video lectures. J Inf Technol Educ 11:227–250
Chase WG, Simon HA (1973) Perception in chess. Cogn Psychol 4:55–81
European Citizen Science Association ECSA (2015) Zehn Prinzipien von Citizen Science. https://ecsa.citizen-science.net/wpcontent/uploads/2020/02/ecsa_ten_principles_of_cs_german.pdf. Accessed 5 Nov 2020
Egan DE, Schwartz BJ (1979) Chunking in recall of symbolic drawings. Mem Cognit 7:149–158
Gedeon K, Grüneberg C, Mitschke A, Sudfeldt C, Eikhorst W, Fischer S, Flade M, Frick S, Geiersberger I, Koop B, Kramer M, Krüger T, Roth N, Ryslavy T, Stübing S, Sudmann SR, Steffens R, Vökler F, Witt K (2014) Atlas Deutscher Brutvogelarten. Atlas of German Breeding Birds. Stiftung Vogelmonitoring Deutschland und Dachverband Deutscher Avifaunisten, Münster
Hart SG, Staveland LE (1988) Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock PA, Meshkati N (eds) Human mental workload. North-Holland, Amsterdam, pp 139–183
Jacobs C, Resch B (2013) Semi-automatisierte Plausibilitätsprüfung in citizen science gestützten Naturbeobachtungen. In: Strobl J, Blaschke T, Griesebner G, Zagel B (eds) Angewandte Geoinformatik. Wichmann Verlag, Heidelberg, pp 350–355
Kalyuga S (2007) Expertise reversal effect and its implications for learner-tailored instruction. Educ Psychol Rev 19:509–539
Kinkeldey C, Schiewe J (2012) Visualisierung thematischer Unsicherheiten mit “Noise Annotation Lines.” KN Kartogr Nachr 62:241–249
Kinkeldey C, MacEachren AM, Schiewe J (2014) How to assess visual communication of uncertainty? A systematic review of geospatial uncertainty visualisation user studies. Cartogr J 51:372–386
Kinkeldey C, MacEachren AM, Riveiro M, Schiewe J (2017) Evaluating the effect of visually represented geodata uncertainty on decision-making. Systematic review, lessons learned, and recommendations. Cartogr Geogr Inf Sci 44:1–21
Lave R (2012) Neoliberalism and the production of environmental knowledge. Environ Soc 3:19–38
MacEachren AM, Robinson A, Hopper S, Gardner S, Murray R, Gahegan M, Hetzler E (2005) Visualizing geospatial information uncertainty. What we know and what we need to know. Cartogr Geogr Inf Sci 32:139–160
Moser J (2018) Neogeographie - Über Chancen und Herausforderungen für die kartographische Forschung. KN Kartogr Nachr 68:113–119
Moser J, Wahl J, Schwan S, Hoyer T, Moritz J, König C, Geidel S (2020) Webbasierte Visualisierung raumbezogener Daten aus Citizen-Science-Projekten. Partizipation, Potentiale und Lösungen am Beispiel des ornitho-Regioportals. Leibniz-Institut für Länderkunde, Leipzig (Forum IfL, Heft 38)
Pettibone L, Vohland K, Ziegler D (2017) Understanding the (inter)disciplinary and institutional diversity of citizen science: a survey of current practice in Germany and Austria. PLoS ONE 12(6):e0178778. https://doi.org/10.1371/journal.pone.0178778
Ratnieks FLW, Schrell F, Sheppard RC, Brown E, Bristow OE, Garbuzov M (2016) Data reliability in citizen science: learning curve and the effects of training method, volunteer background and experience on identification accuracy of insects visiting ivy flowers. Methods Ecol Evol 7:1226–1235. https://doi.org/10.1111/2041-210X.12581
Roth R, Ross K, MacEachren A (2015) User-centered design for interactive maps. A case study in crime analysis. IJGI 4(1):262–301
Soloway E, Ehrlich K (1984) Empirical studies of programming knowledge. IEEE Trans Softw Eng 5:595–609
Van der Meij H, Van Der Meij J (2014) A comparison of paper-based and video tutorials for software learning. Comput Educ 78:150–159
Wahl J, Dannenburg F, Kühn M, König C (2014) ornitho.de - nix Neues an der Küste? Corax 22. Sonderheft 1:9–12
Zhang J, Goodchild MF (2002) Uncertainty in geographical information, 1st edn. CRC Press, London
Acknowledgements
The close cooperation of the participating project partners played a major role for the results, for which we would like to thank especially Johannes Wahl and Christopher König (both DDA), Sebastian Geidel (IfL) and Stephan Schwan (IWM), as well as our assistants Krzysztof Luzar, Robert Schulze, Lena Stephan, Frauke Grieble, Florian Holl and our intern Lisa Sänger. A special thanks goes to all citizen scientists of the ornitho-community for conceptual support, important feedback and participation in the studies.
Funding
Open Access funding enabled and organized by Projekt DEAL. The project has been funded by the German Federal Ministry of Education and Research (BMBF), funding area citizen science, funding code: 01BF1713A-C (01/2018–12/2020).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.
Ethical approval
The plan of the empirical studies had been reviewed by the local ethics board of the Leibniz-Institut für Wissensmedien before data collection and was classified as unproblematic (LEK 2019/024).
Consent to participate
All participants provided written informed consent to participate in the studies prior to testing.
Consent for publication
All participants gave written informed consent that data can be published after data anonymization.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Hoyer, T., Moritz, J. & Moser, J. Visualization and Perception of Data Gaps in the Context of Citizen Science Projects. KN J. Cartogr. Geogr. Inf. 71, 155–172 (2021). https://doi.org/10.1007/s42489-021-00083-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42489-021-00083-2