In September 2015, the United Nations Sustainable Development Summit adopted an international framework to guide global development efforts, entitled ‘Transforming our world: the 2030 Agenda for sustainable development’ (UN 2015). The Agenda includes 17 Sustainable Development Goals (SDGs) and 169 targets relating to global challenges including poverty, inequality, climate, environmental degradation, prosperity, and peace and justice. The UN General Assembly tasked a group of technical and statistical experts with developing a global monitoring framework that would allow the tracking of each SDG target, while at the same time keeping in mind the feasibility and reporting burden of such a monitoring framework. This led to the creation and adoption of the current list of 244 SDG indicators by the UN General Assembly (UN 2017).
The SDG indicator framework is developed based on the existence of a global methodology and data availability. Each SDG indicator was placed into one of three tiers to track progress towards operationalizing the indicator framework. A simplification of the tiering framework is: Tier I: existence of an agreed methodology and good data coverage (at least 50%); Tier II: agreed methodology, but data are lacking (less than 50% data coverage), and Tier III: no established methodology (UN n.d.). The monitoring of the SDGs is expected to happen at the sub-national, national, regional, and global level to ensure the vertical coherence of policies and actions. In terms of official reporting, national governments have the primary responsibility for monitoring the SDG indicators (UN 2017). Each SDG indicator has one or more custodians, who are responsible for (i) developing the global methodology based on the best available science, research, and data expertise, and (ii) identifying the data sources that can contribute to the SDG indicator. Custodians are typically a UN agency or other international organization, who additionally ensure that the methodology is applicable to all countries, regardless of their level of development. National data are expected to be used to populate the official reports of countries on SDG progress (known as Voluntary National Reviews on the SDGs). National data then feed into the UN global database on the SDGs, which is used to analyze regional and global progress, including through an annual report by the Secretary General on the SDGs (UN 2019a). The custodian agency is also responsible for adjusting the data when required to ensure comparability, and for developing techniques to fill data gaps. Since the adoption of the SDG indicator framework, there has been an effort to upscale statistical methodological work and data collection across the SDG indicators. For example, of the SDG indicators that are not related to the environment, only 7% remained Tier III, compared to the 28% that were Tier III at the adoption of the SDG indicators. On the other hand, 26% of the environment-related SDG indicators remained Tier III as of July 2019 (UN 2019b, c, n.d.), and a further 32% have insufficient data available for global tracking (UN Environment 2019).
The funding required to measure all 244 SDG indicators using only traditional data sources creates a barrier for countries to monitor and assess progress toward the SDGs, particularly for developing countries (IAEG Secretariat 2014). Effective and efficient monitoring of the SDGs will require better utilization of new data sources and data techniques (Daguitan et al. 2019a, b). For example, the role of Earth Observation (EO) in addressing societal challenges and supporting the implementation of the 2030 Agenda has been long recognized by governments, industry, and scientific institutions (GEO 2017). EO is a source of data that could help to harmonize information on natural resources, ecosystems, and environmental issues via remote sensing technologies and in-situ measurements (OECD 2017). A number of studies have been conducted by the EO community that aim to define the concrete contributions of EO data in measuring progress towards the SDGs (GEO 2017; CEOS 2018). One such study was jointly conducted, for example, by the Group on Earth Observation (GEO), which is an intergovernmental partnership that improves the availability, access, and use of EO data (GEO 2018), and the Committee on Earth Observation Satellites (CEOS), which is an international consortium involved in the management of international and civil space-borne missions for observing the Earth (CEOS 2015). Their analysis revealed that 29 SDG indicators could be monitored using EO data, and EO could provide support for the achievement of around 71 targets (CEOS 2018).
Official statistics have typically been based on data that are officially collected by national governments (e.g., through surveys, censuses, official sensors) (Daguitan et al. 2019a, b). To date, citizen science methodologies and data are not included in SDG data acquisition. However, doing so may provide data at finer spatial and temporal scales than would otherwise be possible to obtain since data from citizen science are often collected at higher frequencies than the traditional sources of data used as inputs to the SDG indicators, as well as in a spatially disaggregated way. Citizen science, as a concept, has diverse definitions, terms, and interpretations, where no single term or definition is suitable for all contexts (Eitzel et al. 2017). In the scope of our work, we take an all-encompassing approach and define citizen science using three main characteristics: “public participation”, “voluntary contributions”, and “knowledge production” (SDSN TReNDS 2019). Our use of the term citizen science most closely aligns with the concept of “Public Participation in Scientific Research (PPSR)”, which Shirk et al. (2012) describe as “intentional collaborations in which members of the public engage in the process of research to generate new science-based knowledge”. PPSR encompasses projects with five degrees of participation. This includes: (i) contractual projects (where communities approach researchers to conduct a scientific investigation on a particular issue); (ii) contributory projects (usually designed by scientists, and volunteers are involved mainly to contribute data); (iii) collaborative projects (usually designed by scientists and volunteers are involved to contribute data, but also support project design, data analysis, and/or dissemination of results); (iv) co-created projects (designed by scientists and volunteers together, and volunteers are actively involved in most or all aspects of the research) and (v) collegial projects (volunteers conduct research independently expecting different degrees of recognition by professionals) (Shirk et al. 2012). As with PPSR, we have elected to include a wide range of diverse projects from hypothesis-driven science to practices that involve local knowledge and observations for addressing the political and social issues that communities face. Our broad interpretation of citizen science includes any initiative that produces scientific knowledge through the participation of volunteers, such as community-based monitoring (Conrad and Hilchey 2011), community-based participatory research (Asaba and Suarez-Balcazar 2018), participatory action research (MacDonald 2012), citizen-generated data (Datashift 2017), crowdsourcing (Howe 2006; Nov et al. 2010), volunteered geographic information (Sieber and Haklay 2015), and participatory sensing (Coulson et al. 2018), among others.
In the past, there has been research that has proposed the value of including citizen science data in the SDG reporting process (IAEG Secretariat 2014; Flückiger and Seth 2016; Fritz et al. 2019). However, there is currently a lack of comprehensive research that provides systematic evidence regarding where citizen science currently contributes or where it could potentially contribute to the SDGs at an indicator level. The most relevant piece of work to date was undertaken by West and Pateman (2017), who outlined how citizen science approaches could contribute to the definition, monitoring, and/or implementation of the SDGs. In total, they identified 42 of the 169 targets to which citizen science could contribute, but they did not consider the indicator level, nor did they present examples of initiatives to support these suggestions based on a systematic review. Another study by the European Commission, which undertook an inventory of citizen science projects for environmental policies, focused on the goal level (Bio Innovation Service 2018). The results showed that the 503 environmental citizen science initiatives included in the inventory could contribute to all 17 goals, directly or indirectly. In the scope of their work, direct contribution means that the “project aim fits an SDG”, while indirect contribution describes projects that “may contribute to fulfilling an SDG, as a by-product of its activities”. For example, SDG 4 Quality Education, SDG 9 Industry, Innovation and Infrastructure, and SDG 16 Peace, Justice and Strong Institutions are covered by all the projects listed in the inventory, at least indirectly. SDG 3 Good Health and Wellbeing, SDG 13 Climate Action, SDG 15 Life on Land, and SDG 17 Partnerships for the Goals were covered by the majority of the projects (78%, 86%, 75%, and 52%, respectively), both directly and indirectly. SDG 14 Life below Water and SDG 15 Life on Land had the highest direct contributions (18% and 58%, respectively). What is currently missing is an understanding of which SDG indicators citizen science is already contributing to and where its future potential lies. Hence, the aim of this paper is to provide a systematic review of how citizen science data generated by volunteers can provide data for the SDG indicators. This includes filling information gaps at the national level or complementing national level information through improved timeliness or reporting at a higher spatial resolution. As SDG reporting relies on national data compilation by national statistical systems, this paper additionally provides a discussion on the opportunities and challenges for bringing citizen science into the scope of national official statistics.
To systematically analyse which SDG indicators citizen science is already contributing to and where its potential lies in the future, the methodology consists of two main parts. The first part involved reviewing each SDG indicator to identify whether citizen science (i) is currently contributing data to an indicator, (ii) could contribute in the future, or (iii) has no foreseen contribution at present. The second part of the methodology involved summarizing the results of the indicator review in order to understand the current landscape of citizen science contributions to the SDGs.
Systematic review of the SDG indicators
The SDG indicator review process was comprised of a number of steps (Fig. 1). Step 1 was to compile a list of all SDG indicators (see Table S1 in the Supplementary Material for the complete list), which was downloaded from the UN Statistics Division website on 13 April 2019. Step 2 was to consult the metadata documents of the indicators that describe the methodology and data sources associated with each indicator (https://unstats.un.org/sdgs/metadata/). This provided us with an understanding of the current stage of methodological development of the indicators (i.e., Tier I, II or III), what the methodology for the indicator is (if it exists), what data are needed to calculate the indicator (if already decided), and the source of the input data. For cases where there was no metadata or work plan, which occurred for a few of the indicators, the analysis was a judgement based on the indicator title.
Step 3 involved identifying citizen science projects to provide a strategic overview on relevant citizen science initiatives that capture the breadth of different types of projects relevant to each indicator, which was completed using the following procedure:
Reviewed the Inventory of citizen science projects for environmental policies, which was undertaken as part of a study commissioned by the European Commission (Bio Innovation Service 2018), (https://data.jrc.ec.europa.eu/dataset/jrc-citsci-10004);
Queried SciStarter, an online database of citizen science projects from around the world, using respective indicator-relevant keywords (https://scistarter.org/);
Reviewed Zooniverse, an online platform that hosts over 100 virtual citizen science projects from around the world, by respective indicator-relevant topics (https://www.zooniverse.org/);
Queried the Google internet search engine, using keywords from respective indicators and combinations of ‘citizen science’, ‘community-based monitoring’, ‘crowdsourcing’, etc. (https://google.com);
Queried Scopus, an online database for peer-reviewed citations and abstracts, using the same keywords as those listed in the Google queries (https://www2.scopus.com/); and
Used contributions from the co-authors based on their extensive local, regional, and global knowledge of citizen science initiatives and projects.
As mentioned in the introduction, we use citizen science in a broad sense in this paper to include public participation, voluntary contributions and knowledge production (SDSN TReNDS 2019). Hence, the initiatives that fall within this definition were included, even if they do not explicitly use the term citizen science. Regarding public participation, our approach integrates the five models on degree of participation from Shirk et al. (2012), which includes, but is not limited to, projects where members of the public primarily contribute data (at one end of the spectrum) to designing the research together with professional scientists and active involvement in most or all phases of the research (at the other end of the spectrum) (Shirk et al. 2012). Regarding voluntary contributions, our experience is that the term voluntary has different meanings in different contexts. For example, in the context of community-based monitoring activities in the health domain, the terms Community Health Workers (CHWs) or Volunteers are widely used, and are defined as the first point of contact between communities and the health system, usually in low- or middle-income countries. CHWs are trained to provide a specific function, having no formal professional qualifications. They can be employed and salaried by government organizations, non-governmental organizations (NGOs), or perform entirely on a voluntary basis. They may receive different forms of incentives, ranging from uniforms or one-time financial incentives related to health insurance (Ormel et al. 2019). This makes it difficult to determine whether a community-based monitoring program involving CHWs is voluntary or whether they were paid salaries. In our mapping, we did not include projects where participants were paid salaries, but did include examples where participants received small incentives. We excluded those cases from our mapping where we could not be sure about the incentives provided to participants. Finally, in the scope of our study, knowledge production refers to investigation, monitoring, or scientific research as suggested by Eitzel et al. (2017).
Step 4 involved determining the category for the indicator from the following three choices:
already contributing, which means that data from at least one citizen science project are already used for reporting on a specific SDG indicator at the national or global level;
could contribute, where data from at least one citizen science project could be used for a specific SDG indicator, but are not used so far and the project and/or data requirements might need modification/adaptation before the resulting data can be used (provided this modification is feasible); and
no alignment at present.
When a project was found to relate to an indicator topic (step 3), it was then assigned to either “already contributing” to that indicator or “could contribute”. Assignment to the category “currently contributing” involved finding evidence in documents, websites, and in the literature. If the evidence was not present in documents, but a contribution to an indicator was thought to be the case, project staff, custodian agencies, or other organizations involved in reporting efforts for that particular indicator, such as Birdlife International, were consulted to provide the evidence for these contributions. These indicators were then further classified into “direct contributions” or “supplementary contributions”. Direct contributions mean that citizen science data are already contributing or could contribute to the calculation of the official SDG indicator. This would include indicators that could utilize citizen science data as the primary data source (e.g., bird species prevalence can be primarily based on citizen science data from bird watchers) or indicators where citizen science is part of the indicator or used to fill spatial or temporal gaps in data (e.g., air quality reference stations can be coupled with citizen science data on air quality). In the context of this paper, a supplementary contribution means that citizen science data could provide information that is useful to contextualize an SDG indicator or target. For example, the official SDG indicator on poverty aims to capture income poverty, but citizen science data covering the quality of housing or furnishings in homes (e.g., Dollar Street) provide supplementary information that can be used to better understand poverty. If no evidence of a direct contribution could be found but a project was thought to be producing the types of data or similar data to those required by the indicator, then the project was assigned to the category “could contribute”. We note that some of the projects that we identified as “could contribute” may not be collecting the exact data that are needed for the indicator. However, we added them to our mapping as they have great potential to support the indicator if they could be extended to collect the specific data required for the indicator (e.g., through modifications to their data collection protocol). As an example, the Engage TB and One Impact projects could provide supplementary data for indicator 3.3.2 Tuberculosis incidence per 100,000 population. Our suggestion is that, if designed accordingly, they or other similar projects could also support data collection for the HIV-related indicator 3.3.1 Number of new HIV infections per 1,000 uninfected population, by sex, age and key populations. Finally, if no projects were found or the indicator was based on data that were not amenable to collection by citizens, then that indicator was categorized as “no alignment at present”.
Note that we did not consider the robustness of the quality procedures used in the citizen science projects listed in Table S1. Instead, we only considered whether projects are already contributing (where high quality can be assumed) or where they could potentially contribute (where data quality protocols would require compliance if the data were to be used for SDG reporting in the future).
Once the full set of indicators was reviewed, then the first peer review process began in step 5. Each co-author was assigned a different set of indicators to those they initially reviewed to peer review the work of others. This involved reading the metadata for the indicator (step 2), reviewing the category chosen (during step 4), searching the five sources (step 3) and modifying the category of SDG contribution as appropriate. If a citizen science project was listed as contributing to an indicator, the evidence for this claim was reviewed. In the majority of cases, there was consensus. However, two situations of disagreement arose during this process: (i) the peer reviewer identified citizen science projects that were not found by the original reviewer; and (ii) there was disagreement between the original reviewer and the peer reviewer regarding whether a project was considered suitable for a particular indicator. In the first situation, any new citizen science projects identified were discussed between the two reviewers to reach consensus. As this constituted an omission error, consensus was straightforward. The second situation arose because reviewers have different experiences of citizen science that influence their definition of citizen science as a concept. In this situation, reviewers were asked to use the definition of citizen science as set out in the paper and flag any remaining disagreements. The lead author then reviewed any situations with outstanding disagreements in step 6.
Once step 5 was completed, a second peer review process was undertaken (step 6) in two phases. In the first phase of step 6, the lead author of this paper reviewed the mapping done by all other co-authors to ensure that the metadata had been interpreted correctly and to modify those indicators that were initially mapped as having “no alignment”, changing them to “could contribute” where needed, due to additional searching and knowledge of potential initiatives. The lead author also addressed any outstanding disagreements between the reviewer and the peer reviewer identified in step 5. This involved reviewing all the projects identified and all the evidence provided, and then applying the definition of citizen science outlined in the paper to determine whether the citizen science projects are applicable. A final discussion between the lead author, the reviewer and the peer reviewer was then held to reach consensus.
The second phase of step 6 involved a review of all indicators by the chief statistician of UN Environment, who works on the development of the environmental SDG indicators for which UN Environment is a custodian agency. Once again, in the majority of cases, there was consensus between the lead author and the chief statistician, but two situations of disagreement arose. The first case was similar to the situation described above, i.e., disagreements regarding whether a project was citizen science or not, e.g., self-reporting or community-based monitoring, which falls within the definition of citizen science. These disagreements were discussed until consensus was reached. The second situation was similar but occurred when consensus could not be reached, in particular for those SDGs for which UN Environment is not the custodian agency; the chief statistician, therefore, did not have the required expertise. The lead author then contacted other custodian agencies such as the World Bank, UNODC (UN Office on Drugs and Crime), UN FAO (the Food and Agriculture Organization of the UN) and WHO (World Health Organization) for relevant indicators.
In the steps described above, the co-authors have not applied a formal consensus building method or approach over disagreements, but rather encouraged discussions among the reviewers, which allowed different viewpoints and alternative explanations to be introduced, debated, broken down and reassessed. In this way, the authors were able to collectively analyze and evaluate the reasoning and results of others that, in the majority of cases, led to a convergence in reviewers’ perceptions. In three cases, the authors could not reach consensus due to the limited expertise in the particular thematic area of a specific indicator so the authors agreed to leave the judgement to the responsible staff at the relevant custodian agency/ies. This was identified as the best possible resolution to a few minor disagreements among the authors, since custodian agencies are responsible for defining the methodologies of the relevant indicators, including their sources of information and data collection methods, and also have accountability for quality and accuracy of reporting at the global level. In these cases, the lead author consulted the relevant staff members of the custodian agencies and presented the views of different authors to identify the accurate mapping and justification. For example, SDG indicator 16.1.3 is about the proportion of population subjected to (a) physical violence, (b) psychological violence and (c) sexual violence in the previous 12 months. The potential contribution of some of the identified citizen science initiatives, e.g., SafeCity, HarrasMap, etc., to this indicator was unclear to the chief statistician of the UN Environment due to her limited knowledge on the methodology of this indicator. Hence, the lead author consulted the Head of the Research and Trend Analysis Branch of the UNODC, the custodian agency responsible for many of the SDG 16 indicators, who confirmed the potential complementarity of such initiatives to inform this indicator, which was then used in the final mapping. Evidence of these contacts is provided in Table S1. After this second peer review process, the final mapping of citizen science to the SDG indicators was completed in step 7. The final mapping results are provided as Table S1 in the Supplementary Material.
Compilation of results
Once the entire SDG indicator mapping exercise was completed, the results in Table S1 were summarized according to the categories of “already contributing”, “could contribute” and “no alignment at present” to understand the current situation by SDG, tier classification, and custodian agency. The results were also compared to the mapping exercise undertaken by GEO to see where there are overlaps between contributions that could be jointly made from citizen science and EO. Finally, we provide examples by indicator to demonstrate where citizen science is already contributing and where it has the potential to contribute in the future.