Background

In 2008, the World Health Organization’s (WHO) global Commission on Social Determinants of Health (CSDH) called for action on the social determinants of health (SDH), the conditions in which persons are born, grow, work, live, and age, to “close the gap in a generation” [1] (p1). Widespread recognition now exists that taking action on SDH is crucial for reducing and reversing the growing health inequities (i.e., unfair, remediable inequalities in health) [2] that exist within and between countries.

In 2011, a total of 125 countries developed and signed the Rio Political Declaration on Social Determinants of Health (hereafter Rio Political Declaration) [3]. The declaration recommended interventions from governments and international organizations in five Action Areas (Table 1). Building directly on and aligning closely with CSDH’s final recommendations [1], the declaration encompasses 50 pledges for implementing a minimum set of actions that address SDH for improving health equity across diverse sectors. The Rio Political Declaration’s vision of intersectoral and multisectoral action for health was endorsed by the 194 WHO Member States at the 65th World Health Assembly in 2012 (Resolution WHO65.8), and then echoed by 193 Member States of the United Nations (UN) in the 2015–2030 Sustainable Development Goals (SDGs) [4].

Table 1 Five Action Areas of the 2012 Rio Political Declaration on Social Determinants of Health

The interventions on the SDH, recommended in the pledges organized in the five Action Areas, are supported by an increasingly comprehensive body of evidence. For example, research findings continue to support the use of social protection floors and systems over the life course, including social protection benefits (e.g., cash transfers) for children, mothers, the unemployed, occupational injury victims, and older persons (e.g., [5,6,7,8,9,10,11]). Another example is evidence for the effectiveness of early childhood interventions in improving cognitive development, health service use, and health outcomes in both childhood and adulthood [12,13,14]. Moreover, novel approaches in systems theory and causal inference used to evaluate social interventions continue to produce evidence of their beneficial effects on community and population health equity [15, 16].

Health sector monitoring of population health inequities within countries has improved at the global level, including WHO’s Health Equity Monitor [17], the Millennium Development Goals [18], and the progressive realization of the SDGs [19]. The importance of implementation of SDH interventions in the SDG era is gaining recognition [20]. However, specific efforts focused on monitoring SDH actions have only recently received attention, via individual national government commitments and of mandated intergovernmental organizations [16, 21,22,23,24]. A need persists for monitoring SDH actions to align them with their actors, and not just the situation of SDH themselves (where these cannot be aligned with policy responsibilities) without overburdening existing monitoring systems.

Through the Rio Political Declaration and other WHO resolutions (e.g., WHA62.14), WHO has committed to developing a global monitoring system for action on SDH [25, 26]. The goals of the proposed WHO monitoring system are to (1) track the progressive realization of action on SDH through implementation of the Rio Political Declaration at national and international levels and (2) guide continuous improvement in SDH action by UN Member States and within the UN system by providing regular reports about the status of and trends in such action. The WHO global monitoring system for action on the SDH will complement WHO’s existing tracking of key social and environmental determinants of health. That system is analogous to other WHO monitoring systems focused on monitoring financing and implementation for water and sanitation [27] or implementation of the WHO Framework Convention on Tobacco Control [28].

The first step in establishing the SDH action monitoring system is identifying its domains, which are the five Action Areas of the Rio Political Declaration. SDH action is defined in this context as a human rights, governance, policy, or programmatic intervention that improves health. The second step is identifying relevant measurement concepts (i.e., actions that can be defined, conceptualized, and measured). Ideally, each measurement concept already has standard action indicators that are internationally harmonized, meet minimum quality standards, are based on data available for a majority of UN Member States, and are relevant for key stakeholders (e.g., national governments, international organizations, and civil society). SDH action indicators are performance indicators for inputs, outputs, and outcomes of relevant government interventions (e.g., existence of or coverage with laws, policies, or programs). These indicators may measure SDH action by (1) human rights frameworks; (2) governance structures and mechanisms; (3) social policies and programs; and (4) environmental policies and programs [29]. A recent global stocktake identified SDH-focused monitoring systems reporting action indicators in 16 countries and in five regional and global systems (mostly of WHO), although reporting of relevant indicators to the Rio Political Declaration is limited [29]. Canada was among the first countries to monitor SDH action and has qualitatively reported on national actions to advance the five Action Areas of the Rio Political Declaration [21, 22, 30].

We report in this paper on the process undertaken to (1) identify key measurement concepts for each of the five Action Areas of the Rio Political Declaration; (2) identify suitable candidate SDH action indicators and data sources; and (3) propose a core set of the 15–20 selected indicators, considering the needs of key indicator users, including national governments, international organizations, and civil society, which we propose form part of the initial WHO global system for monitoring action regarding SDH. These tasks were accomplished by an interdisciplinary working group (WG) established by WHO and its Canadian partners, the Public Health Agency of Canada (PHAC), and the Canadian Institutes of Health Research—Institute of Population and Public Health (CIHR–IPPH).

Methods

The WG implemented processes for identifying a set of indicators that were aligned with measurement concepts drawn from the Rio Political Declaration [3], had high content validity, and relied on available country-specific monitoring data. The Rio Political Declaration with 50 specific actions built directly on and aligned with the recommendations of the Commission on Social Determinants of Health [1]. Therefore, monitoring implementation of actions taken in response to the organizing policy framework of the Rio Political Declaration is in harmony with the long-term efforts on SDH that Member States have pursued.

Recruitment of WG members was a deliberate and extensive process that sought to bring together those with both technical and policy expertise, and with representation from all six WHO regions. This step was important for reflecting considerations for monitoring in the countries of their particular region. The WG comprised 18 experts, including academics, policy-makers from high-, middle-, and low-income Member States; and representatives from WHO. WG members represented Australia, Brazil, Canada, Chile, India, Italy, Kenya, Morocco, Norway, Rwanda, Switzerland, and the USA. The WG also included persons with expertise in indicator development. This broad representation enabled WG members to identify technically sound, feasible, and acceptable candidate indicators while considering the needs of key users of the indicators, including national governments, international organizations, and civil society. The WG was chaired by Patricia O’Campo PhD, University of Toronto, who is a world leader in the study of SDH. The WG was also supported by a secretariat of four graduate students who screened candidate indicators, drafted documents, and provided administrative support during WG meetings.

During a rapid 4-month period, the expert WG members:

  1. 1.

    identified and prioritized key measurement themes and specific measurement concepts related to action on the SDH in the Rio Political Declaration;

  2. 2.

    identified and selected an initial list of potentially suitable candidate indicators for each measurement concept from multiple databases and systems, including the SDG monitoring system; and

  3. 3.

    applied selection and quality criteria for prioritizing a core set of indicators.

A flow chart of the full process is depicted in Fig. 1. Key terms for the process were also defined early (Table 2).

Fig. 1
figure 1

Flow chart describing the expert working group process for arriving at a core set of proposed indicators for measuring social determinants of health (SDH) action

Table 2 Glossary of terms reflecting the components of the social determinants of health (SDH) action monitoring system and corresponding components of the Rio Political Declaration (from narrow to broad)

Identification of measurement themes and measurement concepts derived from the Rio political declaration action areas and pledges

The WG members reviewed each pledge of the Rio Political Declaration to identify and record the frequency of occurrence of themes related to policy sectoral entry points (e.g., social service and protection policies, development strategies, policies, and rights) or such special population groups such as children; women; indigenous populations; informal workers; and lesbian, gay, bisexual, transgender, and intersex (LGBTI) populations. These entry points formed measurement themes, which were defined to ensure a common understanding of the theme across the WG. Broadly, measurement themes were general ideas that might have been reflected in more than one Rio Political Declaration Action Area or pledge (see glossary Table 2). For example, the measurement theme “build social protection floors” referred to “the extent to which governments provide essential health and economic security to populations in need” (see Table 4 in Appendix) and was linked to pledges in three of the Rio Action Areas: (1) to adopt better governance for health and development (pledge 1.x, promote and strengthen universal access to social services and social protection floors); (2) to further reorient the health sector towards reducing health inequities (pledge 3.iv, build, strengthen and maintain health financing and risk pooling systems and prevent persons from becoming impoverished when they seek medical treatment); and (3) to strengthen global governance and collaboration (pledge 4.ii. support social protection floors as defined by countries to address their specific needs and the ongoing work on social protection within the UN systems, including the work of the International Labour Organization, see Table 4 in Appendix for a list and description of measurement themes by Rio Action Area).

These measurement themes, either in isolation or coupled with back-reference to Rio Political Declaration pledges, guided the formation of the more actionable measurement concepts to facilitate the proposal of suitable indicators of SDH action. Building on the measurement theme related to social protection discussed in an earlier section of the paper, the measurement concept of level of public social protection was identified. Other examples of measurement concepts included level of implementation of mechanisms for participation of civil society; provision of public laws guaranteeing workers human rights for informal workers; level of implementation of mechanisms for ensuring integration of equity into health systems, policies and programs; and North–South, South–South sharing to develop holistic policies addressing inequalities and sustainable development.

The WG members engaged in an iterative process for refining and prioritizing relevant measurement concepts for action on SDH, as follows. Measurement concepts that were mentioned more frequently within or across domains were prioritized over those mentioned less frequently. Preference was given to measurement concepts unique to action regarding SDH and well supported by evidence as recommended by the WG members. Measurement concepts were developed with attention paid to their relevance/applicability across different country contexts. Prioritized measurement concepts also aimed to reflect a balance of indicators for SDH-focused human rights or governance structures or mechanisms and policies or programs on specific determinants of health. Furnishing these discussions with information on data availability by country as described in the next section also shaped the finalization and definition of measurement concepts.

Identification and selection of the most suitable indicators for each measurement concept from multiple databases or systems

We screened existing reports and international databases proposed by WG members and external academic experts that are used for monitoring by international organizations or academic institutions. A prioritized source for indicators was the monitoring system of the SDGs with its 261 indicators [31] because using SDG indicators was regarded as crucial for ensuring alignment of the SDH action monitoring system with the 2015–2030 SDG agenda [32]. Indicators proposed by WHO and the World Bank for universal health coverage were also prioritized to ensure alignment with the universal health coverage agenda [33,34,35]. In total, we screened 21 data sources and references produced by the International Labour Organization, the World Bank, the Pan American Health Organization, and others (see Table 5 in Appendix for a list of sources consulted).

To ensure consistency with best practices of indicator selection, we reviewed existing relevant reports of international monitoring efforts as well as the indicator development literature [21, 36,37,38,39,40,41]. This process guided criteria development for arriving at the core set of proposed indicators, including inclusion or exclusion criteria of screened indicators for an initial list of candidate indicators, and a set of quality criteria for evaluating candidate indicators to form the core set of proposed indicators (described in the next section).

Inclusion and exclusion criteria

To be eligible for inclusion in the initial list, indicators were required to

  1. 1.

    capture a key measurement concept;

  2. 2.

    measure an action (or intervention) regarding SDH, defined as

    1. (a)

      a governance intervention focused on SDH or health equity;

    2. (b)

      a social intervention that improves health or health equity; or

    3. (c)

      an environmental intervention that improves SDH or health equity [29];

  3. 3.

    measure a modifiable action [29, 40];

  4. 4.

    measure a national-level government action (Rio Political Declaration Action Areas 1, 2, 3 and 5) or a global-level action from the countries’ government or international governmental organizations (Rio Political Declaration Action Area 4); and

  5. 5.

    be a quantitative or, if qualitative, at a minimum be a categorical indicator (i.e., existence of laws) [21, 40].

An indicator was excluded from the initial list if it:

  1. 1.

    does not measure an action regarding SDH (i.e., measures an intention as opposed to an action);

  2. 2.

    has no data available or has data becoming available only over the long term (> 3 years);

  3. 3.

    measures an action that cannot be modified; or

  4. 4.

    measures an action taken at the local level only.

The WG debated whether to include only continuous quantitative indicators focusing on the effectiveness of interventions (e.g., coverage), but this criterion was relaxed slightly on the basis of the nature of the measurement concepts as well as data availability constraints.

To further reduce the list of candidate indicators that met the inclusion criteria, we assessed the level of alignment with the measurement concepts. WG members scored each indicator on the measurement concept match criteria, where a score of 1 was assigned when the indicator was not at all in accordance with the corresponding measurement concept, and a score of 7 was assigned when it was completely in accordance with the corresponding measurement concept. A rationale for the score was also provided. Table 6 in Appendix provides detailed examples of candidate indicators that met inclusion criteria for measurement concepts as well as measurement concept alignment scores for domains 1–3.

Application of quality assessment criteria to arrive at a core set of indicators

The initial list of indicators was put forward on the basis of face validity and technical quality. In the process of finalizing the proposal for the monitoring framework the WG members developed and applied a broad set of technical criteria to apply to the indicators. Several of the criteria could be operationalized and formed part of formal documented assessments as described in Table 6 in Appendix. The technical quality assessment criteria were defined and applied as follows:

  1. 1.

    For harmonization purposes, preference was given to indicators from the SDG indicator system and concepts covered in SDG targets [42].

  2. 2.

    Indicators that could demonstrate change over time and were feasible were preferred. This meant preference was given to indicators with regular reporting of at least 5-year intervals, but greater preference was given to indicators with 2–3-year reporting intervals.

  3. 3.

    Indicators should demonstrate benefits for SDH, health service use, health outcomes, or health equity. Although no extensive or formal assessment of studies on the effectiveness of government actions was conducted, where WG members had knowledge of studies describing the links between social determinants and the indicators, the information was noted. Consequently, preference was also given to indicators measuring actions with a stronger evidence base over those with a weaker evidence base [21].

  4. 4.

    Expected acceptability (i.e., user-inspired, responding to data needs, and acceptable to such key stakeholders as national governments or civil society) [38, 41]. The assessment of these criteria was based on the knowledge and background of the WG members, who had been nominated by the WHO regional offices to represent the experiences of the countries of their particular region.

  5. 5.

    Feasibility and cost-effectiveness of providing the information—whether easy to obtain without additional burden for the producer or guardian of the data [38, 41]. Preference was given to official data over other sources because this would enable WHO and other countries to gather data more cost-effectively.

  6. 6.

    Type—preference was given to continuous indicators (e.g., the proportion of a population covered by a social protection floor) over ordinal indicators (e.g., provision of social protection floors was complete versus medium versus low) over binary indicators (e.g., social protection floor was provided versus not provided).

  7. 7.

    Applicability across diverse country contexts and global harmonization [38].

  8. 8.

    SMART criteria [39], as follows:

    Specificity—targets a specific area for improvement; Measurability—is easy to measure, interpret, and communicate with straightforward policy implications (avoid composite indices); Assignability—clearly specifies who will take the action; Realistic—an action that realistically can be taken, given available resources; and Time-relatedness—an action that can be changed over time (e.g., annual changes are feasible). The SMART criteria overlap with some of the criteria described previously, but the WG members believed that making explicit reference to them was important because they are commonly used in indicator development work.

On the basis of the number of technical quality assessment criteria met, the indicator was given a rating on a scale of 1–8 and a rationale for the score.

This quality review also included an assessment of the indicator’s methodologic development and data availability as follows:

  • Tier 1—Represented an indicator that is conceptually clear with established methodology and standards to generate the indicator and data regularly produced by countries.

  • Tier 2—Indicators were conceptually clear on the basis of established methodology and standards but where data were not regularly available across countries.

  • Tier 3—Indicators were not based on established methodology or standards.

Table 6 in Appendix provides detailed examples of candidate indicators and the quality and technical ratings and overall assessment of the availability of indicators by domain.

Results

The WG members developed and prioritized 23 measurement themes, and 19 measurement concepts. Following the iterative process of evaluation, we identified a core set of 36 candidate indicators for the monitoring system on SDH action.

The core set of indicators proposed by the WG is presented in Table 3. Each indicator was uniquely named, with the nomenclature reflecting the indictor’s domain, measurement concept (that at times included an indication of being an equity measure), and the individual indicator. For example, Indicator 3.1I.1 Parity index (by wealth quintile) in coverage with safely managed drinking water is:

  • an indicator from domain 3 (i.e., 3.1I.1)

  • the measurement concept 1I (i.e., 3.1I.1), where the “I” indicates that inequality in an intervention is measured; and

  • the first indicator for this measurement concept (i.e., 3.1I.1).

Table 3 Final core set of 36 indicators and sources of data proposed by the working group to consider for the monitoring system on taking action regarding social determinants of health (SDH)

Five indicators of the core set expressly deal with inequities in intervention coverage, 21 cover governance interventions, and the remaining 10 address social and environmental interventions to improve the SDH.

Tables in the appendices provide information about the results of the development of measurement themes and concepts (Table 4 in Appendix) as well as information about the sources of data consulted to identify the candidate indicators (Table 5 in Appendix). An example of candidate indicators for consideration for the measurement concepts and their levels of alignment, face validity, and technical quality are presented in Table 6 in Appendix. What is not shown in Table 6 in Appendix are measurement concepts for which we could not find relevant indicators. For example, for the following measurement concepts, we were unable to locate candidate indicators: measure extent to which equity impacts of all government policies assessed routinely in decision making (governance); provision public laws guaranteeing self-determination of Indigenous peoples (governance); promote platforms for knowledge exchange of equity-oriented good practices and successful experiences (health sector reorientation); represent level of implementation of international agreements that improve the SDH (global governance); and ensure that justice and accountability are key components of research and evaluations (monitoring and accountability).

Discussion

Summary of findings

We present the methodology used to identify and prioritize key measurement concepts from the Rio Political Declaration and to select relevant, high quality indicators with real-life restrictions on data availability to form the first proposed core set of indicators to guide global monitoring for action on SDH. This set of indicators was presented to a large group of UN Member States and global technical experts at the International Technical Meeting on Measuring and Monitoring Action on the Social Determinants of Health, which took place in Ottawa, Canada, in June 2016 [43] to further guide a broader SDH Action Monitoring System for Member Countries. This is an innovative initiative as previous literature focused explicitly on indicators of determinant vs policy outcomes [34, 44]. Ultimately, this set of indicators will enable policy-makers to track progress in addressing SDH and to build the evidence base of effective actions for reducing health inequities (e.g., [5,6,7,8,9, 23, 45]). Recent evidence emphasizes the importance of cross-government commitment to addressing the root of SDH inequities [33]. Principles guiding identification of the indicator set included input from a diverse working group, attention to equity in the concepts and indicators, and reliance on existing indicators.

Lessons learnt for global monitoring system development

We identified several strengths of our methods for developing a proposed core set of indicators. First, our WG members included diverse technical and policy experts from all six WHO regions. This broad representation enabled the WG to identify technically sound, feasible, and acceptable candidate indicators while considering the needs of key users of the indicators. Second, we drew on best practices for indicator selection [21, 38,39,40,41] in developing the inclusion or exclusion and quality assessment criteria applied to candidate indicators during the screening process. We also adopted an iterative approach that involved consideration of data availability to refine and prioritize measurement concepts.

A guiding principle for the indicator set development was a strong focus on equity. First, some disaggregated indicators (i.e., by sex, age, income, etc.) were included where the concept and data permitted. As is becoming the reference standard in the SDG era with its commitment of leaving no one behind [32], disaggregated indicators better capture how action on the SDH works or does not work to reduce health inequities. To further incorporate inequity measurement into the monitoring system, the WG sought to incorporate indicators that would focus specifically on actions directly affecting the most vulnerable groups, thereby ensuring that no one is left behind. This included indicators that measure inequities experienced by indigenous peoples, children, women, persons living in poverty, LGBTI populations, and informal workers.

We identified six key challenges in the development of the core set of indicators. First, we discovered that indicators monitoring SDH-focused interventions are still not routinely collected for certain areas. For example, indicators for intersectoral actions (e.g., Health in All Policies) have only recently been assessed qualitatively by PAHO [34]. Indicators related to the health expenditures on health promotion are focused only in Organization for Economic Cooperation and Development countries. The WG members therefore relied heavily on the SDG indicator system, which presents a number of novel SDH action indicators.

Second, indicators that are explicitly focused on equity are limited. An equity orientation to monitoring health determinants is a new area of focus for health surveillance and monitoring systems [46], although examples such as indicators for measuring actions related to multisectoral governance and participation are starting to emerge [40]. Moreover, although many populations deserving additional attention (e.g., indigenous peoples, children) during monitoring efforts were considered, our final indicators were limited in number so some such as migrants or persons with disabilities were not reflected in our final set. Similarly, another example of a trade-off was around exclusion of informal payments (tipping or bribes) in health systems from the final set limiting the representation of the domain measurement concept “mechanisms for ensuring integration of equity into health systems, policies and progammes” (Table 3, item 3.a).

Third, although we sought to apply the quality assessment criteria with care, time constraints and limited resources presented some challenges. For example, one criterion required that we give priority to indicators with stronger existing evidence about the benefits of that particular SDH. The application of the criterion relied on the expert panel’s knowledge base; neither time nor resources allowed more extensive literature reviews of this particular topic. Another area affected by our short time frame, and will have to be examined in future work in this area, was our inability to examine and identify process indicators that reflect the dynamic nature of and interaction between two or more measurement concepts or domains.

A fourth challenge identified was identifying how to capture the way interventions are implemented and in what types of contexts including such questions as those pertaining to Indigenous populations, that in some countries are regional concerns and not applicable to the country as a whole. Multiple SDH indicators measure actions taken to improve SDH, but they are not designed to capture the degree of implementation or coverage or whether the action was successful. The political, social, cultural, or community contexts in which the interventions are implemented are not always clear from the indicators. This is important information for determining if or how an intervention works or does not work.

Fifth, assessing the effectiveness and cost-effectiveness of SDH-focused interventions is difficult on the basis of indicators alone. The range of information on cost-effectiveness is limited by the extent of effectiveness studies, which are inherently challenged by the complexity of the SDH interventions [47]. Consolidating the evidence base of countries’ implementations of complex policy-level interventions will be necessary for ensuring that the monitoring system is able to capture SDH actions that are considered best practices. Although systematic reviews are useful for building the evidence base for a single sector, especially health sector initiatives, other methods of summarizing the effective implementation and impacts of complex cross-sectoral policies will likely be required [48, 49].

Finally, the WG also experienced limitations in the available data sources. For instance, the domains of the monitoring system capture all Rio Political Declaration Action Areas, but the scan of globally available indicators revealed a disproportionate number of existing indicators for each Action Area. Action Area 4 strengthen global governance and collaboration for addressing the SDH (domain 4) and Action Area 5 monitor progress and increase accountability (domain 5) had only six and five candidate indicators, respectively. And while data availability was a criterion, conceptual fit and feasibility for data collection were also important considerations. For example, indicator 5.b.1 was only available for Canada but this was an indication of the feasibility of developing and reporting a standard metric. Additional work is needed to develop more suitable indicators to capture action to strengthen global governance and collaboration, and action for monitoring progress and increasing accountability.

Implications for research and practice

More work will be needed as the monitoring system continues to develop. Where data gaps have been identified, further exploration of existing data sources and potential new sources will be necessary. A need also exists for evaluating the monitoring system for relevance, accuracy, and validity, and potentially for adapting indicators as new data sources become available or data quality improves. More research on under-studied but crucial interventions on SDH is needed (e.g., mechanisms and structures for intersectoral action for health and global health governance interventions that protect health and health equity from international trade agreements) to further strengthen the evidence base for SDH action indicators, which might necessitate developing novel research methods.

Conclusions

Because of the pervasive and growing inequalities worldwide, there is an emerging trend towards promoting and monitoring government action on SDH. Yet few existing indicators adequately capture a government’s intent to and implementation of policies and programs to address SDH. Many challenges exist to developing such indicators, some of which have been described here (e.g., those related to data availability). While the methods presented in this paper extend the state of the art in measuring government action on SDH, future efforts should attend to the existing gaps to create a strong set of indicators for monitoring SDH in high- and low-income countries alike.