World Journal of Surgery

, 33:1

Population Health Metrics for Surgery: Effective Coverage of Surgical Services in Low-Income and Middle-Income Countries


    • Division of Pediatric Surgery
  • Renee Hsia
    • University of California, San Francisco
  • Thomas Weiser
    • Department of Health Policy and ManagementHarvard University
  • Richard Gosselin
    • School of Public Health, University of California at Berkeley
  • David Spiegel
    • University of Pennsylvania, Children’s Hospital of Philadelphia
  • Stephen Bickler
    • Division of Pediatric SurgeryUniversity of California at San Diego
  • Peter Dunbar
    • Department of AnesthesiaUniversity of Washington
  • Kelly McQueen
    • Harvard Humanitarian InitiativeHarvard University

DOI: 10.1007/s00268-008-9799-y

Cite this article as:
Ozgediz, D., Hsia, R., Weiser, T. et al. World J Surg (2009) 33: 1. doi:10.1007/s00268-008-9799-y



Access to surgical services is emerging as a crucial issue in global public health. “Effective coverage” is a health metric used to evaluate essential health services in low- and middle-income countries. It measures the fraction of potential health gained that is actually realized for a given intervention by integrating the concepts of need, use, and quality.


This study applies the concept of effective coverage to surgical services by considering injuries and obstetric complications as high-priority surgical conditions in low- and middle-income countries.


Effective coverage for both is poor, but it is less well defined for traumatic conditions compared to obstetric conditions owing to a lack of data.


More primary and secondary data are critical to measure effective coverage and to estimate the resources required to improve access to surgical services in low- and middle-income countries.


The global volume of surgery has recently been estimated at 234 million operations annually [1]. Unsurprisingly, services were unequally distributed, with 26% of these operations occurring in developing countries that account for 70% of the world’s population. Poor access to surgical care results in excessive morbidity and mortality from treatable conditions—injuries, infectious diseases, complications of pregnancy, congenital anomalies, acute abdominal disorders—yet the provision of “essential” surgery as a population-based health strategy has yet to be embraced by the global public health community [2]. As deficiencies in surgical capacity result from limitations in infrastructure and physical and human resources, strengthening surgical care requires improvements at the health systems level.

With limited resources and competing priorities, health planners must choose between interventions. Recently, universal access to a group of “essential” services has been a primary objective of health systems. If surgery is to become a public health priority, surgical conditions must be shown to have a significant impact on population health. The need to measure the impact of surgical care on a population and to monitor the delivery of services within the health system cannot be overstated [3].

Health metrics measure the effectiveness of interventions and the health system in general. These metrics are gaining importance as a means of evaluating the broader impact of public health programs, especially given the health reforms being implemented by many countries [46]. Because traditional public health indicators, such as maternal mortality and under-five mortality, may not capture the full impact of surgical care on population health, specific surgical indicators must be developed. Preliminary estimates from the Disease Control Priorities Project suggest that surgical care may avert 11% of the global burden of disease by reducing disability and premature death [2]. Recent evidence suggests that the cost-effectiveness of basic surgical care compares favorably with other classic public health initiatives such as vaccination programs and antiretroviral treatment for human immunodeficiency virus (HIV) infection [710]. The concept of effective coverage of essential health interventions in low- and middle-income countries has proven to be useful for evaluating their impact on the burden of disease in a specific population.

Effective coverage concept

Effective coverage (EC) estimates the fraction of potential health gain that is actually delivered to those who receive the intervention and integrates the concepts of need, use, and quality, as shown in the equation
$$ {\text{EC}} = {\text{Q}}*({\text{U/N}}) $$
where N is the population in need of an intervention, U is the utilization of the intervention among the population in need, and Q is the quality of the intervention and effect it has on the disease burden [3, 11, 12]. EC can be measured for one or a group of interventions or for the health system as a whole. Figure 1 demonstrates the utility of an overall estimate of the burden of disease as well as the requirement for measures of “met” and “unmet” need. Effective coverage is the ratio of met need to the sum of met and unmet need.
Fig. 1

Burden of disease (BoD) and effective coverage (Graphic courtesy of Dr. Steve Lim, Institute of Health Metrics and Evaluation)

Effective coverage was measured in Mexico for 14 public health interventions and was used to evaluate health system performance [11]. This method allowed measurements of state-level performance, impact of health reforms, and equity through stratification by socioeconomic status. None of the indicators, however, captured effective coverage of surgical services. To apply this concept to surgical services in low- and middle-income countries, some critical questions must be answered: How are surgical conditions defined? Can they be prioritized? How can need, use, and quality can be measured in resource-constrained settings where large proportions of populations with surgical conditions do not interact with the health system?

Defining surgical conditions and assessing priority interventions

Surgical conditions were broadly described in the Disease Control Priorities Project [2, 13]. However, any definition must acknowledge that many “surgical conditions” may require other treatments, such as resuscitation, intubation, or manipulation. For example, surgeons often care for head-injured patients, but only a small fraction require surgery. Therefore, whereas some surgical conditions require procedures only a portion of the time, adequate care requires the presence of surgical infrastructure and clinical expertise at all times. Thus, a surgical condition may be more broadly defined as “any condition for which the most potentially effective treatment is an intervention that requires suture, incision, excision, manipulation, or other invasive procedure that usually, but not always, requires anesthesia.”

There are more than 5000 existing surgical procedures; and effective coverage cannot be measured for all [14]. Measurement for several high-priority conditions may be more feasible. Criteria by which priority interventions were chosen to guide measurement of effective coverage in Mexico were based on the projected impact on the burden of disease, affordability, potential impact on health disparities, and the ability to extrapolate from these interventions to other interventions [11].

The Disease Control Priorities Project estimated that traumatic conditions and complications of pregnancy account for approximately half of the burden of surgically treated disease in Africa and Southeast Asia [2]. Recent data from primary referral facilities in several low-income countries document that essential surgery for obstetric and traumatic conditions can be provided in a cost-effective manner [7, 9, 10]. In addition, there are significant disparities between high- and low-income countries in the care of obstetric and traumatic conditions: 99% of maternal deaths occur in low-income countries, and one million to two million deaths per year could be prevented through improved trauma care [15, 16]. Thus, using criteria similar to those for other priority interventions, trauma care and complications of pregnancy might be a starting point to measure coverage of surgical conditions.

Measuring need for and use of treatment for obstetric complications

Reduction of maternal mortality by 2015 is one of the Millennium Development Goals. Demographic surveillance surveys and vital statistics provide the numerator and denominator for the maternal mortality rate (number of maternal deaths per 100,000 women of reproductive age) and the maternal mortality ratio (the number of maternal deaths during a given time period per 100,000 live births during this same period).

Altogether, 15% of births are expected to require medical attention, and at least 5% require surgical intervention [17, 18]. The actual number of attended obstetric complications seen in a facility during that same time period provide the numerator. This method has, for example, measured the met need for emergency obstetric care (EmOC) in Nicaragua and Chad as 31% and 13%, respectively.

Facility-based evaluations have shown 0% to 25% coverage of basic EmOC, including cesarean sections, in Uganda, Kenya, Tanzania, and southern Sudan; this has been used to estimate met and unmet need [19]. Some of these estimates use antenatal clinic attendance as the denominator, although this is not accurate in places where most women give birth at home [20]. Although these estimates were based on surveys from rural facilities, many estimates of national rates of care are based on urban facilities where the wealthy undergo a disproportionate share of interventions. Thus, an aggregate national rate can make a country appear to perform well when the lowest-wealth quintiles have poor coverage [21].

As obstetric complications cannot be predicted, the United Nations (UN) process indicators of proportion of all births in EmOC facilities can be used as a proxy for utilization. The UN defines the numerator of this measure as the number of women treated in an EmOC facility and the denominator as the estimated number of births in the area during the same time period. With the assumption that 15% of pregnant women develop complications that require medical attention, their calculation of the proportion of women utilizing EmOC facilities can help derive estimates of need [19, 22].

Measuring need for and use of trauma care

Injuries account for approximately 4.3% of total disability-adjusted life years (DALYs) and up to 38% of all surgical DALYs [2]. Road traffic crashes are reaching epidemic proportions in the developing world, where an estimated 25% of all hospital beds are occupied by survivors of road traffic crashes [2327]. Yet trauma systems are grossly inadequate and under-prioritized [2830].

Low utilization rates of health care facilities by injured patients in developing countries are well documented [31, 32]. Injuries treated in health care facilities represent only a fraction of all injuries needing treatment owing to the many barriers to care, although patients with severe nonfatal injuries are more likely to seek help at a health facility. In contrast to obstetric care, most hospital-based reports of surgical output do not categorize treatments for injury, and it is difficult to estimate how many DALYs from trauma could be averted by the provision of quality surgical services [33]. Given the difficulty of measurement, some injuries—such as traumatic amputations, femoral fractures, severe burns—could be used as proxy indicators to estimate the overall unmet need in a given population.

Unlike maternal mortality data, standardized information on injuries and resulting death and disability are lacking. Injury-related premature death can be measured using facility-based and administrative data of injury deaths if collected systematically. Nonfatal outcomes of injury are more challenging to measure owing to a lack of long-term follow-up data. Studies suggest that primary long-term disability from trauma is due in large part to orthopedic injuries [34]. Approximately 90% of all orthopedic surgeons care for 10% of the world’s population, and there are five orthopedic surgeons per 100,000 people in high-income countries compared to less than one per million in sub-Saharan countries. Such data point to the tremendous disparity that exists but also to the potential improvements that could be realized if such inequity was addressed.

Measuring the quality of surgical interventions for effective coverage

Assessment of surgical outcomes is difficult as intervention efficacy is not always immediately apparent, complications are often delayed, and follow-up of patients is limited. The likely outcome without treatment must be considered as some conditions are rapidly fatal, others are disabling, and still others may resolve spontaneously.

Conditions amenable to surgical intervention fall into two categories: those that are strictly an incident and those that are both an incident and prevalent. Strictly incident conditions are usually fatal without surgical treatment. They include obstructed labor and malpresentation, bowel obstruction and perforation, and injuries that are rapidly fatal but treatable—airway compromise, pneumothorax, arterial bleeding, and viscus damage. Patients who undergo an intervention and survive can be expected to either recover completely or be left with some calculable residual disability. Conditions that are both incident and prevalent are diseases that are disabling but rarely fatal. This includes obstetric fistula, hernias, cataracts, and many congenital malformations and nonfatal crippling injuries. Complications and efficacy of care are more difficult to assess because patients require longer follow-up. In addition, some of these conditions are sequelae that develop because of a lack of access to essential surgical care (e.g., obstetric fistula secondary to prolonged obstructed labor or chronic osteomyelitis from delayed fracture care).

Distinguishing between these two categories of conditions is important when evaluating the impact of surgical intervention. Treatment of incident conditions does not need to meet a very high level of efficacy because of the immediate threat to life: Even interventions that have a 50% mortality rate still avert mortality in half of those who would otherwise perish. However, the threshold for success must be considerably higher for interventions that address incident and prevalent conditions as higher-risk interventions may cause more harm than good and are often costly.

An immediate postoperative assessment tool has recently been described to evaluate the risk of complications or death following an operation. The “Surgical Apgar Score” uses three intraoperative measures to create a 10-point score that has been validated as a predictor of risk for complications [35]. It is based on a combination of the estimated blood loss, lowest mean arterial pressure, and lowest heart rate. Although not a substitute for classic outcome measures, it is feasible and immediately available in most institutions meeting minimum standards of intraoperative anesthesia monitoring. Using such a metric can help improve the estimation of surgical outcome in settings where more sophisticated surveillance systems are unavailable.

Limitations of measuring effective coverage for surgery

Need, utilization, and quality are essential to effective coverage but difficult to measure for surgical conditions. Data from health facilities are limited by their accuracy and completeness; population-based data are valuable but expensive alternatives. Facility-based data have, however, yielded important information when used to calculate cumulative incidence of, for example, pediatric surgical conditions in The Gambia [36]. Nonetheless, a more thorough review of existing local data in selected sites is a critical and feasible first step as part of a broader research agenda in global surgery [37]. Model-based estimates of need and utilization with limited local data have driven studies of maternal mortality, and they should be reviewed for applicability to a broader range of conditions [38].

Priority surgical conditions may be added to existing demographic health and surveillance surveys, but these are already lengthy and time-consuming programs. Household and sisterhood survey approaches, when family members are queried regarding deaths of family members who were pregnant, are used commonly for estimates of maternal mortality and might also be used for other common priority surgical conditions such as trauma [39]. More robust epidemiologic data from middle- to high-income countries may be used to extrapolate disease “incidence,” but these have been shown to have limited applicability in low-income settings [40].

The impact of surgical intervention is probably the most difficult to assess. The classic methods of evaluating efficacy are expensive and time-consuming. Risk adjustment is the definitive means for comparing surgeons and institutions, but it requires a robust data collection infrastructure. Simpler methods are necessary in resource-constrained environments. Structural measures are easy to obtain and can measure systems of surgical care. Process measures are also meaningful as surrogate measures for evaluating the quality of care [41]. Outcome measures are the most robust and persuasive when evaluating the results of care. The process of evaluating outcomes of surgical intervention has been described for the purposes of cost-effectiveness and can serve as a useful model for determining burden of disease measurements [7, 9]. This process, however, like those previously described, is resource- and time-intensive.


Effective coverage has been valuable to health system evaluation. It is a useful tool for measuring the full spectrum of surgical care from demand-side barriers to outcomes. It is now a priority for public health surveillance to provide a comprehensive, systematic approach to measuring the burden of surgical disease, the capacity of a health system to provide surgical care, and the quality of care that is provided. As more information is gathered about surgical services on a global level, gaps in knowledge will be narrowed and practical methods to measure access to surgical care in low- and middle-income countries will be developed. A composite index of effective coverage for selected surgical conditions, as has been used for other health interventions, holds great promise as an effective tool for policymakers to target interventions for those most in need. Surgical data from national health care systems, nongovernmental organizations and other providers of international surgical services globally are critical to these efforts.

Copyright information

© Société Internationale de Chirurgie 2008