Developing Metrics to Define Progress in Children’s Surgery

  • Dan PoenaruEmail author
  • Justina Onyioza Seyi-Olajide
Surgical Symposium Contribution


There is a need for relevant, valid, and practical metrics to better quantify both need and progress in global pediatric surgery and for monitoring systems performance. There are several existing surgical metrics in use, including disability-adjusted life years (DALYs), surgical backlog, effective coverage, cost-effectiveness, and the Lancet Commission on Global Surgery indicators. Most of these have, however, not been yet applied to children’s surgery, leaving therefore significant data gaps in the burden of disease, infrastructure, human resources, and quality of care assessments in the specialty. This chapter reviews existing global surgical metrics, identifies settings where these have been already applied to children’s surgery, and highlights opportunities for further inquiry in filling the knowledge gaps. Directing focused, intentional knowledge translation efforts in the identified areas of deficiency will foster the maturation of global pediatric surgery into a solid academic discipline able to contribute directly to the cause of improving the lives of children around the world.


  1. 1.
    Cooper RS, Osotimehin B, Kaufman JS, Forrester T (1998) Disease burden in sub-Saharan Africa: what should we conclude in the absence of data? Lancet 351(9097):208–210CrossRefGoogle Scholar
  2. 2.
    Watters DA, Guest GD, Tangi V, Shrime MG, Meara JG (2018) Global surgery system strengthening: it is all about the right metrics. Anesth Analg 126(4):1329–1339CrossRefGoogle Scholar
  3. 3.
    Collaborative GlobalSurg (2016) Mortality of emergency abdominal surgery in high-, middle- and low-income countries. Br J Surg 103(8):971–988CrossRefGoogle Scholar
  4. 4.
    Murray CJ (1994) Quantifying the burden of disease: the technical basis for disability-adjusted life years. Med Mission Austria 72(3):429–445Google Scholar
  5. 5.
    Murray CJL, Vos T, Lozano R, Naghavi M, Flaxman AD, Michaud C et al (2012) Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 380(9859):2197–2223CrossRefGoogle Scholar
  6. 6.
    Murray CJL, Ezzati M, Flaxman AD, Lim S, Lozano R, Michaud C et al (2012) GBD 2010: design, definitions, and metrics. Lancet 380(9859):2063–2066CrossRefGoogle Scholar
  7. 7.
    Murray CJL, Lopez AD (2013) Measuring the global burden of disease. N Engl J Med 369(5):448–457CrossRefGoogle Scholar
  8. 8.
    Poenaru D, Ozgediz D, Gosselin RA (2014) Burden, need, or backlog: a call for improved metrics for the global burden of surgical disease. Int J Surg 12(5):483–486CrossRefGoogle Scholar
  9. 9.
    Poenaru D (2012) Getting the job done: analysis of the impact and effectiveness of the SmileTrain program in alleviating the global burden of cleft disease. World J Surg 37(7):1562–1570. CrossRefGoogle Scholar
  10. 10.
    Maine RG, Linden AF, Riviello R, Kamanzi E, Mody GN, Ntakiyiruta G et al (2017) Prevalence of untreated surgical conditions in rural Rwanda: a population-based cross-sectional study in Burera district. JAMA Surg 152(12):e174013CrossRefGoogle Scholar
  11. 11.
    Carlson LC, Stewart BT, Hatcher KW, Kabetu C, VanderBurg R, Magee WP Jr (2016) A model of the unmet need for cleft lip and palate surgery in low- and middle-income countries. World J Surg 40(12):2857–2867. CrossRefPubMedGoogle Scholar
  12. 12.
    Lozano R, Soliz P, Gakidou E, Abbott-Klafter J, Feehan DM, Vidal C et al (2006) Benchmarking of performance of Mexican states with effective coverage. Lancet 368(9548):1729–1741CrossRefGoogle Scholar
  13. 13.
    Ozgediz D, Hsia R, Weiser T, Gosselin R, Spiegel D, Bickler S et al (2009) Population health metrics for surgery: effective coverage of surgical services in low-income and middle-income countries. World J Surg 33(1):1–5. CrossRefPubMedGoogle Scholar
  14. 14.
    Shengelia B, Tandon A, Adams OB, Murray CJL (2005) Access, utilization, quality, and effective coverage: an integrated conceptual framework and measurement strategy. Soc Sci Med 61(1):97–109CrossRefGoogle Scholar
  15. 15.
    Bickler S, Ozgediz D, Gosselin R, Weiser T, Spiegel D, Hsia R et al (2010) Key concepts for estimating the burden of surgical conditions and the unmet need for surgical care. World J Surg 34(3):374–380. CrossRefPubMedGoogle Scholar
  16. 16.
    Vos T (2009) Improving the quantitative basis of the surgical burden in low-income countries. PLoS Med 6(9):e1000149CrossRefGoogle Scholar
  17. 17.
    Stokes MAR, Guest GD, Mamadi P, Seta W, Yaubihi N, Karawiga G et al (2016) Measuring the burden of surgical disease averted by emergency and essential surgical care in a district hospital in Papua New Guinea. World J Surg 41(3):650–659. CrossRefGoogle Scholar
  18. 18.
    Bickler SW, Ozgediz D, Poenaru D (2015). Academic global surgery: current research needs and priorities. In: Success in academic surgery, pp 87–98Google Scholar
  19. 19.
    Chao TE, Sharma K, Mandigo M, Hagander L, Resch SC, Weiser TG et al (2014) Cost-effectiveness of surgery and its policy implications for global health: a systematic review and analysis. Lancet Glob Health 2(6):e334–e345CrossRefGoogle Scholar
  20. 20.
    Meara JG, Leather AJM, Hagander L, Alkire BC, Alonso N, Ameh EA et al (2015) Global surgery 2030: evidence and solutions for achieving health, welfare, and economic development. Lancet 386(9993):569–624CrossRefGoogle Scholar
  21. 21.
    Shrime MG, Alkire BC, Grimes C, Chao TE, Poenaru D, Verguet S (2017) Cost-effectiveness in global surgery: pearls, pitfalls, and a checklist. World J Surg 41(6):1401–1413. CrossRefPubMedGoogle Scholar
  22. 22.
    Raykar NP, Ng-Kamstra JS, Bickler S, Davies J, Greenberg SLM, Hagander L et al (2017) New global surgical and anaesthesia indicators in the world development indicators dataset. BMJ Glob Health 2(2):e000265CrossRefGoogle Scholar
  23. 23.
    O’Neill KM, Greenberg SLM, Cherian M, Gillies RD, Daniels KM, Roy N et al (2016) Bellwether procedures for monitoring and planning essential surgical care in low- and middle-income countries: caesarean delivery, laparotomy, and treatment of open fractures. World J Surg 40(11):2611–2619. CrossRefPubMedGoogle Scholar
  24. 24.
    Haider A, Scott JW, Gause CD, Meheš M, Hsiung G, Prelvukaj A et al (2017) Development of a unifying target and consensus indicators for global surgical systems strengthening: proposed by the Global Alliance for Surgery, Obstetric, Trauma, and Anaesthesia Care (The G4 Alliance). World J Surg 41(10):2426–2434. CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    Esquivel MM, Molina G, Uribe-Leitz T, Lipsitz SR, Rose J, Bickler S et al (2015) Proposed minimum rates of surgery to support desirable health outcomes: an observational study based on three strategies. World J Surg 39(9):2126–2131. CrossRefPubMedGoogle Scholar
  26. 26.
    Marks IH, Fong ZV, Stapleton SM, Hung Y-C, Bababekov YJ, Chang DC (2018) How much data are good enough? Using simulation to determine the reliability of estimating POMR for resource-constrained settings. World J Surg 42(8):2344–2347. CrossRefPubMedGoogle Scholar
  27. 27.
    Xu K, Evans DB, Kawabata K, Zeramdini R, Klavus J, Murray CJL (2003) Household catastrophic health expenditure: a multicountry analysis. Lancet 362(9378):111–117CrossRefGoogle Scholar
  28. 28.
    Saksena P, Hsu J, Evans DB (2014) Financial risk protection and universal health coverage: evidence and measurement challenges. PLoS Med 11(9):e1001701CrossRefGoogle Scholar
  29. 29.
    Thong MK, Ho JJ, Khatijah NN (2005) A population-based study of birth defects in Malaysia. Ann Hum Biol 32(2):180–187CrossRefGoogle Scholar
  30. 30.
    Wright NJ, Zani A, Ade-Ajayi N (2015) Epidemiology, management and outcome of gastroschisis in Sub-Saharan Africa: results of an international survey. Afr J Paediatr Surg 12(1):1–6CrossRefGoogle Scholar
  31. 31.
    van der Linde D, Konings EEM, Slager MA, Witsenburg M, Helbing WA, Takkenberg JJM et al (2011) Birth prevalence of congenital heart disease worldwide: a systematic review and meta-analysis. J Am Coll Cardiol 58(21):2241–2247CrossRefGoogle Scholar
  32. 32.
    Wu VK, Poenaru D, Poley MJ (2013) Burden of surgical congenital anomalies in Kenya: a population-based study. J Trop Pediatr 59(3):195–202CrossRefGoogle Scholar
  33. 33.
    Toobaie A, Yousef Y, Balvardi S, St-Louis E, Baird R, Guadagno E et al (2018) Incidence and prevalence of congenital anomalies in low-and middle-income countries: a systematic reviewGoogle Scholar
  34. 34.
    Wilson J, Hodges A (2012) Cleft lip and palate surgery carried out by one team in Uganda: where have all the palates gone? Cleft Palate Craniofac J 49(3):299–304CrossRefGoogle Scholar
  35. 35.
    Butler EK, Tran TM, Nagarajan N, Canner J, Fuller AT, Kushner A et al (2017) Epidemiology of pediatric surgical needs in low-income countries. PLoS ONE 12(3):e0170968CrossRefGoogle Scholar
  36. 36.
    Concepcion T, Mohamed M, Dahir S, Ismail EA, Poenaru D, Rice HE et al. Measuring the burden of pediatric surgical conditions across Somaliland using a national, community-based sampling surveyGoogle Scholar
  37. 37.
    Bickler SW, Sanno-Duanda B (2000) Epidemiology of paediatric surgical admissions to a government referral hospital in the Gambia. Bull World Health Organ 78(11):1330–1336PubMedPubMedCentralGoogle Scholar
  38. 38.
    Delport SD, Christianson AL, van den Berg HJ, Wolmarans L, Gericke GS (1995) Congenital anomalies in black South African liveborn neonates at an urban academic hospital. S Afr Med J 85(1):11–15PubMedGoogle Scholar
  39. 39.
    Yousef Y, Lee A, Ayele F, Poenaru D (2018) Delayed access to care and unmet burden of pediatric surgical disease in resource-constrained African countries. J Pediatr Surg. CrossRefPubMedGoogle Scholar
  40. 40.
    Concepcion TL, Smith ER, Mohamed M, Dahir S, Ismail EA, Andrew JM. Provision of surgical care for children across Somaliland: challenges and policy guidanceGoogle Scholar
  41. 41.
    Goodman LF, Chuluun E, Sanjaa B, Urjin S, Erdene S, Khad N et al (2018) A nationwide assessment of pediatric surgical capacity in Mongolia. World J Surg 42(10):3432–3442. CrossRefPubMedGoogle Scholar
  42. 42.
    Collaborative GlobalSurg (2016) Determinants of morbidity and mortality following emergency abdominal surgery in children in low-income and middle-income countries. BMJ Glob Health 1(4):e000091CrossRefGoogle Scholar
  43. 43.
    Livingston MH, DCruz J, Pemberton J, Ozgediz D, Poenaru D (2015) Mortality of pediatric surgical conditions in low and middle income countries in Africa. J Pediatr Surg 50(5):760–764CrossRefGoogle Scholar
  44. 44.
    Toobaie A, Emil S, Ozgediz D, Krishnaswami S, Poenaru D (2017) Pediatric surgical capacity in Africa: current status and future needs. J Pediatr Surg 52(5):843–848CrossRefGoogle Scholar
  45. 45.
    Burgos CM, Bolkan HA, Bash-Taqi D, Hagander L, Von Screeb J (2018) The met needs for pediatric surgical conditions in Sierra Leone: estimating the gap. World J Surg 42(3):652–665. CrossRefPubMedGoogle Scholar
  46. 46.
    Chirdan LB, Ameh EA (2012) Untreated surgical conditions: time for global action. Lancet 380(9847):1040–1041CrossRefGoogle Scholar
  47. 47.
    Salomon JA, Haagsma JA, Davis A, de Noordhout CM, Polinder S, Havelaar AH et al (2015) Disability weights for the Global Burden of Disease 2013 study—appendix. Lancet Glob Health 3(11):e712–e723CrossRefGoogle Scholar
  48. 48.
    Smith ER, Concepcion T, Lim S, Sadler S, Poenaru D, Saxton AT et al (2018) Disability weights for pediatric surgical procedures: a systematic review and analysis. World J Surg 42(9):3021–3034. CrossRefPubMedGoogle Scholar
  49. 49.
    Poenaru D, Pemberton J, Frankfurter C, Cameron BH, Stolk E (2017) Establishing disability weights for congenital pediatric surgical conditions: a multi-modal approach. Popul Health Metrics 15(1):8CrossRefGoogle Scholar
  50. 50.
    Wu VK, Poenaru D (2013) Burden of surgically correctable disabilities among children in the Dadaab Refugee Camp. World J Surg 37(7):1536–1543. CrossRefPubMedGoogle Scholar
  51. 51.
    Poenaru D, Pemberton J, Cameron BH (2015) The burden of waiting: DALYs accrued from delayed access to pediatric surgery in Kenya and Canada. J Pediatr Surg 50(5):765–770CrossRefGoogle Scholar
  52. 52.
    Yap A, Muzira A, Cheung M, Healy J, Kakembo N, Kisa P et al (2018) A cost-effectiveness analysis of a pediatric operating room in Uganda. Surgery. CrossRefPubMedGoogle Scholar
  53. 53.
    Eeson G, Birabwa-Male D, Pennington M, Blair GK (2015) Costs and cost-effectiveness of pediatric inguinal hernia repair in Uganda. World J Surg 39(2):343–349. CrossRefPubMedGoogle Scholar
  54. 54.
    Magee WP Jr, Vander Burg R, Hatcher KW (2010) Cleft lip and palate as a cost-effective health care treatment in the developing world. World J Surg 34(3):420–427. CrossRefPubMedGoogle Scholar
  55. 55.
    Warf BC, Alkire BC, Bhai S, Hughes C, Schiff SJ, Vincent JR et al (2011) Costs and benefits of neurosurgical intervention for infant hydrocephalus in sub-Saharan Africa. J Neurosurg Pediatr 8(5):509–521CrossRefGoogle Scholar
  56. 56.
    Shrime MG, Sleemi A, Ravilla TD (2015) Charitable platforms in global surgery: a systematic review of their effectiveness, cost-effectiveness, sustainability, and role training. World J Surg 39(1):10–20. CrossRefPubMedGoogle Scholar
  57. 57.
    Saxton AT, Poenaru D, Ozgediz D, Ameh EA (2016) Economic analysis of children’s surgical care in low-and middle-income countries: a systematic review and analysis. PLoS ONE. CrossRefPubMedPubMedCentralGoogle Scholar
  58. 58.
    Alkire B, Hughes CD, Nash K, Vincent JR, Meara JG (2011) Potential economic benefit of cleft lip and palate repair in sub-Saharan Africa. World J Surg 35(6):1194–1201. CrossRefPubMedGoogle Scholar
  59. 59.
    Corlew DS, Alkire BC, Poenaru D, Meara JG, Shrime MG (2016) Economic valuation of the impact of a large surgical charity using the value of lost welfare approach. BMJ Glob Health 1(4):e000059CrossRefGoogle Scholar
  60. 60.
    Poenaru D, Lin D, Corlew S (2016) Economic valuation of the global burden of cleft disease averted by a large cleft charity. World J Surg 40(5):1053–1059. CrossRefPubMedGoogle Scholar
  61. 61.
    Mackinnon N, St-Louis E, Yousef Y, Situma M, Poenaru D (2018) Out-of-pocket and catastrophic expenses incurred by seeking pediatric and adult surgical care at a public, tertiary care centre in Uganda. World J Surg 42(11):3520–3527. CrossRefPubMedGoogle Scholar
  62. 62.
    Casey KM (2007) The global impact of surgical volunteerism. Surg Clin North Am 87(4):949–960CrossRefGoogle Scholar
  63. 63.
    Kagawa RC, Anglemyer A, Montagu D (2012) The scale of faith-based organization participation in health service delivery in developing countries: systematic [corrected] review and meta-analysis. PLoS ONE 7(11):e48457CrossRefGoogle Scholar
  64. 64.
    McQueen KAK, Hyder JA, Taira BR, Semer N, Burkle FM Jr, Casey KM (2010) The provision of surgical care by international organizations in developing countries: a preliminary report. World J Surg 34(3):397–402. CrossRefPubMedGoogle Scholar
  65. 65.
    Walker IA, Obua AD, Mouton F, Ttendo S, Wilson IH (2010) Paediatric surgery and anaesthesia in south-western Uganda: a cross-sectional survey. Bull World Health Organ 88(12):897–906CrossRefGoogle Scholar
  66. 66.
    Shrime MG, Sekidde S, Linden A, Cohen JL, Weinstein MC, Salomon JA (2016) Sustainable development in surgery: the health, poverty, and equity impacts of charitable surgery in Uganda. PLoS ONE 11(12):e0168867CrossRefGoogle Scholar
  67. 67.
    Okoye MT, Ameh EA, Kushner AL, Nwomeh BC (2015) A pilot survey of pediatric surgical capacity in West Africa. World J Surg 39(3):669–676. CrossRefPubMedGoogle Scholar
  68. 68.
    Cairo SB, Kalisya LM, Bigabwa R, Rothstein DH (2018) Characterizing pediatric surgical capacity in the Eastern Democratic Republic of Congo: results of a pilot study. Pediatr Surg Int 34(3):343–351CrossRefGoogle Scholar
  69. 69.
    Goodman LF, St-Louis E, Yousef Y, Cheung M, Ure B, Ozgediz D et al (2018) The Global Initiative for Children’s Surgery: optimal resources for improving care. Eur J Pediatr Surg 28(1):51–59CrossRefGoogle Scholar
  70. 70.
    Chirdan LB, Ameh EA, Abantanga FA, Sidler D, Elhalaby EA (2010) Challenges of training and delivery of pediatric surgical services in Africa. J Pediatr Surg 45(3):610–618CrossRefGoogle Scholar
  71. 71.
    Krishnaswami S, Nwomeh BC, Ameh EA (2016) The pediatric surgery workforce in low- and middle-income countries: problems and priorities. Semin Pediatr Surg 25(1):32–42CrossRefGoogle Scholar
  72. 72.
    Hamad D, Yousef Y, Caminsky NG, Guadagno E, Laberge JM, Emil S, Poenaru D (2018) Defining the critical global pediatric surgical workforce density for children under 15 years of age. Personal CommunicationGoogle Scholar
  73. 73.
    Ford K, Poenaru D, Moulot O, Tavener K, Bradley S, Bankole R et al (2016) Gastroschisis: Bellwether for neonatal surgery capacity in low resource settings? J Pediatr Surg 51(8):1262–1267CrossRefGoogle Scholar
  74. 74.
    Dewan MC, Baticulon RE, Ravindran K, Bonfield CM, Poenaru D, Harkness W (2018) Pediatric neurosurgical bellwether procedures for infrastructure capacity building in hospitals and healthcare systems worldwide. Childs Nerv Syst 34(10):1837–1846CrossRefGoogle Scholar
  75. 75.
    Paradis T, St-Louis E, LaRusso K, Poenaru D, GICS Collaborators (2018) Candidate bellwether procedures for children’s surgery using an iterative e-Delphi methodology. Personal CommunicationGoogle Scholar
  76. 76.
    Farber SH, Vissoci JRN, Tran TM, Fuller AT, Butler EK et al (2016) Geospatial analysis of unmet surgical need in Uganda: an analysis of SOSAS survey data. World J Surg 41(2):353–363. CrossRefGoogle Scholar
  77. 77.
    Zafar SN, Fatmi Z, Iqbal A, Channa R, Haider AH (2013) Disparities in access to surgical care within a lower income country: an alarming inequity. World J Surg 37(7):1470–1477. CrossRefPubMedGoogle Scholar
  78. 78.
    Wang H, Bhutta ZA, Coates MM, Coggeshall M, Dandona L, Diallo K et al (2016) Global, regional, national, and selected subnational levels of stillbirths, neonatal, infant, and under-5 mortality, 1980–2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet 388(10053):1725–1774CrossRefGoogle Scholar
  79. 79.
    Kruk ME, Pate M, Mullan Z (2017) Introducing the Lancet Global Health Commission on high-quality health systems in the SDG era. Lancet Glob Health 5(5):e480–e481CrossRefGoogle Scholar
  80. 80.
    Spence RT, Mueller JL, Chang DC (2016) A novel approach to global benchmarking of risk-adjusted surgical outcomes: beyond perioperative mortality rate. JAMA Surg 151(6):501–502CrossRefGoogle Scholar
  81. 81.
    Anderson GA, Ilcisin L, Ngonzi J, Ttendo S, Twesigye D, Benitez NP et al (2018) Validation of an electronic surgical outcomes database at Mbarara regional referral hospital, Uganda. World J Surg 42(1):54–60. CrossRefPubMedGoogle Scholar
  82. 82.
    Barber RM, Fullman N, Sorensen RJD, Bollyky T, McKee M, Nolte E et al (2017) Healthcare Access and Quality Index based on mortality from causes amenable to personal health care in 195 countries and territories, 1990–2015: a novel analysis from the Global Burden of Disease Study 2015. Lancet 390(10091):231–266CrossRefGoogle Scholar

Copyright information

© Société Internationale de Chirurgie 2018

Authors and Affiliations

  1. 1.Department of Pediatric Surgery, McGill University Health CentreMontreal Children’s HospitalMontrealCanada
  2. 2.Paediatric Surgery Unit, Department of SurgeryLagos University Teaching HospitalIdi-ArabaNigeria

Personalised recommendations