European Journal of Pediatrics

, Volume 177, Issue 6, pp 921–933 | Cite as

Development of a core set of quality indicators for paediatric primary care practices in Europe, COSI-PPC-EU

  • Dominik A. EwaldEmail author
  • Gottfried Huss
  • Silke Auras
  • Juan Ruiz-Canela Caceres
  • Adamos Hadjipanayis
  • Max Geraedts
Original Article


Paediatric ambulatory healthcare systems in Europe are, because of historical reasons, diverse and show strikingly different outcomes. All across Europe, the benchmarking of structures, processes and outcomes could reveal opportunities for improving Paediatric Primary Care (PPC). The aim of this study was to develop a set of Quality Indicators (QIs) to assess and monitor PPC in Europe. In a three-step process, we used the available external evidence and European expert consensus in a modified RAND/UCLA Appropriateness Method (RAM) to develop an indicator set. (1) A broad literature and online research of published QI and guidelines yielded an inventory of 1516 QI. (2) A collaborative panel of paediatric senior experts from the European Academy of Paediatrics (EAP) and the European Confederation of Primary Care Paediatricians (ECPCP) from 15 European countries participated in a first consensus process to reduce the initial indicator inventory by eliminating not PPC-focused indicators and duplicates. (3) In a second consensus process, the panel rated the QI regarding validity and feasibility. The final QI set “COSI-PPC-EU” consists of 42 indicators in five categories of PPC: (A) health promotion/prevention/screening (13 QI), (B) acute care (9 QI), (C) chronic care (8 QI), (D) practice management (3 QI) and (E) patient safety (9 QI).

Conclusion: COSI-PPC-EU represents a consented set of a limited number of valid quality indicators for the application in paediatric primary care in different healthcare systems throughout Europe.

What is Known:

• Paediatric ambulatory healthcare systems in Europe are diverse and show strikingly different outcomes.

• There are known gaps in quality performance measures of paediatric primary care in Europe. Pre-existing sets of quality indicators are predominantly limited to national populations, specific diseases and hospital care.

What is New:

• A set of 42 quality indicators for primary paediatric care in Europe was developed in a multi-country collaborative effort. The method combined a systematic literature review and a consensus process among European paediatric experts.

• The quality indicator set can facilitate quality improvement of PPC. After studying the feasibility, providers can use COSI-PPC-EU to monitor, compare and improve performance of practices, regions and countries.


Quality indicator Quality improvement Quality management Paediatric primary care Community child care Europe Benchmarking EAP ECPCP RAND/UCLA appropriateness method 



Core set of quality indicators


Core set of quality indicators for Paediatric primary care practices in Europe


German Academic Society for General Paediatrics


European Academy of Paediatrics


European Confederation of Primary Care Paediatricians




General practitioner


Grading of Recommendations Assessment, Development and Evaluation, UK


National Health Service, UK


National Institute for Health and Care Excellence, UK


Organisation for Economic Co-operation and Development


Paediatric Primary Care


Quality indicator


RAND/UCLA appropriateness method


Union of European Medical Specialists


United Nations International Children’s Emergency Fund


World Health Organization



We thank the members of the panels for evaluating and rating the indicators: Angel Carrasco Sanz, Björn Wettergren, Elke Jäger-Roman, Arunas Valiulis, Peter Altorjai, Indre Butienne from EAP and several other country delegates, as well as members from EAP, ECPCP and DGAAP.

Authors’ contributions

Dominik A. Ewald and Gottfried Huss acted as European and national study coordinators from 2011 to 2013 and wrote the main part of the manuscript. Silke Auras prepared a draft of the methods and results sections. Juan Ruiz Canela was responsible for the literature research and screened with Gottfried Huss the scientific literature. Adamos Hadjipanayis is the secretary general of the EAP; he reviewed the manuscript and gave valuable advice, and he supported the project through all the years. Max Geraedts was the principal investigator who coordinated the evidence reviews, the foundation of the database and the statistical evaluation of the rating process. All the authors revised the manuscript before submission.


The scientific evaluation of the COSI project by Max Geraedts and Silke Auras was financed by a grant from the European Academy of Paediatrics, EAP. The literature research was possible with the financial support from the Spanish Paediatric Society AEP.

None of the panellists received travel or expense allowance especially for COSI-PPC-EU since the meetings took place at regular meetings of their organizations.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Compliance with ethical standards

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent was not necessary to obtain from individual participants, because there were none included in the study.


  1. 1.
    Altenhofen L, Brech W, Brenner G, Geraedts G, Gramsch E, Kolkmann FW, Rheinberger P (2001) Beurteilung klinischer Messgrößen des Qualitätsmanagements. Zentralstelle der Deutschen Ärzteschaft zur Qualitätssicherung in der Medizin. Gemeinsame Einrichtung der Bundesärztekammer und der Kassenärztlichen Bundesvereinigung. ÄZQ.Google Scholar
  2. 2.
    Arah OA, Westert GP, Hurst J, Klazinga NS (2006) A conceptual framework for the OECD Health Care Quality Indicators Project. Int J Qual Health Care 18:5–13. CrossRefPubMedGoogle Scholar
  3. 3.
    Bach S (2003) International migration of health workers: labour and social issues. Geneva Int. Labour Office. Accessed 01 Aug 2017
  4. 4.
    Brook RH (1995) The RAND/UCLA appropriateness method. Accessed 18 Sep 2016
  5. 5.
    Buchan J (2008) Wie kann die Migration von Gesundheitsfachkräften gesteuert werden, um negative Auswirkungen auf das Angebot zu verhindern? WHO, Regional Office for Europe. Accessed 01 Aug 2017
  6. 6.
    Caplin DA, Rao JK, Filloux F, Bale JF, van Orman C (2006) Development of performance indicators for the primary care management of pediatric epilepsy: expert consensus recommendations based on the available evidence: quality indicators for paediatric epilepsy. Epilepsia 47:2011–2019. CrossRefPubMedGoogle Scholar
  7. 7.
    Cheung CRLH, Gray JAM (2013) Unwarranted variation in health care for children and young people. Arch Dis Child 98:60–65. CrossRefPubMedGoogle Scholar
  8. 8.
    de Cruppé W, Kleudgen S, Diel F, Burgdorf F, Geraedts M (2015) Feasibility of 48 quality indicators in ambulatory care in Germany: a cross-sectional observational study. Z Evid Fortbild Qual Im Gesundhwes 109:682–694. CrossRefGoogle Scholar
  9. 9.
    Del Torso S, Pettoello-Mantovani M, Tenore A et al (2013) A strategic pediatric alliance for the future health of children in Europe. J Pediatr 162:659–660. CrossRefPubMedGoogle Scholar
  10. 10.
    Donabedian A (1966) Evaluating the quality of medical care. Milbank Mem Fund Q 44:166–206. CrossRefGoogle Scholar
  11. 11.
    Donaldson MS, et al (1999) Measuring the quality of health care. The National Roundtable on Health Care Quality, Institute of Medicine. National Academies Press, free downloaded from: Accessed 23 Febr 2017
  12. 12.
    Dougherty D, Schiff J, Mangione-Smith R (2011) The children’s health insurance program reauthorization act quality measures initiatives: moving forward to improve measurement, care, and child and adolescent outcomes. Acad Pediatr 11:S1–S10. CrossRefPubMedGoogle Scholar
  13. 13.
    van den Driessen Mareeuw FA, Hollegien MI, Coppus AMW, Delnoij DMJ, de Vries E (2017) In search of quality indicators for Down syndrome healthcare: a scoping review. BMC Health Serv Res 17:284. CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Engels Y, Campbell S, Dautzenberg M, van den Hombergh P, Brinkmann H, Szécsényi J, Falcoff H, Seuntjens L, Kuenzi B, Grol R, EPA Working Party (2005) Developing a framework of, and quality indicators for, general practice management in Europe. Fam Pract 22:215–222. CrossRefPubMedGoogle Scholar
  15. 15.
    Engels Y, Dautzenberg M, Campbell S, Broge B, Boffin N, Marshall M, Elwyn G, Vodopivec-Jamsek V, Gerlach FM, Samuelson M, Grol R (2006) Testing a European set of indicators for the evaluation of the management of primary care practices. Fam Pract 23:137–147. CrossRefPubMedGoogle Scholar
  16. 16.
    van Esso D, del Torso S, Hadjipanayis A, Biver A, Jaeger-Roman E, Wettergren B, Nicholson A, and the members of the Primary-Secondary Working Group (PSWG) of the European Academy of Paediatrics (EAP) (2010) Paediatric primary care in Europe: variation between countries. Arch Dis Child 95:791–795. CrossRefPubMedGoogle Scholar
  17. 17.
    Faro EZ, Wang CJ, Oyeku SO (2016) Quality Indicator development for positive screen follow-up for sickle cell disease and trait. Am J Prev Med 51:S48–S54. CrossRefPubMedGoogle Scholar
  18. 18.
    Fitch K, Bernstein SJ, Aguilar MD et al (2001) The RAND/UCLA appropriateness method user’s manual. RAND Cooperation, Santa MonicaGoogle Scholar
  19. 19.
    Gill PJ, O’Neill B, Rose P et al (2014) Primary care quality indicators for children: measuring quality in UK general practice. Br J Gen Pract 64:e752–e757. CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Hartveit M, Vanhaecht K, Thorsen O, Biringer E, Haug K, Aslaksen A (2017) Quality indicators for the referral process from primary to specialised mental health care: an explorative study in accordance with the RAND appropriateness method. BMC Health Serv Res 17:4. CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Hasenfuß G, Märker-Hermann E, Hallek M, Sieber C (2016) Klug entscheiden: Allem voran die Indikationsqualität. Internist 57:519–520. CrossRefPubMedGoogle Scholar
  22. 22.
    Hermann RC, Mattke S, Somekh D, Silfverhielm H, Goldner E, Glover G, Pirkis J, Mainz J, Chan JA (2006) Quality indicators for international benchmarking of mental health care. Int J Qual Health Care 18:31–38. CrossRefPubMedGoogle Scholar
  23. 23.
    Katz M, Rubino A, Collier J, Rosen J, Ehrich JHH (2002) Demography of pediatric primary care in Europe: delivery of care and training. Pediatrics 109:788–796CrossRefPubMedGoogle Scholar
  24. 24.
    Kelley E (2006) Beyond the initial indicators: lessons from the OECD Health Care Quality Indicators Project and the US National Healthcare Quality Report. Int J Qual Health Care 18:45–51. CrossRefPubMedGoogle Scholar
  25. 25.
    Kohn L, Corrigan J, Donaldson M (2000) To err is human: building a safer health system. National Academies Press, Washington, D.C.
  26. 26.
    Kraft E, Hersperger M, Herren D (2012) Diagnose und Indikation als Schlüsseldimensionen der Qualität. Schweiz Ärzteztg 93:1485–1489. Accessed 02 Aug 2017
  27. 27.
    Mangione-Smith R, DeCristofaro AH, Setodji CM et al (2007) The quality of ambulatory care delivered to children in the United States. N Engl J Med 357:1515–1523CrossRefPubMedGoogle Scholar
  28. 28.
    Mangione-Smith R, Schiff J, Dougherty D (2011) Identifying children’s health care quality measures for Medicaid and CHIP: an evidence-informed, publicly transparent expert process. Acad Pediatr 11:S11–S21CrossRefPubMedGoogle Scholar
  29. 29.
    Marshall M (2006) OECD Health Care Quality Indicator Project. The expert panel on primary care prevention and health promotion. Int J Qual Health Care 18:21–25. CrossRefPubMedGoogle Scholar
  30. 30.
    Mattke S, Epstein AM, Leatherman S (2006) The OECD Health Care Quality Indicators Project: history and background. Int J Qual Health Care 18:1–4. CrossRefPubMedGoogle Scholar
  31. 31.
    Mattke S, Nicolucci A, Greenfield S (2004) Selecting indicators for the quality of diabetes care at the health systems level in OECD countries. Int J Qual Health Care 18:26–30.
  32. 32.
    Mattke S, Scherer P, Expert Group Members H et al (2006) Health care quality indicators project. OECD Health Working Papers 22.
  33. 33.
    McGlynn EA, Kerr EA, Damberg CL, Asch SM (2000) Quality of care for women. Accessed 18 Sep 2016
  34. 34.
    Mercieca C, Aquilina K, Pullicino R, Borg AA (2012) Freedom of movement across the EU: legal and ethical issues for children with chronic disease. J Med Ethics 38:694–696. CrossRefPubMedGoogle Scholar
  35. 35.
    Nicolucci A, Greenfield S, Mattke S (2006) Selecting indicators for the quality of diabetes care at the health systems level in OECD countries. Int J Qual Health Care 18:26–30. CrossRefPubMedGoogle Scholar
  36. 36.
    Rigby MJ, Köhler LI, Blair ME, Metchler R (2003) Child health indicators for Europe: a priority for a caring society. Eur J Pub Health 13:38–46CrossRefGoogle Scholar
  37. 37.
    Rotar AM, van den Berg MJ, Kringos DS, Klazinga NS (2016) Reporting and use of the OECD health care quality indicators at national and regional level in 15 countries. Int J Qual Health Care J Int Soc Qual Health Care 28:398–404.
  38. 38.
    Ruiz-Canela-Cáceres J, Aquino-Llinares N, Sánchez-Diaz JM, García-Gestoso ML, de Jaime-Revuelta ME, Praena-Crespo M (2015) Indicators for childhood asthma in Spain, using the Rand method. Allergol Immunopathol (Madr) 43:147–156. CrossRefGoogle Scholar
  39. 39.
    Salzer-Muhar U, Pollak A, Aufricht C, Ehrich J, Lenton S (2012) European challenges: cross-border care for children. J Pediatr 161:574–576. CrossRefPubMedGoogle Scholar
  40. 40.
    Skrundz M, Borusiak P, Hameister KA, Geraedts M (2015) Entwicklung und Implementierung von Qualitätsindikatoren in der Sozialpädiatrie für das Krankheitsbild ADHS [The development of quality indicators for management of patients with ADHD in social paediatrics]. Gesundheitswes Bundesverb Ärzte Öffentl Gesundheitsdienstes Ger 77:908–915. CrossRefGoogle Scholar
  41. 41.
    Terracciano L, Brozek J, Compalati E, Schünemann H (2010) GRADE system: new paradigm. Curr Opin Allergy Clin Immunol 10:377–383. CrossRefPubMedGoogle Scholar
  42. 42.
    To T, Guttmann A, Lougheed MD et al (2010) Evidence-based performance indicators of primary care for asthma: a modified RAND appropriateness method. Int J Qual Health Care 22:476–485. CrossRefPubMedGoogle Scholar
  43. 43.
    UNICEF MICS. Monitoring the situation of children and women for 20 years. Accessed 23 Feb 2017
  44. 44.
  45. 45.
    van Velthoven MH, Mastellos N, Majeed A, O’Donoghue J, Car J (2016) Feasibility of extracting data from electronic medical records for research: an international comparative study. BMC Med Inform Decis Mak 16:90. CrossRefPubMedPubMedCentralGoogle Scholar
  46. 46.
    Weir SS (2005) Disease management in primary care: rapid cycle quality improvement of asthma care. NC Med J 66:219–220Google Scholar
  47. 47.
    Wolfe I, Thompson M, Gill P, Tamburlini G, Blair M, van den Bruel A, Ehrich J, Pettoello-Mantovani M, Janson S, Karanikolos M, McKee M (2013) Health services for children in western Europe. Lancet 381:1224–1234. CrossRefPubMedGoogle Scholar
  48. 48.
    Wong ICK, Ghaleb MA, Franklin BD, Barber N (2004) Incidence and nature of dosing errors in paediatric medications. Drug Saf 27:661–670. CrossRefPubMedGoogle Scholar
  49. 49.
    Wroth T, Bols J (2005) Application of quality-improvement methods in a community practice: the Sandhills pediatrics asthma initiative. NC Med J 66:218–220Google Scholar
  50. 50.
    DIRECTIVE 2011/24/EU OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 9 March 2011 on the application of patients’ rights in cross-border healthcare (2011). Accessed 23 Jan 2017
  51. 51.
  52. 52.
    REGULATION (EU) No 282/2014 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 11 March 2014 on the establishment of a third Programme for the Union's action in the field of health (2014-2020) and repealing Decision No 1350/2007/EC. EUR-Lex-32014R0282-EN-EUR-Lex. Accessed 27 Aug 2017
  53. 53.
    Welcome Note - Pediatrician. Eur Acad Paediatr Paediatr Sect. UEMS Accessed 08 Nov 2016
  54. 54.
    WHO - Children’s environmental health indicators. Accessed 18 Sep 2016

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Paediatric Primary Care CentreRegensburgGermany
  2. 2.European Association of Primary Care PaediatriciansECPCPRheinhausenGermany
  3. 3.Institute for Health Systems Research, Faculty of HealthWitten/Herdecke UniversityWittenGermany
  4. 4.Centro de Salud Virgen de ÁfricaSevilleSpain
  5. 5.Medical SchoolEuropean University CyprusNicosiaCyprus
  6. 6.Department of PaediatricsLarnaca General HospitalLarnacaCyprus
  7. 7.Institute for Health Services Research and Clinical Epidemiology, Department of MedicineUniversity of MarburgMarburgGermany

Personalised recommendations