Development of a core set of quality indicators for paediatric primary care practices in Europe, COSI-PPC-EU
Paediatric ambulatory healthcare systems in Europe are, because of historical reasons, diverse and show strikingly different outcomes. All across Europe, the benchmarking of structures, processes and outcomes could reveal opportunities for improving Paediatric Primary Care (PPC). The aim of this study was to develop a set of Quality Indicators (QIs) to assess and monitor PPC in Europe. In a three-step process, we used the available external evidence and European expert consensus in a modified RAND/UCLA Appropriateness Method (RAM) to develop an indicator set. (1) A broad literature and online research of published QI and guidelines yielded an inventory of 1516 QI. (2) A collaborative panel of paediatric senior experts from the European Academy of Paediatrics (EAP) and the European Confederation of Primary Care Paediatricians (ECPCP) from 15 European countries participated in a first consensus process to reduce the initial indicator inventory by eliminating not PPC-focused indicators and duplicates. (3) In a second consensus process, the panel rated the QI regarding validity and feasibility. The final QI set “COSI-PPC-EU” consists of 42 indicators in five categories of PPC: (A) health promotion/prevention/screening (13 QI), (B) acute care (9 QI), (C) chronic care (8 QI), (D) practice management (3 QI) and (E) patient safety (9 QI).
What is Known:
• Paediatric ambulatory healthcare systems in Europe are diverse and show strikingly different outcomes.
• There are known gaps in quality performance measures of paediatric primary care in Europe. Pre-existing sets of quality indicators are predominantly limited to national populations, specific diseases and hospital care.
What is New:
• A set of 42 quality indicators for primary paediatric care in Europe was developed in a multi-country collaborative effort. The method combined a systematic literature review and a consensus process among European paediatric experts.
• The quality indicator set can facilitate quality improvement of PPC. After studying the feasibility, providers can use COSI-PPC-EU to monitor, compare and improve performance of practices, regions and countries.
KeywordsQuality indicator Quality improvement Quality management Paediatric primary care Community child care Europe Benchmarking EAP ECPCP RAND/UCLA appropriateness method
Core set of quality indicators
Core set of quality indicators for Paediatric primary care practices in Europe
German Academic Society for General Paediatrics
European Academy of Paediatrics
European Confederation of Primary Care Paediatricians
Grading of Recommendations Assessment, Development and Evaluation, UK
National Health Service, UK
National Institute for Health and Care Excellence, UK
Organisation for Economic Co-operation and Development
Paediatric Primary Care
RAND/UCLA appropriateness method
Union of European Medical Specialists
United Nations International Children’s Emergency Fund
World Health Organization
We thank the members of the panels for evaluating and rating the indicators: Angel Carrasco Sanz, Björn Wettergren, Elke Jäger-Roman, Arunas Valiulis, Peter Altorjai, Indre Butienne from EAP and several other country delegates, as well as members from EAP, ECPCP and DGAAP.
Dominik A. Ewald and Gottfried Huss acted as European and national study coordinators from 2011 to 2013 and wrote the main part of the manuscript. Silke Auras prepared a draft of the methods and results sections. Juan Ruiz Canela was responsible for the literature research and screened with Gottfried Huss the scientific literature. Adamos Hadjipanayis is the secretary general of the EAP; he reviewed the manuscript and gave valuable advice, and he supported the project through all the years. Max Geraedts was the principal investigator who coordinated the evidence reviews, the foundation of the database and the statistical evaluation of the rating process. All the authors revised the manuscript before submission.
The scientific evaluation of the COSI project by Max Geraedts and Silke Auras was financed by a grant from the European Academy of Paediatrics, EAP. The literature research was possible with the financial support from the Spanish Paediatric Society AEP.
None of the panellists received travel or expense allowance especially for COSI-PPC-EU since the meetings took place at regular meetings of their organizations.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
Compliance with ethical standards
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent was not necessary to obtain from individual participants, because there were none included in the study.
- 1.Altenhofen L, Brech W, Brenner G, Geraedts G, Gramsch E, Kolkmann FW, Rheinberger P (2001) Beurteilung klinischer Messgrößen des Qualitätsmanagements. Zentralstelle der Deutschen Ärzteschaft zur Qualitätssicherung in der Medizin. Gemeinsame Einrichtung der Bundesärztekammer und der Kassenärztlichen Bundesvereinigung. ÄZQ.Google Scholar
- 3.Bach S (2003) International migration of health workers: labour and social issues. Geneva Int. Labour Office. https://www.aspeninstitute.org/content/uploads/files/content/images/Bach%20Health%20worker%20Migration%20WP.pdf. Accessed 01 Aug 2017
- 4.Brook RH (1995) The RAND/UCLA appropriateness method. https://www.rand.org/pubs/reprints/RP395.html. Accessed 18 Sep 2016
- 5.Buchan J (2008) Wie kann die Migration von Gesundheitsfachkräften gesteuert werden, um negative Auswirkungen auf das Angebot zu verhindern? WHO, Regional Office for Europe. http://www.euro.who.int/__data/assets/pdf_file/0008/76427/E93414G.pdf. Accessed 01 Aug 2017
- 6.Caplin DA, Rao JK, Filloux F, Bale JF, van Orman C (2006) Development of performance indicators for the primary care management of pediatric epilepsy: expert consensus recommendations based on the available evidence: quality indicators for paediatric epilepsy. Epilepsia 47:2011–2019. https://doi.org/10.1111/j.1528-1167.2006.00853.x CrossRefPubMedGoogle Scholar
- 11.Donaldson MS, et al (1999) Measuring the quality of health care. The National Roundtable on Health Care Quality, Institute of Medicine. National Academies Press, free downloaded from: http://www.nap.edu/catalog/6418.html. Accessed 23 Febr 2017
- 12.Dougherty D, Schiff J, Mangione-Smith R (2011) The children’s health insurance program reauthorization act quality measures initiatives: moving forward to improve measurement, care, and child and adolescent outcomes. Acad Pediatr 11:S1–S10. https://doi.org/10.1016/j.acap.2011.02.009 CrossRefPubMedGoogle Scholar
- 14.Engels Y, Campbell S, Dautzenberg M, van den Hombergh P, Brinkmann H, Szécsényi J, Falcoff H, Seuntjens L, Kuenzi B, Grol R, EPA Working Party (2005) Developing a framework of, and quality indicators for, general practice management in Europe. Fam Pract 22:215–222. https://doi.org/10.1093/fampra/cmi002 CrossRefPubMedGoogle Scholar
- 15.Engels Y, Dautzenberg M, Campbell S, Broge B, Boffin N, Marshall M, Elwyn G, Vodopivec-Jamsek V, Gerlach FM, Samuelson M, Grol R (2006) Testing a European set of indicators for the evaluation of the management of primary care practices. Fam Pract 23:137–147. https://doi.org/10.1093/fampra/cmi091 CrossRefPubMedGoogle Scholar
- 16.van Esso D, del Torso S, Hadjipanayis A, Biver A, Jaeger-Roman E, Wettergren B, Nicholson A, and the members of the Primary-Secondary Working Group (PSWG) of the European Academy of Paediatrics (EAP) (2010) Paediatric primary care in Europe: variation between countries. Arch Dis Child 95:791–795. https://doi.org/10.1136/adc.2009.178459 CrossRefPubMedGoogle Scholar
- 18.Fitch K, Bernstein SJ, Aguilar MD et al (2001) The RAND/UCLA appropriateness method user’s manual. RAND Cooperation, Santa MonicaGoogle Scholar
- 20.Hartveit M, Vanhaecht K, Thorsen O, Biringer E, Haug K, Aslaksen A (2017) Quality indicators for the referral process from primary to specialised mental health care: an explorative study in accordance with the RAND appropriateness method. BMC Health Serv Res 17:4. https://doi.org/10.1186/s12913-016-1941-1 CrossRefPubMedPubMedCentralGoogle Scholar
- 25.Kohn L, Corrigan J, Donaldson M (2000) To err is human: building a safer health system. National Academies Press, Washington, D.C. https://doi.org/10.17226/9728
- 26.Kraft E, Hersperger M, Herren D (2012) Diagnose und Indikation als Schlüsseldimensionen der Qualität. Schweiz Ärzteztg 93:1485–1489. http://r4.fmh.ch/files/pdf8/GLP_Diagnose_und_Indikation_als_Schlsseldimension_der_Qualitt_SAEZ_Nr._41_20122.pdf. Accessed 02 Aug 2017
- 31.Mattke S, Nicolucci A, Greenfield S (2004) Selecting indicators for the quality of diabetes care at the health systems level in OECD countries. Int J Qual Health Care 18:26–30. https://doi.org/10.1787/165531523300
- 32.Mattke S, Scherer P, Expert Group Members H et al (2006) Health care quality indicators project. OECD Health Working Papers 22. https://doi.org/10.1787/481685177056
- 33.McGlynn EA, Kerr EA, Damberg CL, Asch SM (2000) Quality of care for women. http://www.rand.org/pubs/monograph_reports/MR1284.html. Accessed 18 Sep 2016
- 37.Rotar AM, van den Berg MJ, Kringos DS, Klazinga NS (2016) Reporting and use of the OECD health care quality indicators at national and regional level in 15 countries. Int J Qual Health Care J Int Soc Qual Health Care 28:398–404. https://doi.org/10.1093/intqhc/mzw027
- 40.Skrundz M, Borusiak P, Hameister KA, Geraedts M (2015) Entwicklung und Implementierung von Qualitätsindikatoren in der Sozialpädiatrie für das Krankheitsbild ADHS [The development of quality indicators for management of patients with ADHD in social paediatrics]. Gesundheitswes Bundesverb Ärzte Öffentl Gesundheitsdienstes Ger 77:908–915. https://doi.org/10.1055/s-0034-1389922 CrossRefGoogle Scholar
- 43.UNICEF MICS. Monitoring the situation of children and women for 20 years. http://18.104.22.168/files?job=W1siZiIsIjIwMTUvMDkvMTQvMTcvNTUvMzcvNTI2LzIwMTUwOTEyX01JQ1MyMF9XRUIucGRmIl1d&sha=da0e0b8ac785c628. Accessed 23 Feb 2017
- 44.UNICEF MICS. Definitions of indicators. http://mics.unicef.org/files?job=W1siZiIsIjIwMTUvMDQvMDMvMDYvNDIvNDcvOTYvYXBwZW5kNC5wZGYiXV0&sha=f8f48d64e09eb591. Accessed 23 Feb 2017
- 46.Weir SS (2005) Disease management in primary care: rapid cycle quality improvement of asthma care. NC Med J 66:219–220Google Scholar
- 49.Wroth T, Bols J (2005) Application of quality-improvement methods in a community practice: the Sandhills pediatrics asthma initiative. NC Med J 66:218–220Google Scholar
- 50.DIRECTIVE 2011/24/EU OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 9 March 2011 on the application of patients’ rights in cross-border healthcare (2011). http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2011:088:0045:0065:EN:PDF. Accessed 23 Jan 2017
- 51.Recht auf Behandlung. http://europa.eu/youreurope/citizens/health/planned-healthcare/right-to-treatment/index_de.htm. Accessed 27 Aug 2017
- 52.REGULATION (EU) No 282/2014 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 11 March 2014 on the establishment of a third Programme for the Union's action in the field of health (2014-2020) and repealing Decision No 1350/2007/EC. EUR-Lex-32014R0282-EN-EUR-Lex. http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.L_.2014.086.01.0001.01.ENG. Accessed 27 Aug 2017
- 53.Welcome Note - Pediatrician. Eur Acad Paediatr Paediatr Sect. UEMS http://eapaediatrics.eu/welcome-note/. Accessed 08 Nov 2016
- 54.WHO - Children’s environmental health indicators. http://www.who.int/ceh/publications/cehi_brochure/en/. Accessed 18 Sep 2016