European Journal of Pediatrics

, Volume 178, Issue 5, pp 707–719 | Cite as

Feasibility testing of the Core set of quality Indicators for Paediatric Primary Care in Europe, COSI-PPC-EU

  • Dominik A. EwaldEmail author
  • Gottfried Huss
  • Rike Antje Kraska
  • Max Geraedts
Original Article


There is a need to measure and improve the quality of paediatric primary care in Europe where major differences in the delivery and outcomes of child health care exist. A collaborative panel of paediatric senior experts developed a Core Set of Indicators for Paediatric Primary Care in Europe by compiling 42 quality indicators in a modified consensus process following the RAND/UCLA appropriateness method. The aim of this study was to explore the feasibility of the quality indicator set in European paediatric primary care practices. Seventy-nine practices from eight countries participated in a detailed online interview. The practices rated the applicability, relevance, reliability and acceptance of the 42 quality indicator as well as the availability, technical feasibility and effort to retrieve the needed data from their medical records. Most quality indicators were considered applicable, available, reliable, acceptable and relevant for monitoring quality of care in paediatric primary care. Respondents rated feasibility and effort to retrieve the data lowest because of difficulties collecting the data from the medical records.

Conclusion: European paediatric primary care practices generally agree with the proposed quality indicator set. They document most of the parameters. However, the collection of specific needed values from available routine patient-data is considered technically difficult and time-consuming.

What is Known?

Paediatric primary care systems in Europe show striking differences in their performance. Pre-existing sets of quality indicators are predominantly limited to national populations, specific diseases and hospital care.

A Core Set of 42 quality indicators for paediatric primary care in Europe was developed by European paediatricians using a systematic literature review and a consensus process following a modified RAND/UCLA appropriateness method.

What is New?

Paediatric primary care providers in Europe agree with the idea to use COSI-PPC-EU to monitor and improve the quality of care. The set was considered applicable, available, reliable, acceptable, and relevant for quality improvement.

The score for feasibility and effort to retrieve the data was low, because of technical reasons; the electronical or paper-based medical documentation in most cases does not allow convenient access to all necessary data.


Quality indicator Quality improvement Paediatric primary care Community child care Europe Benchmarking EAP ECPCP Feasibility study 



Continuing medical education


Core Set of Quality Indicators for Paediatric Primary Care in Europe


German Academic Society for General Paediatrics


European Academy of Paediatrics


European Confederation of Primary Care Paediatricians


Electronic Data Processing


Electronic Medical Records


European Union


SPSS Statistics is a software package used for statistical analysis from the IBM Corporation


International Classification of Disease


Models of Child Health Appraised Project


National Institute for Health and Care Excellence, UK


Paediatric Primary Care


Quality Indicator


Quality Management


RAND/University of California Los Angeles Appropriateness Method



We thank the country coordinators which selected and motivated the study participants: Andreas Werner and Liliane Cret (France), Folkert Fehr (Germany), Peter Altorjaj and Arkos Kovacs (Hungary), Shimon Barak and Jakob Urkin (Israel), Laura Reali and Patrizia Calamita (Italy), Beata Kartousova (Slovakia), Denis Baš and Margareta Seher Zupancic (Slovenia), Juan Ruiz Canela and Angel Saenz Carrasco (Spain), Daniela Kasparek and Wilhelm Sedlack (Austria), Hadjigeorgiou Charis (Cyprus), Gabriela Kubatova (Czech Republic), Pirkko Keronen (Finland), Iveta Skurvydiene (Lithuania) and Mario Schumacher (Switzerland). We thank all 79 participating paediatricians for their invested time and engagement, as well as all other country delegates and members from EAP, ECPCP and German Academic Society for General Paediatrics (DGAAP), who supported the study.

Authors’ contributions

Dominik A. Ewald acted as European study coordinator and drafted the manuscript. Gottfried Huss acted as project initiator and European study coordinator. Rike Antje Kraska programmed the questionnaire and performed the statistical analyses. Max Geraedts was the principle scientific study coordinator who coordinated the development of the questionnaire, the data processing and analyses.


The scientific evaluation of the COSI project by Max Geraedts and Rike A. Kraska was financed by a grant from the “Foundation Child and Youth”, Cologne, and by European Confederation of Primary Care Paediatricians (ECPCP) and German Association of Primary Care Paediatrics (DGAAP).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interests.

Informed consent

All authors revised the manuscript for important intellectual content and approved the final version before submission.

Supplementary material

431_2019_3344_MOESM1_ESM.docx (25 kb)
ESM 1 (DOCX 24 kb)


  1. 1.
    Bennett B, Coventry E, Greenway N, Minchin M (2014) The NICE process for developing quality standards and indicators. Z Für Evidenz Fortbild Qual Im Gesundheitswesen 108:481–486. CrossRefGoogle Scholar
  2. 2.
    Beyer M, Chenot R, Erler A, Gerlach FM (2011) Die Darstellung der hausärztlichen Versorgungsqualität durch Qualitätsindikatoren. Z Für Evidenz Fortbild Qual Im Gesundheitswesen 105:13–20. CrossRefGoogle Scholar
  3. 3.
    Campbell SM, Braspenning J, Hutchinson A, Marshall M (2002) Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care 11:358–364. CrossRefGoogle Scholar
  4. 4.
    Cheung CRLH, Gray JAM (2013) Unwarranted variation in health care for children and young people. Arch Dis Child 98:60–65. CrossRefGoogle Scholar
  5. 5.
    de Cruppé W, Kleudgen S, Diel F, Burgdorf F, Geraedts M (2015) Feasibility of 48 quality indicators in ambulatory care in Germany: a cross-sectional observational study. Z Für Evidenz Fortbild Qual Im Gesundheitswesen 109:682–694. CrossRefGoogle Scholar
  6. 6.
    de Cruppé W, Nguyen BH, Weissenrieder N et al (2010) Qualitätsmanagement in kinder- und jugendärztlichen Praxen. Monatsschr Kinderheilkd 159:145–151. CrossRefGoogle Scholar
  7. 7.
    Donabedian A (1966) Evaluating the quality of medical care. Milbank Mem Fund Q 44:166–206. CrossRefGoogle Scholar
  8. 8.
    Ehrich J, Namazova-Baranova L, Pettoello-Mantovani M (2016) Introduction to “diversity of child health Care in Europe: a study of the European Paediatric association/Union of National European Paediatric Societies and Associations”. J Pediatr 177:S1–S10. CrossRefGoogle Scholar
  9. 9.
    Engels Y, Dautzenberg M, Campbell S, Broge B, Boffin N, Marshall M, Elwyn G, Vodopivec-Jamsek V, Gerlach FM, Samuelson M, Grol R (2006) Testing a European set of indicators for the evaluation of the management of primary care practices. Fam Pract 23:137–147. CrossRefGoogle Scholar
  10. 10.
    van Esso D, del Torso S, Hadjipanayis A, Biver A, Jaeger-Roman E, Wettergren B, Nicholson A, and the members of the Primary-Secondary Working Group (PSWG) of the European Academy of Paediatrics (EAP) (2010) Paediatric primary care in Europe: variation between countries. Arch Dis Child 95:791–795. CrossRefGoogle Scholar
  11. 11.
    European Commission, Directorate General for Health & Consumers (2014) The third Health Programme 2014–2020 funding health initiatives. European Commission, BrusselsGoogle Scholar
  12. 12.
    Ewald DA, Huss G, Auras S, Caceres JRC, Hadjipanayis A, Geraedts M (2018) Development of a core set of quality indicators for paediatric primary care practices in Europe, COSI-PPC-EU. Eur J Pediatr 177:921–933. CrossRefGoogle Scholar
  13. 13.
    Fitch K (2001) The Rand/UCLA appropriateness method user’s manual. Rand, Santa MonicaGoogle Scholar
  14. 14.
    Gill PJ, O’Neill B, Rose P, Mant D, Harnden A (2014) Primary care quality indicators for children: measuring quality in UK general practice. Br J Gen Pract 64:e752–e757. CrossRefGoogle Scholar
  15. 15.
    Gold R, Angier H, Mangione-Smith R, Gallia C, McIntire PJ, Cowburn S, Tillotson C, DeVoe JE (2012) Feasibility of evaluating the CHIPRA care quality measures in electronic health record data. Pediatrics 130:139–149. CrossRefGoogle Scholar
  16. 16.
    Hartveit M, Vanhaecht K, Thorsen O, Biringer E, Haug K, Aslaksen A (2017) Quality indicators for the referral process from primary to specialised mental health care: an explorative study in accordance with the RAND appropriateness method. BMC Health Serv Res 17:4. CrossRefGoogle Scholar
  17. 17.
    Holmboe ES, Weng W, Arnold GK, Kaplan SH, Normand SL, Greenfield S, Hood S, Lipner RS (2010) The comprehensive care project: measuring physician performance in ambulatory practice. Health Serv Res 45:1912–1933. CrossRefGoogle Scholar
  18. 18.
    Horodecka AM, Wolniak R (2015) Uwarunkowania rynku włoskiego w zakresie implementacji SZJ norm ISO 9000. Probl Jakości 1:22–28. Google Scholar
  19. 19.
    Kötter T, Blozik E, Scherer M (2012) Methods for the guideline-based development of quality indicators--a systematic review. Implement Sci 7:21. CrossRefGoogle Scholar
  20. 20.
    Mainz J (2003) Developing evidence-based clinical indicators: a state of the art methods primer. Int J Qual Health Care 15:5i–11i.
  21. 21.
    Mainz J (2004) Quality indicators: essential for quality improvement. Int J Qual Health Care 16:i1–i2. CrossRefGoogle Scholar
  22. 22.
    Mangione-Smith R, Roth CP, Britto MT, Chen AY, McGalliard J, Boat TF, Adams JL, McGlynn EA (2017) Development and testing of the pediatric respiratory illness measurement system (PRIMES) quality indicators. Hosp Pediatr 7:125–133. CrossRefGoogle Scholar
  23. 23.
    Mangione-Smith R, Schiff J, Dougherty D (2011) Identifying children’s health care quality measures for Medicaid and CHIP: an evidence-informed, publicly transparent expert process. Acad Pediatr 11:S11–S21CrossRefGoogle Scholar
  24. 24.
    Marshall M, Roland MO, Campbell SM et al (2003) Measuring general practice. Demonstr. Proj. Dev. Test set prim. Care Clin. Qual. Indic. Lond. Nuffield trustGoogle Scholar
  25. 25.
    Marshall MN, Shekelle PG, McGlynn EA, Campbell S, Brook RH, Roland MO (2003) Can health care quality indicators be transferred between countries? Qual Saf Health Care 12:8–12. CrossRefGoogle Scholar
  26. 26.
    Mattke S, Nicolucci A, Greenfield S (2004) Selecting Indicators for the Quality of Diabetes Care at the Health Systems Level in OECD Countries. doi:
  27. 27.
    McGlynn EA (1998) Choosing and evaluating clinical performance measures. Jt Comm J Qual Improv 24:470–479. Google Scholar
  28. 28.
    Peña A, Virk SS, Shewchuk RM et al (2010) Validity versus feasibility for quality of care indicators: expert panel results from the MI-plus study. Int J Qual Health Care 22:201–209. CrossRefGoogle Scholar
  29. 29.
    Rubin HR, Pronovost P, Diette GB (2001) Methodology matters. From a process of care to a measure: the development and testing of a quality indicator. Int J Qual Health Care 13:489–496. CrossRefGoogle Scholar
  30. 30.
    Salzer-Muhar U, Pollak A, Aufricht C, Ehrich J, Lenton S (2012) European challenges: cross-border care for children. J Pediatr 161:574–576. CrossRefGoogle Scholar
  31. 31.
    Stang AS, Straus SE, Crotts J et al (2013) Quality Indicators for High Acuity Pediatric Conditions. Pediatrics peds 132(4):752–762 2013–0854. CrossRefGoogle Scholar
  32. 32.
    Thern J, de With K, Strauss R et al (2014) Selection of hospital antimicrobial prescribing quality indicators: a consensus among German antibiotic stewardship (ABS) networkers. Infection 42:351–362. CrossRefGoogle Scholar
  33. 33.
    To T, Guttmann A, Lougheed MD et al (2010) Evidence-based performance indicators of primary care for asthma: a modified RAND appropriateness method. Int J Qual Health Care 22:476–485. CrossRefGoogle Scholar
  34. 34.
    Wolfe I, Thompson M, Gill P, Tamburlini G, Blair M, van den Bruel A, Ehrich J, Pettoello-Mantovani M, Janson S, Karanikolos M, McKee M (2013) Health services for children in western Europe. Lancet 381:1224–1234. CrossRefGoogle Scholar
  35. 35.
    Wollersheim H, Hermens R, Hulscher M, Braspenning J, Ouwens M, Schouten J, Marres H, Dijkstra R, Grol R (2007) Clinical indicators: development and applications. Neth J Med 65:15–22Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Paediatric Primary Care CentreRegensburgGermany
  2. 2.European Association of Primary Care Paediatricians, ECPCPRheinfeldenGermany
  3. 3.Institute for Health Systems Research, Faculty of HealthWitten/Herdecke UniversityWittenGermany
  4. 4.Institute for Health Services Research and Clinical Epidemiology, Faculty of MedicineUniversity of MarburgMarburgGermany

Personalised recommendations