European Journal of Clinical Pharmacology

, Volume 59, Issue 11, pp 833–840 | Cite as

Validity of performance indicators for assessing prescribing quality: the case of asthma

  • Lisa G. Pont
  • Petra Denig
  • Thys van der Molen
  • Willem Jan van der Veen
  • Flora M. Haaijer-Ruskamp
Pharmacoepidemiology and Prescription



The aim of this study was to assess the concurrent validity between the identification of sub-optimal treatment based on clinical information and computer generated indicators. Indicators that are associated with sub-optimal treatment in one of the four steps of asthma management were assessed.


The ability of each indicator to identify patients with sub-optimal asthma treatment from computerised general practitioner (GP) prescription records was assessed by comparing them with the results of an individual patient assessment using clinical data.


Chronic asthma patients (n=146) registered with 16 Dutch GPs.

Main measures

The sensitivity and positive predictive value (PPV) of each performance indicator was determined.


The step-1 indicator, focusing on patients not prescribed a short-acting β-agonist, had an acceptable sensitivity (0.86), but a low PPV (0.52). The two step-2 indicators, targeting under-prescription of inhaled corticosteroids, had sensitivities of 0.74 and 0.37 and PPVs of 0.46 and 0.71, respectively. The step-3 indicator, which targeted under-dosing of inhaled corticosteroids, had a sensitivity of 0.07 and a PPV of 0.2. The fourth indicator, focusing on under-prescription of long-acting β-agonists, could not be validated due to inadequate numbers of patients with severe asthma in our study sample.


None of the indicators investigated was considered valid for assessing prescriber performance, despite having good face and content validity. Performance indicators that have not been validated can only provide a broad-brush approach for assessing prescribing quality and should be used with extreme caution.


Quality indicators Drug therapy Asthma 



We wish to thank Annet Nicolai for her valuable assistance with the spirometry and patient assessment and Professor Dirkje Postma for her help in preparing this manuscript.


  1. 1.
    Spoeri RK, Ullman R (1997) Measuring and reporting managed care performance: lessons learned and new initiatives. Ann Intern Med 127:726–732PubMedGoogle Scholar
  2. 2.
    Brook RH, McGlynn EA, Shekelle PG (2000) Defining and measuring quality of care: a perspective from US researchers. Int J Qual Health Care 12:281–295CrossRefPubMedGoogle Scholar
  3. 3.
    Sheldon T (1998) Promoting health care quality: what role performance indicators. Qual Health Care 7[Suppl]:S45–S50Google Scholar
  4. 4.
    Poloniecki J (1998) Half of all doctors are below average. BMJ 316:1734–1736PubMedGoogle Scholar
  5. 5.
    Davis RM (1999) Measuring the performance of public health agencies. Government, like doctors and hospitals, should meet quality standards. BMJ 318:889–890PubMedGoogle Scholar
  6. 6.
    Grol R, Wensing M (2001) Implementation: effective change in patient care. (In Dutch: Implementatie: effectieve verandering in de patientenzorg.) Elsevier, MaarssenGoogle Scholar
  7. 7.
    Pringle M, Wilson T, Grol R (2002) Measuring “goodness” in individuals and healthcare systems. BMJ 325:704–707CrossRefPubMedGoogle Scholar
  8. 8.
    Dutch Association of the Research-based Pharmaceutical Industry (2002) Nefarma annual report 2000.Google Scholar
  9. 9.
    Nadzam DM (1991) Development of medication-use indicators by the Joint Commission on Accreditation of Healthcare Organizations. Am J Hosp Pharm 48:1925–1930PubMedGoogle Scholar
  10. 10.
    Rabe KF, Vermeire PA, Soriano JB, Maier WC (2000) Clinical management of asthma in 1999: the Asthma Insights and Reality in Europe (AIRE) study. Eur Respir J 16:802–807PubMedGoogle Scholar
  11. 11.
    National Institute of Health (1992) International consensus report on diagnosis and treatment of asthma. Clin Exp Allergy 22[Suppl1]:1–72Google Scholar
  12. 12.
    National Institutes of Health (1997) Expert panel report 2: Guidelines for the diagnosis and management of asthma. NIH publication number: 97–4051. National Institutes of Health, BethesdaGoogle Scholar
  13. 13.
    National Institutes of Health. (1995) Global strategy for asthma management and for asthma management and prevention NHLBI/WHO workshop. NIH publication number: 95–3659. NHLBI/WHO, BethesdaGoogle Scholar
  14. 14.
    Aveyard P (1997) Assessing the performance of general practices caring for patients with asthma. Br J Gen Pract 47:423–426PubMedGoogle Scholar
  15. 15.
    Shelley M, Croft P, Chapman S, Pantin C (2000) Is the quality of asthma prescribing, as measured by the general practice ratio of corticosteroid to bronchodilator, associated with asthma morbidity? J Clin Epidemiol 53:1217–1221CrossRefPubMedGoogle Scholar
  16. 16.
    Hallas J, Hansen NC (1993) Individual utilization of anti-asthma medication by young adults: a prescription database analysis. J Intern Med 234:65–70PubMedGoogle Scholar
  17. 17.
    Legorreta AP, Christian Herman J, O’Connor RD, Hasan MM, Evans R, Leung KM (1998) Compliance with national asthma management guidelines and specialty care: a health maintenance organization experience. Arch Intern Med 158:457–464CrossRefGoogle Scholar
  18. 18.
    Lagerlov P, Veninga CC, Muskova M, Hummers-Pradier E, Stalsby LC, Andrew M, Haaijer-Ruskamp FM (2000) Asthma management in five European countries: doctors’ knowledge, attitudes and prescribing behaviour. Drug Education Project (DEP) group. Eur Respir J 15:25–29CrossRefPubMedGoogle Scholar
  19. 19.
    Veninga CC, Denig P, Pont LG, Haaijer-Ruskamp FM (2001) Comparison of indicators assessing the quality of drug prescribing for asthma. Health Serv Res 36:143–161PubMedGoogle Scholar
  20. 20.
    Wahlstrom R, Lagerlov P, Lundborg CS, Veninga CCM, Hummers-Pradier E, Dahlgren LO, Denig P (2001) Variations in general practitioners’ views of asthma management in four European countries. Soc Sci Med 53:507–518CrossRefPubMedGoogle Scholar
  21. 21.
    Lagerlov P, Loeb M, Andrew M, Hjortdahl P (2000) Improving doctors’ prescribing behaviour through reflection on guidelines and prescription feedback: a randomised controlled study. Qual Health Care 9:159–165PubMedGoogle Scholar
  22. 22.
    Gaist D, Hallas J, Hansen NC, Gram LF (1996) Are young adults with asthma treated sufficiently with inhaled steroids? A population-based study of prescription data from 1991 and 1994. Br J Clin Pharmacol 41:285–289PubMedGoogle Scholar
  23. 23.
    Veninga CC, Lagerlov P, Wahlstrom R, Muskova M, Denig P, Berkhof J, Kochen MM, Haaijer-Ruskamp FM (1999) Evaluating an educational intervention to improve the treatment of asthma in four European countries. Drug Education Project Group. Am J Respir Crit Care Med 160:1254–1262PubMedGoogle Scholar
  24. 24.
    Veninga CC, Denig P, Zwaagstra R, Haaijer-Ruskamp FM (2000) Improving drug treatment in general practice. J Clin Epidemiol 53:762–772CrossRefGoogle Scholar
  25. 25.
    WHO Collaborating Centre for Drug Statistics Methodology. (1999) ATC index with DDDs. World Health Organization, OlsoGoogle Scholar
  26. 26.
    World organisation of family Doctors (WONCA) (2000) ICPC-2: international classification of primary care, 2nd edn. Oxford Medical Publications, OxfordGoogle Scholar
  27. 27.
    Bland M (2000) An introduction to medical statistics, 3rd edn. Oxford University Press, OxfordGoogle Scholar
  28. 28.
    Altman DG (1997) Practical statistics for medical research. Chapman & Hall, LondonGoogle Scholar
  29. 29.
    Pont LG, van der Werf GT, Denig P, Haaijer-Ruskamp FM (2002) Identifying general practice patients diagnosed with asthma and their exacerbation episodes from prescribing data Eur J Clin Pharm 57:819–825CrossRefGoogle Scholar
  30. 30.
    de Jong-van den Berg LT, Waardenburg CM, Haaijer-Ruskamp FM, Dukes MN, Wesseling H (1993) Drug use in pregnancy: a comparative appraisal of data collecting methods. Eur J Clin Pharmacol 45:9–14PubMedGoogle Scholar
  31. 31.
    Van den Brandt PA, Petri H, Dorant E, Goldbohm RA, Van de Crommert S (1991) Comparison of questionnaire information and pharmacy data on drug use. Pharm Weekbl Sci 13:91–96PubMedGoogle Scholar
  32. 32.
    West SL, Savitz DA, Koch G, Strom BL, Guess HA, Hartzema A (1995) Recall accuracy for prescription medications: self-report compared with database information. Am J Epidemiol 142:1103–1112PubMedGoogle Scholar
  33. 33.
    Cantrill JA, Sibbald B, Buetow SA (1998) Indicators of the appropriateness of long term prescribing in general practice in the United Kingdom: consensus development, face and content validity, feasibility, and reliability. Qual Health Care 7:130–135PubMedGoogle Scholar
  34. 34.
    Campbell SM, Cantrill JA, Roberts D (2000) Prescribing indicators for UK general practice: Delphi consultation study. BMJ 321:425–428CrossRefPubMedGoogle Scholar
  35. 35.
    Hanlon JT, Schmader KE, Samsa GP, Weinberger M, Uttech KM, Lewis IK, Cohen HJ, Feussner JR (1992) A method for assessing drug therapy appropriateness. J Clin Epidemiol 45:1045–1051PubMedGoogle Scholar

Copyright information

© Springer-Verlag 2004

Authors and Affiliations

  • Lisa G. Pont
    • 1
  • Petra Denig
    • 2
  • Thys van der Molen
    • 3
  • Willem Jan van der Veen
    • 4
  • Flora M. Haaijer-Ruskamp
    • 5
  1. 1.Department of Clinical PharmacologyUniversity of GroningenThe Netherlands
  2. 2.Department of Clinical PharmacologyUniversity of GroningenThe Netherlands
  3. 3.Department of General PracticeUniversity of GroningenThe Netherlands
  4. 4.Registration Network Groningen, Department of General PracticeUniversity of GroningenThe Netherlands
  5. 5.Department of Clinical PharmacologyUniversity of GroningenGroningenThe Netherlands

Personalised recommendations