Skip to main content
SpringerLink
Log in

Validity of performance indicators for assessing prescribing quality: the case of asthma

  • Pharmacoepidemiology and Prescription
  • Published:
European Journal of Clinical Pharmacology Aims and scope Submit manuscript

Abstract

Objectives

The aim of this study was to assess the concurrent validity between the identification of sub-optimal treatment based on clinical information and computer generated indicators. Indicators that are associated with sub-optimal treatment in one of the four steps of asthma management were assessed.

Design

The ability of each indicator to identify patients with sub-optimal asthma treatment from computerised general practitioner (GP) prescription records was assessed by comparing them with the results of an individual patient assessment using clinical data.

Setting

Chronic asthma patients (n=146) registered with 16 Dutch GPs.

Main measures

The sensitivity and positive predictive value (PPV) of each performance indicator was determined.

Results

The step-1 indicator, focusing on patients not prescribed a short-acting β-agonist, had an acceptable sensitivity (0.86), but a low PPV (0.52). The two step-2 indicators, targeting under-prescription of inhaled corticosteroids, had sensitivities of 0.74 and 0.37 and PPVs of 0.46 and 0.71, respectively. The step-3 indicator, which targeted under-dosing of inhaled corticosteroids, had a sensitivity of 0.07 and a PPV of 0.2. The fourth indicator, focusing on under-prescription of long-acting β-agonists, could not be validated due to inadequate numbers of patients with severe asthma in our study sample.

Discussion

None of the indicators investigated was considered valid for assessing prescriber performance, despite having good face and content validity. Performance indicators that have not been validated can only provide a broad-brush approach for assessing prescribing quality and should be used with extreme caution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Spoeri RK, Ullman R (1997) Measuring and reporting managed care performance: lessons learned and new initiatives. Ann Intern Med 127:726–732

    PubMed  Google Scholar 

  2. Brook RH, McGlynn EA, Shekelle PG (2000) Defining and measuring quality of care: a perspective from US researchers. Int J Qual Health Care 12:281–295

    Article  PubMed  Google Scholar 

  3. Sheldon T (1998) Promoting health care quality: what role performance indicators. Qual Health Care 7[Suppl]:S45–S50

  4. Poloniecki J (1998) Half of all doctors are below average. BMJ 316:1734–1736

    CAS  PubMed  Google Scholar 

  5. Davis RM (1999) Measuring the performance of public health agencies. Government, like doctors and hospitals, should meet quality standards. BMJ 318:889–890

    PubMed  Google Scholar 

  6. Grol R, Wensing M (2001) Implementation: effective change in patient care. (In Dutch: Implementatie: effectieve verandering in de patientenzorg.) Elsevier, Maarssen

  7. Pringle M, Wilson T, Grol R (2002) Measuring “goodness” in individuals and healthcare systems. BMJ 325:704–707

    Article  PubMed  Google Scholar 

  8. Dutch Association of the Research-based Pharmaceutical Industry (2002) Nefarma annual report 2000.

  9. Nadzam DM (1991) Development of medication-use indicators by the Joint Commission on Accreditation of Healthcare Organizations. Am J Hosp Pharm 48:1925–1930

    CAS  PubMed  Google Scholar 

  10. Rabe KF, Vermeire PA, Soriano JB, Maier WC (2000) Clinical management of asthma in 1999: the Asthma Insights and Reality in Europe (AIRE) study. Eur Respir J 16:802–807

    CAS  PubMed  Google Scholar 

  11. National Institute of Health (1992) International consensus report on diagnosis and treatment of asthma. Clin Exp Allergy 22[Suppl1]:1–72

  12. National Institutes of Health (1997) Expert panel report 2: Guidelines for the diagnosis and management of asthma. NIH publication number: 97–4051. National Institutes of Health, Bethesda

    Google Scholar 

  13. National Institutes of Health. (1995) Global strategy for asthma management and for asthma management and prevention NHLBI/WHO workshop. NIH publication number: 95–3659. NHLBI/WHO, Bethesda

    Google Scholar 

  14. Aveyard P (1997) Assessing the performance of general practices caring for patients with asthma. Br J Gen Pract 47:423–426

    CAS  PubMed  Google Scholar 

  15. Shelley M, Croft P, Chapman S, Pantin C (2000) Is the quality of asthma prescribing, as measured by the general practice ratio of corticosteroid to bronchodilator, associated with asthma morbidity? J Clin Epidemiol 53:1217–1221

    Article  PubMed  Google Scholar 

  16. Hallas J, Hansen NC (1993) Individual utilization of anti-asthma medication by young adults: a prescription database analysis. J Intern Med 234:65–70

    CAS  PubMed  Google Scholar 

  17. Legorreta AP, Christian Herman J, O’Connor RD, Hasan MM, Evans R, Leung KM (1998) Compliance with national asthma management guidelines and specialty care: a health maintenance organization experience. Arch Intern Med 158:457–464

    Article  Google Scholar 

  18. Lagerlov P, Veninga CC, Muskova M, Hummers-Pradier E, Stalsby LC, Andrew M, Haaijer-Ruskamp FM (2000) Asthma management in five European countries: doctors’ knowledge, attitudes and prescribing behaviour. Drug Education Project (DEP) group. Eur Respir J 15:25–29

    Article  PubMed  Google Scholar 

  19. Veninga CC, Denig P, Pont LG, Haaijer-Ruskamp FM (2001) Comparison of indicators assessing the quality of drug prescribing for asthma. Health Serv Res 36:143–161

    PubMed  Google Scholar 

  20. Wahlstrom R, Lagerlov P, Lundborg CS, Veninga CCM, Hummers-Pradier E, Dahlgren LO, Denig P (2001) Variations in general practitioners’ views of asthma management in four European countries. Soc Sci Med 53:507–518

    Article  PubMed  Google Scholar 

  21. Lagerlov P, Loeb M, Andrew M, Hjortdahl P (2000) Improving doctors’ prescribing behaviour through reflection on guidelines and prescription feedback: a randomised controlled study. Qual Health Care 9:159–165

    CAS  PubMed  Google Scholar 

  22. Gaist D, Hallas J, Hansen NC, Gram LF (1996) Are young adults with asthma treated sufficiently with inhaled steroids? A population-based study of prescription data from 1991 and 1994. Br J Clin Pharmacol 41:285–289

    CAS  PubMed  Google Scholar 

  23. Veninga CC, Lagerlov P, Wahlstrom R, Muskova M, Denig P, Berkhof J, Kochen MM, Haaijer-Ruskamp FM (1999) Evaluating an educational intervention to improve the treatment of asthma in four European countries. Drug Education Project Group. Am J Respir Crit Care Med 160:1254–1262

    CAS  PubMed  Google Scholar 

  24. Veninga CC, Denig P, Zwaagstra R, Haaijer-Ruskamp FM (2000) Improving drug treatment in general practice. J Clin Epidemiol 53:762–772

    Article  Google Scholar 

  25. WHO Collaborating Centre for Drug Statistics Methodology. (1999) ATC index with DDDs. World Health Organization, Olso

  26. World organisation of family Doctors (WONCA) (2000) ICPC-2: international classification of primary care, 2nd edn. Oxford Medical Publications, Oxford

  27. Bland M (2000) An introduction to medical statistics, 3rd edn. Oxford University Press, Oxford

  28. Altman DG (1997) Practical statistics for medical research. Chapman & Hall, London

  29. Pont LG, van der Werf GT, Denig P, Haaijer-Ruskamp FM (2002) Identifying general practice patients diagnosed with asthma and their exacerbation episodes from prescribing data Eur J Clin Pharm 57:819–825

    Article  Google Scholar 

  30. de Jong-van den Berg LT, Waardenburg CM, Haaijer-Ruskamp FM, Dukes MN, Wesseling H (1993) Drug use in pregnancy: a comparative appraisal of data collecting methods. Eur J Clin Pharmacol 45:9–14

    PubMed  Google Scholar 

  31. Van den Brandt PA, Petri H, Dorant E, Goldbohm RA, Van de Crommert S (1991) Comparison of questionnaire information and pharmacy data on drug use. Pharm Weekbl Sci 13:91–96

    PubMed  Google Scholar 

  32. West SL, Savitz DA, Koch G, Strom BL, Guess HA, Hartzema A (1995) Recall accuracy for prescription medications: self-report compared with database information. Am J Epidemiol 142:1103–1112

    PubMed  Google Scholar 

  33. Cantrill JA, Sibbald B, Buetow SA (1998) Indicators of the appropriateness of long term prescribing in general practice in the United Kingdom: consensus development, face and content validity, feasibility, and reliability. Qual Health Care 7:130–135

    PubMed  Google Scholar 

  34. Campbell SM, Cantrill JA, Roberts D (2000) Prescribing indicators for UK general practice: Delphi consultation study. BMJ 321:425–428

    Article  PubMed  Google Scholar 

  35. Hanlon JT, Schmader KE, Samsa GP, Weinberger M, Uttech KM, Lewis IK, Cohen HJ, Feussner JR (1992) A method for assessing drug therapy appropriateness. J Clin Epidemiol 45:1045–1051

    PubMed  Google Scholar 

Download references

Acknowledgements

We wish to thank Annet Nicolai for her valuable assistance with the spirometry and patient assessment and Professor Dirkje Postma for her help in preparing this manuscript.

Author information

Authors and Affiliations

  1. Department of Clinical Pharmacology, University of Groningen, The Netherlands

    Lisa G. Pont

  2. Department of Clinical Pharmacology, University of Groningen, The Netherlands

    Petra Denig

  3. Department of General Practice, University of Groningen, The Netherlands

    Thys van der Molen

  4. Registration Network Groningen, Department of General Practice, University of Groningen, The Netherlands

    Willem Jan van der Veen

  5. Department of Clinical Pharmacology, University of Groningen, Ant. Deusinglaan 1, 9713 AV , Groningen, The Netherlands

    Flora M. Haaijer-Ruskamp

Authors
  1. Lisa G. Pont
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Petra Denig
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thys van der Molen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Willem Jan van der Veen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Flora M. Haaijer-Ruskamp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Flora M. Haaijer-Ruskamp.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pont, L.G., Denig, P., van der Molen, T. et al. Validity of performance indicators for assessing prescribing quality: the case of asthma. Eur J Clin Pharmacol 59, 833–840 (2004). https://doi.org/10.1007/s00228-003-0696-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00228-003-0696-x

Keywords

Search

Navigation