, Volume 18, Issue 1, pp 55–62 | Cite as

Using Economic Evaluations to Make Formulary Coverage Decisions

So Much for Guidelines
Original Research Article


Background: It is mandatory for drug manufacturers requesting formulary inclusion under the British Columbia (BC) provincial drug plan to submit a pharmacoeconomic analysis according to published guidelines. These submissions are reviewed by the Pharmacoeconomic Initiative (PI) of BC.

Objective: To assess the compliance of submitted studies with specific criteria outlined in the guidelines, to assess the methodological quality of individual submissions, and to demonstrate the importance of submitting guidelines-compliant pharmacoeconomic analyses.

Data and Methods: All submissions between January 1996 and April 1999 assessed by the PI of BC were included. Submissions were reviewed according to a checklist to establish compliance with respect to choice of comparator drug, study perspective, sensitivity analysis, analytical horizon and discounting. Submissions were examined for association between analytical technique and author, and between source of submission and compliance. Association between compliance and recommendation for approval was also examined.

Results: 95 applications were reviewed. Seven submitted no analyses. There were 25 cost-comparison/consequence, 14 cost-effectiveness, 11 cost-minimisation, 9 cost-utility/benefit and 29 budget-impact analyses. 65 of these 88 submissions failed to comply with guidelines.Of these, 45% used an inappropriate comparator drug, 61% lacked a sensitivity analysis, 73% used a third-party payer and excluded a societal perspective, 66% did not provide a long term evaluation and 25% did not specify any time horizon. 80% of noncompliant studies were cost-comparison/consequence or budget-impact analyses (p < 0.001, Fisher’s Exact). Of 25 cost-comparison/consequence and 29 budget-impact analyses, 19 (76%) and 24 (83%), respectively, were industry-conducted, whereas cost-effectiveness (11 of 14) and cost-utility/benefit (6 of 9) analyses were mostly subcontracted to private consultants or academics (p < 0.001, Fisher’s Exact). 74% of all submissions (compliant and noncompliant) were not recommended by the PI for listing as a provincial drug plan benefit, 16% received approval for restricted benefit and 9%were recommended as full benefit. 80% of the noncompliant submissions were not recommended (p = 0.06, Fisher’s Exact test). Moreover, a strong association between type of analysis and type of recommendation was found (p = 0.03, Fisher’s Exact test). Cost-comparison/consequence and budget-impact analyses were less likely to be recommended.

Implications of Findings: Our findings show poor compliance with guidelines, especially among industry-conducted studies. Possible explanations are lack of expertise in pharmacoeconomics and/or scepticism regarding the importance of guidelines and submission quality in decision making. As corroborated by the strong associations between type of recommendation and compliance, and between type of recommendation and type of analysis, these 2 characteristics have a significant impact on decision making.



Dr Anis’ research programme is supported by a grant from the British Columbia Ministry of Health and the Ministry responsible for Seniors. We would like to gratefully acknowledge the assistance of Ruby Virani, Hector Leon, Daphne Guh, Sophia Wang, Diane Skippen and Fatma Telli. We are also grateful to all members of the Pharmacoeconomic Initiative Scientific Committee and the Drug Benefit Committee for their active participation in, and time commitment to, the drug review and approval process in British Columbia.


  1. 1.
    Detsky A. Guidelines for economic analysis of pharmaceutical products: a draft document for Ontario and Canada. Pharmacoeconomics 1993; 3 (5): 354–61PubMedCrossRefGoogle Scholar
  2. 2.
    Torrance GW, Blaker D, Detsky A, et al. Canadian guidelines for economic evaluation of pharmaceuticals. Pharmacoeconomics 1996; 9 (6): 535–59PubMedCrossRefGoogle Scholar
  3. 3.
    Canadian Coordinating Office for Health Technology Assessment (CCOHTA). Guidelines for economic evaluation of pharmaceuticals. 2nd ed. Ottawa: CCOHTA, 1997Google Scholar
  4. 4.
    Glennie JL, Torrance GW, Baladi JF, et al. The revised Canadian guidelines for the economic evaluation of pharmaceuticals. Pharmacoeconomics 1999; 15 (5): 459–68PubMedCrossRefGoogle Scholar
  5. 5.
    Commonwealth Department of Human Services. Guidelines for the pharmaceutical industry on preparation of submissions to the pharmaceutical benefits advisory committee including major submissions involving economic analyses. Canberra: Australian Government Publishing Service, 1995Google Scholar
  6. 6.
    Gorham P. Cost-effectiveness guidelines: the experience of Australian manufacturers. Pharmacoeconomics 1995; 8 (5): 369–73PubMedCrossRefGoogle Scholar
  7. 7.
    Langley PC. The November 1995 revised Australian guidelines for the economic evaluation of pharmaceuticals. Pharmacoeconomics 1996; 9 (4): 341–52PubMedCrossRefGoogle Scholar
  8. 8.
    Grobler MP, Macarounas-Kirchman K, Pearce GA, et al. Industry comment on the 1995 revised Australian pharmacoeconomic guidelines. Pharmacoeconomics 1996; 9 (4): 343–56CrossRefGoogle Scholar
  9. 9.
    Ikeda S, Naoki I, Oliver AJ, et al. A case for the adoption of pharmacoeconomic guidelines in Japan. Pharmacoeconomics 1996; 10 (6): 546–51PubMedCrossRefGoogle Scholar
  10. 10.
    Garattini L, Grilli R, Scopelliti D, et al. A proposal for Italian guidelines in pharmacoeconomics. Pharmacoeconomics 1995; 7 (1): 1–6PubMedCrossRefGoogle Scholar
  11. 11.
    Drummond MF, Rutten F, Brenna A, et al. Economic evaluation of pharmaceuticals: a European perspective. Pharmacoeconomics 1993; 4: 173–86PubMedCrossRefGoogle Scholar
  12. 12.
    Annemans L, Crott R, De Clercq H, et al. Pricing and reimbursement of pharmaceuticals in Belgium. Pharmacoeconomics 1997; 11 (3): 203–9PubMedCrossRefGoogle Scholar
  13. 13.
    World pharmaceutical news. Scrip 1999 May 14; 2437: 5Google Scholar
  14. 14.
    World pharmaceutical news. Scrip 1999 Apr 23; 2431: 6Google Scholar
  15. 15.
    Anis AH, Rahman T, Schecther MT. Using pharmacoeconomic analysis to make drug insurance coverage decisions. Pharmacoeconomics 1998; 13 (1 Pt 2): 119–26PubMedCrossRefGoogle Scholar
  16. 16.
    Glossary of terms used in health economics, and pharmacoeconomic and quality-of-life analyses. Pharmacoeconomics 2000; 17 (5): 535–8Google Scholar
  17. 17.
    Drummond MF, O’Brien B, Stoddart GL, et al. Methods for the economic evaluation of health care programmes. 2nd ed. Oxford: Oxford University Press, 1997Google Scholar
  18. 18.
    Pharmacoeconomic Initiative (PI) of British Columbia. PI drug submission form [online]. Available from: URL: [Accessed 2000 May 4]
  19. 19.
    Drummond MF. Economic evaluation of pharmaceuticals: science or marketing? Pharmacoeconomics 1992; 1 (1): 8–13PubMedCrossRefGoogle Scholar
  20. 20.
    Drummond MF. A reappraisal of economic evaluation of pharmaceuticals: science or marketing? Pharmacoeconomics 1998; 14 (1): 1–9PubMedCrossRefGoogle Scholar
  21. 21.
    Freemantle N, Maynard A. Something rotten in the state of clinical and economic evaluations. Health Econ 1994; 3: 63–7PubMedCrossRefGoogle Scholar
  22. 22.
    Assiff L, Pollock MR, Manzi P, et al. Health economics in the Canadian pharmaceutical industry. Pharmacoeconomics 1999 Dec; 16 (6): 669–78PubMedCrossRefGoogle Scholar

Copyright information

© Adis International Limited 2000

Authors and Affiliations

  1. 1.Center for Health Evaluation and Outcome SciencesSt Paul’s HospitalVancouverCanada
  2. 2.Department of Healthcare and EpidemiologyUniversity of British ColumbiaVancouverCanada

Personalised recommendations