, Volume 17, Issue 6, pp 545–553 | Cite as

Expert Judgement in Pharmacoeconomic Studies

Guidance and Future Use
  • Christopher EvansEmail author
  • Bruce Crawford
Current Opinion


Research in the field of pharmacoeconomics has increased substantially during the past decade. Much of this research has been on the design and analysis of data concerning the relative merits of one drug or device compared with another in terms of costs and effects. Concomitant with these evaluations has been the development of guidelines for the conduct of economic evaluations in several countries. However, despite an increase in research, little attention has been paid to how different study designs may influence the results of a study. The use of expert judgement in decision analytic modelling is one area where design issues may influence the findings of a study. This issue is examined for the case of modified Delphi and Delphi panels.

Although the use of expert opinion in modelling studies seems to be widespread, there is little consistent application, understanding or reporting of the techniques used. In particular, the definitions of techniques vary between studies, the criteria for determining when consensus is reached vary, and the reporting of these criteria is absent. Future studies using expert judgement should be more aware of the controversies surrounding the issue and provide more reporting of the techniques used. It is proposed that future validation exercises may assist researchers in determining the most appropriate application of methods.


Resource Utilisation Expert Opinion Expert Judgement Pharmacoeconomic Study Delphi Process 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Evans C. The use of consensus methods and expert panels in pharmacoeconomic studies: practical applications and methodological shortcomings. Pharmacoeconomics 1997; 12 (2 Pt 1): 121–9PubMedCrossRefGoogle Scholar
  2. 2.
    Commonwealth Department of Human Services and Health. Guidelines for the pharmaceutical industry on preparation of submissions to the pharmaceutical benefits advisory committee. Canberra: Australian Government Publishing Service, 1995Google Scholar
  3. 3.
    Canadian Coordinating Office for Health Technology Assessment. Guidelines for the economic evaluation of pharmaceuticals: Canada. 2nd ed. Ottawa (ON): Canadian Coordinating Office for Health Technology Assessment (CCOHTA); 1997Google Scholar
  4. 4.
    Jones J, Hunter H. Consensus methods for medical and health services research. BMJ 1995; 311: 376–80PubMedCrossRefGoogle Scholar
  5. 5.
    Stewart D, Shamdasani P. Focus groups: theory and practice. Applied social research methods series, vol. 20. London: Sage Publications, 1990Google Scholar
  6. 6.
    Moore C. Group techniques for idea building (2nd edition): applied social research methods series. London: Sage Publications, 1994Google Scholar
  7. 7.
    Canadian Coordinating Office for Health Technology Assessment. A guidance document for the costing process. Version 1.0. Ottawa (ON): Canadian Coordinating Office for Health Technology Assessment (CCOHTA); 1996Google Scholar
  8. 8.
    Baladi JF, Menon D, Otten N. Use of economic evaluation guidelines: 2 years experience in Canada. Health Economics 1998; 7: 221–7PubMedCrossRefGoogle Scholar
  9. 9.
    National Health Insurance Council. Draft report on the pharmacoeconomic research guideline. The Netherlands: National Health Insurance Council, 1999Google Scholar
  10. 10.
    Gold MR, Seigel JE, Russell LB, et al. Cost-effectiveness in health and medicine. New York (NY): Oxford University Press, 1966Google Scholar
  11. 11.
    Barr JT, Schumacher GE. Using decision analysis to conduct pharmacoeconomic studies. In: Spilker B, editor. Quality of life and pharmacoeconomics in clinical trials. 2nd ed. Philadelphia (PA): Lippincott-Raven, 1996Google Scholar
  12. 12.
    Nuijten MJ, Pronk M, Borerens MJ, et al. Reporting format for economic evaluation part II: focus on modelling studies. Pharmacoeconomics 1998; 14 (3): 259–68PubMedCrossRefGoogle Scholar
  13. 13.
    Halpern M, Luce B, Brown R, et al. Health economic modeling practices: a suggested framework. Value Health 1998; 1 (2): 131–47PubMedCrossRefGoogle Scholar
  14. 14.
    Dalkey N, Helmer O. An experimental application of the Delphi method to the use of experts. Manage Sci 1963; 9 (3): 458–67CrossRefGoogle Scholar
  15. 15.
    Guest J, Hart W, Cookson R, et al. Cost analysis of palliative care for terminally ill cancer patients in the UK after switching from weak to strong opioids. Pharmacoeconomics 1998; 14 (3): 285–97PubMedCrossRefGoogle Scholar
  16. 16.
    Linstone H, Turoff M, editors. The Delphi method: techniques and applications. Reading (MA): Addison-Wesley Publishing Company, 1975Google Scholar
  17. 17.
    Schiebe M, Skutsch M, Schofer J. Experiments in Delphi methodology. In: Linstone H, Turoff M, editors. The Delphi method: techniques and applications. Reading (MA): Addison-Wesley Publishing Company, 1975Google Scholar
  18. 18.
    Jones C. Delphi evaluation of agreement between organizations. In: Linstone H, Turoff M, editors. The Delphi method: techniques and applications. Reading (MA): Addison-Wesley Publishing Company, 1975Google Scholar
  19. 19.
    Antczak-Bouckoms AA, Weinstein M. Cost-effectiveness analysis of periodontal disease control. J Dent Res 1987; 66 (11): 1630–5PubMedCrossRefGoogle Scholar
  20. 20.
    De Lissovoy G, Rentz AM, Dukes EM, et al. The cost-effectiveness of a new chlorhexidine delivery system in the treatment of adult periodontitis. J Am Dent Assoc 1999; 130 (6): 855–62PubMedGoogle Scholar
  21. 21.
    Sackman H. Delphi critique: expert opinion, forecasting and group process. Lexington (MA): Lexington Books, 1975Google Scholar
  22. 22.
    Tversky A, Kahneman D. Judgement under uncertainty: heuristics and biases. Science 1974 Sep; 185: 1124–31PubMedCrossRefGoogle Scholar
  23. 23.
    Nuijten M, Brorens M, Hekster Y, et al. Reporting format for economic evaluation: part I. Application to the Dutch healthcare system. Pharmacoeconomics 1998; 14 (2): 159–63PubMedCrossRefGoogle Scholar
  24. 24.
    Wilson DA, Gammans RE, Evans CJ. Cost of ischemic stroke by functional status: a model comparing citicoline to standard care. Value Health 1999; 2 (3): 152CrossRefGoogle Scholar

Copyright information

© Adis International Limited 2000

Authors and Affiliations

  1. 1.MAPI Values USA LLCBostonUSA

Personalised recommendations