Skip to main content
SpringerLink
Log in

A Note on the Validity and Reliability of Multi-Criteria Decision Analysis for the Benefit–Risk Assessment of Medicines

  • Current Opinion
  • Published:
Drug Safety Aims and scope Submit manuscript

Abstract

The comparative evaluation of benefits and risks is one of the most important tasks during the development, market authorization and post-approval pharmacovigilance of medicinal products. Multi-criteria decision analysis (MCDA) has been recommended to support decision making in the benefit–risk assessment (BRA) of medicines. This paper identifies challenges associated with bias or variability that practitioners may encounter in this field and presents solutions to overcome them. The inclusion of overlapping or preference-complementary criteria, which are frequent violations to the assumptions of this model, should be avoided. For each criterion, a value function translates the original outcomes into preference-related scores. Applying non-linear value functions to criteria defined as the risk of suffering a certain event during the study introduces specific risk behaviours in this prescriptive, rather than descriptive, model and is therefore a questionable practice. MCDA uses weights to compare the importance of the model criteria with each other; during their elicitation a frequent situation where (generally favourable) mild effects are directly traded off against low probabilities of suffering (generally unfavourable) severe effects during the study is known to lead to biased and variable weights and ought to be prevented. The way the outcomes are framed during the elicitation process, positively versus negatively for instance, may also lead to differences in the preference weights, warranting an appropriate justification during each implementation. Finally, extending the weighted-sum MCDA model into a fully inferential tool through a probabilistic sensitivity analysis is desirable. However, this task is troublesome and should not ignore that clinical trial endpoints generally are positively correlated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Guo JJ, Pandey S, Doyle J, Bian B, Lis Y, Raisch DW. A review of quantitative risk-benefit methodologies for assessing drug safety and efficacy—report of the ISPOR Risk-Benefit Management Working Group. Value Health. 2010;13(5):657–66.

    Article  PubMed  Google Scholar 

  2. European Medicines Agency. Benefit-risk methodology project. Work package 2 report: Applicability of current tools and processes for regulatory benefit-risk assessment. EMA/549682/2010. 2010.

  3. Mt-Isa S, Hallgreen CE, Wang N, Callréus T, et al. Balancing benefit and risk of medicines: a systematic review and classification of available methodologies. Pharmacoepidemiol Drug Saf. 2014;23(7):667–78.

    Article  PubMed  Google Scholar 

  4. European Medicines Agency. Benefit-risk methodology project. Work package 4 report: benefit-risk tools and processes. EMA/297405/2012.

  5. Food and Drugs Administration. Structured Approach to Benefit-Risk Assessment in Drug Regulatory Decision-Making. Draft PDUFA V Implementation Plan—February 2013. http://www.fda.gov/downloads/ForIndustry/UserFees/PrescriptionDrugUserFee/UCM329758.pdf. Accessed 25 Mar 2015.

  6. The PROTECT Consortium. Recommendations for the methodology and visualisation techniques to be used in the assessment of benefit and risk of medicines. http://protectbenefitrisk.eu/documents/HughesetalRecommendationsforthemethodologyandvisualisationtechniquestobeusedintheassessmento.pdf. Accessed 25 Mar 2015.

  7. Dodgson J, Spackman M, Pearman A, Phillips L. Multi-criteria analysis: a manual. London: Department of the Environment, Transport and the Regions; 2000.

    Google Scholar 

  8. Mussen F, Salek S, Walker S. A quantitative approach to benefit-risk assessment of medicines. Part 1: the development of a new model using multi-criteria decision analysis. Pharmacoepidemiol Drug Saf. 2007;16(Suppl 1):S2–S15.

  9. Mussen F, Salek S, Walker S. Benefit-risk appraisal of medicines. A systematic approach to decision making. New York: Wiley; 2009.

    Google Scholar 

  10. European Medicines Agency. Benefit-risk methodology project. Work package 3 report: Field tests. EMA/718294/2011. 2011.

  11. Nixon R, Dierig C, Mt-Isa S, Stöckert I, Tong T, Kuhls S, Hodgson G, Pears J, Waddingham E, Hockley K, Thomson A. A case study using the PrOACT-URL and BRAT frameworks for structured benefit risk assessment. Biom J. 2015. doi:10.1002/bimj.201300248 [Epub ahead of print].

  12. Thokala P, Duenas A. Multiple criteria decision analysis for health technology assessment. Value Health. 2012;15:1172–81.

    Article  PubMed  Google Scholar 

  13. European Medicines Agency. Assessment report for Cimzia. 2009. Doc.Ref.: EMEA/664021/2009.

  14. Felson DT, Anderson JJ, Boers M, Bombardier C, Furst D, Goldsmith C, American College of Rheumatology, et al. Preliminary definition of improvement in rheumatoid arthritis. Arthritis Rheum. 1995;38:727–35.

    Article  CAS  PubMed  Google Scholar 

  15. Keeney R, Raïffa H. Decisions with multiple objectives: preferences and value tradeoffs. Cambridge University Press; 1993.

  16. Von Winterfeldt D, Edwards W. Decision analysis and behavioural research. Cambridge University Press; 1986.

  17. Tervonen T, van Valkenhoef G, Buskens E, Hillege HL, Postmus D. A stochastic multicriteria model for evidence-based decision making in drug benefit-risk analysis. Stat Med. 2011;30:1419–28.

    Article  PubMed  Google Scholar 

  18. Broekhuizen H, Groothuis-Oudshoorn CG, van Til J, Hummel JM, IJzerman M. A review and classification of approaches for dealing with uncertainty in multi-criteria decision analysis for healthcare decisions. Parmacoeconomics. 2015. doi:10.1007/s40273-014-0251-x [ahead of print].

  19. Delquié P, Luo M. A simple trade-off condition for additive multiattribute utility. J Multi Crit Decis Anal. 1997;6:248–52.

    Article  Google Scholar 

  20. Kidney Disease: Improving Global Outcomes (KDIGO) Anemia Work Group. KDIGO clinical practice guideline for anemia in CKD. Kidney Int. 2012;2(Suppl. 2012):279–335.

  21. Bleichrodt H, Pinto JL, Wakker P. Using descriptive findings of prospect theory to improve prescriptive applications of expected utility. Manag Sci. 2001;47:1498–514.

    Article  Google Scholar 

  22. Van Osch SMC, Wakker P, van den Hout W, Stiggelbout AM. Correcting biases in standard gamble and time trade-off utilities. Med Decis Mak. 2004;24:511–7.

    Article  Google Scholar 

  23. Edwards A, Elwyn G. Understanding risk and lessons for clinical risk communication about treatment preferences. Qual Health Care. 2001;10(Suppl I):i9–13.

    Article  PubMed Central  PubMed  Google Scholar 

  24. McNeil BJ, Pauker SG, Sox H, Tversky A. On the elicitation of preferences for alternative therapies. N Engl J Med. 1982;306:1259–62.

    Article  CAS  PubMed  Google Scholar 

  25. Howard K, Salkeld G. Does attribute framing in discrete choice experiments influence willingness to pay? Results from a discrete choice experiment in screening for colorectal cancer. Value Health. 2009;12(2):354–63.

    Article  PubMed  Google Scholar 

  26. Pinto-Prades JL, Martinez-Perez JE, Abellán-Perpiñán JM. The influence of the ratio bias phenomenon on the elicitation of health states utilities. Judgem Decis Mak. 2006;1(2):118–33.

    Google Scholar 

  27. Johnson FR, Ozdemir S, Mansfield C, Hass S, Miller DW, Siegel CA, Sands BE. Crohn’s disease patients’ risk-benefit preferences: serious adverse event risks versus treatment efficacy. Gastroenterology. 2007;133(3):769–79.

    Article  PubMed  Google Scholar 

  28. Johnson FR, Van Houtven G, Ozdemir S, Hass S, White J, Francis G, Miller DW, Philips JT. Multiple sclerosis patients’ benefit-risk preferences: serious adverse event risks versus treatment efficacy. J Neurol. 2009;256:554–62.

    Article  PubMed  Google Scholar 

  29. Gafni A. The standard gamble method: what is being measured and how it is interpreted. Health Serv Res. 1994;28:2.

    Google Scholar 

  30. von Neumann J, Morgenstern O. Theory of games and economic behavior. 3rd ed. New York: Wiley; 1953.

    Google Scholar 

  31. Van Osch SMC, Stiggelbout AM. Construction of standard gamble utilities. Health Econ. 2008;17:31–40.

    Article  PubMed  Google Scholar 

  32. Bakker C, Rutten M, van Santen-Hoeufft M, Bolwijn P, van Doorslaer E, Bennett K, van der Linden S. Patient utilities in fibromyalgia and the association with other outcome measures. J Rheumatol. 1995;22(8):1536–43.

    CAS  PubMed  Google Scholar 

  33. Garcia-Hernandez A. Quality-of-life adjusted hazard of death: a formulation of the quality-adjusted life years model of use in benefit risk assessment. Value Health. 2014;17:275–9.

    Article  PubMed  Google Scholar 

  34. Offen WW, Chuang-Stein C, Dmitrienko A, Littman G, et al. Multiple co-primary endpoints: medical and statistical solutions. A report from the multiple endpoints expert team of the Pharmaceutical Research and Manufacturers of America. Drug Inf J. 2007;41:31–46.

    Google Scholar 

  35. Waddingham E, Mt-Isa S, Nixon R, Ashby D. A Bayesian approach to probabilistic sensitivity analysis in structured benefit-risk assessment. Biom J. 2015. doi:10.1002/bimj.201300254 [ahead of print].

  36. Wen S, Zhang L, Yang B. Two approaches to incorporate clinical data uncertainty into multiple criteria decision analysis for benefit-risk assessment of medicinal products. Value Health. 2014;17(5):619–28. doi:10.1016/j.jval.2014.04.008.

    Article  PubMed  Google Scholar 

  37. European Medicines Agency Press Office. Early dialogue between regulators and health technology assessment bodies key to medicine development. EMA/738272/2013. 2013.

Download references

Author information

Authors and Affiliations

  1. Global Data Science, Astellas Pharma Europe B.V., Sylviusweg 62, 2333 BE, Leiden, The Netherlands

    Alberto Garcia-Hernandez

Authors
  1. Alberto Garcia-Hernandez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alberto Garcia-Hernandez.

Ethics declarations

Funding

No sources of funding were used to assist in the preparation of this study.

Conflict of interest

Alberto Garcia-Hernandez has no conflicts of interest that are directly relevant to the content of this study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Garcia-Hernandez, A. A Note on the Validity and Reliability of Multi-Criteria Decision Analysis for the Benefit–Risk Assessment of Medicines. Drug Saf 38, 1049–1057 (2015). https://doi.org/10.1007/s40264-015-0344-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40264-015-0344-2

Keywords

Search

Navigation