Applied Health Economics and Health Policy

, Volume 15, Issue 2, pp 155–162 | Cite as

Amplifying Each Patient’s Voice: A Systematic Review of Multi-criteria Decision Analyses Involving Patients

  • Kevin Marsh
  • J. Jaime Caro
  • Alaa Hamed
  • Erica Zaiser
Systematic Review



Qualitative methods tend to be used to incorporate patient preferences into healthcare decision making. However, for patient preferences to be given adequate consideration by decision makers they need to be quantified. Multi-criteria decision analysis (MCDA) is one way to quantify and capture the patient voice. The objective of this review was to report on existing MCDAs involving patients to support the future use of MCDA to capture the patient voice.


MEDLINE and EMBASE were searched in June 2014 for English-language papers with no date restriction. The following search terms were used: ‘multi-criteria decision*’, ‘multiple criteria decision*’, ‘MCDA’, ‘benefit risk assessment*’, ‘risk benefit assessment*’, ‘multicriteri* decision*’, ‘MCDM’, ‘multi-criteri* decision*’. Abstracts were included if they reported the application of MCDA to assess healthcare interventions where patients were the source of weights. Abstracts were excluded if they did not apply MCDA, such as discussions of how MCDA could be used; or did not evaluate healthcare interventions, such as MCDAs to assess the level of health need in a locality. Data were extracted on weighting method, variation in patient and expert preferences, and discussion on different weighting techniques.


The review identified ten English-language studies that reported an MCDA to assess healthcare interventions and involved patients as a source of weights. These studies reported 12 applications of MCDA. Different methods of preference elicitation were employed: direct weighting in workshops; discrete choice experiment surveys; and the analytical hierarchy process using both workshops and surveys. There was significant heterogeneity in patient responses and differences between patients, who put greater weight on disease characteristics and treatment convenience, and experts, who put more weight on efficacy. The studies highlighted cognitive challenges associated with some weighting methods, though patients’ views on their ability to undertake weighting tasks was positive.


This review identified several recent examples of MCDA used to elicit patient preferences, which support the feasibility of using MCDA to capture the patient voice. Challenges identified included, how best to reflect the heterogeneity of patient preferences in decision making and how to manage the cognitive burden associated with some MCDA tasks.


Health Technology Assessment Discrete Choice Experiment Multiple Criterion Decision Benefit Risk Assessment Weighting Technique 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors would like to thank Evan Davies, Tereza Lanitis, David Neasham and Panos Orfanos at Evidera for their support in reviewing abstracts and full texts.

Compliance with Ethical Standards

Evidera received financial assistance from Sanofi-Genzyme to conduct the study and assist in preparing the manuscript. Authors Kevin Marsh, Erica Zaiser and Jaime Caro are all employees of Evidera. Alaa Hamed is an employee of Sanofi-Genzyme.

Financial support

Financial support for this study was provided entirely by a contract with Sanofi-Genzyme. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report. The following author is employed by the sponsor: Alaa Hamed.

Authors’ contributions

All authors contributed to the conception, design and writing of this manuscript. KM and EZ were responsible for the implementation of the review.


  1. 1.
    oude Egbrink M, Ijzerman M. The value of quantitative patient preferences in regulatory benefit-risk assessment. J Mark Access Health Policy. 2014;2:22761.CrossRefGoogle Scholar
  2. 2.
    Facey K, Boivin A, Gracia J, Hansen HP, Lo Scalzo A, Mossman J, et al. Patients’ perspectives in health technology assessment: a route to robust evidence and fair deliberation. Int J Technol Assess Health Care. 2010;26(3):334–40.CrossRefPubMedGoogle Scholar
  3. 3.
    Fowler FJ Jr, Levin CA, Sepucha KR. Informing and involving patients to improve the quality of medical decisions. Health Aff (Millwood). 2011;30(4):699–706.CrossRefGoogle Scholar
  4. 4.
    Fleurence R, Selby JV, Odom-Walker K, Hunt G, Meltzer D, Slutsky JR, et al. How the Patient-Centered Outcomes Research Institute is engaging patients and others in shaping its research agenda. Health Aff (Millwood). 2013;32(2):393–400.CrossRefGoogle Scholar
  5. 5.
    Pollack A. Viagra for Women’ is back by FDA Panel. New York: The New York Times; 2015.Google Scholar
  6. 6.
    Food and Drug Administration (FDA). FDA approves first treatment for sexual desire disorder. 2015 November 2015. Available from:
  7. 7.
    Patient-Centered Outcomes Research Institute (PCORI). National priorities for research and research agenda. 2012.Google Scholar
  8. 8.
    European Medicines Agency (EMA). Pilot phase to involve patients in benefit/risk discussions at CHMP meetings. EMA/372554/2014—rev. 12014.Google Scholar
  9. 9.
    Mullin T. Patient-focused drug development. 2012 November 2015. Available from:
  10. 10.
    Food and Drug Administration (FDA). About the patient representative program. 2015 November 2015. Available from:
  11. 11.
    Perfetto EM, Burke L, Oehrlein EM, Epstein RS. Patient-focused drug development: a new direction for collaboration. Med Care. 2015;53(1):9–17.CrossRefPubMedGoogle Scholar
  12. 12.
    Weernink MGM, Janus SIM, van Til JA, Raisch DW, van Manen JG, Ijzerman MJ. A systematic review to identify the use of preference elicitation method in healthcare decision making. Pharm Med. 2014;28:175–85.CrossRefGoogle Scholar
  13. 13.
    Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. 2016;19(1):1–13.CrossRefPubMedGoogle Scholar
  14. 14.
    Belton V, Stewart TJ. Multiple criteria decision analysis: an integrated approach. New York: Kluwer Academic Publishers; 2002.CrossRefGoogle Scholar
  15. 15.
    Marsh K, Lanitis T, Neasham D, Orfanos P, Caro J. Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. Pharmacoeconomics. 2014;32(4):345–65.CrossRefPubMedGoogle Scholar
  16. 16.
    Dodgson JS, Spackman M, Pearman A, Phillips LD. Multi-criteria analysis: a manual. London: Department for Communities and Local Government; 2009.Google Scholar
  17. 17.
    Baltussen R, Niessen L. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Eff Resour Alloc. 2006;4:14.CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Devlin N, Sussex J. Incorporating multiple criteria in HTA: methods and processes. Office of Health Economics, London. 2011.Google Scholar
  19. 19.
    Thokala P, Duenas A. Multiple criteria decision analysis for health technology assessment. Value Health. 2012;15(8):1172–81.CrossRefPubMedGoogle Scholar
  20. 20.
    Institute for Quality and Efficiency in Health Care (IQWIG). Determine patient preferences by means of Conjoint Analysis. 2014 April 2015. Available from:
  21. 21.
    Institute for Quality and Efficiency in Health Care (IQWIG). Choice-based conjoint analysis—pilot project to identify, weight, and prioritize multiple attributes in the indication “hepatitis C”. 2014.Google Scholar
  22. 22.
    Dolan JG. Shared decision-making—transferring research into practice: the analytic hierarchy process (AHP). Patient Educ Couns. 2008;73(3):418–25.CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Saaty TL. The analytic hierarchy process: planning, priority setting, resource allocation. New York: McGraw-Hill; 1980.Google Scholar
  24. 24.
    Ryan M, Gerard K, Amaya-Amaya M. Using discrete choice experiments to value health and health care. Berlin: Springer; 2008.CrossRefGoogle Scholar
  25. 25.
    De Montis A, De Toro P, Droste-Franke B, Omann I, Stagl S. Assessing the quality of different MCDA methods. In: Getzner M, Spash C, Stagl S, editors. Alternatives for environmental evaluation. Abingdon, Oxon: Routledge; 2005.Google Scholar
  26. 26.
    International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Multi-criteria decision analysis in health care decision making—emerging good practices task force. 2015 November 2015. Available from:
  27. 27.
    Hummel JM, Snoek GJ, van Til JA, van Rossum W, Ijzerman MJ. A multicriteria decision analysis of augmentative treatment of upper limbs in persons with tetraplegia. J Rehabil Res Dev. 2005;42(5):635–44.CrossRefPubMedGoogle Scholar
  28. 28.
    Airoldi M, Morton A, Smith J, Bevan G. Working paper no. 7. Healthcare prioritisation at the local level: a socio-technical approach. 2011.Google Scholar
  29. 29.
    Broekhuizen H. Integrating patient preferences and clinical trial data in an MCDA model for quantitative benefit-risk assessment. Netherlands: University of Twente; 2012.Google Scholar
  30. 30.
    Youngkong S, Teerawattananon Y, Tantivess S, Baltussen R. Multi-criteria decision analysis for setting priorities on HIV/AIDS interventions in Thailand. Health Res Policy Syst. 2012;10:6.CrossRefPubMedPubMedCentralGoogle Scholar
  31. 31.
    Sussex J, Rollet P, Garau M, Schmitt C, Kent A, Hutchings A. A pilot study of multicriteria decision analysis for valuing orphan medicines. Value Health. 2013;16(8):1163–9.CrossRefPubMedGoogle Scholar
  32. 32.
    Hummel MJ, Volz F, van Manen JG, Danner M, Dintsios CM, Ijzerman MJ, et al. Using the analytic hierarchy process to elicit patient preferences: prioritizing multiple outcome measures of antidepressant drug treatment. Patient. 2012;5(4):225–37.CrossRefPubMedGoogle Scholar
  33. 33.
    Dolan JG. Patient priorities in colorectal cancer screening decisions. Health Expect. 2005;8(4):334–44.CrossRefPubMedPubMedCentralGoogle Scholar
  34. 34.
    Dolan JG, Boohaker E, Allison J, Imperiale TF. Patients’ preferences and priorities regarding colorectal cancer screening. Med Decis Mak. 2013;33(1):59–70.CrossRefGoogle Scholar
  35. 35.
    Hummel JM, Steuten LG, Groothuis-Oudshoorn CJ, Mulder N, Ijzerman MJ. Preferences for colorectal cancer screening techniques and intention to attend: a multi-criteria decision analysis. Appl Health Econ Health Policy. 2013;11(5):499–507.CrossRefPubMedGoogle Scholar
  36. 36.
    Goetghebeur MM, Wagner M, Khoury H, Rindress D, Gregoire JP, Deal C. Combining multicriteria decision analysis, ethics and health technology assessment: applying the EVIDEM decision-making framework to growth hormone for Turner syndrome patients. Cost Eff Resour Alloc. 2010;8:4.CrossRefPubMedPubMedCentralGoogle Scholar
  37. 37.
    Marsh K, IJzerman M, Thokala P, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making—Emerging Good Practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. 2016;19(2):125–37.CrossRefPubMedGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.EvideraLondonUK
  2. 2.EvideraLexingtonUSA
  3. 3.Sanofi GenzymeCambridgeUSA

Personalised recommendations