Multi-criteria Decision Analysis Software in Healthcare Priority Setting: A Systematic Review
The objectives of this systematic review were to identify studies using Multi-Criteria Decision Analysis (MCDA) software tools to support health prioritisation processes and describe the technical capabilities of the MCDA software tools identified.
First, a systematic literature review was conducted in the MEDLINE, EMBASE, Web of Science, EconLit and Cochrane databases in July 2019 to identify studies that have used MCDA software for priority setting in health-related problems. Second, the MCDA software tools found in the review were downloaded (full versions, where freely available, and trial versions otherwise) and tested to extract their key technical characteristics.
Nine studies were included, from which seven different software tools, 1000minds®, M-MACBETH, Socio Technical Allocation of Resources (STAR), Strategic Multi-Attribute Ranking Tool (SMART), Visual PROMETHEE, EVIDEM and the Prioritisation Framework, were identified. These software tools differed in terms of the operating systems (including web interface), MCDA technique(s) available for use, visualisation features, and the capability to perform Value for Money (VfM) and sensitivity analyses.
The use of MCDA software in prioritisation processes has a number of advantages such as inclusion of several types of stakeholders and the ability to analyse a greater number of alternatives and criteria and perform real-time sensitivity analyses. Proprietary software (i.e. software with licensing fees) seemed to have more features than freely available software. However, this field is still developing, with only a few studies where MCDA software was used to support health priority setting and opportunity costs not explicitly captured in many software tools.
The authors would like to acknowledge all of those who contributed to this work.
The manuscript was developed in a doctoral internship funded by the Administrative Department of Science, Technology and Innovation of Colombia (COLCIENCIAS, Bogotá, Colombia), grant number 617.
The original idea and structure of the manuscript were developed by PT and AM. The development of the systematic review was done by AM and TT. AM drafted the manuscript and it was reviewed by PT and TT. All authors contributed to the multiple iterations of the manuscript.
Conflict of interest
Alexander Moreno is a PhD candidate at the Universidad Nacional de Colombia, Bogotá, Colombia. He received a doctoral grant; however, he has no conflicts of interest. Thaison Tong is a Research Associate in Health Economics and Decision Sciences at the School of Health and Related Research, University of Sheffield, Sheffield, UK. He has no financial or non-financial conflicts. Praveen Thokala is a Senior Research Fellow in Health Economics and Decision Sciences at the School of Health and Related Research, University of Sheffield. He has no financial or non-financial conflicts.
- 5.Giedion Ú, Villar M, Ávila A. Los sistemas de salud en Latinoamérica y el papel del seguro privado. 2010. https://www.mapfre.com/ccm/content/documentos/fundacion/cs-seguro/libros/los-sistemas-de-salud-en-latinoamerica-y-el-papel-del-seguro-privado.pdf. Accessed 21 Nov 2019.
- 11.Angelis A, Kanavos P. Value-based assessment of new medical technologies: towards a robust methodological framework for the application of multiple criteria decision analysis in the context of health technology assessment. Pharmacoeconomics. 2016;34(5):435–46.PubMedPubMedCentralCrossRefGoogle Scholar
- 12.Confederation NHS. Priority setting: an overview. London: Primary Care Trust Network; 2007.Google Scholar
- 16.Ghandour R, Shoaibi A, Khatib R, Abu Rmeileh N, Unal B, Sözmen K, et al. Priority setting for the prevention and control of cardiovascular diseases: multi-criteria decision analysis in four eastern Mediterranean countries. Int J Public Health. 2015;60(Suppl 1 S1):S73–81.PubMedCrossRefPubMedCentralGoogle Scholar
- 19.Claxton K. Three questions to ask when examining MCDA. Value Outcomes Spotlight. 2015;1:18–20.Google Scholar
- 20.Mustajoki J, Marttunen M. Comparison of multi-criteria decision analytical software-searching for ideas for developing a new EIA-specific multi-criteria software. Environ Modell Softw. 2013;93(C):78–91.Google Scholar
- 25.Madhavan G, Sangha K, Phelps C, Fryback D, Lieu T, Martinez RM, Committee on Identifying and Prioritizing New Preventive Vaccines for Development; Board on Population Health and Public Health Practice; Board on Global Health; Institute of Medicine, et al., editors. Ranking vaccines: a prioritization framework: phase I: demonstration of concept and a software blueprint. Washington, DC: The National Academies Press; 2012.Google Scholar
- 26.Madhavan G, Sangha K, Phelps C, Fryback D, Rappuoli R, Martinez R, Committee on Identifying and Prioritizing New Preventive Vaccines for Development, Phase II; Board on Population Health and Public Health Practice; Board on Global Health; Institute of Medicine, et al., editors. Ranking vaccines: a prioritization software tool: phase II: prototype of a decision-support system. Washington, DC: The National Academies Press; 2013.Google Scholar
- 27.Madhavan G, Phelps C, Rappuoli R, Martinez RM, King L, Committee on Identifying and Prioritizing New Preventive Vaccines for Development, Phase III; Board on Population Health and Public Health Practice; Board on Global Health; Institute of Medicine; National Academy of Engineering, editors. Ranking vaccines: applications of a prioritization software tool: phase III: use case studies and data framework. Washington, DC: The National Academies Press; 2015.Google Scholar
- 31.Public Health England. The prioritisation framework: making the most of your budget. https://www.gov.uk/government/publications/the-prioritisation-framework-making-the-most-of-your-budget. 2018. Accessed 19 Mar 2019.
- 32.Bana e Costa CA, de Corte J-M, Vansnick J-C. MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique). In: Wiley encyclopedia of operations research and management science. Hoboken: Wiley; 2011.Google Scholar
- 34.Brans JP, Vinke P. Preference ranking organization method for enrichment evaluations. Manag Sci. 1985;31:6. https://difusion.ulb.ac.be/vufind/Record/ULB-DIPOT:oai:dipot.ulb.ac.be:2013/15711/Details. Accessed 17 May 2019
- 37.Abdullah L, Adawiyah CW. Simple additive weighting methods of multi criteria decision making and applications: a decade review. Int J Inf Process Manag. 2014;5(1):39–49.Google Scholar
- 44.Smith PC. Measuring value for money in healthcare: concepts and tools. Quest for quality and improved performance. London: The Health Foundation; 2009. p. 1–54.Google Scholar
- 46.Li Y, Thomas MA. A multiple criteria decision analysis (MCDA) software selection framework. In: Proceedings of the 2014 47th Annual Hawaii international conference on system sciences. Washington, DC: IEEE; 2014. p. 1084–94.Google Scholar
- 47.Weistroffer HR, Li Y. Multiple criteria decision analysis software. In: Greco S, Ehrgott M, Figueira JR, editors. Multiple criteria decision analysis State of the art surveys, vol. 233., International series in operations research & management scienceNew York: Springer; 2016. p. 1301–41.CrossRefGoogle Scholar
- 49.Baizyldayeva U, Vlasov O, Kuandykov AA, Akhmetov TB. Multi-criteria decision support systems. Comparative analysis. Middle East J Sci Res. 2013;16(12):1725–30.Google Scholar