Abstract
Assessing the cost effectiveness of a new health intervention often requires modelling to estimate the impact of the intervention on cost, survival and quality of life over the lifetime of a cohort of patients. Markov modelling is a methodology that is commonly employed to estimate these long-term costs and benefits. As commonly used, these models assume that the patients continue to get the treatments assigned regardless of the change in health states. In this paper, we describe an extension to the Markov modelling approach, called Markov decision modelling. Such a model starts with a set of health states and treatments and optimally assigns treatments to each of the health states.
A Markov decision model can be used to identify the optimal treatment strategy not just for the initial disease state, but also as the disease state changes over time. We present a dynamic programming approach to identifying the optimal assignment of treatments, and illustrate this methodology using an example.
The Markov decision modelling approach provides an efficient way of identifying optimal assignment of treatments to health states, but, like the standard Markov model, may be of limited use when probabilities of future events depend on past history in a complex fashion. Even with its limitations, Markov decision models offer an opportunity for health economists to inform healthcare decision-makers on how to modify current treatment pathways to incorporate new treatments as they become available.
Similar content being viewed by others
References
Raftery J. NICE: faster access to modern treatments? Analysis of guidance on health technologies. BMJ 2001 Dec 1; 323 (7324): 1300–1303
Henry D. Economic analysis as an aid to subsidisation decisions: the development of Australian guidelines for pharmaceuticals. Pharmacoeconomics 1992 Jan; 1 (1): 54–67
Gold MR, Siegel JE, Russell LB, et al., editors. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996
Sonnenberg FA, Beck R. Markov models in medical decision making: a practical guide. Med Decis Making 1993; 13: 322–338
Briggs A, Sculpher M. An introduction to Markov modeling for economic evaluation. Pharmacoeconomics 1998; 13: 397–409
Hillier FS, Lieberman GJ. Introduction to operations research. 3rd ed. San Francisco: Holden-Day Inc, 1980
Claxton K, Thompson KM. A dynamic programming approach to the efficient design of clinical trials. J Health Econ 2001 Sep; 20 (5): 797–822
Chancellor N, Hill AM, Sabin CA, et al. Modelling the cost effectiveness of lamivudine/zidovudine combination therapy in HIV infection. Pharmacoeconomics 1997 Jul; 12 (1): 54–66
Beck JR, Pauker SG. The Markov process in medical prognosis. Med Decis Making 1983; 3 (4): 419–458
Wood E, Hogg RS, Harrigan PR, et al. When to initiate antiretroviral therapy in HIV-1-infected adults: a review for clinicians and patients. Lancet Infect Dis 2005 Jul; 5 (7): 407–414
Schackman BR, Goldie SJ, Weinstein MC, et al. Cost-effectiveness of earlier initiation of antiretroviral therapy for uninsured HIV-infected adults. Am J Public Health 2001 Sep; 91 (9): 1456–1463
Mauskopf J, Kitahata M, Kauf T, et al. HIV antiretroviral treatment: early versus later. J Acquir Immune Defic Syndr 2005 Aug 15; 39 (5): 562–569
Sharma R, Stano M, Haas M. Adjusting to changes in health: implications for cost-effectiveness analysis. J Health Econ 2004; 23: 335–351
Sculpher M, Gafni A. Recognizing diversity in public preferences: the use of preference sub-groups in cost-effectiveness analysis. Health Econ 2001 Jun; 10 (4): 317–324
Bala MY, Zarkin GA. Pharmacogenomics and the evolution of healthcare: is it time for cost-effectiveness analysis at the individual level? Pharmacoeconomics 2004; 22 (8): 495–498
La Caze A. Does pharmacogenomics provide an ethical challenge to the utilisation of cost-effectiveness analysis by public health systems? Pharmacoeconomics 2005; 23 (5): 445–447
Bala MY, Zarkin GA. On pharmacogenomics and cost-effectiveness analysis at the individual level. Pharmacoeconomics 2005; 23 (5): 527
Salomon JA, Weinstein MC, Goldie SJ. Taking account of future technology in cost effectiveness analysis. BMJ. 2004 Sep 25; 329 (7468): 733–736
Acknowledgements
No funding was received for this work. Mohan Bala is an employee of Centocor Inc. Josephine Mauskopf provides consulting services to pharmaceutical and medical device companies.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bala, M.V., Mauskopf, J.A. Optimal Assignment of Treatments to Health States Using a Markov Decision Model. Pharmacoeconomics 24, 345–354 (2006). https://doi.org/10.2165/00019053-200624040-00005
Published:
Issue Date:
DOI: https://doi.org/10.2165/00019053-200624040-00005