Abstract
Partially observable Markov Decision Process POMDP have been suggested a most popular tool to express sequential decision making under uncertainty. The generality of the standard POMDP model, however, limits practical application of the framework due to the computational complexity of associated solution methods. To alleviate this obstacle we have proposed a specialized POMDP form and our goal is to determine an optimal or nearly optimal policy for the treatment of patients with IHD. This well constructed POMDP model has practical advantage over clinical studies (no risk for the life of patients and low cost), though the latter are necessary to perform before making significant changes to treatment guidelines.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Anderson KM (1998) Cardiovascular disease risk profiles. Am Heart J 121:293–298
Bertsekas DP (1998) Stochastic optimal control. The discrete time case. Academic Press, New York
D’Amato RM, D’Aquila RT (1998) Management of antiretroviral therapy for HIV infection: modelling when to change therapy. J Antivir Ther 3:147–158
Goulionis J (2007) Partially observable markov decision processes and applications to the replacement problem. Ph.D. thesis, University of Piraeus
Goulionis J, Stengos D (2008a) Dynamic decision making for clinical patient management with Parkinson’s disease. JP J Biostat 2(3):217–235
Goulionis J, Stengos D (2008b) The POMDP framework in medical decision making. Adv Appl Stat 9(2):205–232
Goulionis J, Stengos D, Tzavelas G (2008) Probabilistic models for optimizing patients survival rates. J Inderdiscip Math 11(5):713–729
Hu C, Lovejoy WS, Shafer SI (1996) Comparison of some suboptimal control policies in medical drug therapy. Oper Res 44:696–709
Lachlan G, Krishnan T (1997) The EM algorithm and extensions. Wiley series in probability and statistics. Wiley, New York
Littman ML (1996). Algorithms for sequential decision making. Ph.D. thesis, Brown University
Pais P, Yusuf S (2001) Increased risk of myocardial infarction associated with cigarette smoking. Indian Heart J 53:731–735
Papadimitriou GH, Tsitsiklis JN (1987) The complexity of Markov decision processes. Math Oper Res 12:441–450
Sondik EJ (1971) The optimal control of partially observable markov processes. Ph.D. dissertation, Stanford University, CA
Tashima KT, Hogan K, Gardner LI (2001) A longitundinal analysis of hospitalization and emergency department use among human immunodeficiency virus-infected women reporting protease inhibitor use. Clin Infect Dis 33(12):2055–2060
Zhang NL, Lin WA (1997) A model approximation scheme for planning in stochastic domains. J Artif Intell Res 7:199–230
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Vozikis, A., Goulionis, J.E. & Benos, V.K. The partially observable Markov decision processes in healthcare: an application to patients with ischemic heart disease (IHD). Oper Res Int J 12, 3–14 (2012). https://doi.org/10.1007/s12351-010-0095-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12351-010-0095-x