Abstract
Cancer is one of the major diseases that seriously threaten the human life. Increasing interest in cancer treatment strategies for chemotherapy treatment planning and optimal drug administration has created new applications for mathematical modeling. In this paper, we develop a finite-horizon Markov decision process (MDP) model for cancer chemotherapy treatment planning that could advise selection of the optimal policy for the chemotherapy regimen according to the patient’s condition. The proposed model uses a finite action space of optimal cancer chemotherapy regimens for gastric and gastroesophageal cancers resulted from the proposed optimization model and a finite state space of patients’ toxicity levels. Results show that the proposed approach yields the optimal sequence of gastric and gastroesophageal cancer chemotherapy treatment regimens for a period of chemotherapy treatment which makes possible designing clinical trials for sequential treatments.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahuja, V., & Birge, J. R. (2019). An approximation approach for response adaptive clinical trial design. SMU Cox School of Business Research Paper, (pp. 18–26).
Alagoz, O., Hsu, H., Schaefer, A. J., & Roberts, M. S. (2010). Markov decision processes: a tool for sequential decision making under uncertainty. Medical Decision Making, 30(4), 474–483.
American Cancer Society (2016). Cancer: What is cancer.
Bazrafshan, N., & Lotfi, M. M. (2016). A multi-objective multi-drug model for cancer chemotherapy treatment planning: A cost-effective approach to designing clinical trials. Computers & Chemical Engineering, 87, 226–233.
Bellman, R. E. (1957). Dynamic programming. Princeton: Princeton University Press.
Bertsimas, D., O’Hair, A., Relyea, S., & Silberholz, J. (2013). An analytics approach to designing clinical trials for cancer. Working paper: Massachusetts Institute of Technology
Chhatwal, J., Alagoz, O., & Burnside, E. S. (2010). Optimal breast biopsy decision-making based on mammographic features and demographic factors. Operations Research, 58(6), 1577–1591.
Choices NHS (2016). Chemotherapy: side effects.
Chow, S. C., & Liu, J. P. (2005). Design and analysis of clinical trials: Concepts and Methodologies (3rd ed.). Hoboken: Wiley.
Collins, L. M., Nahum-Shani, I., & Almirall, D. (2014). Optimization of behavioral dynamic treatment regimens based on the sequential, multiple assignment, randomized trial (SMART). Clinical Trials, 11(4), 426–434.
Denton, B. T., Kurt, M., Shah, N. D., Bryant, S. C., & Smith, S. A. (2009). Optimizing the start time of statin therapy for patients with diabetes. Medical Decision Making, 29(3), 351–367.
Dua, P., Dua, V., & Pistikopoulos, E. N. (2008). Optimal delivery of chemotherapeutic agents in cancer. Computers & Chemical Engineering, 32(1–2), 99–107.
Eghbali-Zarch, M., Tavakkoli-Moghaddam, R., Esfahanian, F., Azaron, A., & Sepehri, M. M. (2019). A Markov decision process for modeling adverse drug reactions in medication treatment of type 2 diabetes. Proceedings of the Institution of Mechanical Engineers. Part H, Journal of Engineering in Medicine, 233(8), 793–811.
Heshmat, M., & Eltawil, A. (2019). Solving operational problems in outpatient chemotherapy clinics using mathematical programming and simulation. Annals of Operations Research. https://doi.org/10.1007/s10479-019-03500-y.
Hughes, T. P., Branford, S., White, D. L., Reynolds, J., Koelmeyer, R., Seymour, J. F., et al. (2008). Impact of early dose intensity on cytogenetic and molecular responses in chronic- phase CML patients receiving 600 mg/day of imatinib as initial therapy. Blood, 112(10), 3965–3973.
Kahruman, S., Ulusal, E., Butenko, S., Hicks, I. V., & Diehl, K. M. (2012). Scheduling the adjuvant endocrine therapy for early stage breast cancer. Annals of Operations Research, 196(1), 683–705.
Macmillan Cancer Support (2016). Chemotherapy : what is chemotherapy.
NCCN. (2016). NCCN clinical practice guidelines in oncology: gastric cancer (3rd ed.). Pennsylvania: National Comprehensive Cancer Network.
Nielsen, L. R., & Kristensen, A. R. (2006). Finding the K best policies in a finite-horizon Markov decision process. European Journal of Operational Research, 175(2), 1164–1179.
Page, R., and Takimoto, C. (2004). Cancer management: A multidisciplinary approach: medical, surgical, and radiation oncology: in principles of chemotherapy. 8th ed, Pazdur, R., New York: PRP, pp. 21–38.
Petousis, P., Winter, A., Speier, W., Aberle, D. R., Hsu, W., & Bui, A. A. (2019). Using sequential decision making to improve lung cancer screening performance. IEEE Access, 7, 119403–119419.
Petrovski, A., & McCall, J. (2001). Multi-objective optimisation of cancer chemotherapy using evolutionary algorithms. Lecture Notes in Computer Science (pp. 531–545).
Powell, W. B. (2011). Approximate dynamic programming (2nd ed.). Hoboken: Wiley.
Puterman, M. L. (2014). Markov decision processes: discrete stochastic dynamic programing. Hoboken: Wiley.
Ramos, C., Cataldo, A., & Ferrer, J. C. (2018). Appointment and patient scheduling in chemotherapy: A case study in Chilean hospitals. Annals of Operations Research, 286(1), 1–29.
Sandıkçı, B., Maillart, L. M., Schaefer, A. J., Alagoz, O., & Roberts, M. S. (2008). Estimating the patient’s price of privacy in liver transplantation. Operations Research, 56(6), 1393–1410.
Sbeity, H., & Younes, R. (2015). Review of optimization methods for cancer chemotherapy treatment planning. Journal of Computer Science & Systems Biology, 8(2), 074–095.
Schaefer, A. J., Bailey, M. D., Shechter, S. M., & Roberts, M. S. (2004). Modeling medical treatment using Markov decision processes. Operations research and health care: A handbook of methods and applications (pp. 593–612). Boston: Springer.
Scherrer, A., Schwidde, I., Dinges, A., Rüdiger, P., Kümmel, S., & Küfer, K.-H. (2015). Breast cancer therapy planning—a novel support concept for a sequential decision making problem. Health Care Management Science, 18(3), 389–405.
Shechter, S. M., Bailey, M. D., Schaefer, A. J., & Roberts, M. S. (2008). The optimal time to initiate HIV therapy under ordered health states. Operations Research, 56(1), 20–33.
Shi, J., Alagoz, O., Erenay, F. S., & Su, Q. (2014). A survey of optimization models on cancer chemotherapy treatment planning. Annals of Operations Research, 221(1), 331–356.
Swan, G. W. (1990). Role of optimal control theory in cancer chemotherapy. Mathematical Biosciences, 101(2), 237–284.
Swan, G., & Vincent, T. (1977). Optimal control analysis in the chemotherapy of IgG multiple myeloma. Bulletin of Mathematical Biology, 39(3), 317–337.
Vogel, C. L. (2002). Efficacy and safety of trastuzumab as a single agent in first-line treatment of HER2-overexpressing metastatic breast cancer. Journal of Clinical Oncology, 20(3), 719–726.
Winston, W. L. (1997). Operations research: applications and algorithms. Duxbury: Duxburry Press.
World Health Organization (2016). Fact Sheets: Cancer.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Bazrafshan, N., Lotfi, M.M. A finite-horizon Markov decision process model for cancer chemotherapy treatment planning: an application to sequential treatment decision making in clinical trials. Ann Oper Res 295, 483–502 (2020). https://doi.org/10.1007/s10479-020-03706-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-020-03706-5