Skip to main content

Advertisement

SpringerLink
Log in

Strategic level proton therapy patient admission planning: a Markov decision process modeling approach

  • Published:
Health Care Management Science Aims and scope Submit manuscript

Abstract

A relatively new consideration in proton therapy planning is the requirement that the mix of patients treated from different categories satisfy desired mix percentages. Deviations from these percentages and their impacts on operational capabilities are of particular interest to healthcare planners. In this study, we investigate intelligent ways of admitting patients to a proton therapy facility that maximize the total expected number of treatment sessions (fractions) delivered to patients in a planning period with stochastic patient arrivals and penalize the deviation from the patient mix restrictions. We propose a Markov Decision Process (MDP) model that provides very useful insights in determining the best patient admission policies in the case of an unexpected opening in the facility (i.e., no-shows, appointment cancellations, etc.). In order to overcome the curse of dimensionality for larger and more realistic instances, we propose an aggregate MDP model that is able to approximate optimal patient admission policies using the worded weight aggregation technique. Our models are applicable to healthcare treatment facilities throughout the United States, but are motivated by collaboration with the University of Florida Proton Therapy Institute (UFPTI).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Adan I, Vissers J (2002) Patient mix optimisation in hospital admission planning: a case study. Int J Oper Prod Manage 22(4):445–461

    Article  Google Scholar 

  2. Agnetis A, Coppi A, Corsini M, Dellino G, Meloni C, Pranzo M (2012) Long term evaluation of operating theater planning policies. Oper Res Health Care 1(4):95–104

    Article  Google Scholar 

  3. Agnetis A, Coppi A, Corsini M, Dellino G, Meloni C, Pranzo M (2014) A decomposition approach for the combined master surgical schedule and surgical case assignment problems. Health Care Manag Sci 17(1):49–59

    Article  Google Scholar 

  4. American Cancer Society (2014) Cancer facts and figures 2014, http://www.cancer.org/acs/groups/content/@research/documents/webcontent/acspc-042151.pdf. Accessed 30 June 2014

  5. Andre D, Russell SJ (2002) State abstraction for programmable reinforcement learning agents. In: AAAI/IAAI, pp 119–125

  6. Averill RF, Muldoon JH, Vertrees JC, Goldfield NI, Mullin RL, Fineran EC, Zhang MZ, Steinbeck B, Grant T (1993) The evolution of casemix measurement using diagnosis related groups (drgs). Wallingford 3M Health Information Systems

  7. Bertsimas D, Demir R (2002) An approximate dynamic programming approach to multidimensional knapsack problems. Manag Sci 48(4):550–565

    Article  Google Scholar 

  8. Bowers J, Mould G (2005) Ambulatory care and orthopaedic capacity planning. Health Care Manag Sci 8:41–47

    Article  Google Scholar 

  9. Cayirli T, Veral E, Rosen H (2006) Designing appointment scheduling systems for ambulatory care services. Health Care Manag Sci 9:47–58

    Article  Google Scholar 

  10. Ciavotta M, Dellino G, Meloni C, Pranzo M (2010) A rollout algorithmic approach for complex parallel machine scheduling in healthcare operations. In: Operations research for patient: centered health care delivery: proceedings of the XXXVI international ORAHS conference, vol 5, pp 316–324

  11. Ciavotta M, Meloni C, Pranzo M (2009) Scheduling dispensing and counting in secondary pharmaceutical manufacturing. AIChE J 55(5):1161–1170

    Article  Google Scholar 

  12. Conforti D, Guerriero F, Guido R (2008) Optimization models for radiotherapy patient scheduling. 4OR 6:263–278

    Article  Google Scholar 

  13. Conforti D, Guerriero F, Guido R (2010) Non-block scheduling with priority for radiotherapy treatments. Eur J Oper Res 201:289–296

    Article  Google Scholar 

  14. Dean T, Givan R, Leach S (1997) Model reduction techniques for computing approximately optimal solutions for markov decision processes. In: Proceedings of the 13th conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc, pp 124–131

  15. Dearden R, Boutilier C (1997) Abstraction and approximate decision-theoretic planning. Artif Intell 89 (1):219–283

    Article  Google Scholar 

  16. Dietterich TG (2000) Hierarchical reinforcement learning with the maxq value function decomposition. J Artif Intell Res 13(1):227–303

    Google Scholar 

  17. Gedik R (2011) Evaluating the capacity of a proton therapy facility. University of Arkansas, Fayetteville

    Google Scholar 

  18. Gedik R (2014) Large-scale solution approaches for healthcare and supply chain scheduling. Phd thesis, University of Arkansas

  19. Gocgun Y, Bresnahan BW, Ghate A, Gunn ML (2011) A markov decision process approach to multi-category patient scheduling in a diagnostic facility. Artif Intell Med 53(2):73–81

    Article  Google Scholar 

  20. Goitein M, Jermann M (2003) The relative costs of proton and x-ray radiation therapy. Clin Oncol 15:S37–S50

    Article  Google Scholar 

  21. Heyman DP, Sobel MJ (2003) Stochastic models in operations research: stochastic optimization, vol 2. Courier Dover Publications

  22. Kapadia AS, Vineberg SE, Rossi CD (1985) Predicting course of treatment in a rehabilitation hospital: a Markovian model. Comput OR 12(5):459–469

    Article  Google Scholar 

  23. Kapamara T, Sheibani K, Haas O, Reeves C, Petrovic D (2006) A review of scheduling problems in radiotherapy. In: Proceedings of the 18th international conference on systems engineering (ICSE2006). Coventry University, UK, pp 201–207

    Google Scholar 

  24. Kerstiens J, Johnstone PA (2014) Proton therapy expansion under current united states reimbursement models. Int J Radiat Oncol Biol Phys 89(2):235–240

    Article  Google Scholar 

  25. Kolesar P (1970) A markovian model for hospital admission scheduling. Manag Sci 16(6):B384–B396

    Article  Google Scholar 

  26. Li L, Walsh TJ, Littman ML (2006) Towards a unified theory of state abstraction for mdps. In: ISAIM

  27. Men C (2009) Optimization models for radiation therapy: treatment planning and patient scheduling. University of Florida

  28. Men C, Salari E, Romeijn HE, Klein SL, Li Z, Mendenhall NP, Palta JR (2008) Report of the strategic model. Technical report, University of Florida

  29. Nunes LGN, de Carvalho SV, Rodrigues RdCM (2009) Markov decision process applied to the control of hospital elective admissions. Artif Intell Med 47(2):159–171

    Article  Google Scholar 

  30. Peeters A, Grutters JP, Pijls-Johannesma M, Reimoser S, Ruysscher DD, Severens JL, Joore MA, Lambin P (2010) How costly is particle therapy? cost analysis of external beam radiotherapy with carbon-ions, protons and photons. Radiother Oncol 95(1):45– 53

    Article  Google Scholar 

  31. Petrovic D, Castro E, Petrovic S, Kapamara T, Urquhart N (2013) Radiotherapy scheduling. In: Uyar AS, Ozcan E (eds) Automated scheduling and planning. Studies in computational intelligence, vol 505. Springer, Berlin, pp 155–189

    Chapter  Google Scholar 

  32. Petrovic D, Morshed M, Petrovic S (2009) Genetic algorithm based scheduling of radiotherapy treatments for cancer patients. In: Combi C, Shahar Y, Abu-Hanna A (eds) Artificial Intelligence in Medicine. Lecture Notes in Computer Science, vol 5651. Springer, Berlin, pp 101–105

    Google Scholar 

  33. Pham DN, Klinkert A (2008) Surgical case scheduling as a generalized job shop scheduling problem. Eur J Oper Res 185:1011–1025

    Article  Google Scholar 

  34. Powell WB (2007) Approximate dynamic programming: solving the curses of dimensionality. Wiley

  35. Powell WB (2009) What you should know about approximate dynamic programming. Nav Res Logist (NRL) 56(3):239–249

    Article  Google Scholar 

  36. Powell WB, George A, Bouzaiene-Ayari B, Simao HP (2003) Approximate dynamic programming for high dimensional resource allocation problems. In: Proceedings of 2005 IEEE International Joint Conference on Neural Networks, 2005 (IJCNN’05), vol 5. IEEE, pp 2989–2994

  37. Powell WB, Simao HP, Bouzaiene-Ayari B (2012) Approximate dynamic programming in transportation and logistics: a unified framework. EURO J Transport Log 1(3):237–284

    Article  Google Scholar 

  38. Salari E, Men C, Li Z, Mendenhall N, Palta J, Romeijn H. (2009) Su-ff-t-467: A new patient scheduling approach for proton therapy treatment delivery. Med Phys 36(6):2630–2630

    Article  Google Scholar 

  39. Sauré A, Patrick J, Tyldesley S, Puterman ML (2012) Dynamic multi-appointment patient scheduling for radiation therapy. Eur J Oper Res 223(2):573–584

    Article  Google Scholar 

  40. Schütz H-J, Kolisch R (2012) Approximate dynamic programming for capacity allocation in the service industry. Eur J Oper Res 218(1):239–250

    Article  Google Scholar 

  41. Shelley W, Brundage M, Hayter C, Paszat L, Zhou S, Mackillop W (2000) A shorter fractionation schedule for postlumpectomy breast cancer patients. Int J Radiat Oncol Biol Phys 47:1219– 1228

    Article  Google Scholar 

  42. Sickinger S, Kolisch R (2009) The performance of a generalized Bailey–Welch rule for outpatient appointment scheduling under inpatient and emergency demand. Health Care Manag Sci 12(4):408–419

    Article  Google Scholar 

  43. Sutton RS, Precup D, Singh S (1999) Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artif Intell 112(1):181–211

    Article  Google Scholar 

  44. Yamada Y, Ackerman I, Franssen E, Mackenzie RG, Thomas G (1999) Does the dose fractionation schedule influence local control of adjuvant radiotherapy for early stage breast cancer. Int J Radiat Oncol Biol Phys 44:99–104

    Article  Google Scholar 

  45. Zhang NL, Zhang W (2001) Speeding up the convergence of value iteration in partially observable markov decision processes. J Artif Intell Res:29–51

  46. Zietman A, Goitein M, Tepper JE (2010) Technology evolution: is it survival of the fittest? J Clin Oncol 28(27):4275– 4279

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mechanical and Industrial Engineering Department Tagliatela College of Engineering, University of New Haven, West Haven, CT, USA

    Ridvan Gedik

  2. Department of Industrial Engineering, University of Arkansas, Fayetteville, AR, USA

    Shengfan Zhang & Chase Rainwater

Authors
  1. Ridvan Gedik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shengfan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chase Rainwater
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shengfan Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gedik, R., Zhang, S. & Rainwater, C. Strategic level proton therapy patient admission planning: a Markov decision process modeling approach. Health Care Manag Sci 20, 286–302 (2017). https://doi.org/10.1007/s10729-016-9354-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10729-016-9354-6

Keywords

Search

Navigation