Using Markov Models for Decision Support in Management of High Occupancy Hospital Care

  • Sally McClean
  • Peter Millard
  • Lalit Garg
Part of the Studies in Computational Intelligence book series (SCI, volume 109)


We have previously used Markov models to describe movements of patients between hospital states; these may be actual or virtual and described by a phase-type distribution. Here we extend this approach to a Markov reward model for a healthcare system with constant size. This corresponds to a situation where there is a waiting list of patients so that the total number of in-patients remains at a constant level and all admissions are from the waiting list. The distribution of costs is evaluated for any time and expressions derived for the mean cost. The approach is then illustrated by determining average cost at any time for a hospital system with two states: acute/rehabilitative and long-stay.

In addition we develop a Markov model to determine patient numbers and costs at any time where, again, there is a waiting list, so admissions are taken from this list, but we now allow a fixed growth which declines to zero as time tends to infinity. As before, the length of stay is described by a phase-type distribution, thus enabling the representation of durations and costs in each phase within a Markov framework. As an illustration, the model is used to determine costs over time for a four phase model, previously fitted to data for geriatric patients. Such an approach can be used to determine the number of patients and costs in each phase of hospital care and a decision support system and intelligent patient management tool can be developed to help hospital staff, managers and policy makers, thus facilitating an intelligent and systematic approach to the planning of healthcare and optimal use of scarce resources.


Markov Model Decision Support System Markov Chain Modelling Clinical Decision Support System Probability Generate Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. Codrington-Virtue, T. Chaussalet, P. Millard, P. Whittlestone, and J. Kelly (2006) A system for patient management based discrete-event simulation and hierarchical clustering, Computer-Based Medical Systems, 2006, 19th IEEE International Symposium, Page(s):800–804Google Scholar
  2. 2.
    M.J. Faddy and S.I. McClean (1999) Analysing data on lengths of stay of hospital patients using phase-type distributions. Applied Stochastic Models and Data Analysis, 15, 311–317zbMATHGoogle Scholar
  3. 3.
    M.J. Faddy and S.I. McClean (2005) Markov chain modelling for geriatric patient care. Methods of Information in Medicine, 44, 369–373Google Scholar
  4. 4.
    G.W. Harrison and P.H. Millard (1991) Balancing acute and long-stay care: the mathematics of throughput in departments of geriatric medicine. Methods of Information in Medicine, 30, 221–228Google Scholar
  5. 5.
    R.L. Himsworth and M.J. Goldacre (1999) Does time spent in hospital in the final 15 years of life increase with age at death? A population based study. British Medical Journal, 319, 1338–1339Google Scholar
  6. 6.
    V. Irvine, S.I. McClean, and P.H. Millard (1994) Stochastic models for geriatric inpatient behaviour. IMA Journal of Mathematics Applied in Medicine and Biology, 11, 207–216zbMATHCrossRefGoogle Scholar
  7. 7.
    D. Ivatts and P.H. Millard (2002) Health care modeling: why should we try? British Journal of Healthcare Management, 8(6), 218–222Google Scholar
  8. 8.
    R.S. Ledley and L.B. Lusted (1959) Reasoning foundations of medical diagnosis. Science, 130, 9–21CrossRefGoogle Scholar
  9. 9.
    S.I. McClean and P.H. Millard (1993) Modelling in-patient bed usage in a department of geriatric medicine. Methods of Information in Medicine, 32, 79–81Google Scholar
  10. 10.
    S.I. McClean, B. McAlea, and P.H. Millard (1998) Using a Markov reward model to estimate spend-down costs for a geriatric department. Journal of Operational Research Society 10, 1021–1025CrossRefGoogle Scholar
  11. 11.
    S.I. McClean, A.A. Papadopolou, and G. Tsaklides (2004) Discrete time reward models for homogeneous semi-Markov systems, Communications in Statistics: Theory and Methods, 33(3), 623–638zbMATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    G.J. Taylor, S.I. McClean, and P.H. Millard (1999). Stochastic models of geriatric patient bed occupancy behaviour. JRSS, Series A, 163(1), 39–48Google Scholar
  13. 13.
    H.R. Warner, A.F. Toronto, and L. Veasy (1964) Experience with Baye’s theorem for computer diagnosis of congenital heart disease. Annals of the New York Academy of Sciences, 115, 2–16Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Sally McClean
    • 1
  • Peter Millard
    • 2
  • Lalit Garg
    • 1
  1. 1.University of UlsterNorthern Ireland
  2. 2.University of WestminsterLondonUK

Personalised recommendations