# DEVS Support for Markov Modeling and Simulation

## Abstract

Markov Modeling is among the most commonly used forms of model expression. Besides their general usefulness, the Markov concepts of stochastic modeling are implicitly at the heart of most forms of discrete-event simulation. This chapter, an addition to the second edition, shows how such concepts are fully compatible with the DEVS characterization of discrete-event models and a natural basis for the extended and integrated Markov Modeling facility developed within the MS4 Me environment. The facility described here offers an easy-to-use set of tools to develop Markov models which are full-fledged DEVS models and able to be integrated with other DEVS models just like other DEVS models. Due to their transition structure, Markov models can be individualized with specific transition probabilities/rates which can be changed during model execution for dynamic structural change. Finally, we discuss case studies where such modeling can provide significant insights through multi-resolution modeling in drug therapy and how speedup using parallel processing is limited by the interprocessor connection network. An appendix gives some background for this chapter on exponential distributions, Poisson processes, and Markov basics.

## References

- Feller, W. (1966).
*An introduction to probability theory and its applications*(pp. 1–2), Wiley.Google Scholar - Karnon, J., & Afzali, H. H. A. (2014). When to use discrete-event simulation (DES) for the economic evaluation of health technologies? A review and critique of the costs and benefits of DES.
*Pharmaco Economics*,*32*, 547–558.Google Scholar - Kemeny, J. G., & Snell, J. L. (1960).
*Finite markov chains*. v. Nostrand.Google Scholar - Kleijnen, J. P. C. (2008).
*Design and analysis of simulation experiments*. Springer.Google Scholar - Nutaro, J., & Zeigler, B. P. (2017). How to apply amdahl’s law to multithreaded multicore processors.
*Parallel and Distributed Computing*. http://dx.doi.org/10.1016/j.jpdc.2017.03.006. - Özmen, Ö., Nutaro, J. J., Pullum, L. L., Ramanathan, A. (2016). Analyzing the impact of modeling choices and assumptions in compartmental epidemiological models.
*Transactions of SCS*,*92*(5), 459–472. Retrieved May 1, 2016, from doi: 10.1177/0037549716640877. - Soares, M.O. and Castro, L.L.C. (2012) ‘Continuous time simulation and, discretized models for cost-effectiveness analysis’,
*Pharmacoeconomics, 30*(12), 1101–1117, doi: 10.2165/11599380-000000000-00000. - Zeigler, B. P. (2017). Using the parallel DEVS Protocol for general robust simulation with near optimal performance.
*IEEE, Computing in Science & Engineering*,*19*, 68, 68–77. doi: 10.1109/MCSE.2017.52.