Variance reduction for Markov chain processes using state space evaluation for control variates

Abstract

We develop a method for reducing variance in Monte Carlo simulation of Markov chain processes based on extracting accurate control variates from state space evaluation functions. An example is given in the form of a simple combat model, where the net variance reduction (adjusted for additional calculation) is larger than a factor of 80. We also indicate how our algorithm may be applied to discrete event simulations and system dynamic models with discrete random events.