Markovian Decision Processes (Semi-Markov and Markov) with Complete Information (Completely Observable)

Abstract

In Figure 4.1(a) a diagrammatic representation of a semi-Markov decision process is presented; special attention to transition probabilities $\left[ {\mathop p\nolimits_{ij,}^k } \right]$ and waiting time probabilities $\mathop h\nolimits_{ij(m)}^k$ is given. A corresponding decision tree for a completely observable semi-Markov process is presented in Figure 4.1(b)