MCMC implementation for Bayesian hidden semi-Markov models with illustrative applications

Abstract

Hidden Markov models (HMMs) are flexible, well-established models useful in a diverse range of applications. However, one potential limitation of such models lies in their inability to explicitly structure the holding times of each hidden state. Hidden semi-Markov models (HSMMs) are more useful in the latter respect as they incorporate additional temporal structure by explicit modelling of the holding times. However, HSMMs have generally received less attention in the literature, mainly due to their intensive computational requirements. Here a Bayesian implementation of HSMMs is presented. Recursive algorithms are proposed in conjunction with Metropolis-Hastings in such a way as to avoid sampling from the distribution of the hidden state sequence in the MCMC sampler. This provides a computationally tractable estimation framework for HSMMs avoiding the limitations associated with the conventional EM algorithm regarding model flexibility. Performance of the proposed implementation is demonstrated through simulation experiments as well as an illustrative application relating to recurrent failures in a network of underground water pipes where random effects are also included into the HSMM to allow for pipe heterogeneity.

Appendices

Appendix A: Pseudo code for simulation experiment with Gaussian observations

Simulate data from 2-state Gaussian HSMM, Eq. (10):

• 1. Set values for μ _S ,σ,ϕ _S and π _S for S=1,2.

• 2. Set i=1 and sample initial state S _i from π={π _S }.

• 3. Sample duration τ _i of S _i , from zero-truncated Poisson with parameter $\phi_{S_{i}}$.

• 4. Sample state transition from distribution corresponding to $S_{1}^{\mathrm{th}}$ row of P.

• 5. Repeat steps 3 and 4 until ∑ _i τ _i ≥250.

• 6. If ∑ _i τ _i >250, truncate so that ∑ _i τ _i =250.

• 7. For each T=1,…,250, sample from N(μ _S ,σ ²) according to which state S the chain is at time step T.

Metropolis-Hastings:

• Set i=0 and initialise parameters $\mu_{S}^{(i)}, \sigma^{(i)}, \phi _{S}^{(i)}$ and $\pi_{S}^{(i)}$.

• Calculate ℓ ⁽ⁱ⁾, the log-likelihood using the forward algorithm in Sect. 3.2.

• Calculate P ⁽ⁱ⁾, the log-posterior by

  

  (15)

  where p(⋅) is the prior for each parameter (Table 1).

Do for i=1,…,M where M is the required number of MCMC iterations:

μ _S ::

$\mu_{S}^{\ast}= \mu_{S}^{(i-1)} + \varepsilon$, $\varepsilon\sim\mbox {N}(0,\sigma^{2}_{\mu})$

Calculate ℓ ^∗ and hence P ^∗ using Eq. (15)

Calculate acceptance probability as η=min(1,Ω) where Ω=exp{P ^∗−P ⁽ⁱ⁻¹⁾}

Sample U from U(0,1) If U≤η, set $\mu_{S}^{(i)}=\mu_{S}^{\ast}$, ℓ ⁽ⁱ⁾=ℓ ^∗ and P ⁽ⁱ⁾=P ^∗.

σ::

σ ^∗=exp{log(σ ⁽ⁱ⁻¹⁾)+ε}, $\varepsilon \sim\mbox{N}(0,\sigma^{2}_{\sigma})$

Calculate ℓ ^∗ and hence P ^∗ using Eq. (15)

Calculate acceptance probability as η=min(1,Ω) where Ω=exp{P ^∗+log(σ ^∗)−[P ⁽ⁱ⁻¹⁾+log(σ ⁽ⁱ⁻¹⁾)]}

Sample U from U(0,1)

If U≤η, set σ ⁽ⁱ⁾=σ ^∗, ℓ ⁽ⁱ⁾=ℓ ^∗ and P ⁽ⁱ⁾=P ^∗.

ϕ _S :

Same as σ, replacing ϕ _S with σ.

π _S :

Sample π ^∗∼Dir(απ ⁽ⁱ⁻¹⁾)

Calculate ℓ ^∗ and hence P ^∗ using Eq. (15)

Calculate acceptance probability as η=min(1,Ω) where

where d(π;θ) is a Dirichlet density with parameter θ

Sample U from U(0,1)

If U≤η, set $\pi_{S}^{(i)}=\pi_{S}^{\ast}$, ℓ ⁽ⁱ⁾=ℓ ^∗ and P ⁽ⁱ⁾=P ^∗.

Adjust $\sigma^{2}_{\mu}, \sigma^{2}_{\sigma}, \sigma^{2}_{\phi}$ and α to achieve desired acceptance rates.

Note that R code for implementing the HSMMs and HMMs from both simulation experiments is available both as supplementary material to the paper and on http://empslocal.ex.ac.uk/people/staff/te201/HSMM/.

Appendix B: Pseudo code for simulation experiment with NHPP observations

Simulate data from 3-state NHPP-HSMM with intensity function λ(t;x|S) as in Eq. (12), for 50 hypothetical objects:

• 1. Set values for θ _S ,β ₀,β ₁,ϕ _S ,π _S and P for S=1,2,3.

• 2. Sample B _j ∼U(80,150), for j=1,…,50, to decide the observation period in discrete time steps for each object.

• 3. Sample x _j ∼U(50,150) to set values for the covariate x.

• 4. Set j=1, i=1 and sample initial state S _i from π={π _S }.

• 5. Sample duration τ _i of S _i , from zero-truncated Poisson with parameter $\phi_{S_{i}}$.

• 6. Sample state transition from distribution corresponding to $S_{1}^{\mathrm{th}}$ row of P.

• 7. Repeat steps 3 and 4 until ∑ _i τ _i ≥B _j .

• 8. If ∑ _i τ _i >B _j , truncate so that ∑ _i τ _i =B _j .

• 9. For each T=1,…,B _j , sample from Pois(Λ([T−1,T]|S)) according to which state S the chain is at time step T, where $\varLambda([T-1,T]|S)=\int_{T-1}^{T} \lambda(t;x|S) dt$.

• 10. Repeat steps 4–9 for j=2,…,50.

Metropolis-Hastings:

• Very similar to the one in Appendix A.

• The forward algorithm needs to be ran for each of the j=1,…,50 objects so that the log-likelihood $\ell^{(i)} = \sum_{j=1}^{50} \ell_{j}^{(i)}$ (assuming independence of the objects).

• The log-posterior is obtained essentially as in Eq. (15). However, we restrict the posterior to be zero in areas of the parameter space other than θ ₁<θ ₂<θ ₃ to impose the label-switching measure.

• Strictly positive parameters such as θ _S and ϕ _S are updated the same as σ and ϕ _S in Appendix A.

• Parameter π and vectors made up of rows from transition matrix P, excluding zero elements, are updated exactly like π in Appendix A.