State-Space Model with One MPP and One Continuous Observation

Abstract

In this chapter, we will derive the EM algorithm equations for a state-space model having an MPP and a continuous-valued variable as its observations. Before looking at the state-space model itself and the equation derivations, we will again first consider a scenario for where the need for such a model arises. We stated earlier that the human body is comprised of multiple internal sub-systems that are networked with one another.

GitHub https://github.com/computational-medicine-lab/A-Tutorial-on-the-Design-of-Bayesian-Filters/tree/main/one_mpp_one_cont

In this chapter, we will derive the EM algorithm equations for a state-space model having an MPP and a continuous-valued variable as its observations. Before looking at the state-space model itself and the equation derivations, we will again first consider a scenario for where the need for such a model arises. We stated earlier that the human body is comprised of multiple internal sub-systems that are networked with one another. The sub-systems perform specialized functions and all work in unison to maintain life. Now multiple functions within the body are regulated by the endocrine system. The endocrine system governs the secretion of a number of hormones that act on different target cells in the body. These hormones largely serve as messengers and help coordinate activities between sub-systems within the body. Functions that hormones are involved in include metabolism, the regulation of blood glucose and appetite, and playing a role in the body’s immune and stress responses, to name a few [101].

The secretory mechanism is pulsatile in the case of a number of hormones. Cortisol is one such example [38]. One of the major functions of cortisol is to raise blood glucose levels in response to external stressors [102, 103]. When the brain interprets sensory inputs as requiring cortisol secretion, the hypothalamus begins to secrete the hormone CRH (corticotropin-releasing hormone). CRH in turn causes the secretion of ACTH (adrenocorticotropic hormone) from the anterior pituitary. Finally, ACTH causes the secretion of cortisol from the adrenal glands [104]. The secretion of cortisol from the adrenal glands has a negative feedback effect suppressing the further secretion of CRH and ACTH [105, 106]. Between 15 and 22 cortisol secretory events typically occur each day in a healthy adult [38, 107]. When cortisol is secreted into the bloodstream, a large percentage of it remains bound [108]. It is the unbound cortisol in the blood that remains physiologically active [109]. This active cortisol aids in energy production at the liver [110, 111]. Since the cortisol concentration in the blood is a continuous variable and its pulsatile secretion forms an MPP, a state-space model for estimating the energy production level related to it should incorporate these types of observations. Similar to the case of skin conductance, a deconvolution procedure can be used to extract the pulsatile profile underlying a series of blood cortisol measurements [107]. Deconvolution also typically yields the infusion and clearance rates necessary to reconstruct a minute-by-minute profile of the cortisol concentration in the blood. Figure 8.1 shows a deconvolved cortisol profile [113].

Fig. 8.1

A deconvolved cortisol profile. Cortisol is secreted in pulses and between 15 and 22 secretory events occur each day in a healthy adult. The figure depicts the blood cortisol measurements taken at 10 min intervals (blue), the reconstructed blood cortisol concentrations at a 1 min resolution (black), and the pulsatile secretions (red). From [112], used under Creative Commons CC-BY license

Alternately, the same MPP plus continuous variable formulation can also be applied to skin conductance. Recall that skin conductance contains both a fast-varying phasic component and a slow-varying tonic component. The phasic component consists of a series of SCRs that are generated by neural impulses. These neural impulses form an MPP. The tonic component, which also reflects sympathetic arousal information, is a continuous observation [60]. Consequently, the state-space model with an MPP and a continuous observation can also be applied to the case of skin conductance. Unlike the case where we had one binary observation and two continuous observations to estimate sympathetic arousal from the same information, the formulation with the MPP and the continuous observation conforms more intuitively to the data itself.

8.1 Deriving the Update Equations in the State Estimation Step

Here again we will assume that $x_{k}$ varies with time following one of the state equations we have already seen. Therefore, no new predict step equations need to be derived.

We made an interesting observation in the previous chapter when deriving the update step equations for the case where $x_{k}$ gives rise to MPP observations. We observed that the update equations switched between those where one binary variable was observed and where both a binary variable and a continuous variable were observed. We will now consider the case where we observe an MPP variable along with a continuous variable. As in (7.9), the density function for our MPP is still

$$\displaystyle \begin{aligned} p(n_{k}\!\cap r_{k}|x_{k})\!=\!\! \begin{cases} 1 - p_{k} = e^{\log(1 - p_{k})}& \!\!\!\text{if } n_{k} = 0 \\ p_{k} \frac{1}{\sqrt{2 \pi \sigma^{2}_{v}}}e^{\frac{-(r_{k} - \gamma_{0} - \gamma_{1}x_{k})^{2}}{2\sigma^{2}_{v}}} = e^{\log(p_{k})} \frac{1}{\sqrt{2 \pi \sigma^{2}_{v}}} e^{\frac{-(r_{k} - \gamma_{0} - \gamma_{1}x_{k})^{2}}{2\sigma^{2}_{v}}} &\!\!\! \text{if } n_{k} = 1, \end{cases} \end{aligned} $$

(8.1)

where $n_{k}$ and $r_{k}$ denote the occurrence of the point process events and the mark amplitudes, respectively. In addition to the MPP, we will now assume that we also observe a continuous variable $s_{k}$ where

$$\displaystyle \begin{aligned} s_{k} &= \delta_{0} + \delta_{1}x_{k} + w_{k}, \end{aligned} $$

(8.2)

and $\delta _{0}$, $\delta _{1}$, and $w_{k}$ have their usual meanings. We observe $s_{k}$ at every point in time. Let us now proceed with deriving the mean and variance for the case when $n_{k} = 0$. The posterior density in this case is

$$\displaystyle \begin{aligned} p(x_{k}|y_{1:k}) &\propto e^{\log(1 - p_{k})} \times e^{\frac{-(s_{k} - \delta_{0} - \delta_{1}x_{k})^{2}}{2\sigma^{2}_{w}}} \times e^{\frac{-(x_{k} - x_{k|k - 1})^{2}}{2 \sigma^{2}_{k|k - 1}}}. \end{aligned} $$

(8.3)

Taking the log on both sides, we have

$$\displaystyle \begin{aligned} q_{1} &= \log(1 - p_{k}) - \frac{(s_{k} - \delta_{0} - \delta_{1}x_{k})^{2}}{2\sigma^{2}_{w}} - \frac{(x_{k} - x_{k|k - 1})^{2}}{2 \sigma^{2}_{k|k - 1}} + \text{constant}. \end{aligned} $$

(8.4)

We will next take the first and second derivatives of $q_{1}$ to obtain the mean and variance.

$$\displaystyle \begin{aligned} \frac{d q_{1}}{d x_{k}} &= -\frac{1}{(1 - p_{k})}p_{k}(1 - p_{k}) + \frac{\delta_{1}(s_{k} - \delta_{0} - \delta_{1}x_{k})}{\sigma^{2}_{w}} - \frac{(x_{k} - x_{k|k - 1})}{\sigma^{2}_{k|k - 1}} = 0 \end{aligned} $$

(8.5)

$$\displaystyle \begin{aligned} \implies 0 &= -p_{k} + \frac{\delta_{1}(s_{k} - \delta_{0} - \delta_{1}x_{k})}{\sigma^{2}_{w}} - \frac{(x_{k} - x_{k|k - 1})}{\sigma^{2}_{k|k - 1}}. \end{aligned} $$

(8.6)

Since $n_{k} = 0$, we can rewrite $-p_{k}$ as $(n_{k} - p_{k})$. Therefore,

$$\displaystyle \begin{aligned} n_{k} - p_{k} + \frac{\delta_{1}(s_{k} - \delta_{0} - \delta_{1}x_{k})}{\sigma^{2}_{w}} &= \frac{(x_{k} - x_{k|k - 1})}{\sigma^{2}_{k|k - 1}}. \end{aligned} $$

(8.7)

But this is identical to (4.21) with $s_{k}$, $\delta _{0}$, and $\delta _{1}$ appearing in the equation instead of $r_{k}$, $\gamma _{0}$, and $\gamma _{1}$. Therefore, similar to (4.26), the update for the mean is

$$\displaystyle \begin{aligned} x_{k} &= x_{k|k - 1} + \frac{\sigma^{2}_{k|k - 1}}{\delta_{1}^{2}\sigma^{2}_{k|k - 1} + \sigma^{2}_{w}}\Big[\sigma^{2}_{w}(n_{k} - p_{k}) + \delta_{1}(s_{k} - \delta_{0} - \delta_{1}x_{k|k - 1})\Big]. \end{aligned} $$

(8.8)

We next take the second derivative of $q_{1}$.

$$\displaystyle \begin{aligned} \frac{d^{2}q_{1}}{dx_{k}^{2}} &= -p_{k}(1 - p_{k}) - \frac{\delta_{1}^{2}}{\sigma^{2}_{w}} - \frac{1}{\sigma^{2}_{k|k -1}}. \end{aligned} $$

(8.9)

This also happens to be identical to (4.28) but with $\delta _{1}$ and $\sigma ^{2}_{w}$ instead of $\gamma _{1}$ and $\sigma ^{2}_{v}$. Therefore, similar to (4.29), the variance update is

$$\displaystyle \begin{aligned} \sigma^{2}_{k|k} &= -\Bigg(\frac{d^{2}q_{1}}{dx_{k}^{2}}\Bigg)^{-1} = \Bigg[\frac{1}{\sigma^{2}_{k|k - 1}} + p_{k|k}(1 - p_{k|k}) + \frac{\delta_{1}^{2}}{\sigma^{2}_{w}}\Bigg]^{-1}. \end{aligned} $$

(8.10)

This is interesting. When we observe both an MPP variable and a continuous variable and $n_{k} = 0$, the update equations are identical to the case where one binary variable and one continuous variable were observed.

We will next consider the case where $n_{k} = 1$ and a non-zero mark $r_{k}$ exists. In this case, the posterior density is

$$\displaystyle \begin{aligned} p(x_{k}|y_{1:k}) &\propto e^{\log(p_{k})} \times e^{\frac{-(r_{k} - \gamma_{0} - \gamma_{1}x_{k})^{2}}{2\sigma^{2}_{v}}} \times e^{\frac{-(s_{k} - \delta_{0} - \delta_{1}x_{k})^{2}}{2\sigma^{2}_{w}}} \times e^{\frac{-(x_{k} - x_{k|k - 1})^{2}}{2 \sigma^{2}_{k|k - 1}}}. \end{aligned} $$

(8.11)

Taking the log value and proceeding to take the first derivative, we have

$$\displaystyle \begin{aligned} & \frac{d q_{2}}{d x_{k}}\!\! =\!\! \frac{1}{p_{k}}p_{k}(1\!-\!p_{k})\! +\! \frac{\gamma_{1}(r_{k}\! -\! \gamma_{0}\! -\! \gamma_{1}x_{k})}{\sigma^{2}_{v}}\! +\! \frac{\delta_{1}(s_{k} - \delta_{0} - \delta_{1}x_{k})}{\sigma^{2}_{w}} - \frac{(x_{k} - x_{k|k - 1})}{\sigma^{2}_{k|k - 1}} \end{aligned} $$

(8.12)

$$\displaystyle \begin{aligned} &= 1 - p_{k} + \frac{\gamma_{1}(r_{k} - \gamma_{0} - \gamma_{1}x_{k})}{\sigma^{2}_{v}} + \frac{\delta_{1}(s_{k} - \delta_{0} - \delta_{1}x_{k})}{\sigma^{2}_{w}} - \frac{(x_{k} - x_{k|k - 1})}{\sigma^{2}_{k|k - 1}}. \end{aligned} $$

(8.13)

Setting this to 0 and replacing $(1 - p_{k})$ with $(n_{k} - p_{k})$ since $n_{k} = 1$, we have

$$\displaystyle \begin{aligned} \frac{d q_{2}}{d x_{k}} &= n_{k} - p_{k} + \frac{\gamma_{1}(r_{k} - \gamma_{0} - \gamma_{1}x_{k})}{\sigma^{2}_{v}} + \frac{\delta_{1}(s_{k} - \delta_{0} - \delta_{1}x_{k})}{\sigma^{2}_{w}} - \frac{(x_{k} - x_{k|k - 1})}{\sigma^{2}_{k|k - 1}} = 0. \end{aligned} $$

(8.14)

But this is identical to (5.21) where we observed one binary variable and two continuous variables. The second derivative of $q_{2}$ yields

$$\displaystyle \begin{aligned} \frac{d^{2}q_{2}}{dx_{k}^{2}} &= -p_{k}(1 - p_{k}) - \frac{\gamma_{1}^{2}}{\sigma^{2}_{v}} - \frac{\delta_{1}^{2}}{\sigma^{2}_{w}} - \frac{1}{\sigma^{2}_{k|k -1}}, \end{aligned} $$

(8.15)

which is the same as (5.28). Therefore, in the case where we observe an MPP variable along with a continuous variable and $n_{k} = 1$, our update equations in the state estimation step are identical to those where we have one binary variable and two continuous variables.

When $x_{k}$ gives rise to MPP observations comprising of the pairs $(n_{k}, r_{k})$ and a continuous observation $s_{k}$, the update equations in the state estimation step are

if $n_{k} = 0$,

$$\displaystyle \begin{aligned} x_{k} &= x_{k|k - 1} + \frac{\sigma^{2}_{k|k - 1}}{\gamma_{1}^{2}\sigma^{2}_{k|k - 1} + \sigma^{2}_{w}}\Big[\sigma^{2}_{w}(n_{k} - p_{k}) + \gamma_{1}(s_{k} - \delta_{0} - \delta_{1}x_{k|k - 1})\Big] \end{aligned} $$

(8.16)

$$\displaystyle \begin{aligned} \sigma^{2}_{k|k} &= \Bigg[\frac{1}{\sigma^{2}_{k|k - 1}} + p_{k|k}(1 - p_{k|k}) + \frac{\delta_{1}^{2}}{\sigma^{2}_{w}}\Bigg]^{-1} \end{aligned} $$

(8.17)

if $n_{k} = 1$,

(8.18)

$$\displaystyle \begin{aligned} \sigma^{2}_{k|k} &= \Bigg[\frac{1}{\sigma^{2}_{k|k - 1}} + p_{k|k}(1 - p_{k|k}) + \frac{\gamma_{1}^{2}}{\sigma^{2}_{v}} + \frac{\delta_{1}^{2}}{\sigma^{2}_{w}}\Bigg]^{-1}. \end{aligned} $$

(8.19)

8.2 Deriving the Parameter Estimation Step Equations

The derivation of the parameter estimation step updates is similar to what we have seen thus far. The updates for the parameters $\gamma _{0}$, $\gamma _{1}$, and $\sigma ^{2}_{v}$ related to $r_{k}$ are calculated based on the subset of values $\tilde {K}$ corresponding to where $n_{k} = 1$. The parameters $\delta _{0}$, $\delta _{1}$, and $\sigma ^{2}_{w}$ corresponding to $s_{k}$ are calculated based on all the observations.

8.3 MATLAB Examples

The MATLAB code examples are contained in the folders shown below:

• one_mpp_one_cont

  • sim∖

    • data_one_mpp_one_cont.mat

    • filter_one_mpp_one_cont.m

  • expm∖

    • expm_data_one_mpp_one_cont.mat

    • expm_filter_one_mpp_one_cont.m

The code itself is quite similar to what we have seen before in earlier examples. It takes in the inputs $n_{k}$, $r_{k}$, and $s_{k}$ denoted by the variables n, r, and s to estimate $x_{k}$. We progress through the repeated predict, update, predict, update, etc. steps with $x_{k|k}$ and $\sigma ^{2}_{k|k}$ being estimated using different equations based on $n_{k}$. The variable names are also largely similar to what we have seen earlier.

8.3.1 Application to Cortisol and Energy

Recall the discussion regarding cortisol at the beginning of this chapter. Cortisol is secreted in pulses and between 15 and 22 of them are secreted by a healthy adult each day. The pulsatile hormone profile forms an MPP. In addition, the amount of unbound cortisol in the blood is biologically active and contributes to energy production. Thus, the observations for estimating the latent cortisol-related energy production state form an MPP and a continuous-valued variable. The cortisol inputs are provided to the code using the variables n, r, and s. The variables n and r denote the MPP observations $n_{k}$ and $r_{k}$. The pulsatile secretions forming the MPP at a resolution of 1 min will need to be extracted via deconvolution (e.g., using [107, 113, 114]). The cortisol infusion and clearance rates yielded by the deconvolution algorithm are used to generate $s_{k}$.

Running the code examples for this particular state-space model produces the results in Fig. 8.2. The code running on experimental data for this model contains a notable difference. In general, when a continuous-valued observation is present, the state estimate can tend to overfit to it. In the experimental code example, the parameter estimation step updates for $\delta _{0}$, $\delta _{1}$, and $\sigma ^{2}_{w}$ (the three parameters related to $s_{k}$) have been adjusted so that only a small step is taken in the direction of the next predicted values at a time. A second change has also been made in that the sensor noise variance $\sigma ^{2}_{w}$ is initialized at a larger value and the same three parameters $\delta _{0}$, $\delta _{1}$, and $\sigma ^{2}_{w}$ are only permitted to update until $\sigma ^{2}_{w}$ reaches a threshold. These two changes greatly help reduce the overfitting to $s_{k}$.

Fig. 8.2

State estimation based on observing one MPP and one continuous variable. The left sub-figure depicts the estimation on simulated data and the right sub-figure depicts the estimation of energy from blood cortisol data. The sub-panels on the left, respectively, depict (a) the MPP observations (blue) and the estimated $r_{k}$ (red), (b) the point process event occurrence probability $p_{k}$ (blue) and its estimate (red), (c) the continuous-valued variable $s_{k}$ (blue) and its estimate (red), (d) the state $x_{k}$ (blue) and its estimate (red), and (e) the QQ plot for the residual error of $x_{k}$. The sub-panels on the right, respectively, depict (a) the deconvolved cortisol pulses (blue) and the fit to $r_{k}$ (red), (b) the reconstructed blood cortisol profile $s_{k}$ (orange) and its estimate (red), (c) the probability of pulse occurrence $p_{k}$ and its 95% confidence limits, and (d) the energy state $x_{k}$ and its 95% confidence limits. From [33], used under Creative Commons CC-BY license