State-Space Model with One Binary, Two Continuous, and a Spiking-Type Observation

Abstract

Spiking-type observations are occasionally recorded in experiments. For instance, neural spiking activity may be recorded from a macaque monkey engaged in a learning experiment or an EKG signal may be recorded from a human subject in an experiment. In such instances, we can model the spiking-type variable using a conditional intensity function (CIF). The CIF is similar to the rate parameter in a Poisson distribution but is more general. With spiking-type observations, we usually assume that our state variable $x_{k}$ affects the rate of spiking through the CIF.

GitHub https://github.com/computational-medicine-lab/A-Tutorial-on-the-Design-of-Bayesian-Filters/tree/main/one_bin_two_cont_one_spk

Spiking-type observations are occasionally recorded in experiments. For instance, neural spiking activity may be recorded from a macaque monkey engaged in a learning experiment or an EKG signal may be recorded from a human subject in an experiment. In such instances, we can model the spiking-type variable using a conditional intensity function (CIF). The CIF is similar to the rate parameter in a Poisson distribution but is more general. With spiking-type observations, we usually assume that our state variable $x_{k}$ affects the rate of spiking through the CIF. Now we need to estimate $x_{k}$ at each time index k. In the case of a spiking-type variable, we typically observe the spiking over a short interval corresponding to time index k. For instance, in the case of a macaque monkey performing a behavioral learning task, we may observe neural spiking over a period of several hundred milliseconds corresponding to each trial k. Each trial duration is then divided into smaller bins indexed over j. Since the spiking-type variables are binary, we assign either $m_{k, j} = 0$ or $m_{k, j} = 1$ within the interval k for each of the smaller time bins j based on spike occurrence. Shown below is an example CIF $\lambda _{k, j}$ used in an experiment where a monkey’s learning state was estimated from measurements that included neural spiking [6].

$$\displaystyle \begin{aligned} \lambda_{k, j} &= e^{\theta_{0} + \psi x_{k} + \sum_{s = 1}^{S}\theta_{s}m_{k, j - s}}. {} \end{aligned} $$

(6.1)

In general, the specific form of the CIF depends on the type of application. In this chapter, we will derive the state and parameter estimation step equations for a model where a spiking-type variable characterized by a general CIF $\lambda _{k, j}$ is observed along with one binary and two continuous variables. We will, however, first consider the need for such a state-space model.

In the preceding chapter, we looked at a state-space model for estimating sympathetic arousal based on one binary and two continuous skin conductance observations. The occurrence of SCRs made up the binary observation $n_{k}$. The continuous observations comprised of a transformed version of the SCR peaks and the tonic level. In reality, the sympathetic nervous system affects a number of organs, not just the skin. We can go to any one of these organs to extract features to estimate arousal. However, not all these organs (or the corresponding physiological signals) are conveniently accessible. The heart is one organ affected by sympathetic activation for which the corresponding signals can be measured easily (e.g., using an EKG). Now sympathetic drive is known to increase heart rate and the force of ventricular contraction [69]. The heart, however, is innervated by both sympathetic and parasympathetic fibers and also has its own pacing mechanism [70]. Consequently, a precise extraction of the sympathetic activation component from an EKG signal is a challenge. In [31], a state-space model based on three skin conductance features (the features just referred to) and EKG signals modeled as spiking observations was used to estimate sympathetic arousal. Here, the model assumed that increased sympathetic arousal caused EKG inter-beat intervals (known as RR-intervals) to decrease (i.e., caused heart rate to increase). The CIF was based on the history-dependent inverse Gaussian (HDIG) probability density function for RR-intervals [71, 72]. The state-space model could be used for wearable healthcare applications (Fig. 6.1). Post-traumatic stress disorder (PTSD), for instance, is known to involve symptoms of hyperarousal [73], while major depression is known to involve low levels of arousal [74]. Thus, a wearable device based on skin conductance and heart rate measurements for monitoring arousal could be used to help care for such patients.

Fig. 6.1

A wearable sensing system for decoding sympathetic arousal. The sweat glands are innervated by sympathetic nerve fibers, and the heart is innervated by both sympathetic and parasympathetic fibers. This information from skin conductance and heart rate can be used to estimate sympathetic arousal. From [26], used under Creative Commons CC-BY license

We also make another notable observation here. The phenomena occurring within the human body and brain are rather complex. Thus, it is likely that no single type of physiological signal or feature captures all the necessary information regarding latent physiological states. If, for instance, both emotional valence and arousal are to be decoded, features from a number of signals could be considered [58, 75,76,77,78]. Signals such as EMG [27, 79,80,81,82], heart rate [83,84,85,86,87], respiration [88,89,90,91,92], and blood flow signals within the brain (functional near infrared spectroscopy) [93,94,95,96,97] all contain information regarding phenomena such as emotion and cognitive effort.

6.1 Deriving the Predict Equations in the State Estimation Step

We have already considered three different cases for the state equation: (i) the simple random walk; (ii) the random walk with a forgetting factor $\rho$; (iii) the random walk with a forgetting factor $\rho$ and an external input $I_{k}$. You would have noticed by now that changes to the state equation primarily affect the predict equations in the state estimation step and not the update equations. The three cases we have considered thus far cover most of the applications that are encountered in typical physiological state estimation problems. In the current state-space model, we will assume that $x_{k}$ evolves with time following one of the state equations we have already seen. Thus no new predict step equations have to be derived. These signals could be used for wearable healthcare applications. A study of how different external stimuli also affect emotion could lead to novel neuromarketing strategies as well [98].

6.2 Deriving the Update Equations in the State Estimation Step

When dealing with a spiking-type observation, we first split our observation interval at time index k into smaller segments and index these smaller bins as $j = 1, 2, \ldots , J$. The joint probability of the spikes over the J observation bins is then [99]

$$\displaystyle \begin{aligned} p(m_{k, 1}, m_{k, 2}, \ldots, m_{k, J}|x_{k}) &= e^{\sum_{j = 1}^{J}\log(\lambda_{k, j}\Delta)m_{k, j} - \lambda_{k, j}\Delta}{}. \end{aligned} $$

(6.2)

Recall from (5.18) that when we had one binary and two continuous observations, the posterior density was

$$\displaystyle \begin{aligned} p(x_{k}|y_{1:k}) &\propto p(n_{k}|x_{k})p(r_{k}|x_{k})p(s_{k}|x_{k})p(x_{k}|n_{1:k - 1}, r_{1:k - 1}, s_{1:k -1 }). \end{aligned} $$

(6.3)

Now that we have the spiking-type observation, we will include $p(m_{k, 1}, m_{k, 2}, \ldots ,$ $m_{k, J}|x_{k})$ in $p(x_{k}|y_{1:k})$ as well. Therefore,

(6.4)

The procedure for deriving the update equations is again similar to what we have seen thus far. As before, we will take the first derivative of the exponent term, set it to 0, and solve for $x_{k}$ to obtain the mean. We will then take the second derivative to obtain the uncertainty or variance associated with the estimate. Taking the log of the posterior density and setting the first partial derivative to 0 yield

$$\displaystyle \begin{aligned} \frac{dq}{dx_{k}} = &\frac{-(x_{k} - x_{k|k - 1})}{\sigma^{2}_{k|k - 1}} + (n_{k} - p_{k}) + \frac{\gamma_{1}(r_{k} - \gamma_{0} - \gamma_{1}x_{k})}{\sigma^{2}_{v}} \\&+ \frac{\delta_{1}(s_{k} - \delta_{0} - \delta_{1}x_{k})}{\sigma^{2}_{w}} + \sum_{j = 1}^{J}\frac{1}{\lambda_{k, j}}\frac{d\lambda_{k, j}}{dx_{k}}(m_{k, j} - \lambda_{k, j}\Delta) = 0 . \end{aligned} $$

(6.5)

Solving for $x_{k}$ is now similar to what we saw in the earlier chapter. We simply need to add and subtract $\gamma _{1}x_{k|k - 1}$ and $\delta _{1}x_{k|k - 1}$ from the terms containing $r_{k}$ and $s_{k}$, respectively. The second partial derivative is

(6.6)

Thus the updates for $x_{k|k}$ and $\sigma ^{2}_{k|k}$ turn out to be

(6.7)

$$\displaystyle \begin{aligned} \sigma^{2}_{k|k} &= \Bigg\{\frac{1}{\sigma^{2}_{k|k - 1}} + p_{k|k}(1 - p_{k|k}) + \frac{\gamma^{2}_{1}}{\sigma^{2}_{v}} + \frac{\delta^{2}_{1}}{\sigma^{2}_{w}} - \sum_{j = 1}^{J} \\ & \Bigg[\frac{1}{\lambda_{k, j|k}}\frac{d^{2}\lambda_{k, j|k}}{dx_{k}^{2}}(m_{k, j} - \lambda_{k, j|k}\Delta) - \frac{m_{k, j}}{\lambda^{2}_{k, j|k}}\Bigg(\frac{d\lambda_{k, j|k}}{dx_{k}}\Bigg)^{2}\Bigg]\Bigg\}^{-1}. \end{aligned} $$

(6.8)

Note that the equations may simplify further depending on the specific form of the CIF. Here we have provided the derivations for the general case.

When $x_{k}$ gives rise to a binary observation $n_{k}$, two continuous observations $r_{k}$ and $s_{k}$ and a spiking-type observation $m_{k, j}$ characterized by the CIF $\lambda _{k, j}$, the update equations in the state estimation step are

(6.9)

(6.10)

6.3 Deriving the Parameter Estimation Step Equations

The state-space model we consider here is an extension of what we considered in the previous chapter that contained one binary and two continuous observations. Therefore, the only new parameter estimation step equations we need to derive are for the spiking-type variable.

6.3.1 Deriving the Coefficients Within a CIF

A CIF can take different forms depending on the type of application. For instance, when neural spiking data are involved, $\log (\lambda _{k, j})$ may be expressed as a linear sum of history-dependent terms and $x_{k}$ as in (6.1). If this is the case, we would have to determine $\psi$ and the $\theta _{s}$’s at the parameter estimation step. When heartbeats are modeled as a spiking-type variable, the CIF involves an inverse Gaussian distribution and could be related to $x_{k}$ through its mean [31]. Thus, the terms to be derived at the parameter estimation step when a spiking-type variable is present are application-specific. In general, due to the rather complicated nature of a CIF, the parameter estimation step updates do not have neat closed-form expressions. Instead, the parameters have to be chosen to maximize the expected log-likelihood

$$\displaystyle \begin{aligned} Q &= \sum_{k = 1}^{K}\sum_{j = 1}^{J}\mathbb{E}\Big[\log(\lambda_{k, j}\Delta)m_{k, j} - \lambda_{k, j}\Delta\Big]{}. \end{aligned} $$

(6.11)

The form of Q can be deduced from (6.2). The trick to maximizing Q is to perform a Taylor expansion around the mean $x_{k|K} = \mathbb {E}[x_{k}]$ for each of the summed terms. Therefore, when the expected value is finally calculated, we will end up with terms like $\mathbb {E}[x_{k} - x_{k|K}]$ and $\mathbb {E}[(x_{k} - x_{k|K})^{2}]$ in the expansion. Now

$$\displaystyle \begin{aligned} \mathbb{E}[x_{k} - x_{k|K}] &= \mathbb{E}[x_{k}] - x_{k|K} \enspace \text{based on (2.2)} \end{aligned} $$

(6.12)

$$\displaystyle \begin{aligned} &= x_{k|K} - x_{k|K} = 0, \end{aligned} $$

(6.13)

and $\mathbb {E}[(x_{k} - x_{k|K})^{2}]$ is the variance $\sigma ^{2}_{k|K}$. These two facts will greatly help simplify the calculation of Q.

Let us now perform the Taylor expansion around $x_{k|K}$ [6]. The summed term within the expected value simplifies to

$$\displaystyle \begin{aligned} \log(\lambda_{k, j}\Delta)m_{k, j} - \lambda_{k, j}\Delta \approx &\log(\lambda_{k, j|K}\Delta)m_{k, j} - \lambda_{k, j|K}\Delta \\ &+ \frac{1}{\lambda_{k, j|K}}\frac{\partial \lambda_{k, j|K}}{\partial x_{k}}(m_{k, j} - \lambda_{k, j|K}\Delta)(x_{k} - x_{k|K}) \\ &+ \frac{1}{2}\Bigg[\frac{1}{\lambda_{k, j|K}}\frac{\partial^{2}\lambda_{k, j|K}}{\partial x_{k}^{2}}(m_{k, j} - \lambda_{k, j|K}\Delta) \\ &- \frac{m_{k, j}}{\lambda^{2}_{k, j|K}}\Bigg(\frac{\partial \lambda_{k, j|K}}{\partial x_{k}}\Bigg)^{2}\Bigg](x_{k} - x_{k|K})^{2}. \end{aligned} $$

(6.14)

Taking the expected value, we have

$$\displaystyle \begin{aligned} \mathbb{E}\big[\log(\lambda_{k, j}\Delta)m_{k, j} - \lambda_{k, j|K}\Delta\big] \approx &\log(\lambda_{k, j|K}\Delta)m_{k, j} - \lambda_{k, j|K}\Delta \\&+ \frac{1}{\lambda_{k, j|K}}\frac{\partial \lambda_{k, j|K}}{\partial x_{k}}(m_{k, j} - \lambda_{k, j|K}\Delta)\mathbb{E}\big[x_{k} - x_{k|K}\big] \\&+ \frac{1}{2}\Bigg[\frac{1}{\lambda_{k, j|K}}\frac{\partial^{2}\lambda_{k, j|K}}{\partial x_{k}^{2}}(m_{k, j} - \lambda_{k, j|K}\Delta)\\ &- \frac{m_{k, j}}{\lambda^{2}_{k, j|K}}\Bigg(\frac{\partial \lambda_{k, j|K}}{\partial x_{k}}\Bigg)^{2}\Bigg]\\ &\times\mathbb{E}\big[x_{k} - x_{k|K}\big]^{2}. \end{aligned} $$

(6.15)

Note the terms $\mathbb {E}[x_{k} - x_{k|K}]$ and $\mathbb {E}[(x_{k} - x_{k|K})^{2}]$ in the expression above. The first of these is 0, and the second is the variance $\sigma ^{2}_{k|K}$. Therefore,

$$\displaystyle \begin{aligned} \mathbb{E}\big[\log(\lambda_{k, j}\Delta)m_{k, j} - \lambda_{k, j|K}\Delta\big] \approx &\log(\lambda_{k, j|K}\Delta)m_{k, j} - \lambda_{k, j|K}\Delta + 0 \\ &+ \frac{1}{2}\Bigg[\frac{1}{\lambda_{k, j|K}}\frac{\partial^{2}\lambda_{k, j|K}}{\partial x_{k}^{2}}(m_{k, j} - \lambda_{k, j|K}\Delta) \\ &- \frac{m_{k, j}}{\lambda^{2}_{k, j|K}}\Bigg(\frac{\partial \lambda_{k, j|K}}{\partial x_{k}}\Bigg)^{2}\Bigg]\sigma^{2}_{k|K}. \end{aligned} $$

(6.16)

Consequently, Q approximately simplifies to

$$\displaystyle \begin{aligned} Q \approx &\sum_{k = 1}^{K}\sum_{j = 1}^{J} \log(\lambda_{k, j|K}\Delta)m_{k, j} - \lambda_{k, j|K}\Delta \\&+ \frac{1}{2}\Bigg[\frac{1}{\lambda_{k, j|K}}\frac{\partial^{2}\lambda_{k, j|K}}{\partial x_{k}^{2}}(m_{k, j} - \lambda_{k, j|K}\Delta) - \frac{m_{k, j}}{\lambda^{2}_{k, j|K}}\Bigg(\frac{\partial \lambda_{k, j|K}}{\partial x_{k}}\Bigg)^{2}\Bigg]\sigma^{2}_{k|K}{}. \end{aligned} $$

(6.17)

In general, Q will have to be maximized with respect to the model parameters in the CIF using numerical methods.

The parameter estimation step updates for the terms in a CIF $\lambda _{k, j}$ when we observe a spiking-type variable $m_{k, j}$ are chosen to maximize

$$\displaystyle \begin{aligned} Q \approx &\sum_{k = 1}^{K}\sum_{j = 1}^{J} \log(\lambda_{k, j|K}\Delta)m_{k, j} - \lambda_{k, j|K}\Delta \\&+ \frac{1}{2}\Bigg[\frac{1}{\lambda_{k, j|K}}\frac{\partial^{2}\lambda_{k, j|K}}{\partial x_{k}^{2}}(m_{k, j} - \lambda_{k, j|K}\Delta) - \frac{m_{k, j}}{\lambda^{2}_{k, j|K}}\Bigg(\frac{\partial \lambda_{k, j|K}}{\partial x_{k}}\Bigg)^{2}\Bigg]\sigma^{2}_{k|K}. \end{aligned} $$

(6.18)

6.4 MATLAB Examples

MATLAB code examples for simulated and experimental data for the state-space model with one binary, two continuous, and one spiking-type observation are provided in the folders shown below:

• one_bin_two_cont_one_spk

  • sim∖

    • data_one_bin_two_cont_one_spk.mat

    • filter_one_bin_two_cont_one_spk.m

  • expm∖

    • expm_data_one_bin_two_cont_one_spk.mat

    • expm_filter_one_bin_two_cont_one_spk.m

6.4.1 Application to Skin Conductance, Heart Rate and Sympathetic Arousal

The state-space model described in this chapter was used in [31] to estimate sympathetic arousal from skin conductance and heart rate measurements. The skin conductance observations are the same three that were used in [29] (discussed in the previous chapter). Thus, the only new observation added here relates to heart rate for which some additional discussion is necessary.

The code examples estimate arousal from the four observations related to skin conductance and heart rate. The R-peaks in the EKG signals are taken to form the spiking observations. If L consecutive R-peaks occur at times $u_{l}$ within $(0, T]$ such that $0 < u_{1} < u_{2} < \ldots < u_{L} \leq T$, and $h_{l} = u_{l} - u_{l - 1}$ is the l^th RR-interval, the HDIG density function for the RR-intervals at $t > u_{l}$ is

$$\displaystyle \begin{aligned} g(t|u_{l}) &= \sqrt{\frac{\theta_{q + 1}}{2 \pi (t - u_{l})^{3}}} ~\text{exp}\Bigg\{\frac{-\theta_{q + 1}[t - u_{l} - \mu]^{2}}{2{\mu}^{2}(t - u_{l})}\Bigg\}, \end{aligned} $$

(6.19)

where q is the model order, $\theta _{q + 1}$ is related to the variance, and the mean is

$$\displaystyle \begin{aligned} \mu = \theta_{0} + \sum_{i = 1}^{q}\theta_{i}h_{l - i + 1} + \eta x_{k}, \end{aligned} $$

(6.20)

where $\eta$ is a coefficient to be determined. Accordingly, a change in sympathetic arousal $x_{k}$ causes the mean of the HDIG density function to shift (i.e., heart rate speeds up or slows down depending on the arousal level). The CIF $\lambda _{k, j}$ is

$$\displaystyle \begin{aligned} \lambda_{k, j} &\triangleq \frac{g(t_{k, j}|u_{k, j})}{1 - \int_{u_{k, j}}^{t_{k, j}}g(z|u_{k, j}) dz}, \end{aligned} $$

(6.21)

where $u_{k, j}$ is the time of occurrence of the last R-peak prior to $t_{k, j}$. The CIF $\lambda _{k, j}$ is calculated every $\Delta = 5$ ms [23, 71]. Since skin conductance is typically analyzed at 4 Hz ($t_{s} = 250$ ms), there are $250 / 5 = 50$ smaller observation bins j for heart rate at each time index k. Due to computational complexity, the $\theta _{i}$’s were estimated separately in an offline manner using maximum likelihood. Now the work by Barbieri et al. [71] was one of the earliest to perform point process analysis of EKG RR-intervals using the HDIG density function.¹ The EM algorithm in [31] was executed for several different values of $\eta$, and the best one was selected based on a maximization of the log-likelihood term in (6.17). Note also that since the experimental code example involves skin conductance and heart rate with $\Delta = 5$ ms bins, the heart rate observations need to be provided to the code in a manner similar to that contained in the .mat file.

The other aspects of the code and the variable names are similar to what was described in earlier chapters. Running the code examples on simulated and experimental data yields the results shown in Fig. 6.2. The experimental data results are from the Pavlovian fear conditioning experiment in [100]. As shown in the figure, the CS+ trials with the electric shock have the highest average responses, while the CS- trials have the lowest average responses for the subject considered. The CS+ trials without the shock have an intermediate response.

Fig. 6.2

State estimation based on observing one binary, two continuous, and one spiking-type variable. The left sub-figure depicts estimation on simulated data, and the right sub-figure depicts the estimation of sympathetic arousal from skin conductance and heart rate data. The sub-panels on the left, respectively, depict: (a) the probability of binary event occurrence $p_{k}$ (blue) and its estimate (red) (the green and black dots above at the top denote the presence or absence of binary events, respectively); (b) the first continuous variable $r_{k}$ (blue) and its estimate (red); (c) the second continuous variable $s_{k}$ (blue) and its estimate (red); (d) the state $x_{k}$ (blue) and its estimate (red) (the cyan and black dots denote the presence or absence of external binary inputs, respectively); (e) the simulated RR-interval sequence (orange) and the fit to the HDIG mean; (f) the QQ plot for the residual error of $x_{k}$. The sub-panels on the right, respectively, depict: (a) the skin conductance signal $z_{k}$; (b) the probability of SCR occurrence $p_{k}$ (the green and black dots on top denote the presence or absence of SCRs, respectively); (c) the phasic-derived variable (green solid) and its estimate (dotted); (d) the tonic level $s_{k}$ (pink solid) and its estimate (dotted); (e) the arousal state $x_{k}$ (the cyan and black dots denote the presence or absence of external binary inputs, respectively); (f) the RR-interval sequence (orange) and the fit to the HDIG mean; (g) the averages corresponding to different trials for skin conductance (CS$-$ —green, CS+ without the shock—mauve and CS+ with the shock—red); (h) the same averages for the state. From [31], used under Creative Commons CC-BY license