An efficient pattern mining approach for event detection in multivariate temporal data

Abstract

This work proposes a pattern mining approach to learn event detection models from complex multivariate temporal data, such as electronic health records. We present recent temporal pattern mining, a novel approach for efficiently finding predictive patterns for event detection problems. This approach first converts the time series data into time-interval sequences of temporal abstractions. It then constructs more complex time-interval patterns backward in time using temporal operators. We also present the minimal predictive recent temporal patterns framework for selecting a small set of predictive and non-spurious patterns. We apply our methods for predicting adverse medical events in real-world clinical data. The results demonstrate the benefits of our methods in learning accurate event detection models, which is a key step for developing intelligent patient monitoring and decision support systems.

Appendix: The Bayesian score: mathematical derivation and computational complexity

In Sect. 5.1, we briefly introduced the Bayesian score of RTP $P$ for predicting class label $y$ compared to a more general group $G$: $G_P\!\subset \!G$. In this “Appendix”, we derive the marginal likelihood for models $M_e,\,M_h$ and $M_l$, which are required for computing the Bayesian score (solving Eq. 1). “The closed-form solution of the marginal likelihood for model $M_e$” of appendix describes the closed-form solution for computing $P(G|M_e)$, which is the marginal likelihood for model $M_e$ (the probability of $y$ is the same inside and outside $G_P$). “Deriving a closed-form solution of the marginal likelihood for model $M_h$” of appendix derives the closed form solution for computing $P(G|M_h)$, which is the marginal likelihood for model $M_h$ (the probability of $y$ in $G_P$ is higher than outside $G_P$). “Four equivalent solutions of the marginal likelihood for model $M_h$” of appendix shows the four equivalent formulas for computing $P(G|M_h)$. “Deriving a closed-form solution of the marginal likelihood for model $M_l$” of appendix illustrates how to obtain the marginal likelihood for model $M_l$ (the probability of $y$ in $G_P$ is lower than outside $G_P$) directly from the solution to $P(G|M_h)$. Finally, “Computational complexity” of appendix analyzes the overall computational complexity for computing the Bayesian score.

1.1 The closed-form solution of the marginal likelihood for model $M_e$

Let us start by defining the marginal likelihood for model $M_e$. This model assumes that all instances in $G$ have the same probability of having label $Y=y$. Let us denote this probability by $\theta$. To represent our uncertainty about $\theta$, we use a beta distribution with parameters $\alpha$ and $\beta$. Let $N_{*1}$ be the number of instances in $G$ with $Y=y$ and let $N_{*2}$ be the number of instances in $G$ with $Y\!\ne \!y$ (i.e., instances that do not have class label $y$). The marginal likelihood for model $M_e$ is as follows:

$$\begin{aligned} Pr(G|M_e)=\int _{\theta =0}^1 {\theta }^{N_{*1}} \cdot {(1-\theta )}^{N_{*2}} \cdot beta({\theta }; {\alpha }, {\beta }) d{\theta } \end{aligned}$$

The above integral yields the following well-known closed-form solution [19]:

$$\begin{aligned} Pr(G|M_e)=\frac{\varGamma (\alpha +\beta )}{\varGamma (\alpha +N_{*1}+\beta +N_{*2})} \cdot \frac{\varGamma (\alpha +N_{*1})}{\varGamma (\alpha )} \cdot \frac{\varGamma (\beta +N_{*2})}{\varGamma (\beta )} \end{aligned}$$

(2)

where $\varGamma$ is the gamma function.

1.2 Deriving a closed-form solution of the marginal likelihood for model $M_h$

Now let us now define the marginal likelihood for model $M_h$. This model assumes that the probability of $Y=y$ for the instances in $G_P$, denoted by $\theta _1$, is higher than the probability of $Y=y$ for the instances of $G$ that are outside $G_P$ (G $\setminus$ G $_P$), denoted by $\theta _2$. To represent our uncertainty about $\theta _1$, we use a beta distribution with parameters $\alpha _1$ and $\beta _1$. To represent our uncertainty about $\theta _2$, we use a beta distribution with parameters $\alpha _2$ and $\beta _2$. Let $N_{11}$ and $N_{12}$ be the number of instances in $G_P$ with $Y=y$ and with $Y\!\ne \!y$, respectively. Let $N_{21}$ and $N_{22}$ be the number of instances outside $G_P$ with $Y=y$ and with $Y\!\ne \!y$, respectively (see Fig. 12).

Fig. 12

A diagram illustrating model $M_h$

The marginal likelihood for model $M_h$ ($Pr(G|M_h)$) is defined as follows:

$$\begin{aligned}&\frac{1}{k}\int _{\theta _1=0}^1 \int _{\theta _2=0}^{\theta _1} {\theta _1}^{N_{11}} \cdot {(1-\theta _1)}^{N_{12}} \cdot {\theta _2}^{N_{21}} \cdot {(1-\theta _2)}^{N_{22}} \cdot beta({\theta _1}; {\alpha _1}, {\beta _1})\nonumber \\&\qquad \cdot beta({\theta _2}; {\alpha _2}, {\beta _2}) d{\theta _2} d{\theta _1}\nonumber \\&\quad =\frac{1}{k}\underbrace{\int _{\theta _1=0}^1 {\theta _1}^{N_{11}} \cdot {(1-\theta _1)}^{N_{12}} \cdot beta({\theta _1}; {\alpha _1}, {\beta _1})}_{f_1}\nonumber \\&\quad \times \underbrace{\int _{\theta _2=0}^{\theta _1} {\theta _2}^{N_{21}} \cdot {(1-\theta _2)}^{N_{22}} \cdot beta({\theta _2}; {\alpha _2}, {\beta _2}) d{\theta _2}}_{f_2} d{\theta _1} \end{aligned}$$

(3)

where $k$ is a normalization constant for the parameter prior. Note that we do not assume that the parameters are independent, but rather we constrain $\theta _1$ to be higher than $\theta _2$.

To solve Eq. 3, we first show how to solve the integral over $\theta _2$ in closed form, which is denoted by $f_2$ in Eq. 3. We then expand the function denoted by $f_1$, multiply it by the solution to $f_2$, and solve the integral over $\theta _1$ in closed form to complete the integration.

We use the regularized incomplete beta function [1] to solve the integral given by $f_2$, which is as follows:

$$\begin{aligned} \int _{\theta =0}^{x} {\theta }^{a-1} \cdot {(1\!-\!\theta )}^{b-1} d{\theta } = \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (a\!+\!b)} \cdot \sum _{j=a}^{a+b-1} \frac{\varGamma (a\!+\!b)}{\varGamma (j\!+\!1) \cdot \varGamma (a\!+\!b\!-\!j)} \cdot x^j \cdot (1\!-\!x)^{a+b-1-j} \end{aligned}$$

(4)

where $a$ and $b$ should be natural numbers.

Note that when $x=1$ in Eq. 4, the solution to the integral in that equation is simply the following:

$$\begin{aligned} \int _{\theta =0}^{1} {\theta }^{a-1} \cdot {(1-\theta )}^{b-1} d{\theta } = \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (a+b)} \end{aligned}$$

(5)

We now solve the integral given by $f_2$ in Eq. 3 as follows:

$$\begin{aligned} f_2&= \int _{\theta _2=0}^{\theta _1} {\theta _2}^{N_{21}} \cdot {(1-\theta _2)}^{N_{22}} \cdot beta(\theta _2; \alpha _2, \beta _2) d \theta _2\\&= \int _{\theta _2=0}^{\theta _1} {\theta _2}^{N_{21}} \cdot {(1-\theta _2)}^{N_{22}} \cdot \frac{\varGamma (\alpha _2+\beta _2)}{\varGamma (\alpha _2) \cdot \varGamma (\beta _2)} \cdot \theta _2^{\alpha _2-1} \cdot (1-\theta _2)^{\beta _2-1} d \theta _2\\&= \frac{\varGamma (\alpha _2+\beta _2)}{\varGamma (\alpha _2) \cdot \varGamma (\beta _2)} \int _{\theta _2=0}^{\theta _1} {\theta _2}^{N_{21}+\alpha _2-1} \cdot {(1-\theta _2)}^{N_{22}+\beta _2-1} d \theta _2\\&= \frac{\varGamma (\alpha _2+\beta _2)}{\varGamma (\alpha _2) \cdot \varGamma (\beta _2)} \int _{\theta _2=0}^{\theta _1} {\theta _2}^{a-1} \cdot {(1-\theta _2)}^{b-1} d \theta _2 \end{aligned}$$

where $a=N_{21}+\alpha _2$ and $b=N_{22}+\beta _2$.

Using Eq. 4, we get the following:

$$\begin{aligned} f_2=\frac{\varGamma (\alpha _2+\beta _2)}{\varGamma (\alpha _2) \cdot \varGamma (\beta _2)} \cdot \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (a+b)} \cdot \sum _{j=a}^{a+b-1} \frac{\varGamma (a+b)}{\varGamma (j+1) \cdot \varGamma (a+b-j)} \cdot \theta _1^j \cdot (1-\theta _1)^{a+b-1-j} \end{aligned}$$

(6)

We now turn to $f_1$, which can be expanded as follows:

$$\begin{aligned} f_1&= \int _{\theta _1=0}^1 {\theta _1}^{N_{11}} \cdot {(1-\theta _1)}^{N_{12}} \cdot beta({\theta _1}; {\alpha _1}, {\beta _1})\nonumber \\&= \int _{\theta _1=0}^1 {\theta _1}^{N_{11}} \cdot {(1-\theta _1)}^{N_{12}} \cdot \frac{\varGamma (\alpha _1+\beta _1)}{\varGamma (\alpha _1) \cdot \varGamma (\beta _1)} \cdot \theta _1^{\alpha _1-1} \cdot (1-\theta _1)^{\beta _1-1}\nonumber \\&= \frac{\varGamma (\alpha _1+\beta _1)}{\varGamma (\alpha _1) \cdot \varGamma (\beta _1)} \int _{\theta _1=0}^1 {\theta _1}^{c-1} \cdot {(1-\theta _1)}^{d-1} \end{aligned}$$

(7)

where $c=N_{11}+\alpha _1$ and $d=N_{12}+\beta _1$.

Now, we combine Eqs. 6 and 7 to solve Eq. 3:

$$\begin{aligned} Pr(G|M_h)&= \frac{1}{k} \cdot f_1 \cdot f_2 d \theta _1\\&=\frac{1}{k} \cdot \frac{\varGamma (\alpha _1+\beta _1)}{\varGamma (\alpha _1) \cdot \varGamma (\beta _1)} \cdot \int _{\theta _1=0}^1 {\theta _1}^{c-1} \cdot {(1\!-\!\theta _1)}^{d-1} \cdot \frac{\varGamma (\alpha _2+\beta _2)}{\varGamma (\alpha _2) \cdot \varGamma (\beta _2)} \cdot \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (a+b)}\\&\quad \cdot \sum _{j=a}^{a+b-1} \frac{\varGamma (a+b)}{\varGamma (j+1) \cdot \varGamma (a+b-j)} \cdot \theta _1^j \cdot (1-\theta _1)^{a+b-1-j} d \theta _1\\&=\frac{1}{k} \cdot \frac{\varGamma (\alpha _1\!+\!\beta _1)}{\varGamma (\alpha _1) \cdot \varGamma (\beta _1)} \cdot \frac{\varGamma (\alpha _2\!+\!\beta _2)}{\varGamma (\alpha _2) \cdot \varGamma (\beta _2)} \cdot \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (a\!+\!b)} \cdot \int _{\theta _1=0}^1 {\theta _1}^{c-1} \cdot {(1\!-\!\theta _1)}^{d-1}\\&\quad \cdot \sum _{j=a}^{a+b-1} \frac{\varGamma (a+b)}{\varGamma (j+1)\cdot \varGamma (a+b-j)} \cdot \theta _1^j \cdot (1-\theta _1)^{a+b-1-j} d \theta _1\\&=\frac{1}{k} \cdot \frac{\varGamma (\alpha _1+\beta _1)}{\varGamma (\alpha _1) \cdot \varGamma (\beta _1)} \cdot \frac{\varGamma (\alpha _2+\beta _2)}{\varGamma (\alpha _2) \cdot \varGamma (\beta _2)} \cdot \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (a+b)}\\&\quad \cdot \sum _{j=a}^{a+b-1} \frac{\varGamma (a\!+\!b)}{\varGamma (j\!+\!1)\cdot \varGamma (a\!+\!b\!-\!j)} \cdot \int _{\theta _1=0}^1 {\theta _1}^{(c+j)-1} \cdot {(1\!-\!\theta _1)}^{(a+b+d-1-j)-1} d \theta _1\\ \end{aligned}$$

which by Eq. 5 is

$$\begin{aligned} \quad Pr(G|M_h)&=\frac{1}{k} \cdot \frac{\varGamma (\alpha _1+\beta _1)}{\varGamma (\alpha _1) \cdot \varGamma (\beta _1)} \cdot \frac{\varGamma (\alpha _2+\beta _2)}{\varGamma (\alpha _2) \cdot \varGamma (\beta _2)} \cdot \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (a+b)}\nonumber \\&\quad \cdot \sum _{j=a}^{a+b-1} \frac{\varGamma (a+b)}{\varGamma (j+1)\cdot \varGamma (a+b-j)} \cdot \frac{\varGamma (c+j) \cdot \varGamma (a+b+d-1-j)}{\varGamma (a+b+c+d-1)}\nonumber \\&=\frac{1}{k} \cdot \frac{\varGamma (\alpha _1+\beta _1)}{\varGamma (\alpha _1) \cdot \varGamma (\beta _1)} \cdot \frac{\varGamma (\alpha _2+\beta _2)}{\varGamma (\alpha _2) \cdot \varGamma (\beta _2)}\nonumber \\&\quad \cdot \sum _{j=a}^{a+b-1} \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (j+1)\cdot \varGamma (a+b-j)} \cdot \frac{\varGamma (c+j) \cdot \varGamma (a+b+d-1-j)}{\varGamma (a+b+c+d-1)} \quad \end{aligned}$$

(8)

where $a=N_{21}+\alpha _2,\,b=N_{22}+\beta _2,\,c=N_{11}+\alpha _1$ and $d=N_{12}+\beta _1$.

We can solve for $k$ (the normalization constant for the parameter prior) by solving Eq. 3 (without the $k$ term) with $N_{11}=N_{12}=N_{21}=N_{22}=0$. Doing so is equivalent to applying Eq. 8 (without the $k$ term) with $a=\alpha _2,\,b=\beta _2,\,c=\alpha _1$ and $d=\beta _1$. Note that $k=\frac{1}{2}$ if we use uniform priors on both parameters by setting $\alpha _1=\beta _1=\alpha _2=\beta _2=1$.

1.3 Four equivalent solutions of the marginal likelihood for model $M_h$

In the previous section, we showed the full derivation of the closed-form solution of the marginal likelihood for model $M_h$. It turned out that there are four equivalent solutions to Eq. 3. Let us use the notations introduced in the previous section: $a=N_{21}+\alpha _2,\,b=N_{22}+\beta _2,\,c=N_{11}+\alpha _1$ and $d=N_{12}+\beta _1$. Also, let us define $C$ as follows:

$$\begin{aligned} C=\frac{1}{k} \cdot \frac{\varGamma ({\alpha _1}+{\beta _1})}{\varGamma ({\alpha _1}) \cdot \varGamma ({\beta _1})}\cdot \frac{\varGamma ({\alpha _2}+{\beta _2})}{\varGamma ({\alpha _2}) \cdot \varGamma ({\beta _2})} \end{aligned}$$

(9)

The marginal likelihood of $M_h$ (Eq. 3) can be obtained by solving any of the following four equations:

$$\begin{aligned} C \cdot \sum _{j=a}^{a+b-1} \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (j+1) \cdot \varGamma (a+b-j)} \cdot \frac{\varGamma (c+j) \cdot \varGamma (a+b+d-j-1)}{\varGamma (a+b+c+d-1)} \end{aligned}$$

(10)

which is the solution we derived in the previous section.

$$\begin{aligned}&C \cdot \sum _{j=d}^{d+c-1} \frac{\varGamma (c) \cdot \varGamma (d)}{\varGamma (j+1) \cdot \varGamma (c+d-j)} \cdot \frac{\varGamma (b+j) \cdot \varGamma (c+d+a-j-1)}{\varGamma (a+b+c+d-1)}\end{aligned}$$

(11)

$$\begin{aligned}&C \cdot \left( \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (a+b)} \cdot \frac{\varGamma (c) \cdot \varGamma (d)}{\varGamma (c+d)} - \sum _{j=b}^{a+b-1} \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (j+1) \cdot \varGamma (a+b-j)} \right. \nonumber \\&\qquad \left. \cdot \frac{\varGamma (d+j) \cdot \varGamma (a+b+c-j-1)}{\varGamma (a+b+c+d-1)} \right) \end{aligned}$$

(12)

$$\begin{aligned}&C \cdot \left( \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (a+b)} \cdot \frac{\varGamma (c) \cdot \varGamma (d)}{\varGamma (c+d)} - \sum _{j=c}^{c+d-1} \frac{\varGamma (c) \cdot \varGamma (d)}{\varGamma (j+1) \cdot \varGamma (c+d-j)} \right. \nonumber \\&\qquad \left. \cdot \frac{\varGamma (a+j) \cdot \varGamma (c+d+b-j-1)}{\varGamma (a+b+c+d-1)} \right) \end{aligned}$$

(13)

1.4 Deriving a closed-form solution of the marginal likelihood for model $M_l$

Lastly, let us define the marginal likelihood for model $M_l$, which assumes that the probability of $Y=y$ for the instances in $G_P$ ($\theta _1$) is lower than the probability of $Y=y$ for the instances of $G$ that are outside $G_P$ ($\theta _2$). The marginal likelihood for $M_l$ is similar to Eq. 3, but integrates $\theta _2$ from 0 to 1 and constrains $\theta _1$ to be integrated from 0 to $\theta _2$ (forcing $\theta _1$ to be smaller than $\theta _2$) as follows:

$$\begin{aligned}&\frac{1}{k}\underbrace{\int _{\theta _2=0}^1 {\theta _2}^{N_{21}} \cdot {(1-\theta _2)}^{N_{22}} \cdot beta({\theta _2}; {\alpha _2}, {\beta _2})}_{f_1} \nonumber \\&\quad \times \underbrace{\int _{\theta _1=0}^{\theta _2} {\theta _1}^{N_{11}} \cdot {(1-\theta _1)}^{N_{11}} \cdot beta({\theta _1}; {\alpha _1}, {\beta _1}) d{\theta _1}}_{f_2} d{\theta _2} \end{aligned}$$

(14)

By solving the integral given by $f_2$, we get:

$$\begin{aligned} f_2&=\frac{\varGamma (\alpha _1+\beta _1)}{\varGamma (\alpha _1) \cdot \varGamma (\beta _1)} \int _{\theta _1=0}^{\theta _2} {\theta _1}^{c-1} \cdot {(1-\theta _1)}^{d-1} d \theta _2\\&=\frac{\varGamma (\alpha _1\!+\!\beta _1)}{\varGamma (\alpha _1) \cdot \varGamma (\beta _1)} \cdot \frac{\varGamma (c) \cdot \varGamma (d)}{\varGamma (c\!+\!d)} \cdot \sum _{j=c}^{c+d-1} \frac{\varGamma (c\!+\!d)}{\varGamma (j\!+\!1) \cdot \varGamma (c\!+\!d\!-\!j)} \cdot \theta _2^j \cdot (1-\theta _2)^{c+d-1-j} \end{aligned}$$

where, as before, $c=N_{11}+\alpha _1$ and $d=N_{12}+\beta _1$.

By solving $f_1$, we get:

$$\begin{aligned} f_1 =\frac{\varGamma (\alpha _2+\beta _2)}{\varGamma (\alpha _2) \cdot \varGamma (\beta _2)} \int _{\theta _2=0}^1 {\theta _2}^{a-1} \cdot {(1-\theta _2)}^{b-1} \end{aligned}$$

where, as before, $a=N_{21}+\alpha _2$ and $b=N_{22}+\beta _2$.

Now, we can solve Eq. 14:

$$\begin{aligned} \quad Pr(G|M_l) = C \cdot \sum _{j=c}^{c+d-1} \frac{\varGamma (c) \cdot \varGamma (d)}{\varGamma (j+1)\cdot \varGamma (c+d-j)} \cdot \frac{\varGamma (a+j) \cdot \varGamma (c+d+b-1-j)}{\varGamma (a+b+c+d-1)} \end{aligned}$$

(15)

where $C$ is the same constant we defined by Eq. 9 in the previous section.

Notice that Eq. 15 [the solution to $Pr(G|M_l)$] can be obtained from Eq. 13 [one of the four solutions to $Pr(G|M_h)$] as follows:

$$\begin{aligned} Pr(G|M_l)=C \cdot \frac{\varGamma (a) \cdot \varGamma (b)}{\varGamma (a+b)} \cdot \frac{\varGamma (c) \cdot \varGamma (d)}{\varGamma (c+d)} - Pr(G|M_h) \end{aligned}$$

(16)

It turns out that no matter which formula we use to solve $Pr(G|M_h)$, we can use Eq. 16 to obtain $Pr(G|M_l)$.

1.5 Computational complexity

Since we require that $N_{11},\,N_{12},\,N_{21},\,N_{22},\,\alpha _1,\,\beta _1,\,\alpha _2$ and $\beta _2$ be natural numbers, the gamma function simply becomes a factorial function: $\varGamma (x)=(x-1)!$. Since such numbers can become very large, it is convenient to use the logarithm of the gamma function and express Eqs. 2, 10, 11, 12, 13 and 16 in logarithmic form in order to preserve numerical precision. The logarithm of the integer gamma function can be pre-computed and efficiently stored in an array as follows:

$$\begin{aligned}&lnGamma[1]=0\\&\quad \hbox {For} \,\,i=2 \,\,\hbox {to}\,\, n\\&\qquad \qquad lnGamma[i]=lnGamma[i-1] + ln(i-1)\\ \end{aligned}$$

We then can use $lnGamma$ in solving the above equations. However, Eqs. 10, 11, 12 and 13 include a sum, which makes the use of the logarithmic form more involved. To deal with this issue, we can define function $lnAdd$, which takes two arguments $x$ and $y$ that are in logarithmic form and returns $ln(e^x + e^y)$. It does so in a way that preserves a good deal of numerical precision that could be lost if $ln(e^x + e^y)$ were calculated in a direct manner. This is done by using the following formula:

$$\begin{aligned} lnAdd(x,y) = x + ln(1+e^{(y-x)}) \end{aligned}$$

Now that we introduced functions $lnGamma$ and $lnAdd$, it is straightforward to evaluate Eqs. 2, 10, 11, 12, 13 and 16 in logarithmic form.

Let us now analyze the overall computational complexity for computing the Bayesian score for a specific rule (solving Eq. 1). Doing so requires computing $Pr(M_e|G),\,Pr(M_h|G)$ and $Pr(M_l|G)$. $Pr(M_e|G)$ can be computed in $O(1)$ using Eq. 2. $Pr(M_h|G)$ can be computed by applying Eqs. 10, 11, 12 or 13. The computational complexity of these equations are $O(N_{22}+\beta _2),\,O(N_{11}+\alpha _1),\,O(N_{21}+\alpha _2)$ and $O(N_{12}+\beta _1)$, respectively. Therefore, $Pr(M_h|G)$ can be computed in $O(min(N_{11}+\alpha _1,N_{12}+\beta _1,N_{21}+\alpha _2,N_{22}+\beta _2))$. $Pr(M_l|G)$ can be computed from $Pr(M_h|G)$ in $O(1)$ using Eq. 16. By assuming that $\alpha _1,\,\beta _1,\,\alpha _2,\,\beta _2$ bounded from above, the overall complexity for computing the Bayesian score is $\varvec{O(min(N_{11},N_{12},N_{21},N_{22}}))$.