Dynamic programming approach for segmentation of multivariate time series

Abstract

In this paper, dynamic programming (DP) algorithm is applied to automatically segment multivariate time series. The definition and recursive formulation of segment errors of univariate time series are extended to multivariate time series, so that DP algorithm is computationally viable for multivariate time series. The order of autoregression and segmentation are simultaneously determined by Schwarz’s Bayesian information criterion. The segmentation procedure is evaluated with artificially synthesized and hydrometeorological multivariate time series. Synthetic multivariate time series are generated by threshold autoregressive model, and in real-world multivariate time series experiment we propose that besides the regression by constant, autoregression should be taken into account. The experimental studies show that the proposed algorithm performs well.

Appendices

Appendix

A: Dynamic programming segmentation

B: Recursive formulas for segment errors of multivariate time series

In this appendix, we will prove the correctness of the recursive formulas Eq. (15) for the computation of segment error $d_{s,t}$ for multivariate time series. As given by Eq. (13),

$$\begin{aligned} d_{s,t}&=|vec({\mathbf {Z}}_{s,t})-vec(\hat{\mathbf {Z}}_{s,t})|^{2}\\&=[vec({\mathbf {Z}}_{s,t})-vec(\hat{\mathbf {Z}}_{s,t})]^{T}[vec({\mathbf {Z}}_{s,t})-vec(\hat{\mathbf {Z}}_{s,t})]\nonumber \\&=vec({\mathbf {Z}}_{s,t})^{T}vec({\mathbf {Z}}_{s,t})\!-\!vec({\mathbf {Z}}_{s,t})^{T}vec(\hat{\mathbf {Z}}_{s,t})\nonumber \\&quad \!-\!vec(\hat{\mathbf {Z}}_{s,t})^{T}vec({\mathbf {Z}}_{s,t})\!+\!vec(\hat{\mathbf {Z}}_{s,t})^{T}vec(\hat{\mathbf {Z}}_{s,t})\nonumber \end{aligned}$$

(27)

Based on $vec(B)^{T}vec(A)=tr(B^{T}A)$, we have

$$\begin{aligned} d_{s,t}=tr({\mathbf {Z}}_{s,t}^{T}{\mathbf {Z}}_{s,t})-tr({\mathbf {Z}}_{s,t}^{T}\hat{\mathbf {Z}}_{s,t})-tr(\hat{\mathbf {Z}}_{s,t}^{T}{\mathbf {Z}}_{s,t})+tr(\hat{\mathbf {Z}}_{s,t}^{T}\hat{\mathbf {Z}}_{s,t}) \end{aligned}$$

(28)

where $tr$ is the trace of a matrix, while based on $\hat{\mathbf {Z}}_{s,t}=\hat{\mathbf {A}}_{s,t}{\mathbf {Y}}_{s,t}$, we have

$$\begin{aligned} d_{s,t+1}&= |vec({\mathbf {Z}}_{s,t+1})-vec(\hat{\mathbf {Z}}_{s,t+1})|^{2}\nonumber \\&= |vec({\mathbf {Z}}_{s,t+1})-vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t+1})|^{2}\nonumber \\&= |vec({\mathbf {Z}}_{s,t})-vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})|^{2}+|{\mathbf {Z}}_{t+1}-\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{t+1}|^{2} \end{aligned}$$

(29)

we can get

$$\begin{aligned}&\quad |vec({\mathbf {Z}}_{s,t})-vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})|^{2}\nonumber \\&=vec({\mathbf {Z}}_{s,t})^{T}vec({\mathbf {Z}}_{s,t})-vec({\mathbf {Z}}_{s,t})^{T}vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})\nonumber \\&\quad -vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})^{T}vec({\mathbf {Z}}_{s,t})+vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})^{T}vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t}) \end{aligned}$$

(30)

Now we will compute the last three terms on the right-hand side in Eq. (30) respectively. With $\mathbf {\delta A}=\hat{\mathbf {A}}_{s,t+1}-\hat{\mathbf {A}}_{s,t}$, we have

$$\begin{aligned} vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})&= vec((\hat{\mathbf {A}}_{s,t}+\mathbf {\delta A}){\mathbf {Y}}_{s,t})\nonumber \\&= vec(\hat{\mathbf {A}}_{s,t}{\mathbf {Y}}_{s,t})+vec(\mathbf {\delta A}{\mathbf {Y}}_{s,t}) \end{aligned}$$

(31)

then we obtain

$$\begin{aligned} vec({\mathbf {Z}}_{s,t})^{T}vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})&= vec({\mathbf {Z}}_{s,t})^{T}vec(\hat{\mathbf {A}}_{s,t}{\mathbf {Y}}_{s,t})+vec({\mathbf {Z}}_{s,t})^{T} vec({\mathbf {\delta A}} {\mathbf {Y}}_{s,t})\nonumber \\&= tr({\mathbf {Z}}_{s,t}^{T}\hat{\mathbf {Z}}_{s,t})+tr({\mathbf {Z}}_{s,t}^{T}{\mathbf {\delta A}} {\mathbf {Y}}_{s,t}) \end{aligned}$$

(32)

Also we have

$$\begin{aligned} vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})^{T}vec({\mathbf {Z}}_{s,t})&= vec(\hat{\mathbf {A}}_{s,t}{\mathbf {Y}}_{s,t})^{T}vec({\mathbf {Z}}_{s,t})+vec({\mathbf {\delta A}} {\mathbf {Y}}_{s,t})^{T}vec({\mathbf {Z}}_{s,t})\nonumber \\&= tr(\hat{\mathbf {Z}}_{s,t}^{T}{\mathbf {Z}}_{s,t})+tr(({\mathbf {\delta A}} {\mathbf {Y}}_{s,t})^{T}{\mathbf {Z}}_{s,t}) \end{aligned}$$

(33)

As $vec(AB)=(I\otimes A)vec(B)$ (Lütkepohl 2005), where $\otimes$ is Kronecker product. What is more, based on the properties of Knonecker product, $(A\otimes B)^{T}=A^{T}\otimes B^{T}$ and $(A\otimes B)(C\otimes D)=AC\otimes BD$ (Lütkepohl 2005), we have

$$\begin{aligned}& vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})^{T}vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})\nonumber \\&\quad=[(I\otimes \hat{\mathbf {A}}_{s,t+1})vec({\mathbf {Y}}_{s,t})]^{T}[(I\otimes \hat{\mathbf {A}}_{s,t+1})vec({\mathbf {Y}}_{s,t})]\nonumber \\&\quad=vec({\mathbf {Y}}_{s,t})^{T}(I\otimes \hat{\mathbf {A}}_{s,t+1})^{T}(I\otimes \hat{\mathbf {A}}_{s,t+1})vec({\mathbf {Y}}_{s,t})\nonumber \\&\quad=vec({\mathbf {Y}}_{s,t})^{T}(I^{T}\otimes \hat{\mathbf {A}}_{s,t+1}^{T})(I\otimes \hat{\mathbf {A}}_{s,t+1})vec({\mathbf {Y}}_{s,t})\nonumber \\&\quad=vec({\mathbf {Y}}_{s,t})^{T}(I^{T}I\otimes \hat{\mathbf {A}}_{s,t+1}^{T}\hat{\mathbf {A}}_{s,t+1})vec({\mathbf {Y}}_{s,t})\nonumber \\&\quad=vec({\mathbf {Y}}_{s,t})^{T}(I\otimes ((\hat{\mathbf {A}}_{s,t}^{T}\!+\!{\mathbf {\delta A}}^{T})(\hat{\mathbf {A}}_{s,t}+{\mathbf {\delta A }})))vec({\mathbf {Y}}_{s,t})\nonumber \\&\quad=vec({\mathbf {Y}}_{s,t})^{T}(I\otimes \hat{\mathbf {A}}_{s,t}^{T}\hat{\mathbf {A}}_{s,t}\!+\!I\otimes \hat{\mathbf {A}}_{s,t}^{T} {\mathbf {\delta A}}\!+\!I\otimes {\mathbf {\delta A}}^{T}\hat{\mathbf {A}}_{s,t}\!+\!I\otimes {\mathbf { \delta A}}^{T} {\mathbf {\delta A}})vec({\mathbf {Y}}_{s,t})\nonumber \\&\quad=vec({\mathbf {Y}}_{s,t})^{T}(vec(\hat{\mathbf {A}}_{s,t}^{T}\hat{\mathbf {A}}_{s,t}{\mathbf {Y}}_{s,t})+vec(\hat{\mathbf {A}}_{s,t}^{T} {\mathbf {\delta A}}{\mathbf {Y}}_{s,t})\nonumber \\&\qquad +vec({\mathbf {\delta A}}^{T}\hat{\mathbf {A}}_{s,t}{\mathbf {Y}}_{s,t})+vec ({\mathbf {\delta A}}^{T}{\mathbf {\delta A}}{\mathbf {Y}}_{s,t})\nonumber \\&\quad=tr({\mathbf {Y}}_{s,t}^{T}\hat{\mathbf {A}}_{s,t}^{T}\hat{\mathbf {A}}_{s,t}{\mathbf {Y}}_{s,t})+tr({\mathbf {Y}}_{s,t}^{T}\hat{\mathbf {A}}_{s,t}^{T} {\mathbf {\delta A}}{\mathbf {Y}}_{s,t})\nonumber \\&\quad +tr({\mathbf {Y}}_{s,t}^{T}{\mathbf {\delta A}}^{T}\hat{\mathbf {A}}_{s,t}{\mathbf {Y}}_{s,t})+ tr({\mathbf {Y}}_{s,t}^{T}\mathbf {\delta A}^{T}{\mathbf {\delta A}}{\mathbf {Y}}_{s,t})\nonumber \\&\quad=tr(\hat{\mathbf {Z}}_{s,t}^{T}\hat{\mathbf {Z}}_{s,t})+tr(\hat{\mathbf {Z}}_{s,t}^{T}{\mathbf {\delta A}}{\mathbf {Y}}_{s,t})+tr({\mathbf {Y}}_{s,t}^{T}{\mathbf {\delta A}}^{T}\hat{\mathbf {Z}}_{s,t})+ tr({\mathbf {Y}}_{s,t}^{T}{\mathbf {\delta A}} ^{T} {\mathbf { \delta A}}{\mathbf {Y}}_{s,t}) \end{aligned}$$

(34)

Replacing Eqs. (32, 33, 34) in Eq. (30), we have

$$\begin{aligned}&\quad |vec({\mathbf {Z}}_{s,t})-vec(\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{s,t})|^{2}\nonumber \\&=tr({\mathbf {Z}}_{s,t}^{T}{\mathbf {Z}}_{s,t})\!-\!tr({\mathbf {Z}}_{s,t}^{T}\hat{\mathbf {Z}}_{s,t})\!-\!tr(\hat{\mathbf {Z}}_{s,t}^{T}{\mathbf {Z}}_{s,t})\!+\! tr(\hat{\mathbf {Z}}_{s,t}^{T}\hat{\mathbf {Z}}_{s,t})\!+\!tr({\mathbf {Y}}_{s,t}^{T}{\mathbf {\delta A}} ^{T}{\mathbf {\delta A}}{\mathbf {Y}}_{s,t})\nonumber \\&=d_{s,t}+tr({\mathbf {Y}}_{s,t}^{T}{\mathbf {\delta A}} ^{T}{\mathbf {\delta A}}{\mathbf {Y}}_{s,t}) \end{aligned}$$

(35)

Further we get that

$$\begin{aligned} d_{s,t+1}=d_{s,t}+tr({\mathbf {Y}}_{s,t}^{T}{\mathbf {\delta A}}^{T}{\mathbf { \delta A}}{\mathbf {Y}}_{s,t})+|{\mathbf {Z}}_{t+1}-\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{t+1}|^{2} \end{aligned}$$

(36)

As $tr(CD)=tr(DC)$, finally we obtain

$$\begin{aligned} d_{s,t+1}=d_{s,t}+tr({\mathbf {\delta A}}{\mathbf {Y}}_{s,t}{\mathbf {Y}}_{s,t}^{T}{\mathbf {\delta A}}^{T} )+|{\mathbf {Z}}_{t+1}-\hat{\mathbf {A}}_{s,t+1}{\mathbf {Y}}_{t+1}|^{2} \end{aligned}$$

(37)