Multivariate functional clustering and its application to typhoon data

Abstract

We propose a multivariate nonlinear mixed effects model for clustering multiple longitudinal data. Advantages of the nonlinear mixed effects model are that it is easy to handle unbalanced data which highly occur in the longitudinal study, and it can take into account associations among longitudinal variables at a given time point. The joint modeling for multivariate longitudinal data, however, requires a high computational cost because numerous parameters are included in the model. To overcome this issue, we perform a pairwise fitting procedure based on a pseudo-likelihood function. Unknown parameters included in each bivariate model are estimated by the maximum likelihood method along with the EM algorithm, and then the number of basis functions included in the model is selected by model selection criteria. After estimating the model, a non-hierarchical clustering algorithm by self-organizing maps is applied to the predicted coefficient vectors of individual specific random effect functions. We present the results of the application of the proposed method to the analysis of data of typhoons that occurred between 2000 and 2017 in Asia.

Appendices

Appendix 1: EM-algorithm for the bivariate nonlinear mixed effects model

We give the EM algorithm for estimating the bivariate nonlinear mixed effects model proposed in Eq. (4).

Step 0 Initialize $\hat{\varvec{\Sigma }}_{(\xi )}^{rs}=\varvec{I}_2$ and $\hat{\varvec{\Delta }}_{(\xi )}^{rs}=\varvec{I}_{2p_r}$ for the iteration number $\xi =0$.

Step 1 Set $\xi =\xi +1$, then update $\hat{\varvec{\alpha }}_{(\xi )}^{rs}$ and $\hat{\varvec{b}}_{i(\xi )}^{rs}$ using

$$\begin{aligned} \hat{\varvec{\alpha }}_{(\xi )}^{rs}= & {} \left( \sum _{i=1}^{n}(\varvec{X}_i^{rs})^{\prime }(\hat{\varvec{V}}_{i(\xi -1)}^{rs})^{-1}\varvec{X}_i^{rs}\right) ^{-1} \left( \sum _{i=1}^{n}(\varvec{X}_i^{rs})^{\prime }(\hat{\varvec{V}}_{i(\xi -1)}^{rs})^{-1}\varvec{y}_i^{rs} \right) ,\\ \hat{\varvec{b}}_{i(\xi )}^{rs}= & {} \hat{\varvec{\Delta }}_{(\xi -1)}^{rs}(\varvec{Z}_i^{rs})^{\prime }(\hat{\varvec{V}}_{i(\xi -1)}^{rs})^{-1}(\varvec{y}_i^{rs} - \varvec{X}_i^{rs}\hat{\varvec{\alpha }}_{(\xi )}^{rs}) \quad (i=1,2,\ldots , n), \end{aligned}$$

where $\hat{\varvec{V}}_{i(\xi -1)}^{rs} = \varvec{Z}_i^{rs}\hat{\varvec{\Delta }}_{(\xi -1)}^{rs}(\varvec{Z}_i^{rs})^{\prime } + \hat{\varvec{\Sigma }}_{(\xi -1)}^{rs}\varvec{I}_{J_i}$.

Step 2 Update $\hat{\sigma }_{r(\xi )}^2$ and $\hat{\varvec{\Delta }}_{(\xi )}^{rs}$ using the following conditional expectations,

$$\begin{aligned} \hat{\sigma }_{k(\xi )}^2= & {} \frac{1}{J}\sum _{i=1}^{n}\left\{ \hat{\varvec{\varepsilon }}_{ik(\xi )}^{\prime } \hat{\varvec{\varepsilon }}_{ik(\xi )} + \hat{\sigma }_{k(\xi -1)}^2\mathrm{tr}[I_{J_i}- \hat{\sigma }_{k(\xi -1)}^2(\varvec{I}_{J_i}|\varvec{0})\varvec{\Gamma }_{i(\xi -1)}^{rs}(\varvec{I}_{J_i}|\varvec{0})^{\prime }]\right\} \quad \ (k=r,s),\\ \hat{\varvec{\Delta }}_{(\xi )}^{rs}= & {} \frac{1}{n}\sum _{i=1}^{n}\left\{ \hat{\varvec{b}}_{i(\xi )}^{rs} \hat{\varvec{b}}_{i(\xi )}^{rs\prime } + \hat{\varvec{\Delta }}_{(\xi -1)}^{rs} - \hat{\varvec{\Delta }}_{(\xi -1)}^{rs}\varvec{Z}_i^{rs\prime }\varvec{\Gamma }_{i(\xi -1)}^{rs}\varvec{Z}_i^{rs}\hat{\varvec{\Delta }}_{(\xi -1)}^{rs} \right\} , \end{aligned}$$

where $J=\sum _{i=1}^nJ_i$, $\hat{\varvec{\varepsilon }}_{ik(\xi )} = \varvec{y}_{ik} - \varvec{X}_{ik}\hat{\varvec{\alpha }}_{k(\xi )} - \varvec{Z}_{ik}\hat{\varvec{b}}_{ik(\xi )}$ and $\varvec{\Gamma }_{i(\xi -1)}^{rs} = (\hat{\varvec{V}}_{i(\xi -1)}^{rs})^{-1} - (\hat{\varvec{V}}_{i(\xi -1)}^{rs})^{-1} \varvec{X}_{i}^{rs} (\sum _{i=1}^{n}(\varvec{X}_i^{rs})^{\prime }(\hat{\varvec{V}}_{i(\xi -1)}^{rs})^{-1}\varvec{X}_{i}^{rs})^{-1}(\varvec{X}_i^{rs})^{\prime }(\hat{\varvec{V}}_{i(\xi -1)}^{rs})^{-1}$.

Step 3 Repeat step 1 and 2 until convergence.

Appendix 2: algorithm of self-organizing maps

Consider classifying the vector $\tilde{\varvec{\gamma }}_i \in \mathbb {R}^d$$(i = 1, \ldots , n)$ into C clusters. Now, let $\{ \varvec{u}_c = (u_{c1}, u_{c2}, \ldots , u_{cd})^{\prime }; c = 1, \ldots , C \}$ be a reference vector for the cth cluster and $\{ \varvec{r}_c; c = 1, \ldots , C \}$ be an output layer for $\varvec{u}_c$, where $\varvec{r}_c$ are placed at equal spaced intervals in two-dimensional space $[1, g_1] \times [1, g_2]$ with $g_1,g_2 \in \mathbb {N}$ and $C=g_1g_2$. Let t be the current iteration and $T=10,000$ be the total number of iterations. Then the SOM algorithm for clustering $\tilde{\varvec{\gamma }}_i$ is given as follows:

• 1. Initialize the reference vector $\varvec{u}_k$$(k = 1, \ldots , K)$ from an uniform distribution U(0, 1).

• 2. Update $\varvec{u}_k$ by applying the following steps to $i = 1, \ldots , n$.

• (a) Define $\tilde{c}$ as follows.

  $$\begin{aligned} \tilde{c} = \arg \min _{k} \left\{ \left\| \tilde{\varvec{\gamma }}_i - \varvec{u}_k \right\| _{2} \right\} . \end{aligned}$$

• (b) Let $U(\varvec{r}_{\tilde{c}})$ be a neighborhood of $\varvec{r}_{\tilde{c}}$ on $\mathbb {R}^2$ with a radius $\sigma (t)$, where $\sigma (t)$ is a monotonically decreasing function for t. Let $N_{\tilde{c}}(t)=\{k; \varvec{r}_k\in U(\varvec{r}_{\tilde{c}})\}$, then for all $k \in N_{\tilde{c}}(t)$, update the reference vector as follows.

  $$\begin{aligned} \varvec{u}_k \leftarrow \varvec{u}_k + h_{\tilde{c}k}(t) \left( \tilde{\varvec{\gamma }}_i - \varvec{u}_k \right) , \end{aligned}$$

  where $h_{\tilde{c}k} (t)$ is a neighborhood function given by

  $$\begin{aligned} h_{\tilde{c}k}(t) = \alpha (t) \exp \left\{ -\frac{\Vert \varvec{r}_{\tilde{c}} - \varvec{r}_k\Vert ^2}{2 \left\{ \sigma (t) \right\} ^2 } \right\} . \end{aligned}$$

  $\alpha (t) \in (0, 1)$ is a learning rate coefficient and is set to be a monotone decreasing function of t. Here, $\alpha (t) = 0.9(1 - t / T)$.

• 3. Repeat step 2 for T times.

• 4. Let $\hat{\varvec{u}}_{k}$ be the reference vector obtained by T iterations. Assign $\tilde{\varvec{\gamma }}_i$$(i = 1, \ldots , N)$ to the $\tilde{c}$th cluster by

  $$\begin{aligned} \tilde{c} = \arg \min _{k} \left\{ \left\| \tilde{\varvec{\gamma }}_i - \hat{\varvec{u}}_{{k}} \right\| _{2} \right\} . \end{aligned}$$