Batch and online variational learning of hierarchical Dirichlet process mixtures of multivariate Beta distributions in medical applications

Abstract

Thanks to the significant developments in healthcare industries, various types of medical data are generated. Analysing such valuable resources aid healthcare experts to understand the illnesses more precisely and provide better clinical services. Machine learning as one of the capable tools could assist healthcare experts in achieving expressive interpretation and making proper decisions. As annotation of medical data is a costly and sensitive task that can be performed just by healthcare professionals, label-free methods could be significantly promising. Interpretability and evidence-based decision are other concerns in medicine. These needs were our motivators to propose a novel clustering method based on hierarchical Dirichlet process mixtures of multivariate Beta distributions. To learn it, we applied batch and online variational methods for finding the proper number of clusters as well as estimating model parameters at the same time. The effectiveness of the proposed models is evaluated on three medical real applications, namely oropharyngeal carcinoma diagnosis, osteosarcoma analysis, and white blood cell counting.

Appendix

1.1 Hyperparameters of equations (30) to (35)

$\rho _{jit}$ as the hyperparameter of $Q(\mathbf {Z})$ is found by:

$$\begin{aligned} \rho _{jit}&= \frac{\exp ({\tilde{\rho }}_{jit})}{\sum _{f=1}^T exp({\tilde{\rho }}_{jit})} \end{aligned}$$

(46)

$$\begin{aligned} {\tilde{\rho }}_{jit}&= \sum _{k=1}^K \big <W_{jtk}\big >\left[ {\tilde{R}}_k\right. + \sum _{d=1}^{D}\big ({\overline{\alpha }}_{kd}-1\big )\ln Y_{jid} \nonumber \\&\quad -\sum _{d=1}^{D}\big ({\overline{\alpha }}_{kd}+1\big )\ln (1 - Y_{jid})\nonumber \\&\quad -\mid {{\overline{\alpha }}_j}\mid \ln \left[ 1+\sum _{d=1}^{D}\frac{ Y_{jid}}{(1-Y_{jid})}\right] +\langle \ln \pi _{jt}^{\prime } \rangle \left. +\sum _{s=1}^{t-1}\langle \ln (1-\pi _{js}^{\prime })\rangle \right] \end{aligned}$$

(47)

${\tilde{R}}_{k}$ following [15] is calculated by:

$$\begin{aligned} {\tilde{R}}_{j}&= \,\ln \frac{\Gamma \big (\sum _{d=1}^{D}{\overline{\alpha }}_{jd}\big )}{\prod _{d=1}^{D}\Gamma \big ({\overline{\alpha }}_{jd}\big )}+ \sum _{d=1}^{D}{\overline{\alpha }}_{jd}\Bigg [\varPsi \Bigg (\sum _{d=1}^{D}{\overline{\alpha }}_{jd}\Bigg )- \varPsi \big ({\overline{\alpha }}_{jd}\big ) \Bigg ]\nonumber \\&\quad \times \Big [\big<\ln \alpha _{jd}\big>-\ln {\overline{\alpha }}_{jd}\Big ]+ \nonumber \\&\frac{1}{2}\sum _{d=1}^{D}{\overline{\alpha }}_{jd}^2\Bigg [\varPsi '\Bigg (\sum _{d=1}^{D}{\overline{\alpha }}_{jd}\Bigg ) - \varPsi '\big ({\overline{\alpha }}_{jd}\big )\Bigg ]\nonumber \\&\quad \times \Big<\big (\ln \alpha _{jd} - \ln {\overline{\alpha }}_{jd}\big )^2\Big> \nonumber \\&+ \frac{1}{2}\sum _{a=1}^{D} \sum _{b=1, a\ne b}^{D}{\overline{\alpha }}_{ja}\,{\overline{\alpha }}_{jb}\Bigg [\varPsi '\Bigg (\sum _{d=1}^{D}{\overline{\alpha }}_{jd}\Bigg ) \nonumber \\& \times \Big (\big<\ln {\overline{\alpha }}_{ja}\big>-\ln {\overline{\alpha }}_{ja}\Big ) \times \Big (\big < \ln {\overline{\alpha }}_{jb}\big >-\ln {\overline{\alpha }}_{jb}\Big )\Bigg ] \end{aligned}$$

(48)

$\varPsi (.)$ and $\varPsi '(.)$ are the digamma and trigamma functions, respectively. Similarly, $\vartheta _{jtk}$ of the factor Q(W) is defined by:

$$\begin{aligned} \vartheta _{jtk}&= \frac{\exp ({\tilde{\vartheta }}_{jtk})}{\sum _{f=1}^K exp({\tilde{\vartheta }}_{jtf})} \end{aligned}$$

(49)

$$\begin{aligned} {\tilde{\vartheta }}_{jtk}&= \sum _{i=1}^N \big <Z_{jit}\big >\Bigg [{\tilde{R}}_k + \sum _{d=1}^{D}\big ({\overline{\alpha }}_{kd}-1\big )\ln Y_{jid} \nonumber \\&-\sum _{d=1}^{D}\big ({\overline{\alpha }}_{kd}+1\big )\ln (1 - Y_{jid})\nonumber \\&\quad -\mid {{\overline{\alpha }}_j}\mid \ln \Big [ 1 +\sum _{d=1}^{D}\frac{ Y_{jid}}{(1-Y_{jid})}\Big ] +\langle \ln \psi _{k}^{\prime } \rangle +\sum _{s=1}^{k-1}\langle \ln (1-\psi _{s}^{\prime })\rangle \Bigg ] \end{aligned}$$

(50)

$a_{jt}$ and $b_{jt}$ as the hyperparameters of the factor $Q(\pi ^\prime )$ are:

$$\begin{aligned} a^*_{jt}=1+\sum _{i=1}^{N}\left\langle Z_{jit}\right\rangle , \quad b^*_{jt}=\lambda _{jt}+\sum _{i=1}^{N} \sum _{s=t+1}^{T}\langle Z_{jis}\rangle \end{aligned}$$

(51)

$c_k$ and $d_k$ as the hyperparameters of the factor $Q(\psi ^\prime )$ are calculated by:

$$\begin{aligned} c^*_{k}=1+\sum _{j=1}^{M} \sum _{t=1}^{T}\left\langle W_{jtk}\right\rangle , \quad d^*_{k}=\gamma _{k}+\sum _{j=1}^{M} \sum _{t=1}^{T} \sum _{s=k+1}^{K}\langle W_{jts}\rangle \end{aligned}$$

(52)

The hyperparameters $u_{kd}^*$ and $\nu _{kd}$ of the factor $Q(\alpha )$ are updated as follows:

$$\begin{aligned} u_{kd}^*&= u_{kd}+\sum _{j=1}^M \sum _{t=1}^T \langle W_{jtk}\rangle \sum _{i=1}^N \big<Z_{jit}\big>{\overline{\alpha }}_{kd} \times \nonumber \\&\Bigg [\varPsi \Bigg (\sum _{s=1}^{D}{\overline{\alpha }}_{ks}\Bigg )-\varPsi \big ({\overline{\alpha }}_{kd}\big )+\sum _{s \ne l}^{D} {\overline{\alpha }}_{ks} \varPsi '\Bigg (\sum _{s=1}^{D}{\overline{\alpha }}_{ks}\Bigg )\nonumber \\&\quad \times \Big (\big <\ln \alpha _{ks}\big >-\ln {\overline{\alpha }}_{ks}\Big )\Bigg ] \end{aligned}$$

(53)

$$\begin{aligned} \nu _{kd}^*&= \nu _{kd}- \sum _{j=1}^M \sum _{t=1}^T \langle W_{jtk}\rangle \sum _{i=1}^N\big <Z_{jit}\big >\nonumber \\&\quad \times \left[ \ln Y_{jid} - \ln (1-Y_{jid}) - \ln \left[ 1+\sum _{d=1}^{D}\frac{Y_{jid}}{(1-Y_{jid})}\right] \right] \end{aligned}$$

(54)

The expected values of above mentioned equations are given by:

$$\begin{aligned} {\bar{\alpha }}_{kd}=\left\langle \alpha _{kd}\right\rangle&=\frac{u^*_{kd}}{v^*_{kd}} \end{aligned}$$

(55)

$$\begin{aligned} \big<Z_{jit}\big>&= \rho _{jit} , \quad \big <W_{jtk}\big >= \vartheta _{jtk} \end{aligned}$$

(56)

$$\begin{aligned} &\big<\ln \pi _{jt}^{\prime }\big>=\varPsi (a_{jt})-\varPsi (a_{jt}+b_{jt}) , \nonumber \\& \big <\ln (1-\pi _{jt}^{\prime })\big>=\varPsi (b_{jt})-\varPsi (a_{jt}+b_{jt}) \end{aligned}$$

(57)

$$\begin{aligned} &\big<\ln \psi _{k}^{\prime } \big>=\varPsi (c_{k})-\varPsi (c_{k}+d_{k}) , \nonumber \\& \big <\ln (1-\psi _{k}^{\prime } )\big >=\varPsi (d_{k})-\varPsi (c_{k}+d_{k}) \end{aligned}$$

(58)

$$\begin{aligned} &\langle \ln \alpha _{kd}\rangle=\varPsi (u_{kd}^{*})-\ln v_{kd}^{*}, \nonumber \\& \Big <\big (\ln \alpha _{kd} - \ln {\overline{\alpha }}_{kd}\big )^2\Big >=\big [\varPsi (u^*_{kd})-\ln u^*_{kd}\big ]^2 + \varPsi ^{\prime }(u^*_{kd}) \end{aligned}$$

(59)

1.2 Hyperparameters of equation (37)

$$\begin{aligned} \rho _{jtr}&= \frac{\exp ({\tilde{\rho }}_{jtr})}{\sum _{f=1}^T \exp ({\tilde{\rho }}_{jtr})} \end{aligned}$$

(60)

$$\begin{aligned} &{\tilde{\rho }}_{jtr}= \sum _{k=1}^K \big <W^{(r-1)}_{jtk}\big >\left[ \tilde{R_k}^{(r-1)}\right. + \sum _{d=1}^{D}\big ({{\overline{\alpha }}_{kd}}^{(r-1)}-1\big )\ln X_{jrl} -\sum _{d=1}^{D}\big ({{\overline{\alpha }}_{kd}}^{(r-1)}+1\big )\ln (1 - X_{jrl}) - \\ \nonumber &\mid {{\overline{\alpha }}_j}\mid \ln \left[ 1+\sum _{d=1}^{D}\frac{ X_{jrl}}{(1-X_{jrl})}\right]+\langle \ln \pi _{jt}^{\prime (r-1)} \rangle\left. +\sum _{s=1}^{t-1}\left\langle \ln (1-\pi _{js}^{\prime (r-1)})\right\rangle \right] \end{aligned}$$

(61)

${\tilde{R}}_{j}$ is calculated by equation (48).

1.3 Equation (40)

$$\begin{aligned} {\tilde{\vartheta }}_{jtk}&= N\rho _{jtr} \left[ \tilde{R_k}^{(r-1)} + \sum _{d=1}^{D}\big ({{\overline{\alpha }}_{kd}}^{(r-1)}-1\big )\ln X_{jrl}\right. -\sum _{d=1}^{D}\big ({{\overline{\alpha }}_{kd}}^{(r-1)}+1\big )\ln (1 - X_{jrl})\nonumber \\&\quad -\mid {{\overline{\alpha }}_j}\mid \ln \left[ 1+\sum _{d=1}^{D}\frac{ X_{jrl}}{(1-X_{jrl})}\right] +\langle \ln \psi _{k}^{\prime (r-1)} \rangle \left. +\sum _{s=1}^{k-1}\langle \ln (1-\psi _{k}^{\prime (r-1)})\rangle \right] \end{aligned}$$

(62)

1.4 Hyperparameters of equations (41) to (43)

$$\begin{aligned} a^{(r)}_{jt}&= a^{(r-1)}_{jt} + \xi _r \varDelta a_{jt}^{(r)}, \quad b^{(r)}_{jt} = b^{(r-1)}_{jt} + \xi _r \varDelta b_{jt}^{(r)} \end{aligned}$$

(63)

$$\begin{aligned} c^{(r)}_{k}&= c^{(r-1)}_{k} + \xi _r \varDelta c_{k}^{(r)} , \quad d^{(r)}_{k} = d^{(r-1)}_{k} + \xi _r \varDelta d_{k}^{(r)} \end{aligned}$$

(64)

$$\begin{aligned} u^{*(r)}_{kd}&= u^{*(r-1)}_{kd} + \xi _r \varDelta u_{kd}^{*(r)} , \quad v^{*(r)}_{kd} = v^{*(r-1)}_{kd} + \xi _r \varDelta v_{kd}^{*(r)} \end{aligned}$$

(65)

$$\begin{aligned} \varDelta a_{jt}^{(r)}= 1 + N \rho _{jtr} - a_{jt}^{(r-1)} , \quad \varDelta b_{jt}^{(r)}= \lambda _{jt} + N \sum _{s=t+1}^{T} \rho _{jsr} - b_{jt}^{(r-1)} \end{aligned}$$

(66)

$$\begin{aligned} &\varDelta c_{k}^{(r)}= 1 + \sum _{j=1}^{K} \sum _{t=1}^{T} \vartheta _{jtk}^{(r)} - c_{k}^{(r-1)} , &\varDelta d_{k}^{(r)} = \gamma _{k} + \sum _{j=1}^{M} \sum _{t=1}^{T} \sum _{m=k+1}^{K} \vartheta _{jtk}^{(r)} - d_{k}^{(r-1)} \end{aligned}$$

(67)

$$\begin{aligned} \varDelta u_{kd}^{*(t)}&=u_{kd}+ N \sum _{j=1}^{M} \sum _{t=1}^{T} \vartheta _{jtk}^{(r)} \rho _{jtr} {\bar{\alpha }}_{kd}^{(r-1)} \left[ \varPsi \left( \sum _{s=1}^{D} {\bar{\alpha }}_{ks}^{(r-1)}\right) \right. \nonumber \\&\quad -\varPsi ({\bar{\alpha }}_{kd}^{(r-1)}) +\sum _{s \ne l}^{D} {\bar{\alpha }}_{ks}^{(r-1)} {\varPsi }^{\prime }\left( \sum _{s=1}^{D} {\bar{\alpha }}_{ks}^{(r-1)}\right) (\langle \ln \alpha _{k s}^{(r-1)}\rangle \nonumber \\&\left. \quad -\ln {\bar{\alpha }}_{k s}^{(r-1)}- \ln {\bar{\alpha }}_{k s}^{(r-1)}\right] - u_{kd}^{*(t-1)} \end{aligned}$$

(68)

$$\begin{aligned} \varDelta v_{kd}^{*(t)}&= v_{kd} - N \sum _{j=1}^{M} \sum _{t=1}^{T} \vartheta _{jtk}^{(r)} \rho _{jtr} \left[ \ln Y_{jid} - \ln (1-Y_{jid}) - \ln \left[ 1+\sum _{d=1}^{D}\frac{Y_{jid}}{(1-Y_{jid})}\right]\right] - v_{kd}^{*(t-1)} \end{aligned}$$

(69)