Non-redundant Combination of Hand-Crafted and Deep Learning Radiomics: Application to the Early Detection of Pancreatic Cancer

Abstract

We address the problem of learning Deep Learning Radiomics (DLR) that are not redundant with Hand-Crafted Radiomics (HCR). To do so, we extract DLR features using a VAE while enforcing their independence with HCR features by minimizing their mutual information. The resulting DLR features can be combined with hand-crafted ones and leveraged by a classifier to predict early markers of cancer. We illustrate our method on four early markers of pancreatic cancer and validate it on a large independent test set. Our results highlight the value of combining non-redundant DLR and HCR features, as evidenced by an improvement in the Area Under the Curve compared to baseline methods that do not address redundancy or solely rely on HCR features.

Appendix

1.1 Estimating the Mutual Information

The Mutual Information (MI) is estimated following the density-ratio trick [8] which requires to train a discriminator $\mathcal {D}_{\lambda }$ predicting whether concatenated radiomics vectors [h, d] come from q(h, d) or q(h)q(d). Samples for training $\mathcal {D}_{\lambda }$ are obtained following the procedure shown in Fig. 4. In practice, $\mathcal {D}_{\lambda }$ is modeled as a 2-layer Multi Layer Perceptron with ReLu activation, which is trained by minimizing a binary cross-entropy (BCE) loss term. Once the discriminator is trained, the MI between HCR and DLR features can be approximated as follows:

$$\begin{aligned} \textrm{MI}(h, d) = {{\,\mathrm{\mathbb {E}}\,}}_{q(h,d)}\biggr [\log \frac{q(h,d)}{q(h)q(d)}\biggr ] \approx \sum _i \textrm{ReLU} \biggr (\biggr [\log \frac{\mathcal {D}_{\lambda }(h_i,d_i)}{1 - \mathcal {D}_{\lambda }(h_i,d_i)}\biggr ] \biggr ). \end{aligned}$$

(5)

Fig. 4.

Training $\mathcal {D}_{\lambda }$. Given three different input images $x^{*}_{i}$, $x^{*}_{j}$ and $x^{*}_{k}$, the corresponding HCR and DLR features are computed: $h_{j}$, $h_{j}$, $h_{k}$ and $d_{i}$, $d_{j}$, $d_{k}$. Samples from q(h, d) are obtained by concatenating features of a same image ($h_i$ and $d_i$ for instance), while samples from q(h)q(d) are obtained by concatenating $h_{k}$ and $d_{j}$ with $k \ne j$.

Table 4. Cancer marker prediction scores for different values of $\kappa$. For each experiment, we report the means and standard deviations of the AUC (in %) obtained by bootstrapping with 10000 repetitions. For each line, best result is in bold.

1.2 Influence of the Hyperparameter $\kappa$

The final loss function for training our model is:

$$\begin{aligned} \mathcal {L} = \mathcal {L}_\textrm{VAE} + \kappa KL[q(h, d) \mid q(h)q(d)] \end{aligned}$$

(6)

where $\kappa$ is a hyperparameter weighting the importance of the the mutual information in the total loss function. Table 4 reports prediction results obtained with different values of $\kappa$. According to these results, $\kappa$ was set to 1 in all our experiments.

1.3 HCR Features Extraction

32 HCR features were extracted using the pyradiomics library [24]:

• 14 shape features describing the size and shape of the pancreas

  • Mesh Volume

  • Voxel Volume

  • Surface Area

  • Surface Area to Volume ratio

  • Sphericity

  • Maximum 3D diameter

  • Maximum 2D diameter in the axial plane

  • Maximum 2D diameter in the coronal plane

  • Maximum 2D diameter in the sagittal plane

  • Major Axis Length

  • Minor Axis Length

  • Least Axis Length

  • Elongation

  • Flatness

• 18 first-order intensity features describing the intensities distribution within the organ

  • Energy

  • Total Energy

  • Entropy

  • Minimum

  • $10^{th}$ percentile

  • $90^{th}$ percentile

  • Maximum

  • Mean

  • Median

  • Interquartile Range

  • Range

  • Mean Absolute Deviation

  • Robust Mean Absolute Deviation

  • Root Mean Squared

  • Skewness

  • Kurtosis

  • Variance

  • Uniformity

More details about each feature can be found on the online documentation.

1.4 Model Architecture

As detailed in Fig. 5, the proposed variational autoencoder (VAE) followed a 3D encoder-decoder architecture. The network topology (number of convolutions per block, filter sizes) was chosen based on the nnU-Net self-configuring procedure [6], resulting in 1, 110, 240 trainable parameters. The VAE was trained on 1000 epochs with a batch size of size 32. Every 5 epochs, the VAE was frozen and the discriminator $\mathcal {D}_{\lambda }$ was trained for 150 epochs with a batch size equal to the total training dataset. The VAE and $\mathcal {D}_{\lambda }$ were optimized using two independent Adam optimizers with a learning rate of $10^{-3}$.

Fig. 5.

Architecture of the proposed VAE