Meningioma segmentation with GV-UNet: a hybrid model using a ghost module and vision transformer

Abstract

Meningiomas are the most common intracranial tumors in adults. The size and shape of a tumor mostly rely on manual measurement by a neurosurgeon. In recent years, deep learning has rapidly developed and has great potential for medical image segmentation. However, most segmentation models still cannot balance the number of parameters and accuracy. In this study, we proposed a novel segmentation network (named GV-UNet) based on a CNN and a transformer for T1-enhanced images of meningiomas to improve the accuracy and efficiency of tumor segmentation. GV-UNet uses an encoder–decoder as the main structure. In the downsampling process, features are extracted through a standard convolutional layer, and a ConvMixer Layer is used to optimize feature extraction with different sizes of meningiomas. Then, a lightweight transformer block is built to model long-range dependencies. In the final layer of the encoder, we propose an innovative Ghost-CA block, which extracts deep features via feature mapping rather than by elevating dimensionality to reduce the number of parameters. In the upsampling process, we add a SimAM that can incorporate a three-dimensional attention mechanism without increasing network parameters, effectively capturing the relationships between features and the spatial structure of the target. GV-UNet was trained and validated using numerous pathologically confirmed T1-enhanced images of meningiomas from Tianjin Huanhu Hospital. We also utilized meningioma images from the Kaggle dataset to test the robustness of the model.

Appendix A

1.1 A.1 Explanation of evaluation indicators

Fig. 5

Trade-off between parameters and segmentation accuracy of different models in Dataset A

Fig. 6

Trade-off between parameters and segmentation accuracy of different models in Dataset B

The DSC is an evaluation metric used to quantify the overlapping ratio between two binary sets, as defined in Eq. (15).

$$\begin{aligned} Dice=\frac{2TP}{FP+2TP+FN} \end{aligned}$$

(15)

where TP represents the true positive; FP represents the false positive; TN represents the true negative; and FN represents the false negative. The range of all the values is [0, 1], and the higher the Dice value is, the closer the tumor segmentation result is to the annotated result, and the better the segmentation effect.

IoU is another evaluation index in segmentation models to measure the overlap between the ground truth and the predicted segmentation, as defined in Eq. (16).

$$\begin{aligned} IoU=\frac{P\bigcap T}{P\bigcup T} \end{aligned}$$

(16)

where P and T represent the predicted result and manually annotated result, respectively.

The Hausdorff distance is a measure of the similarity between two sets of points, describing the maximum shortest distance between the segmentation result and the ground truth, as defined in Eq. (17).

$$\begin{aligned} {{d}_{H}}(A,B)=\max \{{{d}_{AB}},{{d}_{BA}}\} \end{aligned}$$

$$\begin{aligned} max \{\underset{a\in A}{\mathop {\max }}\,\underset{b\in B}{\mathop {\min d}}\,(a,b),\underset{b\in B}{\mathop {\max }} \,\underset{a\in A}{\mathop {\min }}\,d(b,a)\} \end{aligned}$$

(17)

where A and B represent the point set of the model prediction results and the point set of the ground truth results, respectively; d represents the Euclidean distance; and ${{d}_{AB}}$ and ${{d}_{BA}}$ represent the one-way Hausdorff distance from A to B and the one-way Hausdorff distance from B to A, respectively. The smaller the value is, the better the segmentation results.

ASD represents the average symmetrical surface distance, which indicates the average surface distance of all points in the segmented point set and is sensitive to the boundary, as defined in Eq. (18)-(20). The lower the value is, the better the segmentation results.

$$\begin{aligned} {{B}_{seg}}= & {} \left\{ \forall {{p}_{1}}\in {{A}_{seg}},closet\_distance({{p}_{1}},{{p}_{2}})|\exists {{p}_{2}}\in {{A}_{gt}} \right\} \end{aligned}$$

(18)

$$\begin{aligned} {{B}_{gt}}= & {} \left\{ \forall {{p}_{2}}\in {{A}_{gt}},closet\_distance({{p}_{1}},{{p}_{2}})|\exists {{p}_{1}}\in {{A}_{seg}} \right\} \end{aligned}$$

(19)

$$\begin{aligned} ASD= & {} mean(\{{{B}_{seg}},{{B}_{gt}}\}) \end{aligned}$$

(20)

where ${{A}_{seg}}$ represents the pixels of the boundary of the model’s predicted result and ${{A}_{gt}}$ represents the pixels of the boundary of the ground truth.

1.2 A.2 Parameters

The proposed GV-UNet presents a solution to the trade-off between computational power and segmentation accuracy in clinical scenarios involving meningiomas. In this section, we focus on the relationship between model parameters and segmentation accuracy.

As shown in Figs. 5 and 6, the segmentation performance of SegNet, SwinUNet and TransUNet in the two datasets show large gaps, especially TransUNet, which contains many parameters (421.47 M) but exhibits the worst results for meningioma segmentation, indicating that the complex ViT structure cannot achieve the desired performance in a small sample dataset. In addition, UCTransNet is able to maintain a relatively high segmentation accuracy in both datasets, but its parameter quantity is still too large (266.16 M). In contrast, the proposed GV-UNet achieved the highest segmentation accuracy in both datasets with the smallest parameters (50.20 M).

General medical image segmentation networks have potential applicability. However, their complex network structure may lead to the degradation of model performance due to the distinct boundary of meningioma in MRI. In addition, the smaller sample size can easily lead to overfitting of the general networks during training. To address the above problems, we redesigned the encoder structure to limit the downsampling size range to 512. The results demonstrate that the proposed GV-UNet can enhance feature extraction while maintaining the appropriate parameters.

1.3 A.3 Performance analysis of GV-UNet

As shown in Tables 1 and 2, the Dice score, IoU, HD95 and ASD of the proposed GV-UNet outperform the other SOTA models. As shown in Tables 3 and 4, the generalization ability of GV-UNet also outperforms their competitors for several reasons. First, GV-UNet expands the input image into 64-channel feature maps by a simple convolution layer, preserving numerous original features. Second, the large kernel convolution in the ConvMixer Layer is used to blend more distant spatial locations. Third, the MobileViT block is utilized to learn the global representation from different perspectives, which not only has the advantage of inductive bias inherent to CNNs but also enables adaptive weighting and global dependency of ViT. Last, the Ghost-CA layer generates many redundant maps to obtain the final feature information. Table 5 demonstrates the effectiveness of each module. Compared to the baseline network UNet, the Dice scores of GV-UNet improved by 4.4704% (Dataset A) and 2.1865% (Dataset B). As shown in Table 6, the proposed Equilibrium Loss outperforms all competing schemes because Equilibrium Loss not only has the advantage of Dice Loss global evaluation but also utilizes Varifocal Loss to mitigate the extreme imbalance between the foreground and the background.

1.4 A.4 Analysis on redundant maps

As shown in Fig. 2, we discovered that many redundant maps are generated during the feature extraction process. To verify whether these redundant maps contain feature information of the tumor, we performed ablation experiments. As shown in Table 5, the Dice scores of the model with only Ghost-CA added are 90.1068% and 89.0342% in the two datasets, with increases of 1.4409% and 0.1634%, respectively, over those of the baseline network, proving that the redundant maps can enhance the feature extraction capability of the model. Furthermore, Ghost-CA combines simple linear operations with coordinated attention to generate more redundant maps and thus orchestrate both remote dependencies and location capability to enhance the focus on the region of interest.

1.5 A.5 Limitations and future directions

The proposed GV-UNet can be developed into a powerful computer-assisted segmentation system for automating the segmentation process and easing the burden on clinical staff.

Presently, Ghost and ViT are utilized to balance parameters and accuracy. By adding the attention mechanism, the tumor region can be better localized. Although the proposed GV-UNet outperforms other models in terms of the number of parameters, the inherent structure of the transformer leads to the model’s floating points of operations (FLOPs) being unsatisfactory. In addition, our work only focuses on the segmentation performance of the model in T1-enhanced images without trying other modalities. To address these limitations, future work will focus on reducing the model complexity to achieve faster meningioma segmentation, as well as building novel segmentation models for T2, DWI and other modalities. To further address the clinical issues of meningioma, we will also work on the prediction of benign and malignant grades of meningioma by computer technology.