Background

Cervical cancer is one of the most common cancers among women, with an estimated 604,127 new cases and 341,831 deaths from cervical cancer worldwide in 2020 [1]. Treatment options for cervical cancer patients with cervical cancer vary depending on the International Federation of Gynecology and Obstetrics (FIGO) stages and the status of lymph nodes [2]. Lymph node metastasis is one of the most important prognostic factors in patients with cervical cancer [3, 4]. The 5-year survival rate for patients with early-stage cervical cancer without lymph node metastases is reported to be 85-90%, but only 50-55% for those with lymph node metastasis [5]. Furthermore, many patients with early-stage cervical cancer may undergo unnecessary lymph node dissection because of undiagnosed or inaccurate assessment of lymph node status [6, 7]. Therefore, accurate assessment of the preoperative lymph node status has an important impact on the treatment and prognosis of patients with cervical cancer.

The criteria for lymph node diagnosis are histopathologic examination after surgical lymphadenectomy or lymph node biopsy [8]. However, these invasive detection methods have some limitations, such as the potential for infection, nerve or vascular injury, and lower extremity lymphedema from the surgical procedure, while biopsy results are influenced by the presence of abnormal lymphatic drainage, the quality of the preoperative lymphadenectomy, and the experience of the surgeons [9,10,11]. Recently, non-invasive radiomic analysis has been widely used for prognostic assessment [12, 13] and prediction of lymph node metastasis in patients with cervical cancer [14,15,16].

Previous studies have reported the prediction of lymph node metastasis in patients with cervical cancer based on the radiomic features of different images [14,15,16,17,18,19,20,21,22]. Liu et al. reported a radiomic features model based on computed tomography (CT) images to predict lymph node metastasis, and the area under the receiver operating characteristic curve (AUC) of their model was 0.859 [20]. Song et al. predicted lymph node metastasis using a radiomic features model of magnetic resonance imaging (MRI) images with an AUC of 0.75 [15]. The radiomic features models for predicting lymph node metastasis in patients with cervical cancer from previous studies are summarized in Table 1. However, most previous studies have explored the predictive effect of only one modeling approach. In addition, the predictive effect of combining radiomic feature with clinical feature models is not well understood. The modeling method and the features used for modeling are important factors that affect the predictive performance of the model.

Table 1 Overview of studies using radiological features to predict lymph node metastasis in patients with cervical cancer
Full size table

Herein, we aimed to establish a model that combines the radiomic features of MRI images with the clinical features of patients to predict lymph node metastasis in patients with cervical cancer. Seven machine learning methods were used to construct models to identify the optimal model.

Methods

Study population

Data on cervical cancer patients were obtained from The First Affiliated Hospital of Bengbu Medical College between 2018 and 2021. The identification of the patient’s lymph node metastases was based on histopathological examination. Inclusion criteria were as follows: (1) patients aged ≥ 18 years; (2) patients with primary cervical cancer confirmed by histopathological examination; (3) patients who underwent radical hysterectomy and pelvic lymph node dissection; (4) patients who underwent MRI examination within 2 weeks before hysterectomy; and (5) available clinical information. Exclusion criteria were as follows: (1) patients with combined other malignancies; (2) patients with palliative tumor resection; (3) pregnant or lactating women; (4) patients with preoperative chemotherapy or radiation; (5) patients who underwent biopsy puncture or conization before MRI examination; and (6) patients whose MRI imaging did not meet the requirements for post-processing. This cross-sectional study was approved by the Institutional Review Board of The First Affiliated Hospital of Bengbu Medical College (approval number: 2022KY039), and informed consent was obtained from the patients. All methods were carried out in accordance with relevant guidelines and regulations (declaration of Helsinki).

MRI image acquisition

All patients underwent pelvic MRI in the supine position using the same 3.0-T MRI scanner (Siemens AG, Munich, Germany) with 8-channel phased-array coil. Before an examination, patients fasted for at least 4 h and filled the bladder moderately. The scanning sequences included axial and sagittal T1-weighted imaging (T1WI) and axial T2-weighted imaging (T2WI). Pelvic T1WI and T2WI images were obtained after injection of 0.2ml/kg gadopentetate dimeglumine for 80–120 s. All images were Digital Imaging and Communications in Medicine (DICOM) format data. The ITK-SNAP (www.itksnap.org) was used to perform manual 3D segmentation of MRI images. Radiologists experienced (more than 10 years) in pelvic MRI diagnosis performed manual segmentation of the lymph node and tumor region of interest (ROI) on each cross-section to validate the segmentation results for each image. The radiologists were blinded to the patient’s lymph node status.

Radiomics feature extraction

All images are normalized before feature extraction, and the resolution of all images is unified to 1 × 1mm2 by interpolation. The interpolation process uses sitkNearestNeighbor as a resampling interpolator to resample the mask to preserve the label values. Radiomic features were extracted from the processed MRI images by Python 3.8 software (PyRadiomics package) [23]. Specifically, the PyRadiomics package’s “RadiomicsFeatureExtractor()” function was used to preprocess the image and create a feature extraction generator, set an optional custom image type using the “enableImageTypes()” function within the generator, and then use the “execute()” function within the generator to calculate the image label of the original image combined with the ROI to obtain the corresponding type of radiomic features. A total of 122 radiomic features were extracted, including First Order Statistics (18 features), Shape-based (2D) (14 features), Gray Level Cooccurrence Matrix (24 features), Gray Level Run Length Matrix (16 features), Gray Level Size Zone Matrix (16 features), Neighbouring Gray Tone Difference Matrix (5 features), Gray Level Dependence Matrix (14 features), and basic image features (5 features).

Feature selection

The datasets were randomly assigned to the training set and testing set in a ratio of 7:3. All clinical and radiomic features were screened by the least absolute shrinkage and selection operator (LASSO) regression to select the optimal predictive features. Five-fold cross-validation was applied to tune the parameters of the elastic net to select the key features from the high-dimensional feature space and to avoid over-fitting. Finally, 3 clinical features (eosinophil count, red blood cell volume distribution width, squamous cell carcinoma antigen), 2 radiomic features of T1 images (original_firstorder_Range, original_ngtdm_Complexity), and 3 radiomic features of T2 images (diagnostics_Mask-original_VolumeNum, original_glcm_InverseVariance, original_glszm_SmallAreaHighGrayLevelEmphasis) were incorporated into the model.

Construction, validation, and performance of machine learning model

Seven machine learning methods, including eXtreme Gradient Boosting (XGBoost), Logistic Regression, Multinomial Naive Bayes (MNB), Support Vector Machine (SVM), Decision Tree, Random Forest, and Gradient Boosting Decision Tree (GBDT) are used to build prediction models. All models used 8 clinical and radiomic features that were screened out. Receiver operating characteristics (ROC) curve and area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated to assess the performance of the models. The ablation analysis was performed to validate the resilience of the model [24, 25]. The model with the best combined performance was selected, and the performance of models with clinical features only, radiomic features only, and combined features was further compared. The flowchart of this study is shown in Fig. 1.

Fig. 1
figure 1

Flowchart of the study. The composition of the prediction system including image input, feature extraction, feature selection, model building, and model evaluation

Full size image

Statistical analysis

For continuous clinical variables, there were expressed as mean and standard deviation (SD) or median and interquartile range [M (Q1, Q3)], and compared using the Student’s t-test or rank-sum test. Categorical clinical variables were expressed as numbers and percentages [n (%)] and compared using the Chi-square test or Fisher’s exact test. Statistical analyses of clinical data were performed using SAS 9.4 software (SAS Institute Inc., Cary, NC, USA). The extraction of radiomic features and the construction of the model were performed using Python 3.8 software (Python Software Foundation, Delaware, USA). A two-sided P < 0.05 was considered statistically significant.

Results

Clinical characteristics of patients

A total of 259 adult women diagnosed with cervical cancer were selected. Of these patients, 79 women were excluded including 73 patients without MRI image data and 6 patients whose MRI images did not meet post-processing requirements. Table 2 shows the characteristics of the 180 included patients. The mean age was 53.07 ± 9.79 years and 97 (53.89%) patients were postmenopausal women. The number of patients in the FIGO stages was 11 (6.11%) for IA2, 2 (1.11%) for IB1, 45 (25.00%) for IB2, 53 (29.44%) for IB3, 18 (10.00%) for IIA1, and 51 (28.33%) for IIA2. The median squamous cell carcinoma antigen (SCC-Ag) level was 1.63 (0.84, 4.55) ng/mL. There were 49 (27.22%) patients with lymph node metastasis and 131 (72.78%) patients without lymph node metastasis.

Table 2 Clinical characteristics of patients
Full size table

Prediction performance of models

Three clinical features and five radiomic features were selected to build the prediction model. The process of screening features by LASSO regression is presented in Fig. 2. Table 3 shows the performance of different models in predicting lymph node metastasis in patients with cervical cancer. In the training set, the AUCs were 0.939 (95%CI: 0.938–0.940) for the XGBoost model, 0.687 (95%CI: 0.683–0.690) for the Logistic Regression model, 0.611 (95%CI: 0.607–0.615) for the MNB model, 0.830 (95%CI: 0.827–0.832) for the SVM model, 0.691 (95%CI: 0.688–0.694) for the Decision Tree model, 0.875 (95%CI: 0.873–0.877) for the Random Forest model, and 0.997 (95%CI: 0.997–0.997) for the GBDT model. In the testing set, the performance parameters of the models, except for the MNB model, differed significantly between the training and testing sets. The AUC, specificity, and accuracy of the MNB model in the testing set were 0.745 (95%CI: 0.740–0.750), 0.900 (95%CI: 0.807–0.993), and 0.778 (95%CI: 0.667–0.889), respectively. Therefore, the MNB model was used to predict preoperative lymph node metastasis in patients with cervical cancer. The ROC curves and calibrate curves of the MNB model in the training set and testing set are shown in Fig. 3. In addition, the ablation analysis was performed based on the parameter “alpha” of the MNB model. The results showed that when the parameter “alpha” of the MNB model was 5, the test accuracy was best at 77.78% and the test loss was the lowest at 0.8032% (Supplement Table 1).

Fig. 2
figure 2

Feature selection using the least absolute shrinkage and selection operator (LASSO) regression. (a) changes in mean squared error during LASSO regression screening; (b) Changes in the coefficient profiles during LASSO regression screening

Full size image
Table 3 Performances of different models in predicting lymph node metastasis in patients with cervical cancer
Full size table
Fig. 3
figure 3

The receiver operator characteristic (ROC) curves and calibrate curve of the Multinomial Naive Bayes (MNB) model in the training set and testing set. (a) ROC curves and calibrate curve in the training set; (b) ROC curves and calibrate curve in the testing set

Full size image

Performance comparison of models using different features

A comparison of the prediction performance of the MNB model using different features is shown in Table 4. In the testing set, the AUCs of the MNB model with clinical features only, radiomic features only, and combined features were 0.698 (95%CI: 0.692–0.704), 0.632 (95%CI: 0.627–0.637), and 0.745 (95%CI: 0.740–0.750), respectively. Compared with other feature models, the MNB model with radiomic features combined with clinical features had a better performance for predicting preoperative lymph node metastasis in patients with cervical cancer.

Table 4 Comparison of the prediction performance of the Multinomial Naive Bayes (MNB) model using different features
Full size table

Discussion

In this study, we used the radiomic features of MRI images combined with clinical features of patients to predict lymph node metastasis in patients with cervical cancer. We compared the prediction performance of seven machine learning models, among which the MNB model had the best prediction effects in the testing set with an AUC of 0.745. Furthermore, the MNB model with a combination of radiomic and clinical features had better prediction effects than the model with single radiomic features and clinical features.

Predictive models based on radiomic features have been widely used to predict the prognosis of many diseases [26,27,28]. The radiomic feature model provides a non-invasive evaluation method for patient prognosis assessment [29]. Compared to traditional invasive evaluation methods such as biopsy, the radiomic feature models can reduce patient harm and avoid excessive biopsies on patients [30]. Many factors affect the prediction effectiveness of the radiomic feature model, such as the type of image (ultrasound images, CT images, MRI images), the ROI classification, and the method of model construction [31]. Jin et al. constructed a logistic regression model for predicting lymph node metastasis in patients with cervical cancer using the radiomics features of ultrasound images, and the AUC of the model was 0.77 [14]. Chen et al. used clinical features combined with radiomic features of CT images to establish a ridge logistics regression model for predicting lymph node metastasis, and the AUC of the model was 0.75 [22]. More studies have used the radiomic features of MRI images to construct predictive models for lymph node metastasis in patients with cervical cancer [15, 17, 32]. MRI is useful for detecting lymph node metastasis, especially when the tumor size is greater than 4 cm (accuracy 84%) [33]. However, MRI images may miss normal-sized lymph node metastasis and cannot reliably distinguish inflammatory lymph node enlargement from cancer-infiltrating lymph nodes. The radiomic analysis may be able to compensate for this limitation of images alone. This study used the radiomic features of MRI images combined with clinical features to develop a model for predicting lymph node metastasis in patients with cervical cancer. In contrast to previous studies, the current study compared the predictive effects of seven different models. In radiomic analysis, the predictive performance of different machine learning models is different.

Our study screened 3 clinical features and 5 radiomic features for modeling. In clinical features, eosinophils have been reported to be associated with lymph node metastasis in patients with tumors [34]. Red blood cell volume distribution width reflects the size variability of circulating erythrocytes related to chronic inflammation, which is an important influencing factor in the progression of various cancer diseases [35]. Preoperative serum squamous carcinoma antigen levels have also been found to be potentially useful predictors of early lymph node metastasis in squamous cervical cancer [36]. Among our radiomic features, 2 were from T1WI images and 3 were from T2WI images. T2WI can provide information on tumor morphology and stroma information, and T1W can reflect tumor microenvironment and aggressiveness by showing microvascular density and perfusion [37]. The original_firstorder_Range feature is the range of gray values in the ROI, reflecting the heterogeneity within the tumor. The original_ngtdm_Complexity, original_glcm_InverseVariance, and original_glszm_SmallAreaHighGrayLevelEmphasis features are all texture features of the image, and texture can quantify information that is difficult to be perceived simply by vision, such as texture patterns or tissue distribution within the tumor [38]. The original_ngtdm_Complexity indicates the complexity of the image, i.e., the image is non-uniform and there are many rapid changes in gray level intensity. The original_glcm_InverseVariance feature is a measure of image homogeneity. The original_glszm_SmallAreaHighGrayLevelEmphasis feature represents the proportion of the joint distribution of small-sized areas with high gray values in the image, which reflects the gray level changes within peritumoral regions. These radiomic features reflect changes in the tumor and peritumor area.

Among these models, both the XGBoost model and the GBDT model showed good prediction performance in the training set, and their AUCs were more than 0.9. However, these models including the XGBoost model, the GBDT model, the SVM model, the Logistic Regression model, and the Random forest model had significant differences in performance on the training set and testing set. This may suggest an overfitting of these models. The prediction performance of the MNB model was robust in both the training set and testing set, and the AUC in the testing set was 0.745. Furthermore, we compared the predictive performance of the models using radiomic features, clinical features, and radiomic features combined with clinical features, respectively. The results demonstrated that the model with radiomic features combined with clinical features had a better prediction effect. Previous studies have shown that radiomic features combined with patient or tumor characteristics can improve medical decisions through clinical decision support systems, thereby improving diagnostic, prognostic, and predictive accuracies, and facilitating therapeutic research [30, 39]. The prediction performance of the current model has not been significantly improved compared to previous studies. The main direction of future study is to further improve the predictive performance of the model because the AUC of the model was about 0.75 in both the current and previous studies. Furthermore, future studies may need to focus on the metastasis of smaller-diameter lymph nodes.

This study used the radiomic features of MRI images and the clinical features of patients to construct a model for predicting lymph node metastasis in patients with cervical cancer. We compared the prediction performance of seven machine learning models. This study may provide a reference for the selection of different machine learning prediction models. There were several limitations in our study. First, the sample size of patients recruited in this single-center retrospective study was small, and a larger sample size from multiple centers is needed to confirm the predictive effect of our model. Second, serological biomarkers associated with cervical cancer, such as carbohydrate antigen 125 (CA125), carbohydrate antigen 153 (CA153), carbohydrate antigen 199 (CA199) and carcinoembryonic antigen (CEA), were not used to build predictive models due to too much missing data. Third, although the model performed well in internal validation, external validation of the model was also required. Fourth, prospective study design and rigorous study procedures in future studies are needed based on the Radiomics Quality Score of Lambin et al. [39].

Conclusions

This study used the radiomic features of MRI images combined with the clinical features of patients to predict lymph node metastasis in patients with cervical cancer. Seven machine learning methods were used to build models to identify the best modeling method. The MNB model showed the most robust predictive performance, which might be used as a non-invasive tool for the preoperative assessment of lymph node metastasis. Future studies may need to further improve the predictive performance of the model.