To identify optimal machine learning methods for radiomics-based differentiation of local recurrence versus inflammation from post-treatment nasopharyngeal positron emission tomography/X-ray computed tomography (PET/CT) images.
Seventy-six nasopharyngeal carcinoma (NPC) patients were enrolled (41/35 local recurrence/inflammation as confirmed by pathology). Four hundred eighty-seven radiomics features were extracted from PET images for each patient. The diagnostic performance was investigated for 42 cross-combinations derived from 6 feature selection methods and 7 classifiers. Of the original cohort, 70 % was applied for feature selection and classifier development, and the remaining 30 % used as an independent validation set. The diagnostic performance was evaluated using area under the ROC curve (AUC), test error, sensitivity, and specificity. Furthermore, the performance of the radiomics signatures against routine features was statistically compared using DeLong’s method.
The cross-combination fisher score (FSCR) + k-nearest neighborhood (KNN), FSCR + support vector machines with radial basis function kernel (RBF-SVM), FSCR + random forest (RF), and minimum redundancy maximum relevance (MRMR) + RBF-SVM outperformed others in terms of accuracy (AUC 0.883, 0.867, 0.892, 0.883; sensitivity 0.833, 0.864, 0.831, 0.750; specificity 1, 1, 0.873, 1) and reliability (test error 0.091, 0.136, 0.150, 0.136). Compared with conventional metrics, the radiomics signatures showed higher AUC values (0.867–0.892 vs. 0.817), though the differences were not statistically significant (p = 0.462–0.560).
This study identified the most accurate and reliable machine learning methods, which could enhance the application of radiomics methods in the precision of diagnosis of NPC.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Chan ATC, Teo PML, Ngan RK, Leung TW, Lau WH, Zee B, Leung SF, Cheung FY, Yeo W, Yiu HH, Yu KH, Chiu KW, Chan DT, Mok T, Yuen KT, Mo F, Lai M, Kwan WH, Choi P, Johnson PJ (2002) Concurrent chemotherapy-radiotherapy compared with radiotherapy alone in locoregionally advanced nasopharyngeal carcinoma: progression-free survival analysis of a phase III randomized trial. J Clin Oncol 20:2038–2044
Al-Sarraf M, LeBlanc M, Giri PG et al (1998) Chemoradiotherapy versus radiotherapy in patients with advanced nasopharyngeal cancer: phase III randomized intergroup study 0099. J Clin Oncol 16:1310–1317
Han F, Zhao C, Huang S-M, Lu LX, Huang Y, Deng XW, Mai WY, Teh BS, Butler EB, Lu TX (2012) Long-term outcomes and prognostic factors of re-irradiation for locally recurrent nasopharyngeal carcinoma using intensity-modulated radiotherapy. Clin Oncol 24:569–576
Sun X, Su S, Chen C, Han F, Zhao C, Xiao W, Deng X, Huang S, Lin C, Lu T (2014) Long-term outcomes of intensity-modulated radiotherapy for 868 patients with nasopharyngeal carcinoma: an analysis of survival and treatment toxicities. Radiother Oncol 110:398–403
Leung T-W, Tung SY, Sze W-K, Sze WM, Wong VYW, Wong CS, O SK (2000) Salvage radiation therapy for locally recurrent nasopharyngeal carcinoma. Int J Radiat Oncol 48:1331–1338
Comoretto M, Balestreri L, Borsatti E, Cimitan M, Franchin G, Lise M (2008) Detection and restaging of residual and/or recurrent nasopharyngeal carcinoma after chemotherapy and radiation therapy: comparison of MR imaging and FDG PET/CT. Radiology 249:203–211
Yen R-F, Hung R-L, Pan M-H, Wang YH, Huang KM, Lui LT, Kao CH (2003) 18-Fluoro-2-deoxyglucose positron emission tomography in detecting residual/recurrent nasopharyngeal carcinomas and comparison with magnetic resonance imaging. Cancer 98:283–287
Zhou H, Shen G, Zhang W, Cai H, Zhou Y, Li L (2016) 18F-FDG PET/CT for the diagnosis of residual or recurrent nasopharyngeal carcinoma after radiotherapy: a metaanalysis. J Nucl Med 57:342–347
Shuhang N, Tungchieh JC, Shengchieh C et al (2004) Clinical usefulness of 18F-FDG PET in nasopharyngeal carcinoma patients with questionable MRI findings for recurrence. J Nucl Med 45:1669–1676
Wu H-b, Wang Q-s, Wang M-f et al (2011) Preliminary study of 11C-choline PET/CT for T staging of locally advanced nasopharyngeal carcinoma: comparison with 18F-FDG PET/CT. J Nucl Med 52:341–346
Lee N, Yoo IR, Park SY, Yoon H, Lee Y, Oh JK (2015) Significance of incidental nasopharyngeal uptake on 18F-FDG PET/CT: patterns of benign/physiologic uptake and differentiation from malignancy. Nucl Med Mol Imaging 49:11–18
Wahl RL (2009) Principles and practice of PET and PET/CT. Lippincott Williams & Wilkins, Philadelphia
Gerlinger M, Rowan AJ, Horswell S, Math M, Larkin J, Endesfelder D, Gronroos E, Martinez P, Matthews N, Stewart A, Tarpey P, Varela I, Phillimore B, Begum S, McDonald N, Butler A, Jones D, Raine K, Latimer C, Santos CR, Nohadani M, Eklund AC, Spencer-Dene B, Clark G, Pickering L, Stamp G, Gore M, Szallasi Z, Downward J, Futreal PA, Swanton C (2012) Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. N Engl J Med 366:883–892
Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RGPM, Granton P, Zegers CML, Gillies R, Boellard R, Dekker A, Aerts HJWL (2012) Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446
Aerts HJWL (2016) The potential of radiomic-based phenotyping in precision medicine: a review. JAMA Oncol 2:1636
Lambin P, Leijenaar RTH, Deist TM, Peerlings J, de Jong EEC, van Timmeren J, Sanduleanu S, Larue RTHM, Even AJG, Jochems A, van Wijk Y, Woodruff H, van Soest J, Lustberg T, Roelofs E, van Elmpt W, Dekker A, Mottaghy FM, Wildberger JE, Walsh S (2017) Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14:749–762
Coroller TP, Grossmann P, Hou Y, Rios Velazquez E, Leijenaar RTH, Hermann G, Lambin P, Haibe-Kains B, Mak RH, Aerts HJWL (2015) CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiother Oncol 114:345–350
Li H, Zhu Y, Burnside ES, Huang E, Drukker K, Hoadley KA, Fan C, Conzen SD, Zuley M, Net JM, Sutton E, Whitman GJ, Morris E, Perou CM, Ji Y, Giger ML (2016) Quantitative MRI radiomics in the prediction of molecular classifications of breast cancer subtypes in the TCGA/TCIA data set. Npj Breast Cancer 2:16012
Oakden-Rayner L, Carneiro G, Bessen T, Nascimento JC, Bradley AP, Palmer LJ (2017) Precision radiology: predicting longevity using feature engineering and deep learning methods in a radiomics framework. Sci Rep 7:1648
Vallières M, Freeman CR, Skamene SR, El Naqa I (2015) A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities. Phys Med Biol 60:5471–5496
Cheng N-M, Fang Y-HD, Lee L, Chang JTC, Tsan DL, Ng SH, Wang HM, Liao CT, Yang LY, Hsu CH, Yen TC (2015) Zone-size nonuniformity of 18F-FDG PET regional textural features predicts survival in patients with oropharyngeal cancer. Eur J Nucl Med Mol Imaging 42:419–428
Win T, Miles KA, Janes SM, Ganeshan B, Shastry M, Endozo R, Meagher M, Shortman RI, Wan S, Kayani I, Ell PJ, Groves AM (2013) Tumor heterogeneity and permeability as measured on the CT component of PET/CT predict survival in patients with non-small cell lung cancer. Clin Cancer Res 19:3591–3599
Lovinfosse P, Janvary ZL, Coucke P, Jodogne S, Bernard C, Hatt M, Visvikis D, Jansen N, Duysinx B, Hustinx R (2016) FDG PET/CT texture analysis for predicting the outcome of lung cancer treated by stereotactic body radiation therapy. Eur J Nucl Med Mol Imaging 43:1453–1460
Mu W, Chen Z, Liang Y, Shen W, Yang F, Dai R, Wu N, Tian J (2015) Staging of cervical cancer based on tumor heterogeneity characterized by texture features on 18F-FDG PET images. Phys Med Biol 60:5123–5139
Galavis PE, Hollensen C, Jallow N, Paliwal B, Jeraj R (2010) Variability of textural features in FDG PET images due to different acquisition modes and reconstruction parameters. Acta Oncol 49:1012–1016
Doumou G, Siddique M, Tsoumpas C, Goh V, Cook GJ (2015) The precision of textural analysis in 18 F-FDG-PET scans of oesophageal cancer. Eur Radiol 25:2805–2812
Leijenaar RTH, Nalbantov G, Carvalho S, van Elmpt WJC, Troost EGC, Boellaard R, Aerts HJWL, Gillies RJ, Lambin P (2015) The effect of SUV discretization in quantitative FDG-PET radiomics: the need for standardized methodology in tumor texture analysis. Sci Rep 5:11075
Shiri I, Rahmim A, Abdollahi H et al (2017) Applying radiomics and machine learning on PET images to predict lung metastases in soft tissue sarcoma patients. Eur J Nucl Med Mol Imaging 44:S47
Wu W, Parmar C, Grossmann P et al (2016) Exploratory study to identify radiomics classifiers for lung cancer histology. Front Oncol 6:71
Parmar C, Grossmann P, Rietveld D et al (2015) Radiomic machine-learning classifiers for prognostic biomarkers of head and neck cancer. Front Oncol 5:272
Parmar C, Grossmann P, Bussink J, Lambin P, Aerts HJWL (2015) Machine learning methods for quantitative radiomic biomarkers. Sci Rep 5:13087
Hawkins SH, Korecki JN, Balagurunathan Y, Yuhua Gu, Kumar V, Basu S, Hall LO, Goldgof DB, Gatenby RA, Gillies RJ (2014) Predicting outcomes of non-small cell lung cancer using CT image features. IEEE Access 2:1418–1426
Boellaard R, Delgado-Bolton R, Oyen WJG, Giammarile F, Tatsch K, Eschner W, Verzijlbergen FJ, Barrington SF, Pike LC, Weber WA, Stroobants S, Delbeke D, Donohoe KJ, Holbrook S, Graham MM, Testanera G, Hoekstra OS, Zijlstra J, Visser E, Hoekstra CJ, Pruim J, Willemsen A, Arends B, Kotzerke J, Bockisch A, Beyer T, Chiti A, Krause BJ, European Association of Nuclear Medicine (EANM) (2015) FDG PET/CT: EANM procedure guidelines for tumour imaging: version 2.0. Eur J Nucl Med Mol Imaging 42:328–354
Ashrafinia S, Dalaie P, Yan R et al (2018) Application of texture and radiomics analysis to clinical myocardial perfusion SPECT imaging [abstract]. J Nucl Med 59:94
Ashrafinia S (2019) Quantitative nuclear medicine imaging using advanced image reconstruction and radiomics. Johns Hopkins University, Ph.D. dissertation
Zwanenburg A, Leger S, Vallières M, Löck S (2019) Image biomarker standardisation initiative. ArXiv Preprint ArXiv: 1612.07003v7
Brown G, Pocock A, Zhao M-J, Luján M (2012) Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. J Mach Learn Res 13:27–66
Roffo G (2016) Feature selection library (MATLAB Toolbox). ArXiv Preprint ArXiv 1607:01327
Foley D (1972) Considerations of sample and feature size. IEEE Trans Inf Theory 18:618–626
Chalkidou A, O’Doherty MJ, Marsden PK (2015) False discovery rates in PET and CT studies with texture features: a systematic review. PLoS One 10:e0124165
DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845
Cook GJR, Siddique M, Taylor BP, Yip C, Chicklore S, Goh V (2014) Radiomics in PET: principles and applications. Clin Transl Imaging 2:269–276
Yip SSF, Aerts HJWL (2016) Applications and limitations of radiomics. Phys Med Biol 61:R150–R166
Soussan M, Orlhac F, Boubaya M, Zelek L, Ziol M, Eder V, Buvat I (2014) Relationship between tumor heterogeneity measured on FDG-PET/CT and pathological prognostic factors in invasive breast cancer. PLoS One 9:e94017
Yang X, He J, Wang J, Li W, Liu C, Gao D, Guan Y (2018) CT-based radiomics signature for differentiating solitary granulomatous nodules from solid lung adenocarcinoma. Lung Cancer 125:109–114
van Timmeren JE, Leijenaar RTH, van Elmpt W, Reymen B, Oberije C, Monshouwer R, Bussink J, Brink C, Hansen O, Lambin P (2017) Survival prediction of non-small cell lung cancer patients using radiomics analyses of cone-beam CT images. Radiother Oncol 123:363–369
Lv W, Yuan Q, Wang Q, Ma J, Feng Q, Chen W, Rahmim A, Lu L (2019) Radiomics analysis of PET and CT components of PET/CT imaging integrated with clinical parameters: application to prognosis for nasopharyngeal carcinoma. Mol Imaging Biol. https://doi.org/10.1007/s11307-018-01304-3
Yip SSF, Coroller TP, Sanford NN et al (2016) Relationship between the temporal changes in positron-emission-tomography-imaging-based textural features and pathologic response and survival in esophageal cancer patients. Front Oncol 6:72
Coroller TP, Agrawal V, Narayan V, Hou Y, Grossmann P, Lee SW, Mak RH, Aerts HJWL (2016) Radiomic phenotype features predict pathological response in non-small cell lung cancer. Radiother Oncol 119:480–486
Deist TM, Dankers FJWM, Valdes G, Wijsman R, Hsu IC, Oberije C, Lustberg T, Soest J, Hoebers F, Jochems A, el Naqa I, Wee L, Morin O, Raleigh DR, Bots W, Kaanders JH, Belderbos J, Kwint M, Solberg T, Monshouwer R, Bussink J, Dekker A, Lambin P (2018) Machine learning algorithms for outcome prediction in (chemo)radiotherapy: an empirical comparison of classifiers. Med Phys 45:3449–3459
Lu L, Lv W, Jiang J, Ma J, Feng Q, Rahmim A, Chen W (2016) Robustness of radiomic features in [11C]choline and [18F]FDG PET/CT imaging of nasopharyngeal carcinoma: impact of segmentation and discretization. Mol Imaging Biol 18:935–945
Lv W, Yuan Q, Wang Q, Ma J, Jiang J, Yang W, Feng Q, Chen W, Rahmim A, Lu L (2018) Robustness versus disease differentiation when varying parameter settings in radiomics features: application to nasopharyngeal PET/CT. Eur Radiol 28:3245–3254
Wu Z, Zhao J, Gao P, Song Y, Sun J, Chen X, Ma B, Wang Z (2017) Prognostic value of pretreatment standardized uptake value of F-18-fluorodeoxyglucose PET in patients with gastric cancer: a meta-analysis. BMC Cancer 17:275
Hyun SH, Choi JY, Shim YM, Kim K, Lee SJ, Cho YS, Lee JY, Lee KH, Kim BT (2010) Prognostic value of metabolic tumor volume measured by 18F-fluorodeoxyglucose positron emission tomography in patients with esophageal carcinoma. Ann Surg Oncol 17:115–122
Nakamura K, Hongo A, Kodama J, Hiramatsu Y (2011) The measurement of SUVmax of the primary tumor is predictive of prognosis for patients with endometrial cancer. Gynecol Oncol 123:82–87
Brooks FJ, Grigsby PW (2014) The effect of small tumor volumes on studies of intratumoral heterogeneity of tracer uptake. J Nucl Med 55:37–42
Hatt M, Majdoub M, Vallieres M, Tixier F, le Rest CC, Groheux D, Hindie E, Martineau A, Pradier O, Hustinx R, Perdrisot R, Guillevin R, el Naqa I, Visvikis D (2015) 18F-FDG PET uptake characterization through texture analysis: investigating the complementary nature of heterogeneity and functional tumor volume in a multi-cancer site patient cohort. J Nucl Med 56:38–44
This study was supported by the National Natural Science Foundation of China under grant 81871437, and the Guangdong Province Universities and Colleges Pearl River Scholar Funded Scheme (Lijun Lu, 2018).
This retrospective study was approved by the Institutional Review Board and informed consent was waived.
Conflict of Interest
The authors declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
About this article
Cite this article
Du, D., Feng, H., Lv, W. et al. Machine Learning Methods for Optimal Radiomics-Based Differentiation Between Recurrence and Inflammation: Application to Nasopharyngeal Carcinoma Post-therapy PET/CT Images. Mol Imaging Biol 22, 730–738 (2020). https://doi.org/10.1007/s11307-019-01411-9
- Nasopharyngeal carcinoma
- Machine learning