Skip to main content

Advertisement

SpringerLink
Log in

Deep learning and radiomics: the utility of Google TensorFlow™ Inception in classifying clear cell renal cell carcinoma and oncocytoma on multiphasic CT

  • Special Section: Radiogenomics
  • Published:
Abdominal Radiology Aims and scope Submit manuscript

Abstract

Purpose

Currently, all solid enhancing renal masses without microscopic fat are considered malignant until proven otherwise and there is substantial overlap in the imaging findings of benign and malignant renal masses, particularly between clear cell RCC (ccRCC) and benign oncocytoma (ONC). Radiomics has attracted increased attention for its utility in pre-operative work-up on routine clinical images. Radiomics based approaches have converted medical images into mineable data and identified prognostic imaging signatures that machine learning algorithms can use to construct predictive models by learning the decision boundaries of the underlying data distribution. The TensorFlow™ framework from Google is a state-of-the-art open-source software library that can be used for training deep learning neural networks for performing machine learning tasks. The purpose of this study was to investigate the diagnostic value and feasibility of a deep learning-based renal lesion classifier using open-source Google TensorFlow™ Inception in differentiating ccRCC from ONC on routine four-phase MDCT in patients with pathologically confirmed renal masses.

Methods

With institutional review board approval for this 1996 Health Insurance Portability and Accountability Act compliant retrospective study and a waiver of informed consent, we queried our institution’s pathology, clinical, and radiology databases for histologically proven cases of ccRCC and ONC obtained between January 2000 and January 2016 scanned with a an intravenous contrast-enhanced four-phase renal mass protocol (unenhanced (UN), corticomedullary (CM), nephrographic (NP), and excretory (EX) phases). To extract features to be used for the machine learning model, the entire renal mass was contoured in the axial plane in each of the four phases, resulting in a 3D volume of interest (VOI) representative of the entire renal mass. We investigated thirteen different approaches to convert the acquired VOI data into a set of images that adequately represented each tumor which was used to train the final layer of the neural network model. Training was performed over 4000 iterations. In each iteration, 90% of the data were designated as training data and the remaining 10% served as validation data and a leave-one-out cross-validation scheme was implemented. Accuracy, sensitivity, specificity, positive (PPV) and negative predictive (NPV) values, and CIs were calculated for the classification of the thirteen processing modes.

Results

We analyzed 179 consecutive patients with 179 lesions (128 ccRCC and 51 ONC). The ccRCC cohort had a mean size of 3.8 cm (range 0.8–14.6 cm) and the ONC cohort had a mean lesion size of 3.9 cm (range 1.0–13.1 cm). The highest specificity and PPV (52.9% and 80.3%, respectively) were achieved in the EX phase when we analyzed the single mid-slice of the tumor in the axial, coronal and sagittal plane, and when we increased the number of mid-slices of the tumor to three, with an accuracy of 75.4%, which also increased the sensitivity to 88.3% and the PPV to 79.6%. Using the entire tumor volume also showed that classification performance was best in the EX phase with an accuracy of 74.4%, a sensitivity of 85.8% and a PPV of 80.1%. When the entire tumor volume, plus mid-slices from all phases and all planes presented as tiled images, were submitted to the final layer of the neural network we achieved a PPV of 82.5%.

Conclusions

The best classification result was obtained in the EX phase among the thirteen classification methods tested. Our proof of concept study is the first step towards understanding the utility of machine learning in the differentiation of ccRCC from ONC on routine CT images. We hope this could lead to future investigation into the development of a multivariate machine learning model which may augment our ability to accurately predict renal lesion histology on imaging.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Global Burden of Disease Cancer Collaboration. JAMA Oncol 2015; 1: 505-527

    Article  PubMed Central  Google Scholar 

  2. Sasaguri K, Takahashi N. CT and MR imaging for solid renal mass characterization. European Journal of Radiology 2018; 99: 40-54

    Article  PubMed  Google Scholar 

  3. Silverman SG, Israel GM, Herts BR, Richie JP: Management of the incidental renal mass. Radiology 2008; 249: 16-31

    Article  PubMed  Google Scholar 

  4. Liu N, Huang D, Cheng X, et al. Percutaneous radiofrequency ablation for renal cell carcinoma vs. partial nephrectomy: Comparison of long-term oncologic outcomes in both clear cell and non-clear cell of the most common subtype. Urol Oncol 2017; 35(8): 530.e6. https://doi.org/10.1016/j.urolonc.2017.03.014.

  5. Ishigami K, Jones AR, Dahmoush L, Leite LV, Pakalniskis MG, Barloon TJ. Imaging spectrum of renal oncocytomas: a pictorial review with pathologic correlation. Insights into Imaging 2015;6(1):53-64

    Article  PubMed  Google Scholar 

  6. Frank I, Blute ML, Cheville JC, Lohse CM, Weaver AL, Zincke H. Solid renal tumors: an analysis of pathological features related to tumor size. J Urol 2003; 170 (6 Pt 1): 2217-20

    Article  PubMed  Google Scholar 

  7. Guyon I, Elisseeff A. An Introduction to Variable and Feature Selection. J. Mach. Learn. 2003; 3(3): 1157–1182

    Google Scholar 

  8. Aerts HJ, Velazquez E, Leijenaar R, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 2014; 5 (4006). https://doi.org/10.1038/ncomms5006.

  9. Gillies RJ, Kinahan P, Hricak H. Radiomics: Images are more than pictures, they are data. Radiology 2016; 278: 563-577

    Article  PubMed  Google Scholar 

  10. Song J, Liu Z, Zhong W, et al. Non-small cell lung cancer: quantitative phenotypic analysis of CT images as a potential marker of prognosis. Sci. Rep 2016; 6: 38282

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Kickingereder P, Burth S, Wick A, et al. Radiomic Profiling of Glioblastoma: Identifying an Imaging Predictor of Patient Survival with Improved Performance over Established Clinical and Radiologic Risk Models. Radiology 2016; 280(3): 880–889

    Article  PubMed  Google Scholar 

  12. Vallières M, Freeman CR, Skamene SR , El Naqa I. A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities. Phys. Med. Biol. 2015; 60(14), 5471–96.

    Article  PubMed  Google Scholar 

  13. Aerts HJ, Velazquez ER, Leijenaar RT, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 2014; 5: 4006. https://doi.org/10.1038/ncomms5006

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Hatt M, Tixier F, Pierce L, Kinahan PE, Le rest CC, Visvikis D. Characterization of PET/CT images using texture analysis: the past, the present… any future? Eur J Nucl Med Mol Imaging 2017; 44(1): 151-165.

  15. Yip, S. S. F. & Aerts, H. J. W. L. Applications and limitations of radiomics. Phys. Med. Biol. 61(13), R150–R166 (2016)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Parmar C, Grossmann P, Bussink J, Lambin P, Aerts H. Machine Learning method for Quantitative Radiomic Biomarkers. Sci Rep 2015; 5:13087. https://doi.org/10.1038/srep13087

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Gerlinger, Rowan, Horswell S, et al. Intratumor Heterogeneity and Branched Evolution Revealed by Multiregion Sequencing. New England Journal of Medicine 2012; 366(10) 883-92.

  18. Yap F, Hwang D, Cen S, et al. Quantitative Contour Analysis as an Image-Based Discriminator between Benign and Malignant Renal Tumors. Urology 2018; 114:121-127

    Article  PubMed  Google Scholar 

  19. Chen F, Gulati M, Hwang D, et al. Voxel-based whole lesion enhancement parameters: a study of its clinical value in differentiating clear cell renal cell carcinoma from renal oncocytoma. Abdominal Radiology 2017; 42: 552-560

    Article  PubMed  Google Scholar 

  20. Jamshidi N, Jonasch E, Zapala M, et al. The Radiogenomic Risk Score: Construction of a Prognostic Quantitative, Noninvasive Image-based Molecular Assay for Renal Cell Carcinoma. Radiology 2015; 277:114-123

    Article  PubMed  Google Scholar 

  21. Lubner M, Stabo N, Abel EJ, Munoz del Rio A, Pickhardt P. CT Textural Analysis of Large Primary Renal Cell Carcinomas: Pretreatment Tumor Heterogeneity Correlates With Histologic Findings and Clinical Outcomes. AJR 2016; 207: 96-105.

  22. Yin Q, Hung SC, Wang L, et al. Associations between Tumor Vascularity, Vascular Endothelial Growth Factor Expression and PET/MRI Radiomic Signatures in Primary Clear Cell Renal Cell Carcinoma: Proof of Concept Study. Sci Rep 2017; 7: 43356

    Article  PubMed  PubMed Central  Google Scholar 

  23. Shinagare A, Krajewski K, Braschi-Amirfarzan M, Ramaiya N. Advanced Renal Cell Carcinoma: Role of the Radiologist in the Era of Precision Medicine. Radiology 2017; 284(2): 333-351

    Article  PubMed  Google Scholar 

  24. Karlo C.A., Di Paolo P.L., Chaim J., et al. Radiogenomics of clear cell renal cell carcinoma: Associations between CT imaging features and mutations. Radiology. 2014; 270:464–471.

    Article  PubMed  Google Scholar 

  25. Shinagare A.B., Vikram R., Jaffe C., et al. Radiogenomics of clear cell renal cell carcinoma: Preliminary findings of the cancer genome Atlas-Renal Cell Carcinoma (TCGA-RCC) imaging research group. Abdom. Imaging. 2015;40:1684–1692

    Article  PubMed  PubMed Central  Google Scholar 

  26. Yu H, Scalera J, Khalid M, et al. Texture analysis as a radiomic marker for differentiating renal tumors. Abdom Radiol 2017; 42 (10): 2470-2478

    Article  Google Scholar 

  27. Lee HS, Hong H Jung DC, Park s, Kim J. Differentiation of fat-poor angiomyolipoma from clear cell renal cell carcinoma in contrast-enhanced MDCT images using quantitative feature classification. Med Phys. 2017 Jul;44(7):3604-3614.

  28. Jagga Z, Gupta D. Classification models for clear cell renal carcinoma stage progression, based on tumor RNAseq expression trained supervised machine learning algorithms. BMC Proc 2014; 8 (Suppl 6): S2

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Young JR, Margolis D, Sauk S, Pantuck AJ, Sayre J, Raman SS: Clear cell renal cell carcinoma: discrimination from other renal cell carcinoma subtypes and oncocytoma at multiphasic multidetector CT. Radiology 2013; 267(2):444-453

    Article  PubMed  Google Scholar 

  30. Lee-Felker S, Felker E, Tan N, et al: Qualitative and Quantitative MDCT Features for Differentiating Clear Cell Renal Cell Carcinoma From Other Solid Renal Cortical Masses. AJR Am J Roentgenol 2014; 203(5):W516-W524

    Article  PubMed  Google Scholar 

  31. Zhang J, Lefkowitz RA, Ishill NM, et al. Solid renal cortical tumors: differentiation with CT. Radiology 2007; 244(2):494-504

    Article  PubMed  Google Scholar 

  32. Pierorazio PM, Hyams ES, Tsai S, et al. Multiphasic attenuation patterns of small renal masses (≤4cm) on preoperative computed tomography: utility for distinguishing subtypes of renal cell carcinoma, angiomyolipoma, and oncocytoma. Urology 2013; 81(6):1265-1271

    Article  PubMed  PubMed Central  Google Scholar 

  33. Bird V, Kanagarajah P, Morillo G, et al. Differentiation of oncocytoma and renal cell carcinoma in small renal masses (<4cm): the role of 4-phase computerized tomography. Worl J Urol 2011; 29:787-792

    Article  CAS  Google Scholar 

  34. Pano B, Macias N, Salvador R, et al. Usefulness of MDCT to Differentiate Between Renal Cell Carcinoma and Oncocytoma: Development of a Predictive Model. AJR 2016; 764-774.

  35. Coy H, Young JR, Douek M, et al. Quantitative computer-aided diagnostic algorithm for automated detection of peak lesion attenuation in differentiating clear cell from papillary and chromophobe renal cell carcinoma, oncocytoma, and fat-poor amgiomyolipoma on multiphasic multidetector computed tomography. Abdom Radiol 2017; 42(7): 1919-1928

    Article  Google Scholar 

  36. Yates EJ, Yates LC, Harvey H. Machine learning “red dot”: open-source, cloud, deep convolutional neural networks in chest radiograph binary normality classification. Clinical Radiology 2018; 73(9): 827-831

    Article  CAS  PubMed  Google Scholar 

  37. Ha R, Chang P, Karcich J, et al. Convolutional Neural Network Based Breast Cancer Risk Stratification Using a Mammographic Dataset. Academic Radiology 2018 July 31. pii: S1076-6332(18)30334-9. doi: 10.1016/j.acra.2018.06.020. [Epub ahead of print]

  38. Zhang YC, Kagen AC. Machine Learning Interface for Medical Image Analysis. J Digit Imaging 2017; 30(5): 615-621

    Article  PubMed  Google Scholar 

  39. Python Imaging Library (PIL). Available at http://www.pythonware.com/products/pil/. Accessed on 30 Jan 2018.

  40. Tensorflow. Available at https://www.tensorflow.org. Accessed on 30 Jan 2018.

  41. Szegedy C, Vanhoucke V, Loeffe S, Shlens J, and Wojna Z. “Rethinking the Inception Architecture for Computer Vision,” arXiv: 1512.00567v3, 2015. arxiv.org/abs/1512.00567.

  42. Russakovsky O, Deng J, Su H, et al. “Imagenet large scale visual recognition challenge,” arXiv: 1409.0575v3, 2014. arxiv.org/abs/1409.0575.

  43. Varghese B, Chen F, Hwang D, et al. Differentiation of Predominantly Solid Enhancing Lipid-Poor Renal Cell Masses by Use of Contrast-Enhanced CT: Evaluating the Role of Texture in Tumor Subtyping. AJR 2018; 211: 1-9

    Article  Google Scholar 

  44. Kumar V, Gu Y, Basu S, et al. Radiomics: the process and the challenges. Magn Reson Imaging 2012; 30(9): 1234-48

    Article  PubMed  PubMed Central  Google Scholar 

  45. Raman SP, Chen Y, Schroeder JL, Huang P, Fishman EK. CT texture analysis of renal masses. Acad Radiol 2014; 21:1587–1596

    Article  PubMed  PubMed Central  Google Scholar 

  46. Lobo JM, Jimenez-Valverde A, Real R. AUC: a misleading measure of the performance of predictive distribution models. Glob Ecol Biogeogr 2008; 17: 145-151

    Article  Google Scholar 

  47. American Cancer Society (2018) Cancer facts & figures 2016. Atlanta: American Cancer Society

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Radiological Sciences, David Geffen School of Medicine at UCLA, Ronald Reagan-UCLA Medical Center, 924 Westwood Boulevard, Suite 615, Los Angeles, CA, 90024, USA

    Heidi Coy, Mahesh B. Nagarajan, Michael L. Douek & Matthew S. Brown

  2. Department of Computer Science, School of Engineering at UCLA, NRB 635 Charles E. Young Dr. South Suite 116, BOX 951769, Los Angeles, CA, 90095-1769, USA

    Kevin Hsieh

  3. Department of Bioengineering, School of Engineering at UCLA, NRB 635 Charles E. Young Dr. South Suite 116, BOX 951769, Los Angeles, CA, 90095-1769, USA

    Willie Wu

  4. Department of Radiology, City of Hope National Medical Center, 1500 East Duarte Road, Duarte, CA, 91010, USA

    Jonathan R. Young

  5. Department of Neurology, David Geffen School of Medicine at UCLA, NRB 635 Charles E. Young Dr. South Suite 116, BOX 951769, Los Angeles, CA, 90095-1769, USA

    Fabien Scalzo

  6. Department of Radiological Sciences, David Geffen School of Medicine at UCLA, Ronald Reagan-UCLA Medical Center, 757 Westwood Plaza, RRUMC 1621H, Box 957437, Los Angeles, CA, 90095, USA

    Steven S. Raman

Authors
  1. Heidi Coy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kevin Hsieh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Willie Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mahesh B. Nagarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jonathan R. Young
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael L. Douek
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Matthew S. Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Fabien Scalzo
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Steven S. Raman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Heidi Coy.

Ethics declarations

Conflicts of interest

All authors have no conflicts of interest.

Informed consent

All data was acquired with IRB approval, followed HIPAA guidelines, and with a waiver of informed consent.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Coy, H., Hsieh, K., Wu, W. et al. Deep learning and radiomics: the utility of Google TensorFlow™ Inception in classifying clear cell renal cell carcinoma and oncocytoma on multiphasic CT. Abdom Radiol 44, 2009–2020 (2019). https://doi.org/10.1007/s00261-019-01929-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00261-019-01929-0

Keywords

Search

Navigation