Skip to main content

Advertisement

SpringerLink
Log in

Diagnostic performance of commercially available vs. in-house radiomics software in classification of CT images from patients with pancreatic ductal adenocarcinoma vs. healthy controls

  • Pancreas
  • Published:
Abdominal Radiology Aims and scope Submit manuscript

Abstract

Purpose

The purpose of this study is to evaluate diagnostic performance of a commercially available radiomics research prototype vs. an in-house radiomics software in the binary classification of CT images from patients with pancreatic ductal adenocarcinoma (PDAC) vs. healthy controls.

Materials and methods

In this retrospective case–control study, 190 patients with PDAC (97 men, 93 women; 66 ± 9 years) from 2012 to 2017 and 190 healthy potential renal donors (96 men, 94 women; 52 ± 8 years) without known pancreatic disease from 2005 to 2009 were identified from radiology and pathology databases. 3D volume of the pancreas was manually segmented from preoperative CT scans. Four hundred and seventy-eight radiomics features were extracted using in-house radiomics software. Eight hundred and fifty-four radiomics features were extracted using a commercially available research prototype. Random forest classifier was used for binary classification of PDAC vs. normal pancreas. Accuracy, sensitivity, and specificity of commercially available radiomics software were compared to in-house software.

Results

When 40 radiomics features were used in the random forest classification, in-house software achieved superior sensitivity (1.00) and accuracy (0.992) compared to the commercially available research prototype (sensitivity = 0.950, accuracy = 0.968). When the number of features was reduced to five features, diagnostic performance of the in-house software decreased to sensitivity (0.950), specificity (0.923), and accuracy (0.936). Diagnostic performance of the commercially available research prototype was unchanged.

Conclusion

Commercially available and in-house radiomics software achieve similar diagnostic performance, which may lower the barrier of entry for radiomics research and allow more clinician-scientists to perform radiomics research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: Images Are More than Pictures, They Are Data. Radiology 278 (2):563–577. https://doi.org/10.1148/radiol.2015151169

    Article  PubMed  Google Scholar 

  2. Bluemke DA (2020) Top Publications in Radiology, 2019. Radiology 294 (1):2–3. https://doi.org/10.1148/radiol.2019194017

    Article  PubMed  Google Scholar 

  3. Carlos RC, Kahn CE, Halabi S (2018) Data Science: Big Data, Machine Learning, and Artificial Intelligence. J Am Coll Radiol 15 (3 Pt B):497–498. https://doi.org/10.1016/j.jacr.2018.01.029

    Article  PubMed  Google Scholar 

  4. Lubner MG, Smith AD, Sandrasegaran K, Sahani DV, Pickhardt PJ (2017) CT Texture Analysis: Definitions, Applications, Biologic Correlates, and Challenges. Radiographics : a review publication of the Radiological Society of North America, Inc 37 (5):1483–1503. https://doi.org/10.1148/rg.2017170056

  5. Bodalal Z, Trebeschi S, Nguyen-Kim TDL, Schats W, Beets-Tan R (2019) Radiogenomics: bridging imaging and genomics. Abdominal radiology 44 (6):1960–1984. https://doi.org/10.1007/s00261-019-02028-w

    Article  PubMed  Google Scholar 

  6. Chu LC, Park S, Kawamoto S, Fouladi DF, Shayesteh S, Zinreich ES, Graves JS, Horton KM, Hruban RH, Yuille AL, Kinzler KW, Vogelstein B, Fishman EK (2019) Utility of CT Radiomics Features in Differentiation of Pancreatic Ductal Adenocarcinoma From Normal Pancreatic Tissue. AJR American journal of roentgenology:1–9. https://doi.org/10.2214/ajr.18.20901

  7. Kumar V, Gu Y, Basu S, Berglund A, Eschrich SA, Schabath MB, Forster K, Aerts HJ, Dekker A, Fenstermacher D, Goldgof DB, Hall LO, Lambin P, Balagurunathan Y, Gatenby RA, Gillies RJ (2012) Radiomics: the process and the challenges. Magnetic resonance imaging 30 (9):1234–1248. https://doi.org/10.1016/j.mri.2012.06.010

    Article  PubMed  PubMed Central  Google Scholar 

  8. Kocak B, Durmaz ES, Erdim C, Ates E, Kaya OK, Kilickesmez O (2019) Radiomics of Renal Masses: Systematic Review of Reproducibility and Validation Strategies. AJR American journal of roentgenology:1–8. https://doi.org/10.2214/ajr.19.21709

  9. Meyer M, Ronald J, Vernuccio F, Nelson RC, Ramirez-Giraldo JC, Solomon J, Patel BN, Samei E, Marin D (2019) Reproducibility of CT Radiomic Features within the Same Patient: Influence of Radiation Dose and CT Reconstruction Settings. Radiology 293 (3):583–591. https://doi.org/10.1148/radiol.2019190928

    Article  PubMed  Google Scholar 

  10. Yamashita R, Perrin T, Chakraborty J, Chou JF, Horvat N, Koszalka MA, Midya A, Gonen M, Allen P, Jarnagin WR, Simpson AL, Do RKG (2020) Radiomic feature reproducibility in contrast-enhanced CT of the pancreas is affected by variabilities in scan parameters and manual segmentation. European radiology 30 (1):195–205. https://doi.org/10.1007/s00330-019-06381-8

    Article  PubMed  Google Scholar 

  11. Kim H, Park CM, Gwak J, Hwang EJ, Lee SY, Jung J, Hong H, Goo JM (2019) Effect of CT Reconstruction Algorithm on the Diagnostic Performance of Radiomics Models: A Task-Based Approach for Pulmonary Subsolid Nodules. AJR American journal of roentgenology 212 (3):505–512. https://doi.org/10.2214/ajr.18.20018

    Article  PubMed  Google Scholar 

  12. Foy JJ, Robinson KR, Li H, Giger ML, Al-Hallaq H, Armato SG (2018) Variation in algorithm implementation across radiomics software. J Med Imaging (Bellingham) 5 (4):044505. https://doi.org/10.1117/1.jmi.5.4.044505

    Article  Google Scholar 

  13. Orlhac F, Frouin F, Nioche C, Ayache N, Buvat I (2019) Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics. Radiology 291 (1):53–59. https://doi.org/10.1148/radiol.2019182023

    Article  PubMed  Google Scholar 

  14. Park S, Lee SM, Do KH, Lee JG, Bae W, Park H, Jung KH, Seo JB (2019) Deep Learning Algorithm for Reducing CT Slice Thickness: Effect on Reproducibility of Radiomic Features in Lung Cancer. Korean journal of radiology : official journal of the Korean Radiological Society 20 (10):1431–1440. https://doi.org/10.3348/kjr.2019.0212

    Article  Google Scholar 

  15. Zhovannik I, Bussink J, Traverso A, Shi Z, Kalendralis P, Wee L, Dekker A, Fijten R, Monshouwer R (2019) Learning from scanners: Bias reduction and feature correction in radiomics. Clin Transl Radiat Oncol 19:33–38. https://doi.org/10.1016/j.ctro.2019.07.003

    Article  PubMed  PubMed Central  Google Scholar 

  16. Choe J, Lee SM, Do KH, Lee G, Lee JG, Seo JB (2019) Deep Learning-based Image Conversion of CT Reconstruction Kernels Improves Radiomics Reproducibility for Pulmonary Nodules or Masses. Radiology 292 (2):365–373. https://doi.org/10.1148/radiol.2019181960

    Article  PubMed  Google Scholar 

  17. Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, Bussink J, Monshouwer R, Haibe-Kains B, Rietveld D, Hoebers F, Rietbergen MM, Leemans CR, Dekker A, Quackenbush J, Gillies RJ, Lambin P (2014) Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nature communications 5:4006. https://doi.org/10.1038/ncomms5006

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Funding

Linda C. Chu, Seyoun Park, Satomi Kawamoto, Alan L. Yuille, and Elliot K. Fishman received research support from the Lustgarten Foundation. Linda C. Chu, Seyoun Park, and Elliot K. Fishman received additional research support from the Emerson Collective. Other authors have no disclosures.

Author information

Authors and Affiliations

  1. The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, MD, USA

    Linda C. Chu, Berkan Solmaz, Seyoun Park, Satomi Kawamoto & Elliot K. Fishman

  2. Department of Computer Science, Johns Hopkins University, 3400 N. Charles Street, Clark 324B, Baltimore, MD, 21218, USA

    Alan L. Yuille

  3. Sol Goldman Pancreatic Cancer Research Center, Department of Pathology, Johns Hopkins Hospital, Johns Hopkins University School of Medicine, Baltimore, MD, 21287, USA

    Ralph H. Hruban

Authors
  1. Linda C. Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Berkan Solmaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seyoun Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Satomi Kawamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alan L. Yuille
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ralph H. Hruban
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Elliot K. Fishman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Linda C. Chu.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Ethics approval

This was an IRB-approved retrospective study.

Informed consent

Patient consent was waived given retrospective nature of the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chu, L.C., Solmaz, B., Park, S. et al. Diagnostic performance of commercially available vs. in-house radiomics software in classification of CT images from patients with pancreatic ductal adenocarcinoma vs. healthy controls. Abdom Radiol 45, 2469–2475 (2020). https://doi.org/10.1007/s00261-020-02556-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00261-020-02556-w

Keywords

Search

Navigation