Skip to main content

Advertisement

SpringerLink
Log in

Radiomic feature reproducibility in contrast-enhanced CT of the pancreas is affected by variabilities in scan parameters and manual segmentation

  • Hepatobiliary-Pancreas
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

This study aims to measure the reproducibility of radiomic features in pancreatic parenchyma and ductal adenocarcinomas (PDAC) in patients who underwent consecutive contrast-enhanced computed tomography (CECT) scans.

Methods

In this IRB-approved and HIPAA-compliant retrospective study, 37 pairs of scans from 37 unique patients who underwent CECTs within a 2-week interval were included in the analysis of the reproducibility of features derived from pancreatic parenchyma, and a subset of 18 pairs of scans were further analyzed for the reproducibility of features derived from PDAC. In each patient, pancreatic parenchyma and pancreatic tumor (when present) were manually segmented by two radiologists independently. A total of 266 radiomic features were extracted from the pancreatic parenchyma and tumor region and also the volume and diameter of the tumor. The concordance correlation coefficient (CCC) was calculated to assess feature reproducibility for each patient in three scenarios: (1) different radiologists, same CECT; (2) same radiologist, different CECTs; and (3) different radiologists, different CECTs.

Results

Among pancreatic parenchyma-derived features, using a threshold of CCC > 0.90, 58/266 (21.8%) and 48/266 (18.1%) features met the threshold for scenario 1, 14/266 (5.3%) and 15/266 (5.6%) for scenario 2, and 14/266 (5.3%) and 10/266 (3.8%) for scenario 3. Among pancreatic tumor-derived features, 11/268 (4.1%) and 17/268 (6.3%) features met the threshold for scenario 1, 1/268 (0.4%) and 5/268 (1.9%) features met the threshold for scenario 2, and no features for scenario 3 met the threshold, respectively.

Conclusions

Variations between CECT scans affected radiomic feature reproducibility to a greater extent than variation in segmentation. A smaller number of pancreatic tumor-derived radiomic features were reproducible compared with pancreatic parenchyma-derived radiomic features under the same conditions.

Key Points

• For pancreatic-derived radiomic features from contrast-enhanced CT (CECT), fewer than 25% are reproducible (with a threshold of CCC < 0.9) in a clinical heterogeneous dataset.

• Variations between CECT scans affected the number of reproducible radiomic features to a greater extent than variations in radiologist segmentation.

• A smaller number of pancreatic tumor-derived radiomic features were reproducible compared with pancreatic parenchyma-derived radiomic features under the same conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

ACM:

Angle co-occurrence matrix

CCC:

Concordance correlation coefficient

CECT:

Contrast-enhanced computed tomography

DICOM:

Digital Imaging and Communications in Medicine

FD:

Fractal dimension

GLCM:

Gray-level co-occurrence matrix

ICC:

Intraclass correlation coefficient

IH:

Intensity histogram

IQR:

Interquartile range

LBP:

Local binary pattern

PDAC:

Pancreatic ductal adenocarcinoma

RLM:

Run length matrix

References

  1. Siegel RL, Miller KD, Jemal A (2018) Cancer statistics, 2018. CA Cancer J Clin 68:7–30. https://doi.org/10.3322/caac.21442

    Article  Google Scholar 

  2. Garrido-Laguna I, Hidalgo M (2015) Pancreatic cancer: from state-of-the-art treatments to promising novel therapies. Nat Rev Clin Oncol 12:319–334. https://doi.org/10.1038/nrclinonc.2015.53

    Article  CAS  PubMed  Google Scholar 

  3. Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: images are more than pictures, they are data. Radiology 278:563–577. https://doi.org/10.1148/radiol.2015151169

    Article  PubMed  Google Scholar 

  4. Lambin P, Rios-Velazquez E, Leijenaar R et al (2012) Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446. https://doi.org/10.1016/j.ejca.2011.11.036

    Article  PubMed  Google Scholar 

  5. Aerts HJWL, Velazquez ER, Leijenaar RTH et al (2014) Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 5:4006. https://doi.org/10.1038/ncomms5006

    Article  CAS  PubMed  Google Scholar 

  6. Sandrasegaran K, Lin Y, Asare-Sawiri M et al (2019) CT texture analysis of pancreatic cancer. Eur Radiol 29(3):1067–1073. https://doi.org/10.1007/s00330-018-5662-1

  7. Chen X, Oshima K, Schott D et al (2017) Assessment of treatment response during chemoradiation therapy for pancreatic cancer based on quantitative radiomic analysis of daily CTs: an exploratory study. PLoS One 12:e0178961. https://doi.org/10.1371/journal.pone.0178961

    Article  CAS  PubMed  Google Scholar 

  8. Chakraborty J, Langdon-Embry L, Cunanan KM et al (2017) Preliminary study of tumor heterogeneity in imaging predicts two year survival in pancreatic cancer patients. PLoS One 12:e0188022. https://doi.org/10.1371/journal.pone.0188022

    Article  CAS  PubMed  Google Scholar 

  9. Kim BR, Kim JH, Ahn SJ et al (2019) CT prediction of resectability and prognosis in patients with pancreatic ductal adenocarcinoma after neoadjuvant treatment using image findings and texture analysis. Eur Radiol 29(1):362–372. https://doi.org/10.1007/s00330-018-5574-0

  10. Yun G, Kim YH, Lee YJ et al (2018) Tumor heterogeneity of pancreas head cancer assessed by CT texture analysis: association with survival outcomes after curative resection. Sci Rep 8:7226. https://doi.org/10.1038/s41598-018-25627-x

    Article  CAS  PubMed  Google Scholar 

  11. Attiyeh MA, Chakraborty J, Doussot A et al (2018) Survival prediction in pancreatic ductal adenocarcinoma by quantitative computed tomography image analysis. Ann Surg Oncol 25:1034–1042. https://doi.org/10.1245/s10434-017-6323-3

    Article  PubMed  PubMed Central  Google Scholar 

  12. Berenguer R, Pastor-Juan MDR, Canales-Vázquez J et al (2018) Radiomics of CT features may be nonreproducible and redundant: influence of CT acquisition parameters. Radiology 288(2):407–415. https://doi.org/10.1148/radiol.2018172361

  13. Perrin T, Midya A, Yamashita R et al (2018) Short-term reproducibility of radiomic features in liver parenchyma and liver malignancies on contrast-enhanced CT imaging. Abdom Radiol (NY) 43(12):3271–3278. https://doi.org/10.1007/s00261-018-1600-6

  14. van Timmeren JE, Leijenaar RTH, van Elmpt W et al (2016) Test-retest data for radiomics feature stability analysis: generalizable or study-specific? Tomography 2:361–365. https://doi.org/10.18383/j.tom.2016.00208

    Article  PubMed  PubMed Central  Google Scholar 

  15. Summers RM (2017) Texture analysis in radiology: does the emperor have no clothes? Abdom Radiol (NY) 42:342–345. https://doi.org/10.1007/s00261-016-0950-1

    Article  Google Scholar 

  16. Kumar V, Gu Y, Basu S et al (2012) Radiomics: the process and the challenges. Magn Reson Imaging 30:1234–1248. https://doi.org/10.1016/j.mri.2012.06.010

    Article  PubMed  PubMed Central  Google Scholar 

  17. Lambin P, Leijenaar RTH, Deist TM et al (2017) Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14:749–762. https://doi.org/10.1038/nrclinonc.2017.141

    Article  PubMed  Google Scholar 

  18. Zhao B, Tan Y, Tsai W-Y et al (2016) Reproducibility of radiomics for deciphering tumor phenotype with imaging. Sci Rep 6:23428. https://doi.org/10.1038/srep23428

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Pavic M, Bogowicz M, Würms X et al (2018) Influence of inter-observer delineation variability on radiomics stability in different tumor sites. Acta Oncol 57(8):1070–1074. https://doi.org/10.1080/0284186X.2018.1445283

  20. Mingqiang Y, Kidiyo K, Joseph R (2008) A survey of shape feature extraction techniques. Peng-Yeng Yin. Pattern Recognition, IN-TECH, pp 43–90. https://doi.org/10.5772/6237

  21. Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern 3:610–621. https://doi.org/10.1109/TSMC.1973.4309314

    Article  Google Scholar 

  22. Yang X, Tridandapani S, Beitler JJ et al (2012) Ultrasound GLCM texture analysis of radiation-induced parotid-gland injury in head-and-neck cancer radiotherapy: an in vivo study of late toxicity. Med Phys 39:5732–5739. https://doi.org/10.1118/1.4747526

    Article  PubMed  Google Scholar 

  23. Banik S, Rangayyan RM, Desautels JEL (2013) Measures of angular spread and entropy for the detection of architectural distortion in prior mammograms. Int J Comput Assist Radiol Surg 8:121–134. https://doi.org/10.1007/s11548-012-0681-x

    Article  PubMed  Google Scholar 

  24. Tang X (1998) Texture information in run-length matrices. IEEE Trans Image Process 7:1602–1609. https://doi.org/10.1109/83.725367

    Article  CAS  PubMed  Google Scholar 

  25. Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29:51–59. https://doi.org/10.1016/0031-3203(95)00067-4

    Article  Google Scholar 

  26. Pietikäinen M, Hadid A, Zhao G, Ahonen T (2011) Local binary patterns for still images. In: Computer vision using local binary patterns. Springer, London, pp 13–47

    Chapter  Google Scholar 

  27. Mehta R, Egiazarian K (2013) Rotated local binary pattern (RLBP): rotation invariant texture descriptor. In 2nd International Conference on Pattern Recognition Applications and Methods, ICPRAM 2013, Barcelona, Spain, 15.-18.2.2013, pp 497-502. (International Conference on Pattern Recognition Applications and Methods). Institute of Electrical and Electronics Engineers IEEE

  28. Al-Kadi OS, Watson D (2008) Texture analysis of aggressive and nonaggressive lung tumor CE CT images. IEEE Trans Biomed Eng 55:1822–1830. https://doi.org/10.1109/TBME.2008.919735

    Article  PubMed  Google Scholar 

  29. Costa AF, Humpire-Mamani G, Traina AJM (2012) An efficient algorithm for fractal analysis of textures. 25th SIBGRAPI Conference on Graphics, Patterns and Images, Ouro Preto, pp 39–46. https://doi.org/10.1109/SIBGRAPI.2012.15

  30. Chakraborty J, Rangayyan RM, Banik S et al (2012) Statistical measures of orientation of texture for the detection of architectural distortion in prior mammograms of interval-cancer. J Electron Imaging 21:033010–033011. https://doi.org/10.1117/1.JEI.21.3.033010

    Article  Google Scholar 

  31. Chakraborty J, Rangayyan RM, Banik S, et al (2012) Detection of architectural distortion in prior mammograms using statistical measures of orientation of texture. Proc. SPIE 8315, Medical Imaging 2012: Computer-Aided Diagnosis, 831521 (23 February 2012). https://doi.org/10.1117/12.910937

  32. Chakraborty J, Midya A, Mukhopadhyay S, Sadhu A (2013) Automatic characterization of masses in mammograms. 6th International Conference on Biomedical Engineering and Informatics, Hangzhou, pp 111–115. https://doi.org/10.1109/BMEI.2013.6746917

  33. Dice LR (1945) Measures of the amount of ecologic association between species. Ecology 26:297. https://doi.org/10.2307/1932409

    Article  Google Scholar 

  34. ISO 5725-2:1994 - Accuracy (trueness and precision) of measurement methods and results—part 2: basic method for the determination of repeatability and reproducibility of a standard measurement method. https://www.iso.org/standard/11834.html. Accessed 14 Dec 2018

  35. Carrasco JL, Jover L (2003) Estimating the generalized concordance correlation coefficient through variance components. Biometrics 59:849–858. https://doi.org/10.1111/j.0006-341X.2003.00099.x

    Article  PubMed  Google Scholar 

  36. Balagurunathan Y, Gu Y, Wang H et al (2014) Reproducibility and prognosis of quantitative features extracted from CT images. Transl Oncol 7:72–87. https://doi.org/10.1593/tlo.13844

    Article  PubMed  PubMed Central  Google Scholar 

  37. Balagurunathan Y, Kumar V, Gu Y et al (2014) Test-retest reproducibility analysis of lung CT image features. J Digit Imaging 27:805–823. https://doi.org/10.1007/s10278-014-9716-x

    Article  PubMed  PubMed Central  Google Scholar 

  38. Armato SG, Meyer CR, Mcnitt-Gray MF et al (2008) The Reference Image Database to Evaluate Response to therapy in lung cancer (RIDER) project: a resource for the development of change-analysis software. Clin Pharmacol Ther 84:448–456. https://doi.org/10.1038/clpt.2008.161

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Zwanenburg A, Leger S, Vallières M, et al (2016) Image biomarker standardisation initiative. arXiv:1612.07003v7 [cs.CV]

Download references

Acknowledgments

We thank Luz Adriana Escobar Hoyos and Juliana Brooke Schilsky for their support in figure creation and Joanne Chin for editorial assistance.

Funding

This study was funded in part through the 2016 Society of Abdominal Radiology Wylie J. Dodds Research Award, the National Institutes of Health/National Cancer Institute Cancer Center Support Grant P30 CA008748, and the Japan Society Promotion of Science Overseas Research Fellowships JSPS/OT/290125.

Author information

Authors and Affiliations

  1. Department of Radiology, Memorial Sloan Kettering Cancer Center, 1275 York Avenue, New York, NY, 10065, USA

    Rikiya Yamashita, Natally Horvat & Richard K. G. Do

  2. Department of Surgery, Memorial Sloan Kettering Cancer Center, 1275 York Avenue, New York, NY, 10065, USA

    Thomas Perrin, Jayasree Chakraborty, Maura A. Koszalka, Abhishek Midya, Peter Allen, William R. Jarnagin & Amber L. Simpson

  3. Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, 1275 York Avenue, New York, NY, 10065, USA

    Joanne F. Chou & Mithat Gonen

Authors
  1. Rikiya Yamashita
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Perrin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jayasree Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joanne F. Chou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Natally Horvat
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Maura A. Koszalka
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Abhishek Midya
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Mithat Gonen
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Peter Allen
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. William R. Jarnagin
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Amber L. Simpson
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Richard K. G. Do
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard K. G. Do.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Richard K. G. Do, MD, PhD.

Conflict of interest

The authors declare that they have no conflict of interest.

Statistics and biometry

Two of the authors (Joanne Chou and Mithat Gonen, PhD) have significant statistical expertise.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

• retrospective

• cross-sectional study

• performed at one institution

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 24 kb)

ESM 2

(XLSX 17 kb)

ESM 3

(DOCX 16189 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yamashita, R., Perrin, T., Chakraborty, J. et al. Radiomic feature reproducibility in contrast-enhanced CT of the pancreas is affected by variabilities in scan parameters and manual segmentation. Eur Radiol 30, 195–205 (2020). https://doi.org/10.1007/s00330-019-06381-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-019-06381-8

Keywords

Search

Navigation