Skip to main content
SpringerLink
Log in

Benchmarking Various Radiomic Toolkit Features While Applying the Image Biomarker Standardization Initiative toward Clinical Translation of Radiomic Analysis

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

The image biomarkers standardization initiative (IBSI) was formed to address the standardization of extraction of quantifiable imaging metrics. Despite its effort, there remains a lack of consensus or established guidelines regarding radiomic feature terminology, the underlying mathematics and their implementation across various software programs. This creates a scenario where features extracted using different toolboxes cannot be used to build or validate the same model leading to a non-generalization of radiomic results. In this study, IBSI-established phantom and benchmark values were used to compare the variation of the radiomic features while using 6 publicly available software programs and 1 in-house radiomics pipeline. All IBSI-standardized features (11 classes, 173 in total) were extracted. The relative differences between the extracted feature values from the different software programs and the IBSI benchmark values were calculated to measure the inter-software agreement. To better understand the variations, features are further grouped into 3 categories according to their properties: 1) morphology, 2) statistic/histogram and 3)texture features. While a good agreement was observed for a majority of radiomics features across the various tested programs, relatively poor agreement was observed for morphology features. Significant differences were also found in programs that use different gray-level discretization approaches. Since these software programs do not include all IBSI features, the level of quantitative assessment for each category was analyzed using Venn and UpSet diagrams and quantified using two ad hoc metrics. Morphology features earned lowest scores for both metrics, indicating that morphological features are not consistently evaluated among software programs. We conclude that radiomic features calculated using different software programs may not be interchangeable. Further studies are needed to standardize the workflow of radiomic feature extraction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Court, L.E., Fave, X., Mackin, D., Lee, J., Yang, J., Zhang, L.: Computational resources for radiomics. Translational Cancer Research 5(4) (2016). http://tcr.amegroups.com/article/view/8409

  2. Lee, S.H., Cho, H.h., Lee, H.Y., Park, H.: Clinical impact of variability on ct radiomics and suggestions for suitable feature selection: a focus on lung cancer. Cancer Imaging, 19(1):54, 2019. https://doi.org/10.1186/s40644-019-0239-z

  3. Béresová, M., Forgács, A., Bujdosó, B., Székely, A., Varga, J., Berényi, E., Balkay, L.: Comparing the reliability of biomedical texture analysis tools on different image types. Acta Polytechnica Hungarica 15(7), 29–48 (2018). https://doi.org/10.12700/APH.15.7.2018.7.2

  4. Bianchi, J., Gonçalves, J.R., Ruellas, A.C.d.O., Vimort, J.B., Yatabe, M., Paniagua, B., Hernandez, P., Benavides, E., Soki, F.N., Cevidanes, L.H.S.: Software comparison to analyze bone radiomics from high resolution cbct scans of mandibular condyles. Dentomaxillofacial Radiology 48(6), 20190049 (2019). https://doi.org/10.1259/dmfr.20190049. PMID: 31075043

  5. Foy, J.J., Robinson, K.R., Li, H., Giger, M.L., Al-Hallaq, H., Armato, S.G.: Variation in algorithm implementation across radiomics software. Journal of Medical Imaging 5(4), 1 – 10 (2018). https://doi.org/10.1117/1.JMI.5.4.044505

  6. Shafiq-ul Hassan, M., Zhang, G.G., Latifi, K., Ullah, G., Hunt, D.C., Balagurunathan, Y., Abdalah, M.A., Schabath, M.B., Goldgof, D.G., Mackin, D., Court, L.E., Gillies, R.J., Moros, E.G.: Intrinsic dependencies of ct radiomic features on voxel size and number of gray levels. Medical Physics 44(3), 1050–1062 (2017). https://doi.org/10.1002/mp.12123, https://aapm.onlinelibrary.wiley.com/doi/abs/10.1002/mp.12123

  7. Bogowicz, M., Riesterer, O., Bundschuh, R.A., Veit-Haibach, P., Hullner, M., Studer, G., Stieb, S., Glatz, S., Pruschy, M., Guckenberger, M., Tanadini-Lang, S.: Stability of radiomic features in CT perfusion maps. Physics in Medicine and Biology 61(24), 8736–8749 (2016). https://doi.org/10.1088/1361-6560/61/24/8736

  8. Kumar, V., Gu, Y., Basu, S., Berglund, A., Eschrich, S.A., Schabath, M.B., Forster, K., Aerts, H.J., Dekker, A., Fenstermacher, D., Goldgof, D.B., Hall, L.O., Lambin, P., Balagurunathan, Y., Gatenby, R.A., Gillies, R.J.: Radiomics: the process and the challenges. Magnetic Resonance Imaging 30(9), 1234 – 1248 (2012). https://doi.org/10.1016/j.mri.2012.06.010,  http://www.sciencedirect.com/science/article/pii/S0730725X12002202. Quantitative Imaging in Cancer

  9. Schwier, M., van Griethuysen, J., Vangel, M.G., Pieper, S., Peled, S., Tempany, C., Aerts, H.J.W.L., Kikinis, R., Fennessy, F.M., Fedorov, A. Repeatability of multiparametric prostate mri radiomics features. Scientific Reports 9(1), 9441, (2019). https://doi.org/10.1038/s41598-019-45766-z

    Article  Google Scholar 

  10. Mackin, D., Fave, X., Zhang, L., Fried, D., Yang, J., Taylor, B., Rodriguez-Rivera, E., Dodge, C., Jones, A.K., Court, L., Measuring computed tomography scanner variability of radiomics features. Investigative radiology, 50(11), 757–765, (2015). https://doi.org/10.1097/RLI.0000000000000180

  11. van Velden, F.H.P., Kramer, G.M., Frings, V., Nissen, I.A., Mulder, E.R., de Langen, A.J., Hoekstra, O.S., Smit, F.E., and Boellaard, R., Repeatability of radiomic features in non-small-cell lung cancer [18f]fdg-pet/ct studies: Impact of reconstruction and delineation. Molecular Imaging and Biology, 18(5), 788–795 (2016). https://doi.org/10.1007/s11307-016-0940-2

  12. Zwanenburg, A., Valliéres, M., Abdalah, M.A., Aerts, H.J.W.L., Andrearczyk, V., Apte, A., Ashrafinia, S., Bakas, S., Beukinga, R.J., Boellaard, R., et al.: The image biomarker standardization initiative: Standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology 295(2), 328–338 (2020). https://doi.org/10.1148/radiol.2020191145

  13. van Griethuysen, J.J., Fedorov, A., Parmar, C., Hosny, A., Aucoin, N., Narayan, V., Beets-Tan, R.G., Fillion-Robin, J.C., Pieper, S., Aerts, H.J., Computational radiomics system to decode the radiographic phenotype. Cancer Research 77(21), e104–e107 (2017). https://doi.org/10.1158/0008-5472.CAN-17-0339,  https://cancerres.aacrjournals.org/content/77/21/e104

  14. Echegaray, S., Bakr, S., Rubin, D.L., Napel, S.: Quantitative image feature engine (qife): an open-source, modular engine for 3d quantitative feature extraction from volumetric medical images. Journal of Digital Imaging 31(4), 403–414 (2018). https://doi.org/10.1007/s10278-017-0019-x

  15. Aerts, H.J.W.L., Velazquez, E.R., Leijenaar, R.T.H., Parmar, C., Grossmann, P., Carvalho, S., Bussink, J., Monshouwer, R., Haibe-Kains, B., Rietveld, D., Hoebers, F., Rietbergen, M.M., Leemans, C.R., Dekker, A., Quackenbush, J., Gillies, R.J., Lambin, P.: Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nature Communications 5(1), 4006 (2014). https://doi.org/10.1038/ncomms5006

  16. Wu, W., Parmar, C., Grossmann, P., Quackenbush, J., Lambin, P., Bussink, J., Mak, R., Aerts, H.J.W.L., Exploratory study to identify radiomics classifiers for lung cancer histology. Frontiers in Oncology, 6:71, (2016). https://doi.org/10.3389/fonc.2016.00071, https://www.frontiersin.org/article/10.3389/fonc.2016.00071

  17. Coroller, T.P., Grossmann, P., Hou, Y., Velazquez, E.R., Leijenaar, R.T., Hermann, G., Lambin, P., Haibe-Kains, B., Mak, R.H., Aerts, H.J.: Ct-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiotherapy and Oncology 114(3), 345–350 (2015). https://doi.org/10.1016/j.radonc.2015.02.015, http://www.sciencedirect.com/science/article/pii/S0167814015001073

  18. Dou, T.H., Coroller, T.P., van Griethuysen, J.J.M., Mak, R.H., Aerts, H.J.W.L.: Peritumoral radiomics features predict distant metastasis in locally advanced nsclc. PloS one 13(11), e0206108–e0206108 (2018). https://doi.org/10.1371/journal.pone.0206108

  19. van Griethuysen, J.J.M., Lambregts, D.M.J., Trebeschi, S., Lahaye, M.J., Bakers, F.C.H., Vliegen, R.F.A., Beets, G.L., Aerts, H.J.W.L., Beets-Tan, R.G.H., Radiomics performs comparable to morphologic assessment by expert radiologists for prediction of response to neoadjuvant chemoradiotherapy on baseline staging mri in rectal cancer. Abdominal Radiology, 45(3), 632–643 (2020). https://doi.org/10.1007/s00261-019-02321-8

  20. Götz, M., Nolden, M., Maier-Hein. K.: Mitk phenotyping: An open-source toolchain for image-based personalized medicine with radiomics. Radiotherapy and Oncology, 131:108 – 111 (2019). https://doi.org/10.1016/j.radonc.2018.11.021, http://www.sciencedirect.com/science/article/pii/S0167814018336156

  21. Kickingereder, P., Götz, M., Muschelli, J., Wick, A., Neuberger, U., Shinohara, R.T., Sill, M., Nowosielski, M., Schlemmer, H.P., Radbruch, A., Wick, W., Bendszus, M., Maier-Hein, K.H., Bonekamp, D.: Large-scale radiomic profiling of recurrent glioblastoma identifies an imaging predictor for stratifying anti-angiogenic treatment response. Clinical Cancer Research 22(23), 5765–5771 (2016). https://doi.org/10.1158/1078-0432.CCR-16-0702, https://clincancerres.aacrjournals.org/content/22/23/5765

  22. Kickingereder, P., Burth, S., Wick, A., Götz, M., Eidel, O., Schlemmer, H.P., Maier-Hein, K.H., Wick, W., Bendszus, M., Radbruch, A., Bonekamp, D.: Radiomic profiling of glioblastoma: Identifying an imaging predictor of patient survival with improved performance over established clinical and radiologic risk models. Radiology 280(3), 880–889 (2016). https://doi.org/10.1148/radiol.2016160845. PMID: 27326665. 

  23. Nioche, C., Orlhac, F., Boughdad, S., Reuzé, S., Goya-Outi, J., Robert, C., Pellot-Barakat, C., Soussan, M., Frouin, F., Buvat, I.: Lifex: A freeware for radiomic feature calculation in multimodality imaging to accelerate advances in the characterization of tumor heterogeneity. Cancer Research 78(16), 4786–4789, (2018). https://doi.org/10.1158/0008-5472.CAN-18-0125, https://cancerres.aacrjournals.org/content/78/16/4786

    Article  CAS  Google Scholar 

  24. Nioche, C., Orlhac, F., Boughdad, S., Reuze, S., Soussan, M., Robert, C., Barakat, C., Buvat, I.: A freeware for tumor heterogeneity characterization in pet, spect, ct, mri and us to accelerate advances in radiomics. Journal of Nuclear Medicine 58(supplement 1), 1316 (2017).

    Google Scholar 

  25. Nardone, V., Tini, P., Nioche, C., Mazzei, M.A., Carfagno, T., Battaglia, G., Pastina, P., Grassi, R., Sebaste, L., Pirtoli, L.: Texture analysis as a predictor of radiation-induced xerostomia in head and neck patients undergoing imrt. La radiologia medica 123(6), 415–423 (2018). https://doi.org/10.1007/s11547-017-0850-7

    Article  Google Scholar 

  26. Ashrafinia, S.: Quantitative nuclear medicine imaging using advanced image reconstruction and radiomics. Ph.D. thesis, John Hopskins University (2019)

  27. Ashrafinia, S., Dalaie, P., Yan, R., Huang, P., Pomper, M., Schindler, T., Rahmim, A.: Application of texture and radiomics analysis to clinical myocardial perfusion spect imaging. Journal of Nuclear Medicine 59(supplement 1), 94 (2018)

    Google Scholar 

  28. Du, D., Feng, H., Lv, W., Ashrafinia, S., Yuan, Q., Wang, Q., Yang, W., Feng, Q., Chen, W., Rahmim, A., Lu, L.: Machine learning methods for optimal radiomics-based differentiation between recurrence and inflammation: Application to nasopharyngeal carcinoma post-therapy pet/ct images. Molecular Imaging and Biology 22(3), 730–738 (2020). https://doi.org/10.1007/s11307-019-01411-9

  29. Davatzikos, C., Rathore, S., Bakas, S., Pati, S., Bergman, M., Kalarot, R., Sridharan, P., Gastounioti, A., Jahani, N., Cohen, E., Akbari, H., Tunc, B., Doshi, J., Parker, D., Hsieh, M., Sotiras, A., Li, H., Ou, Y., Doot, R.K., Bilello, M., Fan, Y., Shinohara, R.T., Yushkevich, P., Verma, R., Kontos, D.: Cancer imaging phenomics toolkit: quantitative imaging analytics for precision diagnostics and predictive modeling of clinical outcome. Journal of Medical Imaging 5(1), 1–21 (2018). https://doi.org/10.1117/1.JMI.5.1.011018

  30. Pati, S., Singh, A., Rathore, S., Gastounioti, A., Bergman, M., Ngo, P., Ha, S.M., Bounias, D., Minock, J., Murphy, G., Li, H., Bhattarai, A., Wolf, A., Sridaran, P., Kalarot, R., Akbari, H., Sotiras, A., Thakur, S.P., Verma, R., Shinohara, R.T., Yushkevich, P., Fan, Y., Kontos, D., Davatzikos, C., Bakas, S.: The cancer imaging phenomics toolkit (captk): Technical overview. In A. Crimi and S. Bakas, editors, Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, pp. 380–394. 2020. Springer International Publishing, Cham (2020)

    Google Scholar 

  31. Rathore, S., M.D., H.A., Doshi, J., M.D., G.S., Rozycki, M., M.D., M.B., M.D., R.A.L., Davatzikos, C.A.: Radiomic signature of infiltration in peritumoral edema predicts subsequent recurrence in glioblastoma: implications for personalized radiotherapy planning. Journal of Medical Imaging 5(2), 1 – 10 (2018). https://doi.org/10.1117/1.JMI.5.2.021219

  32.  Vallières, M., Freeman, C.R., Skamene, S.R., Naqa. I.E.: A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities. Physics in Medicine and Biology, 60(14), 5471–5496 (2015). https://doi.org/10.1088/2F0031-9155/2F60/2F14/2F5471

  33. Vallières, M., Kay-Rivest, E., Perrin, L.J., Liem, X., Furstoss, C., Aerts, H.J.W.L., Khaouam, N., Nguyen-Tan, P.F., Wang, C.S., Sultanem, K., Seuntjens, J., El Naqa, I.: Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Scientific Reports 7(1), 10117 (2017). https://doi.org/10.1038/s41598-017-10371-5

    Article  Google Scholar 

  34. Lorensen, W.E., Cline, H.E.: Marching cubes: A high resolution 3d surface construction algorithm. In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’87, p. 163–169. Association for Computing Machinery, New York, NY, USA (1987). https://doi.org/10.1145/37401.37422

  35. T. S. Newman and H. Yi. A survey of the marching cubes algorithm. Computers & Graphics 30(5), 854 – 879 (2006). https://doi.org/10.1016/j.cag.2006.07.021, http://www.sciencedirect.com/science/article/pii/S0097849306001336

    Article  Google Scholar 

  36. Lewiner, T., Lopes, H., Vieira, A.W., Tavares, G.: Efficient implementation of marching cubes’ cases with topological guarantees. Journal of Graphics Tools, 8(2), 1–15 (2003). https://doi.org/10.1080/10867651.2003.10487582

  37. Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics, SMC-3(6), 610–621 (1973).

  38. Galloway. M.M.: Texture analysis using gray level run lengths. Computer Graphics and Image Processing 4(2):172 – 179 (1975). https://doi.org/10.1016/S0146-664X(75)80008-6, http://www.sciencedirect.com/science/article/pii/S0146664X75800086

  39. Thibault, G., FERTIL, B., Navarro, C., Pereira, S., L´evy, N., Sequeira, J., MARI, J.L.: Texture indexes and gray level size zone matrix application to cell nuclei classification (2009)

  40. Thibault, G., Angulo, J., Meyer, F.: Advanced statistical matrices for texture characterization: Application to cell classification. IEEE Transactions on Biomedical Engineering 61(3), 630–637, (2014)

    Article  Google Scholar 

  41. Amadasun, M., King, R.: Textural features corresponding to textural properties. IEEE Transactions on Systems, Man, and Cybernetics 19(5), 1264–1274 (1989)

    Article  Google Scholar 

  42. Sun, C., Wee, W.G.: Neighboring gray level dependence matrix for texture classification. Computer Vision, Graphics, and Image Processing 23(3), 341 – 352 (1983). https://doi.org/10.1016/0734-189X(83)90032-4, http://www.sciencedirect.com/science/article/pii/0734189X83900324

  43. Mohanaiah, P., Sathyanarayana, P., GuruKumar, L.: Image texture feature extraction using glcm approach. International Journal of Scientific and Research Publications 3(5) (2013)

  44. Humeau-Heurtier, A.: Texture feature extraction methods: A survey. IEEE Access 7,8975–9000 (2019)

    Article  Google Scholar 

  45. Gade A.A., Vyavahare. A.J.: Feature extraction using glcm for dietary assessment application. International Journal Multimedia and Image Processing (IJMIP) 8(2), 409–413 (2018)

    Article  Google Scholar 

  46. Chernikov, A.N., Xu, J.: A computer-assisted proof of correctness of a marching cubes algorithm. In: J. Sarrate, M. Staten (eds.) Proceedings of the 22nd International Meshing Roundtable, pp. 505–523. Springer International Publishing, Cham (2014)

  47. Delibasis, K., Matsopoulos, G., Mouravliansky, N., Nikita, K.: A novel and efficient implementation of the marching cubes algorithm. Computerized Medical Imaging and Graphics 25(4), 343–352 (2001). https://doi.org/10.1016/S0895-6111(00)00082-3, http://www.sciencedirect.com/science/article/pii/S0895611100000823

  48. Rajon, D., Bolch, W., Marching cube algorithm: review and trilinear interpolation adaptation for image-based dosimetric models. Computerized Medical Imaging and Graphics 27(5), 411 – 435 (2003). https://doi.org/10.1016/S0895-6111(03)00032-6, http://www.sciencedirect.com/science/article/pii/S0895611103000326

    Article  CAS  Google Scholar 

  49. Parmar, C., Leijenaar, R.T.H., Grossmann, P., Rios Velazquez, E., Bussink, J., Rietveld, D., Rietbergen, M.M., Haibe-Kains, B., Lambin, P., Aerts, H.J.: Radiomic feature clusters and prognostic signatures specific for lung and head & neck cancer. Scientific Reports 5(1), 11044 (2015). https://doi.org/10.1038/srep11044

  50. Cuocolo, R., Stanzione, A., Ponsiglione, A., Romeo, V., Verde, F., Creta, M., Rocca], R.L., Longo, N., Pace, L., Imbriaco, M.: Clinically significant prostate cancer detection on mri: A radiomic shape features study. European Journal of Radiology 116, 144–149 (2019). https://doi.org/10.1016/j.ejrad.2019.05.006, http://www.sciencedirect.com/science/article/pii/S0720048X19301664

  51. Zhu, Y., Li, H., Guo, W., Drukker, K., Lan, L., Giger, M.L., Ji, Y.: Deciphering genomic underpinnings of quantitative mri-based radiomic phenotypes of invasive breast carcinoma. Scientific Reports 5(1), 17787 (2015). https://doi.org/10.1038/srep17787

    Article  CAS  Google Scholar 

  52. Varn¨as, K., Halldin, C., Hall, H.: Autoradiographic distribution of serotonin transporters and receptor subtypes in human brain. Human Brain Mapping 22(3), 246–260 (2004). https://doi.org/10.1002/hbm.20035, https://onlinelibrary.wiley.com/doi/abs/10.1002/hbm.20035

  53. THIBAULT, G., FERTIL, B., NAVARRO, C., PEREIRA, S., CAU, P., LEVY, N., SEQUEIRA, J., MARI, J.L.: Shape and texture indexes application to cell nuclei classification. International Journal of Pattern Recognition and Artificial Intelligence 27(01), 1357002 (2013). https://doi.org/10.1142/S0218001413570024

  54. Wahl, R.L., Jacene, H., Kasamon, Y., Lodge, M.A.: From recist to percist: Evolving considerations for pet response criteria in solid tumors. Journal of nuclear medicine : official publication, Society of Nuclear Medicine 50 Suppl 1(Suppl 1), 122S–50S (2009). https://doi.org/10.2967/jnumed.108.057307

  55. Frings, V., van Velden, F.H.P., Velasquez, L.M., Hayes, W., van de Ven, P.M., Hoekstra, O.S., Boellaard, R.: Repeatability of metabolically active tumor volume measurements with fdg pet/ct in advanced gastrointestinal malignancies: A multicenter study. Radiology 273(2), 539–548 (2014). https://doi.org/10.1148/radiol.14132807. PMID: 24865311.

    Article  Google Scholar 

  56. Macyszyn, L., Akbari, H., Pisapia, J.M., Da, X., Attiah, M., Pigrish, V., Bi, Y., Pal, S., Davuluri, R.V., Roccograndi, L., Dahmane, N., Martinez-Lage, M., Biros, G., Wolf, R.L., Bilello, M., O’Rourke, D.M., Davatzikos, C.: Imaging patterns predict patient survival and molecular subtype in glioblastoma via machine learning techniques. Neuro-Oncology 18(3), 417–425 (2015). https://doi.org/10.1093/neuonc/nov127

  57. Naqa], I.E., Grigsby, P., Apte, A., Kidd, E., Donnelly, E., Khullar, D., Chaudhari, S., Yang, D., Schmitt, M., Laforest, R., Thorstad, W., Deasy, J.: Exploring featurebased approaches in pet images for predicting cancer treatment outcomes. Pattern Recognition 42(6), 1162 – 1171 (2009). https://doi.org/10.1016/j.patcog.2008.08.011, http://www.sciencedirect.com/science/article/pii/S0031320308003294. Digital Image Processing and Pattern Recognition Techniques for the Detection of Cancer

  58. Unser, M.: Sum and difference histograms for texture classification. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8(1), 118–125 (1986)

  59. Xu, D.H., Kurani, A., Furst, J., Raicu, D.: Run-length encoding for volumetric texture. The 4th IASTED International Conference on Visualization, Imaging, and Image Processing (2004)

  60. Chu, A., Sehgal, C., Greenleaf, J.: Use of gray value distribution of run lengths for texture analysis. Pattern Recognition Letters 11(6), 415–419 (1990). https://doi.org/10.1016/0167-8655(90)90112-F, http://www.sciencedirect.com/science/article/pii/016786559090112F

  61. Xiaoou Tang. Texture information in run-length matrices. IEEE Transactions on Image Processing 7(11), 1602–1609 (1998)

    Article  CAS  Google Scholar 

  62. Tustison, N., Gee, J.: Run-length matrices for texture analysis. The Insight Journal pp. 1–6 (2008)

Download references

Author information

Author notes

  1. Mingxi Lei and Bino Varghese contributed equally to this work.

Authors and Affiliations

  1. Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, 3740 McClintock Avenue, Los Angeles, CA, 90089, USA

    Mingxi Lei & Steven Cen

  2. Department of Radiology, University of Southern California, 1500 San Pablo Street, Los Angeles, CA, 90033, USA

    Bino Varghese, Darryl Hwang, Xiaomeng Lei, Bhushan Desai, Afshin Azadikhah & Vinay Duddalwar

  3. Department of Aerospace and Mechanical Engineering, University of Southern California, 854 Downey Way, Los Angeles, CA, 90089, USA

    Assad Oberai

Authors
  1. Mingxi Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bino Varghese
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Darryl Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Steven Cen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaomeng Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bhushan Desai
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Afshin Azadikhah
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Assad Oberai
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Vinay Duddalwar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingxi Lei.

Ethics declarations

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (XLSX 19.0 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, M., Varghese, B., Hwang, D. et al. Benchmarking Various Radiomic Toolkit Features While Applying the Image Biomarker Standardization Initiative toward Clinical Translation of Radiomic Analysis. J Digit Imaging 34, 1156–1170 (2021). https://doi.org/10.1007/s10278-021-00506-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-021-00506-6

Keywords

Search

Navigation