Advertisement

European Radiology

, Volume 30, Issue 1, pp 523–536 | Cite as

Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement

  • Ji Eun Park
  • Donghyun Kim
  • Ho Sung KimEmail author
  • Seo Young Park
  • Jung Youn Kim
  • Se Jin Cho
  • Jae Ho Shin
  • Jeong Hoon Kim
Imaging Informatics and Artificial Intelligence

Abstract

Objectives

To evaluate radiomics studies according to radiomics quality score (RQS) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) to provide objective measurement of radiomics research.

Materials and methods

PubMed and Embase were searched for studies published in high clinical imaging journals until December 2018 using the terms “radiomics” and “radiogenomics.” Studies were scored against the items in the RQS and TRIPOD guidelines. Subgroup analyses were performed for journal type (clinical vs. imaging), intended use (diagnostic vs. prognostic), and imaging modality (CT vs. MRI), and articles were compared using Fisher’s exact test and Mann-Whitney analysis.

Results

Seventy-seven articles were included. The mean RQS score was 26.1% of the maximum (9.4 out of 36). The RQS was low in demonstration of clinical utility (19.5%), test-retest analysis (6.5%), prospective study (3.9%), and open science (3.9%). None of the studies conducted a phantom or cost-effectiveness analysis. The adherence rate for TRIPOD was 57.8% (mean) and was particularly low in reporting title (2.6%), stating study objective in abstract and introduction (7.8% and 16.9%), blind assessment of outcome (14.3%), sample size (6.5%), and missing data (11.7%) categories. Studies in clinical journals scored higher and more frequently adopted external validation than imaging journals.

Conclusions

The overall scientific quality and reporting of radiomics studies is insufficient. Scientific improvements need to be made to feature reproducibility, analysis of clinical utility, and open science categories. Reporting of study objectives, blind assessment, sample size, and missing data is deemed to be necessary.

Key Points

• The overall scientific quality and reporting of radiomics studies is insufficient.

• The RQS was low in demonstration of clinical utility, test-retest analysis, prospective study, and open science.

• Room for improvement was shown in TRIPOD in stating study objective in abstract and introduction, blind assessment of outcome, sample size, and missing data categories.

Keywords

Neoplasm Machine learning Quality improvement Computed tomography Magnetic resonance imaging 

Abbreviations

RQS

Radiomics quality score,

TRIPOD

Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis

Notes

Funding

This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (grant number: NRF-2017R1A2A2A05001217 and grant number: NRF-2017R1C1B2007258).

Compliance with ethical standards

Guarantor

The scientific guarantor of this publication is Jeong Hoon Kim.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors has significant statistical expertise (Seo Young Park, 8 years of experience).

Informed consent

Written informed consent was not required because of the nature of our study, which was a study based on research articles.

Ethical approval

Institutional Review Board approval was not required because of the nature of our study, which was a study based on research articles.

Methodology

• retrospective

• cross-sectional study

• performed at one institution

Supplementary material

330_2019_6360_MOESM1_ESM.docx (260 kb)
ESM 1 (DOCX 260 kb)

References

  1. 1.
    Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: images are more than pictures, they are data. Radiology 278:563–577PubMedPubMedCentralGoogle Scholar
  2. 2.
    Lambin P, Leijenaar RTH, Deist TM et al (2017) Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14:749–762Google Scholar
  3. 3.
    Sanduleanu S, Woodruff HC, de Jong EEC et al (2018) Tracking tumor biology with radiomics: a systematic review utilizing a radiomics quality score. Radiother Oncol 127:349–360PubMedGoogle Scholar
  4. 4.
    O'Connor JP, Aboagye EO, Adams JE et al (2017) Imaging biomarker roadmap for cancer studies. Nat Rev Clin Oncol 14:169–186PubMedGoogle Scholar
  5. 5.
    Sung NS, Crowley WF Jr, Genel M et al (2003) Central challenges facing the national clinical research enterprise. JAMA 289:1278–1287PubMedGoogle Scholar
  6. 6.
    Choi YJ, Chung MS, Koo HJ, Park JE, Yoon HM, Park SH (2016) Does the reporting quality of diagnostic test accuracy studies, as defined by STARD 2015, affect citation? Korean J Radiol 17:706–714PubMedPubMedCentralGoogle Scholar
  7. 7.
    Waterton JC, Pylkkanen L (2012) Qualification of imaging biomarkers for oncology drug development. Eur J Cancer 48:409–415PubMedGoogle Scholar
  8. 8.
    Moons KG, Altman DG, Reitsma JB et al (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 162:W1–W73Google Scholar
  9. 9.
    Heus P, Damen JAAG, Pajouheshnia R et al (2018) Poor reporting of multivariable prediction model studies: towards a targeted implementation strategy of the TRIPOD statement. BMC Med 16:120PubMedPubMedCentralGoogle Scholar
  10. 10.
    Whiting PF, Rutjes AW, Westwood ME et al (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155:529–536Google Scholar
  11. 11.
    Aerts HJ, Velazquez ER, Leijenaar RT et al (2014) Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 5:4006PubMedPubMedCentralGoogle Scholar
  12. 12.
    Hawkins S, Wang H, Liu Y et al (2016) Predicting malignant nodules from screening CT scans. J Thorac Oncol 11:2120–2128PubMedPubMedCentralGoogle Scholar
  13. 13.
    Huang Y, Liu Z, He L et al (2016) Radiomics signature: a potential biomarker for the prediction of disease-free survival in early-stage (I or II) non-small cell lung cancer. Radiology 281:947–957PubMedGoogle Scholar
  14. 14.
    Huang YQ, Liang CH, He L et al (2016) Development and validation of a radiomics nomogram for preoperative prediction of lymph node metastasis in colorectal cancer. J Clin Oncol 34:2157–2164Google Scholar
  15. 15.
    Kickingereder P, Bonekamp D, Nowosielski M et al (2016) Radiogenomics of glioblastoma: machine learning-based classification of molecular characteristics by using multiparametric and multiregional MR imaging features. Radiology 281:907–918PubMedGoogle Scholar
  16. 16.
    Kickingereder P, Burth S, Wick A et al (2016) Radiomic profiling of glioblastoma: identifying an imaging predictor of patient survival with improved performance over established clinical and radiologic risk models. Radiology 280:880–889Google Scholar
  17. 17.
    Kickingereder P, Gotz M, Muschelli J et al (2016) Large-scale Radiomic profiling of recurrent glioblastoma identifies an imaging predictor for stratifying anti-angiogenic treatment response. Clin Cancer Res 22:5765–5771PubMedPubMedCentralGoogle Scholar
  18. 18.
    Li H, Zhu Y, Burnside ES et al (2016) MR imaging radiomics signatures for predicting the risk of breast cancer recurrence as given by research versions of MammaPrint, oncotype DX, and PAM50 gene assays. Radiology 281:382–391PubMedPubMedCentralGoogle Scholar
  19. 19.
    Nie K, Shi L, Chen Q et al (2016) Rectal cancer: assessment of neoadjuvant chemoradiation outcome based on radiomics of multiparametric MRI. Clin Cancer Res 22:5256–5264PubMedGoogle Scholar
  20. 20.
    Coroller TP, Agrawal V, Huynh E et al (2017) Radiomic-based pathological response prediction from primary tumors and lymph nodes in NSCLC. J Thorac Oncol 12:467–476PubMedGoogle Scholar
  21. 21.
    Grossmann P, Narayan V, Chang K et al (2017) Quantitative imaging biomarkers for risk stratification of patients with recurrent glioblastoma treated with bevacizumab. Neuro Oncol 19:1688–1697PubMedPubMedCentralGoogle Scholar
  22. 22.
    Hu LS, Ning S, Eschbacher JM et al (2017) Radiogenomics to characterize regional genetic heterogeneity in glioblastoma. Neuro Oncol 19:128–137PubMedGoogle Scholar
  23. 23.
    Liu TT, Achrol AS, Mitchell LA et al (2017) Magnetic resonance perfusion image features uncover an angiogenic subgroup of glioblastoma patients with poor survival and better response to antiangiogenic treatment. Neuro Oncol 19:997–1007PubMedGoogle Scholar
  24. 24.
    Liu Z, Zhang XY, Shi YJ et al (2017) Radiomics analysis for evaluation of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Clin Cancer Res 23:7253–7262PubMedPubMedCentralGoogle Scholar
  25. 25.
    Lohmann P, Stoffels G, Ceccon G et al (2017) Radiation injury vs. recurrent brain metastasis: combining textural feature radiomics analysis and standard parameters may increase (18)F-FET PET accuracy without dynamic scans. Eur Radiol 27:2916–2927Google Scholar
  26. 26.
    Rios Velazquez E, Parmar C, Liu Y et al (2017) Somatic mutations drive distinct imaging phenotypes in lung cancer. Cancer Res 77:3922–3930PubMedGoogle Scholar
  27. 27.
    Song SH, Park H, Lee G et al (2017) Imaging phenotyping using Radiomics to predict micropapillary pattern within lung adenocarcinoma. J Thorac Oncol 12:624–632PubMedGoogle Scholar
  28. 28.
    Wang J, Wu CJ, Bao ML, Zhang J, Wang XN, Zhang YD (2017) Machine learning-based analysis of MR radiomics can help to improve the diagnostic performance of PI-RADS v2 in clinically relevant prostate cancer. Eur Radiol 27:4082–4090Google Scholar
  29. 29.
    Wu S, Zheng J, Li Y et al (2017) A radiomics nomogram for the preoperative prediction of lymph node metastasis in bladder cancer. Clin Cancer Res 23:6904–6911PubMedPubMedCentralGoogle Scholar
  30. 30.
    Yu J, Shi Z, Lian Y et al (2017) Noninvasive IDH1 mutation estimation based on a quantitative radiomics approach for grade II glioma. Eur Radiol 27:3509–3522PubMedGoogle Scholar
  31. 31.
    Yuan M, Zhang YD, Pu XH et al (2017) Comparison of a radiomic biomarker with volumetric analysis for decoding tumour phenotypes of lung adenocarcinoma with different disease-specific survival. Eur Radiol 27:4857–4865Google Scholar
  32. 32.
    Zhang B, Tian J, Dong D et al (2017) Radiomics features of multiparametric MRI as novel prognostic factors in advanced nasopharyngeal carcinoma. Clin Cancer Res 23:4259–4269PubMedGoogle Scholar
  33. 33.
    Zhou H, Vallieres M, Bai HX et al (2017) MRI features predict survival and molecular markers in diffuse lower-grade gliomas. Neuro Oncol 19:862–870PubMedPubMedCentralGoogle Scholar
  34. 34.
    Akbari H, Bakas S, Pisapia JM et al (2018) In vivo evaluation of EGFRvIII mutation in primary glioblastoma patients via complex multiparametric MRI signature. Neuro Oncol 20:1068–1079PubMedPubMedCentralGoogle Scholar
  35. 35.
    Bae S, Choi YS, Ahn SS et al (2018) Radiomic MRI phenotyping of glioblastoma: improving survival prediction. Radiology 289:797–806PubMedGoogle Scholar
  36. 36.
    Beukinga RJ, Hulshoff JB, Mul VEM et al (2018) Prediction of response to neoadjuvant chemotherapy and radiation therapy with baseline and restaging (18)F-FDG PET imaging biomarkers in patients with esophageal cancer. Radiology 287:983–992PubMedGoogle Scholar
  37. 37.
    Bickelhaupt S, Jaeger PF, Laun FB et al (2018) Radiomics based on adapted diffusion kurtosis imaging helps to clarify most mammographic findings suspicious for cancer. Radiology 287:761–770PubMedGoogle Scholar
  38. 38.
    Chen T, Ning Z, Xu L et al (2018) Radiomics nomogram for predicting the malignant potential of gastrointestinal stromal tumours preoperatively. Eur Radiol 29:1074–1082PubMedGoogle Scholar
  39. 39.
    Chen Y, Chen TW, Wu CQ et al (2018) Radiomics model of contrast-enhanced computed tomography for predicting the recurrence of acute pancreatitis. Eur Radiol 29:4408–4417Google Scholar
  40. 40.
    Cui Y, Yang X, Shi Z et al (2018) Radiomics analysis of multiparametric MRI for prediction of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Eur Radiol 29:1211–1220PubMedGoogle Scholar
  41. 41.
    Dong F, Li Q, Xu D et al (2018) Differentiation between pilocytic astrocytoma and glioblastoma: a decision tree model using contrast-enhanced magnetic resonance imaging-derived quantitative radiomic features. Eur Radiol 29:3968–3975Google Scholar
  42. 42.
    Dong Y, Feng Q, Yang W et al (2018) Preoperative prediction of sentinel lymph node metastasis in breast cancer based on radiomics of T2-weighted fat-suppression and diffusion-weighted MRI. Eur Radiol 28:582–591PubMedGoogle Scholar
  43. 43.
    Guo J, Liu Z, Shen C et al (2018) MR-based radiomics signature in differentiating ocular adnexal lymphoma from idiopathic orbital inflammation. Eur Radiol 28:3872–3881PubMedGoogle Scholar
  44. 44.
    Horvat N, Veeraraghavan H, Khan M et al (2018) MR imaging of rectal cancer: radiomics analysis to assess treatment response after neoadjuvant therapy. Radiology 287:833–843PubMedPubMedCentralGoogle Scholar
  45. 45.
    Hu HT, Wang Z, Huang XW et al (2018) Ultrasound-based radiomics score: a potential biomarker for the prediction of microvascular invasion in hepatocellular carcinoma. Eur Radiol 29:2890–2901PubMedGoogle Scholar
  46. 46.
    Kang D, Park JE, Kim YH et al (2018) Diffusion radiomics as a diagnostic model for atypical manifestation of primary central nervous system lymphoma: development and multicenter external validation. Neuro Oncol 20:1251–1261PubMedPubMedCentralGoogle Scholar
  47. 47.
    Kickingereder P, Neuberger U, Bonekamp D et al (2018) Radiomic subtyping improves disease stratification beyond key molecular, clinical, and standard imaging characteristics in patients with glioblastoma. Neuro Oncol 20:848–857PubMedGoogle Scholar
  48. 48.
    Kim JY, Park JE, Jo Y et al (2018) Incorporating diffusion- and perfusion-weighted MRI into a radiomics model improves diagnostic performance for pseudoprogression in glioblastoma patients. Neuro Oncol 21:404–414Google Scholar
  49. 49.
    Kniep HC, Madesta F, Schneider T et al (2018) Radiomics of brain MRI: utility in prediction of metastatic tumor type. Radiology 180946Google Scholar
  50. 50.
    Multiparametric ultrasomics of significant liver fibrosis: a machine learning-based analysis. Eur Radiol 29:1496–1506Google Scholar
  51. 51.
    Li Y, Liu X, Qian Z et al (2018) Genotype prediction of ATRX mutation in lower-grade gliomas using an MRI radiomics signature. Eur Radiol 28:2960–2968PubMedGoogle Scholar
  52. 52.
    Li Y, Liu X, Xu K et al (2018) MRI features can predict EGFR expression in lower grade gliomas: a voxel-based radiomic analysis. Eur Radiol 28:356–362PubMedGoogle Scholar
  53. 53.
    Li ZC, Bai H, Sun Q et al (2018) Multiregional radiomics features from multiparametric MRI for prediction of MGMT methylation status in glioblastoma multiforme: a multicentre study. Eur Radiol 28:3640–3650Google Scholar
  54. 54.
    Liang W, Yang P, Huang R et al (2018) A combined nomogram model to preoperatively predict histologic grade in pancreatic neuroendocrine tumors. Clin Cancer Res 25:584–594Google Scholar
  55. 55.
    Liu H, Zhang C, Wang L et al (2018) MRI radiomics analysis for predicting preoperative synchronous distant metastasis in patients with rectal cancer. Eur Radiol 29:4418–4426PubMedGoogle Scholar
  56. 56.
    Lu CF, Hsu FT, Hsieh KL et al (2018) Machine learning-based radiomics for molecular subtyping of gliomas. Clin Cancer Res 24:4429–4436Google Scholar
  57. 57.
    Lv W, Yuan Q, Wang Q et al (2018) Robustness versus disease differentiation when varying parameter settings in radiomics features: application to nasopharyngeal PET/CT. Eur Radiol 28:3245–3254PubMedGoogle Scholar
  58. 58.
    Meng X, Xia W, Xie P et al (2018) Preoperative radiomic signature based on multiparametric magnetic resonance imaging for noninvasive evaluation of biological characteristics in rectal cancer. Eur Radiol 29:3200–3209PubMedGoogle Scholar
  59. 59.
    Naganawa S, Enooku K, Tateishi R et al (2018) Imaging prediction of nonalcoholic steatohepatitis using computed tomography texture analysis. Eur Radiol 28:3050–3058PubMedGoogle Scholar
  60. 60.
    Niu J, Zhang S, Ma S et al (2018) Preoperative prediction of cavernous sinus invasion by pituitary adenomas using a radiomics method based on magnetic resonance images. Eur Radiol 29:1625–1634PubMedPubMedCentralGoogle Scholar
  61. 61.
    Ortiz-Ramón R, Larroza A, Ruiz-España S, Arana E, Moratal D (2018) Classifying brain metastases by their primary site of origin using a radiomics approach based on texture analysis: a feasibility study. Eur Radiol 28:4514–4523PubMedGoogle Scholar
  62. 62.
    Park YW, Oh J, You SC et al (2018) Radiomics and machine learning may accurately predict the grade and histological subtype in meningiomas using conventional and diffusion tensor imaging. Eur Radiol 29:4068–4076PubMedGoogle Scholar
  63. 63.
    She Y, Zhang L, Zhu H et al (2018) The predictive value of CT-based radiomics in differentiating indolent from invasive lung adenocarcinoma in patients with pulmonary nodules. Eur Radiol 28:5121–5128PubMedGoogle Scholar
  64. 64.
    Shi Z, Zhu C, Degnan AJ et al (2018) Identification of high-risk plaque features in intracranial atherosclerosis: initial experience using a radiomic approach. Eur Radiol 28:3912–3921PubMedPubMedCentralGoogle Scholar
  65. 65.
    Su C, Jiang J, Zhang S et al (2018) Radiomics based on multicontrast MRI can precisely differentiate among glioma subtypes and predict tumour-proliferative behaviour. Eur Radiol 29:1986–1996PubMedGoogle Scholar
  66. 66.
    Suh HB, Choi YS, Bae S et al (2018) Primary central nervous system lymphoma and atypical glioblastoma: differentiation using radiomics approach. Eur Radiol 28:3832–3839PubMedGoogle Scholar
  67. 67.
    Sun H, Chen Y, Huang Q et al (2018) Psychoradiologic utility of MR imaging for diagnosis of attention deficit hyperactivity disorder: a radiomics analysis. Radiology 287:620–630PubMedGoogle Scholar
  68. 68.
    Truhn D, Schrading S, Haarburger C, Schneider H, Merhof D, Kuhl C (2018) Radiomic versus convolutional neural networks analysis for classification of contrast-enhancing lesions at multiparametric breast MRI. Radiology 181352Google Scholar
  69. 69.
    Wu M, Tan H, Gao F et al (2018) Predicting the grade of hepatocellular carcinoma based on non-contrastenhanced MRI radiomics signature. Eur Radiol 29:2802–2811PubMedGoogle Scholar
  70. 70.
    Yang L, Dong D, Fang M et al (2018) Can CT-based radiomics signature predict KRAS/NRAS/BRAF mutations in colorectal cancer? Eur Radiol 28:2058–2067PubMedGoogle Scholar
  71. 71.
    Yin P, Mao N, Zhao C et al (2018) Comparison of radiomics machine-learning classifiers and feature selection for differentiation of sacral chordoma and sacral giant cell tumour based on 3D computed tomography features. Eur Radiol 29:1841–1847PubMedGoogle Scholar
  72. 72.
    Zhang S, Song G, Zang Y et al (2018) Non-invasive radiomics approach potentially predicts non-functioning pituitary adenomas subtypes before surgery. Eur Radiol 28:3692–3701PubMedGoogle Scholar
  73. 73.
    Zhang Y, Zhang B, Liang F et al (2018) Radiomics features on noncontrast-enhanced CT scan can precisely classify AVM-related hematomas from other spontaneous intraparenchymal hematoma types. Eur Radiol 29:2157–2165Google Scholar
  74. 74.
    Zhang Z, Yang J, Ho A et al (2018) A predictive model for distinguishing radiation necrosis from tumour progression after gamma knife radiosurgery based on radiomic features from MR images. Eur Radiol 28:2255–2263Google Scholar
  75. 75.
    Zhu X, Dong D, Chen Z et al (2018) Radiomic signature as a diagnostic factor for histologic subtype classification of non-small cell lung cancer. Eur Radiol 28:2772–2778PubMedGoogle Scholar
  76. 76.
    Zinn PO, Singh SK, Kotrotsou A et al (2018) A coclinical radiogenomic validation study: conserved magnetic resonance radiomic appearance of periostin-expressing glioblastoma in patients and xenograft models. Clin Cancer Res 24:6288–6299Google Scholar
  77. 77.
    Choe J, Lee SM, Do KH et al (2019) Prognostic value of radiomic analysis of iodine overlay maps from dual-energy computed tomography in patients with resectable lung cancer. Eur Radiol 29:915–923Google Scholar
  78. 78.
    Hu T, Wang S, Huang L et al (2019) A clinical-radiomics nomogram for the preoperative prediction of lung metastasis in colorectal cancer patients with indeterminate pulmonary nodules. Eur Radiol 29:439–449Google Scholar
  79. 79.
    Ji GW, Zhang YD, Zhang H et al (2019) Biliary tract cancer at CT: a radiomics-based model to predict lymph node metastasis and survival outcomes. Radiology 290:90–98Google Scholar
  80. 80.
    Kontos D, Winham SJ, Oustimov A et al (2019) Radiomic phenotypes of mammographic parenchymal complexity: toward augmenting breast density in breast cancer risk assessment. Radiology 290:41–49Google Scholar
  81. 81.
    Qu J, Shen C, Qin J et al (2019) The MR radiomic signature can predict preoperative lymph node metastasis in patients with esophageal cancer. Eur Radiol 29:906–914Google Scholar
  82. 82.
    Tan X, Ma Z, Yan L, Ye W, Liu Z, Liang C (2019) Radiomics nomogram outperforms size criteria in discriminating lymph node metastasis in resectable esophageal squamous cell carcinoma. Eur Radiol 29:392–400Google Scholar
  83. 83.
    Wei J, Yang G, Hao X et al (2019) A multi-sequence and habitat-based MRI radiomics signature for preoperative prediction of MGMT promoter methylation in astrocytomas with prognostic implication. Eur Radiol 29:877–888Google Scholar
  84. 84.
    Bonekamp D, Kohl S, Wiesenfarth M et al (2018) Radiomic machine learning for characterization of prostate lesions with MRI: comparison to ADC values. Radiology 289:128–137Google Scholar
  85. 85.
    Park H, Lim Y, Ko ES et al (2018) Radiomics signature on magnetic resonance imaging: association with disease-free survival in patients with invasive breast cancer. Clin Cancer Res 24:4705–4714Google Scholar
  86. 86.
    Sun R, Limkin EJ, Vakalopoulou M et al (2018) A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: an imaging biomarker, retrospective multicohort study. Lancet Oncol 19:1180–1191PubMedGoogle Scholar
  87. 87.
    Wang K, Lu X, Zhou H et al (2018) Deep learning radiomics of shear wave elastography significantly improved diagnostic performance for assessing liver fibrosis in chronic hepatitis B: a prospective multicentre study. Gut 68:729–741PubMedPubMedCentralGoogle Scholar
  88. 88.
    Kessler LG, Barnhart HX, Buckler AJ et al (2015) The emerging science of quantitative imaging biomarkers terminology and definitions for scientific studies and regulatory submissions. Stat Methods Med Res 24:9–26PubMedGoogle Scholar
  89. 89.
    McShane LM, Altman DG, Sauerbrei W et al (2005) Reporting recommendations for tumor marker prognostic studies (REMARK). J Natl Cancer Inst 97:1180–1184PubMedGoogle Scholar
  90. 90.
    Korevaar DA, van Enst WA, Spijker R, Bossuyt PM, Hooft L (2014) Reporting quality of diagnostic accuracy studies: a systematic review and meta-analysis of investigations on adherence to STARD. Evid Based Med 19:47–54PubMedGoogle Scholar
  91. 91.
    Korevaar DA, Wang J, van Enst WA et al (2015) Reporting diagnostic accuracy studies: some improvements after 10 years of STARD. Radiology 274:781–789PubMedGoogle Scholar

Copyright information

© European Society of Radiology 2019

Authors and Affiliations

  1. 1.Department of Radiology and Research Institute of Radiology, Asan Medical CenterUniversity of Ulsan College of MedicineSeoulSouth Korea
  2. 2.Department of RadiologyInje University Busan Paik HospitalBusanSouth Korea
  3. 3.Department of Clinical Epidemiology and Biostatistics, Asan Medical CenterUniversity of Ulsan College of MedicineSeoulSouth Korea
  4. 4.Department of RadiologyKangbuk Samsung Medical CenterSeoulSouth Korea
  5. 5.St. Vincent Hospital, College of MedicineThe Catholic University of KoreaSuwonSouth Korea
  6. 6.Department of Neurosurgery, Asan Medical CenterUniversity of Ulsan College of MedicineSeoulSouth Korea

Personalised recommendations