Skip to main content
SpringerLink
Log in

A systematic review of radiomics in osteosarcoma: utilizing radiomics quality score as a tool promoting clinical translation

  • Musculoskeletal
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

To assess the methodological quality and risk of bias in radiomics studies investigating diagnosis, therapy response, and survival of patients with osteosarcoma.

Methods

In this systematic review, literatures on radiomics in osteosarcoma were included and assessed for methodological quality through the radiomics quality score (RQS). The risk of bias and concern of application was assessed using the Quality Assessment of Diagnostic Accuracy Studies tool. A meta-analysis of studies focusing on predicting osteosarcoma response to neoadjuvant chemotherapy was performed.

Results

Twelve radiomics studies exploring osteosarcoma were identified, and five were included in meta-analysis. The RQS reached an average of 20.4% (6.92 of 36) with good inter-rater agreement (ICC 0.95, 95% CI 0.85-0.99). Four studies validated results with an internal dataset, none of which used external dataset; one study was prospectively designed, and another one shared part of the dataset. The risk of bias and concern of application were mainly related to index test aspect. The meta-analysis showed a diagnostic odds ratio of 43.68 (95%CI 13.5-141.31) for predicting response to neoadjuvant chemotherapy with high heterogeneity and low methodological quality.

Conclusions

The overall scientific quality of included studies is insufficient; however, radiomics remains a promising technology for predicting treatment response, which might guide therapeutic decision-making and related to prognosis. Improvements in study design, validation, and open science needs to be made to demonstrate the generalizability of findings and to achieve clinical applications. Widespread application of RQS, pre-trained RQS scoring procedure, and modification of RQS in response to clinical needs are necessary.

Key Points

• Limited radiomics studies were established in osteosarcoma with mean RQS of 20.4%, commonly due to unvalidated results, retrospective study design, and absence of open science.

• Meta-analysis of radiomics studies predicting osteosarcoma response to neoadjuvant chemotherapy showed high diagnostic odds ratio 43.68, while high heterogeneity and low methodological quality were the main concerns.

• A previously trained data extraction instrument allowed reaching moderate inter-rater agreement in RQS applications, while RQS still needs improvement to become a wide adaptive tool in reviews of radiomics studies, in routine self-check before manuscript submitting and in study design.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

CI:

Confidence intervals

DOR:

Diagnostic odds ratio

HSROC:

Hierarchical summary receiver operating characteristic

ICC:

Correlation coefficient

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-analysis

PROSPERO:

International Prospective Register Of Systematic Reviews

QUADAS:

Quality Assessment of Diagnostic Accuracy Studies

RQS:

Radiomics quality score

TRIPOD:

Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis

References

  1. Fletcher CDM, Bridge JA, Hogendoorn PCW, Mertens F (2013) World Health Organization classification of tumors: WHO classification of tumours of soft tissue and bone, 4th edn. IARC Press, Lyon

    Google Scholar 

  2. Whelan JS, Davis LE (2018) Osteosarcoma, chondrosarcoma, and chordoma. J Clin Oncol 36:188–193

    CAS  PubMed  Google Scholar 

  3. Casali PG, Bielack S, Abecassis N et al (2018) Bone sarcomas: ESMO-PaedCan-EURACAN clinical practice guidelines for diagnosis, treatment, and follow-up. Ann Oncol 29:iv79–iv95

    CAS  PubMed  Google Scholar 

  4. National Comprehensive Cancer Network (2019) NCCN clinical practice guidelines in oncology: Bone Cancer, v1.2020. Available via https://www.nccn.org/professionals/physician_gls/pdf/bone.pdf. Accessed Apr 2020

  5. Link MP, Goorin AM, Miser AW et al (1986) The effect of adjuvant chemotherapy on relapse-free survival in patients with osteosarcoma of the extremity. N Engl J Med 314:1600–1606

    CAS  PubMed  Google Scholar 

  6. Rosen G, Murphy ML, Huvos AG, Gutierrez M, Marcove RC (1976) Chemotherapy, en bloc resection, and prosthetic bone replacement in the treatment of osteogenic sarcoma. Cancer 37:1–11

    CAS  PubMed  Google Scholar 

  7. Rosen G, Caparros B, Huvos AG et al (1982) Preoperative chemotherapy for osteogenic sarcoma: selection of postoperative adjuvant chemotherapy based on the response of the primary tumor to preoperative chemotherapy. Cancer 49:1221–1230

    CAS  PubMed  Google Scholar 

  8. Coffin CM, Lowichik A, Zhou H (2005) Treatment effects in pediatric soft tissue and bone tumors: practical considerations for the pathologist. Am J Clin Pathol 123:75–90

    PubMed  Google Scholar 

  9. Lambin P, Leijenaar RTH, Deist TM et al (2017) Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14:749–762

    PubMed  Google Scholar 

  10. Castellano G, Bonilha L, Li LM, Cendes F (2004) Texture analysis of medical images. Clin Radiol 59:1061–1069

    CAS  PubMed  Google Scholar 

  11. Lambin P, Rios-Velazquez E, Leijenaar R et al (2012) Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446

    PubMed  PubMed Central  Google Scholar 

  12. Gillies RJ, Kinahan PE, Hricak H (2016) Radiomics: images are more than pictures, they are data. Radiology 278:563–577

    PubMed  Google Scholar 

  13. Sullivan DC, Obuchowski NA, Kessler LG et al (2015) Metrology standards for quantitative imaging biomarkers. Radiology 277:813–825

    PubMed  PubMed Central  Google Scholar 

  14. Bi WL, Hosny A, Schabath MB et al (2019) Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J Clin 69:127–157

    PubMed  PubMed Central  Google Scholar 

  15. O’Connor JP, Aboagye EO, Adams JE et al (2017) Imaging biomarker roadmap for cancer studies. Nat Rev Clin Oncol 14:169–186

    PubMed  Google Scholar 

  16. Rogers W, Thulasi Seetha S, Refaee TAG et al (2020) Radiomics: from qualitative to quantitative imaging. Br J Radiol 93:20190948

    PubMed  Google Scholar 

  17. McInnes MDF, Moher D, Thombs BD et al (2018) Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA 319:388–396

    Google Scholar 

  18. Moher D, Shamseer L, Clarke M et al (2015) Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 4:1

    PubMed  PubMed Central  Google Scholar 

  19. Whiting PF, Rutjes AW, Westwood ME et al (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155:529–536

    PubMed  Google Scholar 

  20. Sanduleanu S, Woodruff HC, de Jong EEC et al (2018) Tracking tumor biology with radiomics: a systematic review utilizing a radiomics quality score. Radiother Oncol 127:349–360

    PubMed  Google Scholar 

  21. Ursprung S, Beer L, Bruining A et al (2020) Radiomics of computed tomography and magnetic resonance imaging in renal cell carcinoma-a systematic review and meta-analysis. Eur Radiol 30(6):3558–3566

  22. Cochrane methods screening and diagnostic tests (2017) Handbook for DTA Reviews. Available via https://methods.cochrane.org/sdt/handbook-dta-reviews. Accessed 10 Apr 2020

  23. Bailly C, Leforestier R, Campion L et al (2017) Prognostic value of FDG-PET indices for the assessment of histological response to neoadjuvant chemotherapy and outcome in pediatric patients with Ewing sarcoma and osteosarcoma. PLoS One 12:e0183841

    PubMed  PubMed Central  Google Scholar 

  24. Cho YJ, Kim WS, Choi YH et al (2019) Computerized texture analysis of pulmonary nodules in pediatric patients with osteosarcoma: differentiation of pulmonary metastases from non-metastatic nodules. PLoS One 14:e0211969

    CAS  PubMed  PubMed Central  Google Scholar 

  25. Dufau J, Bouhamama A, Leporq B et al (2019) Prediction of chemotherapy response in primary osteosarcoma using the machine learning technique on radiomic data. Bull Cancer 106:983–999

  26. Jeong SY, Kim W, Byun BH et al (2019) Prediction of chemotherapy response of osteosarcoma using baseline 18-F-FDG textural features machine learning approaches with PCA. Contrast Media Mol Imaging 2019:3515080

    PubMed  PubMed Central  Google Scholar 

  27. Kayal EB, Kandasamy D, Khare K, Bakhshi S, Sharma R, Mehndiratta A (2019) Intravoxel incoherent motion (IVIM) for response assessment in patients with osteosarcoma undergoing neoadjuvant chemotherapy. Eur J Radiol 119:108635

    Google Scholar 

  28. Lee SK, Jee WH, Jung CK, Im SA, Chung NG, Chung YG (2020) Prediction of poor responders to neoadjuvant chemotherapy in patients with osteosarcoma: additive value of diffusion-weighted MRI including volumetric analysis to standard MRI at 3T. PLoS One 15:e0229983

    CAS  PubMed  PubMed Central  Google Scholar 

  29. Lin P, Yang PF, Chen S et al (2020) A Delta-radiomics model for preoperative evaluation of neoadjuvant chemotherapy response in high-grade osteosarcoma. Cancer Imaging 20:7

    PubMed  PubMed Central  Google Scholar 

  30. Sheen H, Kim W, Byun BH et al (2019) Metastasis risk prediction model in osteosarcoma using metabolic imaging phenotypes: a multivariable radiomics model. PLoS One 14:e0225242

    CAS  PubMed  PubMed Central  Google Scholar 

  31. Song H, Jiao Y, Wei W et al (2019) Can pretreatment 18-F-FDG PET tumor texture features predict the outcomes of osteosarcoma treated by neoadjuvant chemotherapy? Eur Radiol 29:3945–3954

    PubMed  Google Scholar 

  32. Wu Y, Xu L, Yang P et al (2018) Survival prediction in high-grade osteosarcoma using radiomics of diagnostic computed tomography. EBioMedicine 34:27–34

    PubMed  PubMed Central  Google Scholar 

  33. Xu L, Yang P, Yen EA et al (2019) A multi-organ cancer study of the classification performance using 2D and 3D image features in radiomics analysis. Phys Med Biol 64:215009

    PubMed  Google Scholar 

  34. Zhao S, Su Y, Duan J et al (2019) Radiomics signature extracted from diffusion-weighted magnetic resonance imaging predicts outcomes in osteosarcoma. J Bone Oncol 19:100263

    PubMed  PubMed Central  Google Scholar 

  35. Valdora F, Houssami N, Rossi F, Calabrese M, Tagliafico AS (2018) Rapid review: radiomics and breast cancer. Breast Cancer Res Treat 169(2):217–229

    PubMed  Google Scholar 

  36. Granzier RWY, van Nijnatten TJA, Woodruff HC, Smidt ML, Lobbes MBI (2019) Exploring breast cancer response prediction to neoadjuvant systemic therapy using MRI-based radiomics: a systematic review. Eur J Radiol 121:108736

    CAS  PubMed  Google Scholar 

  37. Wakabayashi T, Ouhmich F, Gonzalez-Cabrera C et al (2019) Radiomics in hepatocellular carcinoma: a quantitative review. Hepatol Int 13(5):546–559

    PubMed  Google Scholar 

  38. Park JE, Kim HS, Kim D et al (2020) A systematic review reporting quality of radiomics research in neuro-oncology: toward clinical utility and quality improvement using high-dimensional imaging features. BMC Cancer 20(1):29

    CAS  PubMed  PubMed Central  Google Scholar 

  39. Park JE, Kim D, Kim HS et al (2020) Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement. Eur Radiol 30(1):523–536

    PubMed  Google Scholar 

  40. Wang H, Zhou Y, Li L, Hou W, Ma X, Tian R (2020) Current status and quality of radiomics studies in lymphoma: a systematic review. Eur Radiol. https://doi.org/10.1007/s00330-020-06927-1

  41. Stanzione A, Gambardella M, Cuocolo R, Ponsiglione A, Romeo V, Imbriaco M (2020) Prostate MRI radiomics: a systematic review and radiomic quality score assessment. Eur J Radiol 129:109095

  42. Fornacon-Wood I, Faivre-Finn C, O'Connor JPB, Price GJ (2020) Radiomics as a personalized medicine tool in lung cancer: separating the hope from the hype. Lung Cancer 146:197–208

  43. Castillo Tovar JM, Arif M, Niessen WJ, Schoots IG, Veenland JF (2020) Automated classification of significant prostate cancer on MRI: a systematic review on the performance of machine learning applications. Cancers (Basel) 12(6):E1606

  44. Chetan MR, Gleeson FV (2020) Radiomics in predicting treatment response in non-small-cell lung cancer: current status, challenges and future perspectives. Eur Radiol. https://doi.org/10.1007/s00330-020-07141-9

  45. Luo W, Phung D, Tran T et al (2016) Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res 18(12):e323

    PubMed  PubMed Central  Google Scholar 

  46. Jethanandani A, Lin TA, Volpe S et al (2018) Exploring applications of radiomics in magnetic resonance imaging of head and neck cancer: a systematic review. Front Oncol 8:131

    PubMed  PubMed Central  Google Scholar 

  47. Nagendran M, Chen Y, Lovejoy CA et al (2020) Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ 368:m689

    PubMed  PubMed Central  Google Scholar 

  48. Collins GS, Reitsma JB, Altman DG, Moons KG (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 350:g7594

    PubMed  Google Scholar 

  49. European Society of Radiology (ESR) (2020) ESR statement on the validation of imaging biomarkers. Insights Imaging 11(1):76

    Google Scholar 

  50. CONSORT-AI and SPIRIT-AI Steering Group (2019) Reporting guidelines for clinical trials evaluating artificial intelligence interventions are needed. Nat Med 25(10):1467–1468

    Google Scholar 

  51. Mongan J, Moy L, Kahn CE Jr (2020) Checklist for artificial intelligence in medical imaging (CLAIM): a guide for authors and reviewers. Radiology Artificial Intelligence 2(2):e200029

    Google Scholar 

Download references

Acknowledgments

The authors would like to express their gratitude to Prof. Guang Yang and Ms. Chengxiu Zhang for their constructive discussion and suggestions. The authors would like to thank Dr. Guangcheng Zhang for English language editing.

Funding

This study has received funding by National Natural Science Foundation of China (81771790) and Medicine and Engineering Combination Project of Shanghai Jiao Tong University (YG2019ZDB09).

Author information

Author notes

  1. Jingyu Zhong and Yangfan Hu contributed equally to this work.

Authors and Affiliations

  1. Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, No. 1111 Xianxia Road, Changning District, Shanghai, 200050, China

    Jingyu Zhong, Liping Si & Weiwu Yao

  2. Department of Radiology, Shanghai Jiao Tong University Affiliated Sixth People’s Hospital, No. 600 Yishan Road, Xuhui District, Shanghai, 200233, China

    Yangfan Hu, Geng Jia & Yue Xing

  3. Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, No. 197 Ruijin 2nd Road, Huangpu District, Shanghai, 200025, China

    Huan Zhang

Authors
  1. Jingyu Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yangfan Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liping Si
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Geng Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yue Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Huan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Weiwu Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Huan Zhang or Weiwu Yao.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Prof. Weiwu Yao.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors has significant statistical expertise.

Informed consent

Written informed consent was not required for this study because of the nature of our study, which was a systematic review and meta-analysis.

Ethical approval

Institutional Review Board approval was not required because of the nature of our study, which was a systematic review and meta-analysis.

Methodology

• retrospective

• diagnostic or prognostic study

• multicenter study

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 408 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, J., Hu, Y., Si, L. et al. A systematic review of radiomics in osteosarcoma: utilizing radiomics quality score as a tool promoting clinical translation. Eur Radiol 31, 1526–1535 (2021). https://doi.org/10.1007/s00330-020-07221-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-020-07221-w

Keywords

Search

Navigation