Skip to main content

Advertisement

SpringerLink
Log in

Diagnostic Performance of Artificial Intelligence in Detection of Primary Malignant Bone Tumors: a Meta-Analysis

  • Published:
Journal of Imaging Informatics in Medicine Aims and scope Submit manuscript

Abstract

We aim to conduct a meta-analysis on studies that evaluated the diagnostic performance of artificial intelligence (AI) algorithms in the detection of primary bone tumors, distinguishing them from other bone lesions, and comparing them with clinician assessment. A systematic search was conducted using a combination of keywords related to bone tumors and AI. After extracting contingency tables from all included studies, we performed a meta-analysis using random-effects model to determine the pooled sensitivity and specificity, accompanied by their respective 95% confidence intervals (CI). Quality assessment was evaluated using a modified version of Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) and Prediction Model Study Risk of Bias Assessment Tool (PROBAST). The pooled sensitivities for AI algorithms and clinicians on internal validation test sets for detecting bone neoplasms were 84% (95% CI: 79.88) and 76% (95% CI: 64.85), and pooled specificities were 86% (95% CI: 81.90) and 64% (95% CI: 55.72), respectively. At external validation, the pooled sensitivity and specificity for AI algorithms were 84% (95% CI: 75.90) and 91% (95% CI: 83.96), respectively. The same numbers for clinicians were 85% (95% CI: 73.92) and 94% (95% CI: 89.97), respectively. The sensitivity and specificity for clinicians with AI assistance were 95% (95% CI: 86.98) and 57% (95% CI: 48.66). Caution is needed when interpreting findings due to potential limitations. Further research is needed to bridge this gap in scientific understanding and promote effective implementation for medical practice advancement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data Availability

The data that support the findings of this study are available from the authors upon reasonable request.

References

  1. Li J, Li S, Li X, Miao S, Dong C, Gao C, et al. Primary bone tumor detection and classification in full-field bone radiographs via YOLO deep learning model. European Radiology. 2022.

  2. Kerr DL, Dial BL, Lazarides AL, Catanzano AA, Lane WO, Blazer DG, 3rd, et al. Epidemiologic and survival trends in adult primary bone tumors of the spine. Spine J. 2019;19(12):1941-9.

    Article  PubMed  Google Scholar 

  3. Georgeanu VA, Mămuleanu M, Ghiea S, Selișteanu D. Malignant Bone Tumors Diagnosis Using Magnetic Resonance Imaging Based on Deep Learning Algorithms. Medicina (Lithuania). 2022;58(5).

  4. Salazar C, Leite M, Sousa A, Torres J. Correlation between imagenological and histological diagnosis of bone tumors. A retrospective study. Acta Ortop Mex. 2019;33(6):386-90.

    CAS  PubMed  Google Scholar 

  5. Goyal N, Kalra M, Soni A, Baweja P, Ghonghe NP. Multi-modality imaging approach to bone tumors - State-of-the art. J Clin Orthop Trauma. 2019;10(4):687-701.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts H. Artificial intelligence in radiology. Nat Rev Cancer. 2018;18(8):500-10.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Han SH, Kim KW, Kim S, Youn YC. Artificial Neural Network: Understanding the Basic Concepts without Mathematics. Dement Neurocogn Disord. 2018;17(3):83-9.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Driver CN, Bowles BS, Bartholmai BJ, Greenberg-Worisek AJ. Artificial Intelligence in Radiology: A Call for Thoughtful Application. Clin Transl Sci. 2020;13(2):216-8.

    Article  PubMed  Google Scholar 

  9. Nair AV, Ramanathan S, Sathiadoss P, Jajodia A, Blair Macdonald D. Barriers to artificial intelligence implementation in radiology practice: What the radiologist needs to know. Radiologia (Engl Ed). 2022;64(4):324-32.

    Article  CAS  PubMed  Google Scholar 

  10. von Schacky CE, Wilhelm NJ, Schäfer VS, Leonhardt Y, Jung M, Jungmann PM, et al. Development and evaluation of machine learning models based on X-ray radiomics for the classification and differentiation of malignant and benign bone tumors. European Radiology. 2022;32(9):6247-57.

    Article  Google Scholar 

  11. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Annals of internal medicine. 2015;162(1):55-63.

    Article  PubMed  Google Scholar 

  12. Wolff RF, Moons KG, Riley RD, Whiting PF, Westwood M, Collins GS, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Annals of internal medicine. 2019;170(1):51-8.

    Article  PubMed  Google Scholar 

  13. Dwamena B. MIDAS: Stata module for meta-analytical integration of diagnostic test accuracy studies. 2009.

  14. Harbord RM, Whiting P. Metandi: meta-analysis of diagnostic accuracy using hierarchical logistic regression. The Stata Journal. 2009;9(2):211-29.

    Article  Google Scholar 

  15. Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. Journal of clinical epidemiology. 2005;58(9):882-93.

    Article  PubMed  Google Scholar 

  16. Altameem T. Fuzzy rank correlation-based segmentation method and deep neural network for bone cancer identification. Neural Computing and Applications. 2020;32(3):805-15.

    Article  Google Scholar 

  17. Arana E, Marti-Bonmati L, Bautista D, Paredes R. Calvarial eosinophilic granuloma: Diagnostic models and image feature selection with a neural network. Academic Radiology. 1998;5(6):427-34.

    Article  CAS  PubMed  Google Scholar 

  18. Chianca V, Cuocolo R, Gitto S, Albano D, Merli I, Badalyan J, et al. Radiomic Machine Learning Classifiers in Spine Bone Tumors: A Multi-Software, Multi-Scanner Study. Eur J Radiol. 2021;137:109586.

    Article  PubMed  Google Scholar 

  19. Consalvo S, Hinterwimmer F, Neumann J, Steinborn M, Salzmann M, Seidl F, et al. Two-Phase Deep Learning Algorithm for Detection and Differentiation of Ewing Sarcoma and Acute Osteomyelitis in Paediatric Radiographs. Anticancer Research. 2022;42(9):4371-80.

    Article  PubMed  Google Scholar 

  20. Do BH, Langlotz C, Beaulieu CF. Bone Tumor Diagnosis Using a Naïve Bayesian Model of Demographic and Radiographic Features. J Digit Imaging. 2017;30(5):640-7.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Eweje FR, Bao B, Wu J, Dalal D, Liao WH, He Y, et al. Deep Learning for Classification of Bone Lesions on Routine MRI. EBioMedicine. 2021;68:103402.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Gitto S, Cuocolo R, Albano D, Chianca V, Messina C, Gambino A, et al. MRI radiomics-based machine-learning classification of bone chondrosarcoma. Eur J Radiol. 2020;128:109043.

    Article  PubMed  Google Scholar 

  23. Gitto S, Cuocolo R, van Langevelde K, van de Sande MAJ, Parafioriti A, Luzzati A, et al. MRI radiomics-based machine learning classification of atypical cartilaginous tumour and grade II chondrosarcoma of long bones. EBioMedicine. 2022;75.

  24. He Y, Pan I, Bao B, Halsey K, Chang M, Liu H, et al. Deep learning-based classification of primary bone tumors on radiographs: A preliminary study. EBioMedicine. 2020;62:103121.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Ho NH, Yang HJ, Kim SH, Jung ST, Joo SD. Regenerative semi-supervised bidirectional w-network-based knee bone tumor classification on radiographs guided by three-region bone segmentation. IEEE Access. 2019;7:154277-89.

    Article  Google Scholar 

  26. Lee A, Kim MS, Han SS, Park PG, Lee C, Yun JP. Deep learning neural networks to differentiate Stafne's bone cavity from pathological radiolucent lesions of the mandible in heterogeneous panoramic radiography. PLoS ONE. 2021;16(7 July).

  27. Liu H, Jiao M, Yuan Y, Ouyang H, Liu J, Li Y, et al. Benign and malignant diagnosis of spinal tumors based on deep learning and weighted fusion framework on MRI. Insights Imaging. 2022;13(1):87.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Liu H, Jiao ML, Xing XY, Ou-Yang HQ, Yuan Y, Liu JF, et al. BgNet: Classification of benign and malignant tumors with MRI multi-plane attention learning. Front Oncol. 2022;12:971871.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Liu Y, Yang P, Pi Y, Jiang L, Zhong X, Cheng J, et al. Automatic identification of suspicious bone metastatic lesions in bone scintigraphy using convolutional neural network. BMC Med Imaging. 2021;21(1):131.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Pan D, Liu R, Zheng B, Yuan J, Zeng H, He Z, et al. Using Machine Learning to Unravel the Value of Radiographic Features for the Classification of Bone Tumors. Biomed Res Int. 2021;2021:8811056.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Park CW, Oh SJ, Kim KS, Jang MC, Kim IS, Lee YK, et al. Artificial intelligence-based classification of bone tumors in the proximal femur on plain radiographs: System development and validation. PLoS One. 2022;17(2):e0264140.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Reinus WR, Wilson AJ, Kalman B, Kwasny S. Diagnosis of focal bone lesions using neural networks. Investigative Radiology. 1994;29(6):606-11.

    Article  CAS  PubMed  Google Scholar 

  33. Sharma A, Yadav DP, Garg H, Kumar M, Sharma B, Koundal D. Bone Cancer Detection Using Feature Extraction Based Machine Learning Model. Comput Math Methods Med. 2021;2021:7433186.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Zhao K, Zhang M, Xie Z, Yan X, Wu S, Liao P, et al. Deep Learning Assisted Diagnosis of Musculoskeletal Tumors Based on Contrast-Enhanced Magnetic Resonance Imaging. Journal of Magnetic Resonance Imaging. 2022;56(1):99-107.

    Article  PubMed  Google Scholar 

  35. Zhao S, Chen B, Chang H, Chen B, Li S. Reasoning discriminative dictionary-embedded network for fully automatic vertebrae tumor diagnosis. Med Image Anal. 2022;79:102456.

    Article  PubMed  Google Scholar 

  36. Do NT, Jung ST, Yang HJ, Kim SH. Multi-Level Seg-Unet Model with Global and Patch-Based X-ray Images for Knee Bone Tumor Detection. Diagnostics (Basel). 2021;11(4).

  37. Fouad H, Hassanein AS, Soliman AM, Al-Feel H. Internet of Medical Things (IoMT) Assisted Vertebral Tumor Prediction Using Heuristic Hock Transformation Based Gautschi Model–A Numerical Approach. IEEE Access. 2020;8:17299-309.

    Article  Google Scholar 

  38. Gitto S, Bologna M, Corino VDA, Emili I, Albano D, Messina C, et al. Diffusion-weighted MRI radiomics of spine bone tumors: feature stability and machine learning-based classification performance. Radiol Med. 2022.

  39. Gitto S, Cuocolo R, Annovazzi A, Anelli V, Acquasanta M, Cincotta A, et al. CT radiomics-based machine learning classification of atypical cartilaginous tumours and appendicular chondrosarcomas. EBioMedicine. 2021;68:103407.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Ouyang H, Meng F, Liu J, Song X, Li Y, Yuan Y, et al. Evaluation of Deep Learning-Based Automated Detection of Primary Spine Tumors on MRI Using the Turing Test. Front Oncol. 2022;12:814667.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Shung D, Simonov M, Gentry M, Au B, Laine L. Machine Learning to Predict Outcomes in Patients with Acute Gastrointestinal Bleeding: A Systematic Review. Dig Dis Sci. 2019;64(8):2078-87.

    Article  PubMed  Google Scholar 

  42. Montesinos-López O, Montesinos A, Crossa J. Overfitting, Model Tuning, and Evaluation of Prediction Performance. 2022. p. 109–39.

  43. Liu S, Feng M, Qiao T, Cai H, Xu K, Yu X, et al. Deep Learning for the Automatic Diagnosis and Analysis of Bone Metastasis on Bone Scintigrams. Cancer Manag Res. 2022;14:51-65.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Zheng Q, Yang L, Zeng B, Li J, Guo K, Liang Y, et al. Artificial intelligence performance in detecting tumor metastasis from medical radiology imaging: A systematic review and meta-analysis. EClinicalMedicine. 2021;31:100669.

    Article  PubMed  Google Scholar 

  45. Younis MH, Abu-Hijleh HA, Aldahamsheh OO, Abualruz A, Thalib L. Meta-Analysis of the Diagnostic Accuracy of Primary Bone and Soft Tissue Sarcomas by 18F-FDG-PET. Med Princ Pract. 2020;29(5):465-72.

    Article  PubMed  Google Scholar 

  46. Seth N, Seth I, Bulloch G, Siu AHY, Guo A, Chatterjee R, et al. (18) F-FDG PET and PET/CT as a diagnostic method for Ewing sarcoma: A systematic review and meta-analysis. Pediatr Blood Cancer. 2022;69(3):e29415.

    Article  PubMed  Google Scholar 

  47. Zimmer WD, Berquist TH, McLeod RA, Sim FH, Pritchard DJ, Shives TC, et al. Bone tumors: magnetic resonance imaging versus computed tomography. Radiology. 1985;155(3):709-18.

    Article  CAS  PubMed  Google Scholar 

  48. van Dyk DA, Meng X-L. The Art of Data Augmentation. Journal of Computational and Graphical Statistics. 2001;10(1):1-50.

    Article  Google Scholar 

  49. Torrey L, Shavlik J. Transfer learning. Handbook of Research on Machine Learning Applications. 2009.

Download references

Author information

Author notes

  1. Mohammad Amin Salehi and Soheil Mohammadi contributed equally.

Authors and Affiliations

  1. School of Medicine, Tehran University of Medical Sciences, Pour Sina St, Keshavarz Blvd, Tehran, 1417613151, Iran

    Mohammad Amin Salehi, Soheil Mohammadi & Hamid Harandi

  2. School of Medicine, Tabriz University of Medical Sciences, Tabriz, Iran

    Seyed Sina Zakavi

  3. School of Medicine, Guilan University of Medical Sciences, Rasht, Iran

    Ali Jahanshahi

  4. Medical Students Research Committee, Shahed University, Tehran, Iran

    Mohammad Shahrabi Farahani

  5. Department of Radiology, Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Avenue, Boston, MA, 02215, USA

    Jim S. Wu

Authors
  1. Mohammad Amin Salehi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Soheil Mohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hamid Harandi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Seyed Sina Zakavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ali Jahanshahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mohammad Shahrabi Farahani
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jim S. Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Soheil Mohammadi and Mohammad Amin Salehi designed the project, contributed to protocol development, literature search, screening, data extraction, and writing of the original draft. Hamid Harandi contributed to writing of the original draft. Seyed Sina Zakavi and Ali Jahanshahi contributed to screening and protocol development. Mohammad Shahrabi Farahani contributed to the writing of the original draft. Jim S. Wu encouraged and supervised the project, reviewed the manuscript, and contributed to the writing of the original and final draft.

Corresponding author

Correspondence to Soheil Mohammadi.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Salehi, M.A., Mohammadi, S., Harandi, H. et al. Diagnostic Performance of Artificial Intelligence in Detection of Primary Malignant Bone Tumors: a Meta-Analysis. J Digit Imaging. Inform. med. 37, 766–777 (2024). https://doi.org/10.1007/s10278-023-00945-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-023-00945-3

Keywords

Search

Navigation