Abstract
Objectives
To build and validate deep learning and machine learning fusion models to classify benign, malignant, and intermediate bone tumors based on patient clinical characteristics and conventional radiographs of the lesion.
Methods
In this retrospective study, data were collected with pathologically confirmed diagnoses of bone tumors between 2012 and 2019. Deep learning and machine learning fusion models were built to classify tumors as benign, malignant, or intermediate using conventional radiographs of the lesion and potentially relevant clinical data. Five radiologists compared diagnostic performance with and without the model. Diagnostic performance was evaluated using the area under the curve (AUC).
Results
A total of 643 patients’ (median age, 21 years; interquartile range, 12–38 years; 244 women) 982 radiographs were included. In the test set, the binary category classification task, the radiological model of classification for benign/not benign, malignant/nonmalignant, and intermediate/not intermediate had AUCs of 0.846, 0.827, and 0.820, respectively; the fusion models had an AUC of 0.898, 0.894, and 0.865, respectively. In the three-category classification task, the radiological model achieved a macro average AUC of 0.813, and the fusion model had a macro average AUC of 0.872. In the observation test, the mean macro average AUC of all radiologists was 0.819. With the three-category classification fusion model support, the macro AUC improved by 0.026.
Conclusion
We built, validated, and tested deep learning and machine learning models that classified bone tumors at a level comparable with that of senior radiologists. Model assistance may somewhat help radiologists’ differential diagnoses of bone tumors.
Key Points
• The deep learning model can be used to classify benign, malignant, and intermediate bone tumors.
• The machine learning model fusing information from radiographs and clinical characteristics can improve the classification capacity for bone tumors.
• The diagnostic performance of the fusion model is comparable with that of senior radiologists and is potentially useful as a complement to radiologists in a bone tumor differential diagnosis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Abbreviations
- AUC:
-
Area under curve
- DL:
-
Deep learning
References
Gianferante DM, Mirabello L, Savage SA (2017) Germline and somatic genetics of osteosarcoma - connecting aetiology, biology and therapy. Nat Rev Endocrinol 13:480–491
Siegel RL, Miller KD, Jemal A (2020) Cancer statistics, 2020. CA Cancer J Clin 70:7–30
(2020) WHO Classification of Tumours Editorial Board. Soft tissue and bone tumours. Lyon (France): International Agency for Research on Cancer 2020. WHO classification of tumours series, 5th ed Vol 3
Fritzsche H, Schaser KD, Hofbauer C (2017) Benign tumours and tumour-like lesions of the bone : general treatment principles. Orthopade 46:484–497
Antropova N, Huynh BQ, Giger ML (2017) A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets. Med Phys 44:5162–5171
Bestic JM, Wessell DE, Beaman FD et al (2020) ACR Appropriateness Criteria® Primary Bone Tumors. J Am Coll Radiol 17:S226–S238
Miller TT (2008) Bone tumors and tumorlike conditions: analysis with conventional radiography. Radiology 246:662–674
Costelloe CM, Madewell JE (2013) Radiography in the initial diagnosis of primary bone tumors. AJR Am J Roentgenol 200:3–7
Gemescu IN, Thierfelder KM, Rehnitz C, Weber MA (2019) Imaging features of bone tumors: conventional radiographs and MR imaging correlation. Magn Reson Imaging Clin N Am 27:753–767
Do BH, Langlotz C, Beaulieu CF (2017) Bone tumor diagnosis using a naïve Bayesian model of demographic and radiographic features. J Digit Imaging 30:640–647
Tomasian A, Hillen TJ, Jennings JW (2020) Bone biopsies: what radiologists need to know. AJR Am J Roentgenol 215:523–533
Bruno MA, Nagy P (2014) Fundamentals of quality and safety in diagnostic radiology. J Am Coll Radiol 11:1115–1120
Gore JC (2020) Artificial intelligence in medical imaging. Magn Reson Imaging 68:A1–A4
Gorelik N, Gyftopoulos S (2020) Applications of artificial intelligence in musculoskeletal imaging: from the request to the report. Can Assoc Radiol J. https://doi.org/10.1177/0846537120947148:846537120947148
Chea P, Mandell JC (2020) Current applications and future directions of deep learning in musculoskeletal radiology. Skelet Radiol 49:183–197
Vogrin M, Trojner T, Kelc R (2020) Artificial intelligence in musculoskeletal oncological radiology. Radiol Oncol. https://doi.org/10.2478/raon-2020-0068
Alge O, Lu L, Li Z, Hua Y, Najarian K (2020) Automated classification of osteosarcoma and benign tumors using RNA-seq and Plain X-ray2020 42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) in conjunction with the 43rd Annual Conference of the Canadian Medical and Biological Engineering Society,
Bandyopadhyay O, Biswas A, Bhattacharya BB (2019) Bone-cancer assessment and destruction pattern analysis in long-bone X-ray image. J Digit Imaging 32:300–313
He Y, Pan I, Bao B et al (2020) Deep learning-based classification of primary bone tumors on radiographs: a preliminary study. EBioMedicine:62
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision.2818-2826
Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system the 22nd ACM SIGKDD International Conference,
Benndorf M, Neubauer J, Langer M, Kotter E (2017) Bayesian pretest probability estimation for primary malignant bone tumors based on the Surveillance, Epidemiology and End Results Program (SEER) database. Int J Comput Assist Radiol Surg 12:485–491
Wülling M, Engels C, Jesse N, Werner M, Delling G, Kaiser E (2001) The nature of giant cell tumor of bone. J Cancer Res Clin Oncol 127:467–474
Remotti F, Feldman F (2012) Nonneoplastic lesions that simulate primary tumors of bone. Arch Pathol Lab Med 136:772–788
Zhang X, Zhou J, Chai X et al (2018) The application of x-ray, computed tomography, and magnetic resonance imaging on 22 pediatric Langerhans cell histiocytosis patients with long bone involvement: a retrospective analysis. Medicine (Baltimore) 97:e0411
Angelini A, Mavrogenis AF, Rimondi E, Rossi G, Ruggieri P (2017) Current concepts for the diagnosis and management of eosinophilic granuloma of bone. J Orthop Traumatol 18:83–90
Krooks J, Minkov M, Weatherall AG (2018) Langerhans cell histiocytosis in children: history, classification, pathobiology, clinical manifestations, and prognosis. J Am Acad Dermatol 78:1035–1044
Obuchowski NA, Beiden SV, Berbaum KS et al (2004) Multireader, multicase receiver operating characteristic analysis: an empirical comparison of five methods. Acad Radiol 11:980–995
Funding
Presidential Foundation of the Natural Science Foundation of Guangdong Province, China (2019A1515011168); National Key Research and Development Program of China (2019YFC0121900, 2019YFC0121903); National Key Research and Development Program of China (2019YFC0117300, 2019YFC0117301).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Guarantor
The scientific guarantor of this publication is Weiguo Chen.
Conflict of interest
The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.
Statistics and biometry
No complex statistical methods were necessary for this paper.
Informed consent
Written informed consent was waived by the Institutional Review Board.
Ethical approval
Institutional Review Board approval was obtained.
Methodology
• retrospective
• diagnostic or prognostic study
• performed at one institution
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
ESM 1
(DOC 143 kb)
Rights and permissions
About this article
Cite this article
Liu, R., Pan, D., Xu, Y. et al. A deep learning–machine learning fusion approach for the classification of benign, malignant, and intermediate bone tumors. Eur Radiol 32, 1371–1383 (2022). https://doi.org/10.1007/s00330-021-08195-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00330-021-08195-z