Skip to main content

Advertisement

SpringerLink
Log in

Artificial intelligence applied to musculoskeletal oncology: a systematic review

  • Review Article
  • Published:
Skeletal Radiology Aims and scope Submit manuscript

Abstract

Developments in artificial intelligence have the potential to improve the care of patients with musculoskeletal tumors. We performed a systematic review of the published scientific literature to identify the current state of the art of artificial intelligence applied to musculoskeletal oncology, including both primary and metastatic tumors, and across the radiology, nuclear medicine, pathology, clinical research, and molecular biology literature. Through this search, we identified 252 primary research articles, of which 58 used deep learning and 194 used other machine learning techniques. Articles involving deep learning have mostly involved bone scintigraphy, histopathology, and radiologic imaging. Articles involving other machine learning techniques have mostly involved transcriptomic analyses, radiomics, and clinical outcome prediction models using medical records. These articles predominantly present proof-of-concept work, other than the automated bone scan index for bone metastasis quantification, which has translated to clinical workflows in some regions. We systematically review and discuss this literature, highlight opportunities for multidisciplinary collaboration, and identify potentially clinically useful topics with a relative paucity of research attention. Musculoskeletal oncology is an inherently multidisciplinary field, and future research will need to integrate and synthesize noisy siloed data from across clinical, imaging, and molecular datasets. Building the data infrastructure for collaboration will help to accelerate progress towards making artificial intelligence truly useful in musculoskeletal oncology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med [Internet]. 2019 [cited 2019 Jun 16];25:44–56. Available from: http://www.ncbi.nlm.nih.gov/pubmed/30617339.

  2. Shimizu H, Nakayama KI. Artificial intelligence in oncology. Cancer Sci [Internet]. Blackwell Publishing Ltd; 2020 [cited 2021 Mar 8];111:1452–60. Available from: https://pubmed.ncbi.nlm.nih.gov/32133724/.

  3. Gyftopoulos S, Lin D, Knoll F, Doshi AM, Rodrigues TC, Recht MP. Artificial intelligence in musculoskeletal imaging: current status and future directions [Internet]. Am. J. Roentgenol. American Roentgen Ray Society; 2019 [cited 2021 Mar 8]. p. 506–13. Available from: https://pubmed.ncbi.nlm.nih.gov/31166761/.

  4. Kijowski R, Liu F, Caliva F, Pedoia V. Deep learning for lesion detection, progression, and prediction of musculoskeletal disease [Internet]. J. Magn. Reson. Imaging. John Wiley and Sons Inc; 2020 [cited 2021 Mar 8]. p. 1607–19. Available from: https://pubmed.ncbi.nlm.nih.gov/31763739/.

  5. Vogrin M, Trojner T, Kelc R. Artificial intelligence in musculoskeletal oncological radiology [Internet]. Radiol. Oncol. Sciendo; 2020 [cited 2021 Mar 8]. p. 1–6. Available from: https://pubmed.ncbi.nlm.nih.gov/33170144/.

  6. Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A. Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology. Nat Rev Clin Oncol [Internet]. Nature Publishing Group; 2019 [cited 2021 Mar 8];16:703–15. Available from: https://pubmed.ncbi.nlm.nih.gov/31399699/.

  7. Panchmatia JR, Visenio MR, Panch T. The role of artificial intelligence in orthopaedic surgery. Br J Hosp Med [Internet]. MA Healthcare Ltd; 2018 [cited 2021 Mar 8];79:676–81. Available from: https://pubmed.ncbi.nlm.nih.gov/30526106/.

  8. Huynh E, Hosny A, Guthier C, Bitterman DS, Petit SF, Haas-Kogan DA, et al. Artificial intelligence in radiation oncology [Internet]. Nat. Rev. Clin. Oncol. Nature Research; 2020 [cited 2021 Mar 8]. p. 771–81. Available from: https://www.nature.com/articles/s41571-020-0417-8.

  9. Seifert R, Weber M, Kocakavuk E, Rischpler C, Kersting D. Artificial intelligence and machine learning in nuclear medicine: future perspectives [Internet]. Semin. Nucl. Med. W.B. Saunders; 2021 [cited 2021 Mar 9]. p. 170–7. Available from: https://pubmed.ncbi.nlm.nih.gov/33509373/.

  10. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ, editors. Adv Neural Inf Process Syst [Internet]. Curran Associates, Inc.; 2012. Available from: https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf.

  11. Lodwick GS, Haun CL, Smith WE, Keller RF, Robertson ED. Computer diagnosis of primary bone tumors. Radiology [Internet]. Radiological Society of North America (RSNA); 1963 [cited 2021 Mar 10];80:273–5. Available from: https://pubs.rsna.org/doi/abs/https://doi.org/10.1148/80.2.273.

  12. Do BH, Langlotz C, Beaulieu CF. Bone tumor diagnosis using a naïve Bayesian model of demographic and radiographic features. J Digit Imaging [Internet]. Springer New York LLC; 2017 [cited 2021 Mar 10];30:640–7. Available from: https://pubmed.ncbi.nlm.nih.gov/28752323/.

  13. Bandyopadhyay O, Biswas A, Bhattacharya BB. Bone-cancer assessment and destruction pattern analysis in long-bone X-ray image. J Digit Imaging [Internet]. Springer New York LLC; 2019 [cited 2021 Mar 10];32:300–13. Available from: https://pubmed.ncbi.nlm.nih.gov/30367308/.

  14. He Y, Pan I, Bao B, Halsey K, Chang M, Liu H, et al. Deep learning-based classification of primary bone tumors on radiographs: a preliminary study. EBioMedicine [Internet]. Elsevier B.V.; 2020 [cited 2021 Mar 10];62. Available from: https://pubmed.ncbi.nlm.nih.gov/33232868/.

  15. Eweje FR, Bao B, Wu J, Dalal D, Liao W, He Y, et al. Deep learning for classification of bone lesions on routine MRI. SSRN Electron J [Internet]. Elsevier BV; 2021 [cited 2021 Mar 10]; Available from: https://papers.ssrn.com/abstract=3777131.

  16. Ariji Y, Yanashita Y, Kutsuna S, Muramatsu C, Fukuda M, Kise Y, et al. Automatic detection and classification of radiolucent lesions in the mandible on panoramic radiographs using a deep learning object detection technique. Oral Surg Oral Med Oral Pathol Oral Radiol [Internet]. Mosby Inc.; 2019 [cited 2021 Mar 10];128:424–30. Available from: https://pubmed.ncbi.nlm.nih.gov/31320299/.

  17. Santin M, Brama C, Théro H, Ketheeswaran E, El-Karoui I, Bidault F, et al. Detecting abnormal thyroid cartilages on CT using deep learning. Diagn Interv Imaging [Internet]. Elsevier Masson SAS; 2019 [cited 2021 Mar 10];100:251–7. Available from: https://pubmed.ncbi.nlm.nih.gov/30819638/.

  18. Wang B, Perronne L, Burke C, Adler RS. Artificial intelligence for classification of soft-tissue masses at US. Radiol Artif Intell [Internet]. Radiological Society of North America (RSNA); 2021 [cited 2021 Mar 10];3:e200125. Available from: https://pubs.rsna.org/doi/abs/https://doi.org/10.1148/ryai.2020200125.

  19. He Y, Guo J, Ding X, van Ooijen PMA, Zhang Y, Chen A, et al. Convolutional neural network to predict the local recurrence of giant cell tumor of bone after curettage based on pre-surgery magnetic resonance images. Eur Radiol [Internet]. Springer Verlag; 2019 [cited 2021 Mar 10];29:5441–51. Available from: https://pubmed.ncbi.nlm.nih.gov/30859281/.

  20. Lang N, Zhang Y, Zhang E, Zhang J, Chow D, Chang P, et al. Differentiation of spinal metastases originated from lung and other cancers using radiomics and deep learning based on DCE-MRI. Magn Reson Imaging [Internet]. Elsevier Inc.; 2019 [cited 2021 Mar 10];64:4–12. Available from: https://pubmed.ncbi.nlm.nih.gov/30826448/.

  21. Zhang R, Huang L, Xia W, Zhang B, Qiu B, Gao X. Multiple supervised residual network for osteosarcoma segmentation in CT images. Comput Med Imaging Graph [Internet]. Elsevier Ltd; 2018 [cited 2021 Mar 10];63:1–8. Available from: https://pubmed.ncbi.nlm.nih.gov/29361340/.

  22. Huang L, Xia W, Zhang B, Qiu B, Gao X. MSFCN-multiple supervised fully convolutional networks for the osteosarcoma segmentation of CT images. Comput Methods Programs Biomed [Internet]. Elsevier Ireland Ltd; 2017 [cited 2021 Mar 10];143:67–74. Available from: https://pubmed.ncbi.nlm.nih.gov/28391820/.

  23. Klein A, Warszawski J, Hillengaß J, Maier-Hein KH. Automatic bone segmentation in whole-body CT images. Int J Comput Assist Radiol Surg [Internet]. Springer Verlag; 2019 [cited 2021 Mar 10];14:21–9. Available from: https://pubmed.ncbi.nlm.nih.gov/30426400/.

  24. Lindgren Belal S, Sadik M, Kaboteh R, Enqvist O, Ulén J, Poulsen MH, et al. Deep learning for segmentation of 49 selected bones in CT scans: first step in automated PET/CT-based 3D quantification of skeletal metastases. Eur J Radiol [Internet]. Elsevier Ireland Ltd; 2019 [cited 2021 Mar 10];113:89–95. Available from: https://pubmed.ncbi.nlm.nih.gov/30927965/.

  25. Zaman A, Park SH, Bang H, Park C woo, Park I, Joung S. Generative approach for data augmentation for deep learning-based bone surface segmentation from ultrasound images. Int J Comput Assist Radiol Surg [Internet]. Springer; 2020 [cited 2021 Mar 10];15:931–41. Available from: https://pubmed.ncbi.nlm.nih.gov/32399586/.

  26. Leporq B, Bouhamama A, Pilleul F, Lame F, Bihane C, Sdika M, et al. MRI-based radiomics to predict lipomatous soft tissue tumors malignancy: a pilot study. Cancer Imaging [Internet]. BioMed Central Ltd; 2020 [cited 2021 Mar 10];20. Available from: https://pubmed.ncbi.nlm.nih.gov/33115533/.

  27. Yin P, Mao N, Zhao C, Wu J, Sun C, Chen L, et al. Comparison of radiomics machine-learning classifiers and feature selection for differentiation of sacral chordoma and sacral giant cell tumour based on 3D computed tomography features. Eur Radiol [Internet]. Springer Verlag; 2019 [cited 2021 Mar 10];29:1841–7. Available from: https://pubmed.ncbi.nlm.nih.gov/30280245/.

  28. Wang H, Chen H, Duan S, Hao D, Liu J. Radiomics and machine learning with multiparametric preoperative MRI may accurately predict the histopathological grades of soft tissue sarcomas. J Magn Reson Imaging [Internet]. John Wiley and Sons Inc.; 2020 [cited 2021 Mar 10];51:791–7. Available from: https://pubmed.ncbi.nlm.nih.gov/31486565/.

  29. Timbergen MJM, Starmans MPA, Padmos GA, Grünhagen DJ, van Leenders GJLH, Hanff DF, et al. Differential diagnosis and mutation stratification of desmoid-type fibromatosis on MRI using radiomics. Eur J Radiol [Internet]. Elsevier Ireland Ltd; 2020 [cited 2021 Mar 10];131. Available from: https://pubmed.ncbi.nlm.nih.gov/32971431/.

  30. Oh E, Seo SW, Yoon YC, Kim DW, Kwon S, Yoon S. Prediction of pathologic femoral fractures in patients with lung cancer using machine learning algorithms: comparison of computed tomography-based radiological features with clinical features versus without clinical features. J Orthop Surg [Internet]. SAGE Publications Ltd; 2017 [cited 2021 Mar 10];25. Available from: https://pubmed.ncbi.nlm.nih.gov/28659051/.

  31. Gao Y, Kalbasi A, Hsu W, Ruan D, Fu J, Shao J, et al. Treatment effect prediction for sarcoma patients treated with preoperative radiotherapy using radiomics features from longitudinal diffusion-weighted MRIs. Phys Med Biol [Internet]. IOP Publishing Ltd; 2020 [cited 2021 Mar 10];65. Available from: https://pubmed.ncbi.nlm.nih.gov/32554891/.

  32. Peeken JC, Wiestler B, Combs SE. Image-guided radiooncology: the potential of radiomics in clinical application. Recent Results Cancer Res [Internet]. Springer; 2020 [cited 2021 Mar 10]. p. 773–94. Available from: https://pubmed.ncbi.nlm.nih.gov/32594406/.

  33. Crombé A, Fadli D, Italiano A, Saut O, Buy X, Kind M. Systematic review of sarcomas radiomics studies: bridging the gap between concepts and clinical applications? Eur J Radiol [Internet]. Elsevier Ireland Ltd; 2020 [cited 2021 Mar 10];132. Available from: https://pubmed.ncbi.nlm.nih.gov/32980727/.

  34. Chen CY, Chiou HJ, Chou SY, Chiou SY, Wang HK, Chou YH, et al. Computer-aided diagnosis of soft-tissue tumors using sonographic morphologic and texture features. Acad Radiol [Internet]. Acad Radiol; 2009 [cited 2021 Mar 10];16:1531–8. Available from: https://pubmed.ncbi.nlm.nih.gov/19896070/.

  35. Lee YH. Efficiency improvement in a busy radiology practice: determination of musculoskeletal magnetic resonance imaging protocol using deep-learning convolutional neural networks. J Digit Imaging [Internet]. Springer New York LLC; 2018 [cited 2021 Mar 10];31:604–10. Available from: https://pubmed.ncbi.nlm.nih.gov/29619578/.

  36. Groot OQ, Bongers MER, Karhade A V., Kapoor ND, Fenn BP, Kim J, et al. Natural language processing for automated quantification of bone metastases reported in free-text bone scintigraphy reports. Acta Oncol (Madr) [Internet]. Taylor and Francis Ltd.; 2020 [cited 2021 Mar 10];59:1455–60. Available from: https://pubmed.ncbi.nlm.nih.gov/32924696/.

  37. Chen PH, Zafar H, Galperin-Aizenberg M, Cook T. Integrating natural language processing and machine learning algorithms to categorize oncologic response in radiology reports. J Digit Imaging [Internet]. Springer New York LLC; 2018 [cited 2021 Mar 10];31:178–84. Available from: https://pubmed.ncbi.nlm.nih.gov/29079959/.

  38. Wang T, Lei Y, Fu Y, Wynne JF, Curran WJ, Liu T, et al. A review on medical imaging synthesis using deep learning and its clinical applications. J Appl Clin Med Phys [Internet]. John Wiley and Sons Ltd; 2021 [cited 2021 Mar 10];22:11–36. Available from: https://onlinelibrary.wiley.com/doi/https://doi.org/10.1002/acm2.13121.

  39. Isaac A, Dalili D, Dalili D, Weber MA. State-of-the-art imaging for diagnosis of metastatic bone disease [Internet]. Radiologe. Springer Medizin; 2020 [cited 2021 Mar 9]. p. 1–16. Available from: https://doi.org/https://doi.org/10.1007/s00117-020-00666-6.

  40. Erdi YE, Humm JL, Imbriaco M, Yeung H, Larson SM. Quantitative bone metastases analysis based on image segmentation. J Nucl Med. 1997;38:1401–6.

    CAS  PubMed  Google Scholar 

  41. Imbriaco M, Larson SM, Yeung HW, Mawlawi OR, Erdi Y, Venkatraman ES, et al. A new parameter for measuring metastatic bone involvement by prostate cancer: the Bone Scan Index. Clin Cancer Res. 1998;4(4):1765–72.

    CAS  PubMed  Google Scholar 

  42. Sadik M, Hamadeh I, Nordblom P, Suurkula M, Höglund P, Ohlsson M, et al. Computer-assisted interpretation of planar whole-body bone scans. J Nucl Med [Internet]. Society of Nuclear Medicine; 2008 [cited 2021 Mar 9];49:1958–65. Available from: http://www.exini.com.

  43. Ulmert D, Kaboteh R, Fox JJ, Savage C, Evans MJ, Lilja H, et al. A novel automated platform for quantifying the extent of skeletal tumour involvement in prostate cancer patients using the bone scan index. Eur Urol [Internet]. Eur Urol; 2012 [cited 2021 Mar 9];62:78–84. Available from: https://pubmed.ncbi.nlm.nih.gov/22306323/.

  44. aBSI | EXINI Diagnostics AB [Internet]. [cited 2021 Mar 9]. Available from: https://exini.com/products/absi/.

  45. Armstrong AJ, Anand A, Edenbrandt L, Bondesson E, Bjartell A, Widmark A, et al. Phase 3 assessment of the automated bone scan index as a prognostic imaging biomarker of overall survival in men with metastatic castration-resistant prostate cancer a secondary analysis of a randomized clinical trial. JAMA Oncol [Internet]. American Medical Association; 2018 [cited 2021 Mar 9];4:944–51. Available from: https://jamanetwork.com/.

  46. Inaki A, Nakajima K, Wakabayashi H, Mochizuki T, Kinuya S. Fully automated analysis for bone scintigraphy with artificial neural network: usefulness of bone scan index (BSI) in breast cancer. Ann Nucl Med [Internet]. Springer Tokyo; 2019 [cited 2021 Mar 9];33:755–65. Available from: https://pubmed.ncbi.nlm.nih.gov/31317398/.

  47. Zhao Z, Pi Y, Jiang L, Xiang Y, Wei J, Yang P, et al. Deep neural network based artificial intelligence assisted diagnosis of bone scintigraphy for cancer bone metastasis. Sci Rep [Internet]. Nature Research; 2020 [cited 2021 Mar 9];10. Available from: https://pubmed.ncbi.nlm.nih.gov/33046779/.

  48. Shimizu A, Wakabayashi H, Kanamori T, Saito A, Nishikawa K, Daisaki H, et al. Automated measurement of bone scan index from a whole-body bone scintigram. Int J Comput Assist Radiol Surg [Internet]. Springer; 2020 [cited 2021 Mar 9];15:389–400. Available from: /pmc/articles/PMC7036077/.

  49. Papandrianos N, Papageorgiou E, Anagnostis A, Papageorgiou K. Bone metastasis classification using whole body images from prostate cancer patients based on convolutional neural networks application. PLoS One [Internet]. Public Library of Science; 2020 [cited 2021 Mar 9];15. Available from: https://pubmed.ncbi.nlm.nih.gov/32797099/.

  50. Minarik D, Enqvist O, Trägårdh E. Denoising of scintillation camera images using a deep convolutional neural network: a Monte Carlo simulation approach. J Nucl Med [Internet]. Society of Nuclear Medicine Inc.; 2020 [cited 2021 Mar 9];61:298–303. Available from: https://pubmed.ncbi.nlm.nih.gov/31324711/.

  51. Xu L, Tetteh G, Lipkova J, Zhao Y, Li H, Christ P, et al. Automated whole-body bone lesion detection for multiple myeloma on 68 Ga-Pentixafor PET/CT imaging using deep learning methods. Contrast Media Mol Imaging [Internet]. Hindawi Limited; 2018 [cited 2021 Mar 9];2018. Available from: https://pubmed.ncbi.nlm.nih.gov/29531504/.

  52. Zhao Y, Gafita A, Vollnberg B, Tetteh G, Haupt F, Afshar-Oromieh A, et al. Deep neural network for automatic characterization of lesions on 68Ga-PSMA-11 PET/CT. Eur J Nucl Med Mol Imaging [Internet]. Springer; 2020 [cited 2021 Mar 9];47:603–13. Available from: https://pubmed.ncbi.nlm.nih.gov/31813050/.

  53. Moreau N, Rousseau C, Fourcade C, Santini G, Ferrer L, Lacombe M, et al. Deep learning approaches for bone and bone lesion segmentation on 18FDG PET/CT imaging in the context of metastatic breast cancer. Proc Annu Int Conf IEEE Eng Med Biol Soc EMBS [Internet]. Institute of Electrical and Electronics Engineers Inc.; 2020 [cited 2021 Mar 9]. p. 1532–5. Available from: https://pubmed.ncbi.nlm.nih.gov/33018283/.

  54. Lin Q, Luo M, Gao R, Li T, Man Z, Cao Y, et al. Deep learning based automatic segmentation of metastasis hotspots in thorax bone SPECT images. PLoS One [Internet]. Public Library of Science; 2020 [cited 2021 Mar 9];15. Available from: https://pubmed.ncbi.nlm.nih.gov/33270746/.

  55. Lin Q, Li T, Cao C, Cao Y, Man Z, Wang H. Deep learning based automated diagnosis of bone metastases with SPECT thoracic bone images. Sci Rep [Internet]. Nature Research; 2021 [cited 2021 Mar 9];11. Available from: https://pubmed.ncbi.nlm.nih.gov/33608560/.

  56. Mori J, Kaji S, Kawai H, Kida S, Tsubokura M, Fukatsu M, et al. Assessment of dysplasia in bone marrow smear with convolutional neural network. Sci Rep [Internet]. Nature Research; 2020 [cited 2021 Mar 10];10. Available from: https://pubmed.ncbi.nlm.nih.gov/32895431/.

  57. Rehman A, Abbas N, Saba T, Rahman SI ur, Mehmood Z, Kolivand H. Classification of acute lymphoblastic leukemia using deep learning. Microsc Res Tech [Internet]. Wiley-Liss Inc.; 2018 [cited 2021 Mar 10];81:1310–7. Available from: https://pubmed.ncbi.nlm.nih.gov/30351463/.

  58. Arunachalam HB, Mishra R, Daescu O, Cederberg K, Rakheja D, Sengupta A, et al. Viable and necrotic tumor assessment from whole slide images of osteosarcoma using machine-learning and deep-learning models. PLoS One [Internet]. Public Library of Science; 2019 [cited 2021 Mar 10];14. Available from: https://pubmed.ncbi.nlm.nih.gov/30995247/.

  59. Mishra R, Daescu O, Leavey P, Rakheja D, Sengupta A. Convolutional neural network for histopathological analysis of osteosarcoma. J Comput Biol [Internet]. Mary Ann Liebert Inc.; 2018 [cited 2021 Mar 10]. p. 313–25. Available from: https://pubmed.ncbi.nlm.nih.gov/29083930/.

  60. Fu Y, Xue P, Ji H, Cui W, Dong E. Deep model with Siamese network for viable and necrotic tumor regions assessment in osteosarcoma. Med Phys [Internet]. John Wiley and Sons Ltd; 2020 [cited 2021 Mar 10];47:4895–905. Available from: https://pubmed.ncbi.nlm.nih.gov/32677073/.

  61. Doan M, Case M, Masic D, Hennig H, McQuin C, Caicedo J, et al. Label-free leukemia monitoring by computer vision. Cytom Part A [Internet]. Wiley-Liss Inc.; 2020 [cited 2021 Mar 10];97:407–14. Available from: /pmc/articles/PMC7213640/.

  62. Chaber R, Arthur CJ, Łach K, Raciborska A, Michalak E, Bilska K, et al. Predicting Ewing sarcoma treatment outcome using infrared spectroscopy and machine learning. Molecules [Internet]. MDPI AG; 2019 [cited 2021 Mar 12];24. Available from: https://pubmed.ncbi.nlm.nih.gov/30893786/.

  63. PATHFx [Internet]. [cited 2021 Mar 10]. Available from: https://www.pathfx.org/.

  64. Overmann AL, Clark DRM, Tsagkozis P, Wedin R, Forsberg JA. Validation of PATHFx 2.0: an open-source tool for estimating survival in patients undergoing pathologic fracture fixation. J Orthop Res [Internet]. John Wiley and Sons Inc; 2020 [cited 2021 Mar 10];38:2149–56. Available from: https://pubmed.ncbi.nlm.nih.gov/32492213/.

  65. Anderson AB, Wedin R, Fabbri N, Boland P, Healey J, Forsberg JA. External validation of PATHFx version 3.0 in patients treated surgically and nonsurgically for symptomatic skeletal metastases. Clin Orthop Relat Res [Internet]. Lippincott Williams and Wilkins; 2020 [cited 2021 Mar 11];478:808–18. Available from: https://pubmed.ncbi.nlm.nih.gov/32195761/.

  66. Ryu SM, Seo SW, Lee SH. Novel prognostication of patients with spinal and pelvic chondrosarcoma using deep survival neural networks. BMC Med Inform Decis Mak [Internet]. BioMed Central Ltd; 2020 [cited 2021 Mar 10];20. Available from: https://pubmed.ncbi.nlm.nih.gov/31907039/.

  67. Huang R, Xian S, Shi T, Yan P, Hu P, Yin H, et al. Evaluating and predicting the probability of death in patients with non-metastatic osteosarcoma: a population-based study. Med Sci Monit [Internet]. International Scientific Information, Inc.; 2019 [cited 2021 Mar 10];25:4675–90. Available from: https://pubmed.ncbi.nlm.nih.gov/31231119/.

  68. Yan P, Huang R, Hu P, Liu F, Zhu X, Hu P, et al. Nomograms for predicting the overall and cause-specific survival in patients with malignant peripheral nerve sheath tumor: a population-based study. J Neurooncol [Internet]. Springer New York LLC; 2019 [cited 2021 Mar 10];143:495–503. Available from: https://pubmed.ncbi.nlm.nih.gov/31089923/.

  69. Thio QCBS, Karhade A V., Ogink PT, Raskin KA, De Amorim Bernstein K, Lozano Calderon SA, et al. Can machine-learning techniques be used for 5-year survival prediction of patients with chondrosarcoma? Clin Orthop Relat Res [Internet]. Lippincott Williams and Wilkins; 2018 [cited 2021 Mar 10];476:2040–8. Available from: https://pubmed.ncbi.nlm.nih.gov/30179954/.

  70. Bongers MER, Thio QCBS, Karhade A V., Stor ML, Raskin KA, Lozano Calderon SA, et al. Does the SORG algorithm predict 5-year survival in patients with chondrosarcoma? An external validation. Clin Orthop Relat Res [Internet]. Lippincott Williams and Wilkins; 2019 [cited 2021 Mar 11];477:2296–303. Available from: https://pubmed.ncbi.nlm.nih.gov/31107338/.

  71. Bongers MER, Karhade A V., Setola E, Gambarotti M, Groot OQ, Erdoğan KE, et al. How does the skeletal oncology research group algorithm’s prediction of 5-year survival in patients with chondrosarcoma perform on international validation? Clin Orthop Relat Res [Internet]. NLM (Medline); 2020 [cited 2021 Mar 11];478:2300–8. Available from: https://pubmed.ncbi.nlm.nih.gov/32433107/.

  72. Kamalapathy PN, Ramkumar DB, Karhade A V, Kelly S, Raskin K, Schwab J, et al. Development of machine learning model algorithm for prediction of 5-year soft tissue myxoid liposarcoma survival. J Surg Oncol [Internet]. n/a. Available from: https://onlinelibrary.wiley.com/doi/abs/https://doi.org/10.1002/jso.26398.

  73. Seo SW, Kim J, Son J, Lim S. Evaluation of conditional treatment effects of adjuvant treatments on patients with synovial sarcoma using Bayesian subgroup analysis. BMC Med Inform Decis Mak [Internet]. BioMed Central Ltd; 2020 [cited 2021 Mar 11];20. Available from: https://pubmed.ncbi.nlm.nih.gov/33272256/.

  74. Chen W, Zhou C, Yan Z, Chen H, Lin K, Zheng Z, et al. Using machine learning techniques predicts prognosis of patients with Ewing sarcoma. J Orthop Res [Internet]. John Wiley and Sons Inc; 2021 [cited 2021 Mar 11]; Available from: https://pubmed.ncbi.nlm.nih.gov/33458857/.

  75. Huang Z, Hu C, Chi C, Jiang Z, Tong Y, Zhao C. An artificial intelligence model for predicting 1-year survival of bone metastases in non-small-cell lung cancer patients based on XGBoost algorithm. Biomed Res Int [Internet]. Hindawi Limited; 2020 [cited 2021 Mar 11];2020. Available from: https://pubmed.ncbi.nlm.nih.gov/32685470/.

  76. Thio QCBS, Karhade A V., Bindels BJJ, Ogink PT, Bramer JAM, Ferrone ML, et al. Development and internal validation of machine learning algorithms for preoperative survival prediction of extremity metastatic disease. Clin Orthop Relat Res [Internet]. NLM (Medline); 2020 [cited 2021 Mar 10];478:322–33. Available from: https://pubmed.ncbi.nlm.nih.gov/31651589/.

  77. Peeken JC, Goldberg T, Knie C, Komboz B, Bernhofer M, Pasa F, et al. Treatment-related features improve machine learning prediction of prognosis in soft tissue sarcoma patients. Strahlentherapie und Onkol [Internet]. Urban und Vogel GmbH; 2018 [cited 2021 Mar 10];194:824–34. Available from: https://pubmed.ncbi.nlm.nih.gov/29557486/.

  78. Wang Z, Wen X, Lu Y, Yao Y, Zhao H. Exploiting machine learning for predicting skeletal-related events in cancer patients with bone metastases. Oncotarget [Internet]. Impact Journals LLC; 2016 [cited 2021 Mar 10];7:12612–22. Available from: /pmc/articles/PMC4914308/.

  79. Pereira NRP, Janssen SJ, Van Dijk E, Harris MB, Hornicek FJ, Ferrone ML, et al. Development of a prognostic survival algorithm for patients with metastatic spine disease. J Bone Jt Surg - Am Vol [Internet]. Lippincott Williams and Wilkins; 2016 [cited 2021 Mar 11];98:1767–76. Available from: https://pubmed.ncbi.nlm.nih.gov/27807108/.

  80. Alcorn SR, Fiksel J, Wright JL, Elledge CR, Smith TJ, Perng P, et al. Developing an improved statistical approach for survival estimation in bone metastases management: the bone metastases ensemble trees for survival (BMETS) model. Int J Radiat Oncol Biol Phys [Internet]. Elsevier Inc.; 2020 [cited 2021 Mar 11];108:554–63. Available from: https://pubmed.ncbi.nlm.nih.gov/32446952/.

  81. Goldbraich E, Waks Z, Farkash A, Monti M, Torresani M, Bertulli R, et al. Understanding deviations from clinical practice guidelines in adult soft tissue sarcoma. Stud Health Technol Inform. 2015;216:280–4 (IOS Press).

    PubMed  Google Scholar 

  82. Coquet J, Bozkurt S, Kan KM, Ferrari MK, Blayney DW, Brooks JD, et al. Comparison of orthogonal NLP methods for clinical phenotyping and assessment of bone scan utilization among prostate cancer patients. J Biomed Inform [Internet]. Academic Press Inc.; 2019 [cited 2021 Mar 11];94. Available from: https://pubmed.ncbi.nlm.nih.gov/31014980/.

  83. Xuan P, Pan S, Zhang T, Liu Y, Sun H. Graph convolutional network and convolutional neural network based method for predicting lncRNA-disease associations. Cells [Internet]. NLM (Medline); 2019 [cited 2021 Mar 11];8. Available from: https://pubmed.ncbi.nlm.nih.gov/31480350/.

  84. Xuan P, Jia L, Zhang T, Sheng N, Li X, Li J. LDAPred: a method based on information flow propagation and a convolutional neural network for the prediction of disease-associated lncRNAs. Int J Mol Sci [Internet]. MDPI AG; 2019 [cited 2021 Mar 11];20. Available from: https://pubmed.ncbi.nlm.nih.gov/31510011/.

  85. Koelsche C, Schrimpf D, Stichel D, Sill M, Sahm F, Reuss DE, et al. Sarcoma classification by DNA methylation profiling. Nat Commun [Internet]. Nature Research; 2021 [cited 2021 Mar 11];12. Available from: https://pubmed.ncbi.nlm.nih.gov/33479225/.

  86. Chiechi A, Novello C, Magagnoli G, Petricoin EF, Deng J, Benassi MS, et al. Elevated TNFR1 and serotonin in bone metastasis are correlated with poor survival following bone metastasis diagnosis for both carcinoma and sarcoma primary tumors. Clin Cancer Res [Internet]. Clin Cancer Res; 2013 [cited 2021 Mar 12];19:2473–85. Available from: https://pubmed.ncbi.nlm.nih.gov/23493346/.

  87. Hill KE, Kelly AD, Kuijjer ML, Barry W, Rattani A, Garbutt CC, et al. An imprinted non-coding genomic cluster at 14q32 defines clinically relevant molecular subtypes in osteosarcoma across multiple independent datasets. J Hematol Oncol [Internet]. BioMed Central Ltd.; 2017 [cited 2021 Mar 12];10. Available from: https://pubmed.ncbi.nlm.nih.gov/28506242/.

  88. Zhu KP, Zhang CL, Ma XL, Hu JP, Cai T, Zhang L. Analyzing the interactions of mRNAs and ncRNAs to predict competing endogenous RNA networks in osteosarcoma chemo-resistance. Mol Ther [Internet]. Cell Press; 2019 [cited 2021 Mar 12];27:518–30. Available from: https://pubmed.ncbi.nlm.nih.gov/30692017/.

  89. Ren E hui, Deng Y jun, Yuan W hua, Wu Z long, Zhang G zhi, Xie Q qi. An immune-related gene signature for determining Ewing sarcoma prognosis based on machine learning. J Cancer Res Clin Oncol [Internet]. Springer Science and Business Media Deutschland GmbH; 2021 [cited 2021 Mar 12];147:153–65. Available from: https://pubmed.ncbi.nlm.nih.gov/32968877/.

  90. Liu J, Li R, Liao X, Jiang W. Comprehensive bioinformatic analysis genes associated to the prognosis of liposarcoma. Med Sci Monit [Internet]. International Scientific Information, Inc.; 2018 [cited 2021 Mar 12];24:7329–39. Available from: https://pubmed.ncbi.nlm.nih.gov/30317246/.

  91. van IJzendoorn DGP, Szuhai K, Briaire-De Bruijn IH, Kostine M, Kuijjer ML, Bovée JVMG. Machine learning analysis of gene expression data reveals novel diagnostic and prognostic biomarkers and identifies therapeutic targets for soft tissue sarcomas. PLoS Comput Biol [Internet]. Public Library of Science; 2019 [cited 2021 Mar 12];15. Available from: https://pubmed.ncbi.nlm.nih.gov/30785874/.

  92. Cabrera-Andrade A, López-Cortés A, Jaramillo-Koupermann G, González-Díaz H, Pazos A, Munteanu CR, et al. A multi-objective approach for anti-osteosarcoma cancer agents discovery through drug repurposing. Pharmaceuticals [Internet]. MDPI AG; 2020 [cited 2021 Mar 12];13:1–16. Available from: https://pubmed.ncbi.nlm.nih.gov/33266378/.

  93. Shen R, Li Z, Zhang L, Hua Y, Mao M, Li Z, et al. Osteosarcoma patients classification using plain X-rays and metabolomic data. Proc Annu Int Conf IEEE Eng Med Biol Soc EMBS [Internet]. Institute of Electrical and Electronics Engineers Inc.; 2018 [cited 2021 Mar 12]. p. 690–3. Available from: https://pubmed.ncbi.nlm.nih.gov/30440490/.

  94. Huang SC, Pareek A, Seyyedi S, Banerjee I, Lungren MP. Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines [Internet]. npj Digit. Med. Nature Research; 2020 [cited 2021 Jan 8]. p. 1–9. Available from: https://doi.org/https://doi.org/10.1038/s41746-020-00341-z.

  95. Richardson ML, Garwood ER, Lee Y, Li MD, Lo HS, Nagaraju A, et al. Noninterpretive uses of artificial intelligence in radiology. Acad Radiol. 2020;S1076–6332:30039–8.

    Google Scholar 

  96. Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, et al. Transformers: state-of-the-art natural language processing. Proc 2020 Conf Empir Methods Nat Lang Process Syst Demonstr [Internet]. Stroudsburg, PA, USA: Association for Computational Linguistics; 2020 [cited 2021 Jan 4]. p. 38–45. Available from: https://www.aclweb.org/anthology/2020.emnlp-demos.6.

  97. Lundervold AS, Lundervold A. An overview of deep learning in medical imaging focusing on MRI [Internet]. Z. Med. Phys. Elsevier GmbH; 2019 [cited 2021 Mar 12]. p. 102–27. Available from: https://pubmed.ncbi.nlm.nih.gov/30553609/.

  98. Yang J, Sohn JH, Behr SC, Gullberg GT, Seo Y. CT-less direct correction of attenuation and scatter in image space using deep learning for whole-body FDG PET: potential benefits and pitfalls. Radiol Artif Intell [Internet]. Radiological Society of North America (RSNA); 2020 [cited 2021 Mar 12];e200137. Available from: https://pubs.rsna.org/doi/abs/https://doi.org/10.1148/ryai.2020200137.

  99. Li MD, Chang K, Bearce B, Chang CY, Huang AJ, Campbell JP, et al. Siamese neural networks for continuous disease severity evaluation and change detection in medical imaging. npj Digit Med [Internet]. Springer Science and Business Media LLC; 2020 [cited 2020 Jun 20];3:1–9. Available from: https://www.nature.com/articles/s41746-020-0255-1.

  100. Lee CS, Lee AY. Clinical applications of continual learning machine learning [Internet]. Lancet Digit. Heal. Elsevier Ltd; 2020 [cited 2021 Mar 12]. p. e279–81. Available from: www.thelancet.com/digital-health.

Download references

Author information

Authors and Affiliations

  1. Division of Musculoskeletal Imaging and Intervention, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA

    Matthew D. Li & Connie Y. Chang

  2. Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA

    Matthew D. Li, Syed Rakin Ahmed & Jayashree Kalpathy-Cramer

  3. Harvard Medical School, Harvard Graduate Program in Biophysics, Harvard University, Cambridge, MA, USA

    Syed Rakin Ahmed

  4. Geisel School of Medicine At Dartmouth, Dartmouth College, Hanover, NH, USA

    Syed Rakin Ahmed

  5. Division of Hematology Oncology, Department of Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA

    Edwin Choy

  6. Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA

    Santiago A. Lozano-Calderon

Authors
  1. Matthew D. Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Syed Rakin Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edwin Choy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Santiago A. Lozano-Calderon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jayashree Kalpathy-Cramer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Connie Y. Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew D. Li.

Ethics declarations

Conflict of interest

MDL reports a grant from the Radiological Society of North America, outside of the submitted work. JK reports grants from GE Healthcare, a non-financial support from Amazon Web Services, and grants from Genentech Foundation, outside the submitted work. The other authors report no relevant conflict of interest.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Summary statement

This systematic review of artificial intelligence applied to musculoskeletal oncology characterizes the published primary literature, including studies involving data from radiology, nuclear medicine, pathology, clinical research, and molecular biology.

Key points

• Deep learning applied to musculoskeletal oncology research predominantly involves bone scintigraphy for the evaluation of bone metastases, histopathology for the analysis of bone marrow aspirates and other tissues, and various applications for the analysis of radiographs, CT, and MRI studies. Deep learning has rarely been applied to clinical or molecular datasets, though more traditional machine learning approaches are commonly used for those data types.

• There are many different types of data used in musculoskeletal oncology artificial intelligence research, though these are seldom used in combination. The integration of this data could be facilitated by multidisciplinary collaboration between the imaging-focused specialties (radiology and pathology) and clinical specialties (orthopedic oncology surgery, medical oncology, and radiation oncology).

• Few published studies focus on non-interpretive tasks in musculoskeletal oncology, such as those related to the patient experience or physician workflow. This may be an area for future artificial intelligence research that can make an immediate impact on patient care.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, M.D., Ahmed, S.R., Choy, E. et al. Artificial intelligence applied to musculoskeletal oncology: a systematic review. Skeletal Radiol 51, 245–256 (2022). https://doi.org/10.1007/s00256-021-03820-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00256-021-03820-w

Keywords

Search

Navigation