Skip to main content

Advertisement

SpringerLink
Log in

Augmenting Osteoporosis Imaging with Machine Learning

  • IMAGING (H ISAKSSON AND S BOYD, SECTION EDITORS)
  • Published:
Current Osteoporosis Reports Aims and scope Submit manuscript

Abstract

Purpose of Review

In this paper, we discuss how recent advancements in image processing and machine learning (ML) are shaping a new and exciting era for the osteoporosis imaging field. With this paper, we want to give the reader a basic exposure to the ML concepts that are necessary to build effective solutions for image processing and interpretation, while presenting an overview of the state of the art in the application of machine learning techniques for the assessment of bone structure, osteoporosis diagnosis, fracture detection, and risk prediction.

Recent Findings

ML effort in the osteoporosis imaging field is largely characterized by “low-cost” bone quality estimation and osteoporosis diagnosis, fracture detection, and risk prediction, but also automatized and standardized large-scale data analysis and data-driven imaging biomarker discovery.

Summary

Our effort is not intended to be a systematic review, but an opportunity to review key studies in the recent osteoporosis imaging research landscape with the ultimate goal of discussing specific design choices, giving the reader pointers to possible solutions of regression, segmentation, and classification tasks as well as discussing common mistakes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

Not applicable.

Code Availability

Not applicable.

References

Papers of particular interest, published recently, have been highlighted as: •• Of major importance

  1. van Oostwaard M Osteoporosis and the nature of fragility fracture: an overview. In: Hertz K, Santy-Tomlinson J, editors. Fragility fracture nursing: holistic care and management of the orthogeriatric patient. Cham (CH)2018. p. 1-13.

  2. Lewiecki EM, Lane NE. Common mistakes in the clinical use of bone mineral density testing. Nat Clin Pract Rheumatol. 2008;4(12):667–74. https://doi.org/10.1038/ncprheum0928.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Compston J, Cooper A, Cooper C, Gittoes N, Gregson C, Harvey N, Hope S, Kanis JA, McCloskey EV, Poole KES, Reid DM, Selby P, Thompson F, Thurston A, Vine N, National Osteoporosis Guideline G. UK clinical guideline for the prevention and treatment of osteoporosis. Arch Osteoporos. 2017;12(1):43. https://doi.org/10.1007/s11657-017-0324-5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Han W, Qin L, Bay C, Chen X, Yu KH, Miskin N, Li A, Xu X, Young G. Deep transfer learning and radiomics feature prediction of survival of patients with high-grade gliomas. AJNR Am J Neuroradiol. 2020;41(1):40–8. https://doi.org/10.3174/ajnr.A6365.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44. https://doi.org/10.1038/nature14539.

    Article  CAS  PubMed  Google Scholar 

  6. Bengio Y, Courville A, Vincent P. Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell. 2013;35(8):1798–828. https://doi.org/10.1109/TPAMI.2013.50.

    Article  PubMed  Google Scholar 

  7. Bengio Y, Lee H. Editorial introduction to the neural networks special issue on deep learning of representations. Neural Netw. 2015;64:1–3. https://doi.org/10.1016/j.neunet.2014.12.006.

    Article  PubMed  Google Scholar 

  8. Smets J, Shevroja E, Hugle T, Leslie WD, Hans D. Machine learning solutions for osteoporosis-a review. J Bone Miner Res. 2021. https://doi.org/10.1002/jbmr.4292.

  9. Fang Y, Li W, Chen X, Chen K, Kang H, Yu P, Zhang R, Liao J, Hong G, Li S. Opportunistic osteoporosis screening in multi-detector CT images using deep convolutional neural networks. Eur Radiol. 2021;31(4):1831–42. https://doi.org/10.1007/s00330-020-07312-8This is a fully automatic pipeline for vertebral segmentation and BMD assessment for opportunistic osteoporosis screening. Well-designed and relevant study.

    Article  PubMed  Google Scholar 

  10. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention, Pt Iii. 2015;9351:234-41. doi: https://doi.org/10.1007/978-3-319-24574-4_28.

  11. Byra M, Wu M, Zhang X, Jang H, Ma YJ, Chang EY, Shah S, Du J. Knee menisci segmentation and relaxometry of 3D ultrashort echo time cones MR imaging using attention U-Net with transfer learning. Magn Reson Med. 2020;83(3):1109–22. https://doi.org/10.1002/mrm.27969.

    Article  PubMed  Google Scholar 

  12. Bagheri MH, Roth H, Kovacs W, Yao J, Farhadi F, Li X, Summers RM. Technical and clinical factors affecting success rate of a deep learning method for pancreas segmentation on CT. Acad Radiol. 2019. https://doi.org/10.1016/j.acra.2019.08.014

  13. Curtis SL, Norman BP, Milan AM, Gallagher JA, Olsson B, Ranganath LR, Roberts NB. Interference of hydroxyphenylpyruvic acid, hydroxyphenyllactic acid and tyrosine on routine serum and urine clinical chemistry assays; implications for biochemical monitoring of patients with alkaptonuria treated with nitisinone. Clin Biochem. 2019;71:24–30. https://doi.org/10.1016/j.clinbiochem.2019.06.010.

    Article  CAS  PubMed  Google Scholar 

  14. Pang S, Su Z, Leung S, Nachum IB, Chen B, Feng Q, Li S. Direct automated quantitative measurement of spine by cascade amplifier regression network with manifold regularization. Med Image Anal. 2019;55:103–15. https://doi.org/10.1016/j.media.2019.04.012.

    Article  PubMed  Google Scholar 

  15. Xiao P, Zhang T, Dong XN, Han Y, Huang Y, Wang X. Prediction of trabecular bone architectural features by deep learning models using simulated DXA images. Bone Rep. 2020;13:100295. https://doi.org/10.1016/j.bonr.2020.100295.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Ruder S. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:170605098, 2017. 2017.

  17. Lee KS, Jung SK, Ryu JJ, Shin SW, Choi J. Evaluation of transfer learning with deep convolutional neural networks for screening osteoporosis in dental panoramic radiographs. J Clin Med. 2020;9(2). https://doi.org/10.3390/jcm9020392.

  18. Miotto R, Wang F, Wang S, Jiang X, Dudley JT. Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform. 2018;19(6):1236–46. https://doi.org/10.1093/bib/bbx044.

    Article  PubMed  Google Scholar 

  19. Warman A, Warman P, Sharma A, Parikh P, Warman R, Viswanadhan N, Chen L, Mohapatra S, Mohapatra S, Sapiro G. Interpretable artificial intelligence for COVID-19 diagnosis from chest CT reveals specificity of ground-glass opacities. medRxiv. 2020. https://doi.org/10.1101/2020.05.16.20103408.

  20. Wang F, Kaushal R, Khullar D. Should health care demand interpretable artificial intelligence or accept "black box" medicine? Ann Intern Med. 2020;172(1):59–60. https://doi.org/10.7326/M19-2548.

    Article  PubMed  Google Scholar 

  21. Rotemberg V, Halpern A. Towards 'interpretable' artificial intelligence for dermatology. Br J Dermatol. 2019;181(1):5–6. https://doi.org/10.1111/bjd.18038.

    Article  CAS  PubMed  Google Scholar 

  22. Yamamoto N, Sukegawa S, Kitamura A, Goto R, Noda T, Nakano K, Takabatake K, Kawai H, Nagatsuka H, Kawasaki K, Furuki Y, Ozaki T. Deep learning for osteoporosis classification using hip radiographs and patient clinical covariates. Biomolecules. 2020;10(11). https://doi.org/10.3390/biom10111534.

  23. Zhang B, Yu K, Ning Z, Wang K, Dong Y, Liu X, Liu S, Wang J, Zhu C, Yu Q, Duan Y, Lv S, Zhang X, Chen Y, Wang X, Shen J, Peng J, Chen Q, Zhang Y, et al. Deep learning of lumbar spine X-ray for osteopenia and osteoporosis screening: a multicenter retrospective cohort study. Bone. 2020;140:115561. https://doi.org/10.1016/j.bone.2020.115561.

    Article  CAS  PubMed  Google Scholar 

  24. Olczak J, Fahlberg N, Maki A, Razavian AS, Jilert A, Stark A, Skoldenberg O, Gordon M. Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop. 2017;88(6):581–6. https://doi.org/10.1080/17453674.2017.1344459.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Lindsey R, Daluiski A, Chopra S, Lachapelle A, Mozer M, Sicular S, Hanel D, Gardner M, Gupta A, Hotchkiss R, Potter H. Deep neural network improves fracture detection by clinicians. Proc Natl Acad Sci U S A. 2018;115(45):11591–6. https://doi.org/10.1073/pnas.1806905115.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Badgeley MA, Zech JR, Oakden-Rayner L, Glicksberg BS, Liu M, Gale W, MV MC, Percha B, Snyder TM, Dudley JT. Deep learning predicts hip fracture using confounding patient and healthcare variables. NPJ Digit Med. 2019;2:31. https://doi.org/10.1038/s41746-019-0105-1Largest study on fracture detection. The in-depth analysis of the feature is interesting and well executed. It proposed solution for multimodal data integration: imaging and clinical.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Krogue JD, Cheng KV, Hwang K, Toogood P, Meinberg E, Geiger E, Zaid M, Ozhinsky E, Majumdar S, Pedoia V. Automatic Hip fracture identification and functional subclassification with deep learning. Radiol: Artif Intell. 2020;2(2).

  28. Murata K, Endo K, Aihara T, Suzuki H, Sawaji Y, Matsuoka Y, Nishimura H, Takamatsu T, Konishi T, Maekawa A, Yamauchi H, Kanazawa K, Endo H, Tsuji H, Inoue S, Fukushima N, Kikuchi H, Sato H, Yamamoto K. Artificial intelligence for the detection of vertebral fractures on plain spinal radiography. Sci Rep. 2020;10(1):20031. https://doi.org/10.1038/s41598-020-76866-w.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Almog YA, Rai A, Zhang P, Moulaison A, Powell R, Mishra A, Weinberg K, Hamilton C, Oates M, McCloskey E, Cummings SR. Deep learning with electronic health records for short-term fracture risk identification: crystal bone algorithm development and validation. J Med Internet Res. 2020;22(10):e22550. https://doi.org/10.2196/22550.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Muller R. Hierarchical microimaging of bone structure and function. Nat Rev Rheumatol. 2009;5(7):373–81. https://doi.org/10.1038/nrrheum.2009.107.

    Article  PubMed  Google Scholar 

  31. Cheung AM, Adachi JD, Hanley DA, Kendler DL, Davison KS, Josse R, Brown JP, Ste-Marie LG, Kremer R, Erlandson MC, Dian L, Burghardt AJ, Boyd SK. High-resolution peripheral quantitative computed tomography for the assessment of bone strength and structure: a review by the Canadian Bone Strength Working Group. Curr Osteoporos Rep. 2013;11(2):136–46. https://doi.org/10.1007/s11914-013-0140-9.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Langsetmo L, Peters KW, Burghardt AJ, Ensrud KE, Fink HA, Cawthon PM, Cauley JA, Schousboe JT, Barrett-Connor E, Orwoll ES. Osteoporotic Fractures in Men Study Research G. Volumetric bone mineral density and failure load of distal limbs predict incident clinical fracture independent HR-pQCT BMD and failure load predicts incident clinical fracture of FRAX and clinical risk factors among older men. J Bone Miner Res. 2018;33(7):1302–11. https://doi.org/10.1002/jbmr.3433.

    Article  CAS  PubMed  Google Scholar 

  33. Mikolajewicz N, Bishop N, Burghardt AJ, Folkestad L, Hall A, Kozloff KM, Lukey PT, Molloy-Bland M, Morin SN, Offiah AC, Shapiro J, van Rietbergen B, Wager K, Willie BM, Komarova SV, Glorieux FH. HR-pQCT measures of bone microarchitecture predict fracture: systematic review and meta-analysis. J Bone Miner Res. 2020;35(3):446–59. https://doi.org/10.1002/jbmr.3901.

    Article  PubMed  Google Scholar 

  34. Li Y, Sixou B, Burghard B, Peyrin F. Investigation of semi-coupled dictionary learning in 3-D super resolution HR-PQCT imaging. IEEE Transactions on Radiation and Plasma Medical Sciences. 2019;3(2).

  35. Guha I, Nadeem SA, You C, Zhang X, Levy SM, Wang G, Torner JC, Saha PK. Deep learning based high-resolution reconstruction of trabecular bone microstructures from low-resolution CT scans using GAN-CIRCLE. Proc SPIE Int Soc Opt Eng. 2020:11317. https://doi.org/10.1117/12.2549318.

  36. Karasik D, Demissie S, Zhou Y, Lu D, Broe KE, Bouxsein ML, Cupples LA, Kiel DP. Heritability and genetic correlations for bone microarchitecture: the Framingham study families. J Bone Miner Res. 2017;32(1):106–14. https://doi.org/10.1002/jbmr.2915.

    Article  PubMed  Google Scholar 

  37. Biver E, Durosier-Izart C, Chevalley T, van Rietbergen B, Rizzoli R, Ferrari S. Evaluation of radius microstructure and areal bone mineral density improves fracture prediction in postmenopausal women. J Bone Miner Res. 2018;33(2):328–37. https://doi.org/10.1002/jbmr.3299.

    Article  PubMed  Google Scholar 

  38. Atkinson EJ, Therneau TM, Melton LJ 3rd, Camp JJ, Achenbach SJ, Amin S, Khosta S. Assessing fracture risk using gradient boosting machine (GBM) models. J Bone Miner Res. 2012;27(6):1397–404. https://doi.org/10.1002/jbmr.1577This study shows the usage of statistical multi-parametric modeling for fracture discrimination interesting applications and extremely relevant for model interpretability.

    Article  PubMed  Google Scholar 

  39. Treece G, Gee A. Cortical bone mapping: measurement and statistical analysis of localised skeletal changes. Curr Osteoporos Rep. 2018;16(5):617–25. https://doi.org/10.1007/s11914-018-0475-3.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Carballido-Gamio J, Yu A, Wang L, Su Y, Burghardt AJ, Lang TF, Cheng X. Hip fracture discrimination based on statistical multi-parametric modeling (SMPM). Ann Biomed Eng. 2019;47(11):2199–212. https://doi.org/10.1007/s10439-019-02298-x.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Treece GM, Gee AH, Tonkin C, Ewing SK, Cawthon PM, Black DM, Poole KE. Osteoporotic Fractures in Men S. Predicting hip fracture type with cortical bone mapping (CBM) in the Osteoporotic Fractures in Men (MrOS) Study. J Bone Miner Res. 2015;30(11):2067–77. https://doi.org/10.1002/jbmr.2552.

    Article  PubMed  Google Scholar 

  42. Carballido-Gamio J, Bonaretti S, Kazakia GJ, Khosla S, Majumdar S, Lang TF, Burghardt AJ. Statistical parametric mapping of HR-pQCT images: a tool for population-based local comparisons of micro-scale bone features. Ann Biomed Eng. 2017;45(4):949–62. https://doi.org/10.1007/s10439-016-1754-8.

    Article  PubMed  Google Scholar 

  43. Kogan F, Broski SM, Yoon D, Gold GE. Applications of PET-MRI in musculoskeletal disease. J Magn Reson Imaging. 2018;48(1):27–47. https://doi.org/10.1002/jmri.26183.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Kogan F, Fan AP, Monu U, Iagaru A, Hargreaves BA, Gold GE. Quantitative imaging of bone-cartilage interactions in ACL-injured patients with PET-MRI. Osteoarthr Cartil. 2018;26(6):790–6. https://doi.org/10.1016/j.joca.2018.04.001.

    Article  CAS  Google Scholar 

  45. Tibrewala R, Pedoia V, Bucknor M, Majumdar S. Principal component analysis of simultaneous PET-MRI reveals patterns of bone-cartilage interactions in osteoarthritis. J Magn Reson Imaging. 2020;52:1462–74. https://doi.org/10.1002/jmri.27146.

    Article  PubMed  Google Scholar 

  46. Wu PH, Gibbons M, Foreman SC, Carballido-Gamio J, Han M, Krug R, Liu J, Link TM, Kazakia GJ. Cortical bone vessel identification and quantification on contrast-enhanced MR images. Quant Imaging Med Surg. 2019;9(6):928–41. https://doi.org/10.21037/qims.2019.05.23.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Radiology and Biomedical Imaging, University of California San Francisco (UCSF), 1700 Fourth Street, Suite 201, QB3 Building, San Francisco, CA, 94158, USA

    Valentina Pedoia, Francesco Caliva, Galateia Kazakia, Andrew J. Burghardt & Sharmila Majumdar

Authors
  1. Valentina Pedoia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesco Caliva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Galateia Kazakia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrew J. Burghardt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sharmila Majumdar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Valentina Pedoia.

Ethics declarations

Ethics Approval

Not applicable.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Competing Interests

The authors do not have conflicts of interests with the material presented in this manuscript.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Imaging

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pedoia, V., Caliva, F., Kazakia, G. et al. Augmenting Osteoporosis Imaging with Machine Learning. Curr Osteoporos Rep 19, 699–709 (2021). https://doi.org/10.1007/s11914-021-00701-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11914-021-00701-y

Keywords

Search

Navigation