Skip to main content
SpringerLink
Log in

External validation of a deep learning model for predicting bone mineral density on chest radiographs

  • Published:
Archives of Osteoporosis Aims and scope Submit manuscript

Abstract

Summary

We developed a new model for predicting bone mineral density on chest radiographs and externally validated it using images captured at facilities other than the development environment. The model performed well and showed potential for clinical use.

Purpose

In this study, we performed external validation (EV) of a developed deep learning model for predicting bone mineral density (BMD) of femoral neck on chest radiographs to verify the usefulness of this model in clinical practice.

Methods

This study included patients who visited any of the collaborating facilities from 2010 to 2020 and underwent chest radiography and dual-energy X-ray absorptiometry (DXA) at the femoral neck in the year before and after their visit. A total of 50,114 chest radiographs were obtained, and BMD was measured using DXA. We developed the model with 47,150 images from 17 facilities and performed EV with 2914 images from three other facilities (EV dataset). We trained the deep learning model via ensemble learning based on chest radiographs, age, and sex to predict BMD using regression. The outcomes were the correlation of the predicted BMD and measured BMD with diagnoses of osteoporosis and osteopenia using the T-score estimated from the predicted BMD.

Results

The mean BMD was 0.64±0.14 g/cm2 in the EV dataset. The BMD predicted by the model averaged 0.61±0.08 g/cm2, with a correlation coefficient of 0.68 (p<0.01) when compared with the BMD measured using DXA. The accuracy, sensitivity, and specificity of the model were 79.0%, 96.6%, and 34.1% for T-score < -1 and 79.7%, 77.1%, and 80.4% for T-score ≤ -2.5, respectively.

Conclusion

Our model, which was externally validated using data obtained at facilities other than the development environment, predicted BMD of femoral neck on chest radiographs. The model performed well and showed potential for clinical use.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author, YT, upon reasonable request.

References

  1. Salari N, Ghasemi H, Mohammadi L, Behzadi MH, Rabieenia E, Shohaimi S, Mohammadi M (2021) The global prevalence of osteoporosis in the world: a comprehensive systematic review and meta-analysis. J Orthop Surg Res 16:609. https://doi.org/10.1186/s13018-021-02772-0

    Article  PubMed  PubMed Central  Google Scholar 

  2. Wade SW, Strader C, Fitzpatrick LA, Anthony MS, O'Malley CD (2014) Estimating prevalence of osteoporosis: examples from industrialized countries. Arch Osteoporos 9:182. https://doi.org/10.1007/s11657-014-0182-3

    Article  CAS  PubMed  Google Scholar 

  3. Barrett-Connor E (1995) The economic and human costs of osteoporotic fracture. Am J Med 98:3S–8S. https://doi.org/10.1016/s0002-9343(05)80037-3

    Article  CAS  PubMed  Google Scholar 

  4. Burge R, Dawson-Hughes B, Solomon DH, Wong JB, King A, Tosteson A (2007) Incidence and economic burden of osteoporosis-related fractures in the United States, 2005-2025. J Bone Miner Res 22:465–475. https://doi.org/10.1359/jbmr.061113

    Article  PubMed  Google Scholar 

  5. Johnell O (1997) The socioeconomic burden of fractures: today and in the 21st century. Am J Med 103:20S–25S; discussion 25S-26S. https://doi.org/10.1016/s0002-9343(97)90023-1

    Article  CAS  PubMed  Google Scholar 

  6. Rachner TD, Khosla S, Hofbauer LC (2011) Osteoporosis: now and the future. Lancet 377:1276–1287. https://doi.org/10.1016/S0140-6736(10)62349-5

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. World Health Organization (2023) WHO criteria for diagnosis of osteoporosis. 4BoneHealth. http://www.4bonehealth.org/education/world-health-organization-criteria-diagnosis-osteoporosis/2023.5.8. Accessed 10 Aug 2023

  8. Bolotin HH, Sievänen H (2001) Inaccuracies inherent in dual-energy X-ray absorptiometry in vivo bone mineral density can seriously mislead diagnostic/prognostic interpretations of patient-specific bone fragility. J Bone Miner Res 16:799–805. https://doi.org/10.1359/jbmr.2001.16.5.799

    Article  CAS  PubMed  Google Scholar 

  9. Mueller D, Gandjour A (2009) Cost-effectiveness of using clinical risk factors with and without DXA for osteoporosis screening in postmenopausal women. Value Health 12:1106–1117. https://doi.org/10.1111/j.1524-4733.2009.00577.x

    Article  PubMed  Google Scholar 

  10. Jang M, Kim M, Bae SJ, Lee SH, Koh JM, Kim N (2022) Opportunistic osteoporosis screening using chest radiographs with deep learning: development and external validation with a cohort dataset. J Bone Miner Res 37:369–377. https://doi.org/10.1002/jbmr.4477

    Article  PubMed  Google Scholar 

  11. Yamamoto N, Sukegawa S, Kitamura A, Goto R, Noda T, Nakano K, Takabatake K, Kawai H, Nagatsuka H, Kawasaki K, Furuki Y, Ozaki T (2020) Deep learning for osteoporosis classification using hip radiographs and patient clinical covariates. Biomolecules 10:1534. https://doi.org/10.3390/biom10111534

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Widyaningrum R, Sela EI, Pulungan R, Septiarini A (2023) Automatic segmentation of periapical radiograph using color histogram and machine learning for osteoporosis detection. Int J Dent 2023:6662911. https://doi.org/10.1155/2023/6662911

    Article  PubMed  PubMed Central  Google Scholar 

  13. Zhang B, Yu K, Ning Z, Wang K, Dong Y, Liu X, Liu S, Wang J, Zhu C, Yu Q, Duan Y, Lv S, Zhang X, Chen Y, Wang X, Shen J, Peng J, Chen Q, Zhang Y et al (2021) Corrigendum to “Deep learning of lumbar spine X-ray for osteopenia and osteoporosis screening: a multicenter retrospective cohort study” [Bone 140, November 2020, 115561]. Bone 153:116143. https://doi.org/10.1016/j.bone.2021.116143

    Article  PubMed  Google Scholar 

  14. Wang F, Zheng K, Lu L, Xiao J, Wu M, Kuo CF, Miao S (2023) Lumbar bone mineral density estimation from chest X-ray images: anatomy-aware attentive multi-ROI modeling. IEEE Trans Med Imaging 42:257–267. https://doi.org/10.1109/TMI.2022.3209648

    Article  CAS  PubMed  Google Scholar 

  15. Hsieh CI, Zheng K, Lin C, Mei L, Lu L, Li W, Chen FP, Wang Y, Zhou X, Wang F, Xie G, Xiao J, Miao S, Kuo CF (2021) Automated bone mineral density prediction and fracture risk assessment using plain radiographs via deep learning. Nat Commun 12:5472. https://doi.org/10.1038/s41467-021-25779-x

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  16. Sato Y, Yamamoto N, Inagaki N, Iesaki Y, Asamoto T, Suzuki T, Takahara S (2022) Deep learning for bone mineral density and T-score prediction from chest X-rays: a multicenter study. Biomedicines 10:2323. https://doi.org/10.3390/biomedicines10092323

    Article  PubMed  PubMed Central  Google Scholar 

  17. Adams M, Chen W, Holcdorf D, McCusker MW, Howe PD, Gaillard F (2019) Computer vs human: deep learning versus perceptual training for the detection of neck of femur fractures. J Med Imaging Radiat Oncol 63:27–32. https://doi.org/10.1111/1754-9485.12828

    Article  PubMed  Google Scholar 

  18. Altman DG, Vergouwe Y, Royston P, Moons KG (2009) Prognosis and prognostic research: validating a prognostic model. BMJ 338:b605. https://doi.org/10.1136/bmj.b605

    Article  PubMed  Google Scholar 

  19. Chlap P, Min H, Vandenberg N, Dowling J et al (2021) A review of medical image data augmentation techniques for deep learning applications. J Med Imaging Radiat Oncol 65(5):545–563. https://doi.org/10.1111/1754-9485.13261 Epub 2021 Jun 19

    Article  PubMed  Google Scholar 

  20. Tank VH, Ghosh R, Gupta V, Sheth N, Gordon S, He W, Modica SF, Prestigiacomo CJ, Gandhi CD (2016) Drug eluting stents versus bare metal stents for the treatment of extracranial vertebral artery disease: a meta-analysis. J Neurointerv Surg 8:770–774. https://doi.org/10.1136/neurintsurg-2015-011697

    Article  PubMed  Google Scholar 

  21. Kanda Y (2013) Investigation of the freely available easy-to-use software 'EZR' for medical statistics. Bone Marrow Transplant 48:452–458. https://doi.org/10.1038/bmt.2012.244

    Article  CAS  PubMed  Google Scholar 

  22. Mounach A, Abayi DA, Ghazi M et al (2009) Discordance between hip and spine bone mineral density measurement using DXA: prevalence and risk factors. Semin Arthritis Rheum 38(6):467–471. https://doi.org/10.1016/j.semarthrit.2008.04.001 Epub 2008 Jun 24

    Article  CAS  PubMed  Google Scholar 

  23. Lips P, van Schoor NM (2005) Quality of life in patients with osteoporosis. Osteoporos Int 16:447–455. https://doi.org/10.1007/s00198-004-1762-7)

    Article  PubMed  Google Scholar 

  24. Lorentzon M (2019) Treating osteoporosis to prevent fractures: current concepts and future developments. J Intern Med 285(4):381–394. https://doi.org/10.1111/joim.12873. Epub 2019 Jan 18 https://onlinelibrary.wiley.com/journal/13652796

    Article  PubMed  Google Scholar 

  25. Ramspek CL, Jager KJ, Dekker FW, Zoccali C, van Diepen M (2021) External validation of prognostic models: what, why, how, when and where. Clin Kidney J 14:49–58. https://doi.org/10.1093/ckj/sfaa188

    Article  PubMed  Google Scholar 

  26. Collins GS, Moons KGM (2019) Reporting of artificial intelligence prediction models. Lancet 393:1577–1579. https://doi.org/10.1016/S0140-6736(19)30037-6

    Article  PubMed  Google Scholar 

  27. Snell KIE, Archer L, Ensor J, Bonnett LJ, Debray TPA, Phillips B, Collins GS, Riley RD (2021) External validation of clinical prediction models: simulation-based sample size calculations were more reliable than rules-of-thumb. J Clin Epidemiol 135:79–89. https://doi.org/10.1016/j.jclinepi.2021.02.011

    Article  PubMed  PubMed Central  Google Scholar 

  28. Mukaka MM (2012) Statistics corner: a guide to appropriate use of correlation coefficient in medical research. Malawi Med J 24:69–71

    CAS  PubMed  PubMed Central  Google Scholar 

  29. Cetin A, Ertürk H, Celiker R et al (2001) The role of quantitative ultrasound in predicting osteoporosis defined by dual X-ray bsorptiometry. Rheumatol Int 20(2):55–59. https://doi.org/10.1007/pl00006857

    Article  CAS  PubMed  Google Scholar 

  30. Clowes JA, Peel NFA, Eastell R (2006) Device-specific thresholds to diagnose osteoporosis at the proximal femur: an approach to interpreting peripheral bone measurements in clinical practice. Osteoporos Int 17:1293–1302. https://doi.org/10.1007/s00198-006-0122-1

    Article  CAS  PubMed  Google Scholar 

  31. Gemalmaz A, Discigil G, Sensoy N et al (2007) Identifying osteoporosis in a primary care setting with quantitative ultrasound: relationship to anthropometric and lifestyle factors. J Bone Miner Metab 25(3):184–192. https://doi.org/10.1007/s00774-006-0741-9 Epub 2007 Apr 20

    Article  PubMed  Google Scholar 

  32. Trimpou P, Bosaeus I, Bengtsson BA et al (2010) High correlation between quantitative ultrasound and DXA during 7 years of follow-up. Eur J Radiol 73(2):360–364. https://doi.org/10.1016/j.ejrad.2008.11.024 Epub 2009 Jan 8. https://www.sciencedirect.com/science/article/pii/S0720048X08006554

    Article  PubMed  Google Scholar 

  33. Oral A, Esmaeilzadeh S, Yalıman A et al (2019) The ability of calcaneal and multisite quantitative ultrasound variables in the identification of osteoporosis in women and men. Turk J Phys Med Rehabil 65(3):203–215. https://doi.org/10.5606/tftrd.2019.1894 PMID: 31663068; PMCID: PMC6797920. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6797920/

    Article  PubMed  PubMed Central  Google Scholar 

  34. Chanprasertpinyo W, Punsawad C, Khwanchuea R et al (2023) Comparison between calcaneus quantitative ultrasound and the gold standard DXA in the ability to detect osteoporosis in chronic obstructive pulmonary disease patients. J Orthop Surg Res 18(1):778. https://doi.org/10.1186/s13018-023-04211-8 Erratum in: J Orthop Surg Res. 2023 Nov 15; 18(1):868. PMID: 37845656; PMCID: PMC10577968

    Article  PubMed  PubMed Central  Google Scholar 

  35. Jones T, Davie MW (1998) Bone mineral density at distal forearm can identify patients with osteoporosis at spine or femoral neck. Br J Rheumatol 37(5):539–543. https://doi.org/10.1093/rheumatology/37.5.539https://academic.oup.com/rheumatology/article/37/5/539/1783093?login=true

    Article  CAS  PubMed  Google Scholar 

  36. Azami A, Anari H, Iranparvar M et al (2019) Comparison of bone mineral densitometry at 2 sites versus 3 sites in patients suspicious for osteoporosis. Clin Med Insights Arthritis Musculoskelet Disord:12. https://doi.org/10.1177/1179544119849017

  37. Yue C, Ding N, Xu LL et al (2022) Prescreening for osteoporosis with forearm bone densitometry in health examination population. BMC Musculoskelet Disord 23(1):377. https://doi.org/10.1186/s12891-022-05325-6 PMID: 35459140; PMCID: PMC9027342

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Adami S, Zamberlan N, Gatti D et al (1996) Computed radiographic absorptiometry and morphometry in the assessment of postmenopausal bone loss. Osteoporos Int 6(1):8–13. https://doi.org/10.1007/BF01626531

    Article  CAS  PubMed  Google Scholar 

  39. Dey A, McCloskey EV, Taube T et al (2000) Metacarpal morphometry using a semi-automated technique in the assessment of osteoporosis and vertebral fracture risk. Osteoporos Int 11(11):953–958. https://doi.org/10.1007/s001980070034.B4:B32https://link.springer.com/article/10.1007/s001980070034

    Article  CAS  PubMed  Google Scholar 

  40. Hyldstrup L, Nielsen SP (2001) Metacarpal index by digital X-ray radiogrammetry: normative reference values and comparison with dual X-ray absorptiometry. J Clin Densitom 4(4):299–306. https://doi.org/10.1385/jcd:4:4:299https://www.sciencedirect.com/science/article/pii/S1094695006603624

    Article  CAS  PubMed  Google Scholar 

  41. Rosholm A, Hyldstrup L, Backsgaard L et al (2001) Estimation of bone mineral density by digital X-ray radiogrammetry: theoretical background and clinical testing. Osteoporos Int 12(11):961–969. https://doi.org/10.1007/s001980170026

    Article  CAS  PubMed  Google Scholar 

  42. Boonen S, Nijs J, Borghs H et al (2005) Identifying postmenopausal women with osteoporosis by calcaneal ultrasound, metacarpal digital X-ray radiogrammetry and phalangeal radiographic absorptiometry: a comparative study. Osteoporos Int 16(1):93–100. https://doi.org/10.1007/s00198-004-1660-z Epub 2004 Jun 10

    Article  PubMed  Google Scholar 

  43. Lydick E, Cook K, Turpin J, Melton M, Stine R, Byrnes C (1998) Development and validation of a simple questionnaire to facilitate identification of women likely to have low bone density. Am J Manag Care 4:37–48

    CAS  PubMed  Google Scholar 

  44. Cadarette SM, Jaglal SB, Kreiger N, McIsaac WJ, Darlington GA, Tu JV (2000) Development and validation of the Osteoporosis Risk Assessment Instrument to facilitate selection of women for bone densitometry. CMAJ 162:1289–1294

    CAS  PubMed  PubMed Central  Google Scholar 

  45. Cadarette SM, Jaglal SB, Murray TM, McIsaac WJ, Joseph L, Brown JP (2001) Evaluation of decision rules for referring women for bone densitometry by dual-energy X-ray absorptiometry. JAMA 286:57–63. https://doi.org/10.1001/jama.286.1.57

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors express their sincere gratitude to the following healthcare facilities for their invaluable support and cooperation throughout this study: Eniwa Hospital, Japan Community Healthcare Organization Tokyo Shinjuku Medical Center, Kobayakawa Orthopaedic Rheumatology Clinic, Japanese Red Cross Aichi Medical Center Nagoya Daiichi Hospital, Nagoya Medical Center, Gamagori City Hospital, Fujita Health University Hospital, National Center for Geriatrics and Gerontology, Gifu Prefectural Tajimi Hospital, Municipal Ena Hospital, Mie University School of Medicine Nagai Hospital, Yokkaichi Municipal Hospital, Sanda City Hospital, Mitoyo General Hospital, Japan Community Healthcare Organization Ritsurin Hospital, and Naruo Orthopaedic Surgery Hospital. This study would not have been possible without their assistance and collaboration, and all authors are deeply grateful for their contribution to the success of this study. Additionally, the authors would like to extend their appreciation to Yusuke Iesaki, Masahiro Kiyono, Takashi Iwakura, Tetsujiro Ohno, Masahiro Hasegawa, Naoaki Osada, Tomonori Kobayakawa, Yasumori Sobue, Nobuyuki Okui, Shogo Tabata, Terufumi Kokabu, Yuichiro Abe, and Nobuyuki Fujita for their valuable contributions.

Funding

This work was supported by the Japanese Orthopaedic Association [grant numbers 2022-3].

Author information

Authors and Affiliations

  1. Department of Orthopaedic Surgery, Nagoya University Graduate School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, Aichi, 466-8560, Japan

    Takamune Asamoto, Yasuhiko Takegami, Yoichi Sato & Shiro Imagama

  2. Department of Orthopaedic Surgery, Hyogo Prefectural Kakogawa Medical Center, Kakogawa, Hyogo, Japan

    Shunsuke Takahara

  3. Department of Orthopaedic Surgery, Miyamoto Orthopaedic Hospital, Okayama, Japan

    Norio Yamamoto

  4. Department of Orthopaedic Surgery, The Jikei University Kashiwa Hospital, Chiba, Japan

    Naoya Inagaki

  5. Department of Orthopaedic Surgery, Chiba University Graduate School of Medicine, Chiba, Japan

    Satoshi Maki

  6. Department of Orthopaedic Surgery, Jikei University School of Medicine, Tokyo, Japan

    Mitsuru Saito

Authors
  1. Takamune Asamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yasuhiko Takegami
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yoichi Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shunsuke Takahara
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Norio Yamamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Naoya Inagaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Satoshi Maki
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Mitsuru Saito
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Shiro Imagama
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Takamune Asamoto, Yoichi Sato, and Yasuhiko Takegami conducted the experiments and drafted the manuscript. Takamune Asamoto prepared Figs. 1–4 and Table 1. Yasuhiko Takegami, Mitsuru Saito, and Shiro Imagama contributed to the study design and drafted the manuscript. Yoichi Sato, Shunsuke Takahara, Norio Yamamoto, Naoya Inagaki, and Satoshi Maki recruited the participants and collected the data. Yoichi Sato, Takamune Asamoto, and Yasuhiko Takegami analyzed the data. All authors have read and approved the final version of the manuscript.

Corresponding author

Correspondence to Yasuhiko Takegami.

Ethics declarations

Ethics approval

The study protocol was approved by Gamagori City Hospital’s Human Research Ethics Committee and Institutional Review Board. The study procedures were conducted in accordance with the principles embodied in the Declaration of Helsinki.

Consent for publication

Before publishing the research findings, the authors obtained consent to publication from all participants who agreed to have their data included in the study.

Conflicts of interest

None.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Asamoto, T., Takegami, Y., Sato, Y. et al. External validation of a deep learning model for predicting bone mineral density on chest radiographs. Arch Osteoporos 19, 15 (2024). https://doi.org/10.1007/s11657-024-01372-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11657-024-01372-9

Keywords

Search

Navigation