Skip to main content
SpringerLink
Log in

Modernization of bone age assessment: comparing the accuracy and reliability of an artificial intelligence algorithm and shorthand bone age to Greulich and Pyle

Skeletal Radiology volume 49, pages 1449–1457 (2020)Cite this article

Abstract

Greulich and Pyle (GP) is one of the most common methods to determine bone age from hand radiographs. In recent years, new methods were developed to increase the efficiency in bone age analysis like the shorthand bone age (SBA) and automated artificial intelligence algorithms.

Objective

The aim of this study is to evaluate the accuracy and reliability of these two methods and examine if the reduction in analysis time compromises their efficacy.

Methods

Two hundred thirteen males and 213 females had their bone age determined by two separate raters using the SBA and GP methods. Three weeks later, the two raters repeated the analysis of the radiographs. The raters timed themselves using an online stopwatch. De-identified radiographs were securely uploaded to an automated algorithm developed by a group of radiologists in Toronto. The gold standard was determined to be the radiology report attached to each radiograph, written by experienced radiologists using GP.

Results

Intraclass correlation between each method and the gold standard fell within the range of 0.8–0.9, highlighting significant agreement. Most of the comparisons showed a statistically significant difference between the new methods and the gold standard; however, it may not be clinically significant as it ranges between 0.25 and 0.5 years. A bone age is considered clinically abnormal if it falls outside 2 standard deviations of the chronological age; standard deviations are calculated and provided in GP atlas.

Conclusion

The shorthand bone age method and the automated algorithm produced values that are in agreement with the gold standard while reducing analysis time.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Figure 1
Fig. 2

References

  1. Heyworth BE, Osei DA, Fabricant PD, et al. The shorthand bone age assessment: a simpler alternative to current methods. J Pediatr Orthop. 2013;33(5):569–74. https://doi.org/10.1097/BPO.0b013e318293e5f2.

    Article  PubMed  Google Scholar 

  2. Bass S, Pearce G, Bradney M, et al. Exercise before puberty may confer residual benefits in bone density in adulthood: studies in active prepubertal and retired female gymnasts. J Bone Miner Res. 1998;13:500–7.

    Article  CAS  Google Scholar 

  3. Martin DD, Wit JM, Hochberg Z, et al. The use of bone age in clinical practice - part 1. Horm Res Paediatr. 2011;76(1):1–9. https://doi.org/10.1159/000329372.

    Article  CAS  PubMed  Google Scholar 

  4. Satoh M. Bone age: assessment methods and clinical applications. 2015. Clin Pediatr Endocrinol. 2015;24(4):143–52. Published online 2015 Oct 24. https://doi.org/10.1297/cpe.24.143.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Martin DD, Wit JM, Hochberg Z, et al. The use of bone age in clinical practice - part 2. Horm Res Paediatr. 2011;76(1):10–6. https://doi.org/10.1159/000329374.

    Article  CAS  PubMed  Google Scholar 

  6. Makarov MR, Jackson TJ, Smith CM, Jo CH, Birch JG. Timing of epiphysiodesis to correct leg-length discrepancy: a comparison of prediction methods. J Bone Joint Surg Am. 2018;100(14):1217–22. https://doi.org/10.2106/JBJS.17.01380.

    Article  PubMed  Google Scholar 

  7. Diméglio A, Charles YP, Daures JP, de Rosa V, Kaboré B. Accuracy of the Sauvegrain method in determining skeletal age during puberty. J Bone Joint Surg Am. 2005;87(8):1689–96.

    PubMed  Google Scholar 

  8. Bitan FD, Veliskakis KP, Campbell BC. Differences in the Risser grading systems in the United States and France. Clin Orthop Relat Res. 2005;436:190–5. https://doi.org/10.1097/01.blo.0000160819.10767.88.

    Article  Google Scholar 

  9. Wittschieber D, Vieth V, Domnick C, Pfeiffer H, Schmeling A. The iliac crest in forensic age diagnostics: evaluation of the apophyseal ossification in conventional radiography. Int J Legal Med. 2013;127(2):473–9. https://doi.org/10.1007/s00414-012-0763-x.

    Article  PubMed  Google Scholar 

  10. Schmidt S, Schmeling A, Zwiesigk P, Pfeiffer H, Schulz R. Sonographic evaluation of apophyseal ossification of the iliac crest in forensic age diagnostics in living individuals. Int J Legal Med. 2011;125(2):271–6. https://doi.org/10.1007/s00414-011-0554-9.

    Article  PubMed  Google Scholar 

  11. Mughal AM, Hassan N, Ahmed A. Bone age assessment methods: a critical review. Pak J Med Sci. 2014;30(1):211–5. https://doi.org/10.12669/pjms.301.4295.

    Article  Google Scholar 

  12. Su P, Zhang L, Peng Y, Liang A, Du K, Huang D. A histological and ultrastructural study of femoral head cartilage in a new type II collagenopathy. Int Orthop. 2010;34(8):1333–9. https://doi.org/10.1007/s00264-010-0985-9.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Kaur G, Khandelwal N, Jasuja OP. Computed tomographic studies on ossification status of medial epiphysis of clavicle: effect of slice thickness and dose distribution. J Indian Acad Forensic Med. 32(4).

  14. Schmidt S, Mühler M, Schmeling A, Reisinger W, Schulz R. Magnetic resonance imaging of the clavicular ossification. Int J Legal Med. 2007;121(4):321–4.

    Article  Google Scholar 

  15. Hoerr NL. Radiographic atlas of skeletal development of the knee. Springfield: Charles C. Thomas; 1955.

    Google Scholar 

  16. Zafar AM, Nadeem N, Husen Y, Ahmad MN. An appraisal of Greulich-Pyle atlas for skeletal age assessment in Pakistan. J Pak Med Assoc. 2010;60(7):552–5.

    PubMed  Google Scholar 

  17. Gaskin CM, Kahn SL, Bertozzi JC, Bunch PM. Skeletal development of the hand and wrist: a radiographic atlas and digital bone age companion: a radiographic atlas and digital bone age companion. Oxford: Oxford University Press; 2011.

    Google Scholar 

  18. Greulich WW, Pyle SI. Radiograph atlas of skeletal development of the hand and wrist. 2nd ed. Palo Alto: Stanford University Press; 1959.

    Google Scholar 

  19. Halabi SS, Prevedello LM, Kalpathy-Cramer J, et al. The RSNA pediatric bone age machine learning challenge. Radiology. 2019;290(2):498–503. https://doi.org/10.1148/radiol.2018180736.

    Article  PubMed  Google Scholar 

  20. Mukaka MM. A guide to appropriate use of correlation coefficient in medical research. Malawi Med J. 2012;24(3):69–71.

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Nwosu BU, Lee MM. Evaluation of short and tall stature in children. Am Fam Physician. 2008;78(5):597–604.

    PubMed  Google Scholar 

  22. Kim JR, Shim WH, Yoon HM, et al. Computerized bone age estimation using deep learning based program: evaluation of the accuracy and efficiency. AJR Am J Roentgenol. 2017;209(6):1374–80.

    Article  Google Scholar 

  23. Larson DB, Chen MC, Lungren MP, Halabi SS, Stence NV, Langlotz CP. Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology. 2018;287(1):313–22.

    Article  Google Scholar 

  24. Lee H, Tajmir S, Lee J, et al. Fully automated deep learning system for bone age assessment. J Digit Imaging. 2017;30(4):427–41.

    Article  Google Scholar 

  25. Mutasa S, Chang PD, Ruzal-Shapiro C, Ayyala R. MABAL: a novel deep-learning architecture for machine-assisted bone age labeling. J Digit Imaging. 2018;31(4):513–9.

    Article  Google Scholar 

  26. Kaplowitz PB, Slora EJ, Wasserman RC, Pedlow SE, Herman-Giddens ME. Earlier onset of puberty in girls: relation to increased body mass index and race. Pediatrics. 2001;108(2):347–53.

    Article  CAS  Google Scholar 

  27. Herman-Giddens ME, Steffes J, Harris D, et al. Secondary sexual characteristics in boys: data from the pediatric research in office settings network. Pediatrics. 2012;130(5):e1058–68. https://doi.org/10.1542/peds.2011-3291.

    Article  PubMed  Google Scholar 

  28. Ontell FK, Ivanovic M, Ablin DS, Barlow TW. Bone age in children of diverse ethnicity. Am J Roentgenol. 1996;167:1395.

    Article  CAS  Google Scholar 

  29. Loder RT, Estle DT, Morrison K, et al. Applicability of the Greulich and Pyle skeletal age standards to black and white children of today. Am J Dis Child. 1993;147:1329–33.

    CAS  PubMed  Google Scholar 

  30. Zhang A, Sayre JW, Vachon L, et al. Racial differences in growth patterns of children assessed on the basis of bone age. Radiology. 2009;250:228–35.

    Article  Google Scholar 

  31. Martin DD, Neuhof J, Jenni OG, et al. Automatic determination of left- and right-hand bone age in the first Zurich longitudinal study. Horm Res Paediatr. 2010;74:50–5.

    Article  CAS  Google Scholar 

  32. Thodberg HH. Clinical review: an automated method for determination of bone age. J Clin Endocrinol Metab. 2009;94:2239–44.

    Article  CAS  Google Scholar 

  33. Thodberg HH, Jenni OG, Caflisch J, et al. Prediction of adult height based on automated determination of bone age. J Clin Endocrinol Metab. 2009;94:4868–74.

    Article  CAS  Google Scholar 

  34. Thodberg HH, Kreiborg S, Juul A, et al. The BoneXpert method for automated determination of skeletal maturity. IEEE Trans Med Imaging. 2009;28:52–66.

    Article  Google Scholar 

  35. Tanner JM. Assessment of skeletal maturity and prediction of adult height (TW3 method). 3rd ed. London: W.B. Saunders; 2001.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Medicine, University of British Columbia, Vancouver, Canada

    Mina Gerges & Harpreet Chhina

  2. Faculty of Science, University of British Columbia, Vancouver, Canada

    Hayley Eng

  3. Department of Orthopaedics, BC Children’s Hospital, 1D 64 4480 Oak Street, Vancouver, BC, V6H 3V4, Canada

    Harpreet Chhina & Anthony Cooper

  4. Department of Orthopaedics, Faculty of Medicine, University of British Columbia, Vancouver, Canada

    Anthony Cooper

Authors
  1. Mina Gerges
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hayley Eng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Harpreet Chhina
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anthony Cooper
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anthony Cooper.

Ethics declarations

Conflict of interest

Dr. Mark Cicero from 16 Bit Inc. The algorithm adopted in this study is the intellectual property of 16 Bit Inc. Members of the company assisted with the use of the algorithm for the purpose of the study; however, none of the authors are affiliated with the company, nor there are any financial association with the study.

Ethical approval

Ethics approval was obtained from the University of British Columbia's research ethics board (H18-02756)

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 20.1 kb)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gerges, M., Eng, H., Chhina, H. et al. Modernization of bone age assessment: comparing the accuracy and reliability of an artificial intelligence algorithm and shorthand bone age to Greulich and Pyle. Skeletal Radiol 49, 1449–1457 (2020). https://doi.org/10.1007/s00256-020-03429-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00256-020-03429-5

Keywords

search

Navigation