Skip to main content
SpringerLink
Log in

Enhancing YOLO5 for the Assessment of Irregular Pelvic Radiographs with Multimodal Information

  • Published:
Journal of Imaging Informatics in Medicine Aims and scope Submit manuscript

Abstract

Developmental dysplasia of the hip (DDH) is one of the most common orthopedic disorders in infants and young children. Accurate identification and localization of anatomical landmarks are prerequisites for the diagnosis of DDH. In recent years, various works have employed deep learning algorithms on radiography images for DDH diagnosis. However, none of these works have considered the incorporation of multimodal information. The pelvis exhibits distinct structures at different developmental stages, and there are also gender-based differences. In light of this, this study proposes a method to enhance the performance of deep learning models in diagnosing DDH by incorporating age and gender information into the channels. The study utilizes YOLO5 to construct a deep learning network for detecting hip joint landmarks. Moreover, a comprehensive dataset of 7750 pelvic X-ray images is established, covering ages from 4 months to 16 years and encompassing various conditions, such as deformities and post-operative cases, which authentically capture the temporal diversity and pathological complexities of DDH. Experimental results show that the YOLO5 model with integrated multimodal information achieves a mAP0.5–0.95 of 83.1% and a diagnostic accuracy of 86.7% in test dataset. The F1 scores for diagnosing cases of normal (NM), suspected dislocation (SD), mild dislocation (MD), and heavily dislocation (HD) are 90.9%, 79.8%, 63.5%, and 97.4%, respectively. Furthermore, experiments conducted on datasets of different sizes and networks of different sizes demonstrate the beneficial impact of multimodal information in improving the effectiveness of deep learning in diagnosing DDH.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Patel, H., Preventive Health Care, C.T.F., et al.: Preventive health care, 2001 update: screening and management of developmental dysplasia of the hip in newborns. Cmaj 164(12), 1669–1677 (2001)

  2. Broadhurst, C., Rhodes, A., Harper, P., Perry, D., Clarke, N., Aarvold, A.: What is the incidence of late detection of developmental dysplasia of the hip in England?: a 26-year national study of children diagnosed after the age of one. The bone & joint journal 101(3), 281–287 (2019)

    Article  Google Scholar 

  3. Sewell, M.D., Eastwood, D.M.: Screening and treatment in developmental dysplasia of the hip—where do we go from here? International orthopaedics 35, 1359–1367 (2011)

    Article  PubMed  PubMed Central  Google Scholar 

  4. Shorter, D., Hong, T., Osborn, D.A.: Screening programmes for developmental dysplasia of the hip in newborn infants. Cochrane Database of Systematic Reviews (9) (2011)

  5. Ruiz Santiago, F., Santiago Chinchilla, A., Ansari, A., Guzm´an Alvarez, L., ´ Castellano Garc´ıa, M.d.M., Mart´ınez Mart´ınez, A., Tercedor S´anchez, J., et al.: Imaging of hip pain: from radiography to cross-sectional imaging techniques. Radiology research and practice 2016 (2016)

  6. Poyraz, A.K., Onur, M.R., Eroglu, Y., Gurger, M., Goktekin, M.C., Akgol, G.: Computed tomography characteristics of the acetabulum in developmental dysplasia of the hip. Iranian Journal of Radiology 16(1) (2019)

  7. Chen, X., Wang, X., Zhang, K., Fung, K.-M., Thai, T.C., Moore, K., Mannel, R.S., Liu, H., Zheng, B., Qiu, Y.: Recent advances and clinical applications of deep learning in medical image analysis. Medical Image Analysis 79, 102444 (2022)

    Article  PubMed  PubMed Central  Google Scholar 

  8. Liu, C., Xie, H., Zhang, S., Xu, J., Sun, J., Zhang, Y.: Misshapen pelvis landmark detection by spatial local correlation mining for diagnosing developmental dysplasia of the hip. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part VI 22, pp. 441–449 (2019). Springer

  9. Li, Q., Zhong, L., Huang, H., Liu, H., Qin, Y., Wang, Y., Zhou, Z., Liu, H., Yang, W., Qin, M., et al.: Auxiliary diagnosis of developmental dysplasia of the hip by automated detection of sharp’s angle on standardized anteroposterior pelvic radiographs. Medicine 98(52) (2019)

  10. Liu, C., Xie, H., Zhang, S., Mao, Z., Sun, J., Zhang, Y.: Misshapen pelvis landmark detection with local-global feature learning for diagnosing developmental dysplasia of the hip. IEEE Transactions on Medical Imaging 39(12), 3944–3954 (2020)

    Article  PubMed  Google Scholar 

  11. Wu, H., Xie, H., Liu, C., Zha, Z.-J., Sun, J., Zhang, Y.: Circlenet for hip landmark detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12370–12377 (2020)

  12. Park, H.S., Jeon, K., Cho, Y.J., Kim, S.W., Lee, S.B., Choi, G., Lee, S., Choi, Y.H., Cheon, J.-E., Kim, W.S., et al.: Diagnostic performance of a new convolutional neural network algorithm for detecting developmental dysplasia of the hip on anteroposterior radiographs. Korean journal of radiology 22(4), 612 (2021)

    Article  PubMed  Google Scholar 

  13. Li, Q., Yang, W., Xu, M., An, N., Wang, D., Wang, X., Jin, H., Wang, J., Wang, J.: Model construction and application for automated measurement of ce angle on pelvis orthograph based on mask-r-cnn algorithm. Biomedical Physics & Engineering Express 7(3), 035010 (2021)

    Article  Google Scholar 

  14. Thompson, P., Collaborative, M.A., Perry, D.C., Cootes, T.F., Lindner, C.: Automation of clinical measurements on radiographs of children’s hips. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 419–428 (2022). Springer

  15. Xu, W., Shu, L., Gong, P., Huang, C., Xu, J., Zhao, J., Shu, Q., Zhu, M., Qi, G., Zhao, G., et al.: A deep-learning aided diagnostic system in assessing developmental dysplasia of the hip on pediatric pelvic radiographs. Frontiers in Pediatrics 9,785480 (2022)

    Article  PubMed  PubMed Central  Google Scholar 

  16. Fraiwan, M., Al-Kofahi, N., Ibnian, A., Hanatleh, O.: Detection of developmental dysplasia of the hip in x-ray images using deep transfer learning. BMC Medical Informatics and Decision Making 22(1), 1–11 (2022)

    Article  Google Scholar 

  17. Pei, Y., Mu, L., Xu, C., Li, Q., Sen, G., Sun, B., Li, X., Li, X.: Learning-based landmark detection in pelvis x-rays with attention mechanism: data from the osteoarthritis initiative. Biomedical Physics & Engineering Express 9(2), 025001 (2023)

    Article  Google Scholar 

  18. Den, H., Ito, J., Kokaze, A.: Diagnostic accuracy of a deep learning model using yolov5 for detecting developmental dysplasia of the hip on radiography images. Scientific Reports 13(1), 6693 (2023)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Xu, J., Xie, H., Tan, Q., Wu, H., Liu, C., Zhang, S., Mao, Z., Zhang, Y.: Multitask hourglass network for online automatic diagnosis of developmental dysplasia of the hip. World wide web 26(2), 539–559 (2023)

    Article  PubMed  Google Scholar 

  20. Al-Bashir, A.K., Al-Abed, M., Sharkh, F.M.A., Kordeya, M.N., Rousan, F.M.: Algorithm for automatic angles measurement and screening for developmental dysplasia of the hip (ddh). In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 6386–6389 (2015). IEEE

  21. Sahin, S., Akata, E., Sahin, O., Tuncay, C., Ozkan, H.: A novel computer-based ¨ method for measuring the acetabular angle on hip radiographs. Acta orthopaedica et traumatologica turcica 51(2), 155–159 (2017)

    Article  PubMed  PubMed Central  Google Scholar 

  22. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  23. Glenn, J.: Yolov5 release v7.0. https://github.com/ultralytics/yolov5/releases/-tag/v7.0 (2022)

  24. T¨onnis, D.: Indications and time planning for operative interventions in hip dysplasia in child and adulthood. Zeitschrift fur Orthopadie und ihre Grenzgebiete 123(4), 458–461 (1985)

    PubMed  Google Scholar 

Download references

Funding

The Third Affiliated Hospital of Southern Medical University Directors' Fund Project for 2022 (YH202207).

Author information

Author notes

  1. Jing Chen and Xiaoyou Fan contributed equally to this work.

Authors and Affiliations

  1. School of Physics and Optoelectronic Engineering, Guangdong University of Technology, Guangzhou, 510006, Guangdong, China

    Jing Chen, Xiaoyou Fan, Zhen Chen, Lichong Liang & Chengyue Su

  2. Department of Pediatric Orthopedics, Center for Orthopaedic Surgery, Hospital of Guangdong Province, The Third School of Clinical Medicine, Southern Medical University, The Third Affiliated Hospital of Southern Medical University, Guangzhou, 510630, Guangdong, China

    Yichao Peng, Yun Chen & Jinghui Yao

  3. Department of Orthopedics, Academy of Orthopedics Guangdong Province, Guangdong Provincial Key Laboratory of Bone and Joint Degeneration Diseases, Guangzhou, 510630, Guangdong, China

    Yichao Peng & Jinghui Yao

  4. School of Nursing, Southern Medical University, The Third Affiliated Hospital of Southern Medical University, Guangzhou, 510630, Guangdong, China

    Yun Chen

Authors
  1. Jing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyou Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yichao Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lichong Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chengyue Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jinghui Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Jing Chen, Xiaoyou Fan, Yun Chen, and Jinghui Yao made major contributions to the conceptualization and design of this study. Material preparation, data collection, and image annotation were completed by Yichao Peng, Yun Chen, and Jinghui Yao. The implementation and analysis of deep learning were carried out by Xiaoyou Fan, Zhen Chen, Lichong Liang, and Chengyue Su. This manuscript was written by Xiaoyou Fan and Jing Chen, and all authors discussed and revised previous versions of the manuscript. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Yun Chen or Jinghui Yao.

Ethics declarations

Ethics Approval

This research obtained approval from the Ethics Committee of the Third Affiliated Hospital of Southern Medical University and was granted a waiver of informed consent. All data and images used in this study have been desensitized. These data and images used for research do not contain any patient’s private information and do not contain information showing the patient’s identity. Moreover, these images and data are only used for academic research.

Consent to Participate

Informed consent was waived by the Clinical Trial Ethics Committee of the Third Affiliated Hospital of Southern Medical University.

Consent for Publication

Informed consent was waived by the Clinical Trial Ethics Committee of the Third Affiliated Hospital of Southern Medical University.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, J., Fan, X., Chen, Z. et al. Enhancing YOLO5 for the Assessment of Irregular Pelvic Radiographs with Multimodal Information. J Digit Imaging. Inform. med. 37, 744–755 (2024). https://doi.org/10.1007/s10278-024-00986-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-024-00986-2

Keywords

Search

Navigation