Skip to main content
SpringerLink
Log in

Semi-supervised object detection based on single-stage detector for thighbone fracture localization

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The thighbone is the largest bone supporting the lower body. If the thighbone fracture is not treated in time, it will lead to lifelong inability to walk. Correct diagnosis of thighbone disease is very important in orthopedic medicine. Deep learning is promoting the development of fracture detection technology. However, the existing computer-aided diagnosis methods rely on a large number of manually labeled data, and labeling these data costs a lot of time and energy. Therefore, we develop an object detection method with limited labeled image quantity and apply it to the thighbone fracture localization. In this work, we build a semi-supervised object detection framework based on single-stage detector, which includes three modules: adaptive difficult sample oriented (ADSO) module, Fusion Box and deformable expand encoder (Dex encoder). ADSO module takes the classification score as the label reliability evaluation criterion by weighting, Fusion Box is designed to merge similar pseudo boxes into a reliable box for box regression and Dex encoder is proposed to enhance the adaptability of image augmentation. The experiment is conducted on the thighbone fracture dataset, which includes 3484 training thighbone fracture images and 358 testing thighbone fracture images. The experimental results show that the proposed method achieves the state-of-the-art AP in thighbone fracture detection at different labeled data rates, i.e., 1%, 5% and 10%. Besides, we use full data to achieve knowledge distillation, our method achieves 86.2% AP50 and 52.6% AP75. Finally, the effectiveness of our method has also been evaluated using the publicly available datasets COCO and VOC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Jones RM, Sharma A, Hotchkiss R, Sperling JW, Lindsey RV (2020) Assessment of a deep-learning system for fracture detection in musculoskeletal radiographs. NPJ Dig Med 3(1):1–6. https://doi.org/10.1038/s41746-020-00352-w

    Article  Google Scholar 

  2. Georgalis GL, Scheyer TM (2022) Crushed but not lost: a colubriform snake (serpentes) from the miocene swiss molasse, identified through the use of micro-ct scanning technology. Swiss J Geosci 115(1):1–9

    Article  Google Scholar 

  3. Guan B, Yao J, Wang S, Zhang G, Zhang Y, Wang X, Wang M (2022) Automatic detection and localization of thighbone fractures in x-ray based on improved deep learning method. Comput Vis Image Underst 216:103345. https://doi.org/10.1016/j.cviu.2021.103345

    Article  Google Scholar 

  4. Hardalaç F, Uysal F, Peker O, Çiçeklidağ M, Tolunay T, Tokgöz N, Kutbay U, Demirciler B, Mert F (2022) Fracture detection in wrist x-ray images using deep learning-based object detection models. Sensors 22(3):1285. https://doi.org/10.3390/s22031285

    Article  PubMed  PubMed Central  ADS  Google Scholar 

  5. Sha G, Wu J, Yu B (2020) Detection of spinal fracture lesions based on improved yolov2. In: 2020 IEEE international conference on artificial intelligence and computer applications (ICAICA), pp. 235– 238. https://doi.org/10.1109/ICAICA50127.2020.9182582. IEEE

  6. Thian YL, Li Y, Jagmohan P, Sia D, Chan VEY, Tan RT (2019) Convolutional neural networks for automated fracture detection and localization on wrist radiographs. Radiol Artif Intell 1(1):180001. https://doi.org/10.1148/ryai.2019180001

    Article  Google Scholar 

  7. Wu H-Z, Yan L-F, Liu X-Q, Yu Y-Z, Geng Z-J, Wu W-J, Han C-Q, Guo Y-Q, Gao B-L (2021) The feature ambiguity mitigate operator model helps improve bone fracture detection on x-ray radiograph. Sci Rep 11(1):1–10. https://doi.org/10.1038/s41598-021-81236-1

    Article  CAS  Google Scholar 

  8. Lee D-H et al (2013) Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: workshop on challenges in representation learning, ICML vol 3, p 896

  9. Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. Comput Sci 14(7):38–39

    Google Scholar 

  10. Cai Z, Vasconcelos N (2018) Cascade r-cnn: delving into high quality object detection. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp 6154– 6162. https://doi.org/10.1109/CVPR.2018.00644

  11. Cao Y, Xu J, Lin S, Wei F, Hu H (2019) Gcnet: non-local networks meet squeeze-excitation networks and beyond. In: proceedings of the IEEE/CVF international conference on computer vision workshops, pp 1971– 1980. https://doi.org/10.1109/ICCVW.2019.00246

  12. Ding X, Li Q, Cheng Y, Wang J, Bian W, Jie B (2020) Local keypoint-based faster r-cnn. Appl Intell 50(10):3007–3022. https://doi.org/10.1007/s10489-020-01665-9

    Article  Google Scholar 

  13. Tian Z, Shen C, Chen H, He T (2019) Fcos: Fully convolutional one-stage object detection. In: proceedings of the IEEE/CVF international conference on computer vision, pp. 9627– 9636 https://doi.org/10.1109/ICCV.2019.00972

  14. Li B, Liu Y, Wang X (2019) Gradient harmonized single-stage detector. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8577– 8584. https://doi.org/10.1609/aaai.v33i01.33018577

  15. Chen Q, Wang Y, Yang T, Zhang X, Cheng J, Sun J (2021) You only look one-level feature. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13039– 13048. https://doi.org/10.1109/CVPR46437.2021.01284

  16. Zhang L, Hu Y, Chen J, Li C, Li K (2022) Mssif-net: an efficient cnn automatic detection method for freight train images. Neural Comput Appl 35(9):6767–6785. https://doi.org/10.1007/s00521-022-08035-1

    Article  Google Scholar 

  17. Hurtik P, Molek V, Hula J, Vajgl M, Vlasanek P, Nejezchleba T (2022) Poly-yolo: higher speed, more precise detection and instance segmentation for yolov3. Neural Comput Appl 34(10):8275–8290. https://doi.org/10.1007/s00521-021-05978-9

    Article  Google Scholar 

  18. Ren S, He K, Girshick R, Sun J (2016) Faster r-cnn: towards real-time object detection with region proposal networks. In: advances in neural information processing systems, vol. 28

  19. Xu M, Zhang Z, Hu H, Wang J, Wang L, Wei F, Bai X, Liu Z (2021) End-to-end semi-supervised object detection with soft teacher. In: proceedings of the IEEE/CVF international conference on computer vision, pp. 3060–3069. https://doi.org/10.1109/ICCV48922.2021.00305

  20. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: computer vision–ECCV 2014: 13th European conference, Zurich, Switzerland, pp. 740– 755. https://doi.org/10.1007/978-3-319-10602-1_48. Springer

  21. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88:303–338

    Article  Google Scholar 

  22. Hesamian MH, Jia W, He X, Kennedy P (2019) Deep learning techniques for medical image segmentation: achievements and challenges. J Dig Imaging 32(4):582–596. https://doi.org/10.1007/s10278-019-00227-x

    Article  Google Scholar 

  23. Wang W, Huang W, Lu Q, Chen J, Zhang M, Qiao J, Zhang Y (2022) Attention mechanism-based deep learning method for hairline fracture detection in hand x-rays. Neural Comput Appl 34(21):18773–18785

    Article  PubMed  PubMed Central  Google Scholar 

  24. Khurana Y, Soni U (2022) Leveraging deep learning for covid-19 diagnosis through chest imaging. Neural Comput Appl 34(16):14003–14012. https://doi.org/10.1007/s00521-022-07250-0

    Article  PubMed  PubMed Central  Google Scholar 

  25. Shaik NS, Cherukuri TK (2022) Hinge attention network: a joint model for diabetic retinopathy severity grading. Appl Intell 52:15105–15121. https://doi.org/10.1007/s10489-021-03043-5.13

    Article  Google Scholar 

  26. Fouad H, Soliman AM, Hassanein AS, Al-Feel H (2020) Prediction and diagnosis of vertebral tumors on the internet of medical things platform using geometric rough propagation neural network. Neural Comput Appl 24:1–13

    Google Scholar 

  27. Zhang, X., Wang, Y., Cheng, C.-T., Lu, L., Xiao, J., Liao, C.-H., Miao, S (2020) A new window loss function for bone fracture detection and localization in x-ray images with point-based annotation. arXiv preprint arXiv:2012.04066. https://doi.org/10.48550/arXiv.2012.04066

  28. Wang Y, Zheng K, Cheng C-T, Zhou X-Y, Zheng Z, Xiao J, Lu L, Liao C-H, Miao S (2021) Knowledge distillation with adaptive asymmetric label sharpening for semi-supervised fracture detection in chest x-rays. In: international conference on information processing in medical imaging, pp 599– 610. https://doi.org/10.1007/978-3-030-78191-0_46. Springer

  29. Deng J, Xuan X, Wang W, Li Z, Yao H, Wang Z (2020) A review of research on object detection based on deep learning. J Phys Conf Ser 1684:012028

    Article  Google Scholar 

  30. Lee H-L, Kim Y-J, Kim B-G et al (2022) A survey for 3d object detection algorithms from images. J Multim Information Syst 9(3):183–190. https://doi.org/10.33851/JMIS.2022.9.3.183

    Article  ADS  Google Scholar 

  31. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: single shot multibox detector. Europ Conf Comput Vis 14:21–37

    Google Scholar 

  32. Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2117– 2125

  33. Park H-J, Choi Y-J, Lee Y-W, Kim B-G (2022) ssfpn: scale sequence (s\(\hat{~}\) 2) feature based feature pyramid network for object detection. arXiv preprint arXiv:2208.11533 (2022)

  34. Wu F, Jing X-Y, Liu Q, Wu S-S, He G-L (2017) Large-scale image recognition based on parallel kernel supervised and semi-supervised subspace learning. Neural Comput Appl 28(3):483–498. https://doi.org/10.1007/s00521-015-2081-y

    Article  Google Scholar 

  35. Tarvainen A, Valpola H (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Adv Neural Inform Process Syst 30:17

    Google Scholar 

  36. Berthelot D, Carlini N, Cubuk ED, Kurakin A, Raffel C (2019) Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. arXiv preprint arXiv:1911.09785. https://doi.org/10.48550/arXiv.1911.09785

  37. Ma Y, Chen D, Wang T, Li G, Yan M (2022) Semi-supervised partial label learning algorithm via reliable label propagation. Appl Intell. https://doi.org/10.1007/s10489-022-04027-9

    Article  Google Scholar 

  38. Jeong J, Lee S, Kim J, Kwak N (2019) Consistency-based semi-supervised learning for object detection. Adv Neural Inform Process Syst 32:190

    Google Scholar 

  39. Zoph B, Cubuk ED, Ghiasi G, Lin T-Y, Shlens J, Le QV (2020) Learning data augmentation strategies for object detection. In: European conference on computer vision, Springer: London. pp 566– 583. https://doi.org/10.1007/978-3-030-58583-9_34

  40. Yang Q, Wei X, Wang B, Hua X-S, Zhang L (2021) Interactive self-training with mean teachers for semi-supervised object detection. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5941– 5950. https://doi.org/10.1109/CVPR46437.2021.00588

  41. Wang Z, Li Y, Guo Y, Fang L, Wang S (2021) Data-uncertainty guided multi-phase learning for semi-supervised object detection. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4568– 4577. https://doi.org/10.1109/CVPR46437.2021.00454

  42. Zhang Y, Yao X, Liu C, Chen F, Song X, Xing T, Hu R, Chai H, Xu P, Zhang G (2022) S4od: Semi-supervised learning for single-stage object detection. arXiv preprint arXiv:2204.04492. https://doi.org/10.48550/arXiv.2204.04492

  43. Sohn K, Zhang Z, Li C-L, Zhang H, Lee C-Y, Pfister T (2020) A simple semi-supervised learning framework for object detection. arXiv preprint arXiv:2005.04757. https://doi.org/10.48550/arXiv.2005.04757

  44. Liu YC, Ma CY, He Z, Kuo CW, Vajda P (2021) Unbiased teacher for semi-supervised object detection. arXiv preprint arXiv:2102.09480. https://doi.org/10.48550/arXiv.2102.09480

  45. Lin T-Y, Goyal P, Girshick R, He K (2017) Dollár P Focal loss for dense object detection. In: proceedings of the IEEE international conference on computer vision, pp. 2980– 2988. https://doi.org/10.1109/TPAMI.2018.2858826

  46. Zhou Q, Yu C, Wang Z, Qian Q, Li H (2021) Instant-teaching: An end-to-end semi-supervised object detection framework. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4081– 4090. https://doi.org/10.1109/CVPR46437.2021.00407

  47. Sohn K, Berthelot D, Carlini N, Zhang Z, Zhang H, Raffel CA, Cubuk ED, Kurakin A, Li C-L (2020) Fixmatch: simplifying semi-supervised learning with consistency and confidence. Adv Neural Inform Process Syst 33:596–608

    Google Scholar 

  48. Bochkovskiy A, Wang C-Y, Liao H-YM (2020) Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934. https://doi.org/10.48550/arXiv.2004.10934

  49. Rothe R, Guillaumin M, Gool LV (2014) Non-maximum suppression for object detection by passing messages between windows. In: Asian conference on computer vision, pp. 290– 306. https://doi.org/10.1007/978-3-319-16865-4_19. Springer

  50. Zhu X, Hu H, Lin S, Dai J (2019) Deformable convnets v2: More deformable, better results. In: proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 9300– 9308. https://doi.org/10.1109/CVPR.2019.00953

  51. Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: proceedings of the IEEE international conference on computer vision, pp. 764– 773. https://doi.org/10.1109/ICCV.2017.89

  52. Guan B, Yao J, Zhang G, Wang X (2019) Thigh fracture detection using deep learning method based on new dilated convolutional feature pyramid network. Patt Recogn Lett 125:521–526. https://doi.org/10.1016/j.patrec.2019.06.015

    Article  ADS  Google Scholar 

  53. Wang M, Yao J, Zhang G, Guan B, Wang X, Zhang Y (2021) Parallelnet: multiple backbone network for detection tasks on thigh bone fracture. Multim Syst 27(6):1091–1100. https://doi.org/10.1007/s00530-021-00783-9

    Article  Google Scholar 

  54. Hosang J, Benenson R, Dollár P, Schiele B (2015) What makes for effective detection proposals? IEEE Trans Patt Analy Mach Intell 38(4):814–830. https://doi.org/10.1109/TPAMI.2015.2465908

    Article  Google Scholar 

  55. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770– 778

  56. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248– 255. Ieee

  57. Chen K, Wang J, Pang J, Cao Y, Xiong Y, Li X, Sun S, Feng W, Liu Z, Xu J, et al (2019) Mmdetection: open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155. https://doi.org/10.48550/arXiv.1906.07155

  58. Zhu X, Cheng D, Zhang Z, Lin S, Dai J (2019) An empirical study of spatial attention mechanisms in deep networks. In: proceedings of the IEEE/CVF international conference on computer vision, pp 6688– 6697. https://doi.org/10.1109/ICCV.2019.00679

  59. Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B(2021) Swin transformer: hierarchical vision transformer using shifted windows. In: proceedings of the IEEE/CVF international conference on computer vision, pp. 10012– 10022

Download references

Funding

This work is supported by the National Natural Science Foundation of China under Grant No. 62073237. Thanks for the data support of Linyi’s People Hospital.

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Nankai District, Tianjin, 300072, China

    Jinman Wei, Guoshan Zhang, Bin Guan, Yueming Zhang & Shaoquan Wang

  2. Department of Radiology, Linyi People’s Hosptial, Lanshan Districts, Linyi, 276000, Shandong, China

    Jinkun Yao

Authors
  1. Jinman Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinkun Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guoshan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bin Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yueming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shaoquan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guoshan Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships.

Data availability

The datasets generated during and/or analyzed during the current study are available from the authors on reasonable request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, J., Yao, J., Zhang, G. et al. Semi-supervised object detection based on single-stage detector for thighbone fracture localization. Neural Comput & Applic 36, 3447–3461 (2024). https://doi.org/10.1007/s00521-023-09277-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09277-3

Keywords

Search

Navigation