Skip to main content
SpringerLink
Log in

Safety helmet wearing correctly detection based on capsule network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Construction workers protect themselves from accidental injury by wearing safety helmets correctly. The safety helmet wearing detection algorithms currently based on deep learning aim to solve the problem of whether the safety helmet is worn in the head region, rather than whether the safety helmet is worn properly. This paper presents a novel Spatial Position Relation Capsule Network, termed SPR-CapsNet, for addressing safety helmet wearing correctly detection. In the first step, the learned deep feature maps are divided into patches, and each patch is transformed into a vector as a primary capsule, which effectively reduces the computational cost. In the second step, the dynamic routing algorithm is used to learn the spatial relationship between local image features. In the final step, the decision-making is optimized based on the probability of different dimensions of the output vector. It will be recognized as wearing status when the safety helmet is in a more appropriate position relative to the face. SPR-CapsNet is compared with the original Capsule Network, Convolutional Neural Network (CNN), Vision Transformer (ViT), and other models on a large amount of data. Experiments demonstrate the superiority of the proposed network.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The safety helmet wearing detection and face datasets that support the findings of this study are available from here, including SHWD, GDUT-HWD, CHV, and Kinship Verification.

References

  1. Alsaffar M, Mohammed E (2019) Isolation and characterization of pseudomonas aeruginosa from babylon province. Biochem Cell Arch 19(1):203–209

    Google Scholar 

  2. Baker N, Lu H, Erlikhman G, Kellman PJ (2018) Deep convolutional networks do not classify based on global object shape. PLOS Comput Biol 14(12):1–43

    Article  Google Scholar 

  3. Bozkurt F (2022) A deep and handcrafted features-based framework for diagnosis of covid-19 from chest x-ray images. Concurr Comput: Pract Exp 34(5):e6725

    Article  Google Scholar 

  4. Brendel W, Bethge M (2019) Approximating cnns with bag-of-local-features models works surprisingly well on imagenet. In: International conference on learning representations. OpenReview.net

  5. Chen S, Demachi K (2020) A vision-based approach for ensuring proper use of personal protective equipment (ppe) in decommissioning of fukushima daiichi nuclear power station. Appl Sci, 10(15)

  6. Deng J, Guo J, Ververas E, Kotsia I, Zafeiriou S (2020) Retinaface: single-shot multi-level face localisation in the wild. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Computer Vision Foundation / IEEE, pp 5203–5212

  7. Deng L, Li H, Liu H, Gu J (2022) A lightweight yolov3 algorithm used for safety helmet detection. Sci Rep 12(1):10981

    Article  Google Scholar 

  8. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N (2021) An image is worth 16x16 words: transformers for image recognition at scale

  9. Du S, Shehata M, Badawy W (2011) Hard hat detection in video sequences based on face features, motion and color information. ICCRD2011 - 2011 3rd International Conference on Computer Research and Development 4:25–29

    Google Scholar 

  10. Fan Y, Xian Y, Losch MM, Schiele B (2020) Analyzing the dependency of convnets on spatial information, 12544:101–115

  11. Fang R, Tang KD, Snavely N, Chen T (2010) Towards computational models of kinship verification. In: 2010 IEEE International conference on image processing. IEEE, pp 1577–1580

  12. Fekri-Ershad S, Ramakrishnan S (2022) Cervical cancer diagnosis based on modified uniform local ternary patterns and feed forward multilayer network optimized by genetic algorithm. Comput Biol Med 144:105392

    Article  Google Scholar 

  13. Gu Y, Wang Y, Shi L, Li N, Zhuang L, Xu S (2021) Automatic detection of safety helmet wearing based on head region location. IET Image Process 15(11):2441–2453

    Article  Google Scholar 

  14. Hahn T, Pyeon M, Kim G (2019) Self-routing capsule networks. In: Advances in neural information processing systems, vol 32. Curran Associates, Inc., pp 7656–7665

  15. Han G, Zhu M, Zhao X, Gao H (2021) Method based on the cross-layer attention mechanism and multiscale perception for safety helmet-wearing detection. Comput Electr Eng 95:107458

    Article  Google Scholar 

  16. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition, 770–778

  17. Hinton GE, Krizhevsky A, Wang SD (2011) Transforming auto-encoders. In: Artificial Neural Networks and Machine Learning – ICANN 2011, vol 6791. Springer, pp 44–51

  18. Hinton GE, Sabour S, Frosst N (2018) Matrix capsules with em routing. In: International conference on learning representations. OpenReview.net

  19. Hosseini H, Xiao B, Jaiswal M, Poovendran R (2017) On the limitation of convolutional neural networks in recognizing negative images. In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE, pp 352–358

  20. Jixiu W u, Cai N, Chen W, Wang H, Wang G (2019) Automatic detection of hardhats worn by construction personnel: a deep learning approach and benchmark dataset. Autom Constr 106:102894

    Article  Google Scholar 

  21. Li J, Liu H, Wang T, Jiang M, Wang S, Li K, Zhao X (2017) Safety helmet wearing detection based on image processing and machine learning. In: 2017 Ninth International conference on advanced computational intelligence. IEEE, pp 201–205

  22. Li Y, Wei H, Han Z, Huang J, Wang W (2020) Deep learning-based safety helmet detection in engineering management based on convolutional neural networks. Adv Civil Eng 2020:1–10

    Article  Google Scholar 

  23. Nguyen HH, Yamagishi J, Echizen I (2019) Use of a capsule network to detect fake images and videos. arXiv:1910.12467

  24. Noroozi M, Favaro P (2016) Unsupervised learning of visual representations by solving jigsaw puzzles. In: Computer Vision – ECCV 2016, vol 9910. Springer, pp 69–84

  25. Rubaiyat AHM, Toma TT, Kalantari-Khandani M, Rahman SA, Chen L, Ye Y, Pan CS (2016) Automatic detection of helmet uses for construction safety. In: 2016 IEEE/WIC/ACM International Conference on Web Intelligence Workshops (WIW). IEEE Computer Society, pp 135–142

  26. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In: Advances in neural information processing systems, vol 30, pp 3856–3866

  27. Sadiq M, Masood S, Om P (2022) Fd-yolov5: a fuzzy image enhancement based robust object detection model for safety helmet detection. Int J Fuzzy Syst 24(5):2600–2616

    Article  Google Scholar 

  28. Shen J, Xiong X, Li Y, He W, Li P, Zheng X (2021) Detecting safety helmet wearing on construction sites with bounding-box regression and deep transfer learning. Comput-Aided Civil Infrastruct Eng 36(2):180–196

    Article  Google Scholar 

  29. Shrestha K, Shrestha P, Bajracharya D, Yfantis E (2015) Hard-hat detection for construction safety visualization. J Construct Eng 2015:1–8

    Article  Google Scholar 

  30. Singh M, Nagpal S, Vatsa M, Singh R, Majumdar A (2018) Identity aware synthesis for cross resolution face recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. IEEE, pp 479–488

  31. Tsai Y-HH, Srivastava N, Goh H, Salakhutdinov R (2020) Capsules with inverted dot-product attention routing. In: ICLR. OpenReview.net

  32. Veloso e Silva RR, Aires KRT, de Melo Souza Veras R (2014) Helmet detection on motorcyclists using image descriptors and classifiers. In: 2014 27th SIBGRAPI Conference on graphics, patterns and images. IEEE Computer Society, pp 141–148

  33. Wang H, Hu Z, Guo Y, Yang Z, Zhou F, Xu P (2020) A real-time safety helmet wearing detection approach based on csyolov3. Appl Sci, 10(19)

  34. Wang Z, Wu Y, Yang L, Thirunavukarasu A, Evison C, Zhao Y (2021) Fast personal protective equipment detection for real construction sites using deep learning approaches. Sensors 21(10):3478

    Article  Google Scholar 

  35. Waranusast R, Bundon N, Timtong V, Tangnoi C, Pattanathaburt P (2013) Machine vision techniques for motorcycle safety helmet detection. In: 2013 28th International conference on image and vision computing New Zealand. IEEE, pp 35–40

  36. Wöjcik B, Zarski M, Ksiazek K, Miszczak JA, Skibniewski MJ (2021) Hard hat wearing detection based on head keypoint localization. arXiv:2106.10944

  37. Xi E, Bing S, Jin Y (2017) Capsule network performance on complex data. arXiv:1712.03480

  38. Xiong R, Tang P (2021) Pose guided anchoring for detecting proper use of personal protective equipment. Autom Constr 130:103828

    Article  Google Scholar 

  39. Yang Z, Wang X (2019) Reducing the dilution: an analysis of the information sensitiveness of capsule network with a practical improvement method. arXiv:1903.10588

  40. Yi L, Zhang Q, Zhang D, Han J (2019) Employing deep part-object relationships for salient object detection. In: Proceedings of the IEEE/CVF international conference on computer vision. IEEE, pp 1232–1241

  41. Yue S, Zhang Q, Shao D, Fan Y u, Bai J (2022) Safety helmet wearing status detection based on improved boosted random ferns. Multimed Tools Appl 81(12):16783–16796

    Article  Google Scholar 

  42. Zhang X, Zhou X, Lin M, Sun J (2017) Shufflenet: an extremely efficient convolutional neural network for mobile devices, 6848–6856

Download references

Author information

Author notes

    Authors and Affiliations

    1. School of Computer Science and Artificial Intelligence, Changzhou University, Changzhou, 213164, China

      Jun Liu, Xuhua Xian, Zhenjie Hou, Jiuzhen Liang & Hao Liu

    Authors
    1. Jun Liu
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Xuhua Xian
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Zhenjie Hou
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Jiuzhen Liang
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Hao Liu
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Jun Liu.

    Ethics declarations

    Conflict of Interests

    No potential conflict of interest was reported by the authors.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Xuhua Xian, Zhenjie Hou, Jiuzhen Liang and Hao Liu contributed equally to this work.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Liu, J., Xian, X., Hou, Z. et al. Safety helmet wearing correctly detection based on capsule network. Multimed Tools Appl 83, 6351–6372 (2024). https://doi.org/10.1007/s11042-023-15309-w

    Download citation

    • Received:

    • Revised:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11042-023-15309-w

    Keywords

    Search

    Navigation