Skip to main content
SpringerLink
Log in

DeepSafety: a deep neural network-based edge computing framework for detecting unsafe behaviors of construction workers

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Recently, the development and application of artificial intelligence have received widespread research attentions, and one of important applications is accident prevention. Since most accidents on construction sites are caused by construction works’ unsafe behaviors, unsafe behavior detection is desired. Unlike traditional detection model which focuses only on accuracy of detection and ignore efficiency of detection, a deep neural network-based edge computing framework is proposed for detecting unsafe behavior not only efficiently but also precisely. To address efficiency issue, an object detection model and a posture estimation model are established on edge device for extracting feature from surveillance camera streaming, a time series classification model is developed on server for detecting unsafe behavior according to the features extracted from edge devices. Finally, a comprehensive experimental study based on three datasets collected from three real construction sites is conducted. The results showed that the models proposed in this study can achieve 87% in terms of accuracy and 1.5 s in terms of latency. The improve rate of the proposed DeepSafety is higher than 25% in terms of accuracy and 80% in terms of latency. Accordingly, the proposed edge-computing based framework is shown to deliver excellent performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://en.wikipedia.org/wiki/Kinect.

  2. https://skycatch.com.

  3. https://www.newmetrix.com/.

  4. https://pypi.org/project/nanocamera/.

  5. www.reiju.com.tw/en.

  6. www.kedge.com.tw/kedge_en.

References

  • Aggarwal A, Rani A, Sharma P, Kumar M, Shankar A, Alazab M (2022) Prediction of landsliding using univariate forecasting models. Internet Technol Lett 5(1):e209

    Article  Google Scholar 

  • Bochkovskiy A, Wang CY, Liao HYM (2020) Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934

  • Cao Z, Hidalgo Martinez G, Simon T, Wei S, Sheikh YA (2019) Openpose: realtime multi-person 2d pose estimation using part affinity fields. IEEE Trans Pattern Anal Mach Intell 43(1):172–186

    Article  Google Scholar 

  • Cheng B, Xiao B, Wang J, Shi H, Huang TS, Zhang L (2020) Higherhrnet: scale-aware representation learning for bottom-up human pose estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Virtual, pp 5386–5395

  • Chithaluru P, Al-Turjman F, Kumar M, Stephan T (2021) Mtcee-lln: multilayer threshold cluster-based energy-efficient low-power and lossy networks for industrial internet of things. IEEE Internet Things J 9(7):4940–4948

    Article  Google Scholar 

  • Chu X, Yang W, Ouyang W, Ma C, Yuille AL, Wang X (2017) Multi-context attention for human pose estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, Hawaii, USA, pp 1831–1840

  • Delgado MD, Akinade OO, Ahmed AA (2020) Deep learning in the construction industry: a review of present status. J Build Eng 32:101827

    Article  Google Scholar 

  • Ding L, Fang W, Luo H, Love PE, Zhong B, Ouyang X (2018) A deep hybrid learning model to detect unsafe behavior: integrating convolution neural networks and long short-term memory. Autom Constr 86:118–124

    Article  Google Scholar 

  • Guo S, Liu Y, Ni Y, Ni W (2021) Lightweight ssd: real-time lightweight single shot detector for mobile devices. In: Proceedings of 16th international joint conference on computer vision, imaging and computer graphics theory and applications (VISAPP), online streaming, pp 25–35

  • Heinrich HW, Petersen D, Roos N (1980) Industrial accident prevention: a safety management approach. McGraw-Hill, New York, NY

    Google Scholar 

  • Janarthanan R, Maheshwari RU, Shukla PK, Shukla PK, Mirjalili S, Kumar M (2021) Intelligent detection of the pv faults based on artificial neural network and type 2 fuzzy systems. Energies 14(20):6584

    Article  Google Scholar 

  • Kassa S, Gupta P, Kumar M, Stephan T, Kannan R (2021) Rotated majority gate-based 2n-bit full adder design in quantum-dot cellular automata nanotechnology. Circuit World 48(1):48–63

    Article  Google Scholar 

  • Kelm A, Laußat L, Meins-Becker A, Platz D, Khazaee MJ, Costin AM, Helmus M, Teizer J (2013) Mobile passive radio frequency identification (RFID) portal for automated and rapid control of personal protective equipment (PPE) on construction sites. Autom Constr 36:38–52

    Article  Google Scholar 

  • Kumar M, Aggarwal J, Rani A, Stephan T, Shankar A, Mirjalili S (2022) Secure video communication using firefly optimization and visual cryptography. Artif Intell Rev 55(4):2997–3017

    Article  Google Scholar 

  • Liu CC, Ying JJC (2020) Deepsafety: a deep learning framework for unsafe behaviors detection of steel activity in construction projects. In: 2020 international computer symposium (ICS), Tainan, Taiwan. pp 135–140. https://doi.org/10.1109/ICS51289.2020.00036

  • Mikolov T, Karafiát M, Burget L, Černockỳ J, Khudanpur S (2010) Recurrent neural network based language model. In: Eleventh annual conference of the international speech communication association, Makuhari, Chiba, Japan, pp 1045–1048

  • Nadhim EA, Hon C, Xia B, Stewart I, Fang D (2016) Falls from height in the construction industry: a critical review of the scientific literature. Int J Environ Res Public Health 13(7):638

    Article  Google Scholar 

  • Nikouei SY, Chen Y, Song S, Xu R, Choi BY, Faughnan TR (2018) Real-time human detection as an edge service enabled by a lightweight CNN. In: 2018 IEEE international conference on edge computing (EDGE). IEEE, pp 125–129

  • Raheja S, Alshehri M, Mohamed AA, Khaitan S, Kumar M, Stephan T (2022) A smart intuitionistic fuzzy-based framework for round-robin short-term scheduler. J Supercomput 78(4):4655–4679

    Article  Google Scholar 

  • Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, Nevada, USA, pp 779–788

  • Schuster M, Paliwal KK (1997) Networks bidirectional recurrent neural. IEEE Trans Signal Process 45:2673–2681

    Article  Google Scholar 

  • Shi X, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC (2015) Convolutional lstm network: a machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems (NIPS 2015), Montreal, Canada, pp 802–810

  • Tran D, Bourdev L, Fergus R, Torresani L, Paluri M (2015) Learning spatiotemporal features with 3d convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 4489–4497

  • Wang J, Sun K, Cheng T, Jiang B, Deng C, Zhao Y, Liu D, Mu Y, Tan M, Wang X et al (2020) Deep high-resolution representation learning for visual recognition. IEEE Trans Pattern Anal Mach Intell 43(10):3349–3364

    Article  Google Scholar 

  • Wang Y, Liu M, Zheng P, Yang H, Zou J (2020) A smart surface inspection system using faster r-cnn in cloud-edge computing environment. Adv Eng Inform 43:101037

    Article  Google Scholar 

  • Wang C, Zhang F, Zhu X, Ge SS (2022) Low-resolution human pose estimation. Pattern Recognit 126:108579

    Article  Google Scholar 

  • Xiao B, Wu H, Wei Y (2018) Simple baselines for human pose estimation and tracking. In: Proceedings of the European conference on computer vision (ECCV), Munich, Germany, pp 466–481

  • Xu H, Das A, Saenko K (2017) R-c3d: region convolutional 3d network for temporal activity detection. In: Proceedings of the IEEE international conference on computer vision (ICCV),Venice, Italy, pp 5783–5792

  • Yu C, Xiao B, Gao C, Yuan L, Zhang L, Sang N, Wang J (2021) Lite-hrnet: a lightweight high-resolution network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), Virtual, pp 10440–10450

  • Zebin T, Scully PJ, Peek N, Casson AJ, Ozanyan KB (2019) Design and implementation of a convolutional neural network on an edge computing smartphone for human activity recognition. IEEE Access 7:133509–133520

    Article  Google Scholar 

  • Zlatar T, Lago EMG, Soares WA, Baptista JS, Barkokébas B (2019) Falls from height: analysis of 114 cases. Production 29:e20180091. https://doi.org/10.1590/0103-6513.20180091

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics, Physics, and Computing, University of Southern Queensland, Toowoomba, QLD 4350, Australia

    Ji Zhang

  2. Department of Management Information Systems, National Chung Hsing University, Taichung, Taiwan, ROC

    Chia-Chun Liu & Josh Jia-Ching Ying

Authors
  1. Ji Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chia-Chun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Josh Jia-Ching Ying
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Josh Jia-Ching Ying.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Liu, CC. & Ying, J.JC. DeepSafety: a deep neural network-based edge computing framework for detecting unsafe behaviors of construction workers. J Ambient Intell Human Comput 14, 15997–16009 (2023). https://doi.org/10.1007/s12652-023-04554-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-023-04554-4

Keywords

Search

Navigation