Abstract
Accurate and automated classification of workers’ activities is critical for safety and performance monitoring of workers, especially in highly hazardous working conditions. Previous studies have explored automated worker activity classification using wearable sensors with a sole type of data (e.g., acceleration) in controlled lab environments. To further improve the accuracy of worker activity classification with wearable sensors, we collected multimodal data from workers that conduct highway maintenance activities such as crack sealing, and pothole patching, in an Indiana Department of Transportation (INDOT) facility. Several activities were identified through field videos, including crack sealing, transferring material and walking. Two datasets were developed based on the collected data with one containing acceleration data only and the other one fusing acceleration data with multimodal data including heart rate, electrodermal activity (EDA), and skin temperature. The K-nearest neighbors (KNN) models were built to classify workers’ activities for the two datasets respectively. Results showed that the accuracies for detecting crack sealing, transferring material, and walking without the data fusion were 1.0, 1.0 and 0.71. With the data fusion, the accuracies for detecting crack sealing, transferring material, and walking became 1.0, 0.93, and 0.93. The overall accuracy for classifying the three activities increased from 0.9069 to 0.9535 with the data fusion.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brown S, Harris W, Brooks RD, Dong XS (2021) Fatal injury trends in the construction industry. CPWR Data Bull. 1–6
U.S. Bureau of Labor Statistics (2019) Fatal occupational injuries by industry and event or exposure, all United States. https://www.bls.gov/iif/oshwc/cfoi/cftb0331.htm
Fulford R, Standing C (2014) Construction industry productivity and the potential for collaborative practice. Int J Proj Manag 32:315–326. https://doi.org/10.1016/j.ijproman.2013.05.007
Chapman RE, Butry DT, Huang a. L (2010) Measuring and improving U.S. construction productivity. Proc 18th CIB World Build Congr 1–12
Seo J, Han S, Lee S, Kim H (2015) Computer vision techniques for construction safety and health monitoring. Adv Eng Inform 29:239–251. https://doi.org/10.1016/j.aei.2015.02.001
Akhavian R, Behzadan AH (2016) Productivity analysis of construction worker activities using smartphone sensors. In: 16th International Conference on Computer Civil Building Engineering, pp. 1067–1074
Sherafat B, Ahn CR, Akhavian R et al (2020) Automated methods for activity recognition of construction workers and equipment: state-of-the-art review. J Constr Eng Manag 146:03120002. https://doi.org/10.1061/(asce)co.1943-7862.0001843
Akhavian R, Behzadan AH (2016) Smartphone-based construction workers’ activity recognition and classification. Autom Constr 71:198–209. https://doi.org/10.1016/j.autcon.2016.08.015
Bangaru SS, Wang C, Busam SA, Aghazadeh F (2021) ANN-based automated scaffold builder activity recognition through wearable EMG and IMU sensors. Autom Constr 126:103653. https://doi.org/10.1016/j.autcon.2021.103653
Jebelli H, Choi B, Lee S (2019) Application of wearable biosensors to construction sites. i: assessing workers’ stress. J Constr Eng Manag 145:1–12. https://doi.org/10.1061/(asce)co.1943-7862.0001729
Liu W, Meng Q, Li Z, Hu X (2021) Applications of computer vision in monitoring the unsafe behavior of construction workers: current status and challenges. Buildings 11:409. https://doi.org/10.3390/buildings11090409
Ryu J, Seo J, Jebelli H, Lee S (2019) Automated action recognition using an accelerometer-embedded wristband-type activity tracker. J Constr Eng Manag 145:04018114. https://doi.org/10.1061/(asce)co.1943-7862.0001579
Kim K, Cho YK (2021) Automatic recognition of workers’ motions in highway construction by using motion sensors and long short-term memory networks. J Constr Eng Manag 147:04020184. https://doi.org/10.1061/(asce)co.1943-7862.0002001
Sanhudo L, Calvetti D, Martins JP et al (2021) Activity classification using accelerometers and machine learning for complex construction worker activities. J Build Eng 35:102001. https://doi.org/10.1016/j.jobe.2020.102001
Randhawa P, Shanthagiri V, Kumar A, Yadav V (2020) Human activity detection using machine learning methods from wearable sensors. Sens Rev 40:591–603. https://doi.org/10.1108/SR-02-2020-0027
Kim K, Cho YK (2021) Automatic recognition of workers’ motions in highway construction by using motion sensors and long short-term memory networks. J Constr Eng Manag 147:1–12. https://doi.org/10.1061/(asce)co.1943-7862.0002001
Arif A, Javaid N, Aldegheishem A, Alrajeh N (2021) Big data analytics for identifying electricity theft using machine learning approaches in microgrids for smart communities. Concurr Comput Pract Exp 33:1–21. https://doi.org/10.1002/cpe.6316
Figueroa G, Chen YS, Avila N, Chu CC (2018) Improved practices in machine learning algorithms for NTL detection with imbalanced data. IEEE Power Energy Soc Gen Meet 2018:1–5. https://doi.org/10.1109/PESGM.2017.8273852
Huang C, Loy CC, Tang X (2016) Learning Deep Representation for Mood Classification in Microblog. 2016 IEEE Conf Comput Vis Pattern Recognit
Tan M, Zhang S, Wu L (2020) Mutual kNN based spectral clustering. Neural Comput Appl 32:6435–6442. https://doi.org/10.1007/s00521-018-3836-z
Pawlovsky AP, Nagahashi M (2014) A method to select a good setting for the kNN algorithm when using it for breast cancer prognosis. 2014 IEEE-EMBS Int Conf Biomed Heal Informatics. BHI 2014:189–192. https://doi.org/10.1109/BHI.2014.6864336
Nagar N, Ben Tal N, Pupko T (2022) EvoRator: prediction of residue-level evolutionary rates from protein structures using machine learning. J Mol Biol 434:167538. https://doi.org/10.1016/j.jmb.2022.167538
Ogunseiju OR, Olayiwola J, Akanmu AA, Nnaji C (2021) Recognition of workers’ actions from time-series signal images using deep convolutional neural network. Smart Sustain Built Environ 11:812–831. https://doi.org/10.1108/SASBE-11-2020-0170
Acknowledgements
The study is supported by the award SPR 4630 from the Joint Transportation Research Program administered by the Indiana Department of Transportation and Purdue University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tian, C., Chen, Y., Feng, Y., Zhang, J. (2024). Worker Activity Classification Using Multimodal Data Fusion from Wearable Sensors. In: Skatulla, S., Beushausen, H. (eds) Advances in Information Technology in Civil and Building Engineering. ICCCBE 2022. Lecture Notes in Civil Engineering, vol 357. Springer, Cham. https://doi.org/10.1007/978-3-031-35399-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-35399-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35398-7
Online ISBN: 978-3-031-35399-4
eBook Packages: EngineeringEngineering (R0)