Abstract
Human Activity Recognition (HAR) is a branch of computer science that uses raw time-series data information from embedded smartphone sensors and wearable devices to infer human actions. It has aroused considerable interest in various smart home contexts, particularly for constantly monitoring human behavior in an ecologically friendly atmosphere for elderly people and rehabilitation. Data collection, feature extraction from noise and distortion, feature selection, and pre-processing and categorization are among the operating components of a typical HAR system. Extraction of feature and selection strategies have recently been developed using cutting-edge approaches and traditional machine learning classifiers. The majority of the solutions, on the other hand, rely on simple feature extraction algorithms that are unable to detect complex behaviors. Deep learning techniques are often utilized in different HAR approaches to recover features and classification swiftly because of the introduction and development of vast computing resources. The vast majority of solutions, on the other hand, depend on simplistic feature extraction algorithms incapable of recognizing complicated behaviors. Due to advancements in high computational capabilities, deep learning algorithms are now often utilized in HAR methods to efficiently extract meaningful features which can successfully categorize sensor data. In this chapter, we present a hybrid deep learning-based classification model comprising of Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM), which is named CNN-LSTM. The proposed hybrid deep learning model has been tested over three benchmark HAR datasets: MHEALTH, OPPORTUNITY, and HARTH. On the aforementioned datasets, the proposed hybrid model obtained 99.07%, 95.2%, and 94.68% classification accuracies, respectively, which is quite impressive. The source code of the proposed work can be accessed by using the following link: https://github.com/DSharma05/Human-Activity-Recognition-using-hybrid-Deep-learning-approach.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bhattacharya, S., Shaw, V., Singh, P.K., Sarkar, R., Bhattacharjee, D.: SV-NET: a deep learning approach to video based human activity recognition. In: Proceedings of the 11th International Conference on Soft Computing and Pattern Recognition, vol. 1182, pp. 10–20 (2019). https://doi.org/10.1007/978-3-030-49345-5_2
Chakraborty, S., Mondal, R., Singh, P.K., Sarkar, R., Bhattacharjee, D.: Transfer learning with fine tuning for human action recognition from still images. Multimed. Tools Appl. 80, 20547–20578 (2021). https://doi.org/10.1007/s11042-021-10753-y
Banos, O., Villalonga, C., Garcia, R., Saez, A., Damas, M., Holgado-Terriza, J.A., Lee, S., Pomares, H., Rojas, I.: Design implementation and validation of a novel open framework for agile development of mobile health applications. Biomed. Eng. Online 14, 1–20 (2015). https://doi.org/10.1186/1475-925X-14-S2-S6
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Tröster, G., Lukowicz, P., Pirkl, G., Bannach, D., Ferscha, A., Doppler, J., Holzmann, C., Kurz, M., Holl, G., Chavarriaga, R., Sagha, H., Bayati, H., Millán, J.R.: Collecting complex activity data sets in highly rich networked sensor environments. In: Proceedings of the 7th International Conference on Networked Sensing Systems, pp. 233–240 (2010). https://doi.org/10.1109/INSS.2010.5573462
Logacjov, A., Bach, K., Kongsvold, A., Bårdstu, H.B., Mork, P.J.: HARTH: a human activity recognition dataset for machine learning. Sensors 21(23), 1–19 (2021). https://doi.org/10.3390/s21237853
Singh, P.K., Kundu, S., Adhikary, T., Sarkar, R., Bhattacharjee, D.: Progress of human action recognition research in the last ten years: a comprehensive survey. Arch. Comput. Methods Eng. 29(1), 2309–2349 (2022). https://doi.org/10.1007/s11831-021-09681-9
Um, T.T., Babakeshizadeh, V., Kulić, D.: Exercise motion classification from large-scale wearable sensor data using convolutional neural networks. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2385–2390 (2017). https://doi.org/10.1109/IROS.2017.8206051
Lawal, I.A., Bano, S.: Deep human activity recognition with localisation of wearable sensors. IEEE Access 8(1), 155060–155070 (2020). https://doi.org/10.1109/ACCESS.2020.3017681
Qin, Z., Zhang, Y., Meng, S., Qin, Z., Choo, K.K.R.: Imaging and fusing time series for wearable sensor-based human activity recognition. Inf. Fusion 53(1), 80–87 (2020). https://doi.org/10.1016/j.inffus.2019.06.014
Mondal, R., Mukhopadhyay, D., Barua, S., Singh, P.K., Sarkar, R., Bhattacharjee, D.: A study on smartphone sensor-based human activity recognition using deep learning approaches. In: Handbook of Computational Intelligence in Biomedical Engineering and Healthcare, pp. 343–369 (2021). https://doi.org/10.1016/B978-0-12-822260-7.00006-6
Mondal, R., Mukherjee, D., Singh, P.K., Bhateja, V., Sarkar, R.: A new framework for smartphone sensor based human activity recognition using graph neural network. IEEE Sens. J. 21(10), 11461–11468 (2021). https://doi.org/10.1109/JSEN.2020.3015726
Bhattacharya, D., Sharma, D., Kim, W., Ijaz, M.F., Singh, P.K.: Ensem-HAR: an ensemble deep learning model for smartphone sensor-based human activity recognition for measurement of elderly health monitoring. Biosensors 12, 1–25 (2022). https://doi.org/10.3390/bios12060393
Das, A., Sil, P., Singh, P.K., Bhateja, V., Sarkar, R.: MMHAR-EnsemNet: a multi-modal human activity recognition model. IEEE Sens. 21(10), 11569–11576 (2021). https://doi.org/10.1109/JSEN.2020.3034614
Mukherjee, D., Mondal, R., Singh, P.K., Sarkar, R., Bhattacharjee, D.: EnsemConvNet: a deep learning approach for human activity recognition using smartphone sensors for healthcare applications. Multimed. Tools Appl. 79(41), 31663–31690 (2020). https://doi.org/10.1007/s11042-020-09537-7
Banerjee, A., Singh, P.K., Sarkar, R.: Fuzzy integral based CNN classifier fusion for 3D skeleton action recognition. IEEE Trans. Circuits Syst. Video Technol. 31, 2206–2216 (2021). https://doi.org/10.1109/TCSVT.2020.3019293
Ghosh, R., Chattopadhyay, S., Singh, P.K.: Recognizing human activities of daily living using mobile sensors for health monitoring. In: Internet of Things and Data Mining for Modern Engineering and Healthcare Applications (2022). https://doi.org/10.1201/9781003217398-3
Wang, J., Chen, Y., Hao, S., Peng, X., Hu, L.: Deep learning for sensor-based activity recognition: a survey. Pattern Recogn. Lett. 119(1), 3–11 (2019). https://doi.org/10.1016/j.patrec.2018.02.010
Chen, Y., Xue, Y.: A deep learning approach to human activity recognition based on a single accelerometer. In: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, pp. 1488–1492 (2015). https://doi.org/10.1109/SMC.2015.263
Ronao, C.A., Cho, S.B.: Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst. Appl. 59(1), 235–244 (2016). https://doi.org/10.1016/j.eswa.2016.04.032
Wan, S., Qi, L., Xu, X., Tong, C., Gu, Z.: Deep learning models for real-time human activity recognition with smartphones. Mob. Netw. Appl. 25(2), 743–755 (2020). https://doi.org/10.1007/s11036-019-01445-x
Tang, Y., Teng, Q., Zhang, L., Min, F., He, J.: Efficient convolutional neural networks with smaller filters for human activity recognition using wearable sensors (2020). arXiv preprint arXiv:2005.03948
Cheng, X., Zhang, L., Tang, Y., Liu, Y., Wu, H., He, J.: Real-time human activity recognition using conditionally parametrized convolutions on mobile and wearable devices. arXiv preprint arXiv:2006.03259, 1–10 (2020). https://doi.org/10.48550/arXiv.2006.03259
Rashid, N., Demirel, B.U., Faruque, M.A.A.: AHAR: adaptive CNN for energy-efficient human activity recognition in low-power edge devices. IEEE Internet Things J. 9(15), 13041–13051 (2021). https://doi.org/10.1109/jiot.2022.3140465
Zhao, Y., Yang, R., Chevalier, G., Xu, X., Zhang, Z.: Deep residual bidir-LSTM for human activity recognition using wearable sensors. Math. Probl. Eng. 2018(1), 1–13 (2018). https://doi.org/10.1155/2018/7316954
Sun, J., Fu, Y., Li, S., He, J., Xu, C., Tan, L.: Sequential human activity recognition based on deep convolutional network and extreme learning machine using wearable sensors. J. Sens. 2018(1), 1–10 (2018). https://doi.org/10.1155/2018/8580959
Xia, K., Huang, J., Wang, H.: LSTM-CNN architecture for human activity recognition. IEEE Access 8(1), 56855–56866 (2020). https://doi.org/10.1109/ACCESS.2020.2982225
Qi, W., Su, H., Aliverti, A.: A smartphone-based adaptive recognition and real-time monitoring system for human activities. IEEE Trans. Hum.-Mach. Syst. 50(5), 414–423 (2020). https://doi.org/10.1109/THMS.2020.2984181
Wang, H., Zhao, J., Li, J., Tian, L., Tu, P., Cao, T., An, Y., Wang, K., Li, S.: Wearable sensor-based human activity recognition using hybrid deep learning techniques. Secur. Commun. Netw. 2020(1), 1–12 (2020). https://doi.org/10.1155/2020/2132138
Lv, T., Wang, X., Jin, L., Xiao, Y., Song, M.: Margin-based deep learning networks for human activity recognition. Sensors 20(7), 2–19 (2020). https://doi.org/10.3390/s20071871
Gao, W., Zhang, L., Teng, Q., He, J., Wu, H.: DanHAR: dual attention network for multimodal human activity recognition using wearable sensors. Appl. Soft Comput. 111(7), 1–12 (2021). https://doi.org/10.1016/J.ASOC.2021.107728
Ordóñez, F.J., Roggen, D.: Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1), 1–25 (2016). https://doi.org/10.3390/s16010115
Khatun, S., Morshed, B.I.: Fully-automated human activity recognition with transition awareness from wearable sensor data for mHealth. In: Proceedings of the IEEE International Conference on Electro/Information Technology, pp. 0934–0938 (2018). https://doi.org/10.1109/EIT.2018.8500135
Singh, S.P., Sharma, M.K., Lay-Ekuakille, A., Gangwar, D., Gupta, S.: Deep ConvLSTM with self-attention for human activity decoding using wearable sensors. IEEE Sens. J. 21, 8575–8582 (2020). https://doi.org/10.1109/JSEN.2020.3045135
Khatun, M.A., Yousuf, M.A., Ahmed, S., Uddin, M.Z., Alyami, S.A., Ashha, S.A., Akhdar, H.F., Khan, A., Azad, A., Moni, M.A.: Deep CNN-LSTM with self-attention model for human activity recognition using wearable sensor. IEEE J. Transl. Eng. Health Med. 10(1), 1–16 (2022). https://doi.org/10.1109/jtehm.2022.3177710
Gumaei, A., Hassan, M.M., Alelaiwi, A., Alsalman, H.: A hybrid deep learning model for human activity recognition using multimodal body sensing data. IEEE Access 7(1), 99152–99160 (2019). https://doi.org/10.1109/ACCESS.2019.2927134
Debache, I., Jeantet, L., Chevallier, D., Bergouignan, A., Sueur, C.: A lean and performant hierarchical model for human activity recognition using body-mounted sensors. Sensors 20(11), 1–12 (2020). https://doi.org/10.3390/s20113090
Jalal, A., Batool, M., Kim, K.: Stochastic recognition of physical activity and healthcare using tri-axial inertial wearable sensors. Appl. Sci. 10(20), 120 (2020). https://doi.org/10.3390/app10207122
Tahir, S.B., Jalal, A., Kim, K.: Wearable inertial sensors for daily activity analysis based on Adam optimization and the maximum entropy Markov model. Entropy 22(5), 1–19 (2020). https://doi.org/10.3390/e22050579
Jordao, A., Nazare, A.C., Sena, J., Schwartz, W.R.: Human activity recognition based on wearable sensor data: a standardization of the state-of-the-art (2018). arXiv preprint arXiv:1806.05226, 1–11. https://doi.org/10.48550/arXiv.1806.05226
MejÃa, J., Ochoa-Zezzatti, A., Contreras-Masse, R., Rivera, G.: Intelligent system for the visual support of caloric intake of food in inhabitants of a smart city using a deep learning model. In: Applications of Hybrid Metaheuristic Algorithms for Image Processing, pp. 441–455 (2020). https://doi.org/10.1007/978-3-030-40977-7_19
Wang, Z., Oates, T.: Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. AAAI Workshop-Tech. Rep. 2015, 40–46 (2015)
Banerjee, A., Bhattacharya, R., Bhateja, V., Singh, P.K., Lay-Ekuakille, A., Sarkar, R.: COFE-Net: an ensemble strategy for computer-aided detection for COVID-19. Measurement 187, 1–14 (2022). https://doi.org/10.1016/j.measurement.2021.110289
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Sharma, D., Roy, A., Bag, S.P., Singh, P.K., Badr, Y. (2023). A Hybrid Deep Learning-Based Approach for Human Activity Recognition Using Wearable Sensors. In: Rivera, G., Rosete, A., Dorronsoro, B., Rangel-Valdez, N. (eds) Innovations in Machine and Deep Learning. Studies in Big Data, vol 134. Springer, Cham. https://doi.org/10.1007/978-3-031-40688-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-40688-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40687-4
Online ISBN: 978-3-031-40688-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)