Skip to main content
SpringerLink
Log in

A Hybrid Deep Learning-Based Approach for Human Activity Recognition Using Wearable Sensors

  • Chapter
  • First Online:
Innovations in Machine and Deep Learning

Part of the book series: Studies in Big Data ((SBD,volume 134))

  • 247 Accesses

Abstract

Human Activity Recognition (HAR) is a branch of computer science that uses raw time-series data information from embedded smartphone sensors and wearable devices to infer human actions. It has aroused considerable interest in various smart home contexts, particularly for constantly monitoring human behavior in an ecologically friendly atmosphere for elderly people and rehabilitation. Data collection, feature extraction from noise and distortion, feature selection, and pre-processing and categorization are among the operating components of a typical HAR system. Extraction of feature and selection strategies have recently been developed using cutting-edge approaches and traditional machine learning classifiers. The majority of the solutions, on the other hand, rely on simple feature extraction algorithms that are unable to detect complex behaviors. Deep learning techniques are often utilized in different HAR approaches to recover features and classification swiftly because of the introduction and development of vast computing resources. The vast majority of solutions, on the other hand, depend on simplistic feature extraction algorithms incapable of recognizing complicated behaviors. Due to advancements in high computational capabilities, deep learning algorithms are now often utilized in HAR methods to efficiently extract meaningful features which can successfully categorize sensor data. In this chapter, we present a hybrid deep learning-based classification model comprising of Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM), which is named CNN-LSTM. The proposed hybrid deep learning model has been tested over three benchmark HAR datasets: MHEALTH, OPPORTUNITY, and HARTH. On the aforementioned datasets, the proposed hybrid model obtained 99.07%, 95.2%, and 94.68% classification accuracies, respectively, which is quite impressive. The source code of the proposed work can be accessed by using the following link: https://github.com/DSharma05/Human-Activity-Recognition-using-hybrid-Deep-learning-approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bhattacharya, S., Shaw, V., Singh, P.K., Sarkar, R., Bhattacharjee, D.: SV-NET: a deep learning approach to video based human activity recognition. In: Proceedings of the 11th International Conference on Soft Computing and Pattern Recognition, vol. 1182, pp. 10–20 (2019). https://doi.org/10.1007/978-3-030-49345-5_2

  2. Chakraborty, S., Mondal, R., Singh, P.K., Sarkar, R., Bhattacharjee, D.: Transfer learning with fine tuning for human action recognition from still images. Multimed. Tools Appl. 80, 20547–20578 (2021). https://doi.org/10.1007/s11042-021-10753-y

    Article  Google Scholar 

  3. Banos, O., Villalonga, C., Garcia, R., Saez, A., Damas, M., Holgado-Terriza, J.A., Lee, S., Pomares, H., Rojas, I.: Design implementation and validation of a novel open framework for agile development of mobile health applications. Biomed. Eng. Online 14, 1–20 (2015). https://doi.org/10.1186/1475-925X-14-S2-S6

  4. Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Tröster, G., Lukowicz, P., Pirkl, G., Bannach, D., Ferscha, A., Doppler, J., Holzmann, C., Kurz, M., Holl, G., Chavarriaga, R., Sagha, H., Bayati, H., Millán, J.R.: Collecting complex activity data sets in highly rich networked sensor environments. In: Proceedings of the 7th International Conference on Networked Sensing Systems, pp. 233–240 (2010). https://doi.org/10.1109/INSS.2010.5573462

  5. Logacjov, A., Bach, K., Kongsvold, A., Bårdstu, H.B., Mork, P.J.: HARTH: a human activity recognition dataset for machine learning. Sensors 21(23), 1–19 (2021). https://doi.org/10.3390/s21237853

    Article  Google Scholar 

  6. Singh, P.K., Kundu, S., Adhikary, T., Sarkar, R., Bhattacharjee, D.: Progress of human action recognition research in the last ten years: a comprehensive survey. Arch. Comput. Methods Eng. 29(1), 2309–2349 (2022). https://doi.org/10.1007/s11831-021-09681-9

    Article  Google Scholar 

  7. Um, T.T., Babakeshizadeh, V., Kulić, D.: Exercise motion classification from large-scale wearable sensor data using convolutional neural networks. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2385–2390 (2017). https://doi.org/10.1109/IROS.2017.8206051

  8. Lawal, I.A., Bano, S.: Deep human activity recognition with localisation of wearable sensors. IEEE Access 8(1), 155060–155070 (2020). https://doi.org/10.1109/ACCESS.2020.3017681

    Article  Google Scholar 

  9. Qin, Z., Zhang, Y., Meng, S., Qin, Z., Choo, K.K.R.: Imaging and fusing time series for wearable sensor-based human activity recognition. Inf. Fusion 53(1), 80–87 (2020). https://doi.org/10.1016/j.inffus.2019.06.014

    Article  Google Scholar 

  10. Mondal, R., Mukhopadhyay, D., Barua, S., Singh, P.K., Sarkar, R., Bhattacharjee, D.: A study on smartphone sensor-based human activity recognition using deep learning approaches. In: Handbook of Computational Intelligence in Biomedical Engineering and Healthcare, pp. 343–369 (2021). https://doi.org/10.1016/B978-0-12-822260-7.00006-6

  11. Mondal, R., Mukherjee, D., Singh, P.K., Bhateja, V., Sarkar, R.: A new framework for smartphone sensor based human activity recognition using graph neural network. IEEE Sens. J. 21(10), 11461–11468 (2021). https://doi.org/10.1109/JSEN.2020.3015726

    Article  Google Scholar 

  12. Bhattacharya, D., Sharma, D., Kim, W., Ijaz, M.F., Singh, P.K.: Ensem-HAR: an ensemble deep learning model for smartphone sensor-based human activity recognition for measurement of elderly health monitoring. Biosensors 12, 1–25 (2022). https://doi.org/10.3390/bios12060393

    Article  Google Scholar 

  13. Das, A., Sil, P., Singh, P.K., Bhateja, V., Sarkar, R.: MMHAR-EnsemNet: a multi-modal human activity recognition model. IEEE Sens. 21(10), 11569–11576 (2021). https://doi.org/10.1109/JSEN.2020.3034614

    Article  Google Scholar 

  14. Mukherjee, D., Mondal, R., Singh, P.K., Sarkar, R., Bhattacharjee, D.: EnsemConvNet: a deep learning approach for human activity recognition using smartphone sensors for healthcare applications. Multimed. Tools Appl. 79(41), 31663–31690 (2020). https://doi.org/10.1007/s11042-020-09537-7

    Article  Google Scholar 

  15. Banerjee, A., Singh, P.K., Sarkar, R.: Fuzzy integral based CNN classifier fusion for 3D skeleton action recognition. IEEE Trans. Circuits Syst. Video Technol. 31, 2206–2216 (2021). https://doi.org/10.1109/TCSVT.2020.3019293

    Article  Google Scholar 

  16. Ghosh, R., Chattopadhyay, S., Singh, P.K.: Recognizing human activities of daily living using mobile sensors for health monitoring. In: Internet of Things and Data Mining for Modern Engineering and Healthcare Applications (2022). https://doi.org/10.1201/9781003217398-3

  17. Wang, J., Chen, Y., Hao, S., Peng, X., Hu, L.: Deep learning for sensor-based activity recognition: a survey. Pattern Recogn. Lett. 119(1), 3–11 (2019). https://doi.org/10.1016/j.patrec.2018.02.010

    Article  Google Scholar 

  18. Chen, Y., Xue, Y.: A deep learning approach to human activity recognition based on a single accelerometer. In: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, pp. 1488–1492 (2015). https://doi.org/10.1109/SMC.2015.263

  19. Ronao, C.A., Cho, S.B.: Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst. Appl. 59(1), 235–244 (2016). https://doi.org/10.1016/j.eswa.2016.04.032

    Article  Google Scholar 

  20. Wan, S., Qi, L., Xu, X., Tong, C., Gu, Z.: Deep learning models for real-time human activity recognition with smartphones. Mob. Netw. Appl. 25(2), 743–755 (2020). https://doi.org/10.1007/s11036-019-01445-x

    Article  Google Scholar 

  21. Tang, Y., Teng, Q., Zhang, L., Min, F., He, J.: Efficient convolutional neural networks with smaller filters for human activity recognition using wearable sensors (2020). arXiv preprint arXiv:2005.03948

  22. Cheng, X., Zhang, L., Tang, Y., Liu, Y., Wu, H., He, J.: Real-time human activity recognition using conditionally parametrized convolutions on mobile and wearable devices. arXiv preprint arXiv:2006.03259, 1–10 (2020). https://doi.org/10.48550/arXiv.2006.03259

  23. Rashid, N., Demirel, B.U., Faruque, M.A.A.: AHAR: adaptive CNN for energy-efficient human activity recognition in low-power edge devices. IEEE Internet Things J. 9(15), 13041–13051 (2021). https://doi.org/10.1109/jiot.2022.3140465

    Article  Google Scholar 

  24. Zhao, Y., Yang, R., Chevalier, G., Xu, X., Zhang, Z.: Deep residual bidir-LSTM for human activity recognition using wearable sensors. Math. Probl. Eng. 2018(1), 1–13 (2018). https://doi.org/10.1155/2018/7316954

    Article  Google Scholar 

  25. Sun, J., Fu, Y., Li, S., He, J., Xu, C., Tan, L.: Sequential human activity recognition based on deep convolutional network and extreme learning machine using wearable sensors. J. Sens. 2018(1), 1–10 (2018). https://doi.org/10.1155/2018/8580959

    Article  Google Scholar 

  26. Xia, K., Huang, J., Wang, H.: LSTM-CNN architecture for human activity recognition. IEEE Access 8(1), 56855–56866 (2020). https://doi.org/10.1109/ACCESS.2020.2982225

    Article  Google Scholar 

  27. Qi, W., Su, H., Aliverti, A.: A smartphone-based adaptive recognition and real-time monitoring system for human activities. IEEE Trans. Hum.-Mach. Syst. 50(5), 414–423 (2020). https://doi.org/10.1109/THMS.2020.2984181

    Article  Google Scholar 

  28. Wang, H., Zhao, J., Li, J., Tian, L., Tu, P., Cao, T., An, Y., Wang, K., Li, S.: Wearable sensor-based human activity recognition using hybrid deep learning techniques. Secur. Commun. Netw. 2020(1), 1–12 (2020). https://doi.org/10.1155/2020/2132138

    Article  Google Scholar 

  29. Lv, T., Wang, X., Jin, L., Xiao, Y., Song, M.: Margin-based deep learning networks for human activity recognition. Sensors 20(7), 2–19 (2020). https://doi.org/10.3390/s20071871

    Article  Google Scholar 

  30. Gao, W., Zhang, L., Teng, Q., He, J., Wu, H.: DanHAR: dual attention network for multimodal human activity recognition using wearable sensors. Appl. Soft Comput. 111(7), 1–12 (2021). https://doi.org/10.1016/J.ASOC.2021.107728

    Article  Google Scholar 

  31. Ordóñez, F.J., Roggen, D.: Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1), 1–25 (2016). https://doi.org/10.3390/s16010115

    Article  Google Scholar 

  32. Khatun, S., Morshed, B.I.: Fully-automated human activity recognition with transition awareness from wearable sensor data for mHealth. In: Proceedings of the IEEE International Conference on Electro/Information Technology, pp. 0934–0938 (2018). https://doi.org/10.1109/EIT.2018.8500135

  33. Singh, S.P., Sharma, M.K., Lay-Ekuakille, A., Gangwar, D., Gupta, S.: Deep ConvLSTM with self-attention for human activity decoding using wearable sensors. IEEE Sens. J. 21, 8575–8582 (2020). https://doi.org/10.1109/JSEN.2020.3045135

    Article  Google Scholar 

  34. Khatun, M.A., Yousuf, M.A., Ahmed, S., Uddin, M.Z., Alyami, S.A., Ashha, S.A., Akhdar, H.F., Khan, A., Azad, A., Moni, M.A.: Deep CNN-LSTM with self-attention model for human activity recognition using wearable sensor. IEEE J. Transl. Eng. Health Med. 10(1), 1–16 (2022). https://doi.org/10.1109/jtehm.2022.3177710

    Article  Google Scholar 

  35. Gumaei, A., Hassan, M.M., Alelaiwi, A., Alsalman, H.: A hybrid deep learning model for human activity recognition using multimodal body sensing data. IEEE Access 7(1), 99152–99160 (2019). https://doi.org/10.1109/ACCESS.2019.2927134

    Article  Google Scholar 

  36. Debache, I., Jeantet, L., Chevallier, D., Bergouignan, A., Sueur, C.: A lean and performant hierarchical model for human activity recognition using body-mounted sensors. Sensors 20(11), 1–12 (2020). https://doi.org/10.3390/s20113090

    Article  Google Scholar 

  37. Jalal, A., Batool, M., Kim, K.: Stochastic recognition of physical activity and healthcare using tri-axial inertial wearable sensors. Appl. Sci. 10(20), 120 (2020). https://doi.org/10.3390/app10207122

    Article  Google Scholar 

  38. Tahir, S.B., Jalal, A., Kim, K.: Wearable inertial sensors for daily activity analysis based on Adam optimization and the maximum entropy Markov model. Entropy 22(5), 1–19 (2020). https://doi.org/10.3390/e22050579

    Article  Google Scholar 

  39. Jordao, A., Nazare, A.C., Sena, J., Schwartz, W.R.: Human activity recognition based on wearable sensor data: a standardization of the state-of-the-art (2018). arXiv preprint arXiv:1806.05226, 1–11. https://doi.org/10.48550/arXiv.1806.05226

  40. Mejía, J., Ochoa-Zezzatti, A., Contreras-Masse, R., Rivera, G.: Intelligent system for the visual support of caloric intake of food in inhabitants of a smart city using a deep learning model. In: Applications of Hybrid Metaheuristic Algorithms for Image Processing, pp. 441–455 (2020). https://doi.org/10.1007/978-3-030-40977-7_19

  41. Wang, Z., Oates, T.: Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. AAAI Workshop-Tech. Rep. 2015, 40–46 (2015)

    Google Scholar 

  42. Banerjee, A., Bhattacharya, R., Bhateja, V., Singh, P.K., Lay-Ekuakille, A., Sarkar, R.: COFE-Net: an ensemble strategy for computer-aided detection for COVID-19. Measurement 187, 1–14 (2022). https://doi.org/10.1016/j.measurement.2021.110289

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Jadavpur University, Jadavpur University Second Campus, Plot No. 8, Salt Lake Bypass, LB Block, Sector III, Salt Lake City, Kolkata, 700106, West Bengal, India

    Deepak Sharma & Pawan Kumar Singh

  2. School of Computing and Information Technology, Reva University, Bengaluru, 560064, Karnataka, India

    Arup Roy

  3. Department of Medical Biotechnology, College of Life Science and Biotechnology, Dongguk University, Seoul, Republic of Korea

    Sankar Prasad Bag

  4. Pennsylvania State University, Great Valley, 30 East Swedesford Road, Malvern, PA, 19355, USA

    Youakim Badr

Authors
  1. Deepak Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arup Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sankar Prasad Bag
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pawan Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Youakim Badr
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pawan Kumar Singh .

Editor information

Editors and Affiliations

  1. División Multidisciplinaria de Ciudad Universitaria, Universidad Autónoma de Ciudad Juárez, Chihuahua, Mexico

    Gilberto Rivera

  2. Universidad Tecnológica de La Habana, La Habana, Cuba

    Alejandro Rosete

  3. School of Engineering, University of Cadiz, Cadiz, Spain

    Bernabé Dorronsoro

  4. Instituto Tecnológico de Ciudad Madero, Tecnológico Nacional de México, Tamaulipas, Mexico

    Nelson Rangel-Valdez

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Sharma, D., Roy, A., Bag, S.P., Singh, P.K., Badr, Y. (2023). A Hybrid Deep Learning-Based Approach for Human Activity Recognition Using Wearable Sensors. In: Rivera, G., Rosete, A., Dorronsoro, B., Rangel-Valdez, N. (eds) Innovations in Machine and Deep Learning. Studies in Big Data, vol 134. Springer, Cham. https://doi.org/10.1007/978-3-031-40688-1_11

Download citation

Publish with us

Policies and ethics

Search

Navigation