Skip to main content
SpringerLink
Log in

Hybrid deep learning approaches for smartphone sensor-based human activity recognition

  • 1166: Advances of machine learning in data analytics and visual information processing
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Human Activity Recognition (HAR) has become one of the most important research fields to achieve real-time monitoring of human activities for timely decision making in various applications like fall detection, elderly care etc. Now-a-days, most people use smartphones which come with various embedded inertial sensors like accelerometer and gyroscope to monitor acceleration and angular velocity. These smartphone-based sensors have proven to be cost-effective solution in identification of activities belonging to ADL (Activities of Daily Living). Various Machine Learning, Deep learning and hybrid models have been proposed and implemented for HAR. This paper also proposes various hybrid deep learning approaches which combine Deep Neural Networks with other models like LSTM (Long Short Term Memory) Model and GRU (Gated Recurrent Unit) for effective classification of engineered features from CNN (Convolutional Neural Network) Model. A novel architecture that integrates CNN with Random Forest Classifier (DeepCNN-RF) is proposed to add randomness to the model. The proposed models have been tested on publicly available HAR Datasets like UCI HAR and WISDM Activity Recognition Datasets. Experimental results show that the hybrid models outperform the state-of-the-art data mining, machine learning techniques in UCI HAR and WISDM with an overall maximum accuracy of 97.77% and 98.2% respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Abbaspour S, Fotouhi F, Sedaghatbaf A, Fotouhi H, Vahabi M, Linden M (2020) A comparative analysis of hybrid deep learning models for human activity recognition. Sensors 20(19):5707

  2. Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz JL (2012) Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In International workshop on ambient assisted living. Springer, Berlin, Heidelberg, pp 216–223)

  3. Bayat A, Pomplun M, Tran D (2014) A Study on Human Activity Recognition Using Accelerometer Data from Smartphones. Procedia Comput Sci 34:450–457. https://doi.org/10.1016/j.procs.2014.07.009

    Article  Google Scholar 

  4. Bux A, Angelov P, Habib Z (2017) Vision based human activity recognition: a review. In Advances in Computational Intelligence Systems. Cham, Springer, pp 341–371

  5. Feng Z, Mo L, Li M (2015) A random forest-based ensemble method for activity recognition. In 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, pp 5074–5077

  6. Ha S, Yun JM, Choi S (2015) Multi-modal convolutional neural networks for activity recognition. In 2015 IEEE International conference on systems, man, and cybernetics. IEEE, pp 3017–3022

  7. Ignatov A (2017) Real-time human activity recognition from accelerometer data using Convolutional Neural Networks. Appl Soft Comput 62:62–922. https://doi.org/10.1016/j.asoc.2017.09.027

    Article  Google Scholar 

  8. Ijjina EP, Chalavadi KM (2016) Hybrid deep neural network model for human action recognition. Int J Appl Soft Comput 46:936–952

  9. Ioffe S, Szegedy C (2015) Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.” ArXiv abs/1502.03167: n. pag

  10. Jiang W, Yin Z (2015) Human activity recognition using wearable sensors by deep convolutional neural networks. In Proceedings of the 23rd ACM international conference on Multimedia, pp 1307–1310

  11. Jordao A, Nazare Jr. AC, Sena J, Schwartz WR (2018) Human activity recognition based on wearable sensor data: A standardization of the state-of-the-art. arXiv preprint arXiv:1806.05226

  12. Ke SR, Thuc HLU, Lee YJ, Hwang J-N, Yoo J-H, Choi K-H (2013) A review on video-based human activity recognition. Computers 2(2):88–131

    Article  Google Scholar 

  13. Krishnan N, Cook D (2014) Activity recognition on streaming sensor data. Pervasive Mob Comput 10:138–154

    Article  Google Scholar 

  14. Kwapisz JR, Weiss GM, Moore SA (2010) Activity recognition using cell phone accelerometers. Proceedings of the fourth international workshop on knowledge discovery from sensor data (at KDD-10), Washington DC

    Google Scholar 

  15. Liu Q, Zhou Z, Shakya S, Uduthalapally P, Qiao M, Sung A (2018) Smartphone sensor-based activity recognition by using machine learning and deep learning algorithms. Int J Machine Learn Comput 8:121–126. https://doi.org/10.18178/ijmlc.2018.8.2.674

    Article  Google Scholar 

  16. Mandong A, Munir U (2018) Smartphone based activity recognition using k-nearest neighbor algorithm. In Proceedings of the International Conference on Engineering Technologies, Konya, pp 26–28

  17. Mekruksavanich S, Hnoohom N, Jitpattanakul A (2018) Smartwatch-based sitting detection with human activity recognition for office workers syndrome. In 2018 International ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI-NCON). IEEE, pp 160–164

  18. Murad A, Pyun J-Y (2017) Deep recurrent neural networks for human activity recognition. Sensors 17:2556. https://doi.org/10.3390/s17112556

    Article  Google Scholar 

  19. Okai J, Paraschiakos S, Beekman M, Knobbe A, de Sá CR (2019) building robust models for human activity recognition from raw accelerometers data using gated recurrent units and long short term memory neural networks. 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, pp 2486–2491

    Google Scholar 

  20. Ordóñez F, Roggen D (2016) Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16:115. https://doi.org/10.3390/s16010115

    Article  Google Scholar 

  21. Ortiz LJ, Olaya AG, Borrajo D (2011) A Dynamic Sliding Window Approach for Activity Recognition. In: Konstan JA, Conejo R, Marzo JL, Oliver N (eds) User Modeling, Adaption and Personalization. UMAP 2011. Lecture notes in computer science, vol 6787. Springer, Berlin

    Google Scholar 

  22. Ronao C, Cho S-B (2016) Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst Appl 59:235–244. https://doi.org/10.1016/j.eswa.2016.04.032

    Article  Google Scholar 

  23. Sai NL, Samuel K, Naga BK. Vamsidhar E (2019) Performance analysis on human activity detection using knn and random forest. International Journal of Innovative Technology and Exploring Engineering (IJITEE) 8(7):2817–2821

  24. Sani S, Wiratunga N, Massie S, Cooper K (2017) kNN Sampling for Personalised Human Activity Recognition. In: Aha D, Lieber J (eds) Case-Based Reasoning Research and Development. ICCBR 2017. Lecture notes in computer science, vol 10339. Springer, Cham

    Google Scholar 

  25. Sikder N, Chowdhury Md, Arif A, Nahid A (2019) Human Activity Recognition Using Multichannel Convolutional Neural Network

  26. Walse KH, Dharaskar RV, Thakare VM (2016) Pca based optimal ann classifiers for human activity recognition using mobile sensors data. In Proceedings of First International Conference on Information and Communication Technology for Intelligent Systems, vol 1. Springer, Cham, pp 429–436

  27. Wang A, Chen G, Yang J, Zhao S, Chang C (2016) A Comparative Study on Human Activity Recognition Using Inertial Sensors in a Smartphone. IEEE Sens J 16(11):4566–4578. https://doi.org/10.1109/JSEN.2016.2545708

  28. Wang J, Chen Y, Hao S, Peng X, Lisha H (2017) Deep learning for sensor-based activity Recognition: A Survey. Pattern Recognition Lett. https://doi.org/10.1016/j.patrec.2018.02.010

  29. Zeng M (2014) convolutional neural networks for human activity recognition using Mobile sensors, 6th International Conference on Mobile Computing. Applications and Services, Austin, pp 197–205. https://doi.org/10.4108/icst.mobicase.2014.257786

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering (SCOPE ), Vellore Institute of Technology, Chennai, India

    Vasundhara Ghate & Sweetlin Hemalatha C

Authors
  1. Vasundhara Ghate
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sweetlin Hemalatha C
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sweetlin Hemalatha C.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghate, V., C, S.H. Hybrid deep learning approaches for smartphone sensor-based human activity recognition. Multimed Tools Appl 80, 35585–35604 (2021). https://doi.org/10.1007/s11042-020-10478-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10478-4

Keywords

Search

Navigation