Skip to main content
  • 1174: Futuristic Trends and Innovations in Multimedia Systems Using Big Data, IoT and Cloud Technologies (FTIMS)
  • Published:

Human activity recognition by combining external features with accelerometer sensor data using deep learning network model

Multimedia Tools and Applications volume 81, pages 34633–34652 (2022)Cite this article

Abstract

Various Human Activities are classified through time-series data generated by the sensors of wearable devices. Many real-time scenarios such as Healthcare Surveillance, Smart Cities and Intelligent surveillance etc. are based upon Human Activity Recognition. Despite the popularity of local features-based approaches and machine learning approaches, it fails to capture adequate temporal information. In this paper, the deep convolutional neural model has been proposed by combining external features, i.e. orientation invariant (||\(v\)||) and consecutive point trajectory information (||\(\Delta v\)||) with tri-axis data of the accelerometer. The proposed external features based approach experimented on three different deep learning architecture, namely Long-Short Term Memory (LSTM), Convolutional Neural Networks (CNN) and Convolution Long-Short Term Memory (ConvLSTM). Accuracy of the algorithms radically improve with the additional input feature ||\(v\)|| and ||\(\Delta v\)|| along with tri-axis data of accelerometer. The results show that the performance of all three LSTM, CNN and ConvLSTM models is better to compare with the state of art methods on WISDOM dataset and Activity dataset also the performance of ConvLSTM is 98.41% for WISDOM dataset and 98.04 for activity dataset, which is higher than that of CNN and LSTM model used in this paper.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

References

  1. Anjum A, Ilyas MU (2013) Activity recognition using smartphone sensors. In: 2013 ieee 10th consumer communications and networking conference (ccnc). IEEE, pp 914–919

  2. Cao L, Wang Y, Zhang B, Jin Q, Vasilakos AV (2018) GCHAR: An efficient Group-based Context—Aware human activity recognition on smartphone. J Parallel Distrib Comput 1(118):67–80

    Article  Google Scholar 

  3. Chen Y, Xue Y (2015) A deep learning approach to human activity recognition based on single accelerometer. In: Systems, man, and cybernetics (smc). IEEE International Conference, pp 1488–1492

  4. Chen Y, Zhong K, Zhang J, Sun Q, Zhao X (2016) LSTM networks for mobile human activity recognition. In: 2016 International Conference on Artificial Intelligence: Technologies and Applications Atlantis Press

  5. Cowen A (2018) “Exploring Acceleration with a Sensor App: Science Buddies Blog.” Science Buddies, Science Buddies, 9 July 2018. https://www.sciencebuddies.org/google-science-journal-app-tutorial-part5-accelerometer#tryaccelerometer

  6. Figo D, Diniz PC, Ferreira DR, Cardoso JM (2010) Preprocessing techniques for context recognition from accelerometer data. Pers Ubiquit Comput 14(7):645–662

    Article  Google Scholar 

  7. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  8. Jiang W, Yin Z (2015) Human activity recognition using wearable sensors by deep convolutional neural networks. In: proceedings of the 23rd ACM international conference on Multimedia, pp 1307–1310

  9. Khan MA, Roy N (2017) TransAct: Transfer learning enabled activity recognition. In: 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, pp 545–550

  10. Kim Y (2014) Convolutional neural networks for sentence classification, arXiv preprint

  11. Klasnja P, Pratt W (2012) Healthcare in the pocket: Mapping the space of mobile-phone health interventions. J Biomed Inf 45:184–198

    Article  Google Scholar 

  12. Kwapisz JR, Weiss GM, Moore SA (2010) Activity recognition using cell phone accelerometers. In: proceedings of the Fourth International Workshop on knowledge discovery from sensor data (at KDD-10). Washington DC

  13. Kwon MC, Choi S (2018) Recognition of daily human activity using an artificial neural network and smartwatch. Wirel Commun Mob Comput 2018

  14. Lee SM, Yoon SM, Cho H (2017) Human activity recognition from accelerometer data using Convolutional Neural Network. In: 2017 IEEE International conference on big data and smart computing (BigComp). IEEE, pp.131–134

  15. Lu J, Zheng X, Sheng M, Jin J, Yu S (2020) Efficient human activity recognition using a single wearable sensor. IEEE Internet Things J 7(11):11137–11146

    Article  Google Scholar 

  16. Matthews BW (1975) Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim Biophys Acta Protein Structure 405(2):442–451

  17. O'Shea K, Nash R (2015) An introduction to convolutional neural networks. arXiv preprint

  18. Pang J (2018) Human Activity Recognition Based on Transfer Learning. Graduate Theses and Dissertations. https://www.digitalcommons.usf.edu/etd/7558

  19. Peppas K, Tsolakis AC, Krinidis S, Tzovaras D (2020) Real-Time Physical Activity Recognition on Smart Mobile Devices Using Convolutional Neural Networks. Appl Sci 10(23):8482

    Article  Google Scholar 

  20. Pienaar SW, Malekian R (2019) Human activity recognition using LSTM-RNN deep neural network architecture. In: 2019 IEEE 2nd Wireless Africa Conference (WAC). IEEE, pp 1–5

  21. Qin Z, Zhang Y, Meng S, Qin Z, Choo KKR (2020) Imaging and fusing time series for wearable sensor-based human activity recognition. Information Fusion 53:80–87

    Article  Google Scholar 

  22. Ronao CA, Cho SB (2015) Deep convolutional neural networks for human activity recognition with smartphone sensors. In: International Conference on Neural Information Processing. Springer, Cham, pp 46–53

  23. Ronao CA, Cho SB (2016) Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst Appl 15(59):235–244

    Article  Google Scholar 

  24. Shakya SR, Zhang C, Zhou Z (2018) Comparative study of machine learning and deep learning architecture for human activity recognition using accelerometer data. Int J Mach Learn Comput 8:577–582

    Google Scholar 

  25. Shoaib M, Bosch S, Incel OD, Scholten H, Havinga PJ (2014) Fusion of smartphone motion sensors for physical activity recognition. Sensors 14(6):10146–10176

    Article  Google Scholar 

  26. Singh SP, Lay-Ekuakille A, Gangwar D, Sharma MK, Gupta S (2020) Deep ConvLSTM with self-attention for human activity decoding using wearables. arXiv preprint

  27. Sokolova M, Lapalme G (2009) A systematic analysis of performance measures for classification tasks. Inf Process Manage 45(4):427–437

    Article  Google Scholar 

  28. Torres-Huitzil C, Alvarez-Landero A (2015) Accelerometer-based human activity recognition in smartphones for healthcare services. In: Mobile Health. Springer, Cham, pp 147–169

  29. Voicu RA, Dobre C, Bajenaru L, Ciobanu RI (2019) Human physical activity recognition using smartphone sensors. Sensors 19(3):458

    Article  Google Scholar 

  30. Walse KH, Dharaskar RV, Thakare VM (2016) A study of human activity recognition using AdaBoost classifiers on WISDM dataset. The Institute of Integrative Omics and Applied Biotechnology Journal 7(2):68–76

    Google Scholar 

  31. Wang A, Chen G, Yang J, Zhao S, Chang CY (2016) A comparative study on human activity recognition using inertial sensors in a smartphone. IEEE Sens J 16(11):4566–4578

    Article  Google Scholar 

  32. Wu W, Dasgupta S, Ramirez EE, Peterson C, Norman GJ (2012) Classification Accuracies of Physical Activities Using Smartphone Motion Sensors. J Med Internet 14:e130

  33. Xingjian SH, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC (2015) Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In: Advances in neural information processing system, pp 802–810

Download references

Author information

Authors and Affiliations

  1. I K Gujral Punjab Technical University, Kapurthala, Ibban, India

    Neeraj Varshney

  2. I K Gujral Punjab Technical University, Hoshiarpur Campus, India

    Brijesh Bakariya

  3. Guru Ghasidas, Vishwavidyalaya, Bilaspur, India

    Alok Kumar Singh Kushwaha

  4. Dhirubhai Ambani Institute of Information and Communication Technology, Gandhinagar, India

    Manish Khare

Authors
  1. Neeraj Varshney
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brijesh Bakariya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alok Kumar Singh Kushwaha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Manish Khare
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Neeraj Varshney.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Varshney, N., Bakariya, B., Kushwaha, A.K.S. et al. Human activity recognition by combining external features with accelerometer sensor data using deep learning network model. Multimed Tools Appl 81, 34633–34652 (2022). https://doi.org/10.1007/s11042-021-11313-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11313-0

Keywords

  • Convolutional neural network
  • Wearable device Sensor
  • Human activity recognition
  • Deep learning
  • Time-series data