Abstract
In recent years, with the rapid development of artificial intelligence, human activity recognition has become a research focus. The complex, dynamic and variable features of human activities lead to the relatively low accuracy of the traditional recognition algorithms. In order to solve the problem, this paper will propose a novel structure named hierarchical deep LSTM (H-LSTM) based on long short-term memory. Firstly, the original sensor data are preprocessed by smoothing and denoising; then, the feature will be selected and extracted by time–frequency-domain method. Secondly, H-LSTM is applied to the classification of these activities. Three public UCI datasets are used to conduct simulation with the realization of the automatic extraction of feature vectors and classification of outputting recognition results. Finally, the simulation results testify to the outperformance of the H-LSTM network over other deep learning algorithms. The accuracy of H-LSTM network in human activity recognition is proved to be 99.15%.
Similar content being viewed by others
References
S. Ali, N.A. Khan, M. Haneef, X.J.C. Luo, Blind source separation schemes for mono-sensor and multi-sensor systems with application to signal detection. Circuits Syst. Signal Process. 36(11), 4615–4636 (2017)
K. Altun, B. Barshan, O. Tunçel, Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recogn. 43(10), 3605–3620 (2010)
D. Anguita, A. Ghio, L. Oneto, X. Parra, J.L. Reyes-Ortiz, Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine, in International Workshop on Ambient Assisted Living (Springer, Berlin, 2012), pp. 216–223
J. Azorin-Lopez, M. Saval-Calvo, A. Fuster-Guillo, J. Garcia-Rodriguez, A novel prediction method for early recognition of global human behaviour in image sequences. Neural Process. Lett. 43(2), 363–387 (2015)
M. Babiker, O.O. Khalifa, K.K. Htike, A. Hassan, M. Zaharadeen, Automated daily human activity recognition for video surveillance using neural network, in Proceedings of 2017 4th IEEE International Conference on Smart Instrumentation, Measurement and Application (IEEE, 2017), pp. 1–5
O. Banos, J.-M. Galvez, M. Damas, A. Guillen, L.-J. Herrera, H. Pomares, I. Rojas, C. Villalonga, C.S. Hong, S. Lee, Multiwindow fusion for wearable activity recognition, in International Work-Conference on Artificial Neural Networks (Springer, Berlin, 2015), pp. 290–297
A. Bulling, U. Blanke, B. Schiele, A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput. Surv. 46(3), 33 (2014)
P. Casale, O. Pujol, P. Radeva, Human activity recognition from accelerometer data using a wearable device, in Proceedings of Iberian Conference on Pattern Recognition and Image Analysis (Springer, Berlin, 2011), pp. 289–296
L. Chen, J. Hoey, C.D. Nugent, D.J. Cook, Z. Yu, Sensor-based activity recognition. IEEE Trans. Syst. Man, Cybern. C, Appl. Rev. 42(6), 790–808 (2012)
L. Chen, H. Wei, J. Ferryman, ReadingAct RGB-D action dataset and human action recognition from local features. Pattern Recogn. Lett. 50, 159–169 (2014)
W.-H. Chen, C.A.B. Baca, C.-H. Tou, LSTM-RNNs combined with scene information for human activity recognition, in 2017 IEEE 19th International Conference on e-Health Networking, Applications and Services (2017), pp. 1–6
M. Ciman, M. Donini, O. Gaggi, F. Aiolli, Stairstep recognition and counting in a serious game for increasing users’ physical activity. Pers. Ubiquitous Comput. 20(6), 1015–1033 (2016)
K. Cui, X. Jing, Research on prediction model of geotechnical parameters based on BP neural network. Neural. Comput. Appl. (2018). https://doi.org/10.1007/s00521-018-3902-6
M. Edel, E. Köppe, Binarized-blstm-rnn based human activity recognition, in Proceedings of 2016 International Conference on Indoor Positioning and Indoor Navigation (IEEE, 2016), pp. 1–7
P. Esfahani, H.T. Malazi, PAMS: a new position-aware multi-sensor dataset for human activity recognition using smartphones, in Proceedings of 2017 19th IEEE International Symposium on Computer Architecture and Digital Systems (IEEE, 2017), pp. 1–7
L. Fan, Z. Wang, H. Wang, Human activity recognition model based on decision tree, in Proceedings of 2013 International Conference on Advanced Cloud and Big Data (IEEE, 2013), pp. 64–68
F. Gu, K. Khoshelham, S. Valaee, J. Shang, R. Zhang, Locomotion activity recognition using stacked denoising autoencoders. IEEE Internet Things. 5(3), 2085–2093 (2018)
P. Gupta, T. Dallas, Feature selection and activity recognition system using a single triaxial accelerometer. IEEE Trans. Biomed. Eng. 61(6), 1780–1786 (2014)
H.P. Gupta, H.S. Chudgar, S. Mukherjee, T. Dutta, K. Sharma, A continuous hand gestures recognition technique for human–machine interaction using accelerometer and gyroscope sensors. IEEE Sens. J. 16(16), 6425–6432 (2016)
S. Hochreiter, J. Schmidhuber, LSTM can solve hard long time lag problems, in Advances in Neural Information Processing Systems (1997), pp. 473–479
A. Jain, V. Kanhangad, Investigating gender recognition in smart-phones using accelerometer and gyroscope sensor readings, in Proceedings of 2016 International Conference on Computational Techniques in Information and Communication Technologies (2016), pp. 597–602
D. Li, H. Zhang, M.J.C. Zhang, Wavelet de-noising and genetic algorithm-based least squares twin SVM for classification of arrhythmias. Circuits Syst. Signal Process. 36(7), 2828–2846 (2017)
C.L. Liu, C.H. Lee, P.M. Lin, A fall detection system using k-nearest neighbor classifier. Expert Syst. Appl. 37(10), 7174–7181 (2010)
B. Long, M. Li, H. Wang, S.J.C. Tian, Diagnostics of analog circuits based on LS-SVM using time-domain features. Circuits Syst. Signal Process. 32(6), 2683–2706 (2013)
Y. Lu, Y. Wei, L. Liu, J. Zhong, L. Sun, Y. Liu, Towards unsupervised physical activity recognition using smartphone accelerometers. Multimed. Tools Appl. 76(8), 10701–10719 (2017)
D. Micucci, M. Mobilio, P. Napoletano, Unimib shar: a dataset for human activity recognition using acceleration data from smartphones. Appl. Sci. 7(10), 1101 (2017)
M. Milenkoski, K. Trivodaliev, S. Kalajdziski, M. Jovanov, B.R. Stojkoska, Real time human activity recognition on smartphones using LSTM Networks, in Proceedings of 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (IEEE, 2018), pp. 1126–1131
U.M. Nunes, D.R. Faria, P. Peixoto, A human activity recognition framework using max-min features and key poses with differential evolution random forests classifier. Pattern Recogn. Lett. 99, 21–31 (2017)
F. Ordóñez, D. Roggen, Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors. 16(1), 115 (2016)
J. Park, K. Jang, S.-B. Yang, Deep neural networks for activity recognition with multi-sensor data in a smart home, in 2018 4th IEEE World Forum on Internet of Things (IEEE, 2018), pp. 155–160
S.U. Park, J.H. Park, M.A. Al-masni, M.A. Al-antari, M.Z. Uddin, T.S. Kim, A depth camera-based human activity recognition via deep learning recurrent neural network for health and social care services. Proced. Comput. Sci. 100, 78–84 (2016)
S.Y. Park, H. Ju, C.G. Park, Stance phase detection of multiple actions for military drill using foot-mounted IMU. Sensors. 14, 16 (2016)
J.S. Peng, Y.M. Shao, Intelligent method for identifying driving risk based on V2V multisource big data. Complexity. 2018(1), 1–9 (2018)
S.J. Preece, J.Y. Goulermas, L.P. Kenney, D. Howard, A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data. IEEE Trans. Biomed. Eng. 56(3), 871–879 (2009)
J.L. Reyes-Ortiz, L. Oneto, A. Ghio, A. Samá, D. Anguita, X. Parra, Human activity recognition on smartphones with awareness of basic activities and postural transitions, in Proceedings of International Conference on Artificial Neural Networks (Springer, Berlin, 2014), pp. 177–184
G. Sridevi, S.J.C. Srinivas Kumar, Image inpainting based on fractional-order nonlinear diffusion for image reconstruction. Circuits Syst. Signal Process (2019). https://doi.org/10.1007/s00034-019-01029-w
A. Subasi, M. Radhwan, R. Kurdi, K. Khateeb, IoT based mobile healthcare system for human activity recognition, in Proceedings of 2018 15th IEEE Conference on Learning and Technology (IEEE, 2018), pp. 29–34
P. Susarla, U. Agrawal, D.B. Jayagopi, Human weapon-activity recognition in surveillance videos using structural-RNN, in Proceedings of 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence (ACM, New York, 2018), pp. 101–107
G. Vavoulas, M. Pediaditis, E.G. Spanakis, M. Tsiknakis, The MobiFall dataset: An initial evaluation of fall detection algorithms using smartphones, in Proceedings of 13th IEEE International Conference on BioInformatics and BioEngineering (2013), pp. 1–4
T.H. Vu, A. Dang, L. Dung, J.-C. Wang, Self-gated recurrent neural networks for human activity recognition on wearable devices, in Proceedings of the on Thematic Workshops of ACM Multimedia 2017 (ACM, New York, 2017), pp. 179–185
L. Wang, Recognition of human activities using continuous autoencoders with wearable sensors. Sensors. 16(2), 189 (2016)
L. Wang, Three-dimensional convolutional restricted Boltzmann machine for human activity recognition from RGB-D video. Eurasip J. Image Video Process. 2018(1), 120 (2018)
J.H. Wu, W. Wei, L. Zhang et al., Risk assessment of hypertension in steel workers based on LVQ and fisher-SVM deep excavation. IEEE Access. 7, 23109–23119 (2019)
F. Xiao, J. Chen, X.H. Xie, L. Gui, J.L. Sun, W. none Ruchuan, SEARE: a system for exercise activity recognition and quality evaluation based on green sensing. IEEE Trans. Emerg. Top. Comput. (2018). https://doi.org/10.1109/TETC.2018.2790080
A. Yassine, S. Singh, A.J.I.A. Alamri, Mining human activity patterns from smart home big data for health care applications. IEEE Access. 5, 13131–13141 (2017)
H. Zhang, A.C. Berg, M. Maire, J. Malik, SVM-KNN: Discriminative nearest neighbor classification for visual category recognition, in Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06) (IEEE, 2006), pp. 2126–2136
X. Zhang, Y. Li, R. Kotagiri, L. Wu, Z. Tari, M. Cheriet, KRNN: k rare-class nearest neighbour classification. Pattern Recogn. 62, 33–44 (2017)
X.Y. Zhang, F. Yin, Y.M. Zhang, C.L. Liu, Y. Bengio, Drawing and recognizing Chinese characters with recurrent neural network. IEEE Trans. Pattern. Anal. 40(4), 849–862 (2018)
Acknowledgements
This work is supported by the National Natural Science Foundation of Shandong Province (ZR2018BF005), the Scientific Research Foundation of Shandong University of Science and Technology for Recruited Talents (2017RCJJ077), the Shandong Province Higher Educational Science and Technology Program (J17KB167) and the Science and Technology Program of Taian (2017GX0014, 2018ZC0284).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations
Rights and permissions
About this article
Cite this article
Wang, L., Liu, R. Human Activity Recognition Based on Wearable Sensor Using Hierarchical Deep LSTM Networks. Circuits Syst Signal Process 39, 837–856 (2020). https://doi.org/10.1007/s00034-019-01116-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00034-019-01116-y