Abstract
HAR has attained major attention because of its significant use in real-life scenarios like activity and fitness monitoring, rehabilitation, gaming, prosthetic limbs, healthcare, smart surveillance systems, etc. HAR systems provide ways for monitoring human behaviors and detecting body movements and various activities by using sensor data. The collection of sensors available in the mobile and other wearable devices has made most of these HAR applications easily possible. Moreover, Deep Learning (DL) has further accelerated the research on HAR using the data obtained via wearable devices. In this paper, we have discussed the overview of HAR, its applications, and popular benchmark datasets available publicly. Further, we discussed various DL techniques applied for HAR applications. We have also presented the challenges associated with the field and the future directions for performing more vital research in HAR.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yadav, S.K., Tiwari, K., Pandey, H.M., Akbar, S.A.: A review of multimodal human activity recognition with special emphasis on classification, applications, challenges and future directions. Knowl.-Based Syst. 223, 106970 (2021)
Demrozi, F., Pravadelli, G., Bihorac, A., Rashidi, P.: Human activity recognition using inertial, physiological and environmental sensors: a comprehensive survey. IEEE Access 8, 210816–210836 (2020)
Dawadi, P.N., Cook, D.J., Schmitter-Edgecombe, M.: Automated cognitive health assessment using smart home monitoring of complex tasks. IEEE Trans. Syst. Man Cybern. Syst. 43(6), 1302–1313 (2013)
Parsey, C.M., Schmitter-Edgecombe, M.: Applications of technology in neuropsychological assessment. Clin. Neuropsychol. 27(8), 1328–1361 (2013)
Yang, Q.: Activity recognition: linking low-level sensors to high-level intelligence. In: IJCAI, vol. 9, pp. 20–25, Pasadena, California (2009)
Chen, L., Hoey, J., Nugent, C.D., Cook, D.J., Yu, Z.: Sensor-based activity recognition. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 42(6), 790–808 (2012)
Bao, L., Intille, S.S.: Activity recognition from user-annotated acceleration data. In: Ferscha, A., Mattern, F. (eds.) Pervasive Computing. Pervasive 2004. LNCS, vol. 3001, pp. 1–17. Springer, Berlin, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24646-6_1
Figo, D., Diniz, P.C., Ferreira, D.R., Cardoso, J.M.: Preprocessing techniques for context recognition from accelerometer data. Pers. Ubiquit. Comput. 14(7), 645–662 (2010)
Aung, M.S., et al.: Automated detection of instantaneous gait events using time frequency analysis and manifold embedding. IEEE Trans. Neural Syst. Rehabil. Eng. 21(6), 908–916 (2013)
Chaudhuri, A., Sahu, T.P.: Binary Jaya algorithm based on binary similarity measure for feature selection. J. Ambient Intell. Humaniz. Comput. 1–18 (2021)
Chaudhuri, A., Sahu, T.P.: Feature weighting for naïve Bayes using multi objective artificial bee colony algorithm. Int. J. Comput. Sci. Eng. 24(1), 74–88 (2021)
Chaudhuri, A., Sahu, T.P.: Multi-objective feature selection based on quasi-oppositional based Jaya algorithm for microarray data. Knowl.-Based Syst. 236, 107804 (2022)
Anguita, D., Ghio, A., Oneto, L., Parra, X., Reyes-Ortiz, J.L.: A public domain dataset for human activity recognition using smartphones. In: Esann, vol. 3, p. 3 (2013)
Feng, Z., Mo, L., Li, M.: A random forest-based ensemble method for activity recognition. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5074–5077. IEEE (2015)
Nweke, H.F., Teh, Y.W., Al-Garadi, M.A., Alo, U.R.: Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: state of the art and research challenges. Expert Syst. Appl. 105, 233–261 (2018)
Palla, S.R., Sahu, G., Parida, P.: Human gait recognition using firefly template segmentation. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 10(5), 565–575 (2022). https://doi.org/10.1080/21681163.2021.2012829
Dua, N., Singh, S.N., Semwal, V.B.: Multi-input CNN-GRU based human activity recognition using wearable sensors. Computing 103(7), 1461–1478 (2021). https://doi.org/10.1007/s00607-021-00928-8
Jain, R., Semwal, V.B., Kaushik, P.: Deep ensemble learning approach for lower extremity activities recognition using wearable sensors. Expert. Syst. 39(6), e12743 (2022)
Dua, N., Singh, S.N., Semwal, V.B., Challa, S.K.: Inception inspired CNN-GRU hybrid network for human activity recognition. Multimedia Tools Appl. 1–35 (2022)
Raj, M., Semwal, V.B., Nandi, G.C.: Bidirectional association of joint angle trajectories for humanoid locomotion: the restricted Boltzmann machine approach. Neural Comput. Appl. 30(6), 1747–1755 (2018)
Saha, J., Chowdhury, C., Ghosh, D., Bandyopadhyay, S.: A detailed human activity transition recognition framework for grossly labeled data from smartphone accelerometer. Multimedia Tools Appl. 80(7), 9895–9916 (2020). https://doi.org/10.1007/s11042-020-10046-w
Bijalwan, V., Semwal, V.B., Gupta, V.: Wearable sensor-based pattern mining for human activity recognition: deep learning approach. Ind. Robot. 49(1), 21–33 (2022). https://doi.org/10.1108/IR-09-2020-0187
Asteriadis, S., Daras, P.: Landmark-based multimodal human action recognition. Multimedia Tools Appl. 76(3), 4505–4521 (2016). https://doi.org/10.1007/s11042-016-3945-6
Singh, R., Kushwaha, A.K.S., Srivastava, R.: Multi-view recognition system for human activity based on multiple features for video surveillance system. Multimedia Tools Appl. 78(12), 17165–17196 (2019). https://doi.org/10.1007/s11042-018-7108-9
Webber, M., Rojas, R.F.: Human activity recognition with accelerometer and gyroscope: a data fusion approach. IEEE Sens. J. 21(15), 16979–16989 (2021)
Masum, A.K.M., Bahadur, E.H., Shan-A-Alahi, A., Chowdhury, M.A.U.Z., Uddin, M.R., Al Noman, A.: Human activity recognition using accelerometer, gyroscope and magnetometer sensors: deep neural network approaches. In: 2019 10Th International Conference on Computing, Communication and Networking Technologies (ICCCNT), pp. 1–6. IEEE, Kanpur (2019)
Ashry, S., Gomaa, W., Abdu-Aguye, M.G., El-borae, N.: Improved IMU-based human activity recognition using hierarchical hmm dissimilarity. In: Proceedings of the 17th International Conference on Informatics in Control, Automation and Robotics, vol. 1, pp. 702–709 (2020)
Nurhanim, K., Elamvazuthi, I., Izhar, L.I., Capi, G., Su, S.: EMG signals classification on human activity recognition using machine learning algorithm. In: 2021 8th NAFOSTED Conference on Information and Computer Science (NICS), pp. 369–373. IEEE, Hanoi, Vietnam (2021)
Ziaur Rehman, M., et al.: Multiday EMG-based classification of hand motions with deep learning techniques. Sensors 18(8), 2497 (2018)
Ding, Z., Yang, C., Tian, Z., Yi, C., Fu, Y., Jiang, F.: sEMG-based gesture recognition with convolution neural networks. Sustainability 10(6), 1865 (2018)
Jia, R., Liu, B.: Human daily activity recognition by fusing accelerometer and multi-lead ECG data. In: 2013 IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC 2013), pp. 1–4. IEEE, KunMing, China (2013)
Almanifi, O.R.A., Khairuddin, I.M., Razman, M.A.M., Musa, R.M., Majeed, A.P.A.: Human activity recognition based on wrist PPG via the ensemble method. ICT Express (2022)
Zia, S., Khan, A.N., Mukhtar, M., Ali, S.E.: Human activity recognition using portable EEG sensor and support vector machine. In: 2021 International Conference on Engineering and Emerging Technologies (ICEET), pp. 1–6. IEEE, Istanbul, Turkey (2021)
Salehzadeh, A., Calitz, A.P., Greyling, J.: Human activity recognition using deep electroencephalography learning. Biomed. Signal Process. Control 62, 102094 (2020)
Dinarević, E.C., Husić, J.B., Baraković, S.: Issues of human activity recognition in healthcare. In: 2019 18th International Symposium INFOTEH-JAHORINA (INFOTEH), pp. 1–6. IEEE, East Sarajevo (2019)
Hu, F.B., Leitzmann, M.F., Stampfer, M.J., Colditz, G.A., Willett, W.C., Rimm, E.B.: Physical activity and television watching in relation to risk for type 2 diabetes mellitus in men. Arch. Intern. Med. 161(12), 542–1548 (2001)
Schnohr, P., Lange, P., Scharling, H., Jensen, J.S.: Long-term physical activity in leisure time and mortality from coronary heart disease, stroke, respiratory diseases, and cancer. The Copenhagen City heart study. Eur. J. Prev. Cardiol. 13(2), 173–179 (2006)
Bauman, A.E., Reis, R.S., Sallis, J.F., Wells, J.C., Loos, R.J., Martin, B.W.: Correlates of physical activity: why are some people physically active and others not? The Lancet 380(9838), 258–271 (2012). Lancet Physical Activity Series Working Group
Sullivan, A.N., Lachman, M.E.: Behavior change with fitness technology in sedentary adults: a review of the evidence for increasing physical activity. Front. Public Health 4, 289 (2017)
Eskofier, B.M., et al.: Recent machine learning advancements in sensor-based mobility analysis: deep learning for Parkinson’s disease assessment. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 655–658. IEEE, Orlando, USA (2016)
Zhang, S., et al.: CoughTrigger: earbuds IMU based cough detection activator using an energy-efficient sensitivity-prioritized time series classifier. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. IEEE, Singapore (2022)
Meattini, R., Benatti, S., Scarcia, U., De Gregorio, D., Benini, L., Melchiorri, C.: An sEMG-based human–robot interface for robotic hands using machine learning and synergies. IEEE Trans. Compon. Packag. Manuf. Technol. 8(7), 1149–1158 (2018)
Parajuli, N., et al.: Real-time EMG based pattern recognition control for hand prostheses: a review on existing methods, challenges and future implementation. Sensors 19(20), 4596 (2019)
Zhao, H., Ma, Y., Wang, S., Watson, A., Zhou, G.: MobiGesture: mobility-aware hand gesture recognition for healthcare. Smart Health 9, 129–143 (2018)
Reiss, A., Stricker, D.: Introducing a new benchmarked dataset for activity monitoring. In: 2012 16th International Symposium on Wearable Computers, pp. 108–109. IEEE, Newcastle, UK (2012)
Kwapisz, J.R., Weiss, G.M., Moore, S.A.: Activity recognition using cell phone accelerometers. ACM SIGKDD Explor. Newsl. 12(2), 74–82 (2011)
Banos, O., et al.: mHealthDroid: a novel framework for agile development of mobile health applications. In: Pecchia, L., Chen, L.L., Nugent, C., Bravo, J. (eds.) Ambient Assisted Living and Daily Activities. IWAAL 2014. LNCS, vol. 8868, pp. 91–98. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13105-4_14
Chavarriaga, R., et al.: The opportunity challenge: a benchmark database for on-body sensor-based activity recognition. Pattern Recogn. Lett. 34(15), 2033–2042 (2013)
Bachlin, M., et al.: Wearable assistant for Parkinson’s disease patients with the freezing of gait symptom. IEEE Trans. Inf. Technol. Biomed. 14(2), 436–446 (2009)
Ballard, D.H.: Modular learning in neural networks. In: AAAI, vol. 647, pp. 279–284, Washington, DC, USA (1987)
Li, Y., Shi, D., Ding, B., Liu, D.: Unsupervised Feature Learning for Human Activity Recognition Using Smartphone Sensors. In: Prasath, R., O’Reilly, P., Kathirvalavakumar, T. (eds.) Mining Intelligence and Knowledge Exploration. LNCS, vol. 8891, pp. 99–107. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13817-6_11
Mohammadian Rad, N., Van Laarhoven, T., Furlanello, C., Marchiori, E.: Novelty detection using deep normative modeling for imu-based abnormal movement monitoring in parkinson’s disease and autism spectrum disorders. Sensors 18(10), 3533 (2018)
Malekzadeh, M., Clegg, R.G., Haddadi, H.: Replacement autoencoder: a privacy-preserving algorithm for sensory data analysis. In: 2018 IEEE/ACM Third International Conference on Internet-of-Things Design and Implementation (IoTDI), pp. 165–176. IEEE, Orlando, FL, USA (2018)
Jun, K., Choi, S.: Unsupervised end-to-end deep model for newborn and infant activity recognition. Sensors 20(22), 6467 (2020)
Akbari, A., Jafari, R.: An autoencoder-based approach for recognizing null class in activities of daily living in-the-wild via wearable motion sensors. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3392–3396. IEEE, Brighton, UK (2019)
Khan, M.A.A.H., Roy, N.: Untran: recognizing unseen activities with unlabeled data using transfer learning. In: 2018 IEEE/ACM Third International Conference on Internet-of-Things Design and Implementation (IoTDI), pp. 37–47. IEEE, Orlando, FL, USA (2018)
Akbari, A., Jafari, R.: Transferring activity recognition models for new wearable sensors with deep generative domain adaptation. In: Proceedings of the 18th International Conference on Information Processing in Sensor Networks, pp. 85–96. ACM (2019)
Gao, X., Luo, H., Wang, Q., Zhao, F., Ye, L., Zhang, Y.: A human activity recognition algorithm based on stacking denoising autoencoder and lightGBM. Sensors 19(4), 947 (2019)
Hung, B.T., Semwal, V.B., Gaud, N., Bijalwan, V.: Hybrid deep learning approach for aspect detection on reviews. In: Singh Mer, K.K., Semwal, V.B., Bijalwan, V., Crespo, R.G. (eds.) Proceedings of Integrated Intelligence Enable Networks and Computing. Algorithms for Intelligent Systems. Springer, Singapore (2021). https://doi.org/10.1007/978-981-33-6307-6_100
Ha, S., Choi, S.: Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors. In: 2016 International Joint Conference on Neural Networks (IJCNN), pp. 381–388. IEEE, Vancouver (2016)
Cheng, X., Zhang, L., Tang, Y., Liu, Y., Wu, H., He, J.: Real-time human activity recognition using conditionally parametrized convolutions on mobile and wearable devices. IEEE Sens. J. 22(6), 5889–5901 (2022)
Ignatov, A.: Real-time human activity recognition from accelerometer data using convolutional neural networks. Appl. Soft Comput. 62, 915–922 (2018)
Wan, S., Qi, L., Xu, X., Tong, C., Gu, Z.: Deep learning models for real-time human activity recognition with smartphones. Mob. Netw. Appl. 25(2), 743–755 (2020)
Cho, H., Yoon, S.M.: Divide and conquer-based 1D CNN human activity recognition using test data sharpening. Sensors 18(4), 1055 (2018)
Ordóñez, F.J., Roggen, D.: Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1), 115 (2016)
Zhao, Y., Yang, R., Chevalier, G., Xu, X., Zhang, Z.: Deep residual bidir-LSTM for human activity recognition using wearable sensors. Math. Probl. Eng. (2018)
Yu, S., Qin, L.: Human activity recognition with smartphone inertial sensors using bidir-lstm networks. In: 2018 3rd International Conference on Mechanical, Control and Computer Engineering (icmcce), pp. 219–224. IEEE, Huhhot, China (2018)
Ullah, M., Ullah, H., Khan, S.D., Cheikh, F.A.: Stacked LSTM network for human activity recognition using smartphone data. In: 2019 8th European Workshop on Visual Information Processing (EUVIP), pp. 175–180. IEEE, Italy (2019)
Zeng, M., et al.: Understanding and improving recurrent networks for human activity recognition by continuous attention. In Proceedings of the 2018 ACM international symposium on wearable computers, pp. 56–63. ACM, Newyork (2018)
Hammerla, N.Y., Halloran, S., Plötz, T.: Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:1604.08880 (2016)
Lyu, L., He, X., Law, Y.W., Palaniswami, M.: Privacy-preserving collaborative deep learning with application to human activity recognition. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 1219–1228. ACM, Singapore (2017)
Challa, S.K., Kumar, A., Semwal, V.B.: A multibranch CNN-BiLSTM model for human activity recognition using wearable sensor data. Vis. Comput. 1–15 (2021).https://doi.org/10.1007/s00371-021-02283-3
Semwal, V.B., Gupta, A., Lalwani, P.: An optimized hybrid deep learning model using ensemble learning approach for human walking activities recognition. J. Supercomput. 77(11), 12256–12279 (2021). https://doi.org/10.1007/s11227-021-03768-7
Alsheikh, M.A., Selim, A., Niyato, D., Doyle, L., Lin, S., Tan, H.P.: Deep activity recognition models with triaxial accelerometers. In: Workshops at the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona USA (2016)
Zhang, L., Wu, X., Luo, D.: Recognizing human activities from raw accelerometer data using deep neural networks. In: 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pp. 865–870. IEEE, Miami, FL, USA (2015)
Radu, V., Lane, N.D., Bhattacharya, S., Mascolo, C., Marina, M.K., Kawsar, F.: Towards multimodal deep learning for activity recognition on mobile devices. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, pp. 185–188. ACM, Germany, Heidelberg (2016)
Gao, Y., et al.: iHear food: eating detection using commodity bluetooth headsets. In: 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pp. 163–172. IEEE, Washington, DC, USA (2016)
Ramponi, G., Protopapas, P., Brambilla, M., Janssen, R.: T-CGAN: conditional generative adversarial network for data augmentation in noisy time series with irregular sampling. arXiv preprint arXiv:1811.08295 (2018)
Alzantot, M., Chakraborty, S., Srivastava, M.: Sensegen: a deep learning architecture for synthetic sensor data generation. In: 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pp. 188–193. IEEE, Kona, HI, USA (2017)
Wang, J., Chen, Y., Gu, Y., Xiao, Y., Pan, H.: SensoryGANs: an effective generative adversarial framework for sensor-based human activity recognition. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE, Rio de Janeiro, Brazil (2018)
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., Lee, H.: Generative adversarial text to image synthesis. In: International Conference on Machine Learning, pp. 1060–1069. PMLR (2016)
Zhou, Y., Wang, Z., Fang, C., Bui, T., Berg, T.L.: Visual to sound: generating natural sound for videos in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3550–3558. IEEE, Salt Lake City, UT, USA (2018)
Plötz, T., Guan, Y.: Deep learning for human activity recognition in mobile computing. Computer 51(5), 50–59 (2018)
Abdel-Basset, M., Hawash, H., Chang, V., Chakrabortty, R.K., Ryan, M.: Deep learning for heterogeneous human activity recognition in complex iot applications. IEEE Internet Things J. 9(8), 5653–5665 (2020)
Qin, Z., Zhang, Y., Meng, S., Qin, Z., Choo, K.K.R.: Imaging and fusing time series for wearable sensor-based human activity recognition. Inf. Fusion 53, 80–87 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dua, N., Singh, S.N., Challa, S.K., Semwal, V.B., Sai Kumar, M.L.S. (2022). A Survey on Human Activity Recognition Using Deep Learning Techniques and Wearable Sensor Data. In: Khare, N., Tomar, D.S., Ahirwal, M.K., Semwal, V.B., Soni, V. (eds) Machine Learning, Image Processing, Network Security and Data Sciences. MIND 2022. Communications in Computer and Information Science, vol 1762. Springer, Cham. https://doi.org/10.1007/978-3-031-24352-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-24352-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-24351-6
Online ISBN: 978-3-031-24352-3
eBook Packages: Computer ScienceComputer Science (R0)