Human Activity Recognition from Body Sensor Data using Deep Learning

  • Mohammad Mehedi Hassan
  • Shamsul Huda
  • Md Zia Uddin
  • Ahmad Almogren
  • Majed Alrubaian
Mobile & Wireless Health
  • 12 Downloads
Part of the following topical collections:
  1. Convergence of Deep Machine Learning and Nature Inspired Computing Paradigms for Medical Informatics

Abstract

In recent years, human activity recognition from body sensor data or wearable sensor data has become a considerable research attention from academia and health industry. This research can be useful for various e-health applications such as monitoring elderly and physical impaired people at Smart home to improve their rehabilitation processes. However, it is not easy to accurately and automatically recognize physical human activity through wearable sensors due to the complexity and variety of body activities. In this paper, we address the human activity recognition problem as a classification problem using wearable body sensor data. In particular, we propose to utilize a Deep Belief Network (DBN) model for successful human activity recognition. First, we extract the important initial features from the raw body sensor data. Then, a kernel principal component analysis (KPCA) and linear discriminant analysis (LDA) are performed to further process the features and make them more robust to be useful for fast activity recognition. Finally, the DBN is trained by these features. Various experiments were performed on a real-world wearable sensor dataset to verify the effectiveness of the deep learning algorithm. The results show that the proposed DBN outperformed other algorithms and achieves satisfactory activity recognition performance.

Keywords

Human activity recognition Body sensor data Deep learning Deep belief network 

Notes

Acknowledgements

This paper was fully financially supported by King Saud University through the Vice Deanship of Research Chairs: Chair of Pervasive and Mobile Computing.

References

  1. 1.
    Chen, Y., and Shen, C., Performance Analysis of smartphone-sensor behavior for human activity recognition. IEEE Access 5:3095–3110, 2017.CrossRefGoogle Scholar
  2. 2.
    Cornacchia, M., Ozcan, K., Zheng, Y., and Velipasalar, S., A survey on activity detection and classification using wearable sensors. IEEE Sensors 17(2):386–403, 2017.CrossRefGoogle Scholar
  3. 3.
    Campbell, A., and Choudhury, T., From smart to cognitive phones. IEEE Pervasive Computing 11(3):7–11, 2012.CrossRefGoogle Scholar
  4. 4.
    Clarkson, B.P., Life patterns: structure from wearable sensors (Ph.D. thesis), Massachusetts Institute of Technology. 2002.Google Scholar
  5. 5.
    Avci, A., Bosch S., Marin-Perianu M., Marin-Perianu R., Havinga P., Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: a survey. In: International Conference on Architecture of Computing Systems, pp. 1–10. ARCS, Berlin, 2010.Google Scholar
  6. 6.
    Lin, W., Sun, M.-T., Poovandran, R, Zhang, Z., Human activity recognition for video surveillance. In: IEEE International Symposium on Circuits and Systems, pp. 2737–2740. IEEE, Seattle, 2008.Google Scholar
  7. 7.
    Lara, O., and Labrador, M., A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutor. 1:1–18, 2012.Google Scholar
  8. 8.
    Mannini, A., and Sabatini, A. M., Machine learning methods for classifying human physical activity from on-body accelerometers. Sensors 10:1154–1175, 2010.CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Poppe, R., A survey on vision-based human action recognition. Image Vis. Comput. 28:976–990, 2010.CrossRefGoogle Scholar
  10. 10.
    Nham, B., Siangliulue, K., Yeung, S., Predicting mode of transport from iphone accelerometer data. Technical Report, Stanford University, 2008.Google Scholar
  11. 11.
    Tapia, E., Intille, S., Larson, K., Activity recognition in the home using simple and ubiquitous sensors. In: International Conference on Pervasive Computing, pp. 158–175. Springer, Berlin, Heidelberg, 2004.Google Scholar
  12. 12.
    Bao, L., Intille, S., Activity recognition from user-annotated acceleration data. In: International Conference on Pervasive Computing, pp. 1–17. Springer, Berlin, Heidelberg, 2004.Google Scholar
  13. 13.
    Aggarwal, J., and Ryoo, M. S., Human activity analysis: a review. ACM Comput. Surv. 43(3):1–16, 2011.CrossRefGoogle Scholar
  14. 14.
    Tasoulis, S. K., Doukas, N., Plagianakos, V. P., and Maglogiannis, I., Statistical data mining of streaming motion data for activity and fall recognition in assistive environments. Neurocomputing 107:87–96, 2013.CrossRefGoogle Scholar
  15. 15.
    Behera, A., Hogg, D., Cohn, A., Egocentric activity monitoring and recovery. In: Asian Conference on Computer Vision, pp. 519–532. Springer, Berlin, Heidelberg, 2012.Google Scholar
  16. 16.
    D. Townsend, F. Knoefel, R. Goubran, Privacy versus autonomy: a tradeoff model for smart home monitoring technologies. In: 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4749–4752. EMBC, 2011.  https://doi.org/10.1109/IEMBS.2011.6091176.
  17. 17.
    Khan, A. M., Lee, Y. K., Lee, S. Y., and Kim, T. S., A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Trans. Inf. Technol. Biomed. 14(5):1166–1172, 2010.CrossRefPubMedGoogle Scholar
  18. 18.
    Maurer, U., Smailagic, A., Siewiorek, D., and Deisher, M., Activity recognition and monitoring using multiple sensors on different body positions. In: Proc. Int. Workshop Wearable Implantable Body Sens. Netw. pp. 113–116, 2006.Google Scholar
  19. 19.
    Kern, N., Schiele, B., Junker, H., Lukowicz, P., and Troster, G., Wearable sensing t oannotate meeting recordings. Pers. Ubiquit. Comput. 7:263–274, 2003.Google Scholar
  20. 20.
    Minnen, D., Starner, T., Ward, J., Lukowicz, P., and Troester, G., Recognizing and discovering human actions from on-body sensor data. In Proc. IEEE Int. Conf. Multimedia Expo. 1545–1548, 2005.Google Scholar
  21. 21.
    Giansanti, D., Macellari, V., and Maccioni, G., New neural network classifier of fall-risk based on the Mahalanobis distance and kinematic parameters assessed by a wearable device. Physiol. Meas. 29:11–19, 2008.CrossRefGoogle Scholar
  22. 22.
    Narayanan, M. R., Scalzi, M. E., Redmond, S. J., Lord, S. R., Celler, B. G., and Lovell, N. H., A wearable triaxial accelerometry system for longitudinal assessment of falls risk. In: Proc. 30th Annu. IEEE Int. Conf. Eng. Med. Biol. Soc. pp. 2840–2843, 2008.Google Scholar
  23. 23.
    Marschollek, M., Wolf, K., Gietzelt, M., Nemitz, G., Schwabedissen, H. M. Z., and Haux, R., Assessing elderly persons’ fall risk using spectral analysis on accelerometric data—A clinical evaluation study. In: Proc. 30th Annu. IEEE Int. Conf. Eng. Med. Biol. Soc. (2008) 3682–3685.Google Scholar
  24. 24.
    Yang, J. Y., Wang, J. S., and Chen, Y. P., Using acceleration measurements for activity recognition: An effective learning algorithm for constructing neural classifiers. Pattern Recogn. Lett. 29:2213–2220, 2008.CrossRefGoogle Scholar
  25. 25.
    Gao, L., Bourke, A. K., and Nelson, J., A system for activity recognition using multi-sensor fusion 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 7869–7872, 2011.Google Scholar
  26. 26.
    LeCun, Y., Bengio, Y., and Hinton, G., Deep learning. Nature 521(7553):436, 2015.CrossRefPubMedGoogle Scholar
  27. 27.
    Ordóñez, F. J., and Roggen, D., Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16(1):115, 2016.CrossRefPubMedCentralGoogle Scholar
  28. 28.
    Hammerla, N. Y., Halloran, S., and Ploetz, T., Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:1604.08880, 2016Google Scholar
  29. 29.
    Zebin, T., Scully, P. J., and Ozanyan, K. B., Human activity recognition with inertial sensors using a deep learning approach. In SENSORS, 2016 I.E. (pp. 1–3). IEEE, 2016Google Scholar
  30. 30.
    Cheng, L., Guan, Y., Zhu, K., and Li, Y., Recognition of human activities using machine learning methods with wearable sensors. In Computing and Communication Workshop and Conference (CCWC), 2017 I.E. 7th Annual (pp. 1–7). IEEE, 2017.Google Scholar
  31. 31.
    Ha, S., Yun, J. M., and Choi, S., Multi-modal Convolutional Neural Networks for Activity Recognition. In: 2015 I.E. International Conference on Systems, Man, and Cybernetics (SMC), 2015, pp. 3017–3022.Google Scholar
  32. 32.
    Hassan, M. M., Uddin, M. Z., Mohamed, A., and Almogren, A., A robust human activity recognition system using smartphone sensors and deep learning. Futur. Gener. Comput. Syst. 81:307–313, 2018.CrossRefGoogle Scholar
  33. 33.
    Ravi, D., Wong, C., Lo, B., and Yang, G. Z., A deep learning approach to on-node sensor data analytics for mobile or wearable devices. IEEE Journal of Biomedical and Health Informatics. 21(1):56–64, 2017.CrossRefPubMedGoogle Scholar
  34. 34.
    Hinton, G. E., Osindero, S., and Teh, Y.-W., A fast learning algorithm for deep belief nets. Neural Comput. 18(7):1527–1554, 2006.CrossRefPubMedGoogle Scholar
  35. 35.
    Uddin, M. Z., Hassan, M. M., Almogren, A., Zuair, M., Fortino, G., and Torresen, J., A facial expression recognition system using robust face features from depth videos and deep learning. Comput. Electr. Eng., 2017.  https://doi.org/10.1016/j.compeleceng.2017.04.019.
  36. 36.
    Bulling, A., Blanke, U., and Schiele, B., A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys (CSUR) 46(3):33, 2014.CrossRefGoogle Scholar
  37. 37.
    Ebied, H. M., Feature extraction using PCA and Kernel-PCA for face recognition. 8th International Conference on Informatics and Systems (INFOS), 72–77, 2017.Google Scholar
  38. 38.
    Lichman, M., UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science, 2013.

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Chia of Pervasive and Mobile Computing, College of Computer and Information SciencesKing Saud UniversityRiyadhSaudi Arabia
  2. 2.Information Systems DepartmentKing Saud UniversityRiyadhSaudi Arabia
  3. 3.School of ITDeakin UniversityMelbourneAustralia
  4. 4.Department of InformaticsUniversity of OsloOsloNorway

Personalised recommendations