Abstract
Video based fall detection accuracy has been largely improved due to the recent progress on deep convolutional neural networks. However, there still exist some challenges, such as lighting variation, complex background, which degrade the accuracy and generalization ability of these approaches. Meanwhile, large computation cost limits the application of existing fall detection approaches. To alleviate these problems, a video based fall detection approach using human poses is proposed in this paper. First, a lightweight pose estimator extracts 2D poses from video sequences, and then 2D poses are lifted to 3D poses. Second, we introduce a robust fall detection network to recognize fall events using estimated 3D poses, which increases respective field and maintains low computation cost by dilated convolutions. The experimental results show that the proposed fall detection approach achieves a high accuracy of 99.83% on large benchmark action recognition dataset NTU RGB+D and real-time performance of 18 FPS on a non-GPU platform, 63 FPS on a GPU platform.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adhikari, K., Bouchachia, H., Nait-Charif, H.: Activity recognition for indoor fall detection using convolutional neural network. In: MVA, pp. 81–84 (2017). https://doi.org/10.23919/MVA.2017.7986795
Andriluka, M., Pishchulin, L., Gehler, P., Schiele, B.: 2D human pose estimation: new benchmark and state of the art analysis. In: CVPR, June 2014
Cameiro, S.A., da Silva, G.P., Leite, G.V., Moreno, R., Guimarães, S.J.F., Pedrini, H.: Multi-stream deep convolutional network using high-level features applied to fall detection in video sequences. In: IWSSIP, pp. 293–298 (2019). https://doi.org/10.1109/IWSSIP.2019.8787213
Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: Openpose: realtime multi-person 2d pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 172–186 (2021). https://doi.org/10.1109/TPAMI.2019.2929257
Chen, C.H., Ramanan, D.: 3D human pose estimation = 2d pose estimation + matching. In: CVPR, July 2017
Cheng, K., Zhang, Y., Cao, C., Shi, L., Cheng, J., Lu, H.: Decoupling gcn with dropgraph module for skeleton-based action recognition. In: European Conference on Computer Vision, pp. 536–553 (2020)
Cheng, Y., Yang, B., Wang, B., Tan, R.T.: 3D human pose estimation using spatio-temporal networks with explicit occlusion training. In: AAAI, vol. 34, pp. 10631–10638 (2020)
United Nations Department of Economic and Social Affairs: World Population Ageing 2020: Highlights. United Nations (2021)
Gutiérrez, J., RodrÃguez, V., Martin, S.: Comprehensive review of vision-based fall detection systems. Sensors 21(3), 947 (2021). https://doi.org/10.3390/s21030947
He, Y., Yan, R., Fragkiadaki, K., Yu, S.I.: Epipolar transformers. In: CVPR, June 2020
Holschneider, M., Kronland-Martinet, R., Morlet, J., Tchamitchian, P.: A real-time algorithm for signal analysis with the help of the wavelet transform. In: Combes, J.M., Grossmann, A., Tchamitchian, P. (eds.) Wavelets Inverse Problems and Theoretical Imaging, pp. 286–297. Springer, Heidelberg (1990). https://doi.org/10.1007/978-3-642-75988-8_28
Hwang, S., Ahn, D., Park, H., Park, T.: Poster abstract: maximizing accuracy of fall detection and alert systems based on 3d convolutional neural network. In: International Conference on Internet-of-Things Design and Implementation (IoTDI), pp. 343–344 (2017)
Kasturi, S., Filonenko, A., Jo, K.H.: Human fall recognition using the spatiotemporal 3d cnn. In: Proceedings IW-FCV, pp. 1–3 (2019)
Kocabas, M., Athanasiou, N., Black, M.J.: Vibe: Video inference for human body pose and shape estimation. In: CVPR, June 2020
Li, S., Xiong, H., Diao, X.: Pre-impact fall detection using 3d convolutional neural network. In: International Conference on Rehabilitation Robotics (ICORR), pp. 1173–1178 (2019). https://doi.org/10.1109/ICORR.2019.8779504
Li, S., Chan, A.B.: 3D human pose estimation from monocular images with deep convolutional neural network. In: Cremers, D., Reid, I., Saito, H., Yang, M.H. (eds.) ACCV 2014. LNCS, vol. 9004, pp. 332–347. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16808-1_23
Li, X., Pang, T., Liu, W., Wang, T.: Fall detection for elderly person care using convolutional neural networks. In: CISP-BMEI, pp. 1–6 (2017). https://doi.org/10.1109/CISP-BMEI.2017.8302004
Lin, T.Y., et al.: Microsoft coco: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Lu, N., Wu, Y., Feng, L., Song, J.: Deep learning for fall detection: three-dimensional CNN combined with LSTM on video kinematic data. IEEE J. Biomed. Health Inf. 23(1), 314–323 (2019). https://doi.org/10.1109/JBHI.2018.2808281
Ma, C., Shimada, A., Uchiyama, H., Nagahara, H., Taniguchi, R.J.: Fall detection using optical level anonymous image sensing system. Opt. Laser Technol. 110, 44–61 (2019)
Martinez, J., Hossain, R., Romero, J., Little, J.J.: A simple yet effective baseline for 3d human pose estimation. In: ICCV, October 2017
Mehta, D., et al.: Monocular 3d human pose estimation in the wild using improved CNN supervision. In: International Conference on 3D Vision (3DV), pp. 506–516 (2017). https://doi.org/10.1109/3DV.2017.00064
Menacho, C., Ordoñez, J.: Fall detection based on cnn models implemented on a mobile robot. In: International Conference on Ubiquitous Robots (UR), pp. 284–289 (2020). https://doi.org/10.1109/UR49135.2020.9144836
Min, W., Yao, L., Lin, Z., Liu, L.: Support vector machine approach to fall recognition based on simplified expression of human skeleton action and fast detection of start key frame using torso angle. IET Comput. Vis. 12(8), 1133–1140 (2018)
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29
Rahnemoonfar, M., Alkittawi, H.: Spatio-temporal convolutional neural network for elderly fall detection in depth video cameras. In: Big Data, pp. 2868–2873 (2018). https://doi.org/10.1109/BigData.2018.8622342
Senouci, B., Charfi, I., Heyrman, B., Dubois, J., Miteran, J.: Fast prototyping of a SOC-based smart-camera: a real-time fall detection case study. J. Real-Time Image Process. 12(4), 649–662 (2016)
Shahroudy, A., Liu, J., Ng, T.T., Wang, G.: NTU RGB+D: a large scale dataset for 3d human activity analysis. In: CVPR, pp. 1010–1019 (2016)
Shojaei-Hashemi, A., Nasiopoulos, P., Little, J.J., Pourazad, M.T.: Video-based human fall detection in smart homes using deep learning. In: ISCAS, pp. 1–5 (2018). https://doi.org/10.1109/ISCAS.2018.8351648
Tompson, J.J., Jain, A., LeCun, Y., Bregler, C.: Joint training of a convolutional network and a graphical model for human pose estimation. Adv. Neural Inf. Process. Syst. 27, 1799–1807 (2014)
Toshev, A., Szegedy, C.: Deeppose: human pose estimation via deep neural networks. In: CVPR, June 2014
Tsai, T.H., Hsu, C.W.: Implementation of fall detection system based on 3d skeleton for deep learning technique. IEEE Access 7, 153049–153059 (2019). https://doi.org/10.1109/ACCESS.2019.2947518
Wandt, B., Rosenhahn, B.: Repnet: weakly supervised training of an adversarial reprojection network for 3d human pose estimation. In: CVPR, June 2019
Wandt, B., Rudolph, M., Zell, P., Rhodin, H., Rosenhahn, B.: Canonpose: self-supervised monocular 3d human pose estimation in the wild. In: CVPR, pp. 13294–13304, June 2021
Wang, J., et al.: Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 43, 3349–3364 (2020). https://doi.org/10.1109/TPAMI.2020.2983686
Wang, X., Ellul, J., Azzopardi, G.: Elderly fall detection systems: a literature survey. Front. Rob. AI 7, 71 (2020). https://doi.org/10.3389/frobt.2020.00071
WHO: Fall. https://www.who.int/news-room/fact-sheets/detail/fall. Accessed 26 Apr 2021
Xiao, B., Wu, H., Wei, Y.: Simple baselines for human pose estimation and tracking. In: Ferrari, Vi., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 472–487. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01231-1_29
Xu, Q., Huang, G., Yu, M., Guo, Y.: Fall prediction based on key points of human bones. Phys. A Stat. Mech. Appl. 540, 123205 (2020). https://doi.org/10.1016/j.physa.2019.123205
Xu, T., Zhou, Y., Zhu, J.: New advances and challenges of fall detection systems: a survey. Appl. Sci. 8(3), 418 (2018). https://doi.org/10.3390/app8030418
Yang, S., Quan, Z., Nie, M., Yang, W.: Transpose: keypoint localization via transformer. In: ICCV (2021)
Zhang, Z., Tang, J., Wu, G.: Simple and lightweight human pose estimation. arXiv preprint arXiv:1911.10346 (2019)
Zhong, C., Ng, W.W.Y., Zhang, S., Nugent, C.D., Shewell, C., Medina-Quero, J.: Multi-occupancy fall detection using non-invasive thermal vision sensor. IEEE Sens. J. 21(4), 5377–5388 (2021). https://doi.org/10.1109/JSEN.2020.3032728
Zhou, J., Komuro, T.: Recognizing fall actions from videos using reconstruction error of variational autoencoder. In: ICIP, pp. 3372–3376 (2019). https://doi.org/10.1109/ICIP.2019.8803671
Acknowledgments
This work was supported by the National Natural Science Foundation of China under Nos. 61773117 and 62006041.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chen, Z., Wang, Y., Yang, W. (2022). Video Based Fall Detection Using Human Poses. In: Liao, X., et al. Big Data. BigData 2022. Communications in Computer and Information Science, vol 1496. Springer, Singapore. https://doi.org/10.1007/978-981-16-9709-8_19
Download citation
DOI: https://doi.org/10.1007/978-981-16-9709-8_19
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-9708-1
Online ISBN: 978-981-16-9709-8
eBook Packages: Computer ScienceComputer Science (R0)