Abstract
This paper presents a vision-based system for recognizing when elderly adults fall. A fall is characterized by shape deformation and high motion. We represent shape variation using three features, the aspect ratio of the bounding box, the orientation of an ellipse representing the body, and the aspect ratio of the projection histogram. For motion variation, we extract several features from three blocks corresponding to the head, center of the body, and feet using optical flow. For each block, we compute the speed and the direction of motion. Each activity is represented by a feature vector constructed from variations in shape and motion features for a set of frames. A support vector machine is used to classify fall and non-fall activities. Experiments on three different datasets show the effectiveness of our proposed method.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Bergen, G.; Stevens, M. R.; Burns, E. R. Falls and fall injuries among adults aged ⩾ 65 years-United States, 2014. Morbidity and Mortality Weekly Report Vol. 65, No. 37, 993–998, 2016.
Khan, S. S.; Hoey, J. Review of fall detection techniques: A data availability perspective. Medical Engineering & Physics Vol. 39, 12–22, 2017.
World Health Organization. WHO Global Report on Falls Prevention in Older Age. World Health Organization, 2008.
Igual, R.; Medrano, C.; Plaza, I. Challenges, issues, and trends in fall detection systems. BioMedical Engineering OnLine Vol. 12, No. 1, 1–24, 2013.
Zhang, Z.; Conly, C.; Athitsos, V. A survey on visionbased fall detection. In: Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments, Article No. 46, 2015.
Doukas, C. N.; Maglogiannis, I. Emergency fall incidents detection in assisted living environments utilizing motion, sound, and visual perceptual components. IEEE Transactions on Information Technology in Biomedicine Vol. 15, No. 2, 277–289, 2011.
Sree, J.; Umamakeswari, A.; Jenita, B. A survey on technical approaches in fall detection system. National Journal of Physiology, Pharmacy and Pharmacology Vol. 5, No. 4, 275–279, 2015.
Reyes-Ortiz, J. L.; Oneto, L.; Samà, A.; Parra, X.; Anguita, D. Transition-aware human activity recognition using smartphones. Neurocomputing Vol. 171, 754–767, 2016.
Hammerla, N.; Halloran, S.; Ploetz, T. Deep, convolutional, and recurrent models for human activity recognition using wearables. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence, 1533–1540, 2016.
Chelli, A.; Patzold, M. A machine learning approach for fall detection and daily living activity recognition. IEEE Access Vol. 7, 38670–38687, 2019.
Aslan, M.; Sengur, A.; Xiao, Y.; Wang, H. B.; Cevdet Ince, M.; Ma, X. Shape feature encoding via Fisher Vector for efficient fall detection in depth-videos. Applied Soft Computing Vol. 37, 1023–1028, 2015.
Chua, J. L.; Chang, Y. C.; Lim, W. K. A simple vision-based fall detection technique for indoor video surveillance. Signal, Image and Video Processing Vol. 9, No. 3, 623–633, 2015.
Iazzi, A.; Rziza, M.; Oulad Haj Thami, R.; Aboutajdine, D. A new method for fall detection of elderly based on human shape and motion variation. In: Advances in Visual Computing. Lecture Notes in Computer Science, Vol. 10073. Bebis, G. et al. Eds. Springer Cham, 156–167, 2016.
Kumar, D. P.; Yun, Y.; Gu, I. Y. Fall detection in RGBD videos by combining shape and motion features. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 1337–1341 2016.
Rougier, C.; Meunier, J.; St-Arnaud, A.; Rousseau, J. Robust video surveillance for fall detection based on human shape deformation. IEEE Transactions on Circuits and Systems for Video Technology Vol. 21, No. 5, 611–622, 2011.
Harrou, F.; Zerrouki, N.; Sun, Y.; Houacine, A. An integrated vision-based approach for efficient human fall detection in a home environment. IEEE Access Vol. 7, 114966–114974, 2019.
Kepski, M.; Kwolek, B. Fall detection using ceilingmounted 3D depth camera. In: Proceedings of the International Conference on Computer Vision Theory and Applications, Vol. 2, 640–647, 2014.
Nguyen, V. A.; Le, T. H.; Nguyen, T. T. Single camera based fall detection using motion and human shape features. In: Proceedings of the 7th Symposium on Information and Communication Technology, 339–344, 2016.
Fan, K. B.; Wang, P.; Zhuang, S. Human fall detection using slow feature analysis. Multimedia Tools and Applications Vol. 78, No. 7, 9101–9128, 2019.
Ismail, M. M. B.; Bchir, O. Automatic fall detection using membership based histogram descriptors. Journal of Computer Science and Technology Vol. 32, No. 2, 356–367, 2017.
Fan, K. B.; Wang, P.; Hu, Y.; Dou, B. J. Fall detection via human posture representation and support vector machine. International Journal of Distributed Sensor Networks Vol. 13, No. 5, 155014771770741, 2017.
Núñez-Marcos, A.; Azkune, G.; Arganda-Carreras, I. Vision-based fall detection with convolutional neural networks. Wireless Communications and Mobile Computing Vol. 2017, Article ID 9474806, 2017.
Akagunduz, E.; Aslan, M.; Sengu, A.; Wang, H. B.; Ince, M. C. Silhouette orientation volumes for efficient fall detection in depth videos. IEEE Journal of Biomedical and Health Informatics Vol. 21, No. 3, 756–763, 2017.
Abdulmunem, A.; Lai, Y. K.; Sun, X. F. Saliency guided local and global descriptors for effective action recognition. Computational Visual Media Vol. 2, No. 1, 97–106, 2016.
Vishwakarma, V.; Mandal, C.; Sural, S. Automatic detection of human fall in video. In: Pattern Recognition and Machine Intelligence Lecture Notes in Computer Science, Vol. 4815. Ghosh, A.; De, R. K.; Pal, S. K. Eds. Springer Berlin Heidelberg, 616–623, 2007.
Akagündüz, E. Shape recognition using orientational and morphological scale-spaces of curvatures. IET Computer Vision Vol. 9, No. 5, 750–757, 2015.
Yun, Y. X.; Gu, I. Y. Human fall detection in videos via boosting and fusing statistical features of appearance, shape and motion dynamics on Riemannian manifolds with applications to assisted living. Computer Vision and Image Understanding Vol. 148, 111–122, 2016.
Wiskott, L.; Sejnowski, T. J. Slow feature analysis: Unsupervised learning of invariances. Neural Computation Vol. 14, No. 4, 715–770, 2002.
Krizhevsky, A.; Sutskever, I.; Hinton, G. E. ImageNet classification with deep convolutional neural networks. Communications of the ACM Vol. 6, No. 6, 84–90, 2012.
Stauffer, C.; Grimson, W. E. L. Adaptive background mixture models for real-time tracking. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 246–252, 1999.
Kim, K.; Chalidabhongse, T. H.; Harwood, D.; Davis, L. Background modeling and subtraction by codebook construction. In: Proceedings of the International Conference on Image Processing, Vol. 5, 3061–3064, 2004.
McFarlane, N. J. B.; Schofield, C. P. Segmentation and tracking of piglets in images. Machine Vision and Applications Vol. 8, No. 3, 187–193, 1995.
Yu, M.; Yu, Y. Z.; Rhuma, A.; Naqvi, S. M. R.; Wang, L.; Chambers, J. A. An online one class support vector machine-based person-specific fall detection system for monitoring an elderly individual in a room environment. IEEE Journal of Biomedical and Health Informatics Vol. 17, No. 6, 1002–1014, 2013.
Minaee, S.; Wang, Y. An ADMM approach to masked signal decomposition using subspace representation. IEEE Transactions on Image Processing Vol. 28, No. 7, 3192–3204, 2019.
Bakkay, M. C.; Rashwan, H. A.; Salmane, H.; Khoudour, L.; Puig, D.; Ruichek, Y. BSCGAN: Deep background subtraction with conditional generative adversarial networks. In: Proceedings of the 25th IEEE International Conference on Image Processing, 4018–4022, 2018.
Gomes, V.; Barcellos, P.; Scharcanski, J. Stochastic shadow detection using a hypergraph partitioning approach. Pattern Recognition Vol. 63, 30–44, 2017.
Gonzalez, R. C.; Woods, R. E.; Masters, B. R. Digital image processing third edition. Journal of Biomedical Optics Vol. 14, No. 2, 029901, 2008.
Yang, L.; Ren, Y. Y.; Zhang, W. Q. 3D depth image analysis for indoor fall detection of elderly people. Digital Communications and Networks Vol. 2, No. 1, 24–34, 2016.
Yu, M.; Rhuma, A.; Naqvi, S. M.; Wang, L.; Chambers, J. A posture recognition-based fall detection system for monitoring an elderly person in a smart home environment. IEEE Transactions on Information Technology in Biomedicine Vol. 16, No. 6, 1274–1286, 2012.
Minaee, S.; Boykov, Y.; Porikli, F.; Plaza, A.; Kehtarnavaz, N.; Terzopoulos, D. Image segmentation using deep learning: A survey. arXiv preprint arXiv:2001.05566 2020.
Badrinarayanan, V.; Kendall, A.; Cipolla, R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 12, 2481–2495, 2017.
Farnebäck, G. Two-frame motion estimation based on polynomial expansion. In: Proceedings of the 13th Scandinavian Conference on Image Analysis, 363–370, 2003.
Bouguet, J. Y. Pyramidal implementation of the Lucas Kanade feature tracker description of the algorithm. Intel Corporation Microprocessor Research Labs.
Chang, C.-C.; Lin, C.-J. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology Vol. 2, No. 3, Article No. 27, 2011.
Bae, T. M.; Jin, S. H.; Ro, Y. M. Video segmentation using hidden Markov model with multimodal features. In: Image and Video Retrieval. Lecture Notes in Computer Science, Vol. 3115. Enser, P.; Kompatsiaris, Y.; O'Connor, N. E.; Smeaton, A. F.; Smeulders, A. W. M. Eds. Springer Berlin Heidelberg, 401–409, 2014.
Zhai, Y.; Shah, M. Video scene segmentation using Markov chain Monte Carlo. IEEE Transactions on Multimedia Vol. 8, No. 4, 686–697, 2006.
Charfi, I.; Miteran, J.; Dubois, J.; Atri, M.; Tourki, R. Definition and performance evaluation of a robust SVM based fall detection solution. In: Proceedings of the 8th International Conference on Signal Image Technology and Internet Based Systems, 218–224, 2012.
Auvinet, E.; Rougier, C.; Meunier, J.; St-Arnaud, A.; Rousseau, J. Multiple cameras fall dataset. Technical Report No. 1350. DIRO-Université de Montréal, 2010.
Khan, S.; Vizzari, G.; Bandini, S.; Basalamah, S. Detecting dominant motion flows and people counting in high density crowds. Journal of WSCG Vol. 22, Nos. 1–2, 21–30, 2014.
Acknowledgements
This research is part of the ANGEL project supported by MERSFC and CNRST.
Author information
Authors and Affiliations
Corresponding author
Additional information
Abderrazak Iazzi received his bachelor of mathematics and computer science (MIT) degree in 2011 and his master degree in computer science and telecommunications in 2013 from the Faculty of Sciences, Rabat, Morocco. He is currently pursuing his Ph.D. degree in the Laboratory of Computer Science and Telecommunications (LRIT) at Mohammed V University, Rabat, Morocco, where he is studying fall detection for the elderly by exploiting video information.
Mohammed Rziza received his national doctorate in engineering sciences, with image processing speciality, from the Faculty of Science of the Mohammed V-Agdal University, Rabat, Morocco, in 2002. He joined the Faculty of Science, Rabat, Morocco, in 2003 first as an assistant professor, and then as an associate professor in 2010. Since 1997, he has been a member of the LRIT Laboratory and full professor since 2015 at Mohammed V University, Faculty of Sciences. His research interests include image processing, pattern recognition, and computer vision.
Rachid Oulad Haj Thami received his Ph.D. degree in computer science from the Faculty of Sciences Ben MSik Sidi Otthman, Casablanca, Morocco, in 2002. He is currently a full professor of computer engineering with the Higher National School of Computer Science and Systems Analysis (ENSIAS), Rabat IT Center, Mohammed V University, Rabat, Morocco. His research interests include multimedia and information retrieval, image and video analysis, intelligent video surveillance, and health applications.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Iazzi, A., Rziza, M. & Thami, R.O.H. Efficient fall activity recognition by combining shape and motion features. Comp. Visual Media 6, 247–263 (2020). https://doi.org/10.1007/s41095-020-0183-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-020-0183-7