Skip to main content

Advertisement

SpringerLink
Log in

Fall detection method based on Spatio-temporal feature fusion using combined two-channel classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Nowadays, the growing population of senior citizens is a challenge for almost all developing countries. New technologies can help monitor elderlies at home by providing an innovative and secure environment and further enhancing their quality of living. Vision-based systems offer promising results in analyzing human posture and detecting abnormal events like falls. Falls appear to possess the most considerable risk for seniors living alone. In this article, a new fall detection method is proposed based on a fusion of motion-based and human shape-based features. Motion History Images (MHI) represent the temporal feature in our approach. Simultaneously, the height-to-width ratio and centroid of the moving person represent the spatial features. A two-channel classification model is designed using a threshold-based and a keyframe-based approach. The two channels are further combined based on any classification disparity for which more information is used to classify between falls and daily activities. Keyframes are selected based on the displacement of the spatial features having a threshold higher than a preset value. Keyframes are subject to a K-NN classification. The proposed algorithm delivers promising results on the UR fall detection dataset’s simulated fall and daily activity sequences. It provides satisfactory performance compared to existing state-of-the-art methods and shows a peak accuracy of 98.6% and recall of 100% in detecting falls. Specificity and precision are over 96%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Ahad MAR, Tan JK, Kim H, Ishikawa S (2012) Motion history image: its variants and applications. Mach Vis Appl 23(2):255–281

    Article  Google Scholar 

  2. Barron JL, Fleet DJ, Beauchemin SS (1994) Performance of optical flow techniques. Int J Comput Vis 12:43–77. https://doi.org/10.1007/BF01420984

    Article  Google Scholar 

  3. Bobick AF, Davis JW (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267

    Article  Google Scholar 

  4. Chelli A, Patzold M (2019) A machine learning approach for fall detection and daily living activity recognition. IEEE Access 7:38670–38687. https://doi.org/10.1109/ACCESS.2019.2906693

    Article  Google Scholar 

  5. Chen M-C, Liu Y-M(2013) An indoor video surveillance system with intelligent fall detection capability. Math Probl Eng 2013:1–8

    Google Scholar 

  6. Directalert. Wireless emergency response system [Online]. Available: http://www.directalert.ca/emergency/help-button.php. Accessed 2 July 2021

  7. Garripoli C, Mercuri M, Karsmakers P, Soh PJ, Crupi G, Vandenbosch GA, Schreurs D (2014) Embedded DSP-based tele health radar system for remote in-door fall detection. IEEE J Biomed Health Inform 19(1):92–101

    Article  Google Scholar 

  8. Gracewell JJ, Pavalarajan S (2019) Fall detection based on posture classification for smart home environment. J Ambient Intell Humaniz Comput 2019:1–8

    Google Scholar 

  9. Gunale K, Mukherji P (2018) Indoor human fall detection system based on automatic vision using computer vision and machine learning technique. J Eng Sci Technol 13(8):2587–2605

    Google Scholar 

  10. Gutiérrez J, Rodríguez V, Martin S (2021) Comprehensive review of vision-based fall detection systems. Sensors 21:947. https://doi.org/10.3390/s21030947

    Article  Google Scholar 

  11. Han Q, Zhao H, Min W, Cui H, Zhou X, Zuo K, Liu R (2020) A two-stream approach to fall detection with MobileVGG. IEEE Access 8:17556–17566

    Article  Google Scholar 

  12. Hao Z, Duan Y, Dang X, Xu H (2019) KS-FALL: indoor human fall detection method under 5 GHz wireless signals. IOP Conf Ser Mater Sci Eng 569(3):8

    Article  Google Scholar 

  13. Htun SN, Zin TT, Tin P (2020) Image processing technique and hidden Markov model for an elderly care monitoring system. J Imaging 6(49)

  14. Huang Y, Chen W, Chen H, Wang L, Wu K (2019) G-Fall: device-free and training-free fall detection with geophones. In: 2019 16th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON). Boston, MA, USA, pp. 1–9

  15. Hussain F, Hussain F, Ehatisham-ul-Haq M, Azam MA (2019)Activity-aware fall detection and recognition based on wearable sensors. IEEE Sensors J 19(12):4528–4536

    Article  Google Scholar 

  16. Jamil N, Sembok TMT, Bakar ZA (2008) Noise removal and enhancement of binary images using morphological operations. In: 2008 International Symposium on Information Technology. Kuala Lumpur, pp 1–6

  17. Kalinga T, Sirithunge C, Buddhika A, Jayasekara P, Perera I (2020) A fall detection and emergency notification system for elderly. In: Proceedings of the 2020 6th International Conference on Control, Automation and Robotics (ICCAR). Singapore, pp 706–712

  18. Kepski M, Kwolek B (2014) Human fall detection on embedded platform using depth maps and wireless accelerometer. Comput Methods Prog Biomed 117(3):489–501

    Article  Google Scholar 

  19. Kepski M, Kwolek B (2015) Embedded system for fall detection using body-worn accelerometer and depth sensor. In: Proceedings of IEEE 8th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), pp 755-759

  20. Kerdjidj O, Ramzan N, Ghanem K, Amira A, Chouireb F (2019) Fall detection and human activity classification using wearable sensors and compressed sensing. J Ambient Intell Humaniz Comput 2020:1–13

  21. Khaire P, Kumar P, Imran J (2018) Combining CNN streams of RGB-D and skeletal data for human activity recognition. Pattern Recogn Lett 115:107–116

    Article  Google Scholar 

  22. Krumholz A, Hopp J (2008) Falls give another reason for taking seizures to heart. Neurology 70:1874–1875

    Article  Google Scholar 

  23. Luo Y, Zhou H, Tan Q, Chen X, Yun M (2018) Key frame extraction of surveillance video based on moving object detection and image similarity. Pattern Recog Image Anal 28:225–231. https://doi.org/10.1134/S1054661818020190

    Article  Google Scholar 

  24. Makandar A, Mulimani D, Jeevor M (2015) Preprocessing step—review of key frame extraction techniques for object detection in video. Int J Curr Eng Technol 5(3):2036–2039

    Google Scholar 

  25. Makhlouf A, Boudouane I, Saadia N, Amar RC (2018) Ambient assistance service for fall and heart problem detection. J Ambient Intell Humaniz Comput 10(4):1527–1546

    Article  Google Scholar 

  26. Merrouche F, Baha N (2017) Fall detection using head tracking and centroid movement based on a depth camera. In: Proceedings of ACM International Conference on Computing for Engineering and Sciences (ICCES), pp 29-34

  27. Paul M, Haque SME, Chakraborty S (2013) Human detection in surveillance videos and its applications—a review. EURASIP J Adv Signal Proc 176

  28. Peng Y, Peng J, Li J, Yan P, Hu B (2019) Design and development of the fall detection system based on point cloud. Procedia Comput Sci 147:271–275

    Article  Google Scholar 

  29. Queralta JP, Gia T, Tenhunen H, Westerlund T (2019)Edge-AI in Lora-based health monitoring: fall detection system with fog computing and LSTM recurrent neural networks. In: 2019 42nd International Conference on Telecommunications and Signal Processing (TSP) (IEEE), pp 601–604. https://doi.org/10.1109/TSP.2019.8768883

  30. Ramachandran A, Karuppiah A (2020) A survey on recent advances in wearable fall detection systems. Biomed Res Int 2020:1–17. https://doi.org/10.1155/2020/2167160

    Article  Google Scholar 

  31. Rubenstein LZ (2006) Falls in older people: epidemiology, risk factors and strategies for prevention. Age Ageing 35:ii37–ii41

    Article  Google Scholar 

  32. Russell-Jones DL, Shorvon SD (1989) The frequency and consequences of head injury in epileptic seizures. J Neurol Neurosurg Psychiatry 52:659–662

    Article  Google Scholar 

  33. Sabatini AM, Ligorio G, Mannini A, Genovese V, Pinna L (2016)Prior-to- and post-impact fall detection using inertial and barometric altimeter measurements. IEEE Trans Neural Syst Rehabil Eng 24(7):774–783

    Article  Google Scholar 

  34. Sadreazami H, Bolic M, Rajan S (2019) TL-FALL: contactless indoor fall detection using transfer learning from a pretrained mode. In: Proceedings of the 2019 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Istanbul, Turkey, pp 1–5

  35. Shu F, Shu J (2021) An eight-camera fall detection system using human fall pattern recognition via machine learning by a low-cost android box. Sci Rep 11:2471. https://doi.org/10.1038/s41598-021-81115-9

    Article  Google Scholar 

  36. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149). Fort Collins, CO, USA, pp 246–252

  37. Sterling DA, O’connor JA, Bonadies J (2001) Geriatric falls: injury severity is high and disproportionate to mechanism. J Trauma Inj Infect Crit Care 50:116–119

    Article  Google Scholar 

  38. Sulman N, Sanocki T, Goldgof D, Kasturi R (2008) How effective is human video surveillance performance? In: 19th International Conference on Pattern Recognition (ICPR 2008). pp 1–3

  39. Tian Y, Lee G-H, He H, Hsu C-Y, Katabi D (2018)RF-based fall monitoring using convolutional neural networks. Proc ACM Interact Mob Wearable Ubiquit Technol 2(3):1–24. https://doi.org/10.1145/3264947

    Article  Google Scholar 

  40. United Nations (2017) World population ageing 2017—Highlights (ST/ESA/SER.A/397). Department of Economic and Social Affairs, Population Division

  41. Vollset SE, Goren E, Yuan C-W, Cao J, Smith AE, Hsiao T, Bisignano C, Azhar GS, Castro E, Chalek J, Dolgert AJ, Frank T, Fukutaki K, Hay SI, Lozano R, Mokdad AH, Nandakumar V, Pierce M, Pletcher M, … Murray CJL (2020) Fertility, mortality, migration, and population scenarios for 195 countries and territories from 2017 to 2100: a forecasting analysis for the global burden of disease study. Lancet 396:1285–1306. https://doi.org/10.1016/S0140-6736(20)30677-2

    Article  Google Scholar 

  42. Wang H, Zhang D, Wang Y, Ma J, Wang Y, Li S (2017) RT-fall: a real-time and contactless fall detection system with commodity wifi devices. IEEE Trans Mob Comput 16(2):511–526

    Article  Google Scholar 

  43. Wang H, Gao Z, Lin W (2019) A fall detection system based on convolutional neural networks. In: Proceedings of ACM the International Conference on Robotics, Intelligent Control and Artificial Intelligence (RICAI 2019), Shanghai, China, pp 242–246. https://doi.org/10.1145/3366194.3366236

  44. Wang X, Ellul J, Azzopardi G (2020) Elderly fall detection systems: a literature survey. Front Robot AI 7:71. https://doi.org/10.3389/frobt.2020.00071

    Article  Google Scholar 

  45. WHO (2018) Falls. https://www.who.int/news-room/fact-sheets/detail/falls. Accessed 16 Jan 2018

  46. Xi X, Jiang W, Lü Z, Miran SM, Luo Z-Z(2020) Daily activity monitoring and fall detection based on surface electromyography and plantar pressure. Complexity 2020:1–12. https://doi.org/10.1155/2020/9532067

    Article  Google Scholar 

  47. Zhang J, Wu C, Wang Y (2020) Human fall detection based on body posture Spatio-temporal evolution. Sensors 20(3):946. https://doi.org/10.3390/s20030946

    Article  Google Scholar 

  48. Zitouni M, Pan Q, Brulin D, Campo E (2019) Design of a smart sole with advanced fall detection algorithm. J Sensor Technol 9:71–90

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, NIT Agartala, Agartala, India

    Anurag De & Ashim Saha

  2. Department of Computer Science and Engineering, VNIT Nagpur, Nagpur, India

    Praveen Kumar

  3. Department of Computer Science and Engineering, TIT Narsingarh, Agartala, India

    Gautam Pal

Authors
  1. Anurag De
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ashim Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Praveen Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gautam Pal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anurag De.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

De, A., Saha, A., Kumar, P. et al. Fall detection method based on Spatio-temporal feature fusion using combined two-channel classification. Multimed Tools Appl 81, 26081–26100 (2022). https://doi.org/10.1007/s11042-022-11914-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-11914-3

Keywords

Search

Navigation