Keywords

1 Introduction

Falls among the elderly are currently a real human and financial scourge. 25 to 30% of people over 65 and 50% of those over 80 fall at least once a year. These are the first cause of emergency visits for people over 75 and the second cause of death by unintentional trauma. A significant proportion of falls result in femoral neck fractures, a real public health issue (Katsoulis, 2017 [1] Oberlin, 2016 [2]). Furthermore, research has demonstrated that the apprehension of falling is linked to unfavorable outcomes, including reduced engagement in everyday activities, decreased level of physical activity, increased likelihood of falling, symptoms of depression, and a lower overall quality of life [3]. According to INSERM (2017) [4], between 50,000 and 80,000 individuals in France, and approximately 1.5 million around the world (Silver Eco 2017 portal) experience fractures of the neck of the femur. This type of fracture has severe implications as one-quarter of people pass away within one year, and 50% of those who survive experience a considerable loss of autonomy. This work is part of a project led by Indienov, a company that created a smart airbag belt capable of analyzing the movements of individuals and detecting anomalies (in this particular case falls and pre-falls), and instantly activate a perfectly positioned airbag prevent fractures of the femoral neck or greater trochanter (Fig. 1). The belt will also be able to detect critical situations such as lying for a prolonged time.

Fig. 1.
figure 1

Belt Airbag illustration

Full size image

2 Literature Review

A large variety of fall detection systems are available and rely on diverse sensors (cameras, infrared motion sensors, LIDAR, and inertial motion units). These fall detection systems are generally classified into two categories: fixed systems (in the environment) and mobile/wearable systems (necklaces, watches, belts, phones, etc.) [5]. We focused on mobile systems using Inertial Motion Units.

2.1 State of the Art

Initial research concerning fall detection systems employed threshold-based algorithms, and demonstrated that by applying thresholds to vertical and horizontal velocities, it is possible to differentiate between daily activities and falls [6, 7] with remarkable accuracy. In a study based on a sensor located on the chest [20], a fall detection algorithm is optimized using the Upper Fall Threshold (UFT) and Lower Fall Threshold (LFT) of the accelerometer combined with the UFT of the gyroscope for fall detection. The experimental results showed that the algorithm was able to detect falls compared to other daily movements with a sensitivity and specificity of 96.3% and 96.2%, respectively. The addition of gyroscope data significantly improves specificity over published results based solely on accelerometer data, as changes in angular velocity provide an additional indication of a fall event compared to other activities that may also experience high acceleration spikes. The use of LFTs and UFTs is equivalent to determining the window of fall and detecting it by means of its start and end.

Alternatively, Li et al. [8] used static postures and dynamic transitions between them to detect falls, defining a fall as a static “lying” posture with an unintentional transition to it. Numerous other studies have examined fall detection, including reviews, learning algorithms with automatic feature extraction, etc. [9,10,11,12,13,14,15,16,17,18], some of which have been found to be highly reliable [19].

Detecting the onset of a fall, or pre-fall detection, presents a challenging problem. Unlike fall detection, which relies on identifying the impact of a fall visible on acceleration signals, pre-fall detection requires detecting the early signs of an impending fall. As a result, accelerometer and gyroscope signals may not yet exhibit features indicative of a fall. The primary hurdle in solving this issue is to distinguish the crucial warning signs that trigger a fall while minimizing the number of inaccurate detections (false positive cases).

In the context of pre-fall detection, one of the earliest fall protection systems was proposed by G. Shi et al. [21] and later enhanced [22]. It relies on an inertial motion unit and airbag deployment, along with a Support Vector Machine (SVM) prediction model and an embedded Digital Signal Processing (DSP) unit.

Table 1. Fall prediction algorithms comparison
Full size table

In recent years, machine learning models have gained popularity due to their good performances in fall and pre-fall detection. In a comparative study conducted by Yu et al. [24], a hybrid ConvLSTM model was evaluated and compared to other existing models. The results showed that the ConvLSTM model outperformed other models for all three classes (no drop, drop before impact, and drop), with average sensitivities of 93.15%, 93.78%, and 96.00% for no drop, drop before impact, and drop, respectively. Furthermore, the model demonstrated higher specificities for all three classes (96.59%, 94.49%, and 98.69%) compared to the LSTM and CNN models. These results suggest that the proposed hybrid model outperforms the LSTM and CNN models, providing high detection accuracy (particularly for the pre-impact drop). The combination of convolution and recurrent models allows feature extraction and temporal linkage between the data, leading to satisfactory results compared to using these models separately.

Table 2. Comparison of the Kfall model performances on simulated data and real data. Source : [27]
Full size table

Recent fall prediction methods have shown very promising results (as shown in Table 1). However, one important issue highlighted in the literature is the need to compare the performance of fall detection algorithms using real-world data, as significant discrepancies have been observed between the results obtained from simulated falls data of healthy individuals and real falls data from at-risk populations [11]. The Bourke and Chen algorithms perform the best in terms of sensitivity and specificity, but all algorithms exhibit lower performance on real falls compared to simulated falls. The latter being also true for the work from Yu et al. (as shown in Table 2) despite the very high performance obtained on simulation data (99.32%, 99.03% of sensitivity, and specificity, respectively). This difference in performance was mainly attributed to the fact that the movements in real falls do not exhibit the same signatures as simulated falls, especially in terms of the amplitude before impact [28].

3 Methods and Material

3.1 Data Collection

As mentioned above, the problem of motion and fall data for at-risk individuals currently remains one of the biggest challenges. The first step consists of obtaining real, reliable motion data that comes from our target audience. Then, in addition to retrieving widely used movement data from the internet (especially in research, e.g. SisFall [30], MobiFall [29], kFall [27] etc.), a specific protocol based on [29, 30] was developed to gather experimental data (Table 3 and 4).

Table 3. Public Data Overview
Full size table
Table 4. Acquired Data Overview
Full size table

A data collection campaign has been initiated in EHPAD (institutions for dependent elderly people) (Dataset_03, Dataset_04), in which multiple volunteers were equipped with dataloggers (embedded sensors with IMU + SD card - attached to a belt worn at the waist) to monitor their activities throughout the day (and in some cases, the entire week from morning to evening). The purpose of the database obtained from this data collection is to develop solutions and algorithms adapted to the target population, the elderly, where detecting pre-fall events is of utmost importance. As the participants’ profiles are highly suitable for this purpose, the collected data is expected to provide valuable insights and aid in validating the developed solutions and algorithms.

3.2 Configuration and Setup

After examining the SisFall and MobiFall datasets, we discovered that they were not optimal for predicting falls because they lacked temporal information on the pre-fall phase. Even though there has been a lot of research on fall detection, there hasn’t been much done on prefall detection. The key contrast between prefall detection and fall detection is found here. Prefall detection focuses on detecting the beginning of a fall, which happens over a brief period of time and then passes. In other words, throughout the fall, we can only see the beginning of the fall during a specified time period. On the other side, fall detection focuses on identifying a fall when it really occurs, generally after the impact has taken place. At any time after the fall, it is relatively simple to identify because the impact of the fall clearly leaves an imprint in the signals. Therefore, we had to identify and label the pre-fall phase, which may have introduced bias into the labeling process. In contrast, the kfall dataset contained more detailed information on the onset and impact of falls, making it more useful for fall prediction. Additionally, the age and profiles of the participants in the various datasets were not very representative, so we generated new data for activities of daily living and falls that included a wide range of realistic scenarios.

For the labeled data, participants followed a predefined protocol that included activities from the SisFall dataset as well as additional daily activities such as walking, using stairs with a cane or walker, doing household chores, and dancing. We also included wheelchair activities. Unlabeled data were collected from participants engaged in their normal activities throughout the day, resulting in data that better reflects real-world scenarios. For data recording, we used dataloggers, which are developed by the same company, and we recorded the videos of these simulations. The IMU (MPU6050) that we used was equipped with a 3-axis accelerometer and gyroscope, with a resolution of \(\mathrm {\pm }\)8 [g] for the accelerometer and \(\mathrm {\pm }\)1000\(^{\circ }\)/s for the gyroscope. The sampling frequency (200 [Hz]) is defined based on the data analysis and signal processing.

3.3 Data Processing and Feature Extraction

Data Cleaning and Labelling: To reduce accelerometer sensitivity noise, we used a 3rd order Butterworth lowpass filter with a cutoff frequency of 20 [Hz] as first step. We then divided the fall data into four stages (Fig. 2): non-risk, pre-fall, the impact, and the post-fall. This categorization was based on notes taken during fall simulations (such as the type of fall, subject, and duration) and recorded video data (including the start time of the fall, the end time of the pre-fall phase, the time of ground contact, and the time of impact).

Fig. 2.
figure 2

Fall Data Segmentation.

Full size image
Fig. 3.
figure 3

Activities signal examples - Acc: accelerometer (acc_x, acc_y, acc_z), Gyro: gyroscope (gyro_x, gyro_y, gyro_z) - 3 Normal Activities (1, 2, 3) vs one Fall Activity (4)

Full size image

Data Analysis and Feature Selection: An initial visual inspection was performed to analyze the collected data. This enabled the identification of movement-specific signatures and variable behavior, and the tracking of their fluctuations over time in relation to distinct activities and subject profiles (as shown in Fig. 3). A statistical examination of the information obtained from various activities was performed with the aim of identifying indicators that can distinguish between ADLs and falls (Fig. 4). Once this stage was completed, we selected the variables that would be used in computing the features (listed in Table 5). Subsequently, statistical computations were carried out on each variable, resulting in a collection of potential features for pre-fall detection (Table 6). The features were calculated in sliding windows of 0.5[s].

Fig. 4.
figure 4

Analysis of the data distribution examples - compare the data distribution over different activities (ADLs vs FALLs)

Full size image
Table 5. Selected Variables for Feature calculation
Full size table
Table 6. Selected Features for Calculation
Full size table

Algorithm and Features: For feature selection, we started an optimization process over all the features (cited on Table 6), based on recursive elimination of features by KNN (K Nearest Neighbors) and RF (Random Forest) algorithms, so we tested several combinations depending on the type of falls we were proposing and the performances we were getting. Ultimately, we chose an algorithm based on the RF (random forest) and the most important features, which we selected for their relevance to decision making. Our solution detects pre-falls and falls (Fig. 5), then, sends an alert to relatives and caregivers (previously selected).

Fig. 5.
figure 5

Fall Detection/Prediction System.

Full size image

3.4 Model Deployment in Embedded Architecture

Due to their inherent resource constraints, embedded systems are unable to support sophisticated inference models. As a result, implementing AI models in these systems becomes difficult, especially in real-time scenarios when characteristics like response speed, space complexity, and computing complexity are critical. Focusing on low complexity choices is necessary since these limits impose extra factors to take into account when choosing the right model. Deep learning techniques, for instance, may not be practical due to their inference, time, and space complexity, necessitating careful attention throughout implementation. As a result, it may be necessary to develop customized methods to optimize model training, conversion, and implementation for better optimization and utilization of hardware resources such as DSP and FPU. Various methods (weight pruning, quantization, encoding, etc.) and solutions (TensorFlow Lite Micro, EdgeML, STM32Cube.AI, etc.) are available to address this challenge [31], but they require careful consideration and planning to achieve the desired results in the context of TinyML.

4 Experiments and Evaluation

4.1 Experiments

Initially, we tested our solution on daily life and fall acquired data to ensure its functionality (this was the Experiment_00). Our ultimate goal was to develop a functional and customized solution for the airbag belt. Therefore, we implemented and deployed the solution in the embedded system (belt) and conducted three types of experiments.

  • Experiment_00: In this experiment, we have tested our solution on the acquired data (which contains ADLs and Falls).

  • Experiment_01: The first test involved wearing the belt and performing guided activities (ADLs and falls).

  • Experiment_02: The second test involved wearing the belt throughout the day and engaging in their daily activities.

  • Experiment_03: Finally, we tested the belt on elderly people in EHPAD for a whole day and evaluated the performance recorded in the log files.

The first and second tests were conducted by young and healthy individuals.

4.2 Evaluation

We established a set of evaluation criteria for our solutions, which serve to measure their quality and degree of reliability. These criteria include:

.:

Sensitivity: measures the solution’s ability to detect and predict falls accurately. \(\mathrm {Sensitivity = \frac{TP}{TP + FN}}\), \(\textrm{TP}\) : true positive, \(\textrm{FN}\) : false negative. (7)

.:

Specificity: measures its ability to correctly identify activities of daily living (ADLs). \(\mathrm {Specificity = \frac{TN}{TN + FP}}\), \(\textrm{TN}\) : true negative, \(\textrm{FP}\) : false positive. (8)

.:

Time Lead: which refers to the time between the prediction of a fall and the actual impact of the fall. Our objective is to detect pre-fall movements at least 220 [ms] before impact (safe deployment of airbags).

Finally, we use performance as an overall measure of how well our solution performs based on the aforementioned criteria. This value is calculated using a weighted sum of sensitivity, specificity, and time lead. Based on the desired outcomes, it is possible to set the weights of the parameters, such as emphasizing sensitivity, specificity, or lead time.

\(\mathrm {Performances = (P_{sn} * Sensivity) + (P_{sp} * Specifity) + (P_{tm} * time\_lead)}\). (9)

\(\mathrm {Performances \in [0, 1]}\), P: weights, \(\mathrm {P_i \in [0, 1], \displaystyle \sum P_i = 1}\), time_lead normalized

5 Experimental Result Analysis

The Dataset_02 was used for training, and subsequent optimization techniques were implemented to select a subset of the data that would yield lightweight and tailored models optimized for embedded systems. To assess the effectiveness of our solution, we employed different methods depending on the experiment. For Exp. 0 and Exp. 1, we evaluated the performance based on the number of simulations (each activity labeled as ADL or Fall), enabling us to determine false positive and false negative rates. For other experiments (exp. 2 and exp. 3), we based our evaluations on the number of activity hours and participant-reported information. If the participant doesn’t fall but a fall was predicted or detected, it is considered a false positive. If the participant falls, but we didn’t detect anything, it is considered a false negative. However, if the person falls and we predict it, it was considered as a true positive.

Generally, falls in real life are infrequent, so our solution specificity is highlighted in exp. 2 and exp. 3. When we encounter false positives or false negatives, we analyze the data and incorporate it into subsequent training sessions. The benefit of these experiments is that we can evaluate and validate our solution under real-world conditions with participants who are natural and free, unlike other evaluation tests. Therefore, our solution is exposed to situations (activities) not seen during training, representing a challenge for us. The evaluation results that we currently have are presented in (Table 7). It should be noted that the evaluation of our solution is still ongoing, with particular focus on (Exp. 03).

Table 7. Experimental results
Full size table

6 Discussion

Based on the initial results from experiments (exp. 00 and exp. 01), it appears that fall detection is facilitated by the fact that signals already contain the necessary information to accurately detect falls. This is supported by the high level of sensitivity and specificity observed. The fall prediction models showed a decrease in performance when the environment and subjects were changed (between exp. 00 and exp. 01), which is likely attributed to the prior compression of the models before deployment on embedded devices. This compression was intended to reduce the models space-time complexity and make them more lightweight. The two additional experiments exposed our solution to real-world conditions, which allowed participants greater freedom of movement, resulting in signals that differed, sometimes markedly, from what we had observed in our learning database. Moreover, as the belt-wearing was subtle, participants behaved more naturally, revealing movements we had not encountered during training.

7 Conclusion

This study involved the development of a fall detection system and a fall prediction system, which were adapted and deployed on an airbag belt to test their effectiveness in real-life conditions. While the theoretical results were comparable to other works, the goal was to create a functional system that performed well in both theory and practice. This is in contrast to some studies that relied on theoretical results or private motion data to which the researchers did not have access. However, the lack of movement data from elderly people and real fall data posed significant challenges. Therefore, the plan for the future is to launch a collection campaign in EHPAD to gather natural data over a long period of time, including real falls that can be added to the learning base.