Keywords

1 Introduction

Gait is a very common behavior of people in daily life [1, 2] to exhibit the physical and psychological states of people. In the existing researches, gait is treated as a biometric feature for automatic identification and verification of human identities [3, 4, 5, 6, 7]. However, gait can disclose more than identity and it provides an informative indication of the physical condition of a person, such as tumbles, jumps, and limps, which are regarded abnormal gait patterns.

A couple of clinical gait analysis studies have shown that abnormal gait can reflect a person’s physical problems or the deterioration of a patient’s health [17, 18, 19]. So, computer-aided gait analysis provides decision support to doctors for more reliable and accurate diagnosis of a disease. Moreover, it acts as an essential means in terms of healthcare for elder people, which can alarm physical problems of elder people in early time.

Depending on the sensors to collect the motion profiles of people, the existing researches in terms of abnormal gait detection can be sorted into two categories: Wearable device-based and video surveillance-based.

For wearable sensor-based abnormal gait detection, in [8], the gait data are acquired using six body-worn tags and wall-mounted sensors and a classifier is adopted for recognition of health problems in the context of machine learning. In [9], the falling-down detection is performed based on a system of wearable sensors. In [10], some fine-grained gait features representing stability, symmetry, and harmony are extracted from plantar pressure data collected from the sensors placed under toes and heels, and finally the gait patterns of Parkinson’s disease are identified by employing Back Propagation (BP) Neural Network. Wearable devices require people to carry some extra equipment and most people are unwilling to wear all day. So, the scenarios of applications are limited.

For abnormal gait detection in surveillance videos, first, gait silhouettes of human bodies should be detected, and then motion features should be extracted from gait silhouettes for anomaly detection. In [11], a system to recognize abnormal activities for elderly care is developed based on R-transform and kernel discriminant. In [12], grid-based partition of body silhouettes is applied to detect abnormal gait patterns with support vector machine. In [13], the authors characterize gait patterns using silhouette-masked flow histograms. In [14], the gait patterns are classified as normal and abnormal using K-Nearest-Neighbor (KNN) classifier.

All of the aforementioned works suffer from the following problems:

  1. (1)

    The methodology in terms of supervised machine learning relies on learning the prototypes of a couple of known types of anomalies. However, it is not practical to foresee or include all types of anomalies in the training data. Once some novel types of anomalies appear, the prototypes of which have never been applied to the classifier before, the anomaly detection system will fail to recognize such novel patterns.

  2. (2)

    To render a high-precision anomaly detection system in the framework of supervised machine leaning, a large-scale training data is usually needed, which incurs time-consuming machine learning over large-scale high-dimensional data.

In view of the limits of the existing works, our work is based on such an intuition that normal gait is nearly periodic signal and anomalies usually disturb such periodicity. By evaluating whether the signal of working rhythm is periodical in terms of signal processing, the proposed method does not need any machine learning. As a one-dimensional statistic over FFT to measure anomalies in the present of a periodical signal, the proposed method is non-parametric and quite efficient.

The contribution of the proposed method is as follows:

  1. (1)

    Abnormal gait detection is formulated as an unsupervised leaning problem and solved in the context of outlier detection over time series, which is different from the existing researches based on supervised machine learning. Such a solution is more practical in that no prior knowledge regarding the prototypes of abnormal walking patterns is required and the anomalies not consistent with the known prototypes can also be detected.

  2. (2)

    The time-varying ratio of width to height of a walking person forms a time series to characterize the rhythm of walking, namely, aspect ratio [16]. Then, we transform the time series of the aspect ratio to frequency domain using Fast Fourier Transform (FFT) and compute the standard deviation of the corresponding spectral magnitude as an indicator for anomaly detection, which is subject to any sudden change or strong pulse-like disturbance in the normally periodical time series of aspect ratio. Such a scheme for change detection in time series has a solid background in terms of signal processing.

Since only one-dimensional measure is applied to anomaly detection, the proposed solution is non-parametric and quite efficient in contrast to supervised machine learning, which rely on large-scale training data and time-consuming machine learning over large-scale high-dimensional data.

The experimental results show that the method promises high precision in detecting abnormal gait patterns from normal cases even in the case that the walking subjects exhibit a variety of abnormal gait motions, where no training data is required and the computational load is low.

2 The Proposed Method

Although there are many researches on gait based on computer vision, the most researches are focus on automatic identification and verification of human identities. There are few researches on the analysis and recognition of abnormal gait by computer vision technology. The main purpose of our work is to realize the abnormal gait recognition by analyzing the gait, so as to infer whether the individual is in an abnormal state.

In the literature, the time series of aspect ratio [16] is an important feature of human gait, which is the ratio of width to height changing over time to reflect the walking rhythm. Intuitively, the time series of aspect ratio can be regarded as a pseudo periodical signal for normal gait, and the periodicity of the aspect ratio should be unstable in case any abnormal gait appears.

The proposed method begins with extraction of human silhouettes from input videos using background subtraction. Then we use bounding box to enclose human gait silhouette in each frame and get the aspect ratio of the gait silhouette. So we can get the aspect ratio time series from input videos. Using Fast Fourier transform, we can observe and analyze the time series of aspect ratio in the frequency domain. Finally, we use the standard deviation of spectral magnitude to distinguish abnormal gaits from normal ones.

In Fig. 1, the whole procedure of the pipelined processing for feature extraction is illustrated.

Fig. 1.
figure 1

Flowchart of the proposed method.

Full size image

2.1 Gait Silhouette Extraction

The proposed method begins with extraction of human silhouettes from input videos. Here, the environment is continuously monitored by a stationary video camera, which provides the required input for gait detection. The input video is segmented into clips, each of which is composed of a couple of continuous frames of images for further processing. Initially, a background model of the environment is obtained and then, every frame in the video sequence is subtracted by the background model [15] to obtain the foreground image as shown in Fig. 2.

Fig. 2.
figure 2

Gait silhouette extraction: (a) Original image; (b) Background image; (c) Human silhouette.

Full size image

In Fig. 2, (a) is a frame of video, (b) is the background image obtained from the first 100 frames of the video sequence, (c) is the human silhouette obtained by the differential processing.

2.2 Aspect Ratio

In Fig. 3, we use bounding box to enclose human gait silhouette, so we can get the width and height of the gait silhouette of interest.

Fig. 3.
figure 3

Aspect ratio: (a) Bounding box of gait silhouette; (b) Width and height of the silhouette to compute aspect ratio.

Full size image

Then, the aspect ratio [16] is defined as the ratio of the width of the person to the height of the person in Eq. (1), which is one of the most important features of human gait.

$$ {\text{Aspect}}\;{\text{Ratio}} = {\text{Width/Height}} $$
(1)

The aspect ratio of the human body in a single frame image is a single value. Therefore, we need to extract gait silhouette from the video sequence, according to the changes of gait in the process of dynamic change of human walking, and extract the characteristic of the aspect ratio of the human silhouette in each frame. When the aspect ratio time series of the human silhouette is obtained, we focus on the change of the aspect ratio time series of the human body in the video sequence. In Fig. 4 shows the variation of the aspect ratio in a video sequence.

Fig. 4.
figure 4

The aspect ratio time series of the normal gait.

Full size image

As can be seen from Fig. 4, the aspect ratio of the normal gait shows a quasi- periodic variation with time evolution with regular rhythm of peaks and bottoms to appear.

Examples of four types of gaits are shown in Fig. 5, which are normal, tumble, jump and limp.

Fig. 5.
figure 5

Examples of four types of gaits: (a) Normal; (b) Tumble; (c) Jump; (d) Limp

Full size image

The corresponding silhouettes, aspect ratio, and standard deviation of the spectral representation of aspect ratio are shown in Fig. 6.

Fig. 6.
figure 6

The exemplar silhouettes, aspect ratio, and standard deviation of spectral magnitude of aspect ratio for four types of gaits: (a), (b), (c): normal; (d), (e), (f): tumble; (g), (h), (i): jump; (j), (k), (l): limp. Here, (a), (d), (g), (j) are the silhouettes; (b), (e), (h), (k) are the corresponding aspect ratio changing over time; (c), (f), (i), (l) are the standard deviations of the frequency.

Full size image

As shown in Fig. 6(b), the aspect ratio of the moving human silhouette shows periodic variation with time with regular rhythm of peaks and bottoms to appear. For the abnormal cases shown in Fig. 6(e), (h), (k), it can be observed that the periodicity of the time series of aspect ratio is disturbed.

2.3 Statistic over FFT Regarding Periodicity

Using Fast Fourier transform, we can observe and analyze the time series of aspect ratio in the frequency domain. It is well-known that by means of FFT, any periodical signal can be decomposed into a couple of basic periodical signals in terms of sine or cosine waveforms with different frequencies. So, FFT is a primary analytical tool to observe periodicity of signals in frequency domain. This motivates us to use the standard deviation of spectral magnitude to distinguish abnormal gaits from normal ones. In our method, the aspect ratio time series is first segmented according to the extreme value, and then the fast Fourier transform is performed on each segment. Finally, the standard deviation of each segment is obtained.

3 Experiments

3.1 Data Collection

Since there is no benchmark for abnormal gait detection in unsupervised manner, For performance evaluation, we collect video data to do research by ourselves. In the experiment, the camera is fixed to observe the lateral image of human walking. In total, eight subjects are required to walk across the same background both normally and in an unusual way, such as tumble, skip, and limp. Some sample images are shown in Fig. 5. The data sets include 32 sequences, and an average length of each sequence is about 300 frames.

3.2 Performance Evaluation

As shown in Fig. 6, we can observe a strong regularity from the periodicity of normal gait. In such a case, the standard deviation of the spectral magnitude is relatively small, and on the contrary, the feature of abnormal gaits is large.

Figure 7 shows that the normal and abnormal gaits correspond with smaller and bigger standard deviation, respectively. The X axis in the figure is a section of the aspect ratio time series according to the extreme value. The Y axis in the figure is the standard deviation of the frequency of each segment.

Fig. 7.
figure 7

Comparison of the FFT features of different gaits.

Full size image

For performance evaluation, the accuracy of the detection results is defined as the ratio of the correctly detected anomalies to the number of all anomalies as follows:

$$ Accuracy = \frac{1}{N}\sum\limits_{i = 1}^{N} {\delta (F_{i} > \theta )} $$
(2)

where Fi represents the FFT feature of the ith gait sample, N the number of all abnormal gait samples, θ a predefined threshold, and δ(Fi > θ) = 1 if Fi > θ, which functions to count the correct hits of anomalies. The setting of parameter θ can affect the accuracy of abnormal gait detection. Figure 8 shows the effects of different parameter values on the detection results, where the parameter θ is assumed to be between 0.00009 and 0.004. Note that the detection results can reach 100% if the parameter value is properly set.

Fig. 8.
figure 8

Precision of anomaly detection against parameter θ.

Full size image

Similar to our works, the accuracy of the detection method using KNN classifier in work [14] is 77%, and the method in [13] using silhouette-masked flow histograms to analyze abnormal walking gait, the accuracy is 90%. Obviously, the accuracy of our algorithm is higher.

The proposed method is implemented on a computer configured with Core i5 and 8 GB RAM. As our method does not need any training, the processing time for each frame is 256 ms under MATLAB, which is obviously efficient.

4 Conclusions

Human gait contains a lot of information, and gait characteristics of human are difficultly to disguise and are easily to obtain. Gait can exhibit the physical and psychological states of people to some degree. Abnormal gait may represent some problems of the physical and psychological states of people, so it is very important to detect abnormal gait rapidly, for treatment diseases or prevention threats to public safety.

In this paper, An unsupervised approach for abnormal gait detection is proposed, which requires no prior knowledge regarding prototypes of abnormal gaits, nor any machine learning. Here, the periodicity of the time series of aspect ratio is measured in frequency domain as the standard deviation of the spectral magnitude, which acts as the indicator of how the nearly periodical walking rhythm in normal case is disturbed. The experimental results show that the proposed method is not only promising in terms of precision but also efficient due to the one-dimensional non-parametric feature adopted. The method has the potential to be applied to a variety of applications, including unobtrusive clinical gait analysis and automated video surveillance.

Currently, our methods cannot solve the problems in different scales and the camera motion. In the future, we will work to study the problem in different, unconstrained real world settings. In addition, we are working to adopt our approach to situations with several people and find the method to extract and analyze several silhouettes simultaneously.