Background

Multiple sclerosis (MS) is a common chronic inflammatory and neurodegenerative disease that normally begins in young adulthood, typically affecting patient quality of life and leading to high rates of early retirement [1–4]. Impairment of gait and balance are major factors that restrict daily activity [5] and may occur as early as after a first clinical episode [6–8].

In current clinical practice, balance and gait impairment are quantified using a combination of clinical examination and patients’ reported maximum walking distance: The Kurtzke‘s Functional Systems and Expanded Disability Status Scale (EDSS) [9] are widely used both in clinical practice and for clinical trials. The EDSS provides a good overview of current neurologic status, gait impairment and mobility dysfunction, but it has limitations. It is a relatively subjective measure with high intra- and inter-rater variability and quantifying mild symptoms and symptom progression is challenging [10, 11]. The Timed 25-Foot Walk (T25FW) measures the time a patient takes to walk 25 feet at maximum speed. As part of the Multiple Sclerosis Functional Composite (MSFC), the T25FW is currently the most widely implemented method to objectively quantify gait disability in clinical MS trials and, to a lesser extent, clinical practice [12, 13]. Although an excellent method of quantifying overall gait disability, the T25FW measures only the time taken to walk a set distance [14]. As alternatives to the T25FW, timed walking tests might improve reliability. In particular, the 2-minute walk test is proposed as an additional outcome parameter in clinical trials [15].

Recently, more objective motion capture systems (for an overview see [16]) have been proposed as tools for detecting the pattern of gait impairment more precisely than clinical examination using conventional tools (i.e. EDSS) [17, 18]. However, none of these systems has found its way into clinical routine yet, and are therefore rarely available in outpatient clinics and neurologic practices.

With an infrared light camera system, the Microsoft Kinect™, which was originally developed for video gaming, detects anatomical landmark positions in three dimensions (3D) [19]. This function is based on a decision forest method that harnesses mass data of sensor-recorded skeletal joint movements [20]. Using the programming interface made available via the Kinect software development kit (SDK) [21], it has in the past been employed to detect balance and motion of healthy subjects [22, 23] and stroke patients [24, 25]. Furthermore, some few studies have reported methods for gait parameter approximation using Kinect [26–28], however, none of these has been applied to patients with MS.

The objective of this study was to evaluate the feasibility of computerized versions of walking tests using Microsoft Kinect and to compare the resulting findings for MS patients with low to moderate neurological impairment to those for healthy controls. We established the Short Maximum Speed Walk test (SMSW) as a measure to analyse patients’ gait speed and degree of sway. We report feasibility, reliability and correlation of results with those of EDSS and the T25FW.

Methods

Ethics

The local ethics committee of the Charité – Universitätsmedizin Berlin approved the study (EA1/225/12). It was conducted in accordance to the Declaration of Helsinki in its currently applicable version. All patients and healthy subjects gave written informed consent.

Patients

Twenty-two patients with diagnosed MS according to the current panel criteria [29] (age mean ± SD 43 ± 9 years, 13 female) and 22 age and sex matched healthy controls (HC) (age 37 ± 11 years, 13 female) were enrolled in the study. MS patients were recruited from on-going clinical trials at Charité’s neuroimmunology outpatient clinic. HC were recruited from volunteers. Blinding was not attempted, since the operators knew many subjects and visual contact was mandatory. MS patients were first clinically graded using EDSS (median 3.0, range 0.0 – 6.0) under supervision of a neurologist. The EDSS is calculated from a detailed neurological examination using functional system scores that assess the visual system function, brainstem function, pyramidal tract function, cerebellar function, sensory system function, bowel and bladder function, cerebral function and patient ambulation [9]. Subjects additionally performed the T25FW component of the Multiple Sclerosis Functional Composite (MSFC) [30]. T25FW results were not available from eight of the 22 HC, and one of the 22 MS patients. The first 8 HC had been tested before the T25FW was added to the study protocol and the MS patient could not complete the study in full due to time constraints.

Perceptive computing assisted motor assessment

We used custom-built software running on a Windows 8 (Microsoft Corp., Redmond, WA, USA) computer with Kinect Software Development Kit (SDK) version 1.7. A Microsoft Kinect sensor was attached to a fixed pole, thereby covering a triangular area of roughly 2.5×2.5 m. The Kinect system records live videos with a conventional camera and combines these with depth information comprising a combined feed from an infrared projector and an infrared camera. The Kinect Software Development Kit (SDK) then detects the human subject in the 3D video in real-time and models an artificial skeleton with 20 joints of an individual subject and their movement over time [19, 21]. A sample of the process is shown in Figure 1 and Supplementary Video 1. Each subject was tested in an evenly lit room in a single session of sequential tests. All subjects were given the same instruction as specified in a standardized test procedure: “Walk as fast as you can towards the sensor”. The starting point for walking was approximately two metres outside the detection range of the sensor. We postulated that this would allow the subject to reach maximum walking speed before reaching the measurement zone. The start was given as a voice command. An automatic computer-generated sound signalled the subject to end the experiment after leaving the opposite edge of the sensor measurement zone.

Figure 1
figure 1

System setup. A) Schematic system setup (view from above): The Kinect sensor (K) was positioned 140 cm above ground, angled -9° towards the floor. Starting (S) about 2 metres in front of the beam path (light blue area), subjects walked with maximum speed towards the Kinect camera (red arrow). Recording started and stopped automatically as sensors detected the subjects entering and leaving the detection zone (dark blue area). B) Sample screenshot of a healthy subject during the test with skeleton projection (green lines). The hip-centre joint (marked as red dot) was used as the data source for analysis. C) Sample data from single healthy subject after normalization. At the position of the red arrow a measurement frame was removed due to a detected calibration jump.

Full size image

Short Maximum Speed Walk test analysis

The SDK’s recorded skeletal information was post-processed at a frame-rate of 30/s. To assess walking speed and related parameters we extracted the hip-centre joint coordinates over time. An overview of all parameters is given in Table 1. In detail, the data post-processing and analysis comprised the following steps:

Table 1 Overview of SMSW parameters
Full size table

First, joint coordinates were normalized, correcting for sensor tilt and walking direction of the subject by computing a linear approximation of the hip-centre movement, translating the start of the movement to the point of origin and rotating the direction of movement onto the positive Z-axis.

As in any automated assessment tool, artefacts (in the form of tracking errors, or calibration jumps) were inevitable. Specifically, detection of the spatial location of the crucial hip-centre joint by the SDK system was clearly sometimes erroneous as determined by visual inspection. Furthermore, the SDK’s in-built error detection system (SDK Recognition Quality) was unable to identify these artefacts with sufficient accuracy, and instead falsely reported all joint detections as successful. To address this problem, we developed an alternative error correction technique. Here, we mathematically extracted the acceleration of the joint movement between two frames as recorded by the SDK system. We analysed the transition between the points in terms of a subject’s overall acceleration and averaged these values across the study cohort. We found that if aggregated acceleration over all coordinate axes exceeded a threshold of 0.6 m/frames2, the frames were likely to be an artefact. As such they were flagged as suspected tracking errors (“calibration jump”) and excluded from further analysis.

Using the calibration jump-corrected data, we computed the SMSW test parameters given in Table 1, which were then used for statistical analysis.

All data post-processing and analysis was performed using Matlab 2012A (Mathworks, Ismaning, Germany).

Analysis of measurement quality

We derived several quality parameters and investigated the recognition quality of the central joint hip-centre (Table 2). The frequency of calibration jumps was comparable between HC and MS patients. However, the average jump size was significantly higher in HC than MS patients. There was a strong correlation between walking speed and average jump distance (Pearson’s R = 0.778, p < 0.001), suggesting that the reason for this difference was the higher speed of HC ambulation. Furthermore, the faster people were walking, the shorter were both the observed time (Pearson’s R = -0.967, p < 0.001) and the observed distance (Spearman’s Rho = -0.686, p < 0.001). This was probably caused by the short initialisation period the system needed when a subject walked into the detection range. As a result, MS patients were measured over a minimally longer time period and distance than HC, although the predefined test criteria were identical (Table 2).

Table 2 Quality parameters
Full size table

Analysis of immediate retest reliability

T25FW tests were performed twice and the SMSW three times in a row to assess the retest reliability. T25FW, average speed and SMSW showed excellent retest reliability with ICC > 0.9 in HC and MS. Speed deviation and up/down deviation showed good reliability, whereas reliability for 3D direction deviation and left/right deviation was only moderate (for details, see Table 3). When data was not corrected for calibration jumps, ICC was considerably lower (not shown).

Table 3 Immediate retest reliability
Full size table

Statistical analysis

Immediate retest reliability was analysed using intra-class correlation coefficients (ICC) with a one-sided, random model. Test results between MS patients and HC were compared using t-tests. When equal variances could be assumed (Levene’s test p > 0.05), p-values from independent sample Student’s t-tests are given. When equal variances could not be assumed (Levene’s test p < 0.05), results from the Welch’s t-test are given. The association between T25FW and SMSW measures was investigated using Pearson’s correlation analysis and a Bland-Altman analysis. Correlation with EDSS and functional system scores was investigated with Spearman’s Rho analysis due to the ordinal nature of EDSS data.

Statistical analysis was performed using SPSS version 21 (IBM, Armonk, NY, USA). A p < 0.05 was deemed significant. All tests should be understood as exploratory data analysis as no prior power calculation and subsequent corrections for multiple testing were applied.

Results

Five gait parameters were derived from the hip-centre joint movement during the walk test and compared between HC and MS patients. Average walking speed was the main test parameter, while the four other parameters described gait stability in different dimensions (Table 1). MS patients were significantly slower than HC in average walking speed. In MS patients, 3D direction deviation and left/right deviation were significantly higher than in HC, whereas there was no significant difference between speed deviation and up/down deviation (Table 4 and Figure 2).

Table 4 Comparison between healthy controls and multiple sclerosis patients
Full size table
Figure 2
figure 2

Test outcome differences. Box plots of test outcome measurements (red = MS patients, grey = healthy subjects). Significance levels from t-tests: *** = p < 0.001; ** = p < 0.01; * = p < 0.05.

Full size image

Comparison between SMSW and T25FW

There was a significant but only moderate correlation between T25FW and SMSW average walking speed (Pearson’s R = -0.447, p = 0.042) in MS patients (Figure 3). In a Bland-Altman analysis comparing T25FW with SMSW and including HC and MS patients, the difference between the measures was 0.4 ± 1.3 sec (mean ± SD) (Figure 3). There was no correlation between the test difference and observed time, meaning that these differences could not be explained by a higher estimation error in tests with shorter measurement times.

Figure 3
figure 3

SMSW in comparison with the T25FW. Measuring agreement of gait speed detection of MS patients (red dots) and healthy subjects (grey triangles) with standard T25FW and SMSW. To be able to directly compare SMSW and T25FW in a Bland-Altman analysis, the SMSW time equivalent of walking 25-feet at the detected average speed was calculated. A) Results are plotted against each other. The solid line represents an R2 of 0.392 from a linear regression analysis; the long dashed lines are 95% confidence intervals to the mean. The short-dashed line represents the theoretical test equality (T25FW = SMSW). B) Bland-Altman plot comparison between T25FW and SMSW average speed. B) The difference of the mean is -0.4 sec (solid line). Long dashed lines are 2x standard deviation.

Full size image

Association with EDSS and functional system scores

Both T25FW and SMSW correlated equally well with EDSS total score (Table 5A) and EDSS ambulation score (Table 5C). In the EDSS functional system scores (Table 5B), T25FW correlated mainly with visual function, cerebral function but not with pyramidal function. In contrast, SMSW did not correlate with visual function but with brainstem function, pyramidal and cerebral function (Table 5). Gait stability as expressed by the four deviation parameters correlated to a lesser extent with EDSS (Table 5).

Table 5 Correlation between test outcomes and EDSS
Full size table

Discussion

In this cross-sectional study we investigated the applicability and feasibility of perceptive computing assisted motion analysis in MS patients using the Microsoft Kinect system. Our primary question was whether skeletal tracking data recorded by the device’s SDK program would yield reliable information for gait analysis. Using only the hip-centre joint, we established the test SMSW to analyse MS patients’ gait at maximum walking speed. Additionally, we analysed the SMSW left/right and up/down deviation of MS patients gait in comparison to HC.

Using the SMSW, we were able to detect differences in average walking speed between MS patients and HC. Several gait stability parameters also showed promising results: Especially 3D direction deviation and left/right deviation showed prominent differences between HC and MS patients. The lateral sway of MS patients (described by the parameter left/right deviation) detected by our test suggests that stability impairments are indeed detectable by perceptive computing systems, even over limited observation time and distance (although some types of gait variability may only become evident over longer distances) [31].

The linear distance covered by the recognition area was only 2.2 metres (see Figure 1 and Table 2), and consequently only a few steps of each subject were analysed. Despite this short walking and recording time, the overall detection quality of the target hip-centre joint of the SMSW over time was excellent. Calibration jumps (i.e. tracking errors by the SDK system) presented regularly (see Table 2), however, these could be filtered out by applying a custom-built filter developed during the analysis of this experiment. After filtering out the calibration jumps, the retest reliability of the SMSW average walking speed parameter was excellent and on par with T25FW. The gait stability parameters showed less retest reliability. This was likely due to the fact that most differences between MS patients and HC were only marginally significant. The reliability of the gait stability measurements was superior in MS patients, where a higher signal-to-noise ratio was expected due to weaker gait stability. Additionally, the detection limit was heavily influenced by the joint recognition quality of the sensor. In fact, one measure (3D direction deviation), originally designed to detect changes in walking stability, proved to be very sensitive to calibration jumps in walking direction, which primarily occurred at faster walking speed.

In combination, these data suggest that an assessment employing gait analysis at lower walking speed might improve detection of gait variability and stability in mobility impaired patients. This would not only lower detection errors in the form of calibration jumps but would also allow the system to record several step cycles for a more robust analysis. This is supported by a previous study investigating a Kinect-based system for gait measurements using 3D point cloud information instead of skeletal tracking data, which reported accurate estimation of gait parameters compared to a Vicon motion analysis system in healthy volunteers [32]. A second study used skeletal data from all reported joints to analyse a postural control assessment battery in healthy subjects [23]. Our study adds to these findings and shows that Kinect-based motion analysis is also feasible in MS patients and is able to reliably detect gait differences in comparison to healthy controls. Although the experimental setups and analysis approaches differ, all three studies show that inexpensive 3D perceptual computing can be reliably used to quantify motion impairment. Furthermore, a recent work of Stone and Skubic showed, Kinect can also be installed in the patient’s home as a means of frequently and conveniently assessing gait [28].

We expected that SMSW average speed and the T25FW would yield similar results. However, MS patients’ results sometimes differed strongly in the two tests. Although typical MS symptoms like reduced muscular strength, spasticity or balance deficiency are widely known to fluctuate [33], it is unlikely that random irregularities or gait variability are the key factors in this discrepancy, because retest reliability was high in both tests.

Instead, a different influence of functional impairments on both tests seems possible. SMSW and T25FW correlated well with overall EDSS degree of neurological disability. However, when drilling-down by comparing individual parameter results to the corresponding EDSS functional system scores (FS), we were surprised to find a difference between the SMSW and the T25FW: Whereas SMSW average walking speed showed a moderate correlation to cerebral function, T25FW results correlated with visual function, and SMSW walking speed showed correlation with the pyramidal scores of the EDSS.

Pyramidal impairment such as muscular weakness and spasticity clearly restricts the gait of many MS patients [34, 35]. However, as stated above, we found no correlation between the T25FW and the pyramidal function score of the EDSS. This is in discrepancy with previous findings, such as those of Phan-Ba et al. [36], and likely explained by the only mild pyramidal impairment of many patients in our study (41% patients with FS ≤ 2 ). In contrast to T25FW, SMSW correlated with pyramidal functional system scores. This might be explained by findings from previous studies where MS patients, performing the T25FW, accelerated walking speed after a relatively slow start [36, 37]. Because muscular rigidity is triggered by postural changes [38], such as starting from a static position to walk, we may have detected a starting weakness of spastic patients. Thus, a patient, who begins moving more slowly, might show a slower walking time at SMSW average walking speed compared to T25FW. In the T25FW a slower speed at start might level over the longer distance. In this regard SMSW average walking speed may be more sensitive to impaired initiation of movement in MS patients than the T25FW. We did not evaluate spasticity with a validated assessment, so we were not able to further pursue the question of whether the SMSW could be a marker for minor pyramidal weakness.

An exploratory pilot study of this type has some limitations. Only Kinect-derived skeletal tracking information generated by the SDK was used [19]. Although not within the scope of the present study, it is conceivable that exclusive or hybrid inclusion of raw sensor data might yield additional or better results. Furthermore, we neither selected MS patients with specific and well-established motor symptoms or gait disability, nor did we compare SMSW test results to objective gait measures other than T25FW. The analysis between SMSW, T25FW and EDSS functional system scores should be treated as preliminary and the results should be interpreted with care. However, the demonstrated correlations fit well within the expected clinical framework and the results of previous studies. As such, our results can be expected to serve as a basis for confirmatory studies. Most importantly, the ability of SMSW measurements to detect changes longitudinally and the retest reliability of the test needs to be validated in a follow-up study.

Conclusion

Perceptive computing-based detection of ambulation speed via the joint hip-centre was feasible and reliable. SMSW average walking speed was a valid parameter as demonstrated by retest reliability results and the strong correlation with established clinical scores, such as the T25FW and EDSS. A notable difference was the slower walking speed of MS patients measured using SMSW (mean distance: 2.2 metres) compared to T25FW (mean distance: 7.6 metres). The gait stability parameters showed greater left/right deviation in MS patients compared to HC. The analysis approach presented here shows promise as an objective technique for detecting and assessing of gait pattern and as a simple and affordable tool in the clinician’s toolbox. Finally, this study provides evidence that further investigation of gait with periodic Kinect-measurements may give new insight into the disease progression of MS.