1 Introduction

Functional recovery and regain of independence following neurological injury are commonly assessed via clinical scores, comprising capacity and time measures, and subjective questionnaires [11]. Objective information on arm use in the home environment could provide a valuable complement to insights gained from clinical assessments.

Wearable sensor technology has enabled unobtrusive monitoring of arm movements in the natural environment, with accelerometry representing the most established approach. Activity counts (AC) derived from the acceleration signals provide quantitative information about arm activity, such as total duration [27, 28] and intensity of movements [5, 23]. However, AC provide only information about the amount of arm use but not about functional arm use. For example, accelerometry in wrist-worn applications is sensitive to any kind of movements, e.g., passive arm swinging during ambulatory activities, which leads to an overestimation of real-world arm activity [1, 12]. Ambulatory activities can be detected by means of accelerometry [18, 33], and upper extremity activity can be adjusted accordingly [22], but this requires additional, temporally synchronized sensors which may negatively impact patient compliance. To reject the influence of ambulatory activities, the ratio of arm use between paretic and non-paretic arm is commonly used [2, 26, 28–30]. However, ratios are a relative measure of intensity or duration of use and therefore are not suitable to capture potential changes in absolute activity or use.

Whereas clinical tests assess function and impairment, accelerometry measures only the effects of impairment in the form of reduced activity levels or reduced duration of arm use, but does not consider motion patterns. In contrast to accelerometers only, inertial measurement units (IMUs) comprising accelerometers and gyroscopes (angular rate sensors) can be used to precisely measure and monitor movements of body segments and have been proposed for portable motion capture of gait [6, 13] or arm movements [35, 36]. The inclusion of gyroscopes is especially beneficial as they allow to measure angular displacement during highly dynamic activities. Inertial sensors worn in a clinical setting during assessments such as the Functional Ability Scale of the Wolf Motor Function Test [7, 10, 19, 20] or the Fugl-Meyer Assessment [3], have allowed the reconstruction of these scores based on the recorded signals, demonstrating that qualitative information of arm movements can also be extracted. Still, there is a need for a metric motivated by the characteristics of typical object grasping and manipulation movements, complementing the quantitative measures of accelerometers with measures of functional arm use. Single-sensor setups are preferable because there is no risk of swapping sensors and no need for sophisticated synchronization, and compliance in patients as well as clinicians is increased.

We propose a novel method to qualitatively assess functional arm use in the home environment, relying on only a single wrist-worn sensor module. To evaluate this method, we monitored 10 subacute/chronic stroke survivors in their home environment during 48 h using five inertial measurement units (IMUs) placed on wrists, shanks and waist. We compare our method against setups involving sensors worn at multiple anatomical locations and investigate its ability to reject influence of ambulatory activities on arm use. Performance measures are compared with clinical measures for gross manual dexterity.

2 Methods

2.1 Rationale

The goal of this work is to assess functional arm use, as in the case of reaching to and manipulating an object, with a single wrist-worn IMU. IMUs are subject to drift, especially in the horizontal plane, and measurement of absolute wrist orientation can therefore not be guaranteed. The use of magnetometers could add an absolute reference in the horizontal plane and reduce drift, but these are “difficult to use in the vicinity of ferromagnetic metals” [16], which are often encountered in daily settings (e.g., doors, elevators, speakers). Robust measurement of wrist orientation and position relative to the trunk during ADL would require absolute tracking as provided by magnetic sensing [8, 24]. However, when considering only short time windows of sensor recordings, the effects of drift can be neglected, and angular displacements can be measured reliably.

Reaching and manipulation movements performed during ADL typically involve grasping objects placed on a table or shelf and displacing them by moving the arm over a certain angle in the horizontal or vertical plane, or a combination of the two. During walking, in return, the forearm is mostly oriented toward the ground and swings passively. It has been shown that, during ADL, wrist position is mostly constrained around the sagittal plane [8] and above the waist [31], and thus absolute orientation with respect to the trunk, which could be influenced by sensor drift in the horizontal plane, is not essential. We assume that forearm elevation (the angle between the forearm axis and a horizontal plane) and/or yaw (the angle between the forearm axis and the sagittal plane) change significantly during functional arm use. Based on these assumptions, we propose to infer functional arm use by measuring the relative angular change in forearm orientation induced by such movements.

To derive a metric (subsequently called gross arm movements, GM) based on these assumptions, thresholds for forearm elevation and lateral movement have to be determined. By observing reaching and manipulation tasks we defined a threshold of \(30^\circ\) for angular change in elevation and/or yaw in order to identify such movements (Fig. 2). Further, by constraining the elevation of the forearm to a range of \(-30^\circ\) to \(+30^\circ\), the influence of pro- and supination movements of the forearm (which we assume is not indicative of functional movements) on the proposed metric can be reduced. This is an issue as pronosupination movements at forearm postures around \(\pm 90^\circ\) are indistinguishable from lateral movement with the available sensor information. Furthermore, this guarantees that the analysis of arm movements is not affected by singularities (where subsequent behavior can not be predicted) occurring at \({\pm }90^\circ\) of forearm elevation. Additionally, by excluding forearm elevation lower than \(-30^\circ\) from the analysis, the influence of ambulatory activities can be reduced.

Stroke survivors show reduced activity of the paretic arm compared to the non-paretic arm in ADL as shown by reduced AC on the paretic side [12] and we assume that this discrepancy is also reflected in a reduction of GM performed with the paretic arm during ADL. Also, lifting the affected arm against gravity is impaired by paresis in stroke survivors, and we therefore hypothesize that forearm elevation of left and right arm differ in stroke survivors, and that average forearm elevation is lower compared to the unaffected arm. We further hypothesize that the proposed metric describes meaningful arm function as assessed through the Box and Block Test (BBT) and is robust against the influence of ambulatory activities.

2.2 Subjects

Ten subacute/chronic stroke survivors were enrolled in this pilot study after providing written informed consent. Persons with inability to give informed consent were excluded from the study. The study was approved by the local ethics commission. Subject demographics are shown in Table 1. All subjects were independent walkers. Subjects’ gross manual dexterity was assessed with the BBT [21] by a trained clinician either at their homes or during a therapy session in the clinic.

Subjects wore five wearable sensor modules for 48 h, one on the dorsal side of each wrist, one on the lateral side of each shank right above the ankle and one around the waist. Sensors were donned by a clinician either at their home or during a therapy session in the clinic. The modules at wrist and shanks were fixed with silicone straps, while the waist sensor was fixed with an elastic band. Subjects were instructed to wear all sensor modules during all activities and to keep them donned for bathing or showering in order to eliminate the risk of swapping the modules or placing them at a different location. The sensors were returned to the clinic by surface mail. Data from the waist sensor were excluded from the analysis.

Table 1 Demographics and Box and Block Test (BBT) scores of the 10 subacute/chronic stroke survivors who participated in this study
Full size table

2.3 Sensors

For the purpose of precisely recording human posture and motion data over a period of 48 h we used ReSense, a low-power 10-degrees-of-freedom (DOF) inertial measurement unit (IMU) comprising a 3-axis accelerometer, a 3-axis gyroscope, a 3-axis magnetometer and a barometric pressure sensor [14]. This module can record continuously for at least 24 h at a sampling rate of 50 Hz, respectively up to 48 h through intelligent power management. Multiple modules can be synchronized temporally via a base station prior to the measurement and the calibrated clock in each unit guarantees <90 ms drift per day between modules. The system (Fig. 1) was specifically designed for use by clinical staff and self-use by patients.

Fig. 1
figure 1

ReSense sensor modules and base station [14]. The base station allows data readout, battery charging and temporal synchronization of up to 5 sensor modules simultaneously via a USB 2 port

Full size image

2.4 Data processing

All data processing was performed in MATLAB (2014a, The MathWorks, Natick MA, USA).

2.4.1 Preprocessing

Raw data from all sensors were resampled at 50 Hz and gaps due to sensor standby during inactive time periods were filled by means of zero-order hold interpolation for acceleration. Angular rate was set to zero during these periods. Resampling was necessary because of the inaccuracy of the sampling rate of the low-cost MEMS sensors, which, according to our experience, may vary by up to 2 %. Residual zero offset and drift of the gyroscope data was then corrected linearly interpolating averaged signals from still phases. These phases were identified by means of a moving variance filter. Data from barometer and magnetometer were excluded from the data analysis.

2.4.2 Walking identification

Identification of walking phases was based on acceleration signals from the shank-worn sensor module at the non-paretic leg [15]. Data streams were segmented into windows (data segments) of 7.5 s duration (corresponding to 375 datapoints) and fed into a support vector machine classifier previously trained on data from 12 stroke survivors [15]. The output of the classifier was further processed by morphological filters which filled gaps (non-walking) of \(\le\)4 windows (30 s) and removed small walking bouts of only 1 window (7.5 s).

2.4.3 Manual labeling

Manual labeling of corrupted/faulty segments and nightly sleeping phases was performed by visual inspection of the raw data of wrist and shank sensors and the identified walking phases. Faulty segments could occur when subjects did not wear their sensor modules for some time or when they accidentally swapped or inverted them, despite being clearly instructed not to do so. These periods were manually identified and excluded from the analysis. Nightly sleeping phases were defined manually from the time point where activity marginally dropped, after the last walking bout, and mostly static signals were visible until the time point where activity increased and the first walking bout could be identified. Active phases during nightly bed time were labeled as sleep phases even when they clearly included walking. Inactive periods during daytime were only labeled as sleep in case the duration exceeded one hour.

2.4.4 Activity counts

Acceleration from the wrist-worn sensors was high-pass-filtered at 0.3Hz in order to reduce the influence of gravity on the final score. AC for epochs with window size of 1 min (3000 samples) and no overlap were calculated by means of Eq. 1 adapted from [9].

$$\begin{aligned} AC = \frac{1}{f_\mathrm{s} \cdot \mathrm{Epoch}}\sum _{n=1}^{N} \sqrt{a_x[n]^2+a_y[n]^2+a_z[n]^2} \end{aligned}$$
(1)

where \(N = f_s \cdot \mathrm{Epoch} \cdot 60\); Epoch, duration of an epoch in minutes; \(f_\mathrm{s}\), sampling frequency in Hz; \(a_{x/y/z}\), acceleration in direction x y z.

A threshold of 0.05 g was applied on the acceleration magnitude to suppress very low sensor excitation mainly due to sensor noise.

AC were calculated during awake time including and excluding walking phases. To exclude walking phases from AC, the acceleration of wrist-worn sensors was set to zero during the periods where walking was detected by means of the shank sensor. The final outcome is the average number of counts per minute during awake time. When excluding walking phases from AC, the average was calculated over the duration of awake time minus walking time. Additionally, also the duration of arm use was calculated by thresholding the accelerometric data, as proposed by Uswatte et al. [27].

2.4.5 Forearm elevation

Acceleration and angular rate from wrist-worn sensors were used to estimate the forearm orientation relative to the earth referential frame. For this purpose the gradient descent orientation filter proposed by Madgwick et al. [17] was selected, which fuses sensor measurements of gravity and angular rate into an optimal orientation estimate and also assures convergence from initial conditions and compensates for eventual drift in a vertical plane (the weighting of the accelerometer measurements in the error correction was set to \(\beta = 0.03\)). The filter outputs orientation in a quaternion representation \({\mathbf {q}}=[q_0,~q_1,~q_2,~q_3]\), which can be transformed into a \(3\times 3\) direction cosine matrix \({\mathbf {R}}\) [34]. To calculate the forearm elevation (Fig. 2), the forearm vector \(\mathbf {a_s} = [1~0~0]^\top\) was expressed in the earth fixed referential e: \(\mathbf {a_e}={\mathbf {R}}^\top \mathbf {a_s} = {\mathbf {R}}^\top [1~0~0]^\top\). The elevation \(\Theta\) of the forearm vector \(\mathbf {a_e} = [a_{ex}~a_{ey}~a_{ez}]\) can then be computed as:

$$\begin{aligned} \Theta = \mathrm{arctan} \left( \frac{a_{ez}}{\sqrt{a_{ex}^2+a_{ey}^2}} \right) \end{aligned}$$
(2)

The normalized probability distribution of forearm elevation \(\Theta\) during daily routines was established in a polar representation between \(-90^\circ\) and \(90^\circ\) with a histogram bin-size of \(1^\circ\) (Fig. 3). Phases of inactivity were detected by means of a threshold of 0.05 g on the acceleration magnitude and were excluded from the probability distribution. Probability distributions were established for both forearms during awake time, including and excluding walking phases. The first moments (mean) were extracted from the distributions and plotted against BBT scores to examine its value as a performance metrics.

Fig. 2
figure 2

Illustration of the forearm angles extracted from the IMU. Elevation is the angle between the horizontal and the forearm axis and yaw is the angle covered in lateral movements. The red area in the vertical plane illustrates the region where gross arm movements are identified and movements outside of this area are not captured by the algorithm. Note that shoulder abduction, flexion and rotation as well as elbow flexion can influence elevation and yaw

Full size image

2.4.6 Gross arm movement identification

GM identification is based on the orientation estimate determined for forearm elevation. In addition to elevation \(\Theta\), the rotation around the vertical axis in the earth referential (collinear with gravity), i.e., the yaw, was calculated and also considered (Fig. 2). Yaw (\(\Phi\)) was computed by numerically integrating the angular rate \(\omega _{ez}\) over time:

$$\begin{aligned} \Phi [k]=\frac{1}{f_{\rm s}}\sum _{n=1}^{k}\omega _{ez}[n] \end{aligned}$$
(3)

where \(\omega _{\mathbf {e}}=[\omega _{ex},~\omega _{ey},~\omega _{ez}]={\mathbf {R}}^\top \omega _{\mathbf {s}}\) \(f_\mathrm{s}\) is the sampling frequency in Hz, \(k=[1 \dots N]\) where N is the total number of samples to be integrated.

Elevation and yaw were then segmented into windows of 2 s with 75 % overlap, to increase the probability of capturing movements within a window that match the defined heuristic rules. In each of the windows, the following heuristic rules were tested

$$\begin{aligned} \left| \Theta \right| \le 30^\circ ~\mathrm{and}~ \mathrm{range}(\Theta )+\mathrm{range}(\Phi ) \ge 30^\circ \end{aligned}$$
(4)

where \(\mathrm{range}(x) = \mathrm{max}(x)-\mathrm{min}(x)\) and true was assigned for valid conditions, false otherwise.

Put simply, arm movement is counted as GM if the sum of a change of forearm orientation in yaw and elevation is more than \(30^\circ\) within a time period of 2 s, but only if the movement occurs within a range of forearm elevation between \(-30^\circ\) and \(+30^\circ\). Angular ranges were defined based on the assumptions listed in the Rationale section. The final output was the total amount of windows labeled true multiplied with the difference of window size minus window overlap, thus resulting in the total duration of GM during the recording period.

2.4.7 Statistical analysis

Pearson’s correlation coefficients were calculated for the BBT of the paretic arm with: the difference of forearm elevation probability distribution’s mean between paretic and non-paretic arm (excluding walking), AC of the paretic arm (including and excluding walking), GM of the paretic arm (including and excluding walking), and duration of paretic arm use (including and excluding walking). Correlation coefficients for the ratio of the BBT paretic/non-paretic arm were calculated with: the ratio of AC paretic/non-paretic arm (including and excluding walking), the ratio of GM paretic/non-paretic arm (including and excluding walking), and the ratio of arm use duration paretic/non-paretic arm (including and excluding walking). Correlations of AC and GM and arm use duration were compared with the Steiger’s Z-test for dependent correlations [25] in order to identify significant differences in correlation strength.

3 Results

Forearm elevation distributions of two representative subjects are depicted in Fig. 3. The distributions of paretic and non-paretic forearm elevation differed in a subject with a low BBT score of 6, with the mean of the paretic forearm displaying a lower elevation during activity. In return, these were similar for a subject with a high BBT score of 65. During walking, in average 64 % \((\pm 22.7\,\%)\) of movements of the impaired arm were measured below \(-30^\circ\) (average mean \(-38.2^\circ ~\pm 14.1^\circ\)), and removing walking phases from the recordings resulted in less activity with an elevation lower than \(-30^\circ\) and thus raised the mean. Similar results were found in all 10 subjects. Pearson’s correlation coefficients of BBT score of the paretic arm correlated with the difference of means (paretic–nonparetic) of ten chronic stroke survivors is \(r=0.68\) (\(p=0.03\)) (Fig. 3).

Fig. 3
figure 3

Top: Polar representation of the probability distribution of the forearm elevation of subject 2 (top, BBT score of 6) and subject 7 (bottom, BBT score of 65) during activity of paretic (red) and non-paretic (blue) arm, including (left) and excluding (right) walking phases. The bold circles mark the first moment (mean) of the distributions. \(90^\circ\) means the forearm points upward against gravity, while \(-90^\circ\) means the forearm points downward along gravity. The forearm angle is independent of upper arm orientation. Bottom: Scatter plot of the difference of forearm elevation probability distribution’s mean between paretic and non-paretic arm (excluding walking phases) with the Box and Block Test, showing a high correlation (color figure online)

Full size image
Fig. 4
figure 4

Scatter plots of activity counts (AC) and the Box and Block Test (blue squares) and gross movements and the Box and Block Test (red dots). Absolute counts on paretic arm including walking (top left) and excluding walking (top right) and ratios between paretic and non-paretic arms including walking phases (bottom left) and excluding walking phases (bottom right). The lines represent the total least squares fit. The correlations of AC and gross movements (GM) were compared using the Steiger’s Z-test. Data have been normalized for display purposes in the plots in the top half (color figure online)

Full size image

Correlations of arm AC and arm GM with the BBT are shown in Fig. 4. Correlation of total paretic arm AC including walking with BBT was \(r=0.69\) (\(p=0.029\)) and total paretic arm AC excluding walking with BBT was \(r=0.93\) (\(p<0.001\)). Steiger’s Z-test on the two latter correlations was \({\bar{Z}}=2.99\), and this value, when compared to the normal curve rejection points of \(\pm 1.96\), is significant. Correlation of the ratio of paretic/non-paretic arm AC including walking with ratio of the BBT was \(r=0.49\) (\(p=0.155\)) and the ratio of paretic/non-paretic paretic arm AC excluding walking with the ratio of the BBT was \(r=0.84\) (\(p=0.002\)). Steiger’s Z-test again showed a significant difference (\({\bar{Z}}=2.95\)). GM correlated significantly better with BBT than AC with BBT when walking phases are included. This applies to the ratios (\({\bar{Z}}=-4.82\)) and absolute values (\({\bar{Z}}=-2.58\)). Correlation coefficients increased in the case of AC with BBT when walking phases are excluded and no significant difference compared to GM with BBT could be identified using Steiger’s Z-test. In all four tested conditions, the correlation of GM with BBT resulted in high correlation coefficients \(r\ge 0.90\) (\(p<0.001\)) and no significant difference could be identified by means of the Z-test. Duration of paretic arm use including walking correlated with BBT was \(r=0.77\) (\(p=0.009\)) and the correlation differed significantly from GM correlated with BBT (\({\bar{Z}}=-2.51\)). The three remaining conditions of arm use duration correlated well with the BBT (\(r\ge 0.90\); \(p<0.001\)) and there was no significant difference between the correlation coefficients in comparison to GM.

4 Discussion

AC are an established means for assessing daily arm use but are sensitive to ambulatory activity. Ambulatory activities can lead to overestimation of AC and consequently weaker correspondence with clinical scores, as already reported previously [2, 30]. In our study, absolute AC of the paretic arm during awake time correlated significantly better with a clinical score of arm function when walking phases, detected by a shank-worn sensor on the unaffected side, were excluded (\(r=0.93\)) compared to when they were not excluded (\(r=0.69\)) (Fig. 4). In contrast, our results demonstrate that measuring GM is robust against arm movements during ambulatory activities, as shown by the strong correlation with the BBT (\(r=0.95\)). Arm movements during ambulation mostly occur below \(-30^\circ\), and oscillations in the forearm elevation due to walking are typically below the amplitude threshold we have set, and therefore these arm movements remain undetected by the GM method. Further, we observed that using activity ratios between the paretic and non-paretic arms can only partially reject the influence of ambulatory activities.

Previous research suggested to quantify arm use by thresholding accelerometric data [27] as an outcome measure which can be robust against ambulatory activities [28]. Indeed, the ratios correlated well with the BBT (\(r=0.95\)), but this approach, however, rejects intensity of arm use and can therefore not be considered a measure of functional arm use. Also, in a single-sensor setup, GM correlated significantly more strongly with BBT (\(r=0.95\)) than duration of impaired arm use (\(r=0.77\)).

While GM correlated strongly with the BBT score, the latter is sensitive to both arm and hand function (e.g., difficulty in grasping a block), whereas our proposed method only assesses arm use. Nevertheless, similar correlations could also be found with other clinical tests such as the Chedoke Arm and Hand Activity Inventory (\(r\ge 0.91\); \(p<0.001\)).

The strong correlation of the proposed metric with the BBT suggests that it may provide valuable insights into the true use and performance of the paretic arm. In comparison with the BBT administered in a clinical environment and at few discrete time points, the proposed method is a cost-effective tool to continuously track recovery. The ability of this tool to capture intra- and inter-day-variability might additionally increase the overall reliability of these assessments, and provide additional insights on the recovery process. It has to be noted, however, that the proposed single-sensor method neither allows to directly measure reaching distance/workspace, nor to isolate the impairments resulting in reduced GM score. In order to obtain such information, additional inertial sensors would need to be placed on each segment of the arm as well as on the trunk to allow reconstruction of arm kinematics. Such a setup, however, would likely negatively impact usability and compliance

Comparing the forearm elevation distributions including and excluding walking phases illustrates that, in case of a paretic arm with a low BBT score, arm swing during walking may constitute a substantial part of captured overall activity. This motivated the rejection of movements with an elevation of less than \(-30^\circ\). This may have biased activity levels in previous studies [1, 4, 26, 30]. The difference of means of the paretic and non-paretic forearm elevation distribution correlated well with the BBT, suggesting that this measure could be a clinically relevant parameter of arm impairment. These findings are in accordance with measurements of vertical wrist position [32], where stroke survivors showed a reduced amount of activity with their affected wrist when located above mid-trunk level.

Results show that forearm elevation, which we assume to be linked to arm function, can be monitored during daily life by means of wrist-worn 6-DOF IMUs. By using gyroscopes, the precise orientation of a sensor module could be reconstructed, even during fast movements, and also changes in orientation in the horizontal plane could be captured, which is impossible with accelerometers only.

5 Conclusions

We conclude that GM measured with one 6-DOF IMU worn at the paretic wrist qualifies as assessment of functional arm use in real life. The proposed method is sensitive to absolute changes in arm activity and is robust against overestimation of passive arm movements, e.g., by ambulation and has the advantage of relying on a single-sensor unit as opposed to accelerometers on both arms to calculate ratios, or on additional sensors on the shank to exclude walking episodes. A single-sensor setup is highly desirable as it may improve user compliance and ease usability. We identified that patients removed and accidentally swapped sensor modules even though they were instructed not to do so. Such a risk could be minimized in a single-sensor setup. In order to consolidate our findings further validation needs to be performed with a larger sample size. Nevertheless, the proposed method promises more detailed and qualitative insights into functional arm use in the home environment beyond what is possible with pure accelerometry.