Brain–computer interfaces for human gait restoration

Abstract

In this review article, we present more than a decade of our work on the development of brain–computer interface (BCI) systems for the restoration of walking following neurological injuries such as spinal cord injury (SCI) or stroke. Most of this work has been in the domain of non-invasive electroencephalogram-based BCIs, including interfacing our system with a virtual reality environment and physical prostheses. Real-time online tests are presented to demonstrate the ability of able-bodied subjects as well as those with SCI to purposefully operate our BCI system. Extensions of this work are also presented and include the development of a portable low-cost BCI suitable for at-home use, our ongoing efforts to develop a fully implantable BCI for the restoration of walking and leg sensation after SCI, and our novel BCI-based therapy for stroke rehabilitation.

Introduction

Neurological conditions, such as spinal cord injury (SCI), stroke, or traumatic brain injury (TBI), can compromise gait function and leg sensation. These deficits have a profoundly negative impact on the independence and quality of life of the affected populations. In the USA alone, the primary and secondary healthcare costs of SCI and stroke are estimated at $90B/year [1, 2], which represent a significant public health burden. There are currently limited options to restore motor and sensory functions after SCI, and up to 60% of stroke survivors have long-term gait deficits despite spontaneous recovery and intense physiotherapy [3, 4]. Therefore, novel approaches to the restoration of gait function and leg sensation after SCI and stroke are in high demand.

Brain–computer interfaces (BCIs) represent one such novel approach. Generally, BCIs are systems that record neural correlates of users’ intentions, decode these signals in real time, and generate control commands for external end-effectors, such as computer applications, wheelchairs, robotic prostheses/orthoses, and muscle stimulators [5]. BCIs enable those with paralysis to bypass the site of neurological injury and assume direct brain control of external devices (see Fig. 1). Principally, BCIs can be classified as invasive or noninvasive, depending on the way they acquire brain signals. An overwhelming majority of noninvasive BCIs rely on scalp-recorded electroencephalogram (EEG). Invasive BCIs primarily utilize action and local field potentials recorded by intracortically implanted microelectrode arrays [6, 7]. These high-fidelity brain signals enable a BCI performance that is superior to those achieved by EEG-based BCIs. They also require brain surgeries, which carry non-negligible health risks, and the longevity of these brain implants is limited [8].

Fig. 1

Elementary BCI framework. A person with paralysis due to SCI generates movement intentions whose neural correlates are decoded real-time and used to control an external end-effector. Typical feedback is visual, although other sensory modalities can be employed

Early BCI applications targeted the most severe forms of paralysis, such as amyotrophic lateral sclerosis (ALS) or brainstem stroke. For example, the earliest proof-of-concept BCIs enabled people with ALS to communicate with their environment [9,10,11]. Similarly, those with severe tetraplegia due to cervical SCI [12] or brainstem stroke [13] used invasive BCIs to control a variety of devices including robotic arm prostheses. However, there were relatively few attempts to develop BCIs for leg paralysis (paraplegia) due to conditions such as mid-thoracic or lumbar SCI, or subcortical stroke. Motivated by this knowledge gap, we developed several BCI systems to address lower extremity paralysis and this review summarizes more than a decade of our work in this area. In Sect. 2, we describe our approach to designing a non-invasive BCI system for the restoration of walking after SCI. Subsequently, we integrated this BCI with a virtual reality environment (VRE) and tested its performance first in a population of able-bodied individuals and then in those with SCI. These results are summarized in Sects. 3 and 4, respectively. Once we verified its function in a VRE, we integrated our BCI system with physical prostheses, first with a robotic gait orthosis for treadmill walking (see Sect. 5) and subsequently with a functional electrical stimulation (FES) system for free-overground walking (see Sect. 6). We provide conclusions and future directions in Sect. 7.

Noninvasive BCIs for restoration of walking after SCI

Wheelchair mobility remains the primary mode of ambulation for individuals with paraplegia due to SCI. However, prolonged wheelchair use and sedentary lifestyle associated with SCI lead to a number of comorbid conditions [14,15,16], which contribute to the majority of SCI-related healthcare costs. These problems have inspired the pursuit of novel approaches to the restoration of walking after SCI. Examples include cell-based therapies, which have shown promise in preclinical studies [17], followed by clinical trials designed to ascertain their safety [18, 19]. Another example is a neuromodulation-based approach, whereby electrical stimulation is delivered to the spinal cord, below the lesion. This method enabled those with motor-complete SCI to regain volitional leg movements [20, 21]. While orthogonal to cell-based and neuromodulation-based therapies, BCIs are inherently complementary to these approaches due to their unique ability to provide real-time access and interpretation of brain signals. Therefore, BCIs will continue to play an important role in the restoration of walking after SCI, either as a stand-alone or adjunct technology.

We first sought to test the feasibility of developing an EEG-based BCI to control walking [22]. In doing so, we established the following design criteria: (1) intuitive control, (2) short training time and (3) robust operation. To achieve intuitive control, subjects with SCI initiated walking by either attempting to walk or creating the kinesthetic motor imagery (KMI) of walking, i.e., a mental rehearsal of walking without any overt movement execution [23]. To cease walking, they simply relaxed. This control strategy intuitively matches the task at hand and is in contrast to other BCI approaches that often involve the KMI of unrelated body parts, such as fist pumping or tongue protrusion. Short training time and robust control were achieved by employing a data-driven subjects-specific decoder design. This approach generally permitted BCI-naive subjects or those with very little prior BCI experience to assume purposeful control of an end-effector after a 15-min training/calibration procedure. Other BCI approaches typically require a significantly more extensive training time [24, 25]. Our approach also allowed a fixed decoder to be used over time despite the non-stationary nature of EEG signals.

At the core of our BCI system is a state decoder designed as a binary-state machine (see Fig. 2). Its transitions are driven by a probabilistic input, $P(W\,\vline \,f^\star )$, which represents the posterior probability of Walk state given the observed EEG feature $f^\star$. Note that $P(I\,\vline \,f^\star )=1-P(W\,\vline \,f^\star )$, so the knowledge of $P(W\,\vline \,f^\star )$ is sufficient for defining state transitions. This probability is compared to fixed thresholds, $T_I$ and $T_W$, which are found through a calibration process. Since $T_I<T_W$, the binary-state machine (BSM) exhibits a hysteretic behavior. Such an asymmetric threshold structure also minimizes the mental fatigue of BCI participants. For example, to transition from Idle to Walk state, the input $P(W\,\vline \,f^\star )$ must exceed $T_W$. However, to remain in Walk state, it is sufficient for $P(W\,\vline \,f^\star )$ to remain above $T_I$, which is typically lower than $T_W$ and requires much less mental effort. Likewise, to transition from Walk to Idle state, $P(W\,\vline \,f^\star )$ must be brought below $T_I$; however, to remain in Idle state, it is sufficient for the posterior probability to remain below $T_W$.

Fig. 2

a The state transition diagram of the BCI decoder with a probabilistic input, $P(W\,\vline \,f^\star )$. b The state-input space of the system with the transition thresholds $T_I$ and $T_W$ ($T_I<T_W$). When $T_W>0.5$ and $T_I<0.5$, these transition rules are consistent with Bayes factors [26]

Another challenge in the design of our BCI decoder is the small sample size problem [27]. Namely, obtaining accurate posterior probability estimates $P(W\,\vline \,d^\star )\propto p(d^\star \,\vline \,W)P(W)$ from observed EEG data $d^\star \in \mathbb {R}^{N_c\times N_t}$ is difficult given the high number of EEG channels, $N_c$, and the high sampling rate. For example, a typical 750-ms, 64-channel EEG data segment sampled at 256 Hz ($N_t=192$), would result in a $\sim$ 12,288-dimensional data. In the face of limited number of data segments, the likelihood function estimate $p(d\,\vline \,W)$ is typically meaningless and so low-dimensional features $f=\varphi (d)$ must be pursued. In the simplest form, feature extraction maps, $\varphi \,:\, \mathbb {R}^{N_c\times N_t}\mapsto \mathbb {R}^m$, are linear. Using information-theoretic arguments [28], we designed a piecewise linear feature extraction map that maximizes the separability of features f under different class assignment [29, 30]. For binary class problems (e.g., Fig. 2), this method often yields optimal results in one-dimensional (1D) feature space ($m=1$), which is consistent with the feature dimension of the theoretical Bayes classifier [31]. Since EEG signals often exhibit rhythmic behavior, our analysis is performed in the frequency domain, i.e., $d\in \mathbb {R}^{N_c\times N_b}$, where $N_b$ is the number of frequency bands and the elements of d are the power spectra.

Self-paced BCI control of ambulation in a virtual reality environment

We recruited eight able-bodied subjects and one individual with mid-thoracic SCI to participate in this study. The participants were fitted with an actively-shielded, 63-channel EEG cap connected to an amplifier array. They were seated in front of a computer screen which showed textual cues prompting them to alternate between relaxing (Idle state) and KMI of walking (Walk state). During this procedure, their EEG data were recorded (sampling rate 256 Hz), labeled by the state information, and stored on a computer for subsequent analysis. This training data collection lasted 10 min and included 10 alternating Idle and Walk epochs, each lasting 30 s. A more detailed description of these procedures can be found in [22].

From each Idle and Walk epoch, we extracted 5 non-overlapping 4 s-long trials of EEG data. This created a training database of 100 trials (50 trials in each state). These trials were then fast Fourier transformed (FFT) and their power spectra were integrated in 2 Hz bins centered at $1, 3,\ldots ,39$ Hz. Note these frequencies were chosen since most of the motor-related EEG signals are confined to $\leqslant 40$ Hz frequency band. This transformation resulted in spatio-spectral data $d\in \mathbb {R}^{63\times 20}$, which, at 50 trials per class, create a severe small sample size problem. Therefore, the data were vectorized and subjected to a combination of dimensionality reduction via classwise principal component analysis (CPCA) [29] and feature extraction via approximate information discriminant analysis (AIDA) [30]:

$$\begin{aligned} f = T_{\text {A}}{\varPhi} _{\text {C}}(d), \end{aligned}$$

(1)

where ${\varPhi} _\text {C}$ is a piecewise linear CPCA mapping and $T_{\text {A}}$ is an AIDA feature extraction matrix. Through this procedure, we extracted 1D features, f, which enabled the accurate estimate of the likelihood $p(f\,\vline \,W)$ as a univariate Gaussian function and in turn the posterior probability $P(W\,\vline \,f)\propto p(f\,\vline \,W)P(W)$.

Subsequently, we found the BSM transition thresholds, $T_I$ and $T_W$ (Fig. 2), through the following calibration procedure. The subjects were instructed through verbal cues to alternate between short epochs of relaxing and walking KMI for a total of $\sim$ 2 min. During this period, their EEG data were acquired in real time and the most recent 750-ms-long data segment was transformed into the frequency domain as explained above. Subsequently, we extracted 1D features $f^\star$ using Eq. (1) and calculated the posterior probability $P(W\,\vline \,f^\star )$ using the Bayes rule. Ideally, the samples of $P(W\,\vline \,f^\star \in I)$ should cluster around 0 and those of $P(W\,\vline \,f^\star \in W)$ should cluster around 1. To account for noise, we chose $T_I = \text {median}\{P(W\,\vline \,f^\star \in I)\}$ and $T_W = \text {median}\{P(W\,\vline \,f^\star \in W)\}$. To further smooth the state transitions, the calibration procedure often required averaging the posterior probabilities $P(W\,\vline \,f^\star )$ across two or three consecutive 750-ms data segments. Another option is to smooth the posterior probabilities through a recursive Bayesian update [32]:

$$\begin{aligned} P(W\vert f_k^\star ) = \frac{p(f_k^\star \,\vline \,W) P(W\,\vline \,f_{k-1}^\star )}{p(f_k^\star )}, \end{aligned}$$

where $f_{k-1}^\star$ and $f_{k}^\star$ are the observed EEG features across two consecutive data segments. The feature extraction transformations, the parameters of the likelihood function, and the thresholds $T_I$ and $T_W$ were then saved for each subject. This concluded the design of data-driven subject-specific BCI decoders.

We quantified the BCI system’s performance through real-time online tests consisting of walking in a VRE. Within the VRE, each subject was assigned a BCI-controlled avatar that had to be walked in a straight line and stopped by each of the 10 non-player characters (NPCs) for at least 2 s (see Fig. 3). The goal of the game was to finish the course as quickly as possible while making all 10 stops. Once the game started, the subject’s EEG data were recorded in real time with a refresh rate of 0.5 s (limited by the computer processing speed). From the most recent 750-ms data segment, the posterior probabilities $P(W\,\vline \,f^\star )$ were estimated as above using the subjects-specific parameters. To walk, the subjects had to use the KMI of walking and to dwell by each NPC, they had to relax. The state transitions were controlled by the subject-specific BSM with transition thresholds determined through the above calibration procedure. Each subject performed the walking task 5 times, with all the tasks completed in a single day. A video showing an example of a real-time BCI test can be found here: https://www.youtube.com/watch?v=GXmovT3BxEo. Figure 4 shows the space-time plot of a representative test for one of the able-bodied subjects.

Fig. 3

Reprinted with permission from [22]. Copyright IOP Publishing

A BCI-controlled avatar within a VRE, operated in a third-person view. The subject uses walking KMI to move the avatar from one NPC to another at a constant speed. The subject relaxes to dwell by each NPC for $\geqslant$ 2 s.

Fig. 4

Reprinted with permission from [22]. Copyright IOP Publishing

The space-time plot of a representative online walking test for an able-bodied subject A8. The pink areas mark designated stopping zones (the location of the NPC ± one body length tolerance). Orange segments mark false starts. The game finishes after the avatar walks past the last stopping zone. There were no false stops in this particular test.

Table 1 shows the real-time online performances of all study participants, averaged over 5 walking tests. The performances are quantified using the completion time and the number of stops made. Dwelling for less than 2 s at each NPC’s location incurred a partial stop score. Note that these two performance metrics are inherently traded off, as shorter completion time can be achieved at the expense of not making designated stops. Monte Carlo simulations showed that all but one of the 45 total online tests performed by the subjects were statistically significant, i.e., they could not be achieved by uniformly drawing $P(W\,\vline \,f^\star )$ between 0 and 1. The performance achieved with a manually controlled joystick is also shown for reference. For a more detailed presentation of results, the reader is referred to [22].

Table 1 Average BCI-VRE performance for 8 able-bodied subjects (A1–A8) and one subject with paraplegia due to SCI (S1). The average performance with a manually controlled (MC) joystick is also shown

In summary, this study demonstrated that a non-invasive EEG-based BCI could be used to restore walking in a VRE. These results were achieved while adhering to the principles of intuitive control, short training time, and robust operation. Intuitive control was achieved by subjects utilizing the KMI of walking and relaxation to respectively initiate virtual walking and standing. All subjects achieved purposeful control of the BCI after undergoing a 10-min training data collection and 2-min calibration. This is in contrast to similar BCI-VRE studies [24, 25], which respectively required 3–5 and 4 months of training. Robust operation was achieved by utilizing a data-driven, subject-specific approach to the decoder design. This approach allowed the BCI walking tasks to be repeated over time without making adjustments for the non-stationarity of EEG data due to factors, such as impedance drifts, sweat formation, and human–computer co-adaptation [33]. Our study also demonstrated that some subjects (e.g., A8 and A2) were only marginally inferior to manually controlled joystick in terms of completion time (Table 1) and several participants achieved competitive stop scores, e.g., $\geqslant 9.0$. Especially encouraging is the performance of subject S1, who despite being paralyzed for 11 years, was able to achieve purposeful BCI control of walking. These results warranted an extension of this study to a population of individuals with SCI.

BCI control of ambulation in a virtual reality environment after SCI

Encouraged by the results from Sect. 3, we sought to test the performance of our BCI-VRE system in a population of people with SCI. We also wanted to verify whether purposeful control could be retained over multiple days. To this end, we recruited 5 individuals with SCI and tested their ability to operate our BCI system over 5 non-consecutive days. Four participants had paraplegia due to thoracic SCI and one (S3) had tetraplegia due to syringomyelia in the cervical spine. All were considered to be in the chronic phase of SCI with post-injury times ranging from 1 year (S5) to 14 years (S3) and ages ranging from 21 (S5) to 59 (S4). A more detailed demographic description of the participants can be found in [34].

The experimental procedures were identical to those described in Sect. 3, except that they were repeated over 5 non-consecutive days. On each experimental day, the subjects underwent training EEG data collection (10 min), followed by a decoder design, including a 2-min calibration procedure, and between 2 and 8 real-time online BCI-VRE walking tests. These tests were identical to those in the previous section (see also Fig. 3). The total number of real-time online walking tests on a per-subject basis ranged from 19 to 29. Subject S2 attained purposeful BCI control on the second experimental day, with all the other participants being able to do so on the very first day.

A representative video of a real-time online BCI-VRE test for subject S3 can be found at https://www.youtube.com/watch?v=K4Frq9pwAz8. Table 2 shows a more detailed performance breakdown including the total number of tests performed by each subject, their best experimental day, and the level of SCI injury and paralysis type. Since these performances are characterized by the completion time and stops made, which are opposing criteria, determining the best day’s performance included combining these measures into a single composite score. The performances and experimental days could then be ordered based on the composite score. To ascertain the statistical significance of these results, we performed Monte Carlo simulations with subject-specific BSMs and the posterior probabilities drawn uniformly between 0 and 1. Across 124 real-time online BCI-VRE tests performed by all participants, only 5 tests had performances that could have been achieved by Monte Carlo simulations, with the majority of these non-significant performances attributed to S1. For an in-depth presentation of the results, the reader is referred to [34].

Table 2 Average BCI-VRE performance for five participants with SCI (S1–S5)

These results demonstrate that a population of people with SCI could learn to operate an intuitive, KMI-based BCI within a relatively short time period. All subjects attained purposeful BCI control, as ascertained by statistical tests, on the very first experimental day, except for subject S2, who did it on the second day. These times are still significantly shorter than those reported in related BCI-VRE studies [24, 25]. Our data-driven, subject-specific approach to the BCI decoder design yielded a robust performance in the face of EEG signal non-stationarity. Anecdotally, we tested these decoders across multiple days, and while they still yielded purposeful control, the performances were better when a new decoder was designed for each experimental day. Given the relatively short decoder training and calibration process, this did not impose additional burden on the participants and/or experimenters. All subjects (except S4) achieved their best performance on the fourth or fifth experimental day, suggesting that the performance may improve over time due to human-computer co-adaptation [33]. Other factors such as the motor imagery induced reactivation of the previously dormant brain areas [35] may also have played a role. Especially encouraging are the performances of subjects S3 and S5, who on their best day achieved the perfect stop score, while being marginally slower than the manually controlled joystick. It should also be noted that subject S3 was the most severely paralyzed and could not use his arms or breathe on his own. Such as high level of control achieved by a diverse population of people with SCI indicates that BCI-controlled lower extremity prostheses for gait rehabilitation or restoration may be feasible. Our BCI-VRE system may also serve as a training platform for such prostheses if/when they become available.

BCI control of robotic gait orthosis

After extensively testing our BCI-VRE system in able-bodied population (Sect. 3) and people with SCI (Sect. 4), we explored whether it was possible to interface this system with a physical prosthesis [36]. To this end, we integrated our EEG-based BCI with a commercial robotic gait orthosis (RoGO) system (see Fig. 5). This system consists of a RoGO, a support harness, a treadmill, and a computer control module. When in motion, both the RoGO and treadmill move in a synchronized manner, i.e., with the step rate that is proportional to the treadmill speed. The system can accommodate different body habitus and provide different levels of body-weight support, which may be needed when the system is used by those with SCI. While conceptually similar, there are several important differences between this system and our BCI-VRE for walking. First, the BCI-RoGO system is operated in an up-right position. The same goes for the training and calibration processes. Second, walking on a treadmill causes vibrations and movement artifacts that can easily be picked up by EEG. In general, these artifacts must be appropriately dealt with. Finally, feedback in the BCI-RoGO system is no longer visual and it involves a complex combination of tactile, proprioceptive, and auditory sensations.

Fig. 5

Reprinted from [36] under the Creative Commons License Attribution

A subject wearing an EEG cap suspended in the RoGO via a harness. Surface EMG electrodes and a gyroscope are respectively measuring the muscle activity and movement of the left leg. When movement intentions are decoded from EEG, the BCI sets the RoGO and the treadmill in motion, which provides feedback to the subject.

We recruited one able-bodied participant (Subject 1) and one participant with mid-thoracic T6 SCI (Subject 2) to participate in this study. They, respectively, had 5 and 3 hours of prior BCI experience, mostly by participating in our BCI-VRE experiments. They were fitted with an actively shielded 63-channel EEG cap and placed in the RoGO. Both had a gyroscope mounted above their left ankle to measure leg trajectories. Subject 1 also had three electromyogram (EMG) channels to measure the activity of the left quadriceps (thigh), tibialis anterior (shin) and gastrocnemius (calf) muscles. These EMG signals were measured in the able-bodied subject to rule out BCI control by voluntary leg movements. Both subjects then completed 10 min of training data collection, by alternating between epochs of relaxation and the KMI of walking. The decoder was then designed from these training data using a similar procedure to those presented in the previous sections. Subsequently, a 5-min calibration was performed, where EEG data were acquired every 0.25 s, while the subjects were instructed verbally to alternate between idling and the KMI of walking. We then calculated the posterior probabilities, $P(W\,\vline \,f^\star )$, from the most recent 750-ms-long EEG data segment and determined the state transition thresholds as explained in the previous sections. A more detailed account of these procedures can be found in [36].

A real-time online walking test consisted of subjects being prompted by a computer screen to alternate between 1-min-long epochs of idling and walking for a total of 5 min. By alternating between relaxation and the KMI of walking they, respectively, stopped/moved the RoGO. Their EEG data were recorded every 0.25 s and the posterior probabilities were computed in real time from the most recent 750-ms data segment. This resulted in the state of the BSM (Fig. 2) being updated at a rate of 4 Hz. Therefore, to complete the whole 5-min task correctly, 1200 EEG data segments must be correctly decoded in succession. A video showing a representative example of a real-time online walking test can be found at https://www.youtube.com/watch?v=HXNCwonhjG8. Both subjects completed 5 such tests during a single experimental day. Figure 6 shows the time course of representative walking tests for Subject 1 and Subject 2. We subsequently analyzed these online walking tests to assess the performance. Specifically, we calculated the normalized cross-correlation between the instructional cues and BCI-RoGO walking, as determined from the gyroscope data. We also calculated the rate of omissions (failure to activate BCI-RoGO walking during Walk cues) and false alarms (initiation of BCI-RoGO walking during Idle cues). Finally, we performed Monte Carlo simulations to establish the empirical p-values and ascertain the statistical significance of the achieved performances.

Fig. 6

Reprinted from [36] under the Creative Commons License Attribution

Representative real-time online walking tests for Subject 1 (able-bodied) and Subject 2 (SCI), showing epochs of idling and BCI-RoGO walking. The walking epochs (green blocks) were estimated from the gyroscope data. The system has a built-in startup/stopping cycles, during which the treadmill speed and RoGO stepping rates are gradually increased/decreased. The red line shows the states decoded by the BCI (thin-idle, thick-walk), while the blue line (thin-idle, thick-walk) marks the instructional cues. Corresponding EMG (gold: quadriceps; teal: tibialis anterior; purple: gastrocnemius) are also shown.

Table 3 summarizes these results across five real-time online BCI-RoGO walking tests. In all ten tests, the achieved performances were statistically significant, i.e., they could not have been achieved a the random sampling of posterior probabilities. Both subjects achieved cross-correlations in excess of 0.8 at the average lag of $\sim$7 s. Note that most of this lag is imposed by the RoGO system’s built-in stopping cycle, over which the treadmill velocity is gradually brought to 0. A small source of additional lag is the averaging of posterior probabilities across multiple 750-ms segments, although, as mentioned in Sect. 3, this smoothing procedure reduces state misclassifications. Indeed, both subjects had no omissions and had on average < 1 false alarm per 5-min test. The average duration of these false alarms was $\sim$ 7 s and was largely dominated by the RoGO system’s stopping cycle. The examples in Fig. 5 show no false alarms for Subject 1 and two false alarms for Subject 2. Analysis of EMG data for Subject 1 showed that there was no significant EMG activity in either muscle group prior to the transition from Idle to Walk state. This confirms that the able-bodied subject did not use volitional leg movement to initiate the transition to Walk state. Additionally, the levels of EMG activity during the BCI-RoGO walking tests were consistent with passive leg movements, indicating that BCI-RoGO was not controlled by the subject actively moving his legs. A detailed discussion of these and related points can be found in [36].

Table 3 BCI-RoGO performances for the two participants, averaged over five real-time online walking tests

Finally, we turn our attention to the information-theoretic feature extraction maps defined by Eq. (1). While solely data-driven, these maps have an intuitive physiological interpretation. Namely, for each frequency bin, the coefficients of the piecewise linear CPCA-AIDA transformation can be mapped onto the EEG electrodes, interpolated, and visualized as brain topographies. Figure 7 shows an example of these maps for Subject 2, which had the most salient coefficients at the 10–12 Hz frequency bin. This is consistent with the loss of EEG power in the $\mu$-frequency band (8–12 Hz) that tends to happen with the imagination or execution of movements [37]. Spatially, the areas of importance appear to overlay the leg and arm sensorimotor representation areas, which is consistent with prior studies [22, 34]. For example, the involvement of the leg and bilateral arm areas has been reported in our BCI-VRE experiments and is likely associated with the imagery of leg movements and arm swings.

Fig. 7

Modified from [36] under the Creative Commons License Attribution

The CPCA-AIDA feature extraction maps at the 10–12 Hz frequency bin for Subject 2. The piecewise linear nature of the map is represented as two images (one for each state). Topographic areas with values close to + 1 or $-1$ are the most important for distinguishing between Idle and Walk states at this frequency.

In summary, these results demonstrate that purposeful BCI control of leg prostheses for the restoration of walking is feasible. It is particularly notable that both subjects had very little prior BCI experience, yet they attained highly accurate BCI control of the RoGO system on their first attempt. Our BCI-RoGO system retained the intuitiveness, short training, and robust performance—the design criteria established with our BCI-VRE system. Moreover, the performances achieved in this study were superior to those achieved with our BCI-VRE system. We hypothesize that this gain in performance could be attributed to a more natural up-right operation of the BCI-RoGO system and a more biomimetic nature of feedback consisting of a combination of tactile, proprioceptive, and auditory sensation. To the best of our knowledge, this was the first-ever demonstration of a person with SCI using a BCI to operate a leg prosthesis. It also represents an important intermediate step toward the restoration of free overground walking using neurorestoration technologies.

BCI control of free overground walking

Following up on our BCI-VRE and BCI-RoGO studies [22, 34, 36], we explored the possibility of integrating our EEG-based BCI system with a prosthesis for free overground walking [38]. Specifically, we chose a U.S. Food and Drug Administration (FDA)-approved FES system (Parastep, Sigmedics, Fairborn, OH) which achieves ambulation by electrically activating the quadriceps and tibialis anterior muscles in a coordinated manner [39]. Coupled with the user’s anterior-lateral shifting maneuvers, this procedure allows those with complete paraplegia to walk overground. The Parastep system is equipped with a front-wheel walker for stability and additional support (see Fig. 8). The standing function and individual steps are normally controlled manually by pressing the corresponding buttons. To enable BCI control of the system, we used a microcontroller unit (MCU) and digital relays to interface with the “left step”, “right step”, and “stand” buttons of the Parastep system.

Fig. 8

Modified from [38] under the Creative Commons License Attribution

Integrated BCI-Parastep system for free overground walking. The subject is placed in a safety harness of a body-weight support system (ZeroG, Aretech, Ashburn, VA) for added safety. EEG amplifiers are placed in a backpack and EEG data are sent wirelessly (via Bluetooth) to the BCI computer, which processes the data in real-time and wirelessly sends the BCI commands to a microcontroller (placed in the belt-pack). A laser distance meter (placed on the rail of ZeroG) and two gyroscopes (placed above the ankles) are added to measure the subject’s position and leg trajectories.

We recruited a single subject with paraplegia due to SCI (T6, 6 years post-injury), who had no motor/sensory function below the level of injury. He underwent a battery of screening procedures to rule out severe spasticity, osteoporosis, lower extremity fractures, pressure ulcers, orthostatic hypotension, contractures, and restricted range of motion. We also confirmed that he had an adequate neuromuscular response to FES. Subsequently, the participant began a BCI practice using our BCI-VRE system (see Sect. 4). He simultaneously started FES training in order to recondition his leg muscles for weight bearing and overground walking, and to improve his cardiopulmonary endurance. The FES training continued until the subject could walk the length of the 3.66-m course (12 ft). This procedure was also used to determine the optimal parameters for the Parastep system, such as the step rate and stimulation amplitude. For safety purposes and partial body-weight support, he was placed in the ZeroG body-weight support system (Fig. 8).

After $\sim$ 12 weeks of BCI practice and FES training, we began testing the subject’s ability to walk using the BCI system. The walking tests entailed the subject receiving verbal cues to start and stop walking, while his EEG data were recorded every 0.25 s, analyzed in real time using the procedures similar to those described in the previous sections, and decoded to issue the corresponding command to the Parastep system. To facilitate free overground walking, we untethered the subject from the BCI system by using a wireless communication protocol (Bluetooth) for the transfer of EEG data and Parastep control commands. We had to reduce the number of EEG channels from 63 to 24 to accommodate the wireless transfer bandwidth limit. This reduction in the number of channels did not adversely affect the performance as 24 channels still provided a good coverage of motor cortical areas. The walking task began with the subject standing still for $\sim$20 s, followed by walking to the mid-point of the course (marked by a traffic cone), standing there for $\sim$ 20 s, finishing the course, and standing there for another $\sim$ 20 s. A video showing a representative example of one real-time online walking test can be found here: https://www.youtube.com/watch?v=RBAeR-Z0EHg. The participant performed between one and six such overground walking tests per visit. Over the course of 19 weeks, he had 11 visits during which he performed a total of 30 walking tests. Figure 9 shows data from a representative walking test.

Fig. 9

Reprinted from [38] under the Creative Commons License Attribution

Graphical representation of a BCI-controlled overground walking test. The beginning and end of yellow blocks mark the onset of Walk and Idle verbal cues, respectively. Red blocks represent periods when the BCI system decoded Walks state; otherwise, the system is in Idle state. Green and blue blocks mark leg movements recorded by the gyroscopes. The laser signal (blue trace) depicts the participant’s position over time, as measured by the laser distance meter. The beginning, mid-point, and end of the course are marked by traffic cones.

The subject’s performances during overground walking tests were characterized based on the video, BCI state, and gyroscope data. Specifically, we calculated the normalized cross-correlation between verbal cues (recorded by the camera) and BCI-FES mediated responses (recorded by the gyroscope), as well as the lag at which the maximum cross-correlation occurs. Similar to our BCI-RoGO study (Sect. 5), the statistical significance of these cross-correlations was established by comparing them to those achieved through Monte Carlo simulations. To make our study comparable to related BCI studies, we expressed these performances using an information transfer rate (ITR). The ITR (bit/s) was calculated as

$$\begin{aligned} \text {ITR} = \mathcal {B}\mathcal {I}(D,T), \end{aligned}$$

(2)

where $\mathcal {B}$ is the number of decisions per unit of time (4/s in this study) and $\mathcal {I}(D,T):=H(D)-H(D\,\vline \,T)$ is the mutual information between the true state of the nature T (determined from the cue) and the decoded state D. The entropy H(D) and conditional entropy $H(D\,\vline \,T)$ can be found from the four terms of the confusion matrix, viz. the probability of: correctly decoding Idle state, correctly decoding Walk state, omission (incorrectly decoding Idle state), and false alarm (incorrectly decoding Walk state). Further derivation of this formula can be found in [38].

Table 4 shows the average performance of the subject across 30 overground walking tests performed during 11 visits, as well as the performance during the best session (recorded on the 9th visit). The achieved cross-correlations were similar to those reported in Sect. 5 with a significantly shorter lag. Monte Carlo simulations confirmed that in all 30 tests, the achieved cross-correlations were statistically significant. Similarly, there were no omissions and the subject had $\sim$2 false alarms per test. Notably, the false alarm rate decreased toward the end of the study, presumably due to learning. Finally, the average ITR of 2.3 bit/s is comparable to the values recorded with our EEG-based BCI for communication [40]. Finally, the analysis of the peak performance indicates that a very high level of control can be achieved with this system. Specifically, the cross-correlations as high as 0.987 are achievable with no omissions or false alarms. In addition, ITRs as high as 3.7 bit/s were observed, which is significantly superior to 1 bit/s—the hypothesized upper ITR bound of EEG-based BCIs [5, 41].

Table 4 Cross-correlation, $\rho$, between verbal cues and gyroscope movement, ITR, number of false alarms, and false alarm rate for the 30 overground walking tests performed

This study represents the first demonstration of a person with paraplegia due to SCI purposefully operating a noninvasive BCI-FES system for overground walking. The participant achieved highly accurate control of the system and maintained this level of performance during a 19-week period. The subject was also able to lightly converse during these experiments without interfering with the function of the system. These results provide a proof-of-concept for direct brain control of the leg prosthesis for the restoration of elementary overground walking to those with leg paralysis. Further studies are necessary to test the performance of this system in a population of individuals with SCI.

Conclusion and future directions

In this article, we summarized more than a decade of our work on the development of EEG-based BCIs for the restoration of walking after SCI. Our design process was guided by the principles of intuitiveness, short training time, and robustness. We first integrated our BCI system with a VRE and tested the ability of able-bodied individuals to virtually walk in this environment [22]. We then repeated this study with a cohort of people with SCI [34]. We demonstrated that participants in these studies could master the BCI-mediated virtual walking with very little or no prior BCI experience, in a relatively short period of time (often on the very first try). We attribute these results to using a combination of intuitive control strategy based on KMI and subject-specific data-driven decoders. We subsequently integrated our BCI system with physical prostheses, such as gait orthosis and FES for walking, and respectively, tested their performances in treadmill [36] and free overground walking [38] conditions. At the time, these studies represented first-of-a-kind demonstrations of individuals with paraplegia using their brain signals to regain elementary walking functions. Similar studies involving a larger number of subjects with paralysis due to SCI were subsequently reported by other groups [42], who also observed partial neurological improvements in response to long-term BCI use.

There are several potential extensions and improvements of our work. For example, conventional EEG-based BCIs, including the one presented in the previous sections, are generally expensive, complex to operate, require a long set-up time, and lack portability. These factors limit their use to laboratory settings. These problems can be mitigated using a fewer number of EEG electrodes [43] and custom-designing EEG amplifiers for portability and compactness [44]. Additional savings can be achieved by replacing the BCI computer with a dedicated embedded processing system [45, 46]. Motivated by these shortcomings, we developed a portable low-cost BCI platform and compared its performance to that of our conventional BCI system [47]. Specifically, we designed a custom 4-channel amplifier array and integrated it with an open-source MCU and a touchscreen. When blind-tested in a population of able-bodied subjects, the real-time online decoding performance of the low-cost system was statistically indistinguishable from that of a conventional BCI. To ascertain its clinical utility, our future work will be directed toward testing the function of this system in a population of individuals with SCI or other neurological conditions.

While we demonstrated that highly accurate control is possible with EEG-based BCIs, a real-world deployment of these systems may require much higher levels of performance. For example, decoding errors, such as omissions or false alarms, may have serious implications for the safety of BCI users. Achieving error rates that are within an acceptable safety margin may not be feasible given the limited spatio-temporal resolution of EEG signals and their susceptibility to biological and non-biological artifacts. A potential solution to this problem is to employ invasively recorded brain signals. Having access to surgically implanted electrodes may also provide a conduit for delivering biomimetic feedback by means of cortical electrostimulation. We are currently in the process of developing a fully implantable BCI for the restoration of walking and leg sensation in people with paraplegia due to SCI. For this purpose, we intend to use subdurally recorded electrocorticogram (ECoG). In a simplified form, ECoG signals can be viewed as a high spatio-temporal resolution instantiation of EEG [48]. They are also largely immune to artifacts. ECoG is typically recorded by subdurally placed electrode grids, which while invasive, do not penetrate the brain tissue and have been shown to remain stable for up to 9 years [49,50,51]. ECoG grids are also FDA-approved for cortical electrostimulation, which when delivered over the sensory cortex, can elicit artificial sensation [52, 53]. Our preliminary results with subjects implanted with ECoG grids show distinct patterns of walking-induced activity over the leg motor cortex that extend well into the $\gamma$ (40–160 Hz) frequency band [54]. Our offline analysis demonstrated that the state information could be decoded from ECoG data with an unprecedented accuracy of 99.8%. Additionally, using a Bayesian filter approach, we achieved an average correlation coefficient between the decoded and true step rates of 0.934 [55]. We are currently in the process of developing custom integrated circuits (ICs) for ECoG recording [56], wireless data transmission [57], and cortical electrostimulation [58]. These ICs were designed to conform to the stringent requirements of fully implantable devices in terms of size and power consumption. Also, we are developing novel methods for the suppression of stimulation artifacts [59] in order to accommodate simultaneous recording and cortical electrostimulation. Finally, we recently developed a computational framework to estimate the thermal impact of the various components of our implantable BCI system [60] and calculate the power budget of underlying ICs. A proof-of-concept demonstration of an ECoG-based BCI for the restoration of walking after SCI has recently been reported by others [61]. Earlier studies have reported on ECoG-based BCIs for the restoration of arm movements in those with tetraplegia [62]. While neither of these systems were fully implantable and provided no biomimetic feedback, their results are encouraging and warrant further development of ECoG-based BCIs, including fully implantable ones.

Finally, the most important benefit of EEG-based BCIs for lower extremities may be in the realm of neurorehabilitation, where such systems would be used to develop novel therapies for motor rehabilitation of individuals with incomplete paralysis due to conditions such as stroke, TBI, or incomplete SCI. For example, by interfacing our EEG-based BCI with an FES placed over the tibialis anterior muscle, a co-activation of upper motor neurons (via attempted movements) and lower motor neurons (via electrical stimulation) may be achieved. This near-simultaneous activation of two neuronal populations has been hypothesized to strengthen the synaptic coupling between them [63]—a mechanism known as Hebbian learning. This could potentially lead to rehabilitation of the residual motor and sensory functions beyond what is achievable with standard therapies. We tested this concept in a small population of chronic stroke survivors with foot drop [64], which is a major contributor to gait impairments after stroke [65, 66]. After 12 hours of BCI-FES therapy, we found that 5 out of 9 subjects demonstrated a detectable increase in their gait speed and six-minute walk distance. Furthermore, these gains were retained 4 weeks post therapy. Encouraged by these results, we are now in the process of conducting a randomized controlled trial [67] to formally ascertain the effectiveness of this therapy.