Intracortical Brain–Machine Interfaces

Abstract

A brain–machine interface, or BMI, directly connects the brain to the external world, bypassing damaged biological pathways. It replaces the impaired parts of the nervous system with hardware and software that translate a user’s internal motor commands into action. In this chapter, we will discuss the four basic components of an intracortical BMI: an intracortical neural recording, a decoding algorithm, an output device, and sensory feedback. In Sect. 5.2 we will discuss intracortical signals, the electrodes used to record them, and where in the brain to record them. The salient features of the neural signal useful for control are extracted with a decoding algorithm. This algorithm translates the neural signal into an intended action which is executed by an output device, such as a robotic limb, the person’s own muscles, or a computer interface. In Sect. 5.3 we will discuss classification decoders and how they can be implemented in a BMI for communication. In Sect. 5.4 we will discuss continuous decoders for moment-by-moment control of a computer cursor or robotic arm. In Sect. 5.5, we will discuss a BMI that controls electrical stimulation to directly activate a patient’s own paralyzed muscles and reanimate their arm. Finally, in Sect. 5.6, we will discuss ongoing work toward expanding sensory feedback with the goal of making intracortical BMIs a clinically viable option for treating paralysis, as well as other research trends.

Homework

• 1. Consider designing a BMI to classify movement to the right or left, and we want to test how well it works with one neuron. If the BMI user intends to move right, the neuron’s firing rate is drawn from a Gaussian distribution with mean μ_right = 8 spikes/second and standard deviation σ _right = 5 spikes/second. If the BMI user intends to move left, the neuron’s firing rate is drawn from a Gaussian distribution with mean μ_left = 12 spikes/second and standard deviation σ _left = 6 spikes/second.

  • (a) Suppose we make one measurement of the firing rate, y, and we assume the prior probability of “left” and “right” are equal. For each of the cases below, would we classify “left” or “right”?

    y (Spikes/second)	2	5	8	11	14	17
    Classification

  • (b) Suppose we now assume that the BMI user moves “left” twice as often as “right” (i.e., P(left) = 2/3 and P(right) = 1/3. For each of the cases below, would we classify “left” or “right”?

    y (Spikes/second)	2	5	8	11	14	17
    Classification

• 2. In Sect. 5.3 we showed how to implement a classifier with Gaussian firing statistics, where the neural activity for class k is modeled as y∼N(μ _k, Σ_k), where μ _k ∈ R ^d and Σ_k ∈ R ^d × d are the mean and covariance of the activity of a population of d neurons. Here we will assume that the covariance matrix is the same for each class k = 1, …, K (i.e., Σ₁ = Σ₂ = … = Σ_K).

  • (a) First, suppose we have a new recording of neural activity, y. Also, suppose that P(c _k) = π _k. Using Bayes’rule, find logP(c _k| y), up to the normalizing constant.

  • (b) Find the decision boundary used for determining whether the point y came from class j or class k, and simplify the expression.

  • (c) Is the decision boundary linear?

• 3. In this problem we will derive the equations to implement a classifier based on Poisson spike counts. The spike count of neuron i given class k is Poisson-distributed with parameter λ_ik. We will assume that the D neurons, y ₁, …, y _D are conditionally independent given the class j. In other words, given neural activity y ∈ R^D, the probability that y came from class k is as follows:

  $$ P\left(\boldsymbol{y}|{c}_k\right)={\prod}_{\mathrm{i}=1}^DP\left({y}_i|{c}_k\right),\kern0.5em \mathrm{where} $$
  $$ P\left({y}_i|{c}_k\right)=\exp \left(-{\uplambda}_{ik}\right){\uplambda}_{ik}^{y_i}/{y}_i! $$

  • (a) Let P(c _k) = π _k. Find P(c _k| y) using Bayes rule.

  Now simplify the expression above by taking the log: log P(c _k | y).

  • (b) Given a new point y, we want to determine to which class this point belongs. Derive the decision boundary that determines whether we classify a new point y as belonging to either class j or class k. Use the expression that you derived in part a.

  • (c) Is the decision boundary linear?

• 4. In Sect. 5.3 we provided the following expressions for the training phase of a classifier: ${\boldsymbol{\upmu}}_k=\frac{1}{N}{\sum}_{i=1}^N{\boldsymbol{y}}_i$ (Eq. 5.9) and ${\Sigma}_k=\frac{1}{N}{\sum}_{i=1}^N\left({\boldsymbol{y}}_i-{\boldsymbol{\upmu}}_k\right){\left({\boldsymbol{y}}_i-{\boldsymbol{\upmu}}_k\right)}^{\top }$(Eq. 5.10), where y _i ∈ R ^d for all i = 1, …, N is the neural activity recorded with class k, μ _k ∈ R ^d, and Σ^k ∈ R ^d × d. Show that these values of μ _k and Σ_k maximize the following equation for the likelihood:

  $${\renewcommand\theequation{\normalsize\thechapter.\arabic{equation}} {\begin{array}{lll}& L\left({\boldsymbol{\mu}}_k,{\Sigma}_k\ |{\boldsymbol{y}}_1,\dots, {\boldsymbol{y}}_{N,}{c}_k\right) =P\left({\boldsymbol{y}}_1,\dots, {\boldsymbol{y}}_N\ |{c}_k\right)\end{array}}}$$
  $$\begin{array}{ll}&={\prod}_{i=1}^N{\left(2\pi \right)}^{-d/2}{\left|{\Sigma}_k\right|}^{-1/2}\\&\exp \left(-\frac{1}{2}{\left({\boldsymbol{y}}_i-{\boldsymbol{\mu}}_k\right)}^{\top }{\Sigma_k}^{-1}\left({\boldsymbol{y}}_i-{\boldsymbol{\mu}}_k\right)\right)\end{array}$$

• 5. In Sect. 5.4, we considered a two-neuron example of the PVA decoder where the neurons had orthogonal preferred directions (e.g., one neuron preferred 90°, while the other neuron preferred 180°). Show that if the two neurons do not have orthogonal tuning directions, the directions decoded by PVA will be biased.

• 6. Show that a neuron that exhibits cosine tuning also shows linear tuning to velocity. That is, suppose that given a reach in the θ direction with speed s, a neuron’s firing rate can be written as $y={b}_0+ ms\ \cos \left(\theta -{\theta}_{\overrightarrow{p}}\right)$, where b ₀ is the neuron’s baseline firing rate, m is its modulation depth, and ${\theta}_{\overrightarrow{p}}$ is the neuron’s preferred direction. Show that this means we can also write y = b₀ + b ^Tv, where b and v are both 2D vectors.

• 7. Derive the expressions for the training phase of the Kalman filter in Sect. 5.4:

  • $B=\left(\sum_{t=1}^T{\mathbf{y}}_t{\mathbf{x}}_t^T\right){\left(\sum_{t=1}^T{\mathbf{x}}_t{\mathbf{x}}_t^T\right)}^{-1}$

  • $\Sigma\!\!\!\!=\!\!\!\!\frac{1}{T}\sum_{t=1}^T\left({\mathbf{y}}_t-B{\mathbf{x}}_t\right){\left({\mathbf{y}}_t-B{\mathbf{x}}_t\right)}^T$ (Note that here we use the B found above.)

  • $A=\left(\sum_{t=2}^T{\boldsymbol{x}}_t{\boldsymbol{x}}_{t-1}^T\right){\left(\sum_{t=2}^T{\boldsymbol{x}}_{t-1}{\boldsymbol{x}}_{t-1}^T\right)}^{-1}$

  • $Q\!=\!\frac{1}{T-1}\sum_{t=2}^T\left({\boldsymbol{x}}_t-A{\boldsymbol{x}}_{t-1}\right){\left({\boldsymbol{x}}_t\!-\!A{\boldsymbol{x}}_{t-1}\right)}^T$ (Note that here we use the A found above.)

• 8. Consider using a BMI to play Pong with one neuron. That is, we will use a Kalman filter to decode position along a one-dimensional axis from the firing rate of a single neuron. Let the state model be x _t = x _t − 1 + ω_t, ω_t∼N(0, q) and the observation model be y _t = bx _t + ε_t, ε_t∼N(0, σ).

  • (a) Show that the estimate of the position on time step t, μ _t, can be written in the form

  $$ {\mu}_t=\left(1-\alpha \right){\mu}_{t-1}+\alpha \left(\frac{{\mathrm{y}}_t}{\mathrm{b}}\right) $$

  • (b) Prove that 0 ≤ α ≤ 1.

  • (c) When does α approach 0? Under this case, why does it make sense for μ _t = μ _t − 1?

  • (d) When does α approach 1? Under this case, why does it make sense for μ _t = y _t/b?

• 9. You decide to speed up the implementation of your Kalman filter by skipping the one-step prediction. Whereas normally you would solve the measurement update (Eq. 5.17) and one-step predictions iteratively on each time step (Eq. 5.16).

  • One-step prediction: $P\left({\boldsymbol{x}}_t|{\left\{\boldsymbol{y}\right\}}_1^{t-1}\right)=\int P\left({\boldsymbol{x}}_t|{\boldsymbol{x}}_{t-1}\right)P\left({\boldsymbol{x}}_{t-1}|{\left\{\mathbf{y}\right\}}_1^{t-1}\right){d\boldsymbol{x}}_{t-1.}$

  • Measurement update: $P\left({\boldsymbol{x}}_t|{\left\{\boldsymbol{y}\right\}}_1^t\right)=\frac{P\left({\boldsymbol{y}}_t|{\boldsymbol{x}}_t\right)P\left({\boldsymbol{x}}_t|{\left\{\boldsymbol{y}\right\}}_1^{t-1}\right)}{P\left({\boldsymbol{y}}_t|{\left\{\boldsymbol{y}\right\}}_1^{t-1}\right)}$

  You instead decide to just iterate the measurement update step, by directly plugging in the velocity estimate from the previous time step, P $\left({\boldsymbol{x}}_{t-1}|{\left\{\boldsymbol{y}\right\}}_1^{t-1}\right)$, without making a one-step prediction:

  $$ P\left({\boldsymbol{x}}_t|{\left\{\boldsymbol{y}\right\}}_1^t\right)=\frac{P\left({\boldsymbol{y}}_t|{\boldsymbol{x}}_t\right)P\left({\boldsymbol{x}}_{t-1}|{\left\{\boldsymbol{y}\right\}}_1^{t-1}\right)}{P\left({\boldsymbol{y}}_t|{\left\{\boldsymbol{y}\right\}}_1^{t-1}\right)} $$

  Describe qualitatively what will happen to the velocity estimate over time.

  (Hint: when in doubt, try simulating it or solving the 1D case.)

• 10. The goal of the measurement update of the Kalman filter is to find P(x _t| y ₁, …, y _t). To do so, we adopted the strategy in Sect. 5.4 whereby we would first find the joint distribution P(x _t, y _t| y ₁, …, y _t − 1), and then use the theorem of conditioning for jointly Gaussian random variables to find P(x _t| y ₁, …, y _t). Here we will derive the means and covariances of the joint distribution

  $$\left[\begin{array}{l}\boldsymbol{y}_t\mid \boldsymbol{y}_1,\dots, \boldsymbol{y}_{t-1} \\ \boldsymbol{x}_t\mid \boldsymbol{y}_1,\dots, \boldsymbol{y}_{t-1}\end{array}\right] \sim \mathrm{N}\left(\left[\genfrac{}{}{0pt}{}{B\boldsymbol{\mu}_{\mathrm{t}}^{-}}{\boldsymbol{\mu}_{\mathrm{t}}^{-}}\right],\left[\genfrac{}{}{0pt}{}{B{\varPhi}_t^{-}{B}^{\mathrm{T}}+\Sigma}{\varPhi_t^{-}{B}^{\mathrm{T}}}\genfrac{}{}{0pt}{}{\ B{\varPhi}_t^{-}}{\ {\varPhi}_t^{-}}\right]\right)$$

  • (a) Find the mean of $\boldsymbol{y}_{t}\mid \boldsymbol{y}_{1}, \ldots, \boldsymbol{y}_{t-1}$.

  • (b) Find the variance of $\boldsymbol{y}_{t}\mid \boldsymbol{y}_{1}, \ldots, \boldsymbol{y}_{t-1}$.

  • (c) Find the covariance of x _t, y _t when both are conditioned on y _t, …, y _t − 1.

  11–12. We have provided a dataset (https://github.com/emilyoby/bmi-data-set) consisting of center-out arm reaches and neural activity recorded from a Utah electrode array implanted in M1. The following describes the data format. The .mat file has two data structures: ‘trainTrials’ contains 180 trials to be used as training data, and testTrials contains 8 trials to be used as test data. Each data structure contains ‘spikes’, ‘handPos’, and ‘handVel’ variables, representing the spiking activity, hand position, and hand velocity, respectively, on each trial in which a monkey reached to one of eight different targets. The ‘spikes’ variable contains, for each trial, the number of threshold crossings in 50 ms bins recorded simultaneously from the 91 electrodes and has dimensions (n time steps) × (91 electrodes), where n is the number of time steps within a particular trial. For example, ‘trainTrials.spikes{i}(n,k)’ contains the number of threshold crossings recorded on the kth electrode in the nth time step of the ith trial. The ‘handPos’ and ‘handVel’ variables are structured similarly and contain the 2D hand position (in mm) and velocity (in mm/sec), respectively, for the same time steps as in the ‘spikes’ variable.

  For the problems below, use the provided neural and kinematic data to implement the continuous decoders discussed in Sect. 5.4.

• 11. Use PVA decoder to estimate the movement velocity during a center out task.

  • (a) Fit the parameters of the decoder using the 180 trials of training data.

  • (b) Test the decoder on the eight test trials. Plot the decoded trajectories and the actual movement trajectories on the same plot.

  • (c) Try improving the decoding by smoothing the firing rates by using a running average of the firing rates during the previous 250 ms.

• 12. Use a Kalman filter decoder to estimate the movement trajectory for each trial.

  • (a) Fit the parameters of the decoder using the 180 trials of training data.

  • (b) Test the decoder on the eight test trials. Plot the decoded trajectories and the actual movement trajectories on the same plot.