Building an internal model of a myoelectric prosthesis via closed-loop control for consistent and routine grasping
- 420 Downloads
Prosthesis users usually agree that myoelectric prostheses should be equipped with somatosensory feedback. However, the exact role of feedback and potential benefits are still elusive. The current study investigates the nature of human control processes within a specific context of routine grasping. Although the latter includes a fast feedforward control of the grasping force, the assumption was that the feedback would still be useful; it would communicate the outcome of the grasping trial, which the subjects could use to learn an internal model of feedforward control. Nine able-bodied subjects produced repeatedly a desired level of grasping force using different control configurations: feedback versus no-feedback, virtual versus real prosthetic hand, and joystick versus myocontrol. The outcome measures were the median and dispersion of the relative force errors. The results demonstrated that the feedback was successful in limiting the variability of the routine grasping due to uncertainties in the system and/or the command interface. The internal models of feedforward control could be employed by the subjects to control the prosthesis without the loss of performance even after the force feedback was removed. The models were, however, unstable over time, especially with myocontrol. Overall, the study demonstrates that the prosthesis system can be learned by the subjects using feedback. The feedback is also essential to maintain the model, and it could be delivered intermittently. This approach has practical advantages, but the level to which this mechanism can be truly exploited in practice depends directly on the consistency of the prosthesis control interface.
KeywordsUpper limb prostheses Grasping force Closed-loop control Internal models Feedforward Feedback
Grasping is a complex task that involves a simultaneous and coordinated activation of many degrees of freedom (MacKenzie and Iberall 2010). Yet, it is performed routinely by human subjects as a smooth and fast movement. From the perspective of a high-level control, the human hand is therefore a reliable end-effector with a controllable and consistent response. Such an impressive performance with a system of a highly complex structure is afforded by the model-based feedforward control mechanisms (Johansson and Cole 1994; Hermsdörfer et al. 2011), established through an extensive practice during development. It is hypothesized that in order to control predictively, able-bodied subjects acquire internal models of system dynamics (Wolpert and Miall 1996; Kawato and Wolpert 1998; Haruno et al. 2001). It is also acknowledged that sensory feedback is likely instrumental for learning, utilization, maintenance, and updating of these models (Augurelle et al. 2003; Hermsdörfer et al. 2008).
After an amputation of the hand, the lost limb of a transradial amputee can be substituted by a myoelectrically controlled hand prosthesis (Belter et al. 2013) to restore the grasping function. However, the restoration is only partial, since direct somatosensory feedback is still unavailable in the commercial devices (Ning et al. 2012), except for one recent system (VINCENTevolution 2, Vincent Systems GmbH, DE). The lack of direct feedback can be somewhat compensated by using alternative sources of information (e.g., visual assessment, motor sound), which can be effectively exploited for control (Ninu et al. 2014). However, a drawback is that other senses (vision, audition) are allocated to the prosthesis control task. The provision of somatosensory feedback could therefore facilitate intuitive control with less effort as well as promote the feeling of embodiment through sensory–motor integration.
The methods to provide direct somatosensory feedback are being addressed in the recent scientific literature (Antfolk et al. 2013; Ninu et al. 2014; D’Alonzo et al. 2013; Saunders and Vijayakumar 2011). The general approach known as sensory substitution includes reading the data from the sensors embedded into the prosthesis and delivering this information to the residual limb using patterned mechanical or electrical stimulation (Kaczmarek et al. 1991; Szeto and Saunders 1982). In most studies (Patterson and Katz 1992; Cipriani et al. 2008; Chatterjee et al. 2008; Patterson and Katz 1992; Jorgovanovic et al. 2014), the grasping force has been considered as the information to be fed back to the user since this variable is not directly assessable through vision. Although most users as well as researchers would agree that feedback should be provided, there is no general consensus about the exact role and benefit of the closed-loop control, especially regarding the possible improvements in utility and effectiveness. The experimental outcomes of the recent studies are inconsistent: some demonstrated improvement in performance (Patterson and Katz 1992; Jorgovanovic et al. 2014), whereas others reported no difference with respect to the no-feedback condition (Cipriani et al. 2008), or an improvement limited to only specific subjects and/or conditions (Saunders and Vijayakumar 2011; Chatterjee et al. 2008).
These conflicting results point out to the necessity to understand the feedback from a more fundamental perspective, i.e., by regarding it as a component of a human control system integrating multiple other processes (e.g., learning, prediction, and feedforward control). Recent studies demonstrated that prosthesis users, similar to able-bodied subjects, might employ internal models for feedforward control of prostheses with no somatosensory feedback (Lum et al. 2014; Metzger et al. 2010; Weeks et al. 2000). However, there is only a single study (Saunders and Vijayakumar 2011) addressing the role of feedforward and feedback processes when using a closed-loop prosthesis. When the prosthesis was controlled consistently and predictably, using a switch to close the hand at a constant speed and increase the force at a constant rate, the subjects successfully accomplished the grasping task even in the condition of full feedback deprivation. The addition of vibrotactile force feedback improved the performance only when random time delays with respect to the onset of movement and onset of the force ramp were implemented.
The aim of the present study was to explore the nature of control processes when operating a myoelectric prosthesis. A prosthetic hand can be used for different tasks, from careful manipulation of a delicate object to a routine grasping of daily-life items, and the focus of the present study was on the latter. Ideally, a prosthesis user should be able to grasp an object fast and smoothly, in a consistent and straightforward manner by generating a feedforward command to close the hand and directly produce the desired force.
In the present study, the routine grasping task was investigated using several control configurations: feedback versus no-feedback control, ideal versus real controlled system, and reliable versus noisy command interface. The aim was twofold. The first objective was to assess how consistently the subjects could reproduce the same level of force when performing a sequence of routine grasping trials in different control conditions. The consistency of grasping is regarded as an important measure of the prosthesis controllability, which can directly affect the utility as well as the feeling of embodiment. If the prosthesis responds consistently to the user intentions, as the normal hand would, this might lead to better control (fast feedforward) and facilitate the integration of the prosthesis into the body scheme of the user. The second objective was to evaluate the specific contribution of the force feedback in facilitating the grasping consistency as well as in promoting the acquisition and maintenance of the internal models, when performing a sequence of routine grasping trials. The role of feedback in this context is not straightforward, since the routine grasping is executed using feedforward control. Therefore, the force feedback is not utilized for online modulation during the progression of the grasp. However, it provides the user with the resulting grasping force, and this information can be used to update the feedforward command delivered to the hand in the next grasping trial. Considered over many trials, this could allow the user to learn a stable internal model of the prosthesis behavior, i.e., a direct mapping between the desired output and the necessary command input [i.e., an inverse model (Wolpert and Kawato 1998)], eventually rendering the continuous feedback truly redundant. The latter could, however, hold only in certain conditions (e.g., reliable control interface).
Closed-loop control system
Both command interfaces provided a single continuous signal normalized to the interval [0, 1], proportionally controlling the velocity of closing and increase in grasping force. The release and opening of the hand were automatically triggered when the command signal returned from a positive value back to zero. The hand opened completely, and the fully open hand was the starting position in each grasping trial. The joystick (JOY) was operated using the index finger and calibrated from the center position (0) to the maximum left inclination (1). Two standard Ag/AgCl electrodes (Neuroline 720, Ambu, US) were placed over the wrist and finger flexor muscles. Myocontrol was calibrated so that approximately 80 % of the maximum voluntary contraction resulted in the maximum command signal (1). The control and system parameters for the virtual hand model and real prosthesis were constant throughout the experimental session.
The virtual hand model emulated the response of the real hand. The model was realized as a state machine switching between an integrator for the velocity control of aperture, before contact, and a gain for the proportional control of force, after contact. The model implemented non-backdrivable behavior, as the real prosthesis. The purpose of the virtual hand was to investigate the routine grasping with an ideal controlled system (i.e., time invariant, deterministic, and no time delays).
Experimental setup and protocol
The amplifier sampled the EMG at 1 kHz and computed root mean square (RMS) internally, over 250-ms data segments with 80 % of overlap, sending every 50 ms a new RMS value to the host PC through a USB connection. The control loop at the PC operated five times faster to oversample the RMS data, ensuring the responsiveness to the signal changes. The time delay for data transfer from the amplifier to the host PC was approximately 10 ms. The joystick provided an analog voltage signal digitized by a data acquisition card at the rate of the control loop (@100 Hz). The host PC was a standard desktop computer running the closed-loop system and communicating with the prosthesis via a Bluetooth. The prosthesis controller (AxonMaster) sampled the sensor data at 100 Hz and sent them to the host. The time delay for data transfer over Bluetooth was approximately 50 ms. This was the time delay between the joystick command and the prosthesis reaction. In the case of myocontrol, the total time included the data transfer from the amplifier to the host PC and then to the prosthesis, resulting in approximately 60 ms. Importantly, the delays in the control loop were unlikely to affect the performance in the current experiment, since the control within the trial was feedforward (no online modulation).
The tests were performed on nine able-bodied volunteers (25 ± 4 years) who signed an informed consent for the experimental protocol that was approved by the local ethics committee. The subjects were seated comfortably in a chair in front of a desk, and they used their dominant hand to control the prosthesis. The prosthesis was placed on a separate table behind the subject (Fig. 1b). The hand was fixed using a clamp and positioned so that it grasped the target object (wooden piece) when it was closed. The subjects wore noise-cancelation headphones and was thereby fully detached from the prosthetic hand, removing all potential sources of indirect feedback.
Each subject was tested in four experimental conditions (EMG/JOY × VH/RH) in a randomized order to avoid the potential interactions due to systematic arrangement and minimize the effect of indirect factors (e.g., mental or muscle fatigue). The task for the subjects was to close the hand and generate a predefined level of grasping force equal to 60 % of the prosthesis maximum (~100 N). The subjects were instructed to grasp in a feedforward and predictive manner: set the command signal to close the hand at a certain speed so that, after the hand contacted the object, the grasping force jumped directly to the target force level. Therefore, there was no online modulation of the closing speed and grasping force. The maximum force reached during a grasping trial was adopted as the trial outcome. When controlling the virtual hand, the subjects performed three blocks of 50 grasping trials each, and the grasping force feedback was provided in all the trials. These blocks served to determine the benchmark closed-loop control performance (i.e., explicit force feedback and ideal system). With the real hand, the first and second block comprised 80 and 60 trials, respectively, where the last 30 trials were performed without the grasping force feedback (i.e., force bars removed from the scene). The latter resembled a common real-life scenario in which the user grasps an object while looking into the hand. In the third block with the real hand, comprising 60 trials, the gripper was also hidden from the view during the second half of the block (30 trials), and the grasping was therefore performed completely blindly. The blocks with the real hand assessed both the closed-loop control performance when controlling the physical system and the retention of the feedforward models when the direct force feedback was removed. Therefore, the blocks with the real hand comprised more trials (with and without feedback) compared to the blocks with the virtual hand (with feedback only). A short 2-min break was inserted between the blocks and 5-min between the conditions. In the blocks with myocontrol, the subjects produced brief contractions (1–3 s) at about 50 % of MVC and they did not report muscle fatigue. The experimental session lasted approximately 2 h. Before starting the blocks in each condition, the subject received a short training in hand control (~ten grasps).
For each grasping trial, the absolute deviation of the generated grasping force from the target force level was computed. The relative error was then calculated as a percent of the target force level. The data were not normally distributed (Kolmogorov–Smirnov test). To assess the quality of force control, a median and interquartile range (IQR) of relative errors were calculated as the measures of grasping accuracy and consistency (precision), respectively. For the virtual hand, the data from all three blocks of trials were pooled together (VH group). For the real prosthesis, each block was separated into trials with feedback (groups RH 1, 2, and 3) and without feedback (groups RH 1 and 2 no force, RH 3 no force/hand). Significant differences in medians between the groups were tested using Kruskal–Wallis test, while the post hoc pairwise comparison was performed using Wilcoxon rank sum test with Bonferroni correction. To evaluate the significant differences in dispersions, Bartlett multi-sample test and two-sample F test for equal variances with Bonferroni correction for multiple comparisons were applied. The threshold for the statistical significance was set at p < 0.05 for both medians and dispersions. Furthermore, the results of all subjects were pooled and organized in the groups of five consecutive trials, with four trials of overlap (i.e., sliding-window segments). The number of trials within the sliding window was selected heuristically as a trade-off between the variance and resolution. Five was the smallest number still revealing a consistent trend across the block. This analysis was performed separately over the trials with and without force feedback. The baseline performance, last 30 trials with feedback, was then compared to the pooled sliding-window results (baseline vs. each window). Using this analysis, it could be determined not only whether the removal of feedback influenced the average performance per group of trials, but also in which point across trials the change in performance became significant. Therefore, the blocks/conditions could be compared with respect to the stability of the acquired inverse models across trials. The data analysis and statistical tests were performed in MATLAB 2013b.
The same sequence of events can be recognized in myocontrol (Fig. 2b). However, with EMG, the subject was far less consistent in generating the control signals. The consecutive commands were variable (rippled waveforms), especially regarding the maximum amplitude. Since the commands were noisy, it was difficult for the subjects to achieve a consistent control at the point of contact and thereby consistent grasping forces. The signals of similar magnitudes could produce very different force levels (see Fig. 2b, arrow annotations).
With myocontrol (Fig. 4b), the performance was substantially worse compared to that achieved using the joystick. Controlling an ideal system (virtual hand) with force feedback using myocontrol resulted in a similar performance as when controlling a real system (prosthetic hand) using the joystick but with no force feedback, i.e., there were no statistically significant differences between EMG/VH in Fig. 4b versus JOY/RH 1 no force and JOY/RH 2 no force in Fig. 4a. Therefore, a consistent interface could compensate for the addition of the mechanical factors (transition from virtual to real system) as well as the lack of explicit force feedback. Furthermore, with myocontrol, contrary to what was observed for the joystick, there was no difference in the closed-loop control performance when controlling ideal versus real system. Again, the median error and dispersion were similar across the blocks. Removing the force feedback substantially increased the error median and dispersion to 22/26 % and 17/26 % in block 1 and 2, respectively, but unlike with the joystick, removing the hand visualization did not further decrease the performance (e.g., no statistical differences between RH 2 no force and RH 3 no force/hand). The performance of the myocontrol with no force feedback was similar to that of the joystick control during blind grasping.
Sources of variability
It is well known that the EMG signals are inherently noisy, leading to the variability at the prosthesis control input. In addition, the prosthesis itself is a nonideal physical system, exhibiting complex mechanical phenomena (Engeberg and Meek 2008), resulting in the variability in the prosthesis output. The impact of mechanical effects can be substantial, as illustrated for force tracking in (Dosen et al. 2014). The aim of the present study was to quantify the influence of these two sources of variability in the case of routine grasping task. Controlling the virtual hand, a system with an ideal response, using the joystick as a precise and reproducible command interface, was used to establish the benchmark performance. Substituting the ideal system with the real prosthesis increased the variability (3–4 %) and did not change the accuracy (similar median error). Therefore, the prosthesis responded reliably when the subject was able to generate a consistent sequence of feedforward commands. A slight increase in variability was due to the nonideal properties of the real system [backlash, friction effects (Engeberg and Meek 2008)] and thereby increased uncertainty and more challenging control. When the subjects switched to myocontrol, however, the performance deteriorated substantially; both the error median and variability increased almost twofold compared to the joystick controlling the real system and even threefold with respect to the joystick operating the virtual hand. The main reason for this increase was the inconsistency inherent to the command interface. The generated feedforward command waveforms were noisy and variable, and this effect completely dominated the influence of the physical system properties, i.e., with myocontrol, contrary to the joystick, there was no difference in performance when operating an ideal system versus a real prosthesis. The prosthesis can be therefore regarded as a system with a reliable response. The full capabilities of this system are not exploited, however, due to the inconsistency of the myocontrol interface. Improving the latter by using new technical (Engeberg et al. 2008) or signal processing solutions (Sanger 2007) is thus an important future goal.
The role of feedback
The second aim of this study was to investigate the role of feedback in the context of routine grasping. Although the execution of a single routine grasping trial was a feedforward process, the assumption was that the subject could use the feedback to iteratively refine the feedforward command (Kawato et al. 1987; Kawato and Gomi 1992). Ideally, after the model was learned, the subject would be able to control the system without the force feedback or even without seeing the hand. And indeed, when the joystick was used to operate the prosthesis, few initial trials with feedback were needed for the force to converge, after which the performance was consistent. The performance temporarily remained at the benchmark level even after the force feedback was removed and the hand was still visible (block 2). Importantly, the hand closing velocity is proportional to the grasping force after contact, and therefore, the velocity can be used as an indirect feedback for the control of grasping force (Ninu et al. 2014). The subjects could maintain the desired force by maintaining the same velocity of closing. The velocity could be used to retune the feedforward command instead of the explicit force information. Therefore, this condition can still be regarded as a closed-loop scenario, although with a nonideal feedback. However, this strategy was only short-term effective. Once the explicit feedback of force was removed, the feedforward command and thereby the generated grasping force started drifting from the desired level, eventually resulting in the statistically significant difference with respect to the baseline. With myocontrol, the internal models were even less stable without regular explicit force feedback updates. The control performance dropped almost immediately after the force feedback was removed. Because the myoelectric signals are noisy, the information about the prosthesis velocity was not enough to maintain the consistency of the feedforward command across trials. Finally, completely blind grasping was characterized by unstable internal models, and this was irrespective of the control interface. Therefore, relying on the memory of the learned feedforward command and the sense of effort was not enough to maintain the inverse model. The results thus demonstrated that the feedback could be used to acquire the feedforward model of grasping and that it was also essential for maintaining the model across trials, especially when using myocontrol. The force feedback allowed the subjects to learn the feedforward command (inverse model) so that they could drive the prosthesis within the ballpark of the target force. With the reliable control interface (joystick), the learned model could be maintained at the same level of accuracy for a certain number of trials even when the direct force information was substituted with an indirect feedback source (hand closing velocity). Finally, even though with myocontrol and/or full feedback deprivation the performance deteriorated already in the first few trials, the time course of performance degradation (e.g., drift with saturation) still implies that the preceding phase of feedback-facilitated learning had an effect that was visible until the end of the block. The learning could become more stable with the longer training or by acquiring the models at the lower target force levels, since the variability of the EMG depends on the contraction level.
It was reported in the recent study (Saunders and Vijayakumar 2011) that the feedback was redundant for the control of prosthesis grasping when the conditions were ideal (e.g., consistent control interface). This is in line with the results for the joystick control in the current study demonstrating the successful retention of the models in the absence of feedback. However, the current study also pointed out that this process has a time dimension and that the model will inevitably deteriorate if not updated again, even in the ideal conditions. The conclusion of another, very recent study (Lum et al. 2014) was that the users of open-loop prostheses employ internal models for grasping, but that the accuracy of these models is poor. This is in accordance with what was observed here for the myoelectric control.
As mentioned in the Introduction, the results in the literature investigating the benefits of the closed-loop control in improving the utility of myoelectric prostheses are inconsistent. One reason for this inconsistency is that many motor activities in the daily life of able-bodied subjects (Wolpert and Kawato 1998) as well as amputees (Ninu et al. 2014) can be performed by relying mainly on the feedforward control. In this case, even when provided, the feedback might not be exploited by the prosthesis user to modulate the online execution of the movement. This context was investigated in the present study using the routine grasping task. The proposition is that the feedback can be considered as an instrument for learning through repeated practice, with the most visible effect in the initial phase of practicing, i.e., before the user of the prosthesis becomes well trained. However, the quality and stability of this learning depends directly on the reliability of the feedforward control interface. This view on the function of feedback in prosthetics parallels the insights about the role of feedback in human motor control (Kawato and Wolpert 1998; Haruno et al. 2001; Davidson and Wolpert 2005). Considering this viewpoint, an economical feedback protocol can be hypothesized, in which the amount of feedback is reduced gradually during the training by delivering the feedback intermittently, frequently enough to refresh and maintain what has been learned.
For the practical application, the force information could be provided using electrotactile or vibrotactile stimulation. A simple method would be to implement a spatial coding using an array of stimulators, where the activity of each stimulator denotes a certain level of force (tactile force bar). The stimulation could be delivered intermittently, as indicated above. This would have several practical advantages: avoiding interference between stimulation and recording, power saving, less intrusive feedback, decreasing habituation, etc. However, the feasibility and details of the intermittent feedback protocol have yet to be investigated. The training could start with visual feedback and then continue in daily life using electrotactile interface. In the latter case, the provision of feedback could be even under user control. When the user feels confident in the prosthesis use, he/she can decide to switch the feedback off, resulting in less intrusive system operation. The future studies need also to consider more comprehensive scenarios, including a range of target forces and longer training.
In the present study, the conditions were randomized in order to investigate them individually. However, arranging the conditions in a particular order could facilitate the acquisition and maintenance of the internal models, forming the basis for the training protocol. For example, the subjects could start with the virtual hand, building up the appropriate EMG command, and then switch to the real hand to fine-tune the EMG, accommodating the physical effects due to the real prosthesis. In addition, to characterize the effect of training, the initial performance of the routine grasping without direct force feedback (hand visible) as well as without any external feedback (blind grasping) should be assessed and compared to the performance achieved in the same conditions after the training. The performance would likely depend on the previous experience in myocontrol. Experienced subjects might be able to generate less variable myoelectric signals. This could lead to a more consistent force control with feedback and more stable feedforward models in the absence of feedback. However, the inherent variability of the EMG signals might still be an important limiting factor, regardless of the subject experience, especially at high contraction levels, as demonstrated in (Ninu et al. 2014).
Finally, future research should also consider that there might be other sources of feedback, which the prosthesis users could exploit, as pointed out in Introduction. It is known from motor control studies that humans can integrate sensory information from multiple sources based on their relative uncertainty using Bayesian fusion (Körding and Wolpert 2004). In principle, the visual force feedback used in the present study can be regarded as the final output of this integration process. In a practical situation, this information might be contributed using electrotactile feedback but also by any other available cue (e.g., motor sound). The latter might be as reliable as the former or even more accurate, especially in the case of an experienced user (who has learned to “listen” to his/her prosthesis). For learning and maintaining the feedforward models, it is important that the feedback information be provided, whereas it is less relevant how exactly this information has been obtained. Therefore, in order to determine the utility of the direct somatosensory feedback, the nature and quality of the alternative sources and the specifics of the aforementioned sensor fusion mechanisms have to be revealed.
This work is financially supported by the German Ministry for Education and Research (BMBF) via the Bernstein Focus Neurotechnology (BFNT) Göttingen under the Grant Nos. 01GQ0817 and 01GQ0810, and the European Commission under the MYOSENS (FP7-PEOPLE-2011-IAPP-286208) projects.