This paper describes the design and preliminary testing of FINGER (Finger Individuating Grasp Exercise Robot), a device for assisting in finger rehabilitation after neurologic injury. We developed FINGER to assist stroke patients in moving their fingers individually in a naturalistic curling motion while playing a game similar to Guitar Hero®a. The goal was to make FINGER capable of assisting with motions where precise timing is important.
FINGER consists of a pair of stacked single degree-of-freedom 8-bar mechanisms, one for the index and one for the middle finger. Each 8-bar mechanism was designed to control the angle and position of the proximal phalanx and the position of the middle phalanx. Target positions for the mechanism optimization were determined from trajectory data collected from 7 healthy subjects using color-based motion capture. The resulting robotic device was built to accommodate multiple finger sizes and finger-to-finger widths. For initial evaluation, we asked individuals with a stroke (n = 16) and without impairment (n = 4) to play a game similar to Guitar Hero® while connected to FINGER.
Precision design, low friction bearings, and separate high speed linear actuators allowed FINGER to individually actuate the fingers with a high bandwidth of control (−3 dB at approximately 8 Hz). During the tests, we were able to modulate the subject’s success rate at the game by automatically adjusting the controller gains of FINGER. We also used FINGER to measure subjects’ effort and finger individuation while playing the game.
Test results demonstrate the ability of FINGER to motivate subjects with an engaging game environment that challenges individuated control of the fingers, automatically control assistance levels, and quantify finger individuation after stroke.
Over the past several decades, researchers have developed robotic devices for rehabilitation therapy after stroke. This is in response to a sizable need, with nearly 800,000 people per year suffering a stroke in the United States alone . Of the survivors, approximately two-thirds experience long-term impairment of their affected upper-extremity . Robotic therapy devices can automate the repetitive and strenuous aspects of conventional physical therapy. Furthermore, robotic therapy devices can serve as scientific instruments for quantifying the recovery process, and thus may provide insight that is not normally available with conventional therapy alone.
Robot assisted therapy of the upper extremity following stroke has been shown to be as effective as, and in some cases modestly more effective than, conventional therapy (for reviews see [3–7]). Research with robotic therapy devices supports the contention that motor recovery increases with therapy intensity , i.e. more practice is better. What remains unclear, however, is how a rehabilitation robot should interact with the patient in order to optimize recovery during practice. One approach is to help patients practice movements that they cannot complete without assistance, which may foster somatosensory stimulation that induces brain plasticity . Indeed, most rehabilitation robots are strong enough to complete movements even when patients are completely impaired and/or when tone and spasticity act in opposition. However, care must be taken so that the robot does not “take over” the movement practice from the patient, which may cause the patient to “slack” and reduce their effort at the task being practiced [9, 10]. Patient effort is considered crucial to increasing motor-plasticity during rehabilitation therapy [11, 12]. Thus, it seems important for robotic rehabilitation devices to simultaneously enable movement practice and encourage patient effort during therapy.
Numerous control strategies for robotic therapy have been successfully implemented and tested, as summarized in . Of specific interest are “assist-as-needed” control strategies, which change assistance in response to perceived effort, typically correlated in some way to performance error (tracking error or similar). These controllers alter the assistance level by modifying controller parameters (e.g. feedback gains, desired trajectory shape and/or timing, model based terms, etc.) [9, 14–18]. Tests with these controllers suggest that increased error encourages patient effort, and vice-versa, although the relationship remains unclear. Additional experiments may clarify this and other relationships affecting motor recovery during rehabilitation therapy, although the ultimate validation clearly depends on therapeutic efficacy.
Effectively exploring the factors that promote functional recovery during movement therapy and evaluating “assist-as-needed” and other control strategies depends on the control fidelity of the robotic platform. To quantify baseline motor ability, ideally, the robot should be able to appear both massless and frictionless to the patient, and should be highly compliant and backdriveable. However, it is also important to have a high bandwidth of force control, as to not limit the response of robot during interaction with the patient. Improving the control and impedance characteristics of a rehabilitation robot has the potential to make such devices better scientific instruments as well as allowing more precise investigation of motor learning and the mechanisms of neuroplasticity, as suggested by .
Another critical consideration for understanding the mechanisms by which rehabilitation robots promote recovery is the limb of application of the robot. As an integral part of activities of daily living (ADLs), rehabilitation of the hand is particularly important, and a significant need exists for improved hand rehabilitation, as most of those who have suffered a stroke experience some impairment in hand function . Furthermore, the hand and fingers have a highly developed neuro-muscular system to which the brain has dedicated a large portion of resources.
Designing a robot to actuate the hand or finger is a significant challenge, as evidenced by the large variety of robotic devices that have been developed for hand and finger therapy. Previous work has focused often on re-creating the complexity of hand and finger movements, often at the expensive of actuation and control. These hand robots typically fall into the category of end-effector or exoskeleton (for review see ). End-effector devices attach distally and do not attempt to align with the joint axis of the patient, as exoskeleton devices typically do.
In the work presented here, we sought to maximize controller fidelity and minimize the mechanical impedance of the device, at the expense of the robot’s degrees-of-freedom. Although each finger in the human hand has multiple degrees-of-freedom, most ADLs incorporate a simple finger curling motion, similar to a power grasp . Thus, an opportunity existed to create a finger-curling robot with one degree-of-freedom, high control fidelity, and low friction.
FINGER, the finger curling robot presented here, is capable of individually assisting both the index and middle fingers through a natural grasping motion (Additional file 1). Each finger is individually guided by an 8-bar mechanism that controls the orientation and position of the proximal phalanx and the position of the middle phalanx. Each 8-bar mechanism has a single degree-of-freedom and is actuated by a high bandwidth and low-friction linear electric actuator. Further friction reduction is achieved through feed-forward control compensation.
In the sections that follow, we present the design, controller development, and preliminary testing of FINGER. We present the mechanism synthesis, which is based on motion capture of finger grasping motion, first. We then describe the mechanical design, including sizing adjustments and patient-robot interface. In the third section, we describe the actuation including controller development and friction compensation. Finally, we present some results from pilot testing with several subjects who have suffered a stroke. Portions of this work have appeared previously in conference paper format [22, 23].
Finger curling data acquisition and analysis
This section describes motion capture and data analysis used to characterize a basic finger curling motion, similar to a power grasp . Although the human hand can perform many differing grasps and grips in order to manipulate objects, the basic curling motion is the most common and therefore we focused on it for administering and studying finger movement therapy.
Color-based motion tracking  was used to record the path of the index finger as the hand performed a curling motion. Subjects’ hands were filmed with a single camera from above, where the index and middle fingers curled in a plane perpendicular to the camera. Any out-of-plane motion during curling was small and treated as noise. The back of the hand was placed against a rest aligning it with the x-axis. Four brightly colored felt dots of differing colors were attached to the index finger, two each on the center-lines of the proximal and middle phalanges, which are the defined attachment points for the 8-bar mechanism. Likewise, the colored dots were attached using hook-and-loop straps that were the same thickness as the planned mechanism straps. The placement of the felt dots allowed the centerlines of the proximal and middle phalanges to be recorded throughout the curling motion. See Figure 1.
Seven healthy adult subjects were asked to curl their hand, meeting the thumb in a circle, for a minimum of 10 times. They were not given any further instructions regarding how to curl their hand, in order to produce the most natural motion. Dimensions of the index and middle fingers and hand were recorded for each subject using calipers. The lengths of the proximal and middle phalanges for the index and middle fingers were recorded in a similar fashion as . The distances between creases for both the index and middle fingers were also recorded in the same manner as .
The path of the four felt dots was regressed against a planar, two revolute joint model, shown in Figure 1. The regression model determines 5 dimensions using the system of equations given below in (1). These equations are defined for each frame of the motion capture.
In (1) above, m1, m2, p1 and p2 are the positions of the four markers, and O x and O y define the location of the metacarpophalangeal (MCP) joint of the index finger. During mechanism design, this point is assumed to be the origin. The length of the proximal phalanx is denoted l p and the final two parameters are the distances to the proximal and middle strap attachment from the previous joint, referred to as the proximal, r p , and middle, r m , radii. It may seem that these radii should center the straps along the proximal and middle phalanges, but in practice the position along the proximal phalanges that is most comfortable for a strap is significantly forward of the center of the phalanx. For example, in Figure 1 it can be seen that the hook and loop strap holding the felt dots to the proximal phalanx sits comfortably at more than half the distance along the proximal phalanx from the MCP joint. The same relationship is true for the middle phalanx.
The mean length of the index finger proximal phalanx determined by the motion capture and regression analysis was 42 mm, with a standard deviation of 3 mm. This mean value was compared to  which contains a statistical analysis of 4000 hand samples. The ratio of this mean proximal length to the mean proximal length reported in  was multiplied by the standard deviation also reported in , producing a scaled standard deviation 3 mm, which is close to the standard deviation of the small data sample used.
Following the approach in , the length change of the proximal phalanx between successive mechanism sizes was chosen to be two standard deviations. By scaling the other variables accordingly, the complete dimensions for the other finger sizes may be found, as given in Table 1. This range of finger sizes provides an acceptable coverage of the population of hand sizes.
This approach to regression has the advantage of determining the dimensions of the finger independent of the motion type. Thus, the angular relationship between the phalanges can be used independently to define the finger motion. Specifically, regression is used to determine the middle phalanx angle, θ m , as a function of the proximal phalanx angle, θ p , using a second-order polynomial equation.
Figure 2 shows the relationship between the middle phalanx angle and the proximal phalanx angle for the curling motion collected from all 7 motion capture subjects. The black line is a quadratic curve-fit. With significant variance in hand sizes, the relationship between the two angles appears uncorrelated to hand size.
The 2nd order equation used to fit the data as shown in Figure 2 is:
This equation and finger dimensions given in Table 1 were used to generate the 15 target points for each mechanism size, consisting of 15 desired points and angles of the proximal phalanx and 15 desired points of the middle phalanx, repeated for each of three sizes: extra-small, small, medium, and large. This number of points was chosen in order to maintain a good spatial resolution while keeping computational complexity of the design reasonable. The target points were created using the 2 revolute joint planar model with the angle of proximal phalanx, θ p , varied from 0° to 60° discretized into 15 evenly spaced target angles. The angle of the middle phalanx, θ m , was determined from (2) and the target points for both the proximal and middle phalanges were defined at 19 mm behind the center-line of the finger to allow for a means of connecting the robot to the hand.
Designing a mechanism to reach multiple end-effector configurations, known as mechanism synthesis, is a well-studied research area . This particular application, however, has a unique twist. In this case, there is not a single desired configuration but rather two that are correlated; one for the proximal phalanx (position and angle), and one for the middle phalanx (position only). Furthermore, the design specifies a planar grasping motion with a single DOF for each finger. Planar mechanisms, with their multiple varieties of single DOF configurations, provide an adequate solution base for this design problem.
Initial mechanism synthesis attempts explored multiple configurations of Watt type six-bar chains , but were ultimately unsuccessful in reproducing the desired output configuration. The final design uses an eight-bar mechanism (Chain 1 from ) with revolute joints (see Figure 3). The goal configurations consist of the position (P G ) and angle (μ P ) of the proximal phalanx and the position of the middle phalanx (M G ). The mechanism is made up of 10 revolute joints (G, G1, W, W1, W2, H, H2, Y, Y1, and Y2) and 7 links defined by the kinematic chains GW, WHW1, G1W1W2, HPYH2, W2H2Y2, Y1Y2, and YMY1. These links are defined by seven structural angles (α, α2, δ, δ2, γ, γ2, and μ) and 13 structural lengths (d1-11, m, and m2). Figure 3 also shows the seven configuration angles (θ, θ1, ϕ, ϕ1, ϕ2, ψ, and ψ1) that changes as the mechanism moves. The mechanism has 1 DOF so that specifying one of these configuration angles specifies the complete configuration of the mechanism.
The preliminary optimization of this mechanism was presented in . The approach here is similar, but here the mechanism configuration has changed so that the middle phalanx end-effector is connected to link Y1Y2 rather than YY1. This change improved the ability of the optimization process to reach desired middle phalanx target points, and also made the resulting mechanism easier to manufacture.
Mechanism design equations
The design equations that define the 8-bar mechanism consist of both path and loop equations. The path equations consist of three separate kinematic paths through the mechanism from one of the fixed pivots to the each of the end effectors (P and M). The path equations are similar to those presented in  but have changed to improve optimization and manufacturability, based on a trial and error design process. For the proximal phalanx, the shortest path equation is:
where c and s stand for cosine and sine, respectively, and the other parameters are previously defined and shown in Figure 3. The other two path equations for the proximal phalanx are defined by the kinematic chains G1W2H2HP and G1W2Y2Y1YHP. As previously mentioned, the middle phalanx path equations have changed more significantly. Using the same notation, the shortest path to the middle phalanx is
The other two kinematic paths to the middle phalanx are defined by the kinematic chains G1W2Y2M and GWHYY1Y2M. The design equations also include 3 internal loop constraint equations which must be satisfied to keep the design viable. The inner loop equation is
The other two equations, for the middle and outer loops are defined by the kinematic chains GWHH2W2G1 and GWHYY1Y2W2G 1 .
As mentioned before, the design specifies a goal angle of the proximal phalanx, μ P , in addition for each goal position of the proximal phalanx, P g . Using the relationship
the goal angle for the proximal phalanx is substituted into the previously presented path and loop equations to constrain the configuration angle ψ to the goal angle of the proximal phalanx, μ P , and the structural angles α2, δ, and μ.
With the path and loop equations defined, the design problem becomes a function minimization problem. The objective is to find the structural variables and the set of n = 15 configuration angles that best reach the 15 desired configurations. To achieve this, a cost function is created consisting of the sum of the squares of the distance between the desired end-effector points P and M and the goal positions P G and M G for each of the 15 desired configurations. The cost function is defined
where P n is the position of point P at angle θ n and PG,n is the nth desired configuration (with similar definitions for M and M G ). Only one configuration angle is necessary as the mechanism has only 1 DOF, and θ was arbitrarily chosen (the other possibility was θ1), even though in the final design we selected θ1 as the input angle for connecting the actuator (based on the locations of G and G1).
Mechanism design equation constraints
In addition to the cost function, a large set of constraints are required based on the overall design goals and manufacturing considerations. Our preliminary approach to consstraints was presented in . Some constraints require only upper and lower bounds, such as the location of the base points, G and G1, and structure variables d1 − 11, m, and m2. Other constraints require additional calculations, such as the link dimensions not specifically specified by the structural dimensions (the distance , for example). These constraints are summarized, with brief explanations, in Table 2.
These constraints govern the structural dimensions of the 8-bar mechanism, but do not limit the location of the free joints (W, W1, W2, H, H2, Y, Y1, and Y2) as the mechanism moves through the desired configurations. For example, any two joints should not overlap during the motion of the mechanism. This requires that the location of each joint be calculated at each of the 15 goal configurations. In total there are 29 joint pairs with the potential to overlap which are constrained to keep the distance between joints from going below a manufacturable distance. The complete list of joint-to-joint distances to calculate at each of the 15 goal configurations is given in Table 3.
One of the most important set constraints concerns the location of the joints with respect to the hand and fingers throughout the motion. One of the main goals for the mechanism was that it be entirely located behind the hand during operation. This goal was chosen to allow the hand to be easily attached to the robot, to allow stacking of mechanisms for individual fingers, and to facilitate providing sensory stimulation to the volar surface of the hand. For instance, a soft object can be mounted towards the palm of users’ hands to be touched while practicing grasp motions. In the presented experiment, however, this feature is not utilized. This requires all of the joints to be behind the hand and fingers at each of the 15 goal configurations. Thus the constraint area is constantly changing. We implemented this constraint area by creating three separate unit vectors, one at the back of the middle phalanx, u m , one at the back of the proximal phalanx, u p , and one at the back of the wrist, u w . These vectors are illustrated in Figure 4. These unit vectors at the phalanges point away from the hand at an angle perpendicular to the phalanx and are different for each of the 15 goal configurations. The unit vector at the back of the wrist also points away from the hand, but does not move for different goal configurations. Furthermore, three vectors are created for each of the free joints at each of the 15 goal configurations. These three vectors, v m , v p , and v w , all point to the aforementioned joint and originate from the base of the unit vectors u m , u p , u w (see Figure 4). The maximum of the dot products between the corresponding u and v vectors gives the distance, as a positive value, that the joint is from the back of the hand/finger. This value is constrained to between 12.7 and 76.2 mm, so that the joints clear the back of the hand during motion but are also kept from begin excessively away from the hand.
Mechanism cost function minimization
The cost function (7) was minimized within the bounds of the constraints detailed in the previous section using a constrained multivariable minimization optimizer (Matlab function “fmincon”). The minimization process was repeated numerous times with randomized initial conditions until viable solutions were found. The minimization was first performed on the goal configurations generated from the dimensions of the medium size given in Table 1. The resulting structural parameters of the medium mechanism are given in Table 4. The total error after cost function minimization was 2.14 mm for all 30 of the target points for the 15 medium finger size goal configurations. This is an average error of 0.071 mm per target point.
After the structural parameters for the medium finger size were determined, the cost function was re-minimized for the large, small, and extra small finger sizes using the medium structural parameters as the initial conditions. During re-minimization, all of the structural parameters were fixed except for five (m, m2, δ, δ2, & μ) which allow the new mechanism it to reach the goal configurations of the other finger sizes. Although other choices of parameters also allow the minimization to reach the trajectories for the other finger sizes, changing these five parameters can be accomplished by only modifying the shape the two end-effector links. This allows the re-sizing to be accomplished without the need to disassemble the mechanism when changing between sizes. The resulting values of these structural parameters and the minimized cost function for the other mechanism sizes are given in Table 5. A visual depiction of the ability of the four different mechanism sizes to reach the four sets of 15 desired configurations is shown in Figure 5.
The current version of FINGER has two identical planar 8-bar mechanisms to individually curl the index and middle fingers through a naturalistic motion. The mechanism, actuators, and adjustment assemblies are located behind the hand. As mentioned above, this allows contact of the volar surface of the hand with objects during therapy, and makes it easier to attach the hand of a subject to the robot.
Each 8-bar mechanism was designed with alternating inner links and outer link pairs, overlapping at joints to balance bearing forces and keep friction low. Two ABEC 5 bearings and one precision shoulder bolt were used for each joint. The links were designed in Solidworks to the dimensions determined from the mechanism synthesis and machined from aluminum using a three-axis, computer numerical control (CNC) milling machine. The linkage design includes mechanical hard stops to limit the range of motion to the desired range.
Finger cups with custom ratcheting straps (see Figure 6) are located at the two end effectors of each mechanism to attach the robot to the subjects’ proximal and middle phalanges. The middle phalanx finger-cup allows for rotation while the proximal finger cup is fixed, as per desired kinematic design. Each of the 8-bar mechanisms includes adjustability for different finger lengths. After inspecting the results of the mechanism synthesis, it was apparent that the locations of the end effector M as defined for the four hand sizes (Table 5) are very nearly located on a line with respect to link MY1Y2. The same is true for the proximal phalange end effector on PHY. This fact simplified the mechanical design, allowing for infinite positioning of the finger cups over the full adjustment range. The middle phalanx length adjustment is shown in Figure 6.
The location of each of the 8-bar mechanisms may be adjusted vertically to align them with the plane of the subjects’ index and middle fingers (see Figure 7). Furthermore, the wrist of the subjects is secured in a wrist cuff, of which the height and angle may be adjusted as necessary for alignment and comfort during gameplay (see Figure 7).
Each 8-bar mechanism is independently actuated. The two linear actuators are mounted on top of each other with a fixed vertical distance from the base plate while they can freely rotate about an axis normal to it. The detailed specifications of the actuators are explained in the sections that follow. The entire assembly is shown in Figure 8.
Robotic actuation and performance
FINGER uses two brushless linear motors (“Servo Tube” actuators, Dunkermotoren STA116-168-S-S03C) to independently actuate the 8-bar finger curling mechanisms. These actuators were chosen for their unique combination of high speed, low friction, and large stroke length. Because they lack any gearing or cables, they exhibit good backdrivability. This is an important feature for robot assisted therapy; the ideal rehabilitation actuator would be able to apply any force at any point during the desired motion, including zero-force, allowing the subject to see the results of their efforts.
This particular model of Servo Tube actuator can produce a continuous force of 26.75 N with a peak of 91.9 N. Current to the actuator is controlled by an amplifier (Copely Controls ACJ-055-09-S), which allows a voltage or PWM setpoint signal. The Servo Tube actuator has built-in Hall Effect sensors and outputs an emulated quadrature encoder position signal of up to 8 microns of resolution. Accelerometers (Analog Devices ADXL325EB) mounted to the end of the actuator rod measures actuator accelerations with a range of ±6 g.
The controller is implemented on a PC using Matlab® xPC Target, with a sampling frequency of 1000 Hz. A National Instruments 6221 DAQ card (16-Bits, 250kS/s) is used to acquire voltage signals from the accelerometers, read the quadrature outputs from the Servo Tube actuators, and send the forces commands to the actuators.
To evaluate the control fidelity of FINGER, we conducted a closed loop frequency response test. A Proportional-Integral-Derivative (PID) controller was used to follow desired sinusoidal trajectories with a magnitude of 75% of the range of motion and frequencies from 0.15 to 100 Hz. The PID controller gains, chosen by trial and error, for this test were KP = 8 N/m, KI = 8 N/m∙s and KD = 2 N∙s/m, respectively. The results, shown in Figure 9, show a −3 db magnitude reduction at approximately 8 Hz. The corresponding jump in phase lag indicates the nonlinearity in the system at high speeds.
Although the built-in position sensor of the Servo Tube actuator has a very high resolution, using a discreet derivative of the position signal can be very noisy, especially at low velocities. In order to obtain a smooth velocity estimate, a Kalman-Filter was designed that uses the actuator’s position signal and an acceleration signal from an accelerometer mounted at the end of the actuator rod. The Kalman-Filter gains were calculated using the Matlab LQR function (Linear Quadratic Regulator). The Kalman-Filter design is similar to the one used in .
Minimizing friction was a top priority during the design and manufacturing of FINGER. This goal guided the mechanism design, manufacturing process, and the selection of bearings and actuators. Figure 10 shows the static friction force for one of the 8-bar mechanisms as a function of actuator stroke. These static friction forces were determined experimentally as the force required by the actuator to move the mechanism from a rest position. Because the position dependency in the static friction is minimal, the average static friction force (0.0137 ± 0.0015 N SD) was used to construct a feedforward friction compensator. Assuming a simple Coulomb friction model, the compensator adds this average friction force along the direction of the estimated velocity. To prevent chattering, the compensator only applied the static force after a minimum velocity magnitude is achieved (see the dotted box in Figure 11).
Figure 11 shows the block diagram of the robot control system. The trajectory planning block includes the computer game with predefined desired trajectories, sent to the robot controller. Based on therapeutic preferences, different games can be used or designed as an interface between the subjects and the robot. For each game, the subjects are instructed to move the robot that is attached to their fingers according to the tasks dictated by the game. The robot moves by the combination of subject and actuator forces (Figure 11). The actuator force is a function of the controller type; hence, the controller structure determines how the subject will be assisted by the robot. Various controller types with different characteristics have been used in assistive devices to fulfil different therapeutic hypotheses . The controller used for the testing described herein is a linear Proportional-Derivative (PD) controller, whose gains vary during the gameplay according to an algorithm that will be described in the following sections.
Pilot testing with individuals with stroke
Eleven male and five female volunteers with stroke related motor impairment on the right side participated in the study. The average age of the subjects was 57.8 +/− 12.5 SD and they were 3.3+/−1.8 SD years post stroke. Eight subjects reported that their stroke was ischemic; three reported that their stroke was haemorrhagic; and five did not know. Level of impairment was assessed using both the upper extremity Fugl-Meyer (FM) test and the box and blocks (BB) test [31, 32]. For the FM test, a trained therapist asked subjects to perform 33 test movements and scored them 0 (can’t do), 1 (can do partially), or 2 (can do), then summed the scores. For the BB test, subjects moved as many blocks as possible over a divider in a one minute period. The average FM scores for the group were found to be 41.6 ± 15.8 SD out of 66, and average BB scores were found to be 25.1 ± 21.9 (compared to a score of 75.2 ± 11.9 reported in literature for healthy subjects) . Based on these scores, nine of the subjects were classified as highly impaired (FM < 40 & BB < 20), and the remaining seven subjects were classified as moderately impaired. For comparison, four healthy subjects (3 male/1 female, average age 33.5 ± 9.4 SD) were also included in the study. All subjects provided informed consent, and all procedures were approved by the institutional review board at U.C. Irvine.
Therapeutic game play
To demonstrate its potential as both a rehabilitative tool and a platform for exploring the factors that promote functional recovery, FINGER was used to test the hypothesis that subjects will be most engaged in the rehabilitation therapy presented to them when they are at their optimal challenge level. To test this hypothesis, FINGER was used to assist subjects in playing a custom-designed game similar to Guitar Hero®, which is the third largest video game franchise in history (Additional file 1). Prior to gameplay, the subjects were asked to put their hand in FINGER, and the proximal and middle phalange attachment points were adjusted to finger size until the subjects were able to comfortably curl their fingers through the full range of motion. Additional support under elbow was provided as needed to put the subject’s arm and wrist in a comfortable orientation. This game requires subjects to play along with a song by attempting to hit notes streaming down a visual display as shown in Figure 12. In order to hit these notes, the subjects were required to flex their fingers to a desired angle and stop at the correct time while receiving performance-based assistance from the robot. During the game, subjects were presented with three types of notes corresponding to flexion of the index finger, the middle finger and both fingers together. After successfully hitting a note, the subjects were required to extend their fingers back to a neutral position before the game would credit them with hitting future notes. During extension to the neutral position, subjects received the same amount of assistance as they received during flexion. While subjects attempted to flex their fingers to the correct positions, small dark balls hovering above the fretboard were displayed to provide the subjects with visual feedback of their finger position (see Figure 12). The song used in this experiment was “Happy Together” by the Turtles, and it required 104 notes to be hit over the course of a 160 second game. Timing of the notes was the same during all the experiments and only the level of assistance changed to modulate subjects’ success rates as will be discussed later. Portions of this experiment have been published in conference paper format .
Success rate algorithm
During the game, FINGER was used to both assist the subjects in completing the desired task and to monitor their performance. Although FINGER can be operated under a variety of control paradigms, this experiment used a PD controller whose gains were intermittently updated by an algorithm which attempts to control the subjects’ probability of hitting notes successfully . Our contention is that by controlling subjects’ success rate, we will be able to control their challenge level. According to the challenge point framework (CPF), determining the optimal challenge level is crucial to optimality of motor learning, particularly in rehabilitation . CPF states there is an ideal amount of information which when presented to the learner will optimize the learning process. In other words, to achieve the best learning rate, the task shouldn’t be too easy or too difficult. This ideal amount of information varies with the skill level of the learner. By controlling the controller gains, we can control the game difficulty and hence the level of challenge the subjects experience, regardless of their impairment level” with “ By changing the feedback gains, we can control the game difficulty and hence the level of challenge the subjects experience, regardless of their impairment level.
Determining the optimal challenge point for a particular task is difficult because it requires measuring long-term learning at a variety of challenge levels in a large number of subjects. However, one determinant of the optimal challenge point is likely effort – i.e. the more engaged a subject is, the more learning will likely occur. Effort can be measured in real-time and thus has the potential to serve as a means to identify when conditions are at least partially conducive for learning. Thus, we studied how effort, quantified by how much force the subjects exerted during the game (see below), varied with success rate.
The success rate algorithm mentioned above works as follows: For each successful note, the algorithm reduced the gains on the corresponding finger by an amount ρ, and for every missed note the gains on the corresponding finger were increased by an amount α∙ρ. As shown in , this simple algorithm eventually forces the subjects’ probability of success to converge on a value dependent only on α as shown in Equation 8 below.
Subjects were seated in front of a visual display, and the proximal and middle phalanges of their index and middles fingers were securely attached to the end effectors of the FINGER robot. Subjects were then instructed how to play the game and were asked to familiarize themselves with the task by playing through a song at a success rate of 75%. Data from this initial trial were excluded from the final analysis.
After the familiarization task, the robot was used to measure the subjects’ range of motion and maximum isometric force in both flexion and extension. Measurements were taken from the index and middle fingers both individually and together. These measurements were repeated at the end of the experiment. Then subjects were asked to play through the same song twice at each of the three randomly presented success rates (50%, 75%, and 99%).
On a randomly selected subset consisting of roughly 15% of the notes in every song, the robot’s gains were set to a fixed value and the robot was used to block the subject’s movements instead of assisting them. During these blocked trials, the amount of force exerted against the robot was taken as a measure of the subject’s effort in the task. Subject performance during these trials was not used to adapt the robot’s gains, and once the blocked notes passed the control gains were returned to their previous values.
The instantaneous success rate at each note was calculated by dividing the number of successful trials within a moving window containing the 25 preceding notes by the size of the window. The peak force applied against the robot during blocked trials was used to quantify subject effort by normalizing it to the subject’s maximum force for the corresponding finger as measured during isometric trials. An unbalanced 2 factor mixed measures ANOVA with repeated measures applied to the success rate variable was used to test the effects of success rate and impairment level on subjects’ effort.
During blocked notes for the index and middle fingers, the robot restricted the motion of both the correct and the incorrect fingers. An estimate of finger individuation was thus obtained by comparing the force generated by the finger that was supposed to move to the force generated by the finger that was not. Forces measured from both fingers were first normalized by their corresponding maximum force values from isometric trials. A measure of individuation was then calculated by dividing the average maximum normalized force applied by the incorrect finger by that of the correct finger. For blocked notes in which the force applied by the incorrect finger was greater than 1.25 times the force applied by the correct finger, it was assumed that the subject accidentally tried to hit the wrong note. Similarly, for trials in which the subjects did not apply any measurable force with either finger, it was assumed the notes were completely missed. These blocked notes were not included in the individuation analysis. An unbalanced three factor mixed measures ANOVA with repeated measures on the finger variable and the success rate variable was used to determine whether finger, success rate, or impairment level had any significant effect on the subject’s individuation value.
Average probability of success in hitting correct notes during gameplay versus time for the sixteen impaired and the four healthy subjects is shown in Figure 13. At the desired success rates of 50%, 75% and 99% the impaired subjects converged to the average actual success rates of 47.7+/−9.6%, 73.8+/−7.1%, and 97.6+/−1.9%. However, the unimpaired subjects converged to the average actual success rates of 72.2+/−19.5%, 79.3+/−4%, and 99+/−1.1%. This result shows that the algorithm explained in 4.3 is successful in assisting subjects to achieve a desired success rate. It is not surprising that the healthy subjects could achieve success rates higher than algorithm’s desired success rate, because the algorithm doesn’t prevent subjects from hitting more correct notes than desired. In order to effectively challenge the unimpaired subject, the algorithm would need to have been able to make the game more difficult than it would naturally be with the assistance turned completely off. This is not necessary for the impaired subjects, whose reduced neuromuscular ability provided the increased difficulty.
We also measured how success rate and impairment level affected the subjects’ effort while playing the game. Success rate was found to have a significant effect on subjects’ effort (p = 0.0024, degrees of freedom = 2). The effects of impairment level on effort, approached but did not achieve significance (p =0.0785, degrees of freedom = 2). As shown in Figure 14, effort decreased when subjects’ success rate increased.
Figure 15 shows the effects of impairment level and the finger being used on finger individuation. Both the finger being used and impairment level were found to have a significant effect on finger individuation (p = .0001, degrees of freedom = 1 and p = .0062, degrees of freedom = 2, respectively). As can be seen in Figure 15, individuation scores of the index finger were consistently better than those of the middle finger. This means that when the subject tried to move the index finger, he was more successful at moving the index finger only, as compared to when he tried to move the middle finger. Success rate was not found to have a significant effect on finger individuation, and so we combined data across success levels, resulting in Figure 16, which shows the effect of subjects’ impairment level on finger individuation. Subjects’ with higher impairment had lower individuation ability. The ability to individuate the index finger was higher than the ability to individuate the middle finger.
This paper described the design and preliminary evaluation of FINGER (Finger Individuating Grasp Exercise Robot). FINGER makes use of individual single degree-of-freedom 8-bar mechanisms to assist patients in making a naturalistic grasping motion with different fingers, together or separately. The kinematic and mechanical design work was guided by the overall goal of creating a robot with high-control fidelity as an instrument for testing and implementing the widest possible range of control strategies. Thus, we paired the lightweight, low-friction mechanism with high-speed and un-geared linear actuators. The resulting robotic mechanism has a closed loop frequency response of approximately of −3 dB at approximately 8 Hz. The fast speed and frequency response of FINGER make it a good candidate for evaluating control algorithms and therapy tasks that require fast movements and/or precise timing.
Another unique feature of FINGER is that, in contrast to most exoskeleton designs that attempt to align the joints of the robot with the joints of the body, the joints of the 8-bar mechanisms of FINGER are kept to the back of the hand and wrist throughout the curling motion. This facilitates easy attachment to the user and stacking of the mechanisms for multiple fingers, and allows for the possibility of applying sensory stimuli to the volar surface of the palm, for example by having individuals grasp real objects with assistance from FINGER.
The physical parameters of the 8-bar mechanism were determined through a mechanism synthesis process that achieved desired end-effector locations using cost function minimization. Four different sets of desired end-effector locations were created to generate a mechanism that could be easily adjusted to accommodate four different hand sizes. Using low-friction bearings and a balanced joint design, we were able to achieve smooth, low-friction 8-bar mechanisms that are easily backdriven. Further design features include finger-to-finder width adjustment, finger length adjustment, and wrist alignment.
Future upgrades to FINGER are currently under development. Possible upgrades and improvements include adding direct force sensing and impedance control, implementing unstructured “assist-as-needed” adaptive control, and adding a thumb exoskeleton mechanism.
The preliminary tests of FINGER showed that it can allow individuals with a range of impairment levels to play an engaging video game similar to Guitar Hero®. We used FINGER with a simple gain-adaptation algorithm to test the hypothesis that we can assist subjects as needed in achieving predefined success levels at the game, which we confirmed. We also found that the effort of both high level and low level subjects decreased when their success rate increased; this is consistent with previous observations of slacking when a robotic device over-assists its user [9, 10].
According to CPF, there is an intermediate success rate in which learning is maximal. We do not find a success level at which effort was optimal. One possibility is that effort may not decrease unless success is below 50%, the lowest level we tested. Determining the relationship between measures of effort and the optimal challenge point is an important direction for future research.
These tests also demonstrated the ability of FINGER to quantify finger individuation. Using measurements during blocked trials based on patients’ force applied by the wrong finger, we found that patients with higher impairment levels individuated less than those with lower levels of impairment. This result supports the findings in the previous literature on individuation that found that stroke reduced the ability to perform selective individuated finger motions, and specifically that the independence of the middle finger is more impaired than that of the index finger [36, 37]. A significant result is that we were able to quantify individuation during the normal course of game play of the game similar to Guitar Hero®. The possibility of generating quantitative measures of movement ability while therapy is delivered may increase the frequency at which these measures can be obtained .
The results of the preliminary tests with FINGER demonstrate its unique capabilities to study and implement finger therapy after stroke. Additional testing with FINGER may add insight to the effects of success rate on motor learning and finger movement recovery. We also plan to further explore the mechanisms of finger individuation in subjects with impairment due to stroke. Such knowledge will guide the use of FINGER for post-stroke movement therapy.
aGuitar Hero® is a trademark of Activision Publishing, Inc.
Roger VL, Go AS, Lloyd-Jones DM, Adams RJ, Berry JD, Brown TM, Carnethon MR, Dai S, de Simone G, Ford ES: Heart disease and stroke statistics—2011 update1. Circulation 2011, 123: e18-e209. 10.1161/CIR.0b013e3182009701
Broeks J, Lankhorst G, Rumping K, Prevo A: The long-term outcome of arm function after stroke: results of a follow-up study. Disabil Rehabil 1999, 21: 357-364. 10.1080/096382899297459
Kwakkel G, Kollen BJ, Krebs HI: Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabil Neural Repair 2008, 22: 111.
Prange GB, Jannink MJA, Groothuis CGM, Hermens HJ, IJzerman MJ: Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke. J Rehabil Res Develop 2006, 43: 171-184. 10.1682/JRRD.2005.04.0076
Lum PS, Burgar CG, Shor PC, Majmundar M, Van der Loos M: Robot-assisted movement training compared with conventional therapy techniques for the rehabilitation of upper-limb motor function after stroke. Arch Phys Med Rehabil 2002, 83: 952-959. 10.1053/apmr.2001.33101
van der Lee JH, Snels IAK, Beckerman H, Lankhorst GJ, Wagenaar RC, Bouter LM: Exercise therapy for arm function in stroke patients: a systematic review of randomized controlled trials. Clin Rehabil 2001, 15: 20. 10.1191/026921501677557755
Platz T: Evidence-based arm rehabilitation - a systematic review of the literature. Nervenarzt 2003, 74: 841−+.
Rossini PM, Dal Forno G: Integrated technology for evaluation of brain function and neural plasticity. Phys Med Rehabil Clin N Am 2004, 15: 263-306. 10.1016/S1047-9651(03)00124-4
Wolbrecht ET, Chan V, Reinkensmeyer DJ, Bobrow JE: Optimizing compliant, model-based robotic assistance to promote neurorehabilitation. Neural Syst Rehab Eng, IEEE Trans 2008, 16: 286-297.
Israel JF, Campbell DD, Kahn JH, Hornby TG: Metabolic costs and muscle activity patterns during robotic-and therapist-assisted treadmill walking in individuals with incomplete spinal cord injury. Phys Ther 2006, 86: 1466. 10.2522/ptj.20050266
Lotze M, Braun C, Birbaumer N, Anders S, Cohen LG: Motor learning elicited by voluntary drive. Brain 2003, 126: 866-872. 10.1093/brain/awg079
Kaelin-Lang A, Sawaki L, Cohen LG: Role of voluntary drive in encoding an elementary motor memory. 2005, 93: 1099-1103.
Marchal-Crespo L, Reinkensmeyer DJ: Review of control strategies for robotic movement training after neurologic injury. J Neuroeng Rehab 2009, 6: 20. 10.1186/1743-0003-6-20
Cai LL, Fong AJ, Otoshi CK, Liang Y, Burdick JW, Roy RR, Edgerton VR: Implications of assist-as-needed robotic step training after a complete spinal cord injury on intrinsic strategies of motor learning. J Neuroscience 2006, 26: 10564-10568. 10.1523/JNEUROSCI.2266-06.2006
Colombo R, Pisano F, Micera S, Mazzone A, Delconte C, Carrozza MC, Dario P, Minuco G: Robotic techniques for upper limb evaluation and rehabilitation of stroke patients. Neural Syst Rehabil Eng, IEEE Trans [see also IEEE Trans Rehabil Eng] 2005, 13: 311-324. 10.1109/TNSRE.2005.848352
Krebs HI, Palazzolo JJ, Dipietro L, Ferraro M, Krol J, Rannekleiv K, Volpe BT, Hogan N: Rehabilitation robotics: performance-based progressive robot-assisted therapy. Auton Robot 2003, 15: 7-20. 10.1023/A:1024494031121
Riener R, Lunenburger L, Jezernik S, Anderschitz M, Colombo G, Dietz V: Patient-cooperative strategies for robot-aided treadmill training: first experimental results. Neural Syst Rehabil Eng, IEEE Trans [see also IEEE Trans Rehabil Eng] 2005, 13: 380-394. 10.1109/TNSRE.2005.848628
Kahn LE, Rymer WZ, Reinkensmeyer DJ: Adaptive assistance for guided force training in chronic stroke. Engineering in Medicine and Biology Society, 2004 EMBC 2004 Conference Proceedings 26th Annual International Conference 2004, 1: 2722-2725.
Nowak D: The impact of stroke on the performance of grasping: usefulness of kinetic and kinematic motion analysis. Neurosci Biobehav Rev 2008, 32: 1439-1450. 10.1016/j.neubiorev.2008.05.021
Balasubramanian S, Klein J, Burdet E: Robot-assisted rehabilitation of hand function. Curr Opin Neurol 2010, 23: 661-670. 10.1097/WCO.0b013e32833e99a4
Cutkosky MR, Kao I: Computing and controlling compliance of a robotic hand. Robotic Autom, IEEE Trans 1989, 5: 151-165. 10.1109/70.88036
Taheri H, Rowe JB, Gardner D, Chan V, Reinkensmeyer DJ, Wolbrecht ET: Robot-assisted guitar hero for finger rehabilitation after stroke. Engineering in medicine and biology society (EMBC), 2012 annual international conference of the IEEE; aug. 28 2012-sept. 1 2012 2012, 3911-3917.
Wolbrecht ET, Reinkensmeyer DJ, Perez-Gracia A: Single degree-of-freedom exoskeleton mechanism design for finger rehabilitation. IEEE Int Conf Rehabil Robot: [proceedings] 2011, 2011: 5975427.
Heisele B, Kressel U, Ritter W: Tracking non-rigid, moving objects based on color cluster flow. Computer vision and pattern recognition, 1997 proceedings, 1997 IEEE computer society conference on; 17–19 jun 1997 1997, 257-260.
Book NAS: Volume II: a handbook of anthropometric data. NASA Reference: Publication; 1978:1024.
Serban I, Baritz M, Rosca IC, Cotoros LD: Statistical Analysis of Anthropometric and Physiologic Performance of the Hand. IFMBE Proceedings Volume 36. In International Conference on Advancements of Medicine and Health Care through Technology. Edited by: Vlad , Simona , Ciupa , RaduV . Berlin Heidelberg: Springer; 2011:380-383. Chapter: 80
Dreyfuss H, Tilley A: Measure of man and woman: human factors in design. New York City, NY: Watson-Guptill Publications; 1993.
Sandor GN, Erdman AG: Advanced mechanism design: analysis and synthesis: vol. 2. Englewood Cliffs: NJ, Prentice-Hall; 1984.
Soh GS, McCarthy JM: The synthesis of six-bar linkages as constrained planar 3R chains. Mech Mach Theory 2008, 43: 160-170. 10.1016/j.mechmachtheory.2007.02.004
Wolbrecht ET, Reinkensmeyer DJ, Bobrow JE: Pneumatic control of robots for rehabilitation. Int J Robot Res 2010, 29: 23-38. 10.1177/0278364909103787
Fugl-Meyer AR, Jaasco L, Leyman L, Olsson S, Steglind S: The post-stroke hemiplegic patient. Scand Journal Rehab Med 1975, 7: 13-31.
Mathiowetz V, Volland G, Kashman N, Weber K: Adult norms for the box and block test of manual dexterity. Am J Occup Ther 1985, 39: 386-391. 10.5014/ajot.39.6.386
Taheri H, Rowe JB, Gardner D, Chan V, Reinkensmeyer DJ, Wolbrecht ET: Robot-assisted guitar hero for finger rehabilitation after stroke. San Diego, CA: Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE; 2012.
Spencer SJ: Movement training and post-stroke rehabilitation using a six degree of freedom upper-extremity robotic orthosis and virtual environment. Irvine: University of California; 2012.
Guadagnoli MA, Lee TD: Challenge point: a framework for conceptualizing the effects of various practice conditions in motor learning. J Motor Behav 2004, 36: 212-224. 10.3200/JMBR.36.2.212-224
Lang CE, Schieber MH: Differential impairment of individuated finger movements in humans after damage to the motor cortex or the corticospinal tract. J Neurophysiol 2003, 90: 1160-1170. 10.1152/jn.00130.2003
Lang CE, Schieber MH: Reduced muscle selectivity during individuated finger movements in humans after damage to the motor cortex or corticospinal tract. J Neurophysiol 2004, 91: 1722-1733. 10.1152/jn.00805.2003
Zariffa J, Kapadia N, Kramer J, Taylor P, Alizadeh-Meghrazi M, Zivanovic V, Albisser U, Willms R, Townson A, Curt A: Relationship between clinical assessments of function and measurements from an upper-limb robotic rehabilitation device in cervical spinal cord injury. Neural Syst Rehabil Eng, IEEE Trans 2011, 341-350.
The authors gratefully acknowledge the support for the work described herein by NIH-R01HD062744 from the National Center for Medical Rehabilitation Research at the National Institute of Child Health and Human Development, and the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant UL1 TR000153. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
The authors declare that they have no competing interests.
All authors contributed to draft and review the manuscript. ETW, HT, DG, KG and CB contributed to the development and manufacturing of the robot. HT, JBR, DJR and ETW implemented the control design, mechatronics and the game interface. JBR, HT, VC, DG, KG, DJR and ETW carried out the testing of the platform. JBR, DJR, ETW and HT contributed to data analysis. DJR and ETW contributed in the supervision of the research study. All authors read and approved the final manuscript.
Electronic supplementary material
Additional file 1: A video of FINGER being tested. A lady suffering from stroke is playing the custom designed game similar to GuitarHero® using FINGER. The video game and robot assembly can be seen in the video from two different angles. The MPG video clip can be played using standard video players such as Windows Media Player. Informed consent to publish the video was obtained from the subject. (MPEG 20 MB)
About this article
Cite this article
Taheri, H., Rowe, J.B., Gardner, D. et al. Design and preliminary evaluation of the FINGER rehabilitation robot: controlling challenge and quantifying finger individuation during musical computer game play. J NeuroEngineering Rehabil 11, 10 (2014) doi:10.1186/1743-0003-11-10
- Robotic rehabilitation
- Motor control
- Mechanism synthesis
- Finger individuation
- Color-based motion capture