Keywords

1 Introduction

Hands, as the most dexterous part of our body, are of vital importance to our everyday life. However, since hands are extensively used in nearly all tasks, they are exposed in more dangerous environment than any other parts. Overwork, injury and geratic complications, such as stroke can all cause hand afunction, totally or partially, which directly diminish the quality of life. According to the statistics from World Health Organization (WHO) 2014, rehabilitation and palliative care has not been given enough attention and essential basic equipment is still not available everywhere, not only in most developing countries, but also in some developed countries [1], thus many patients of hand afunction cannot be provided prevention, diagnosis or treatment.

In addition, unlike other diseases, hand impairment and afunction require sufficient patient’s exercise besides doctor’s treatment. Exact motion exercise can profoundly help with patient’s rehabilitation [2]. However, currently there is no other method than doctor’s judgment telling patients what to exercise and how they exercise, and patients only exercise their finger and thumb by repeatedly fisting or picking up small items during their long time staying at home, without doctor’s instructions. Thus, there is a potential demand of designing a hand rehabilitation system to utilize patients’ time at home to exercise accurate motion training.

The key procedure of designing such an exercising system is to reproduce the training atmosphere in hospital for patients who need hand rehabilitation [3]. Previous studies [4, 5] have shown that following and imitating are effective in motivating training and increasing exercising accuracy. We used the user-centered design approach to innovate an interactive rehabilitation system for patients to encourage their exercises at home and meanwhile provide them with direct instructions. Their finger movements above the Leap Motion sensor are tracked and projected to a computer screen where they can see standard motions and their own motions at the same time. The Leap Motion-based rehabilitation system, both easy and efficient to exercise, is considered to be very helpful in motivating patients to exercise at home as well as correcting their inaccurate motions.

This article firstly introduces an analysis of interviews we conducted with patients, doctors. We collected their opinions and preferences for both real and interactive exercise to generate a list of requirements which is used in the later design of the system. Shortcomings of current indoor sports equipment are summarized with a brief introduction of the Leap Motion-based system in Sect. 4. A detailed illustration of the design of the rehabilitation system is then presented. Some improvements based on a usability test are suggested. Discussion and conclusions are summarized in the last section.

2 Background and Related Work

Generally, hand rehabilitation, which focuses on recovery of both strength and nimbleness [2, 3], causes for professional training in rehabilitation center and long-term persistent self-exercise. According to present medical care situation in China, most patient can only afford one or two treatments in hospitals per week, which is considerably insufficient for rehabilitation. Improving the efficiency of patients’ self-exercise at home is a key method to reduce rehabilitation time and increase rehabilitation effect.

The human-computer interaction community has made attempts and carried out researches to enhance hand rehabilitation with HCI technologies. Khademi, M. et al. [6] modified the game “Fruit Ninja” with Leap Motion controller for patients with stroke to practice arm and hand. Boulanger et al. [8] built a game, as well, using Microsoft Surface’s hand position. These games and researches involve new or existing interaction games and adapt them into hand control mode to encourage rehabilitation. Another example of hand rehabilitation is an interactive glove proposed by Hallam et al. [9] to help patients with stroke with hand reuniting. There is also another glove-based treatment system called HandTutor [10] focusing on finger rehabilitation. The above researches show examples about tangible and intangible interaction rehabilitation designs. Yet, rehabilitation system using free-hand interaction for patients’ self-exercise and introducing doctor’s instruction still remains to be investigated.

3 User Requirement Gathering: Interview

System design was started by an interview of 4 doctors and 8 patients, who need hand rehabilitation, in a Grade III-A Hospital in Beijing, China, which is a top grade hospital according to Chinese grading system and runs fully functional rehabilitation center.

3.1 Interview with Doctors

To study the general procedure of hand rehabilitation and know better about hospital’s demand for interactive system, we conducted a series of interviews with 4 doctors of rehabilitation in January 2014. 3 diagnosticians and 1 physical therapist participated to answer a list of 8 questions about their current treatment procedure and concerns they had when carrying out therapeutic scheme. They are each in charge of diagnosing injured parts and severity, designing rehabilitation project, and assisting rehabilitation training.

During the interviews, we mainly investigated their common cases, general procedure, usual training motions and equipment, and opinions about self-exercise at home. Since doctors’ instructions will be introduced into this rehabilitation system, we also surveyed their preference about operation process and interface about online system. Additional concerns about rehabilitation they had were also included in generating a list of requirements we need to consider in designing the system.

They introduced the whole process of rehabilitation and gave their ideas about in what stage they hope auxiliaries can help. They also showed around the rehabilitation center about the common equipment and standard training plans. Besides, they stated that, according to their clinical experience, flexibility recovery was far more important than strength recovery, which most rehabilitation equipment failed to help, and that they regarded patients’ time at home was not effectively used for rehabilitation, because they could only repeat simple motions, such as clenching fist, for fear that they exercised wrong without professional instructions and feedback. As for the system design, they mentioned the existing prescription system since they were quite familiar to that and suggested the rehabilitation system to follow that procedure and be added to the existing system.

3.2 Interview with Patients

We also randomly recruited 8 patients in the rehabilitation center who were under treatment of those doctors. The 8 patients—consisted of two elderly people suffering from geratic complications and six other patients who injured their hands.

During the investigation, we asked about their rehabilitation circumstances, including the duration and frequency they came to hospital, their exercise at home, and their attitude towards independent training and assisting auxiliary. This investigation consists of more than 16 open-ended questions. Since the online system requires user to operate by gestures, we also surveyed their IT background and experience of using computers and interactive devices. Additional concerns about rehabilitation they had were also included in generating a list of requirements we need to consider in designing the system.

As identified, most patients regard their exercise at home useless and usually cannot maintain a tight and long-term exercise schedule because they do not set regular time for simple training and often forget to exercise daily. Instead of exercise at home, they trust doctors much more, but most of them cannot afford a whole course with doctor’s instruction. Generally, they spent the first few weeks at hospital, having professional training every day, yet after they rehabilitate to some degree, they move back home and visit hospital one or two times every week considering time and expense. All of them take positive attitude towards self-exercise equipment, saying that they would like to try such system if doctor permits. As for the type of training, they claimed that they prefer a series of training lessons which each focus on one part of hand, so that they can pertinently exercise according to their disease. When the interface and operation design was mentioned, they worried about whether their IT background could handle the system, since they may cannot use their finger flexibly and operate the mouse. They further added a requirement to play some similar game because they hope to increase entertainment.

3.3 User Requirements

Results from the interview shed light on the demand of a new system for online hand rehabilitation with doctors’ instructions. All subjects showed interests in the idea we put forward about a series motions for exercises at home and were willing to have a try. They expected to have an interaction device of good usability, great efficiency, high safety, and social connectivity at an affordable price. A list of user requirements for the hand rehabilitation is summarized as below.

  • The system should have adequate but accurate exercises to be effective, but the duration of each exercise motion should be limited to avoid overtiredness.

  • The system should be easy to start with and self-explainable. It should be easy for elderly people and people without abundant IT knowledge to understand.

  • The system should apply gesture to interaction with computers considering the patients’ low level of control to their fingers.

  • The system should provide specific training motions for exercise need for different parts of hand. Different motions should enable patients to exercise different hand parts, especially their fingers.

  • The system should encourage patients to maintain a clear schedule by doing hand rehabilitation every day. And if patients fail to follow the schedule, the system should remind them and contact doctors to adjust their rehabilitation plan.

  • The system should give clear feedbacks to patients for them to correct their gestures and improve, and record their performance for themselves and their doctors to check.

  • The system would be better if social connectivity is provided between patients.

  • The system should allow doctors to check their patients’ performance and give instructions easily and synchronously.

  • The system should provide convenient access for doctors to manage and monitor their patients.

4 Improvement of Existing Hand Rehabilitation Equipment

There are a large number of patients who need hand rehabilitation in the word today according to status from WHO, and attentions on equipment for that are still inadequate. Currently, products designed for hand rehabilitation are limited, which we proposed that with the help of some well-designed device and systems, patients can be instructed and encouraged to rehabilitate safely, correctly and, most important, efficiently by themselves.

4.1 Shortcomings of Existing Equipment

Current hand rehabilitation equipment, which consists of two types: wearable devices and interactive video games, both have their own focuses. Thus when used by doctors for assistance, they could have several inherent drawbacks as follows.

  • Wearable devices, such as hand dynamometer, and gloves (e.g. MusicGlove [12], Gloreha [11] as shown in Fig. 1) consist of mechanical parts and are fixed on hands by elasticity or else, which may all add extra compression and cause reinjury.

    Fig. 1.
    figure 1

    Product demos of musicglove snd gloreha

    Full size image

  • Interactive video games, for example Khademi’s Ninja game with Leap Motion controller, focus only on encouraging users to exercise their hand, paying no attentions on precise training motions, while nimbleness recovery which is of equal importance causes for precise exercise.

  • Both of these products are designed for patients to exercise all by themselves. However, doctors’ opinions and instructions may give much help for rehabilitation, which should be taken into account.

4.2 Improvement Using Leap Motion Controller

The Leap Motion controller is an excellent sensor which can tracking hands and fingers with high speed and precision [13]. It enables a series of operation in interaction with computers. These unique advantages could improve current hand rehabilitation products in following points.

  • The Leap Motion Controller enables doctors to record standard exercise motion beforehand by tracking their example motions through its camera and IR sensors without contact, which can be shown to patients and instruct their self-training.

  • Besides beforehand recording, the Leap Motion controller could also real-time process the data it tracks. By comparing real-time data with recorded data, users’ motions can be judged whether matching with the standard motions or not.

As a result, the Leap Motion controller has been chosen to realize most of the requirements doctors and patients proposed, because of its well performance in tracking users’ hand, as well as sending and processing real-time hand data. Users only need to move their fingers above the Leap Motion controller. The prototype system refers a set if sample codes in Leap Motion SDK, with 3D hands modeling and displaying.

5 Prototype Design and Development

5.1 Prototype Apparatus

The online hand rehabilitation system consists of four parts: users (include patients and doctors), a Leap Motion controller, a personal computer and a display screen. Connecting with the Leap Motion controller and display screen, the PC processes the real-time data of hands obtained from the Leap Motion controller and displays on the screen when the system is running. Patients only needs to sit in front of the PC and hold hand above the Leap Motion controller in order that his or her hands can be captured and tracked. Patients are able to wave his or her hands to select and click on the buttons presented on the screen and operate the system, as shown in Fig. 2.

Fig. 2.
figure 2

Prototype apparatus

Full size image

5.2 System Design

This rehabilitation system has special designs for patients with hand afunction to help them with hand rehabilitation and communicate with their doctors.

As the start of the whole rehabilitation process, doctor check patient’s hand situation and have a basic diagnosis. Then instead of writing a prescription and teaching patients how to exercise, doctor log on his or her account and add a new patient’s document under his account. (Of course, if this patient is come for a return visit, doctor can open his or her existing document and work on that instead of creating a new one.) Doctor needs to fill in patient’s basic information and diagnosis before he gives prescription. He can choose among all the exercise motions, which have been recorded by Leap Motion and saved in the system, for patients to exercise with instructions about frequency and duration, etc., like prescriptions about taking medicine. After all these operations end, doctor will generate an account for patient, with a unique username and password, and rent a Leap Motion controller to patients. The main interface is shown in Fig. 3, where doctors can add new patients and enter certain patient’s record. As for the patient’s record, the interface arranges as shown in Fig. 4 with patient’s basic information, diagnose, prescription and score for each exercise. Here, doctor can add comments as well.

Fig. 3.
figure 3

Doctor’s main interface

Full size image
Fig. 4.
figure 4

Patient’s record interface

Full size image

Then comes to the patient’s self-exercise. Patients can open the website and login with the username and password which is given by their doctors. After a basic calibration with Leap Motion, patients enter their homepage, as shown in Fig. 5 where they can choose to exercise with standard motion, view their score and play interactive games which encourage them to move their finger and wrist. When entering into the exercise part, as shown in Fig. 6, patients can see the standard motion, prescribed by their doctor, presents on the left for them to follow and at the same time, their real motion will present on the right with feedbacks showing where to improve. Moreover, a score based on the similarity between the standard motion and real one will appear on the left, for patients themselves and their doctor to estimate their rehabilitation situation.

Fig. 5.
figure 5

Patients’ homepage

Full size image
Fig. 6.
figure 6

Patients’ exercise page

Full size image

6 Prototype Evaluation: An Interview

In order to check whether this online system can satisfy patients’ and doctors’ demand and improve hand rehabilitation’s efficiency, a prototype system was developed and evaluated by 2 doctors in June 2014.

The evaluation interview consisted of three steps: a brief introduction to the usage and design of the system, a short operation trial for doctors and a follow-up interview to collect their feedback. Since we don’t have enough time to recruit more patients, this evaluation was carried out only with doctors.

According to doctors’ feedback, they both acknowledged that the system met their demand and they believed it to be useful for hand rehabilitation. Also, they liked the interface and structure of the website. Besides, they mentioned that the interaction with Leap Motion and computer was easy for ordinary person, but they were not sure whether patients with hand afunction could also interact in this way easily.

7 Future Plan and Discussion

Compared with current wearable devices and interactive video games for hand rehabilitation, this online system is better for patients in the following three aspects.

First, the online interactive system for hand rehabilitation add no extra compression to patients. Since patients who suffer from hand afunction have limited control to their finger and are easier to be wounded, wearable devices may accidentally hurt patients’ hands. However, using Leap Motion as a distant sensing device, patients are free to interact with the computer with their hand without touch, thus avoiding further damage.

Second, the online interactive system provides precise rehabilitation training, comparing with existing hand interactive game. Those task oriented game, for example, Fruit Ninja, usually set a task for players to finish, and the game only focuses on the whether the task is finished instead of how to finish it, thus cannot instruct accurate motion. However, this system teaches patients the rehabilitation motion and monitors their performance, at the same time, gives feedbacks for patients to correct their motion themselves. In this method, precise rehabilitation training can be realized.

Third, via the online system, doctors and patients can communicate at any time, instead of only during weekly visit back to the hospital. Patients’ video and score will be shared with their doctors and doctors’ comments and instructions appear on patients’ interface as well. This has not only increased efficiency of the rehabilitation, but also make full use of patients’ time at home.

We have to admit that because of time limitation, we didn’t carry out evaluation experiment with patients for long term test. Although the two doctors showed their great interest in this system after a brief introduction to this system, there may be some problems we cannot find until the patients try this. So for the next stage, we need to cooperate with some local hospitals and get more detailed feedbacks which can point to the directions we need to consider to further improve our rehabilitation system.

In conclusion, in this study, we carried out a “user-entered design” for patients in developing a Leap-Motion-based online interactive hand rehabilitation system. User requirements were collected from an interview with both doctors in hand rehabilitation and patients who need rehabilitation. Based on these requirements, a prototype system has been designed and developed with Leap Motion sensor. We briefly evaluated the system with an introduction and presentation to two relevant doctors, and furthermore, a more detailed experiment with patients need to be carried out. Compared with the existing wearable devices and interactive games, this online system performs better in safety, accuracy, efficiency in assisting patients with hand afunction in rehabilitation training.