Towards a low-cost interactive system for motor self-rehabilitation after stroke

- Virtual reality technologies have been experimented for several years for post-stroke motor rehabilitation, but there is too little diffusion of these systems among medical facilities and none among patients. Our objective is the development of an interactive sys-tem to assist motor rehabilitation of the upper limb after a stroke, which retains the medical beneﬁts of traditional post-stroke methods while reducing human costs (usable with minimal supervision) and materials (general public), and facilitating active patient participation. System architecture, 3D interactions and virtual content are based on an iterative, user-centered design methodology with patients and therapists. The system allows users to perform repetitive and intensive tasks with the upper limb. The paretic hand is tracked with a low-cost depth sensor. Kinematic performance is monitored and visual feedbacks are proposed. Preliminary tests were conducted on a non-immersive prototype, with eight patients and a target pointing task. The results showed good usability and high acceptance from the users.


I. INTRODUCTION
Stroke is a leading cause of disability worldwide.The hemiparesis of the contralateral upper limb is the most frequent motor deficit (9).Because of the aging population, the costs of stroke care will become a greater problem in the years to come (20).It is now well established that post-stroke motor recovery is activity-dependent and can be produced with training programs that are repetitive (large amount of movements), intensive (in terms of time and active participation of the patient) and goal-oriented (i.e.organized around clear and concrete objectives, rather than focusing solely on particular movements or muscles) (24).The greater part of recovery is reported to take place in the first three months following stroke (22), but progress can also be seen throughout the chronic phase (after six months) (23).Rehabilitation should then be done as soon as and as long as possible.
Nevertheless, effective therapy requires intensive therapist support, which can be expensive, and is often limited by the low compliance and lack of motivation of patients.That is why the last fifteen years have seen the emergence of innovative therapeutic approaches, notably based on robotics and Virtual Reality (VR) (14).Several clinical studies have demonstrated the benefits of these approaches in addition to conventional rehabilitation (9).
In this context, we propose a VR-based system for assisting motor recovery of the upper limb, which retains at least the medical benefits of traditional post-stroke methods, while reducing human and material costs and facilitating active patient participation.

II. POST-STROKE REHABILITATION OF THE UPPER LIMB USING VIRTUAL REALITY
Many studies have focused on the use of VR technologies for post-stroke motor rehabilitation.They are characterized by the use of specific interfaces to detect the movements of the patient.Data gloves or marker-based vision devices can be used to capture hands/fingers movements (3; 5) but these tracking technologies are invasive (1).They potentially increase the setup time, have problematics of hygiene and are sometimes difficult to wear because of the paretic arm and hand.Wiimote and 3D joysticks have also been tested (18), but they require gripping capabilities in patients that they not always have.Therefore the majority of solutions are today focused on video capture via mono-or stereo-scopic cameras (e.g.Kinect) (12; 19).Concerning the visual interfaces, there are multiple levels of immersion.CAVEs are only used in specific research cases because of the required space and price.HMDs are sometimes used for supervised rehabilitation but not when the patient is alone because it can be difficult to install with a paretic arm.The most conventional are still PC screens or projectors ranging from standard to large size.Finally, conventional speakers are usually added.An effective rehabilitation requires repetitive, intensive and task-oriented motor activities.This is why some previous works focus on a specific task such as: following a trajectory (21), grasping, placing an object on a shelf (21), reaching targets (4; 7), performing life-relevant tasks like cutting nails, buttoning tee-shirt (17) or pouring water into a glass (13).The conditions of the task are usually either strict exercises or games.When it is an exercise, the movements are designed to maximize recovery but are often boring.When it is a game, the task is more engaging but less effective for recovery because the movements performed are not always those recommended.The best activity is a com-promise between the two: "rehabilitation gaming", a series of engaging tasks specifically designed for recovery (16).
In terms of user interactions, patients can perform selections and manipulations: • in a 3D space (4; 7; 17; 19; 21): most common and transferable to the real life activities; • in a vertical plane (11; 13): no depth, requires strength to fight against gravity, can reinforce muscle; • in a horizontal plane (13): no height, the hand moves on a surface, less transferable but easier to practice for severe impaired patients.Feedbacks, as defined in motor learning theory, are an essential aspect of motor rehabilitation.Extrinsic (or "augmented") feedbacks are largely present in VR rehabilitation and focus on the "knowledge of results", the outcome of the movement (18): display of a score, remaining or elapsed time, sound effects to indicate success or failure, etc. Knowledge of performance (i.e. the kinematic quality of the movement) is not treated effectively although it seems essential and difficult to perceive in conventional therapies.For example, the visualization of the ideal trajectory (17; 21) helps patients to know if their movement is correct.The "force" of the movement can also be displayed (18).Concerning intrinsic feedback (which corresponds to the internal sensory information provided by vision, hearing and proprioception), it may be incomplete in non-immersive rehabilitation systems due to indirect visualization of the virtual movements, and latency.
An important feedback is the representation of the user in the virtual environment.For now, the impact of this avatar on embodiment feeling and on recovery is not completely understood, but kinesthetic illusion can potentially produce some human motor plasticity (cortical reorganization) (15).We can distinguish four types of representations: • real image of the patient (11): hypothesis is that the mirror neurons are activated when the individual is subjected to his own image and this would facilitate motor learning; • 3D character / avatar (18; 19): commercial games have by default this representation; • arms and hands only (4; 7); • abstract representation as a sphere, tool, ... (13; 17; 21).
In addition to all these elements prevalent in VR rehabilitation applications, some recent works have proposed methods to address specific problematics: using the relative position of the hand with respect to the trunk to avoid compensatory trunk movements (24), using visual movement amplification to counteract the learned non-use effect (suppress the use of the affected extremity) (2), integrating cognitive challenge to improve the global efficiency of motor rehabilitation (8), providing coaching instead of gaming to generate higher activity levels (6) or integrating dynamic difficulty adaptation to obtain a continuous challenge and contribute to the intensity and motivation (5; 10).
Although many encouraging results have been encountered on VR rehabilitation, VR is still not recommended by health agencies because of the small amount of patients involved in the studies and the high risk of bias (14).In addition, the proposed systems are rarely used by patients in practice, let alone at home.

Principles
Our objective is to develop a low-cost, intuitive, noninvasive, safe and clinically validated VR system to assist self-rehabilitation at home and counteract the learned noneuse effect with an active and regular practice of motor activity.The main components and architecture of our approach are depicted in Figure 1.They are based on core principles identified in the state-of-the-art, and medical recommendations for efficient recovery.Task In order to maximize recovery, the patient must perform a goal-oriented (focused on a goal and not only a movement), repetitive and intensive (in terms of activity time and challenge) task, involving movements of the paretic arm.The activity needs to be adapted to the patient's motor and cognitive abilities through various parameters, in particular levels of difficulty.

Feedbacks and Motor learning
The patient perceives intrinsic and extrinsic feedbacks which can promote motor learning.In particular, a synchronized avatar could reinforce functional recovery based on cortical plasticity (mirror neurons system).Other feedbacks could improve knowledge of performance (quality of movement) and give cues on how to improve it.
Monitoring The system monitors performance of the patient over time: success, failure, speed, fluidity, etc. Indicators are extracted from the measures and communicated to the therapist and the patient in the form of scores and graphs.In this way, the therapist can follow the patient's progress (possibly remotely) and improve the rehabilitation process with personalized recommendations and adjustments to the difficulty.The patient can also see and understand what aspects of the activity he/she is progressing and which ones still have to be worked on.
Motivation The first essential step of effective rehabilitation is adherence (or "compliance"): the extent to which a patient is properly following medical treatment.In order to improve motivation and commitment, it is essential to propose an interesting activity which gives the feeling of being useful and which is neither too difficult nor too easy.To do this we draw inspiration from the principles of (serious) games to design the task and the virtual environment.In particular, adaptive difficulty (based on calibration assessment) and virtual coaching are two ways to always challenge the patient.

Prototype
Our system runs on a standard PC.The patient sits in front of a table, his/her paralyzed arm is placed on the table.The virtual environment is displayed on a standard monitor.The tracking of the hands is based on a low-cost non-invasive IR sensor (Leap Motion, available for about $80 USD).The sensor is mounted upside down on top of the monitor in order to allow patients to keep or rest their hand on the surface if necessary.To prevent infrared reflections and increase the reliability of the tracking, a matt surface is placed on the table.The virtual reality application is designed using Unity 3D with C#.The hand tracking is based on Leap Motion SDK Orion 3.1.3.The user view of our final prototype is schematized in Figure 2.

Environment, Interactions and Feedbacks
We followed a multidisciplinary (VR researchers, neurologists, therapists and patients) and user-centered design process.Each iteration is the succession of identification, specification, development and evaluation phases.The main objective is to involve end-users (patients and therapists in our case) in the development process from the very beginning to take into account needs and constraints (cognitive impairments for example) and to make usability tests regularly.
Figure 3: The virtual environment for the pointing task.Patient has to reach 3D targets (white) with his/her virtual realistic hand, and go back to a rest area (blue).Time and score are displayed.
The first activity we designed is an uni-manual and repetitive pointing task, which is one of the major tasks in the rehabilitation of the upper limb.The therapeutic goal is focused on the elbow and shoulder movements, without particular movement of the fingers.Patients can make the connection with a task of everyday life (designating an object).Figure 3 illustrates the virtual environment of the task.
Task The goal is to perform hand movements between a rest zone and randomly selected virtual targets.To succeed, the patient must reach the target proposed by the system in a predefined maximum time and then return to the rest zone (no time limit for return).The rest area has the shape of a parallelepiped to facilitate people to rest in the most comfortable position (often with their elbow on the table) whatever their dominant hand.A new objective is then highlighted.If the time is exceeded, it is a failure for this target.The exercise continues until a predefined number of targets have been proposed.The goal is to succeed as much as possible in a minimum time.Since stroke is a source of fatigue, the user can manually stop the activity.When the hands are not detected for two seconds, the application pauses automatically.
Targets We designed a pattern of nine 3D targets, defined by two angles (horizontal (xz) and vertical (yz) deviations) and a distance from the center of the rest area.These targets are oriented towards the middle of the rest zone to encourage patients to follow straight trajectories and avoid bias in the trajectory strategy.The targets were also modeled to avoid trajectory bias.The form of buzzers intuitively suggests a linear displacement of the moving part whatever the angle of approach of the hand.Their form can also be used to test the strength and accuracy of reaching the end point, thus bringing a bit of playful motivation ("the more you press the button, the more points you get").

Measures
The system tracks the position of the hand over time and computes multiple parameters: the mean time to reach a target, the mean velocity, the number of success and failures, the number of pauses as well as "attempts".An "attempt" corresponds to a beginning of motion without success, and is detected when the hand leaves the rest zone.This allows us to differentiate a failure with or without an effort.This is also supposed to limit discouragements associated with failures.
Virtual environment and feedbacks The depth perception of 3D environment on a standard monitor is difficult and impacts the performance of 3D interactions.In order to increase the depth perception, the virtual environment is delimited by graduated walls which also represent the tracking volume of the IR camera.Occlusion, lighting and shadows are used to improve the perspective effect.The application should be usable simply without long explanations, and accessible to patients who may have cognitive impairment.For this, the environment is sober and simple semantic colors are used (white = neutral, blue = object to reach, green = success).The elapsed time, and the number of proposed targets in relation to the objective of the exercise, are displayed at the top-center of the screen.This position takes into account the potential patients with hemispatial neglect.
Avatar The virtual hand is displayed on the screen.We have chosen a realistic, non-animated avatar in a horizontal rest position.Despite the ability of our sensor to precisely measure the position of the fingers, we chose not to display the hands of the patients identically.These hands being often deficient and retracted, this is supposed to give a more positive perception to patients.For the same reason, the movements of the virtual hand are synchronized with the movements of the real hand only along the three axes of translation and the vertical axis of rotation.The hand is opaque but the arm is not visible to reduce occlusion with targets.We expect that these choices could induce a partial embodiment effect that can be beneficial for cortical reorganization.

IV. PRELIMINARY RESULTS AND DISCUSSION
Preliminary tests (Figure 4) were conducted with patients at the Neurorehabilitation Unit at Les Trois Soleils Rehabilitation Center (Boissise le Roi, France) to evaluate the usability of the system and the pointing exercise.Eight naive participants participated in this study (5 male, 3 female; aged 26-77, mean=51.2;6 with right hand paresis; seven patients in chronic phase -more than 6 months after stroke -including one patient with sickle cell anemia and one patient with a cervical spondylotic myelopathy).The activity was performed in the presence of a therapist and filmed to identify The participants were asked to reach 40 times randomly selected targets among 9, in a minimum amount of time but with maintaining a right posture and correct movement.All subjects quickly understood how to use the system and the purpose of the activity.They completed the exercise between 2 and 5 minutes.6 participants were completely successful (40 targets reached), 1 failed completely (insufficient mobility) and 1 was moderately successful (75% of targets reached).
Our results suggest that the system is usable in partial autonomy by a patient to practice rehabilitation of the upper limbs after a stroke.Our observations during the experiment show that the activity does not require therapeutic accompaniment to ensure the safety, understanding and use of the system.Even though the system does not yet have specific components for motivation, user acceptance of the system was high and patients were highly committed.For example, one patient said it was "fun" and "like a video game".This may be due to the "wow" effect of new technologies (3d hands recreated as if by magic).Another one is ready to re-use the system with the current task.One is even ready to buy the whole system for less than 100 euros if it were marketed.Some therapists also commented on the system.They found it "simple and efficient", "easy" and "quick to install and understand".Generally, patients and therapists find the system pleasant, easy to use, simple and interesting.
As for the patient who was moderately successful, we speculate that this was mainly related to fatigue due to her cervical spondylotic myelopathy.However, repeated and personalized encouragement from a therapist helped the participant to complete the exercise.A challenge is therefore not to forget the human dimension and the personalized support in the rehabilitation, although it is done in autonomy and with new technologies to reduce the costs.A research direction is the integration of a concept of encouragement directly into the application, in order to improve the motor demand after a pause caused by fatigue.

VI. CONCLUSION AND FUTURE WORKS
Our interactive system to assist motor rehabilitation of the upper limb after stroke is based on Virtual Reality because of its flexibility and the possibility to deliver non-invasive tracking.The first preliminary tests delivered to eight patients with the uni-manual pointing task are encouraging.
Current work focuses on the co-design of new gamified exercises to perform different types of movements (pointing, supination, etc.) and the validation of the monitoring components (measures and indicators).Then, the research will be the integration of adaptive difficulty functionality, before a formal clinical evaluation.

Figure 1 :
Figure 1: The architecture and core components of our VR rehabilitation system.

Figure 2 :
Figure 2: The external view of the system.

Figure 4 :
Figure 4: The system used by a patient.