A PRACTICAL EXAMPLE USING VR IN THE ASSESSMENT OF BRAIN INJURY

Virtual Reality (VR) as a complementary tool for medical practitioners in the assessment and rehabilitation of people who have suffered a traumatic brain injury (TBI) is discussed. A pilot-study has been undertaken on a prototype VR assessment tool. The design involved nine occupational therapists with expertise in the care of traumatic brain injured patients and one (computer experienced) patient. The aim was to begin a dialogue and to ascertain the potential of a VR system. A common method for occupational therapists to assess function and ability is to ask a patient to brew coffee. From the performance of such a task, an individual’s “functional signature” can be determined. The prototype was built using Superscape, a personal computer based VR system, to be close to the real coffee making task, including effects of making mistakes, realistic graphics and sound effects. The world was designed to be as easy to use and intuitive as possible, though problems of mental abstraction level, transfer of training and realistic interaction have yet to be resolved. The comments from the test participants have highlighted problem areas, given positive insight and pointed out other scenarios where VR may be of use in the rehabilitation of people with a traumatic brain injury.


Introduction
Virtual Reality (VR) has shown great potential in various training and visualisation fields.
In the rehabilitation of people with Traumatic Brain Injury (TBI), the challenge of appraising and using a new tool is beginning to be taken. A few groups, notably; Andrews et al (1995), Rose (1996), Pugnetti et al (1995) and Rushton et al (1996) have shown that VR technology can provide new and useful techniques for rehabilitation. For example, training using VR could help to alleviate the problems of under-stimulation during recuperation (so called fourth level damage -degeneration of brain tissue due to an impoverished environment) by providing continuous and early sensory stimulation (Rose, 1996). Similarly, early assessment of cognitive ability can allow a rehabilitation programme to be instigated quickly (Rose, 1996). Recent work by Christiansen et al, 1998 gives the results from a reliability study using VR to assess basic living skills in people with TBI. Though the results seem positive, many questions are left open as to the relevance of the task breakdown, transfer of training from VR to reality, usefulness, usability and comparison with existing assessment techniques. Furthermore, the tool was designed as a stand-alone assessment tool whereas a more easily constructed tool, more likely to be accepted by the medical profession, would be as an aid to the rehabilitation expert in an assessment situation.
Such articles suggest that further work is both warranted and needed in this area. In particular, VR is an attractive training medium since: It allows control over which details are to be included in an environment, leaving out those that may be distracting or irrelevant. The environment can be structured so that exactly the same situation can be repeated as many times as the trainee (or trainer) desires. The trainee receives positive reinforcement in the form of direct feedback, rewards for correct responses, and help from the computer. The trainee can work independently, with the training sequences being recorded for later appraisal by a human teacher. Dangerous or costly situations can be trained without risk. Training can occur despite physical disability.
A common problem with brain injury is a reduction in motivation. Given that computer programmes can be highly motivational for some people (particularly younger people), VR based training may allow patients to learn despite themselves.
This will, of course, require careful consideration of behavioural training methods and make use of direct feedback and rewards from the computer.
One issue that is not clear is what sort of information can be successfully transferred to the real-world situation from training using VR. This is entirely dependent on the cognitive abilities of the trainee. There have been a number of successful instances of training people with mental disabilities using VR. For example Standen and Cromby (1995) have used desktop VR for education and training of people with learning difficulties, and researchers at VIRART at the university of Nottingham have shown that VR has a positive effect in special needs education (eg. Brown and Stewart, 1996, Brown et al, 1998and Cobb et al, 1998. It is hypothesised that since people with mental disabilities can be taught using VR technology, it may also work for people with a TBI. Nevertheless, it would be presumptuous to assume that VR will provide the perfect tool, indeed, research has shown that care must be taken where VR is applied -not all such training is beneficial. It is generally accepted that people can generalise concepts and procedures, even emotive conditions from VR to the real world, for example, consider the work of North et al, (1999), however, does this hold true for people with TBI? Kozak et al, (1993), for example, have found no evidence of transfer of training using VR for physical tasks due mainly to the lack of tactile feedback. However recent work by Rose et al (1998) has demonstrated a positive transfer effect from the virtual environment to the real and that there is an equivalence in cognitive strategy and load. Nevertheless is backed up by Wilson (1997), where he warns against seeing VR (or indeed any new technology) as a panacea and to critically compare new methods with existing techniques.
Furthermore, side effects of immersive VR usage can be uncomfortable, if not damaging. Such effects include overloading of neck muscles, nausea (Regan, 1995) and optical disturbances (Mon-Williams et al, 1992) which would be augmented and unpredictable on people with TBI. Christiansen et al (1998) reported no such effects, however the tests were short and problems could still arise if a person with TBI were to be immersed for a long period. Note that these effects are absent in desktop VR systems.
Finally, the ethics of trying a new technology on people who have been through a traumatic experience must be considered. The accepted rehabilitation methods cannot be delayed in order to test a new procedure if it is not guaranteed to provide at least the same level of help. Therefore, experiments must be carefully and ethically designed.

USES OF VR IN REHABILITATION AFTER BRAIN INJURY
VR has the potential to be used in three main areas: assessment of the extent of a person's remaining cognitive function; training weak cognitive areas; and as a tool for everyday living. The first two are discussed below.
1.1.1 Assessment. There are two methods of assessment currently used that can be roughly classified as theoretical and practical.
The theoretical method entails a one to one discussion between a medical practitioner and the patient and includes pen-and-paper tests designed to highlight particular cognitive functions. These tests have been developed and validated over many years and are only recently being augmented by computerised versions (Bradley et al, 1993). It is not expected that VR can take the place of any of these tests in the short term, mainly due to the rigorous and timeconsuming validations required. However, the practitioner not only uses tests, but also experience and subtle clues in behaviour to pinpoint the precise nature of the disability. It is in this area that VR could be useful.
Practical assessment occurs in the rehabilitation hospital by the nurses and occupational therapists caring for and working with the patient.
The responsibilities of an occupational therapist include evaluation and treatment of patients who, due to some functional limitation, find it difficult to contend with daily activities. A TBI can mean that a person is hampered in the performance of daily activities by lack of motor or cognitive proficiency or both. Reduction in motor proficiency can be seen as difficulties in walking, bending down, lifting, gripping and co-ordinating.
Reduction in cognitive proficiency often has implications for how independent a person can be. Examples of such are; taking initiative, being attentive and able to set goals, choosing and managing artefacts, beginning and ending an activity, proceeding through a sequence in logical order, and learning from previous experience.
Assessments are carried out in this environment as well, though these tend to be more of a practical nature, based on daily activities. VR could also be of help here, particularly in the early assessment of people who have other physical injuries that restrict movement.

Training.
Rehabilitation of people with TBI entails preparing them for re-entry into daily life, and retraining them in everyday activities necessary to partake in society and work and to look after themselves. This is a timeconsuming process involving practice in real situations and can be stressful since the patient often has to go into the outside (unprotected) world, for example to learn how to catch a train. Some of these situations may be possible to represent in VR, such as using the train ticketing machine or an automatic bank teller.
The cognitive processes that VR can help with are: following instructions; sequencing; attention to periphery information (audible and visual); increasing the brain's working speed; problem solving; and navigation.

PHYSICAL AND COGNITIVE BURDENS IMPOSED BY COMPUTERISED TOOLS
A cognitive (and physical) burden is placed on a person to manage a tool to complete a task. Most tools are, of course, designed to simplify the completion of the task, but often require training to be used effectively. In the extreme case, using the tool is too complex and completely detracts attention from the task (for example, a non-typist trying to use a keyboard for writing). This is particularly relevant for computerised tools, where it is often not known beforehand if the tool will actually simplify the process. If the issue is further complicated by involving people who may well have abstraction difficulties with everyday objects, then extreme care must be taken in feeling a way forward in the design of a computerised tool.
In perceiving depth in the real world, both monocular and binocular cues are used (Eysenck and Keane, 1995). Many of the monocular cues such as linear perspective, aerial perspective (haziness of far-off objects), interposition (nearer objects obscure farther ones), shading, texture, familiarity with object's size and motion parallax can be implemented in VR. This is indeed vital if we are not to tax the patient's abstraction understanding too much, for all the binocular cues (convergence of the eyes, stereopsis and accommodation) are lost when we put the picture on a computer screen. To complicate the issue, the patient also has to understand that the image on the screen is a two dimensional rendition of a three dimensional model which in turn represents a real environment. It is expected that some brain injured patients will not be able to overcome these abstraction barriers, particularly if their three dimensional perception is damaged.
Clearly, one of the first tasks is to identify exactly which group of patients can benefit from the technology.

A Study In Assessment Using VR
It was decided to conduct an exploratory study as a lead-in to further work to assess where VR could in practice be applied to TBI rehabilitation and to which groups of patients. A collaborative project was initiated between the Lund University Ergonomics department and the Lund University Hospital Rehabilitation Centre to build and test a simple desktop VR world as an assessment tool and to gather ideas from the Occupational Therapists who work with brain injured patients. The task chosen was a reproduction of one of the tests currently used for practical assessment -that of brewing coffee.
Brewing coffee is a frequent and common task in the Swedish culture. A task such as this is usually subdivided into both physical and cognitive components (Table 1) which are used by the occupational therapist in the assessment process. Whilst the physical components cannot be assessed using desktop VR technology, over half are cognitive, suggesting that VR could indeed have a role in early assessment using this task. The cognitive aspects can be further divided into operational and tactical proficiencies. The operational aspects depend on variables such as ability to visually scan the locale, orientation in the environment and action readiness or confusion with complex motions. Such behaviours or deficiencies are recognisable in an individual regardless of the task's character.
Tactical proficiency is more problem or task orientated and tied to the completion of an activity and the specific requirements of the task. In this area are qualities such as problem solving and judgement of the actual situation, risk assessment, simultaneous capacity, planning of procedures and behaviour control. The current coffee brewing test is described in a document used in the rehabilitation field which details the exact process to be carried out and what the therapist should be on the look out for (Fischer, 1995). This was used as a basis for the design of the VR world which was contrived to look like a simplified kitchenette (fig 1).
The following aspects were taken into account -some of which come from Fischer (1995) whilst others are standard human-machine interaction considerations: 1. Task concentration. The user has to be able to concentrate on the task, not on using the tool. This is particularly important in this case, as the users may have no computer experience at all. Furthermore, any assessment using the tool is going to be coloured by the effects of the tool itself. This leads to an important research question -can VR be used for assessment, or is the weight of usage too great to be overcome by brain-injured patients? Methods of reducing tool distraction are discussed below. 2. Sound. Not only vision is important -sound, too, plays a role in determining what is occurring in an environment. For example, the sound of running water reminds us that the tap is still on. Some brain-injured people have trouble incorporating such peripheral information into their task performance. Therefore, realistic and directional sound effects were included for all objects. 3. Controlling the world: One method of reducing distraction by the tool is to simplify the input procedure. This is particularly important in this case as a 3D mouse or dataglove is both expensive and may be hard to use. Hence, all interactions were made to be by single mouse clicks with automatic positioning of the viewpoint. It was hoped that by restricting in this way, the user would quickly learn the tool. Similarly, all interaction was via objects in the environment -no abstract symbols, icons or hidden keypresses.

Abstraction level, visibility of objects and affordances.
To decrease the cognitive load, the level of abstraction from the image on the screen to the real world had to be minimised. In essence, this entailed making the VR world as realistic as possible within the hardware and software limitations. For example, the textures on the objects were scanned from the real objects to give them added realism. There were some problems, however, in keeping items of interest always visible and of a sufficient size to be clicked upon without drawing undue attention to the object. An interesting question would be whether a split-screen method is better. A further issue was whether the objects gave sufficient affordance to being clicked upon. 5. Making mistakes. A vital part of the assessment process is noticing what sort of mistakes the patient makes. For example, not being able to follow a procedure might manifest in putting the coffee grounds in the machine before the filter. Therefore, the VR world had to allow for mistakes to be made and produce a sensible result, so forgetting to put the coffee grounds in should result in hot water in the coffee pot, not coffee. This differs from the usual situation where mistaken avenues of action cannot be taken (for programming efficiency) since the objects are not made active until the correct prerequisites have been fulfilled. Users also had to have the freedom to take alternative routes since not everybody makes coffee in the same way. An important aspect to consider was whether the patient would be able to manage the VR tool, and whether he or she could overcome the cognitive (and physical) burdens imposed by the tool (as opposed to those demanded by the coffee brewing process). These are estimated to be as follows in this prototype: Being able to move a mouse with enough precision to point and click. Perceiving the coupling of mouse movement with cursor on the screen. Understanding that clicking with the left button can cause an event to occur. Realising that clicking an object means "pick up this object and carry it".
Knowing that clicking another object whilst holding the first means to "perform an operation using the first object on the second". Comprehending that the flat two dimensional picture is a representation of a three dimensional worldand that this, in turn, is a representation of a real environment.

Figure 1: A scene from the VR coffee brewing environment. {{The web version contains the actual world}}
It has not yet been investigated whether the patient is able to understand these concepts, nor the detriment of these on the performance of the task and the usability of the tool for assessment of traumatic brain injury -this is the subject of further research.

Method
Superscape® was chosen as the VR world building environment since it works on ordinary personal computers, such as those found in a rehabilitation centre and allows quick and easy prototyping and programming of complex object behaviours. Desktop VR was used rather than immersive, again due to the availability of hardware, and also because of the aforementioned side effects.

SUBJECTS
The subjects were nine occupational therapists (aged from 31 to 49, 6 women, 3 men) who work with brain injured patients on a daily basis as well as one brain injured patient (aged 19, male). Only the patient had significant computer experience, the others were 'word-processing' level users. Since the intention was to gather ideas and show the potential of the system, it was considered too early to include more patients, the one included was comfortable with technology and was included as an expert rather than a 'patient'.

PROCEDURE
The subjects were made aware of the purpose of the test and asked beforehand to consider the potential of the system. They were told to brew coffee as they usually do, that the program was controlled by single mouse clicks and performed the task between one and five times. The time taken to perform the task was recorded. The subjects were then asked for criticisms about the graphics, animation sequences and tempo as well as to give suggestions about other activities or areas where VR technology could be used. Note that this was not a real user-test, rather a preliminary test to gather in expertise from the experts in brain injury care and assessment.

Results
With 28 tests by 10 people, time varied from 1.17 minutes to 12 minutes to complete the task, with a tendency to reduction on subsequent tests by an individual. The average time taken was 3.9 minutes with a standard deviation of 2.6 minutes. Two attempts were stopped prematurely due to computer problems; all others finished when the coffee was successfully brewed. The following observations and comments from the users were gathered and divided into three areas -graphics, animation sequences and tempo:

GRAPHICS
Eight of the ten participants had problems reading the packets and identifying what the objects actually were. Some considered that simply clicking with the mouse was confusing and restricting and that dragging objects should be possible. Some technical problems (bugs) became apparent, such as milk cartons left hanging in the air and a disappearing coffeepot. Some participants thought it disturbing the way objects suddenly came towards them and just hung in the air when being carried -this could be frightening for patients. It was not possible to see the coffee grounds going into the filter. More objects should be put onto the shelves and a clock on the wall. Colouring of the objects was questioned (such as the water in the coffeepot). Superscape® does not allow shadowing, and in this model, all the objects were flat-shaded to increase speed. This gave a cartoon-like quality to the pictures.

ANIMATION SEQUENCES
Missing being able to open the coffee packet. Being able to see the sink, coffee machine and bench together was judged desirable. Need an overview of the whole area.
Water runs too fast into the coffeepot. Some "reality problems" were discovered such as water not seen to be pouring into the machine, though ending up there anyway. One person commented that, due to the hopping backwards and forwards of the viewpoint and automatic movements of objects, that it was difficult to know what was going to happen next. The one person that commented on sound said it was suitable. Not possible to have two things in the hands at once.

TEMPO
The tempo was thought to be reasonable, though some parts were considered too fast (such as filling the coffeepot with water), whilst others too slow and jerky.
Some of the comments can be traced back to the computer experience of the participants that, though limited, nevertheless gave expectations of how a graphical interface should work. Completely computer untrained people would probably not have commented in the same way.
The users were also asked for suggestions for other applications and came up with: automatic bank teller, automatic ticket machine, making dinner, laying the breakfast table, performing a transaction at the bank or post office, training in planning, training in traffic and finding one's way. Suggestions covered both assessment and training. The general feeling was positive and that such technology is exciting and could be more motivating for younger patients than household tasks. It was also felt that adaptability to an individual's needs would be easier and the ability to repeat difficult situations a definite bonus, particularly where the consequences of a wrong action can be expensive or dangerous (such as with a ticket machine or in traffic).

Discussion
This preliminary test and participative design has identified some areas where VR technology can be used in the rehabilitation of people who have suffered a TBI and indeed desirable in the following circumstances: as an early assessment tool for patients who are also physically injured; as a training tool for difficult or dangerous situations; and as a positive training motivator for technology interested (perhaps younger) patients. However, the disturbance effect of using a VR tool on task performance and assessment potential has yet to be measured. Actual user tests under real conditions are planned using a more advanced prototype to determine the suitability of the tool and whether any transfer of training actually occurs. Results will be compared with existing assessment and training methods.
This prototype design posed some problems and forced often arbitrary choices of interface. Comparative studies are warranted to test alternative methods of control and the appropriate trade-off between level of realism and computer ability. These are essential due to the lack of information on the ability of brain injured patients in coping with the abstraction required in using a computerised tool. Such studies could be: Comparison of the current point-and-click mouse control with drag-and-drop and other input devices such as a dataglove, touch screen or force-feedback device. Comparing the effect of picture realism (realistic lighting, shadows and realistic object behaviours) with the current more primitive version (no lighting effects or shadows, flat, unshaded surfaces and somewhat unreal water behaviour) on performance and abstractability. Determining the effect of sound effects -how much sound is required and how realistic should it be?
It was also difficult at times to find a suitable viewpoint where the user could both see and click on the objects that should be available at a particular stage in the brewing process. Alternative organisations of the objects could be tested against, perhaps, a split screen showing the coffee machine and a roving viewpoint.
Other VR technologies could also be tested but a careful analysis of cost vs. benefit must be made concerning the availability of suitable computer equipment in the rehabilitation centres and the possible dangers of such technology. This includes stereoscopic glasses to give depth vision to the screen, to immersive VR using a head mounted display or even projected immersive VR as well as alternative input devices as mentioned above.
During the test, many comments were gathered on both the negative and positive aspects of the tool. Despite problems such as unpredictable behaviour of the viewpoint and occasional bugs, the average time taken by the participants to brew the coffee was less than four minutes with no prior training and they showed a tendency for improvement over successive trials. This is a significant achievement and suggests that the usability level of the tool is indeed at an appropriate level, at least for non-brain injured people, though how it will be received by patients cannot be concluded from this test. The negative comments could be mostly tied back to hardware or software deficiencies, programming bugs or limitations in the prototype. Furthermore, there was little mention made of reality problems such as not-quite-realistic sound effects, flat shading of objects and totally unrealistic water behaviour. This may be explained by considering the expectations of the users -if the scene is too realistic, people tend to notice the deficiencies, if it looks like a model, they tend to accept reality deficiencies more easily. Whether people with TBI will do the same should be investigated in the context of the trade-off between reality level and hardware costs. The positive comments suggested that the tool does indeed have potential, though it is obvious more supporting evidence will need to be gathered before the medical practitioners and occupation therapists put any trust in a tool based on technology that many see as solely for games or researcher's ramblings.