A Virtual Radial Arm Maze for the Study of Multiple Memory Systems in a Functional Magnetic Resonance Imaging Environment

(cid:178) An increasing number of functional brain imaging studies are employing computer-based virtual reality (VR) to study changes in brain activity during the performance of high-level psychological and cognitive tasks. We report the development of a VR radial arm maze that adapts for human use in a scanning environment with the same general experimental design of behavioral tasks as that has been used with remarkable effectiveness for the study of multiple memory systems in rodents. The software platform is independent of specific computer hardware and operating systems, as we aim to provide shared access to this technology by the research community. We hope that doing so will provide greater standardization of software platform and study paradigm that will reduce variability and improve the comparability of findings across studies. We report the details of the design and implementation of this platform and provide information for downloading of the system for demonstration and research applications


INTRODUCTION
Learning and memory are not unitary processes in the brain; rather, multiple neural systems support the functioning of multiple forms of learning and memory.One classical distinction between memory systems, for example, involves neural systems in the striatum and mesial temporal lobe.The hippocampus and amygdala in the mesial temporal lobe mediate memories for conscious facts, previous experiences, memory system) [1].In contrast, the caudate nucleus and putamen together compose the striatum and mediate memories -response (S- [2,3].Both animal [2,[4][5][6] and human studies [7][8][9] have demonstrated the mediation of habit learning by the basal ganglia (striatum is a part of the basal ganglia), and the independence of habit learning from declarative learning and memory functions based Manuscript Received on Novermber 1, 2011 within medial temporal lobe structures.Studies of humans who have undergone resections or who have suffered damage to the hippocampus and parahippocampal cortex have suggested that these regions support declarative memory, in particular, spatial memory [10][11][12][13][14]. Studies of memory systems in rodents, which defined the field of multiple memory systems, showed, for example, that electrolytic or neurochemical lesions of the dorsal striatum impair performance on habit learning tasks but not on declarative memory tasks, whereas lesions of the hippocampal system impair performance of declarative memory tasks but not on habit learning tasks [4,5,[15][16][17][18].The independent role of these two brain structures in declarative and habit learning memory systems have been documented not only in rodent models, but also in nonhuman primates [19] [20].This specific and differential effect that lesions to different brain systems have on specific types of learning and memory is constitute powerful evidence for the independence and functional specificity of neural systems.
In recent years, investigators began to use imaging approaches to study multiple memory systems, and preliminary functional magnetic resonance imaging (fMRI) studies of navigation tasks in humans using virtual reality (VR) have already added significantly to our understanding of the neural networks involved in learning and memory [21][22][23][24].By imaging a group of subjects using positron emission tomography (PET) during navigation of a virtual city in three different modes, investigators found that right hippocampus and right caudate nucleus were strongly associated with navigation memory while brain structures in the left-hemisphere (hippocampus, frontal cortex) were involved in non-spatial aspects of navigations [25].With a similar virtual city and an aerial view of the town in a map, investigators employed fMRI to study the functioning of brain regions involved in navigation tasks, and the difference of navigation ability between adolescents and adults [22].This study found that right side in the thalamus and the junction of amygdala and anterior hippocampus were statistically involved, and adolescents showed consistently higher activations than did adults who performed much better than adolescents in interpreting 3D city using abstract symbols on a 2D map, indicating the existence of a neurodevelopmental progress.Some other functional imaging studies on spatial learning and navigation in humans have also found significant activations in medial temporal lobe [26,27], in posterior parietal and frontal cortices [23], and in the closely related parahippocampus [28,29].Thus, the findings of prior VR paradigms to study learning and memory systems have been quite variable.
These discrepancies in findings may be attributable, at least in part, to differences in design of the navigation tasks.One feature of study design that accounts for a major portion of variability across studies is how well the control condition accounts for neural activity during the task that is necessary but not sufficient for learning and memory.Unfortunately, variability in the choice and adequacy of control conditions likely have contributed to the variability in the patterns of activation that have been reported in prior imaging studies using VR memory tasks.For example, in the PET imaging study [25], the participants were required to execute 4 tasks.The 4 tasks differed in a number of aspects, instead of only in that related to spatial learning.Also, extra cues were used in one of the 4 tasks, and thus introduced to the fMRI contrast disturbances that were not either further investigated nor excluded in the analysis stage.In the fMRI study using a VR radial arm maze to investigate changes in brain activity while human subjects adopted different strategies to navigation [23], the control session did not match the experiment sessions in bait configuration, experiment duration, bait visibility, number of accessible arms (4 versus 8).Moreover, the views (colorful landscape versus plain walls) were significantly different in order to either show or hide the landmarks.Therefore, whether or not the fMRI contrast thus obtained was attributable to spatial memory is questionable.Also, this study did not have an efficient mechanism to control the navigation strategy that the participants would employ to retrieve the baits, but relied on their debriefing after the experiments, making the analysis not completely quantitative and objective.All such variances across the differing task conditions of the maze would subsequently cause difference in action planning and execution, thereby including unrelated neural activations in the final contrast that was supposed only related to memory activities.It therefore seems that the extant virtual reality platforms for the study of multiple memory systems thus far have not been ideally controlled even for basic features on the stimulus properties or task instructions across the active and control conditions, likely producing activations associated with sensory and cognitive processes that have little or nothing to do with memory.
In general, we expect an ideal VR paradigm for studying multiple memory systems to have well-designed control conditions matching the active experiments, thereby limiting the contamination of the planned contrast by neural activity associated with features of the task that are not specific to learning and memory.Navigation task is usually a complex procedure involving multiple levels of activities in many brain regions (Fig. 1).Schematically, each lower level of information processing constitutes necessary but not sufficient basis for the higher level of processing.The output On+1at level n+1 is a function f of information In both the active and control experiments, each module constitutes a necessary but not sufficient condition for the following module.If the corresponding modules are perfectly comparable across both experiments, the contrast of them will only attribute to brain activities related to memory systems.processing based on the output On of the lower level n plus some additional input information In+1: On+1 = f(On, In+1).This applies to both the active and control experiments.Functional MRI data thus collected therefore correspond to OK, supposing K levels are involved in the procedure.Ideally, if the active experiment contains N levels of information processing with the last one being the expected neuro-activities and the control experiment contains N-1 levels perfectly matching the active experiment in every aspect, the final contrast between ON(active) and ON-1(control) will be brain activities purely related to IN, i.e., the expected high-level neuropsychological brain activities in the active experiment.The multiple levels of activities involved in navigating a VR paradigm move gradually from posterior of the brain to anterior, representing a procedure of information processing from lower levels to higher levels.In general, the participant receives low order visual stimuli (e.g., hue, texture, luminance, color) in the visual cortex, process and encodes visual features (e.g., spatial awareness [30]) in the temporal parietal cortex, recognizes particular symbols or identifiers and plan the next steps in the frontal cortex, then executes the planned actions in the frontal cortex; the subject then evaluates the success of the actions and adjusts accordingly in the anterior cingulate.In an active spatial learning task, the brain will then memorize the successful trial using hippocampus and parietal cortex.Therefore, an ideal control experiment should include all the counterparts except and only leave out the last portion, which involves the memory-related activities.The contrast between the active and control experiments can consequently fully attribute to memory activities, without disturbance from any other neither necessary nor specific brain activities.In cases when the control experiments do not match the counterparts in the active experiment, extra neural activities that cannot be offset from the difference of the two experiments will then contaminate the resulting contrast, making it unreliable.Whereas no experiment condition can be perfect, the design of the control conditions should at least minimize such contamination.
Taking advantage of the modern VR technology, our VR system facilitates the study of multiple memory systems of human brains.We design the VR system in every aspect to maximize comparability across the active and control experiments, except that the active experiment contains one additional module that involves memory activities.This VR system involves processing of both low-and high-order visual stimuli, identity recognition, path planning, decision making, plan execution, monitoring success and memorizing the success or failure, producing activations across many or most regions the brain, particularly in the visual cortex, temporal lobe, frontal cortex, anterior cingulate, hippocampus, and parietal cortex.In this VR system, we have designed the walls of the maze, the luminance, hue, texture of the maze and environment, the nature of the rewards to be as identical as possible across all conditions.Moreover, the number of possible trials across sessions, accessibility to arms, and familiarity to the environment are also kept to be the same (see details later in the text).Our VR system provides a more ecologically valid experimental setting, because the task is much more real world-like than non-VR tasks, such as word recall [31,32], recognition memory tests [33,34].It may therefore be more clinically relevant for assessing memory in persons who have learning and memory disturbances that impair real-world functioning, although their performance on pencil and paper tests is relatively unimpaired.
Our system also features the following innovations.First, it provides far greater control over navigation strategy.The direction to which the participant faces at the beginning of each trial during an experiment is randomized in the virtual reality, which is impossible in reality.This design thus undercuts the so-called chaining strategy (or, nonspatial strategy in [23]), by which a person can routinely visit each arm one by one in spatial order to complete the task, while using little or no spatial memory.This chaining strategy could not be identified in previous studies (e.g.[23]), if the participants chose not to report.In addition, we randomize the scene segments outside of the maze in the control conditions for destroying the spatial cues that the person must use to navigate the maze successfully.Because the segments are from the same scene as in active conditions, visual familiarity remains roughly the same but spatial cues are no longer useful.Second, we use exactly the same number of trials in the control conditions as that the person has used in the active experiment, and we ensure that the frequency of reward and failure are exactly the same in the control condition as in the prior active task.The control of reward frequency ensures good control of the cognitive and emotional experiences associated with reward and failure in the active and control conditions.The control of both the number of trials and the frequency of reward thus maximize the comparability across the active and control experiments, and are helpful in providing the same statistical power across conditions in post-experiment analyses of behavioral performance and brain activation.Third, we have designed and added an additional experiment of control condition to inspect those potential signals arisen from arguable factors (detailed in Section 3.3) for further validating that our control conditions offsets signals from unrelated brain activities.Fourth, our system minimizes differential desensitization and habituation across the active and control tasks.Participants do not need to be trained in the system in order to become familiar with the VR maze.A brief description and instruction appearing in text prior to the beginning of the experiment can do the work.Fifth, while all other aspects are kept identical, the only difference between active and control experiments is the spatial cues, which are destroyed in the control conditions by destructing the outside scene into segments and subsequently the segments are recombined, making the experience of depending on spatial cues in the active mode no longer useful.Contrast between the active and the control thus should only relate to this difference, i.e., the spatial memory.Sixth, the design of our VR maze is t, as it is adapted directly from maze paradigms used in rodents, in which the presence of multiple memory systems have been identified and most rigorously studied.In contrast, most of the VR employed in previous functional imaging studies have not typically been the subject makes, and therefore can provide encoded and scored measures to be associated to brain regions and activities.This system thus allows one to completely re-experience what a participant has experienced and acted.Moreover, the recorded behavioral data and brain imaging data thus allow us to study the reward mechanism obtained through this experiment [35], because this mechanism in the brain is independent of the task.Also, the detailed information thus recorded allows further explorations in the future whenever one realizes any new findings could have been possibly mined from the treasuring data.

II. BACKGROUND
The radial arm maze consists of a central area and eight arms radiating out from the central area, first described in 1976 [36] for studying memory systems using an animal model of rats.It can be adapted to a wide range of behavioral tests, in particular, hippocampal and striatal learning tests.Using such an eight-arm radial maze, the neural underpinnings of the different radial arm maze tasks were dissociated [16].Based on a study of the effects of lesions to three brain regions, a dissociation was found between the conditioned cue preference task (CCP) (impaired by amygdala lesions), the win-shift task (impaired by hippocampal lesions) and the win-stay task (impaired by striatal lesions).Damage to the hippocampal system impaired the acquisition of win-shift task but neither the win-stay nor the CCP task.Damage to the lateral amygdala impaired the acquisition of CCP task but not the win-stay or win-shift task.Damage to the dorsal striatum impaired the acquisition of the win-stay task but not the win-shift or the CCP task [16].The win-shift and win-stay tasks thus became a popular and nearly standard approach to probing brain working memory and learning ability.

Radial Arm Maze Win-shift
The win-shift task is a test of spatial working memory.In the test of using animal model, food pellets are placed in all eight arms.A mouse is placed in the central arena and is allowed to freely explore.The mouse enters an arm, finds a food pellet and eats it.When it returns to the centre, doors lower to temporarily close off all the arms.After a delay, the doors are opened and the mouse is once again allowed to enter an arm -the whole process is then repeated.In order to obtain all the food, the mouse has to go to each arm in turn.The trial continues until the mouse finds all eight food pellets.The experimenter records every action the mouse takes and grades its behavior.It loses marks if it returns to a previously-visited arm.This test was shown to be hippocampus-dependent [37], related to spatial learning and declarative memory.

Radial Arm Maze Win-stay
The win-stay task runs in a dark room.Four of the eight arms of the maze are baited with food pellets; these arms are also selectively illuminated.The mouse starts in the center of the maze and is allowed to freely explore.Upon entering one of the Fig 2 .An Overview of the Virtual Reality (VR) System.The VR is running on a PC with dual monitors.Visual stimuli to be displayed in the MR-compatible goggle is simultaneously displayed on one of the two monitors, while the other monitor is used as console f participant lying in the scanner navigates and acts through use of a MR-compatible joystick or button-box.The VR is launched before the scanning session is started, and waits for a signal from the scanner for trigging synchronization between the scanning session and the fMRI task.Every movement that the participant makes after the signal of synchronization will be recorded for post-experiment analysis and replay.lit arms, it will discover and eat the food pellet therein.It then returns to the centre, at which point entry to the arms is blocked by descending doors.The mouse stays blocked in the centre for about 10 seconds, during which time the food it has eaten is replenished.The doors are then opened and it is again allowed e is to return to the first arm, where it will again discover food.Upon returning to the centre for the second time, the light in that arm is switched off.The procedure can then be repeated for the three remaining illuminated and baited arms.The task was shown to be striatum-dependent [16], related to S-R or habit learning.

III. GENERAL DESIGN OF THE SYSTEM
The VR system that we developed directly adapted from and rigorously simulates the mouse experiment [36] to maximize the translational power.The virtual environment is composed of an eight-arm radial maze with a short-out rampart of bricks under blue sky, surrounded by naturalistic landscape (e.g., mountains, grassland and trees) that appear to be in the later afternoon.The landscape out of the maze constitutes the extra-maze cues for use during spatial navigation.Instead of the food pellets, however, rewards denoted by a large icon of a US dollar sign ($) are used and are hidden at the very end of the arms of the maze.We acquire functional MRI data by scanning a human participant using a GE magnetic resonance (MR) scanner (General Electric Corp, Milwaukee, WI) when the participant is executing the experiments.
This VR system is compatible with MR scanners of major brands.We used Microsoft Visual C/C++ for programming the cross-platform software package of OpenGL for graphics application on PC/Windows XP (Fig. 2).To coordinate the VR task with the scanner, we implemented a synchronization device to facilitate post-experimental analysis of imaging data.The signal of synchronization is passed from the scanner to the VR task by a standard S-R box, which is independent of scanners and platforms for communicating status and responses of the participant to the VR system.While all other necessary pieces of peripheral device are products of commercial standard, our VR system can run with Phillips and Siemens scanners, although we developed our system based on a GE MR-scanner.
The system usually runs on a PC with dual monitors, interacting with the participant of the experiment through a pair of video goggles and a joystick or button box, all MR-compatible.The visual stimuli delivered to the goggles are simultaneously displayed on one of the two monitors, while the other monitor is used as a console for the operator of the experiment to supervise the status of the experiment and the navigates the virtual maze and acts himself through use of the joystick or button-box, which communicates with the console PC through an interfacing response box or S/R-box.The VR needs to be launched before the scanning session is started, and will wait for a signal from the scanner for trigging the VR system so that the navigation task synchronizes with the fMRI scanning session.Every movement that the participant makes after the synchronization signal will be recorded for post-experiment analysis, or replay of the whole navigation procedure.
Fig3.Map of the Virtual Maze.The virtual maze has eight arms stretching radically.The maze is divided into three regions for internal coding: the central area (green), the arm passage (light blue), and the baiting area (dark blue).An invisible threshold is set at the 10% length from the entrance of each arm toward the other end of the arm.Once this threshold is passed, the participant is regarded as entering an arm.The outmost 30% length of each arm is the bait area, whereas the area between the 10% and 30% lengths is denoted as the passage area.The rest region inside the maze is defined as the central area.The notation to the upper-left corner of the map indicates the current experiment: either win-shift, win-stay, control, or control with arrow.The lower left corner displays in real time a summary statistics of the current experiment episode.The colorful curves inside the maze are the routes that the participant has navigated, of which the green route corresponds to that in the episode of active win-shift and the red route corresponds to that in the episode of the control with arrow.The triangle with a red tip in the central area of the maze is the current location of the participant in the VR maze, with the red tip indicating the forwarding direction.
We designed the system to be used in three different modes: real experiment mode, training mode and replay mode.In the real mode, one may navigate the maze in three different conditions, i.e. regular active experiments of winshift or winstay, a control experiment and a control experiment with leading arrow (in which an arrow will show for leading the participant to a randomly selected arm).The experiments of these conditions are either with useful visual cues or useless cues both inside and outside of the maze.When the VR system runs in the training mode, it works exactly the same as does in the real mode, except that no synchronization will be triggered for coordinating the VR and MRI scanner systems.Thus, the VR system can run alone in training mode independently, detached from the MR scanner system.Finally, the replay mode allows replaying of an experiment or training session by taking in a recorded log file of behavioral data, allowing investigators to re-experience vividly in person what and how the a participant has performed.
While this VR maze is fully functioning on PC/MS-Windows platform and can coordinate with GE MRI scanners properly, we also provide two simplified versions compatible with SUN/Solaris and SGI/IRIX system for training purposes, which allow use of a mouse or keyboard instead of a joystick for people to familiarize themselves with the system.The system is available upon request and accessible through the following link, after filling out a registration and request form: http://www.columbia.edu/~dx2103/Gamepage.Below we elaborate the specific features of this VR system of radial maze.

The Layout of the VR Maze
Geometrically, the virtual maze is set up on a plane, with eight arms stretching radically and distributing evenly on the 2D plane (Fig. 3).The central region of the maze is roughly round with a radius of 60 pixel units, and each arm is 120 pixels in length by 32 in width.In the VR system, the maze appears to be a brick structure in gray sitting on grassland, surrounded by woods and hills under blue sky in a later afternoon of the day.
Internally, the maze is divided into three different regions (Fig. 3) for conveniently monitoring and assessing the facilitating post-experiment analysis.The three regions are the baiting area, the arm passage area and the central area.The Shown here is a snapshot of a typical immersive view of the virtual maze during the session of control with arrow that a participant will see through the MR-compatible goggle.The red arrow was initialized and generated in the beginning of each trip for seeking a reward, pointing to a randomly selected arm that may or may not contain a reward.This red arrow is therefore providing possibly misleading information to the participant.outmost 10% length of each of the eight arms is the baiting areas, where a hidden reward may be retrieved.At the 10% length interior to the entrance of each arm, an invisible doorsill is defined.A participant is regarded as entering an arm only when the participant passes through this sill toward the exterior end of the arm.The area between this doorsill and the baiting area is the passage area.Finally, the rest region within the maze is defined as the central area.
In the very beginning of each experiment session and every trial in each session, the participant is virtually positioned at the center of the maze, facing to a randomly decided orientation on the plane of the maze.

Interactions
Our VR maze system provides two separated views: the console view of maze map for supervision (Fig. 3) and the immersive view of the VR maze for participant in the MR scanner(Fig.4).In addition, a control panel (Fig. 5) is provided for overseeing the running status of the system.While all these components can be displayed in one physical monitor as an option for running the system for regular exercising of using the system on low-end computers, running in dual-monitor mode is recommended, especially in real experiment mode.Using dual monitors, the immersive view of the maze occupies the whole screen on one of the two monitors, and the console view of maze map along with the control panel will be displayed on the other screen.Only the immersive view of the VR will be dispatched simultaneously to the MRI-compatible goggle as visual stimuli; and the other two components will be visible only to the operator of the VR system at the console, providing performance.
The operator of the system at console can choose to run different experiment episodes (see the following subsections), set a time limit for ending the experiment, launch or terminate an experiment, examine routes that the participant has navigated performance during an experiment.
The general procedure of running the system (Fig. 6) is: When an fMRI pulse sequence is ready to start, the operator of this experiment launches the VR system, and initializes those corresponding parameters and set options on the control panel, including choice of experiment type, time allowance, and path visibility.Once the operator clicks the Start button (Fig. 5), a short message will appear in the maze window alerting that the VR is ready and is waiting to be triggered by the scanner for synchronization.Then the operator should start the fMRI sequence, allowing the scanner to automatically trigger and synchronize with the VR.The control of the VR is then immediately transferred to the participant being scanned, who is lying inside the bore of the MRI scanner.In the VR maze, the participant will find himself initially positioned at the center of the maze, facing to a randomized orientation.The participant then can navigate the virtual maze by using a joystick to navigate inside the maze.Despite that different experiment tasks (winshift, winstay, and their control episodes) have differing rewarding configurations and mechanisms, the participant needs to enter an arm and walk all the way to the end of an arm to retrieve a potential reward.If there is a reward, a reward sign will be displayed, indicating this current trial is successful.In any case, once the participant reaches the baiting area, either with or without a reward, the participant will be put back to the center of the maze again with a randomized orientation, after a short pause of one second in the bating area.
The subject will have to use spatial cues to identify a next arm for visit.After running a set of such experiments or when time is up, the system will terminate automatically.A detailed log of along with a summary and simple statis performance, briefing the total number of visits, total time spent, rate of success and average speed of navigation, etc.The control of the system is then deprived from the participant and transferred completely back to the VR operator who runs this experiment at the console.
In cases that the participant fails to retrieve all hidden rewards within the predefined time allowance, the experiment will also terminate.This setting gives an alternative way to control the execution of the VR tasks.
We have purposely designed in the system that a participant is only allowed to move forward and rotate either leftwards or rightwards, but restricted from walking backwards.We anticipate that this mechanism will force the participant to perceive actively the spatial cues in the surroundings instead of neglecting them while retracting in cases when the participant changes mind in moving direction.
At any time of the experiment, the system operator always has the privilege of controlling the VR system.In particular, a Stop button is provided on the control panel at the console for terminating proactively the running of the system at any time in case of evacuation or emergencies, so that the scanner does not have to wait until the end of this VR task to start a new session.However, no function for resuming a stopped session is provided as such experiments always need to be restarted from scratch for synchronization.

Interfacing with the MRI Scanner
As indicated, synchronization between the VR system and the fMRI pulse sequence on the MR-scanner is necessary to ease post-experiment analysis of the imaging and behavioral data.A Cedrus product, LUMINA Response Pad for MRI Model LSC-400 Controller, is used in our system for this purpose.This response pad is connected to a series port (COM 1 ) on the console computer for receiving the signal of synchronization sent from the scanner, which is defined as a -time by this response pad.The VR program is launched in advance and set to idle, waiting for this triggering signal, by continuously querying the COM 1 port.Once the signal is received, the control of the maze is granted to the participant at experiment, who can then use the joystick to navigate the maze and perform the task.
The immersive view of the maze is displayed to the subject through a pair of MR-compatible video goggle (Model FIU004US, Current Design Inc., Philadelphia PA), and the navigation is controlled by the participant through using an MR-compatible joystick (Model VOU-RTC2k, Resonance Technology Inc.,).Both these two pieces of device are connected directly to the console computer, goggle at the port of video output and joystick at any available USB slot.

Experiment Configurations
Simulating exactly the version of animal model of this experiment [16], we have implemented in this VR system two sets of tasks, i.e. the win-shift task and win-stay task.Each of the two sets contains a series of three experiments, including a regular active experiment (either win-shift or win-stay), a control experiment and a control experiment with an arrow that could be misleading.
In the active win-shift experiment, all eight arms are baited.The subject is instructed to retrieve all the eight rewards, best by visiting each arm once and only once.The outside scenery remains unchanged during this experiment, so that the participant can rely on the landmarks in the scene as spatial cues for retrieving rewards.However, the outside scenery is designed to be very similar in all spatial orientations with only delicate differences, thus the participant has to fully engage his memory and learning functions to tell the visited arms from unvisited ones.
In the active win-stay task, only four of the eight arms of the maze are randomly selected for baiting, and each of the four contains two hidden baits.In addition to the unchanged outside scenery as spatial cues, the baited arms are illuminated by a lamp fixture on the wall at the end of the arms.The lamp is turned on when a reward is still available in that arm, serving as an additional cue to hint the participant.
In the control experiments for both of these two tasks, nearly everything remains to be exactly the same so that the brain activities in the control episodes can be comparable with the proceeding active tasks.For example, exactly the same number of attempts as made in proceeding active task will be allowed, by guaranteeing that the last trial in the control experiments gets the eighth reward, which is the last reward.However, the outside scene is actually shuffled after each visit to the arms, thereby making the outside cues no longer reliable, although the outside scene appears similar to that in the active experiments as well as across the trials within the same experiment episode.We thus destroy any possibility of using spatial learning to perform the control task, making the memory activities the only difference from the proceeding active tasks.
However, the random spatial cues that is no longer reliable in the control episodes may possibly cause additional emotions, such as arousal, anxiety or perhaps frustration, when the participant fails to rely on cues that once worked.To confirm whether or not such emotional effects exist, we add one additional control experiment in which we try to explicit these emotions that are caused by misleading information.In this third episode corresponding to either win-stay or win-shift, i.e. the additional control episode with a possibly misleading arrow, we keep everything exactly the same as in the proceeding control task.We first instruct the participant in a message to follow an arrow for retrieving rewards, and then show a red arrow pointing to a randomly selected arm.Nevertheless, the pointed arm may or may not contain a reward.The arrow is therefore possibly misleading for purpose (Fig. 4).This episode is thus used for further controlling the side effects in experiments.If those mentioned emotions do exist, we should be able to find imaging signals in corresponding brain regions.Consequently, we get confirmed that something must be offset from the results obtained in the proceeding control tasks, and more importantly, we now know what needs to be removed.If those side-effects do not exist or are neglectable, we find nothing prominent in the imaging data; then we confirm that the proceeding control conditions are perfect.Either way will help us to obtain definitely reliable data for studying memory system.
By providing reliable, unreliable or possibly misleading spatial cues in the series of experiments, we expect imaging signals from different brain activities can be isolated and irrelevant brain activities will be offset.
Consequently brain activities associated to multiple memory system can be clearly identified and studied.

Three Operating Modes
The system provides three different operating modes, for training, experiment and re-experiencing recorded behaviors.
The previous subsection described the normal procedure in experiment mode.Training mode differs from the experiment mode only in the very beginning right after the system is loaded.Instead of waiting for an actual triggering signal from the scanner for synchronization, the training mode will just flash a waiting screen and then automatically move to the next stage of the experiment only after a short pause without needing a triggering signal from the MR-scanner.Everything else remains exactly the same as in the real experiment mode, including the final output log file of behavioral data.The replay mode is specially designed for investigators re-experiencing in person the behaviors of a participant.Launched in this replay mode, the system will ask for a recorded log file so that all recorded behavior can be simulated realistically, by re-experiencing exactly the same location, route and action at each time point.A different control panel (Fig. 7) will show in the replay mode, through which the operator of the VR system can choose the information to be visualized in the maze map and also how the recorded navigation is replayed.One can select to display the route of one individual experiment episode or opt to see a combination of the routes that the participant has navigated in the corresponding experiment episodes.One can also choose to jump to a particular time point, pause in the middle of the replaying session, continue to replay from a time point, or replay only a segment of the behavior data by setting the starting and ending time points and clicking a responsible button (Fig. 6).
The replay mode is able to reemerge the recorded behaviors, no matter these are from a real experiment or from a training session.

Data Recording and Post Analysis
The behavior log-file recorded from an experiment contains four parts: environment variables of the task, architectural information of the maze, paths navigated, and behavior statistics (Table 1).
The first part records information of the general setting of the task.Noted in this part are the time limit for performing the task, the duration of the pause after a reward is retrieved, and the velocity of navigation (pace).
The second part describes the basic parameters of the maze: the logical size in pixel of the virtual environment, including the radiuses of the central area and the circumcircle of the maze, as well as the width and length of the radial arms.In addition, the reward configurations for both the winshift and winstay experiments are defined.
The third part is the main body of this log file, which is usually quite lengthy.This part records every action a participant makes during the experiment sessions.Seven items are recorded for every single action, including the 2D the 2D viewing point (obj_x, obj_y), the time mark of this current event, the reward status and the current stage of where the participant currently stands.The coordinates of the viewing point together with the coordinates of the eye point decides the viewing direction of the participant, which is also the expected direction of the next step.The time mark records in millisecond when this event happens, and the reward status indicates whether or not a reward is retrieved and if yes, which reward this is.The reward status also codes in part (1) the current area code of the subject in terms of the three predefined regions in the maze, (2) success or failure of retrieving a reward if the subject is in a baiting area, and (3) the experiment stage of the current experiment in terms of the winstay/winshift, control or control with arrow (Tables 2 and 3).Moreover, the stage code of the experiment indicates whether the current event happens within a particular experiment episode (Table 3).
The fourth part provides a brief statistical report of the conducted experiment episodes, summarizing for each episode the time point of synchronization, total number of entries to arms, number of failed attempts, number of failures in the first ten attempts, total duration of the experiment and the average performance for retrieving a reward successfully in terms of time (Table 1).
This log file is dumped to harddisk at the end of the experiment in one standard format of MS-Excel, which can be read in text format, or reloaded back to the VR maze for replay.

Other Technological Aspects:
The system employs OpenGL and its utility package Glut, for the development of the graphics components of this VR maze system.OpenGL/Glut is a standard cross-platform Application Programming Interface (API) for computer graphics, available on multiple computer operating systems and platforms.Therefore, the basic components of our system can be transplanted to and run on various computer platforms, including PC/Windows, SUN/Solaris, SGI/IRIX and PC/Linux.With the facilities provided by OpenGL, we mapped texture of an extensively wide picture of natural scenery (W4096 × H512) to a remote frame surrounding the virtual maze, mimicking a realistic view of natural environment in the open air.The upper portion of this wide picture was manually touched so that the scene of landscape could naturally merge into the blue sky of a bright afternoon.Also, the two ends of this wide picture were meticulously processed to provide a seamless connection when it is posted to the outside circumstance frame.Moreover, the texture of the natural scene is readily to be replaced by a handful of different views that we prepared.
In addition, GLUI, a freeware of Glut-based C++ user interface library that provides controls (e.g, buttons, check boxes, etc.), is also employed in this system for programming the Graphical User Interface (GUI) of control panels.
To access the serial COM 1 port on the computer for synchronization signal from the scanner to trigger the virtual maze, we used NTPort, a free software product for trial by Zeal Soft Studio.

IV. PERFORMANCE
To date, we have tested the VR maze on 54 participants in an fMRI study.We continue running it in this study for scanning new participants.The parameters of the fMRI sequence that we used in the study are: The fMRI data were first preprocessed using SPM2 [38] (http://www.fil.ion.ucl.ac.uk /spm/ ), an fMRI analysis software package built on top of MATLAB, to adjust head motion and slice timing difference.The same package is also used to normalize the functional data to a standard MNI template space, and finally to smooth the normalized data for reducing spatial noise.The preprocessed data were then analyzed in SPM2 to detect the brain regions with significant activation or deactivation during the performance of the tasks, using a general linear model (GLM), for testing whether the blood-oxygen-level dependent (BOLD) response to the tasks could significantly match a hypothesized hemodynamic response function (HRF) derived from the task.
As expected, we detected brain activities in hippocampus and putamen regions due to the nature of spatial learning and memory in this navigation task of VR Maze, as well as in the cerebellum, motor cortex regions and visual cortex regions because this task also involves motor planning and visual perception (Fig. 8).

V. DISCUSSION
Following the famous report of working memory on rats [16] and the subsequent experiments using animal models, we have developed a radial virtual reality maze with eight arms for translational studies of human memory systems based on the win-shift and win-stay tasks, allowing functional imaging of the working brains during performing the navigation task that is

Hippocampus Putame Cerebellum
Visual Cortex not possible in animal models.The design of this VR system has embedded in it well controlled experiments for contrast, including one regular control experiment and one additional control experiment for double-confirmation, so that irrelevant signals to multiple memory systems and their functions are minimized in the resulting contrast of the imaging data.This VR system also takes into consideration the complicated issues of various existing platforms that are based on different computer and scanner systems, so it has been designed to be based on software and hardware packages that are platform independent, i.e. the OpenGL / Glut / GLUI / NTPORT and the Response Pad.Taking advantage of the VR technology, this system breaks through the limits of time and space so that it can easily immerse the participant to any desired location inside the virtual environment at any particular moment, and can vividly replay what has happened to the volunteer participants.Moreover, this system has been fully tested on a 3T GE MR scanner, and achieved satisfactory results after using it for 54 human subjects in real experiments.We hope this software package thus can benefit the research community by providing a common basis for studying multiple memory sysmtes of human brains.The ability of interaction of the VR system with the scanner for synchronization is a very important aspect of this system.Ours actually has provided two solutions, i.e., using the standard response pad controller, or using a SR-box provided with the popular E-Prime software package for fMRI studies.Both of these two devices connect to a series port COM1 on the console PC.However, people sometimes feel inconvenient with using these measures because these involve machine-level programming for accessing the serial port on a personal computer (PC).We therefore prepared a third option for greater flexibility in case a user seeks an alternative approach for synchronizing the VR system.This solution uses a simplified keyboard, taking the signal from the SR-box directly and connecting to an available USB port on the PC, which does not need additional programming of machine-level.The synchronization signal sent from the scanner will thus be converted to a simulated character in this simplified keyboard and sent directly to the computer, readily to be picked up by the operating system as if someone has pressed the character on a real keyboard that is already connected to the computer.As keyboard events are much easier to be caught than do those at a series port, this solution will free the VR package from using the third party software NTport.Nevertheless, this solution needs a keyboard chip (Silitek PBAAS336) and needs some extra work to construct the driving circuit for triggering the key stroke at the simulated keyboard (Fig. 9).As this last option requires some knowledge in hardware of integrated circuits, we do not list it as a regular configuration in our system but only make it available herein as an option upon request.
All the extant fMRI studies in the literature did not mention how the imaging protocols were synchronized with the execution of the fMRI/VR tasks.In our system, the log file that we recorded during the experiments denotes every action of a participant in the precision of milliseconds.Along with the embedded synchronization mechanism, our system thereby allows for any particular moment the identification of accurate correspondence across the tasks, the imaging data and the behaviors.This consequently greatly facilitates the post analysis of the imaging data and behavior study, warranting a higher precision that has never been possible.
Compared with all those existing VR paradigms for fMRI study of memory and learning systems, ours is truly a translational VR system from the one using animal models of rats that has already been well established, because our system has rigidly duplicated the Win-stay and Win-shift experiments of animal models.The value of using translational paradigms is: (1) People can analogical reasoning with probes used in animals that cannot be used in humans; (2) By imaging the entire brain in humans rather than lesioning or stimulating a single brain region in a single experiment, people can learn about activity in regions outside of the hippocampus and striatum and how they interact functionally with these memory systems.Therefore, our system can be more useful in studying multiple memory systems by verifying the agreement between human and animal models, thereby allowing more comprehensive understanding of the brain functions through complimenting and translating findings using both models.
Lastly, although the work reported in this paper is based on the Win-stay and Win-shift tasks for studying the brain functions of navigation, the framework of our system provides a template and is readily to be used for other VR tasks in an MRI environment.The kernel of this VR software can be easily replaced with a different cognitive or psychological paradigm for studying a different brain function that is of interest.In fact, the principles of designing the fMRI task and its corresponding control experiments, the concept and design of the human-computer interface, the communication between the console computer and the peripheral device, synchronization between the MRI scanner and the cognitive tasks are universally suitable for any fMRI studies.
The software package can be downloaded for demonstration at: http://www.columbia.edu/~dx2103/Game/Gamepage.html.And a runnable version for GE platform is available upon request.

ACKNOWLEDGEMENT
This work was supported by a NIBIB grant 1R03EB008235-01A1, NIDA grant DA017820, NIMH grants MH068318, K02-74677 and K01-MH077652, a grant from the Simons Foundation, the Suzanne Crosby Murphy Endowment at Columbia University College of Physicians and Surgeons, a grant from the National Alliance for Research on Schizophrenia and Depression (NARSAD), and by funding from the Sackler Institute for Developmental Psychobiology, Fig9.An Alternative Solution to Synchronizing the Paradigm with the MR Scanner.Triggering signal sent out from the MR scanner will be converted to a simulated keyboard keystroke and then relayed to the PC running the paradigm for synchronizing the virtual maze program.

Fig 1 .
Fig 1.Schematic Modules of Information Processing in the Virtual Reality Task.In both the active and control experiments, each module constitutes a necessary but not sufficient condition for the following module.If the corresponding modules are perfectly comparable across both experiments, the contrast of them will only attribute to brain activities related to memory systems.

Fig 4 .
Fig 4. The Immersive View of the Virtual Reality Maze.Shown here is a snapshot of a typical immersive view of the virtual maze during the session of control with arrow that a participant will see through the MR-compatible goggle.The red arrow was initialized and generated in the beginning of each trip for seeking a reward, pointing to a randomly selected arm that may or may not contain a reward.This red arrow is therefore providing possibly misleading information to the participant.

Fig 5 .
Fig 5.The General Control Panel of the Virtual Maze System.This panel shows up during an experiment for initializing and managing the experiment.When the

Fig6.
Fig6.Runtime Flow Chart of the VR Maze.

Fig 7 .
Fig 7. Control Panel for the Replay Mode.

Fig 8 .
Fig 8.An Example of fMRI Activations Acquired from One Participant Executing the Win-shift Task.This healthy participant was a 29 years old, male, right-handed, African American.While he was executing the Win-shift task, activations were detected in hippocampus, putamen, visual cortex, motor cortex, and cerebellum for the memory, visual, motion activities in the brain.