Immersive Virtual Reality Implementations in Developmental Psychology
Abstract
In recent years, immersive virtual reality technology (IVR) has seen a substantial improvement in its quality, affordability, and ability to simulate the real world. Virtual reality in psychology can be used for three basic purposes: immersion, simulation, and a combination of both. While the psychological implementations of IVR have been predominately used with adults, this review seeks to update our knowledge about the uses and effectiveness of IVR with children. Specifically, its use as a tool for pain distraction, neuropsychological assessment, and skills training. Results showed that IVR is a useful tool when it is used either for immersive or simulative purposes (e.g., pain distraction, neuropsychological assessment), but when its use requires both simulation (of the real world) and immersion (e.g., a vivid environment), it is trickier to implement effectively.
Introduction and basic definitions of key concepts
The introduction of virtual reality (VR) as a concept can be traced back to 1957 when Morton Heilig, a cinematographer, thought audiences would be more engaged with narratives if all their senses could be stimulated. The focus on immersion in Heilig's version of VR remains an important concept today, particularly in the realm of video gaming, where theorists and designers regularly assess VR environments for their impressiveness. Immersion in this context is the feeling of complete engagement or involvement in a 3-D, computer-generated world that perceptually surrounds the viewer using images, sound, or other stimuli to provide a captivating environment (Fallis, 2013). In contrast to entertainment VR, which focuses on immersion, applied commercial and industry uses of VR focus on simulation. As such, the success of the applied uses of VR depends on the extent to which the virtual experience simulates the real-world environment or situation for the user (Turner et al., 2016).
The concepts of presence, immersion and simulation are frequently discussed when considering virtual reality environments. Slater and Wilbur (1997) defined presence as the inherent function of the user’s psychology. In other words, presence is linked to the mental processes that occur when the user gets psychologically drawn into a virtual world, focusing on the experience of being part of the virtual environment (Biocca, 1997). Immersion, on the other hand, can be considered as a quality of the system’s technology and is related to its ability to present a vivid environment while shutting out the physical reality (Cummings & Bailenson, 2016).
According to Slater and Wilbur (1997), a system is more likely to shut out physical reality if it offers three different aspects: 1) high fidelity environments through stimulation of multiple senses, 2) accurately represents the users’ physical movements with virtual body actions, and 3) blocks the external world from the participant’s mind, permitting the user to then become psychologically engaged in the virtual environment. Therefore, as Cummings and Bailenson (2016, p. 274) state “the more immersive the system, the more likely an individual will feel present within the mediated environment and the more likely that the virtual setting will dominate over physical reality in determining user responses”.
Moreover, simulation is defined as an imitation of a particular appearance or form, and this term is used in virtual reality as the attempt to recreate characteristics of the real world (Beaubien & Baker, 2004). VR experiences can focus on different aspects of the technology. There are VR applications where its success relies on the capacity of the technology to accurately replicate the physical world (e.g., in military or aviation training). There are other VR applications where the simulation of the real world is not an essential aspect of the experience; its success relies on how well the user is psychologically involved in the environment (e.g., pain distraction experiences).
This review focuses on three different dimensions of VR technology to evaluate its effectiveness on different uses with children: immersion (vivid environment that blocks or distracts the user’s mind from the real world), simulation (accurate representation of the real world) and the combination of the two, immersion and simulation.
There are six different types of VR systems: computer-based VR (also known as non-immersive VR), mirror systems, vehicle-base systems, Cave automatic virtual environment (CAVE), immersive virtual reality (IVR) and augmented reality (Biocca & Levy, 1995). However, state-of-the-art VR systems typically make use of either IVR or CAVE systems, which are currently used in psychological implementations (Pan & Hamilton, 2018). Both types of systems involve certain level of immersion, and the use of multisensory computer-simulated environments that perceptually surround an individual, leading the user to believe they have stepped inside or can interact with the generated world.
--------------------------------
The following abbreviations are used throughout this review:
ABI acquired brain injury
ADHD attention deficit hyperactivity disorder
ASD autism spectrum disorders
CAVE cave automatic virtual environment
CPT continuous performance test
HMD head-mounted display
IVR immersive virtual reality
TD typically developing
TOVA Test of Variables of Attention
VR virtual reality
1.1. Virtual reality in psychological research and practice
A key reason for the growing profile of IVR in psychological research is that IVR is a multifaceted technology that can serve a range of purposes to create a VR environment that is immersive, simulative, or both. Psychological uses of VR vary in the extent to which these properties are exploited in a psychological implementation; for example, the immersive properties of VR have been investigated as tools in immersion therapy to help adults overcome phobias (Rothbaum et al., 2015). VR's simulative properties have also been used as a skills training tool for pilots, astronauts, and others (Oberhauser et al., 2018).
While the psychological implementations of IVR have been predominately used with adults, there is a growing interest in evaluating how IVR can be used with children (Araiza-Alba et al., 2020; Araiza-Alba et al., 2020; Aubrey et al., 2018). A specific focus on children is important for a number of reasons. As pointed out by Sharer et al. (2007), children's expectations while immersed in a VR system may differ from adults' expectations, leading to different outcomes when assessing the effectiveness of an IVR-based program. Similarly, children and adults may differ in the extent to which they react to a VR environment as a real physical experience, potentially influencing the effectiveness of VR used for immersive and simulative purposes. Additionally, using IVR-based interventions with children may provoke concern in a society that is already somewhat skeptical of children's growing media exposure (Huber et al., 2018). Therefore, researching the effectiveness of applied pediatric psychology uses of IVR is important to determine if the benefits of such programs outweigh the risks of additional media exposure. Here we offer a literature review of three uses of IVR with developmental populations: 1) for pain distraction for children; 2) as a neuropsychological diagnostic tool; and 3) as a social-skills training tool for children with autism spectrum disorders (ASD). These uses were specifically chosen because they differ in the extent to which their success relies on immersion, simulation, or a combination of the two. We were guided by our hypothesis that IVR would be a useful tool when it is either immersive or simulative, but cases that require both simulation (of the real world) and immersion would be more complicated to implement and therefore less effective. The primary purpose of this review is to provide a summary of which implementations of VR technology are more likely to be effective with children and what does not work with this population, as well as offer a starting point for future research. To that end, we describe the key procedural details and main findings of our studies in this review. We complement these descriptive summaries with the relevant statistical details (including when possible, means, standard deviations, inferential statistics, and effect sizes) in our supplemental analyses.
VR technology used as pain distraction for children
2.1. Historical context
While treating children with critical illness and injuries, the pediatric clinicians conduct a variety of distressing and painful procedures every day. Coping strategies and less painful interventions have been investigated for years. One of the most well-known coping strategies is distraction (Gershon et al., 2003), which has been used for many years with significant results. Its basic principle is that attention is diverted away from an unpleasant stimulus and is focused instead on a more pleasant one (Mccaul & Malott, 1984).
Some studies have shown that distraction decreases the affective and sensory mechanism of pain by reducing the activation of the brain areas associated with it, making distraction an effective strategy for reducing procedural pain, fear, and distress. When an individual's attention is occupied by a distracting task, the brain responds differently and results in a reduction of pain (Windich-Biermeier et al., 2007). The effectiveness of combining pharmacologic treatments and distraction techniques for pain management has been known for many years, and the introduction of non-immersive VR distractions seems to be one of the many ways researchers and clinicians are trying to reduce distress during medical procedures (Koller & Goldman, 2012).
Non-immersive VR (VR experiences that use desktop or laptop screens to present the virtual environment to a user) seems to work well as a distraction strategy in a number of medical contexts for children experiencing painful or fear-induce medical procedures. For example, playing video games during burn-wound care can lower pain and anxiety and also help with faster wound re-epithelialization (Brown et al., 2014). Nilsson et al., 2009) also reported that children with cancer who used video games while undergoing a needle-related procedure had reduced procedural pain and distress. Similarly, Gerçeker et al. (2018) found that the use of videos (watched on a HMD) and external cold and vibration are effective in reducing the pain in children during phlebotomy.
In the next section, we describe uses of IVR as a distraction from medical procedures, thus reducing pain and anxiety in children, and we also explore the idea that greater immersion should result in greater distraction and less pain. The main goal of VR in pain distraction is to immerse the mind of the patient in a world unconnected to the one in which they are undergoing a medical procedure, thereby taking their mind away from the present situation. The studies discussed next focus on the distraction potential of VR technology and do not rely on accurate simulations of the real world.
2.2. Literature review strategy
The purpose of our literature review was to identify papers that examined the effectiveness of IVR as a pain distraction tool. Our literature search was guided by the population, intervention, comparison, and outcomes of interest (PICO) model (Schardt et al., 2007), with the following inclusion requirements:
Population: The study must evaluate children and/or adolescents (aged 0–18 years old) and must have a total sample size of at least 10 participants per group.
Intervention: Must include an intervention using IVR as a distraction tool.
Comparison: Must include a non-IVR distraction intervention as a comparison group.
Outcomes: Must report on the effectiveness of IVR as a distraction tool.
2.2.1. Search strategy and study selection
The search strategy was devised through scoping the literature for keywords indexed in published journal articles using the terms: (child* OR paediatric OR pediatric) AND (pain OR anxiety) AND distract* AND "virtual reality.” Our original search was conducted in August 2018 and was updated in February 2019. Papers that self-identified as pilot or feasibility studies were excluded. An exhaustive search of the literature was conducted on (a) SCOPUS, which indexes the literature from life sciences, social sciences, physical sciences and health sciences and b) MEDLINE, which focuses on life sciences and biomedical literature. The search produced 72 results after removing duplicates.
The PICO inclusion criteria were applied to these papers, resulting in 27 studies that satisfied at least one criterion and 6 studies that satisfied all of them. The majority of the excluded studies did not use IVR, only tested adults, or did not compare IVR distraction to non-IVR distraction.
2.3. Review results
Our results found that IVR was most commonly used to distract children from the pain of medical procedures during treatment for burns or chorionic wounds, during venous access, and oncological care; however, the majority of the studies were either pilot, feasibility, or case studies (e.g., Gershon et al. (2004); Wint et al. (2002)) or did not satisfy the inclusion criteria.
From the six studies that satisfied all of the PICO inclusion criteria (see Table 1), five of them reported that VR is a useful and effective technology that can be used as a distraction tool with children. The majority of the results showed a significant reduction in pain and/or anxiety in children using IVR distraction compared with control groups or standard distractions (e.g., toys, television, books, and parental comforting) in a variety of pediatric medical procedures. The majority of the results were taken from the reports of the patients and their caregivers. Key examples of effective use of IVR to reduce pain or anxiety are described in the following sections.
2.3.1. Wound and burn care
Hua et al. (2015) demonstrated analgesic results by using IVR while treating patients with chronic wounds (e.g. leg ulcer). Sixty-five children, aged 4 to 16 years, with chronic wounds on their lower limbs, were randomly divided into two experimental groups, VR group and a control group that received standard distraction methods (toys, television, books, and parental comforting). The VR group was instructed to play a VR game, Ice Age 2, an immersive virtual game in which the participant controls Sid the Sloth, who slides down a snowy path collecting acorns and avoiding obstacles. The game was displayed in an HMD while the patients underwent dressing changes, including removing the dressings, cleaning the wound, and applying a new dressing.
They used three scales to measure the children’s pain before, during, and after the dressing change. Critically, the before pain scores were measured once the child was involved in the distraction intervention. Children self-reported their pain on the Wong-Baker FACES pain rating scale, caregivers rated their children’s observed pain on the visual analogue scale (VAS), and nurses rated the children’s observed pain and distress using the Face, Legs, Activity, Cry, Consolability (FLACC) scale. They also measured how long it took to complete the dressing change for each child. The VR distraction group had significantly lower pain responses compared to the control group on 8 of the 9 measures; the only measure that was not statistically significant was for the nurses’ ratings of pain before the dressing change. In addition, it took less time to complete the dressing change for children in the VR group compared to the control group. As reported in Supplemental Table S1. 1, these significant results were accompanied by medium to large effect sizes (0.55 ≥ ds ≤ 1.87).
A few limitations are worth noting. First, it is difficult to control for previous pain experiences because chronic wounds generally require multiple dressing changes. Second, the participants, caregivers, and nurses were not blinded to the participants’ assigned condition. Lastly, no pain or anxiety scores were taken before the intervention; a baseline score for pain would rule out the alternative explanation that the observed differences between the two groups were driven by pre-existing differences in pain rather than the intervention.
Similar pain distraction results were found by Jeffs et al. (2014) when using IVR with children and adolescents involved in burn treatment. Thirty children and adolescents 10 to 17 years of age with burn wounds were randomly assigned to one of three groups (28 participants completed the entire study). In the standard care group, participants (n = 10) received typical comfort from nursing staff and parents. In the passive distraction condition, participants (n = 10) watched a movie. In the IVR distraction group, participants (n = 8) played a VR interactive game, Snow World, where users scored points by throwing snowballs at snowmen, penguins, etc. This immersive VR environment was displayed using a tripod-arm device holding the VR glasses to avoid the exclusion of patients with burns to the head. Importantly, even though this equipment did not allow the same mobility and tracking of head movements as a regular HMD, the results were promising.
The Adolescent Pediatric Pain Tool word graphic rating scale was completed before assignment to condition (baseline, preprocedural measure) and after the burn wound care procedure. Participants also completed a number of other measures, which were controlled for in the analyses (e.g., state anxiety). Participants in the IVR group reported significantly less procedural pain than the PD group and was the only group to demonstrate a significant decrease in pain perception from baseline (preprocedural pain) to pain experienced during the procedure (Jeffs et al., 2014).
Jeffs et al. (2014) did not share two of Hua et al. (2015) limitations. Namely, they included a baseline measure of pain and also blinded the staff until the pre-procedure measures were completed. Nonetheless, it was not possible to blind staff to the intervention condition for the entirety of the study. Another limitation is the small number of participants in this three-group, between-subjects study. Additionally, the variation in days from the original burn injury and the individual's experience with previous wound care episodes could have potentially affected participants' perception of anxiety and pain. Finally, they used a generic VR environment rather than one specifically designed to reduce pain.
Kipping et al. (2012) also provides evidence of the potential of IVR as a distraction tool for burn patients. Forty-one children and adolescents (11–17 years) undergoing burn-wound care were divided to receive either a standard distraction (e.g., music, TV, or stories) or IVR distraction—a 3-D game, either Chicken Little or Need for Speed, where participants used a joystick to interact with the environment). Both groups received identical wound-care procedures and medication protocols.
Kipping et al. (2012) measured various acute pain outcomes, including adolescent self-reports of pain and nausea and caregivers' ratings of their pain (both using the Visual Analogue Scale [VAS]), nurses' ratings of their pain (using the FLACC scale), physiological measures (i.e., heart rate and oxygen saturation), and whether the patients required rescue doses of pain medication (Entonox). All measurements were taken at three time periods: baseline, after dressing removal, and during dressing application, and they analysed mean change scores by subtracting the baseline measures from the removal and application scores. Of the 8 pain continuous variables, the only significant difference was the nurses rated patients in the standard distraction group as experiencing more pain during the dressing removal than those in the VR distraction group. As reported in Supplemental Table S1. 2, this comparison produced a medium to large effect (d = .73, 95% CI [0.10, 1.37]). Kipping et al. also reported a reduction in the number of rescue doses given 3/20 (VR distraction group) vs. 9/21 (standard distraction group); however, the significance of this effect is questionable (see our supplementary analyses). Similar to the previous studies presented above, the limitations of this study are related to the impossibility of a double-blind study and the use of a generic VR equipment and game that were not tailored for pain relief or children's use.
2.3.2. Research of pain control (cold-pressor test)
Apart from the uses mentioned above where IVR has been implemented during a pediatric treatment, IVR has also been used in other research of pain control for children, demonstrating its potential as a distraction. Sil et al. (2012) and Dahlquist et al. (2009, 2010) used IVR as a distraction tool for a cold-pressor test. The three studies showed that children using IVR significantly improved their pain tolerance relative to the baseline. The IVR game, however, did not outperform traditional interactive video games. For instance, Sil et al. (2012) found no significant difference between the regular and VR videogame conditions on children's pain tolerance. Moreover, Dahlquist et al. (2009) was the only study to demonstrate that the distracting benefits of IVR might be stronger for older children than younger children. Please see our Supplemental Analyses for summary and inferential statistics for these studies.
2.4. Conclusion: effectiveness of IVR for pain distraction
The results of this review reveal that when IVR distraction is used along with standard analgesic therapy, pediatric patients can have a clinically meaningful degree of pain relief while undergoing a range of medical procedures. This finding is consistent with other reviews that have focused primarily on adult patients (Mallari et al., 2019) and with a recent metanalysis done with pediatric population (Eijlers et al., 2019) where large effect sizes indicate that VR is an effective distraction intervention to reduce pain in pediatric patients undergoing a wide variety of medical procedures. The apparent effectiveness of IVR for pain distraction in children is consistent with our guiding hypothesis that applications requiring immersion but not simulation would be more easily implemented than applications requiring both simulation and immersion. In the literature described above, the common feature of each application was that it was designed primarily to distract children from the painful procedure taking place in reality. Whether or not the VR environment was effective at simulating a real environment was largely irrelevant.
2.5. Gaps and suggestions for future research
IVR clearly has potential as a pain distraction tool with children. However, an interesting question arose while studying the relationship between IVR distraction and pain management—does the effectiveness of IVR distraction techniques continue during consecutive treatments or uses? The vast majority of the studies in children measured the effect of IVR during a single session, but Faber et al. (2013) demonstrated that immersive VR distraction has the potential to continue effectiveness when used for three (or possibly more) consecutive treatments during severe burn-wound debridement in adults. Therefore, further research with children is needed to assess the efficacy of IVR as a possible tool used for children with chronic pain and as a tool for long-term pain rehabilitation.
Treatment of chronic pain is another area where IVR may have benefits; however, in general there is not much research on treatments for children. Most of the treatments are normally extrapolated from research on adults, and even those studies show that the evidence for effectiveness is limited (Eccleston & Malleson, 2003). Information is lacking about the effectiveness of IVR combined with other nonpharmacological treatment modalities and about the possibility that IVR can generate neurophysiologic changes leading to a reduction in need for pain medication (Gupta et al., 2018; Li et al., 2011; Shiri et al., 2013). To date, only a few studies have investigated VR for chronic pain management, and the data are preliminary or they focused on an adult population (Wiederhold et al., 2014). Therefore, there is a need for further research that explores these possibilities in children.
Another important question about the value of IVR as a distraction tool is related to its potential to enhance the distraction effect compared to traditional video games. For example, in the studies by Sil et al. (2012) and Dahlquist et al. (2009, 2010) described above, IVR did not enhance the effect of distraction over traditional video games. Nonetheless, it is possible that these results could be attributed to the use of out-of-date VR technology (not as compelling and immersive as the latest VR technology). Hence, studies of IVR are necessary to investigate not only the possible enhancement of distraction effects when using leading-edge technology, but also the possible difference between using IVR in a passive way (e.g., using the HMD to watch a VR film or calm/meditation experience) or in an interactive way (e.g., IVR interactive game). During our literature review, we found two studies (that did not fit in our inclusion criteria) that investigated the difference between interactive IVR versus passive IVR (Dahlquist et al., 2007; Law et al., 2011); however more research is need to determine what aspects of the interactivity, such as type of game or cognitive processes involved during the activity, are needed to generate greater benefits as a distraction tool.
Additionally, future research efforts should investigate the idea of IVR as a relaxation tool (to prepare the patient for a medical procedure) rather than just as a distraction mechanism and analyze the effects that this technology could have on reducing the fear of routine medical procedures in children. The preliminary results of pilot studies have shown the potential of VR relaxation to manage pain and distress or anxiety (Arane et al., 2017). However, more studies are needed to reproduce these results in large, randomized control studies.
Finally, there is an important limitation on the study of IVR as a distraction tool for medical procedures. The majority of studies that compare the effectiveness of IVR with other types of distraction do not take in consideration that perhaps the mere fact of wearing an HMD (that blocks the view of the medical procedure) is the responsible from the distraction and reduction of pain effect, and not the immersion generated from the technology. To overcome this limitation, future research should compare IVR with other interventions in which the medical procedure is blocked from the children’s view.
Table 1: IVR used as pain distraction for children
Authors |
Population N (age range) |
Intervention |
Comparison |
Outcomes |
Type of pain treatment or cold-pressor pain test |
Type of VR technology |
|---|---|---|---|---|---|---|
41 children (6–14 years old) |
VR game: participants used a joystick to play a videogame that was presented through a 3-D HMD helmet with integrated headphones |
Video game: presented via a computer screen and stereo speakers Baseline: no distraction |
Both distraction conditions resulted in improved pain tolerance relative to baseline. Older children showed more pain tolerance in the VR condition than younger children. |
Cold-pressor pain |
HMD with a joystick |
|
Dahlquist al. (2010) |
50 children (6–10 years old) |
VR game: participants used a joystick to play a videogame that was presented through a 3-D HMD helmet with integrated headphones |
Video game: presented via a computer screen and stereo speakers Baseline: no distraction |
Children demonstrated significant improvements in pain tolerance during both distraction conditions. No significant difference between the traditional and VR videogame conditions. |
Cold-pressor pain |
HMD with a joystick |
65 children (4–16 years old) |
VR game using an HMD and a joystick. |
Standard distraction methods: toys, TV, books, and parental comforting |
Children in the IVR group reported significantly less pain compared with the control group before, during, and after dressing changes than the control group. The caregivers’ and nurses’ ratings of the children’s pain was also lower for the IVR group than the control group. |
Pain relief of chronic wound during dressing change |
HMD with a joystick |
|
30 children (10–17 years old) |
VR game with a joystick. |
Standard care: nurse and parents’ comfort; Passive distraction: movie |
IVR group was the only group to have an estimated decrease in pain perception from baseline (preprocedural pain) to procedural pain. |
Pain reduction in burn patients |
HMD on a custom-built, articulated-arm tripod device |
|
41 children (11–17 years old) |
VR game with a joystick |
Standard distraction: access to TV, stories, music, caregivers, or no distraction in the treatment room, per personal choice and standard practice |
Nursing staff reported a statistically significant reduction in pain scores during dressing removal. Fewer patients in the VR group than the standard distraction group required rescue doses. No significant differences between groups in patients’ self-reported pain or caregivers’ ratings of patients’ pain. |
Pain reduction in burn care |
HMD |
|
62 children (6–13 years old) |
VR game |
Traditional videogame: Nintendo Wii console operated by a wireless handheld controller; Baseline: control condition with no distraction |
Children demonstrated significant improvement in pain tolerance during both videogame distraction conditions compared to the baseline. No significant difference between the traditional and VR videogame conditions. |
Cold-pressor pain |
HMD |
IVR used as a neuropsychological tool for children
3.1. Historical context
Neuropsychological testing involves the use of structured-systematic behavioral observations, paper-and-pencil tasks, and computerized psychometrics to assess brain and psychological functioning (Walsh & Darby, 1999). Although, technological advancement has influenced methods for neuropsychological testing (Brahnam et al., 2011), computer-based neuropsychological assessment in clinical diagnostic practice has recently become more common as key advantages to adopting technology have been identified. Such advantages include more time-efficient scoring and reducing errors in test administration (Galindo-Aldana et al., 2018). Moreover, when used with children, other benefits have been noted. For example, children find the computerized testing format to be more interesting and motivating than paper-and-pencil testing (Luciana, 2003).
The idea of potentially using VR in the area of neuropsychology first emerged in the mid-1990s; however, the technology was not fully ready to deliver compelling experiences that could recreate a real environment and provide a cognitive challenge for the participants. Initial work in this area was led by a small number of researchers exploring VR tools to assess cognitive performance in patients with central nervous system dysfunction (Cromby et al., 1996; Rose et al., 2001). Although, the majority of their work did not involve immersive VR systems, the results highlighted the potential value of this technology to assess cognitive function while maintaining decent ecological validity (Brahnam et al., 2011; Rizzo et al., 2004)
Whether neuropsychological tests involve computer-based testing or not, neuropsychology testing arguably suffers from low to moderate ecological validity because the subjects are tested in isolation from real-life complexity. For example, testing normally occurs in a quiet room free of distractions, which does not represent the challenges that people face in everyday life (Chaytor & Schmitter-Edgecombe, 2003; Nolin et al., 2016; Parsons et al., 2007). Rizzo et al. (2004) has suggested VR as a possible means to enhance the ecological validity of neuropsychology testing because it is capable of introducing visual complexity by simulating a real environment (e.g., adding distractions, interaction with 3-D objects, etc.), therefore allowing greater control and replicability during diagnostic testing (Rizzo et al., 2004). This technology has already been applied to testing procedures for a number of cognitive domains with adults, including executive function, attention, impulsivity, cognitive and motor inhibition, memory and learning, spatial abilities, and visuospatial neglect (Neguţ et al., 2016).
Next, we detail our literature review on the uses of IVR for neuropsychological diagnosis or assessment in children. The following studies rely on the potential of IVR to generate an effective, engaging simulation of the real world to replicate the neurocognitive challenges that the person could face in real life; for example, maintaining attention while distractions are happening.
3.2. Literature review strategy
The purpose of this literature review was to identify papers that examined the effectiveness of using immersive VR in the area of neuropsychology with children. Our search was guided by the PICO model for literature reviews (Schardt et al., 2007), with the following inclusion requirements:
Population: The study must evaluate and test children and/or adolescents (aged 0–18 years old) and must have a total sample size of at least 10 per group.
Intervention: Must include implementation of IVR (with an HMD) as a neuropsychological assessment or training tool.
Comparison: Studies must have a control group (traditional neuropsychological assessment or non-IVR intervention) and provide comparison with an IVR assessment or training tool (Traditional vs IVR).
Outcomes: Must report on the effectiveness of the neuropsychological assessment or training tool using IVR.
3.2.1. Search strategy and study selection
The search strategy was devised through scoping the literature for keywords indexed on published journal articles using the “terms”: "virtual reality" AND child* AND (neuropsych* OR assessment OR rehabilitation OR "attention deficit" OR "brain injury" OR disorder). Our original search was conducted in August 2018 and was updated in February 2019. Papers that self-identified as pilot or feasibility studies were excluded. An exhaustive search of the literature was conducted on (a) SCOPUS, which indexes the literature from life sciences, social sciences, physical sciences and health sciences and b) MEDLINE, which focuses on life sciences and biomedical literature. The search produced 529 results after removing duplicates. We applied the PICO inclusion criteria to these papers, resulting in 17 studies that satisfied at least one criterion and only 7 studies that satisfied all of them. The majority of the excluded studies did not use an immersive VR system or did not test children.
3.3. Neuropsychology review results
Immersive virtual reality technology has been used in the area of child neuropsychology mostly as an assessment tool to evaluate attention processes in children (Nolin et al., 2016) with attention hyperactivity disorder (ADHD) and acquired brain injury (Díaz-Orueta et al., 2014; Gilboa et al., 2015; Neguț et al., 2016; Rodríguez et al., 2018). It had also been used as a training tool for attention enhancement for children with ADHD (Bioulac et al., 2012; Blume et al., 2017) and with behavioral problems (Cho et al., 2002), and as a neuromotor rehabilitation tool for children with cerebral palsy (Bortone et al., 2018). However, most of the research related to IVR as a rehabilitation tool is either proof of concept or in the early stages of testing the study protocol for a randomized controlled trial (e.g., Bortone et al., 2018; Biffi et al., 2017; Bioulac, 2015).
Our review identified three distinct streams of research into the effectiveness of IVR as a neuropsychological diagnostic tool with children. The first stream includes research that specifically assessed the validity and reliability of an IVR assessment by comparing it to a traditional gold-standard assessment. Included in the second stream is research that sought to determine if IVR diagnostic tools may be more effective than traditional tools to identify children with ADHD. Research in the third stream more generally examined the effectiveness of an IVR tool to assess attention performance not necessarily related to a specific neuropsychological diagnosis (see Table 2). The main research findings from each of these streams is summarized below.
3.3.1. Validity and reliability studies
Two studies identified in this review were aimed at assessing the validity and reliability of IVR versions of a continuous performance test (CPT), normally used to measure a person's attention while screening for ADHD. Díaz-Orueta et al. (2014) and Nolin et al. (2016) compared an immersive virtual reality (IVR) CPT test to traditional tests of attention function, Conners CPT and VIGIL-CPT.
Díaz-Orueta et al. (2014) explored the convergent validity between the AULA Nesplora test (IVR test; Climent and Banterla (2010) and the Conners CPT (traditional test). The Conners CPT is a computerized test that is widely accepted as a gold standard for assessment of attention-related problems; it consists of 360 stimuli (letters) that appear on a screen, and the participant needs to press a button every time a letter appears on the screen, except for the letter “x” (inhibition of the response). Similarly, the AULA Nesplora test is a CPT that takes place in a virtual school, where the child is situated in a virtual context shown through an HMD. Stimuli are presented both on a visual and auditory basis, while distractors of an ecological nature appear progressively (e.g., sound of door closing, children chatting). The participant is asked to do two different tasks: either press the button every time they perceive the target stimulus or press the button when they do not perceive it.
In this study, 57 children, aged 6–16 years, with a diagnosis of ADHD underwent both tests. The results of the two tests were significantly correlated (rs ranged from .303–.786), demonstrating convergent validity of the AULA Nesplora test of attention and impulsivity. Moreover, the AULA Nesplora test (but not Conners CPT) was able to differentiate between ADHD in children with and without pharmacological treatment for a wide range of measures related to inattention, impulsivity, processing speed, motor activity, and quality of attention focus, making this test a useful complementary tool for the diagnosis of ADHD (see our Supplemental Analysis). Some limitations found in this study are that the sample size could be considered as a relatively small clinical sample of children with ADHD and that comorbidity with other psychological diagnosis was not considered as part of the inclusion/exclusion criteria (Díaz-Orueta et al., 2013).
Similarly, Nolin et al. (2016) conducted a concurrent validity and reliability study that explored the relationship between performance on the standard VIGIL-CPT, presented on a computer monitor, and ClinicaVR: Classroom-CPT, an IVR version of the same test. Interestingly, the ClinicaVR test adds distractions for the children to cope with during testing by placing children inside a virtual classroom, rather than in a quiet office typical of CPT testing. Participants could look 360 degrees around themselves, an experience designed to create an impression that they were in a real classroom with typical visual and auditory distractions (e.g., knocking on the door, bells, children talking, etc.). Their participants were 102 children and adolescents, ranging in ages from 7–16 years old. Despite the differences between the ClinicaVR test and the traditional version of the test, participants' scores on the overlapping components (correct responses, commissions, and reaction time) were significantly correlated (rs = .63, .50, .82, respectively). In addition to demonstrating convergent validity, the study also demonstrated that ClinicaVR had good test–retest reliability with a one-month period between tests. Reaction time was the only variable that showed poor reliability (r = .13, ns); all other variables were significantly correlated (rs = .34–.61). Nolin et al. argued the IVR version may have an advantage in terms of ecological validity because it includes distractions likely to occur in real-life scenarios. This possibility is revisited in the following section.
3.3.2. Relative effectiveness of IVR versus traditional ADHD diagnostics
Although the studies described in the previous section aimed to establish the validity of new IVR tests by comparing them to traditional tests (e.g., TOVA, Conners CPT, VIGIL-CPT) as the accepted standard, recent IVR studies have gone further to investigate if IVR tests might be more effective at diagnosing attention deficits in children (Neguț et al., 2016; Pollak et al., 2009; Rodríguez et al., 2018).
Neguț et al. (2016) tested 7–13 year old children with a traditional CPT (displayed on a computer screen) or with ClinicaVR: Classroom-CPT (described above). The children were classified as typically developing (n = 42) or as being diagnosed with ADHD (n = 33). Each child completed the test with and without distractions (Neguț et al., 2016). For the CPT with distractions, the audio recording from the ClinicaVR was played through headphones.
The key dependent variables were total correct responses, errors of commission, errors of omission, and mean reaction time. Neguț et al. (2016) found that, independent of their test condition, children with ADHD made fewer correct responses, committed more errors of commission and omission, and had slower reaction times compared with the typically-developing children (see our Supplemental Table S2. 1). They did not find a significant difference between the effectiveness of ClinicaVR and traditional CPT in identifying children with ADHD.
These results are in line with Pollak et al. (2009) who compared three CPT's: (i) VR-CPT (similar to the ClinicalVR used in Neguț et al., 2016), (ii) the same CPT without VR (No VR-CPT), and (iii) the TOVA. They used a within-subjects design so that the children (20 boys with ADHD and 17 typically-developing boys) completed all three tests. Their four key measures were reaction time, variability of reaction time, errors of omission, and errors of commission. All three tests were able to distinguish children with ADHD from the control group based on their increased errors of omission and commission (with large effect sizes, ds ≥ 0.88; see Supplemental Table S2. 2). The results from the reaction time measures were more variable (as reflected in the range of effect sizes, ds = 0.21–1.10), but only reaction time was associated with a significant Group x Test interaction. With the measure of reaction time, only the VR-CPT found significant differences between the two groups. With regard to reported experience with the assessment, participants enjoyed the VR-CPT more than the TOVA.
Taken together, the results from Neguț et al. (2016) and Pollak et al. (2009) further demonstrate the validity of IVR tests of attention, but do not show that these tests are any more sensitive for identifying attention difficulties compared with traditional testing. In contrast, Rodríguez et al. (2018) suggested that the AULA Nesplora IVR version of the CPT has better specificity and sensitivity to identify ADHD in children than the traditional TOVA-CPT. Their participants were between 6 and 16 years of age; 101 children were in the control group and 237 had ADHD diagnoses. Given their large sample of children with ADHD, they were able to classify them according to whether they presented as inattentive (n = 108), impulsive hyperactive (n = 52), or combined (n = 77).
Children were randomly assigned to complete the IVR-CPT test or the traditional TOVA-CPT assessment (Rodríguez et al., 2018). The IVR-CPT test consisted of two tasks that were explained by a virtual teacher in the virtual classroom (a standard classroom with lines of desks and a board in the front). The first phase of the test was based on the “no-go” paradigm in which the participant must press a button when they saw a stimulus on the virtual board or heard a stimulus, but only if the stimulus was not “apple.” In the second phase of the test, a “go” task was included; participants needed to press a button when they saw or heard the number “seven.” Auditory and visual distractions were included during the duration of the test. The TOVA-CPT was administered using a computer screen and a push-button. The test presented two simple images; the first part of the test showed the stimulus at the top of the screen and the second at the bottom. Participants were instructed to press the button every time they saw the target figure and not press it when the nontarget figure appeared.
Both tests in Rodríguez et al. (2018) measured errors of omissions, errors of commissions, response time, and variability of response time, but the tests reported them differently. The IVR-CPT reported these measures separately for each task (“no-go” and “go”), whereas the TOVA-CPT split the measures across four quartiles and two halves. The results reported by Rodríguez et al. (2018) demonstrated that the AULA Nesplora IVR-CPT was able to discriminate between children with and without ADHD symptoms, whereas the TOVA was not able to discriminate between the two groups (see our Supplemental Analysis).
The studies in this area demonstrated that the VR tasks are at least as useful as the traditional tasks in neuropsychological testing for attention difficulties. Whether they are an improvement over traditional methods is harder to determine given the mixed results. Therefore, at this stage, it is unclear if using a VR version of a traditional neuropsychological test with added distractors increases the effectiveness, sensitivity and specificity of the test to identify ADHD in children (Areces et al., 2016; Rizzo et al., 2000).
3.3.3. IVR to assess sustained attention performance
While the studies described above were specifically aimed at examining the diagnostic effectiveness of IVR for ADHD, IVR has also been used to examine attention performance more generally in two additional studies (Bioulac et al., 2012; Gilboa et al., 2015). Bioulac et al. investigated IVR as a tool to assess the sustained attention performance of 36 boys (20 with ADHD and 16 without), aged 7–10 years old. All boys were tested first with the traditional CPT test and then with an IVR test (virtual classroom). They used a French version of the virtual classroom developed by Rizzo et al. (2000), which consisted of a standard rectangular classroom environment with desks, blackboard, windows, and so forth. Within this scenario, children's attention performance was assessed while a series of common classroom distractors (e.g., ambient classroom noise, activity occurring outside the window, etc.) were systematically played within the virtual environment. Participants were asked to focus and maintain their attention on the letters appearing on the backboard to identify every time that the letter “K” appeared after being immediately preceded by the letter “A”. To do so, the participants needed to avoid the distractions. The virtual classroom test consisted of five blocks (for a period of 100 s each) with 20 targets, whereas the CPT comprised six blocks (for a period of 140 s each) with up to 54 targets.
Bioulac et al. (2012) showed that participants with ADHD performed worse than the control subjects on some, but not all, measures of both tests. Specifically, for the virtual classroom, significant effects emerged on total correct hits and total errors of commission, but not on the three reaction time measures. For the CPT, significant effect emerged on total correct hits and the two reaction time measures, but not on errors of commission (see Supplemental Table S2.3). Moreover, the virtual classroom was more precise at identifying deteriorating performance over time, whereas the traditional test was not (see Supplemental Table S2.4). The authors argued that this difference could be attributed to the involvement of more complex cognitive mechanisms in the IVR test. That is, the traditional test examined inhibitory processes, but only minimal working-memory load was required to solve the task. In contrast, in the IVR test, inhibitory processes were necessary to avoid distractors and working memory was necessary for keeping in mind the last stimulus presented on the blackboard and identifying the correct sequence, thereby increasing the cognitive load required to solve the test. Bioulac et al. concluded that the realistic and lifelike environment that the IVR test offers, along with its ability to measure deterioration over time, make IVR a useful tool to assess attention. However, two limitations of Bioulac et al. must be acknowledged: they had a sample size and the participants were exclusively boys.
Comparably, Gilboa et al. (2015) found IVR to be a useful, sensitive, and ecologically valid tool to assess attention among children with acquired brain injury (ABI). Gilboa et al. used a similar version of the virtual classroom described in Bioulac et al. 2012 (Rizzo et al., 2000) and compared it with two classical neuropsychological tests: (i) four sub-tests of the Test of Everyday Attention for Children (TEA-Ch), and (ii) Conners' Parent Rating Scales–Revised: Short Form (CPRS-R:S) questionnaire.
Children and adolescents aged 8 to 18 years with ABI (n = 41) and without ABI (n = 35) completed the TEA-Ch (completed only by those with ABI) and the VC assessment, and their parents completed the CPRS-R:S. In terms of the effectiveness of the virtual classroom assessment in differentiating between children with and without ABI, the two groups were significantly different on only one of the measures: the number of correct hits. The two groups did not significantly differ on errors of commission, reaction time, and head movements, as measured by the virtual classroom. The CPRS-R:S test scores also showed limited ability to differentiate between the two groups; after correcting for multiple comparisons, the authors reported that the only significant effect emerged on the ADHD index, but not on the opposition, inattention, or hyperactivity clinical subscales. Our re-analysis (see Supplementary Table S2. 5), however, found differing results, such that the ADHD index comparison did not reach significance (p = .0163 fell below the corrected alpha of .0125). Compared to the norms of the standardized TEA-Ch, the patients with ABI had significant impairments on the four sub-tests that measured various domains of attention (Gilboa et al., 2015).
Gilboa et al. (2015) also examined the relationships between the three tests and found that the virtual classroom measures were significantly correlated with several, but not all, components of the traditional tests, indicating some degree of concurrent validity. For example, for the ABI group, the virtual classroom's total correct hits significantly correlated with the Sky search scale of the TEA-Ch (r = 0.35, p < 0.01). Likewise, the virtual classroom's reaction time measure was negatively correlated with the TEA-Ch's Sky search dual task sub-test (r = -0.54, p < .005). Of the sixteen correlations between the four virtual classroom measures and the four clinical scales of the CPRS-R:S, only three were statistically significant. Specifically, the virtual classroom's total correct hits were negatively correlated with two clinical subscales: inattention (r = -0.34, p < 0.01) and ADHD index (r = -0.31, p < 0.01). In addition, the virtual classroom's hit reaction time was positively correlated with the inattention subscale (r = 0.28, p < 0.05).
Moreover, Gilboa et al. (2015) highlighted correlations of the virtual classroom with age and injury-severity variables, reflecting the ability of the virtual classroom to detect developmental aspects of attention in ABI. They concluded that IVR provides information beyond the scope of currently available assessments, essentially because it allows the user to interact in real-time with the lifelike simulated environment, with relative ecological validity and no risk of harm. This study had two key limitations: (a) the ABI group was, on average, 12 months older than the control group, and (b) the control group did not complete the TEA-Ch sub-tests, limiting the number of comparisons possible across groups.
3.4. Conclusion: effectiveness of IVR in neuropsychological testing
Taken together, the results of this review show that neuropsychological IVR testing is a useful technique when used to simulate real-life environments for assessment purposes. First, it is a valid and reliable technology that can assist in neuropsychological assessment for conditions such as ADHD and ABI. Second, it is a useful tool to assess attention performance in general, not necessarily related to a specific neuropsychological diagnosis. Importantly, research in each stream suggested that IVR improves upon the low to moderate ecological validity of standard psychometric instruments; this technology is capable of reasonably simulating the complexity and challenges that children confront in real life. The apparent effectiveness of IVR as a neuropsychological tool is consistent with our guiding hypothesis that it could be a valid and useful implementation when used to simulate the real world to test the cognitive abilities of children.
3.5. Gaps and suggestions for future research
Although the aforementioned studies demonstrated that IVR could be a useful technology for neuropsychological assessment, studies using this technology must overcome some limitations before IVR can be used as a conventional psychometric instrument. More research is needed with larger sample sizes, narrow age-range groups, and with more diverse populations.
From a cognitive standpoint, research is needed to ascertain how and why IVR-based assessments may allow for improved diagnostics. Specifically, research to date has assumed that adding virtual distractions would manifest in more ecologically-valid testing. However, the diagnostic effects of the nature and number of distractions has not been systematically investigated. For example, are tests improved with constant or intermittent distractions? Should distractions have visual and/or auditory components? Could the ideal distraction parameters be patient-specific? More theoretically, why is it that distractions affect diagnostic utility? The latter may relate to increased cognitive load, or it may allow the test to better identify attention problems in situations that are more contextually realistic to learning in a school environment (where attention problems are often pronounced). Clearly, continued research is vital in this important area.
Additionally, IVR could be a useful tool to assess the therapeutic effectiveness of interventions; for example, the results of neurocognitive training in ADHD could be better tested in IVR. This technology allows a more naturalistic environment in which the participant’s ability to control or avoid distractions can be tested in more ecologically valid environment. Immersing the participant in a computer-generated environment that simulates the real world and represents the challenges that a child with ADHD could face in an actual classroom might be a more effective way to assess therapeutic outcomes.
Importantly, the research described above related entirely to neuropsychological studies of attention assessment. Consequently, future study is needed of the cost-effectiveness and applicability of IVR for other executive functions and skills, such as inhibitory control, working memory, cognitive flexibility, reasoning, problem solving, and motor and perception functioning. Such research is essential to ascertain the potential for IVR to take on a wider role in neuropsychology. Furthermore, assessing if IVR also has the potential to be a useful tool in rehabilitating cognitive functions would be valuable. For example, researchers could study if the level of immersion and interactivity that IVR offers is a crucial factor in ecological validity, children’s motivation, and transfer of learning to real life.
Table 2 IVR used as a neuropsychological tool for children
Type of use |
Authors |
Population N (age range) |
Intervention |
Comparison |
Outcomes |
Type of VR technology |
|---|---|---|---|---|---|---|
Validity and reliability studies |
Díaz-Orueta et al. (2013) |
56 children with ADHD (6–16 years old) |
AULA Nesplora test (IVR test) |
Traditional test: Conners CPT (computerized test) |
The two tests were significantly correlated, demonstrating convergent validity of the AULA Nesplora test of attention and impulsivity. AULA Nesplora test (but not Conners CPT) was able to differentiate between ADHD children with and without pharmacological treatment. |
IVR using HMD |
102 typically- developing children (7–16 years old) |
ClinicaVR: Classroom-CPT (IVR test) |
Traditional test: VIGIL-CPT |
Good concurrent validity; ClinicaVR measures were significantly correlated with the VIGIL-CPT. Good test–retest reliability of the ClinicaVR. |
IVR using HMD |
||
Effectiveness of IVR vs traditional ADHD diagnostics |
33 children with ADHD and 42 children without ADHD (7–13 years old) |
ClinicaVR: Classroom-CPT (IVR test) |
Traditional CPT (displayed on a screen computer) |
The results showed that, independent of the type of test, children with ADHD made fewer correct responses, committed more errors of commission and omission, and had slower reaction times than typically-developing children. The study did not find a significant difference between the effectiveness of ClinicaVR and traditional CPT in identifying children with ADHD. |
IVR using HMD |
|
20 children with ADHD and 17 children without ADHD (9–17 years old). |
VR-CPT (IVR Test) |
Traditional tests: No VR-CPT (displayed on a screen computer) & TOVA |
Children with ADHD made more errors of omission and commission than the control group on all three tests. For response time, only the VR-CPT differentiated between the two groups. Participants reported enjoying the VR-CPT more than the TOVA. |
IVR using HMD |
||
237 children with ADHD and 101 without ADHD (6–16 years old) |
VR-CPT (AULA Nesplora) |
Traditional test: TOVA |
VR-CPT (AULA Nesplora) showed better sensitivity and specificity than the traditional CPT (TOVA). The VR-CPT was able to discriminate between children with and without ADHD symptoms, whereas the TOVA could not. |
IVR using HMD |
||
IVR to assess sustained attention performance |
20 boys with ADHD and 16 boys without ADHD (7–10 years old) |
Virtual Classroom (IVR Test) |
Traditional test: Continuous CPT |
Participants with ADHD performed worse than the control group on two measures of the IVR test and three measures of the CPT test. However, the IVR test was more precise in identifying the deterioration of performance over time than the traditional test. |
IVR using HMD |
|
41 children with ABI and 35 children without ABI (8–18 years old) |
Virtual Classroom (IVR test) |
Traditional tests: Test of Everyday Attention for Children & Conners’ Parent Rating Scales–Revised: Short form questionnaire |
The virtual classroom was sensitive to effects of an ABI, but only for total correct hits. The other measures (commission errors, reaction time, and head movements) were not significantly different between groups. The virtual classroom measures significantly correlated with some measures from the traditional tests, indicating a degree of concurrent validity. |
IVR using HMD |
IVR used as a social-skills training tool for children with autism spectrum disorder
4.1. Historical context
For more than 20 years, researchers have suggested that VR technology could be used as an effective therapeutic intervention for children with ASD (e.g., Parsons & Mitchell, 2002). In their review, Parsons and Mitchell identified several studies suggesting that VR could help children and adults with autism by providing social-skills training. VR might allow those with autism to practice their social skills through role-playing exercises in virtual environments that would be less likely to cause high levels of stress often experienced by people with autism in live social interactions (Bozgeyikli et al., 2016; Ke and Im, 2013; S. Parsons, 2016; S. Parsons and Cobb, 2011; Ramachandiran et al., 2015).
Yet, while the promise of VR has been acknowledged repeatedly (Didehbani et al., 2016; Ramachandiran et al., 2015; Strickland, 1997), most published work in this area is best described as feasibility or proof-of-concept studies with small samples sizes and without conditions to control for time, practice effects, and other confounds associated with pre- and post-test research (e.g., Zhao et al., 2018; Kuriakose & Lahiri, 2017; Halabi et al., 2017). Moreover, very few used IVR technology, the main focus of this review.
Such feasibility studies have not provided satisfying answers to the question of whether IVR, or VR more generally, could be useful as part of an autism intervention program. Instead, these studies showed that for children with and without autism, interacting in a virtual environment was generally interesting and motivational (Halabi et al., 2017) and may have allowed them to engage in simulated interactions without significant stress (Kuriakose & Lahiri, 2017). Children with autism also seemed able to manipulate the equipment, when necessary (Strickland et al., 1996). Only recently however, have well-controlled studies using IVR technology begun to assess the effectiveness of potential interventions; we discuss these studies in more detail in the Section 4.3.
4.2. Literature review strategy
The purpose of our literature review was to identify papers that examined the effectiveness of autism therapies involving the use of IVR. Our search was guided by the PICO model for literature reviews (Schardt et al., 2007), with the following inclusion requirements:
Population: The study must evaluate and test children and/or adolescents (aged 0–18 years old) with autism and must have a total sample size of at least 10 participants per group.
Intervention: Must include a therapy program that involved the use of IVR.
Comparison: Must include a non-IVR control group.
Outcomes: Must report on the effectiveness of the IVR therapy.
4.2.1. Search strategy and study selection
The search strategy was devised through scoping the literature for keywords indexed in published journal articles using the “terms”: (“Virtual reality” AND “autism” AND children). Our original search was conducted in August 2018 and was updated in February 2019. Papers that self-identified as pilot or feasibility studies were excluded. An exhaustive search of the literature was conducted on (a) SCOPUS, which indexes the literature from life sciences, social sciences, physical sciences and health sciences, and b) MEDLINE, which focuses on life sciences and biomedical literature. The search produced 110 results after removing duplicates. The PICO inclusion criteria were applied to these papers, resulting in 27 studies that satisfied at least one criterion and only 2 studies that satisfied all of them. The majority of studies did not use an HMD or were done with adults.
4.3. IVR used as a social-skills training tool
4.3.1. Review results
Only two IVR studies in our search provided potentially useful data on the effectiveness of VR as a social-skills training tool for children with autism (see Table 3). Lorenzo et al. (2016) developed and evaluated an elaborate, IVR-based program created to provide an opportunity for children with ASD to practice a range of social and emotional skills. The study used a system with L-shaped screens called “semi-CAVE” wherein one can navigate with special 3-D glasses and included a face-recognition system that automatically recorded children's facial expressions to assess mood and emotional reactions. Participants were 40 children (aged 7–12 years) diagnosed with ASD and were assigned to one of two groups: the experimental group, which experienced the semi-CAVE system, and the control group, which used a non-immersive VR application.
The scope Lorenzo et al. (2016) sets itself apart from other research in this area in a number of ways. First, unlike other studies that typically examined children for very brief periods of time, this study examined children over 40 sessions during a 10-month period. Second, this study examined behavior both during the training sessions and in the classroom environment. During the training sessions, children navigated the VR environment in which they experienced a range of simulated social situations with the guidance of a real-life evaluator. Some of these situations included interactions at a birthday party, conversations with other students outside the classroom, participation in games such as hide-and-seek, etc. The evaluator asked the children during these sessions to explain how the other virtual children were feeling (based on the situation and their apparent affect).
Lorenzo et al. (2016) reported that children in the experimental training group demonstrated significantly greater improvement in nearly all the measured social skills, relative to the children in the control group. For example, children in the experimental group improved in identifying emotions and starting a conversation at appropriate times. The face-recognition data was consistent with this finding, demonstrating that, with training, children in the experimental group made fewer inadequate emotional expressions and displayed more emotionally appropriate behaviors, whereas children in the control group improved less. During the course of the study, the children's teachers (in their actual school) reported that those in the experimental group showed significantly more improvement in their emotional behaviors than children in the control group, suggesting that the program's benefits transferred to children's daily lives.
Though the results of Lorenzo et al. (2016) are certainly promising, the study's interpretability has some notable limitations. The authors do not report if the children's teachers were blind to participants' assigned condition; thus, some of the positive findings possibly could have resulted from observer bias. Additionally, the authors reported very little information about the age of the participants beyond the fact that they were between 7 and 12 years of age. The intervention may have been most effective for a more specific age group; moreover, the mean age for participants was not reported, making it difficult to assess whether this program was effective for this age group as a whole. Most notably, however, the data reported was incomplete and the data analytic choices were less than ideal (i.e., more complex analyses would be more appropriate in identifying changes over the 40 sessions). We discuss this further in our Supplemental Analyses.
The other recent key study was conducted by Ip et al. (2018) who tested the effectiveness of VR-enabled training on the development of emotional- and social-adaptation skills in a 14-week, 28-session study. They randomly assigned 72 children (aged 7–10 years) with ASD to either a control group (did not receive any intervention) or an experimental group (received VR training). The training took a group-therapy approach with three to four children (of similar ages) participating together. Each child in the group took turns interacting with the virtual environment and then received feedback from the other children about their behaviors. The VR training incorporated scenarios that focused on emotional control, relaxation, and the simulation of different social situations where children had to identify emotions and use social-adaptation skills. The learning scenarios were presented to the children via a 4-sided, immersive VR environment (CAVE).
Ip et al. (2018) used a number of assessments before and after training to determine the program's effectiveness, including the Faces Test (Baron-Cohen et al., 1997) and the Eyes Test (Baron-Cohen et al., 2001) to test for emotion recognition, as well as the Psychoeducational Profile 3 (Schopler et al., 2005) to assess social skills, such as social reciprocity, emotional expression and regulation.
Ip et al. (2018) found that children in the VR training improved (relative to baseline) in the areas of social reciprocity and in emotion expression and regulation (as measured by the Social Reciprocity and Affective Expressions subtests, respectively, of the Psychoeducational Profile 3). Furthermore, as reported in our Supplemental Analyses, we found a significant difference between the groups on the post-training scores for emotion expression and regulation and social reciprocity. For both measures the experimental group outperformed the control group. No significant improvements were found for emotional recognition or other secondary measures.
Thus, the results of Ip et al. (2018) were mixed compared to Lorenzo et al. (2016). Although the reported improvements in social skills may be promising, the children in the experimental group spent considerable time interacting with one another, whereas those in the control group had no comparable experience. Therefore, additional research will be required to determine whether the VR aspect of the program played any role in these improvements. One possibility is that the VR experience helped children for reasons consistent with the anxiety-reducing hypothesis previously discussed. Alternatively, the VR experience may have helped simply because it maintained children's engagement with the training exercises. Based on the results of this one study, the VR component of the training may have no effect on social learning, and a similar program that engaged children in group discussion about emotions could have been more beneficial.
4.4. Conclusion: effectiveness of IVR used as a social-skills training tool for children with ASD
Unfortunately, only a few studies have investigated IVR as a training tool with children with ASD; thus, there is no clear answer to whether this technology is a useful training tool for social skills or if it is a helpful tool to reduce the anxiety levels of real social interactions for this type of population. The apparent effectiveness of IVR as a training tool for social skills in children with ASD is consistent with our guiding hypothesis that when applications require both immersion and simulation, implementation and effectiveness are more challenging to execute effectively.
4.5. Gaps and suggestions for future research
Despite more than 10 years of research in this area, new investigations will be necessary to gauge the potential effectiveness of VR training programs for children with ASD. Only the two published research studies described above have examined this question with sufficient sample sizes and control groups to provide data points that go beyond establishing proof of concept. Moreover, it is important to note that neither study compared VR-program effectiveness to established therapy programs. For these reasons, we suggest that a key future direction for research in this area would be registered trials of a VR-based program designed to improve social outcomes for children with autism that includes comparisons to existing non-VR interventions.
In addition to registered trials, other conceptual investigations can lead to important discoveries. Most importantly, the work conducted in this area to date has not investigated if and how anxiety experienced during VR simulations relates to VR-intervention effectiveness. Indeed, the original inspiration for VR-based interventions was based on the idea that social anxiety is a major obstacle for many people with ASD, which limits their ability to practice social understanding and social behavior skills in the real world (Moore et al., 2000). VR was envisioned as a way to present practice social situations while reducing the anxiety accompanying real-life interactions, with the hope that VR practice would manifest in better social interactions and less anxiety in everyday life as a result (Parsons & Mitchell, 2002). Neither of the key VR-intervention programs discussed above attempted to establish if program effectiveness resulted from a reduction in social anxiety relative to real-life interactions. Moreover, the converse notion should also be investigated: VR training programs for children with ASD might be effective because they foster at least a minimal sense of social anxiety, which is necessary for the children to transfer their learning to the real world where social anxiety is more commonly experienced. That is, perhaps too much social anxiety prevents social engagement, but too little would not provide enough emotional contextual learning to be accessed during an actual social simulation. Indeed, many questions about the underlying role of anxiety in VR-based interventions could be assessed.
In addition to these key questions for future research, opportunities exist to examine the importance of the specific VR technology used. Both Lorenzo et al. (2016) and Ip et al. (2018) used CAVE systems with their experimental groups. How well similar programs with IVR systems would work is currently unknown and is an important question to address. If IVR systems are found to be similarly effective, researchers can assess training programs with considerably less cost and more accessibility to families of children with autism.
Future research could also examine VR-based therapies for younger children. The two studies detailed in this section examined children over 6 years of age. As psychological interventions are generally found to be more effective at younger ages (Rogers & Vismara, 2008), examining these therapeutic approaches for younger children would be worthwhile, taking into account developing research findings on safe use of VR for young children. Additionally, future work could help clarify age effects with more specific analysis and reporting of age effects, as neither study examined the possible role of age on program effectiveness.
Table 3 IVR used as a social-skills training tool for children with autism spectrum disorder
Authors |
Population of children with ASD N (age range) |
Intervention with IVR |
Comparison non-IVR control group |
Evaluation report on the effectiveness of IVR therapy |
Type of VR technology |
|---|---|---|---|---|---|
72 (7–10 years old) |
IVR-enabled program designed to enhance emotional and social-adaptation skills |
No intervention |
The experimental training significantly improved children’s emotion expression and regulation and social reciprocity compared to control group. No differences on other measures. |
Semi-CAVE system (4-sided IVR environment) with special 3-D glasses |
|
40 (7–12 years old) |
IVR program designed to allow children to train in and develop different social situations in a structured, visual, and continuous manner |
Non-immersive VR-desktop software application |
Children in the experimental group demonstrated significantly greater improvement in social skills relative to children in the control group. |
Semi-CAVE system with special 3-D glasses |
Limitations
The following limitations should be taken into account when interpreting the results of the current review. First, our review only included studies published in peer-reviewed journals. This means that no conference proceedings, unpublished studies or studies in book chapters were included in the data analysis. This could possibly lead to an overestimation of the effectiveness of IVR with a developmental population due to the file drawer effect. That is, studies that fail to reach significance or fail to reject the null hypothesis are less likely to be published in journals (Dickersin et al., 1987). Second, the included studies applied different types of VR software, which could have influenced the amount of immersion, and VR effectivity and efficiency. Third, there was a difference in the effect and effectiveness of VR for different pediatric procedures / developmental uses or procedures, so one should be careful when generalizing the suggested effect for VR to clinical practice. However, the results of this review still provide useful information and guidance into the effectiveness of the use of IVR in developmental psychology. A final limitation is that we were unable to support or refute our hypothesis that immersion and simulation would be harder to implement effectively because none of the studies were specifically designed to test this claim.
General conclusions
In this review, we summarized the uses of IVR with a developmental population in three main areas: (1) pain distraction, (2) neuropsychology, and (3) social-skills training. The review aimed to identify the effectiveness of IVR when used as an immersion or simulation tool or when used as a combination of the two.
Firstly, the results showed that IVR is a useful tool when used to immerse participants in the VR experience to distract them from the real world. Its effectiveness does not rely on the ability to simulate a real environment, making it easier to use for any practice where children need to be distracted from a real situation (e.g., pain distraction). Secondly, regarding its use as a simulation tool, results showed that IVR is a valid and useful tool that has the capacity to simulate the challenges of real-life experiences. Therefore, it is a useful and practical technology to be used in any psychological application that requires the reconstruction of an environment (e.g., neuropsychology assessment). Lastly, when IVR is used to simulate and immerse the participant with the intention to use it as a training tool for children with ASD, the results are not as clear as in the previous two applications of the technology. Evidence of its effectiveness needs additional investigation to determine if IVR is effective, and whether this effectiveness is the result of presenting children with social situations that may induce less anxiety or because children can practice multiple times and receive immediate feedback.
Results of this review showed IVR can be a useful tool in psychology-related practices with children; however, further and more rigorous research is needed to examine the most promising uses of IVR with this population.
Acknowledgements
PAA express her gratitude to the Mexican National Council for Science and Technology (CONACYT) and to Swinburne University of Technology for research scholarship funding that supported this research.
Conflict of Interest Statement
The authors declare no conflicts of interest.
NOTES
References
- Araiza-Alba P., Keane T., Chen W.S., Kaufman J., 2020. Immersive virtual reality as a tool to learn problem-solving skills. PsyArXiv. doi:10.31234/osf.io/q857x
- Araiza-Alba P., Keane T., Kaufman J., 2020. Are we ready for Virtual Reality in K-12 classrooms? EdArXiv. doi:10.35542/osf.io/ux8hs
- Arane K., Behboudi A., Goldman R.D., 2017. Virtual reality for pain and anxiety management in children. Canadian Family Physician Medecin de Famille Canadien, 63(12), 932–934. http://www.ncbi.nlm.nih.gov/pubmed/29237632
- Areces D., Rodríguez C., García T., Cueli M., González-Castro P., 2016. Efficacy of a Continuous Performance Test Based on Virtual Reality in the Diagnosis of ADHD and Its Clinical Presentations. Journal of Attention Disorders, 22(11), 1081–1091. doi:10.1177/1087054716629711
- Aubrey J.S., Robb M., Bailey J., Bailenson J., 2018. Virtual Reality 101: What you need to know about VR and Kids. 1–30.
- Baron-Cohen S., Wheelwright S., Hill J., Raste Y., Plumb I., 2001. The “Reading the Mind in the Eyes”; Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 42(2), 241–251.
- Baron-Cohen S., Wheelwright S., Jolliffe T., 1997. Is there a “language of the eyes”? Evidence from normal adults, and adults with autism or Asperger Syndrome. Visual Cognition, 4(3), 311–331. doi:10.1080/713756761
- Beaubien J.M. and Baker D.P., 2004. The use of simulation for training teamwork skills in health care: How low can you go? In Quality and Safety in Health Care (Vol. 13, Issue SUPPL. 1, pp. 51–56). BMJ Publishing Group Ltd. doi:10.1136/qshc.2004.009845
- Biffi E., Beretta E., Cesareo A., Maghini C., Turconi A.C., Reni G., Strazzer S., 2017. An Immersive Virtual Reality Platform to Enhance Walking Ability of Children with Acquired Brain Injuries. Methods of Information in Medicine, 56(02), 119–126. doi:10.3414/ME16-02-0020
- Biocca F., 1997. The Cyborg’s Dilemma: Progressive Embodiment in Virtual Environments. Journal of Computer-Mediated Communication, 3(2), 0–0. doi:10.1111/j.1083-6101.1997.tb00070.x
- Biocca F. and Levy M.R., 1995. Communication in the Age of Virtual Reality. Lawrence Erlbaum Associates Inc.
- Bioulac S., 2015. Use of virtual classroom software as an evaluation tool and for treatment for children with Attention Deficit Hyperactivit. Enfance, 2015(1), 141–158. doi:10.4074/S0013754515001093
- Bioulac S., Lallemand S., Rizzo A., Philip P., Fabrigoule C., Bouvard M.P., 2012. Impact of time on task on ADHD patient’s performances in a virtual classroom. European Journal of Paediatric Neurology, 16(5), 514–521. doi:10.1016/j.ejpn.2012.01.006
- Blume F., Hudak J., Dresler T., Ehlis A.-C., Kühnhausen J., Renner T.J., Gawrilow C., 2017. NIRS-based neurofeedback training in a virtual reality classroom for children with attention-deficit/hyperactivity disorder: study protocol for a randomized controlled trial. Trials, 18(1), 41. doi:10.1186/s13063-016-1769-3
- Bortone I., Leonardis D., Mastronicola N., Crecchi A., Bonfiglio L., Procopio C., Solazzi M., Frisoli A., 2018. Wearable Haptics and Immersive Virtual Reality Rehabilitation Training in Children with Neuromotor Impairments. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26(7), 1469–1478. doi:10.1109/TNSRE.2018.2846814
- Bozgeyikli L., Bozgeyikli E., Raij A., Alqasemi R., Katkoori S., Dubey R., 2016. Vocational training with immersive virtual reality for individuals with autism: towards better design practices. 2016 IEEE 2nd Workshop on Everyday Virtual Reality (WEVR), 21–25. doi:10.1109/WEVR.2016.7859539
- Brahnam S., Jain L., Nicolson R.I., Fawcett, A.J., 2011. Advanced Computational Intelligence Paradigms in Healthcare 6: Virtual Reality in Psychotherapy, Rehabilitation, and Assessment. Advanced Computational Intelligence, 337(March 2011), 147–164. doi:10.1007/978-3-642-17824-5
- Brown N.J., Kimble R.M., Rodger S., Ware R.S., Cuttle L., 2014. Play and heal: Randomized controlled trial of DittoTM intervention efficacy on improving re-epithelialization in pediatric burns. Burns, 40(2), 204–213. doi:10.1016/j.burns.2013.11.024
- Chaytor N. and Schmitter-Edgecombe M., 2003. The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills. Neuropsychology Review, 13(4).
- Cho B.H., Ku J., Jang D.P., Kim S., Lee Y.H., Kim I.Y., Lee J.H., Kim S.I., 2002. The Effect of Virtual Reality Cognitive Training for Attention Enhancement. CyberPsychology & Behavior, 5(2), 129–137. doi:10.1089/109493102753770516
- Climent G. and Banterla F., 2010. AULA, evaluación ecológica de los procesos atencionales AULA, ecological evaluation of attentional processes. San Sebastián: Nesplora.
- Cromby J.J., Standen P.J.P., Newman J., Tasker H., 1996. Successful transfer to the real world of skills practised in a virtual environment by students with severe learning difficulties. Proc. 1st European Conference on …, 8–10. doi:10.1007/s11739-010-0427-0
- Cummings J.J. and Bailenson J.N., 2016. How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence. Media Psychology, 19(2), 274. doi:10.1080/15213269.2015.1015740
- Dahlquist L.M., McKenna K.D., Jones K.K., Dillinger L., Weiss K.E., Ackerman, C.S., 2007. Active and Passive Distraction Using a Head-Mounted Display Helmet: Effects on Cold Pressor Pain in Children. Health Psychology, 26(6), 794–801. doi:10.1037/0278-6133.26.6.794
- Dahlquist L.M., Weiss K.E., Clendaniel L.D., Law E.F., Ackerman C.S., McKenna K.D., 2009. Effects of Videogame Distraction using a Virtual Reality Type Head-Mounted Display Helmet on Cold Pressor Pain in Children. Journal of Pediatric Psychology, 34(5), 574–584. doi:10.1093/jpepsy/jsn023
- Dahlquist L.M., Weiss K.E., Law E.F., Sil S., Herbert L.J., Horn S.B., Wohlheiter K., Ackerman C.S., 2010. Effects of videogame distraction and a virtual reality type head-mounted display helmet on cold pressor pain in young elementary school-aged children. Journal of Pediatric Psychology, 35(6), 617–625. doi:10.1093/jpepsy/jsp082
- Díaz-Orueta U., Garcia-López C., Crespo-Eguílaz N., Sánchez-Carpintero R., Climent G., Narbona J., 2014. AULA virtual reality test as an attention measure: Convergent validity with Conners Continuous Performance Test. Child Neuropsychology, 20(3), 328–342. doi:10.1080/09297049.2013.792332
- Dickersin K., Chan S., Chalmersx T.C., Sacks H.S., Smith H., 1987. Publication bias and clinical trials. Controlled Clinical Trials, 8(4), 343–353. doi:10.1016/0197-2456(87)90155-3
- Didehbani N., Allen T., Kandalaft M., Krawczyk D., Chapman S., 2016. Virtual Reality Social Cognition Training for children with high functioning autism. Computers in Human Behavior, 62, 703–711. doi:10.1016/j.chb.2016.04.033
- Eccleston C. and Malleson P., 2003. Managing chronic pain in children and adolescents. We need to address the embarrassing lack of data for this common problem. BMJ (Clinical Research Ed.), 326(7404), 1408–149. doi:10.1136/bmj.326.7404.1408
- Eijlers R., Utens E.M.W.J., Staals L.M., De Nijs P.F.A., Berghmans J.M., Wijnen R.M.H., Hillegers M.H.J., Dierckx B., Legerstee J.S., 2019. Systematic Review and Meta-analysis of Virtual Reality in Pediatrics. Anesthesia & Analgesia, 129(5), 1344–1353. doi:10.1213/ANE.0000000000004165
- Faber A.W., Patterson D.R., Bremer M., 2013. Repeated use of immersive virtual reality therapy to control pain during wound dressing changes in pediatric and adult burn patients. Journal of Burn Care and Research, 34(5), 563–568. doi:10.1097/BCR.0b013e3182777904
- Fallis A., 2013. Application of Virtual Reality Technology in Simulation of Automated Workplaces. Journal of Chemical Information and Modeling, 53(9), 1689–1699. doi:10.1017/CBO9781107415324.004
- Galindo-Aldana G., Meza-Kubo V., Castillo-Medina G., Ledesma-Amaya I., Galarza-Del-Angel J., Padilla-López A., Morán, A.L., 2018. Computer-Based Neuropsychological Assessment: A Validation of Structured Examination of Executive Functions and Emotion (pp. 306–316). doi:10.1007/978-3-319-91122-9_26
- Gerçeker G., Binay Ş., Bilsin E., Kahraman A., Yılmaz H.B., 2018. Effects of Virtual Reality and External Cold and Vibration on Pain in 7- to 12-Year-Old Children During Phlebotomy: A Randomized Controlled Trial. Journal of Perianesthesia Nursing, 33(6), 981–989. doi:10.1016/j.jopan.2017.12.010
- Gershon J., Zimand E., Lemos R., Rothbaum B.O., Hodges L., 2003. Use of Virtual Reality as a Distractor for Painful Procedures in a Patient with Pediatric Cancer: A Case Study. CyberPsychology & Behavior, 6(6), 657–661. doi:10.1089/109493103322725450
- Gershon J., Zimand E., Pickering M., Rothbaum B.O., Hodges L., 2004. A Pilot and Feasibility Study of Virtual Reality as a Distraction for Children With Cancer. Journal of the American Academy of Child & Adolescent Psychiatry, 43(10), 1243–1249. doi:10.1097/01.chi.0000135621.23145.05
- Gilboa Y., Kerrouche B., Longaud-Vales A., Kieffer V., Tiberghien A., Aligon D., Mariller A., Mintegui A., Canizares C., Abada G., Paule Chevignard M., 2015. Describing the attention profile of children and adolescents with acquired brain injury using the Virtual Classroom. Brain Injury, 9052(October), 1–10. doi:10.3109/02699052.2015.1075148
- Gupta A., Scott K., Dukewich M., 2018. Innovative Technology Using Virtual Reality in the Treatment of Pain: Does It Reduce Pain via Distraction, or Is There More to It? Pain Medicine, 19(1), 151–159. doi:10.1093/pm/pnx109
- Halabi O., Abou El-Seoud S., Alja’am J., Alpona H., Al-Hemadi M., Al-Hassan D., 2017. Design of Immersive Virtual Reality System to Improve Communication Skills in Individuals with Autism. International Journal of Emerging Technologies in Learning (IJET), 12(05), 50. doi:10.3991/ijet.v12i05.6766
- Hua Y., Qiu R., Yao W.Y., Zhang Q., Chen, X.L., 2015. The Effect of Virtual Reality Distraction on Pain Relief During Dressing Changes in Children with Chronic Wounds on Lower Limbs. Pain Management Nursing, 16(5), 685–691. doi:10.1016/j.pmn.2015.03.001
- Huber B., Highfield K., Kaufman J., 2018) Detailing the digital experience: Parent reports of children's media use in the home learning environment. British Journal of Educational Technology, 49(5), 821-833. doi:10.1111/bjet.12667
- Ip H.H.S., Wong S.W.L., Chan D.F.Y., Byrne J., Li C., Yuan V.S.N., Lau K.S.Y., Wong, J.Y.W., 2018. Enhance emotional and social adaptation skills for children with autism spectrum disorder: A virtual reality enabled approach. Computers & Education, 117, 1–15. doi:10.1016/J.COMPEDU.2017.09.010
- Jeffs D., Dorman D., Brown S., Files A., Graves T., Kirk E., Meredith-Neve S., Sanders J., White B., Swearingen C.J., 2014. Effect of Virtual Reality on Adolescent Pain During Burn Wound Care. Journal of Burn Care and Research, 35(5), 395–408. doi:10.1097/BCR.0000000000000019
- Ke F. and Im T., 2013. Virtual-Reality-Based Social Interaction Training for Children with High-Functioning Autism. The Journal of Educational Research, 106(6), 441–461. doi:10.1080/00220671.2013.832999
- Kipping B., Rodger S., Miller K., Kimble R.M., 2012. Virtual reality for acute pain reduction in adolescents undergoing burn wound care: A prospective randomized controlled trial. In Burns (Vol. 38, Issue 5, pp. 650–657). doi:10.1016/j.burns.2011.11.010
- Koller D. and Goldman R.D., 2012. Distraction Techniques for Children Undergoing Procedures: A Critical Review of Pediatric Research. In Journal of Pediatric Nursing (Vol. 27, Issue 6, pp. 652–681). doi:10.1016/j.pedn.2011.08.001
- Kuriakose S. and Lahiri U., 2017. Design of a Physiology-Sensitive VR-Based Social Communication Platform for Children With Autism. IEEE Transactions on Neural Systems and Rehabilitation Engineering : A Publication of the IEEE Engineering in Medicine and Biology Society, 25(8), 1180–1191. doi:10.1109/TNSRE.2016.2613879
- Law E.F., Dahlquist L.M., Sil S., Weiss K.E., Herbert L.J., Wohlheiter K., Horn S.B., 2011. Videogame distraction using virtual reality technology for children experiencing cold pressor pain: The role of cognitive processing. Journal of Pediatric Psychology, 36(1), 84–94. doi:10.1093/jpepsy/jsq063
- Li A., Montaño Z., Chen V.J., Gold, J.I., 2011. Virtual reality and pain management: current trends and future directions. Pain Management, 1(2), 147–157. doi:10.2217/pmt.10.15
- Lorenzo G., Lledó A., Pomares J., Roig R., 2016. Design and application of an immersive virtual reality system to enhance emotional skills for children with autism spectrum disorders. Computers & Education, 98, 192–205. doi:10.1016/j.compedu.2016.03.018
- Luciana M., 2003. Practitioner Review: Computerized assessment of neuropsychological function in children: clinical and research applications of the Cambridge Neuropsychological Testing Automated Battery (CANTAB). Journal of Child Psychology and Psychiatry, 44(5), 649–663. doi:10.1111/1469-7610.00152
- Mallari B., Spaeth E.K., Goh H., Boyd, B.S., 2019. Virtual reality as an analgesic for acute and chronic pain in adults: A systematic review and meta-analysis. In Journal of Pain Research (Vol. 12, pp. 2053–2085). Dove Medical Press Ltd. doi:10.2147/JPR.S200498
- Mccaul K.D. and Malott J.M., 1984. Distraction and Coping With Pain. 95(3), 516–533.
- Moore D., McGrath P., Thorpe J., 2000. Computer-Aided Learning for People with Autism – a Framework for Research and Development. Innovations in Education and Training International, 37(3), 218–228. doi:10.1080/13558000050138452
- Neguț A., Jurma A.M., David D., 2016. Virtual-reality-based attention assessment of ADHD: ClinicaVR: Classroom-CPT versus a traditional continuous performance test. Child Neuropsychology, 23(6), 692–712. doi:10.1080/09297049.2016.1186617
- Neguţ A., Matu S.A., Sava F.A., David D., 2016. Task difficulty of virtual reality-based assessment tools compared to classical paper-and-pencil or computerized measures: A meta-analytic approach. Computers in Human Behavior, 54, 414–424. doi:10.1016/j.chb.2015.08.029
- Nilsson S., Finnström B., Kokinsky E., Enskär K., 2009. The use of Virtual Reality for needle-related procedural pain and distress in children and adolescents in a paediatric oncology unit. European Journal of Oncology Nursing, 13(2), 102–109. doi:10.1016/j.ejon.2009.01.003
- Nolin P., Stipanicic A., Henry M., Lachapelle Y., Lussier-Desrochers D., Rizzo A., Allain P., 2016. ClinicaVR: Classroom-CPT: A virtual reality tool for assessing attention and inhibition in children and adolescents. Computers in Human Behavior, 59, 327–333. doi:10.1016/j.chb.2016.02.023
- Oberhauser M., Dreyer D., Braunstingl R., Koglbauer I., 2018. What’s Real About Virtual Reality Flight Simulation?: Comparing the Fidelity of a Virtual Reality With a Conventional Flight Simulation Environment. Aviation Psychology and Applied Human Factors, 8(1), 22–34. doi:10.1027/2192-0923/a000134
- Pan X. and Hamilton A.F.d.C., 2018. Why and how to use virtual reality to study human social interaction: The challenges of exploring a new research landscape. British Journal of Psychology, 109(3), 395–417. doi:10.1111/bjop.12290
- Parsons S., 2016. Authenticity in Virtual Reality for assessment and intervention in autism: A conceptual review. Educational Research Review, 19, 138–157. doi:10.1016/j.edurev.2016.08.001
- Parsons S. and Cobb S., 2011. State-of-the-art of virtual reality technologies for children on the autism spectrum. European Journal of Special Needs Education, 26(3), 355–366. doi:10.1080/08856257.2011.593831
- Parsons S. and Mitchell P., 2002. The potential of virtual reality in social skills training for people with autistic spectrum disorders. Journal of Intellectual Disability Research : JIDR, 46(5), 430–443. doi:10.1046/j.1365-2788.2002.00425.x
- Parsons T.D., Bowerly T., Buckwalter J.G., Rizzo A.A., 2007. A controlled clinical comparison of attention performance in children with ADHD in a virtual reality classroom compared to standard neuropsychological methods. Child Neuropsychology, 13(4), 363–381. doi:10.1080/13825580600943473
- Pollak Y., Weiss P.L., Rizzo A.A., Weizer M., Shriki L., Shalev R.S., Gross-Tsur V., 2009. The Utility of a Continuous Performance Test Embedded in Virtual Reality in Measuring ADHD-Related Deficits. Journal of Developmental and Behavioral Pediatrics, 30(1), 2–6. doi:10.1097/DBP.0b013e3181969b22
- Ramachandiran C.R., Jomhari N., Thiyagaraja S., Maria M., 2015. Virtual reality based behavioural learning for autistic children. Electronic Journal of E-Learning, 13(5), 357–365.
- Rizzo A.A., Buckwalter J.G., Bowerly T., Zaag C., Humphrey L., Neumann U., Chua C., Kyriakakis C., Rooyen A., Sisemore D., 2000. The Virtual Classroom: A Virtual Reality Environment for the Assessment and Rehabilitation of Attention Deficits. Cyberpsychology & Behavior, 3(3).
- Rizzo A.A., Schultheis M., Kerns K.A., Mateer C., 2004. Analysis of assets for virtual reality applications in neuropsychology. Neuropsychological Rehabilitation, 14(1-2 SPEC. ISS.), 207–239. doi:10.1080/09602010343000183
- Rodríguez C., Areces D., García T., Cueli M., González-Castro P., 2018. Comparison between two continuous performance tests for identifying ADHD: Traditional vs. virtual reality. International Journal of Clinical and Health Psychology, 18(3), 254–263. doi:10.1016/j.ijchp.2018.06.003
- Rogers S.J. and Vismara L.A., 2008. Evidence-Based Comprehensive Treatments for Early Autism. Journal of Clinical Child & Adolescent Psychology, 37(1), 8–38. doi:10.1080/15374410701817808
- Rose F.D., Attree E.A., Brooks B.M., Andrews T.K., 2001. Learning and Memory in Virtual Environments: A Role in Neurorehabilitation? Questions (and Occasional Answers) from the University of East London. Presence: Teleoperators and Virtual Environments, 10(4), 345–358. doi:10.1162/1054746011470208
- Rothbaum B.O., Rizzo A.S., McDaniel D.D., Zanov M.V., 2015. Virtual Reality Exposure Therapy. In Encyclopedia of Mental Health: Second Edition (pp. 370–374). doi:10.1016/B978-0-12-397045-9.00196-8
- Schardt C., Adams M.B., Owens T., Keitz S., Fontelo P., 2007. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Medical Informatics and Decision Making, 7, 16. doi:10.1186/1472-6947-7-16
- Schopler E., Lansing M.D., Reichler R.J., Marcus L.M., 2005. Psychoeducational Profile-Third Edition (PEP-3). doi:10.1007/s10803-013-1794-y
- Shiri S., Feintuch U., Weiss N., Pustilnik A., Geffen T., Kay B., Meiner Z., Berger I., 2013. A Virtual Reality System Combined with Biofeedback for Treating Pediatric Chronic Headache—A Pilot Study. Pain Medicine, 14(5), 621–627. doi:10.1111/pme.12083
- Sil S., Dahlquist L.M., Thompson C., Hahn A., Herbert L., Wohlheiter K., Horn S., 2012. The effects of coping style on virtual reality enhanced videogame distraction in children undergoing cold pressor pain. Journal of Behavioral Medicine, 37(1), 156–165. doi:10.1007/s10865-012-9479-0
- Slater M. and Wilbur S., 1997. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence: Teleoperators and Virtual Environments, 6(6), 603–616. doi:10.1162/pres.1997.6.6.603
- Strickland D., 1997. Virtual reality for the treatment of autism. Studies in Health Technology and Informatics, 44, 81–86. doi:10.3233/978-1-60750-888-5-81
- Strickland D., Marcus L., Mesibov, G.B., Hogan K., 1996. Brief report: Two case studies using virtual reality as a learning tool for autistic children. In Journal of Autism and Developmental Disorders (Vol. 26, Issue 6, pp. 651–659).
- Turner C.J., Hutabarat W., Oyekan J., Tiwari A., 2016. Discrete Event Simulation and Virtual Reality Use in Industry: New Opportunities and Future Trends. IEEE Transactions on Human-Machine Systems, 46(6), 882–894. doi:10.1109/THMS.2016.2596099
- Walsh K. and Darby D., 1999. Neuropsychology: A Clinical Approach (4th editio). Churchill Livingstone.
- Wiederhold B.K., Gao K., Sulea C., Wiederhold M.D., 2014. Virtual reality as a distraction technique in chronic pain patients. Cyberpsychology, Behavior and Social Networking, 17(6), 346–352. doi:10.1089/cyber.2014.0207
- Windich-Biermeier A., Sjoberg I., Dale J.C., Eshelman D., Guzzetta C.E., 2007. Effects of Di straction on Pain, Fear, and Distress During Venous Port Access and Venipuncture in Children and Adolescents With Cancer. Journal of Pediatric Oncology Nursing, 24(1), 8–19. doi:10.1177/1043454206296018
- Wint S.S., Eshelman D., Steele J., Guzzetta C.E., 2002. Effects of Distraction Using Virtual Reality Glasses During Lumbar Punctures in Adolescents With Cancer. Oncology Nursing Forum, 29(1), E8–E15. doi:10.1188/02.ONF.E8-E15
- Zhao H., Swanson A.R., Weitlauf A.S., Warren, Z.E., Sarkar N., 2018. Hand-in-Hand: A Communication-Enhancement Collaborative Virtual Reality System for Promoting Social Interaction in Children with Autism Spectrum Disorders. IEEE Transactions on Human-Machine Systems, 48(2), 136–148. doi:10.1109/THMS.2018.2791562
Attachments
Data Analysis
In this document, we report the key findings from the papers we discussed in our manuscript. Many of the empirical papers did not report complete statistical analyses or effect sizes; therefore, when possible, we calculated these using the information provided in the original papers. The data and code for these analyses are available on the Open Science Framework at: https://osf.io/f96hc/
DownloadArticle statistics
Views: 2608
Downloads
PDF: 530
XML: 4