Introduction

When human beings evolve in a 3D world, most of their experiences are spatialized. Being able to encode, store and retrieve the spatial properties of what we experienced is important for adaptability to our environment and is a key feature of episodic memory. In addition, the integration of the spatial components of the experience in long-term memory is enhanced by a period of rest and/or sleep (Orban et al., 2006; Rauchs et al., 2008). However, in a research context, spatialized memory tasks that are presented on 2D screens can reasonably be considered far removed from typical real-life situations, especially in terms of spatiality. Virtual reality (VR) allows researchers to present three-dimensional objects in specific positions in a manner similar to reality while providing the same experimental control as one offered by other digital technologies (Sauzéon et al., 2012). The presentation of a memory task with a spatial component has been facilitated by the fact that virtual reality headsets have become more affordable and reliable (Smith, 2019).

The elements we encounter in our daily lives are not memorized in isolation. Indeed, the way in which these elements are associated and related to each other is decisive for memory performance. Moreover, the sleep (a full night and a day nap) is posited to benefit on many aspects of cognition and specifically to memory (Rasch & Born, 2013; Ficca et al., 2010). For instance, Lahl et al. (2008) demonstrated that sleep benefits in general memory for naps as short as 6 minutes. More specifically, sleep improves performance for both the associative and relational properties of memory (Rasch & Born, 2013). For example, when the spatial properties of an episode are reactivated in the hippocampus, these reactivations are supposed to contribute to the consolidation of the whole memory (Talamini et al., 2008). This associative and relational improvement has been studied over periods ranging from twenty minutes of offline delay (delay without rehearsal) to several days with a typical sleep-wake cycle (Ellenbogen et al., 2007). To our knowledge, memory improvement for specifically associative and relational performance has not yet been demonstrated for a period of 20-minutes or less using VR design. Our study aims at evaluating the effect of a 20-minute rest period with a nap, on the memory performance for associated and related items (also referred as object in this study), using an associative inference task in VR.

Aims and hypothese

The present study aims at evaluating the effect of a short rest period on associative and relational performance on a memory task proposed in virtual reality. We expect that participants in the rest group will recall more correct associations and relations than participants in the wake group.

Methods

Material and Measures

The headset used in this study was an HTC Vive displaying 1080x1200 pixel for each eye. This headset yields a 100° horizontal field of view with a refresh rate of 90 Hz. It was connected to an HP ZBook 17 G3 with an Intel Core i7 (Quad-Core 2.6 GHz / 3.5 GHz Turbo), 8 GB RAM and a NVIDIA Quadro M5000M graphic card. The environment was developed using Unreal Engine 4 (UE4).

Associative Spatialised Memory Room

During a pre-test, some participants looked surprised when they took off the headset and indicated that this surprise was probably due to the unexpected difference in size between the real test room and the virtual environment. As we did not know whether this surprise would affect the study or not, we decided to reduce this difference by matching the size of the rooms and placing a door in the same place.

In the environment, 12 display stands were disposed in front of the participant, 4 at the left side, 4 at the back side and 4 at the right side. Each display stand was covered to only show the item when the participants pressed a button placed on the stand. For each revelation, a luminous mark on the button indicated which display stand was to be opened. By pressing the button with a standard controller, the participant would hear a clicking sound. The room was illuminated using an omnidirectional light source placed in the middle of display stands. A black cross was drawn on the floor to indicate to the participant where he or she needed to go after each item reveal. During the associative recall, a supplementary display stand was shown in the center of the room to display the item for which the participant had to indicate the location. A laser pointer was available and movable by the participant to indicate which stand he or she had previously seen the item displayed.

Figure 1: Screen capture of the Associative Spatialised Memory Room.

Save View full size Expand inline Collapse inline

Items Selection

The items, also referred as object in this study, were selected from 3D models proposed online on the Unreal engine market and other websites with a CCO or CC BY license. Two pre-test sessions were conducted with undergraduate students, a PhD student specialising in VR, a professor of neuropsychology, and a research engineer in order to ensure good readability of the items and environment, to maximise the items uniqueness, and to avoid phobogenic items. Out of 41 3D models, 25 were selected and another one was randomly excluded to fit the final 24-items list composed of: Green plant, Computer, Spray can, Sledge, Car, Cactus, Gift box, Rock, Camera, Stool, Ball, Mushroom, Hat, Toilet, Pumpkin, Fire extinguisher, Violin, Dice, Bread, Hammer, Crow, Fish, Saw, Armchair.

The graphic appearance of the objects was preferably chosen for the realistic and compatibility with an accurate representation of the object (as opposed to stylized or cartoonish). However, as the models came from different authors, the graphics identity were certainly not identical between the objects. A figure presenting the objects is available on request to the corresponding author of the present article (to ensure the novelty of the objects in the context of future studies).

Scores composition

Each correct recall granted the participant 1 point. Scores for immediate free recall, delayed free recall, immediate associative recall and delayed associative recall were integers (min = 0/24, max = 24/24). Score for relational recall was integer (min = 0/12, max = 12/12).

Subjective Evaluation of Sleep and Actimetry

All participants completed an additional two-part questionnaire (20 minutes after the task and 40 minutes after the task). Participants in the rest group were asked whether or not they had slept, how long they estimated their nap and whether or not they had dreamt during this short sleep. All participants in the rest group were tracked by actimetry using an Actiwatch 2 (Rupp & Balkin, 2011). Actimetry limited the possibility of a Type 1 error (indicating that one is asleep when one is not). Data from 8 of the 55 participants in the rest group were not recorded by the watch because of a failure in memory management by the researcher. However, all participants who reported being asleep were monitored.

Procedure[1]

Participants were placed in the virtual environment. For the learning task, each participant was instructed for each association, to stand on the black cross and identify the display stand with the luminous marker. They were then requested to go to the display and open it by placing his or her hand on the button. The participant proceeded in the same way for the next display. Participants performed the recall task while starring at the back wall. The instruction for free recall test was to indicate aloud which objects were seen in the learning phase. The instruction for associative recall was to indicate the display stand containing the object using the laser. Finally, the instruction for relational recall test was to indicate aloud which object had the same position as the object displayed in front of the participant. During this relational recall, participants were placed in a circular room to ensure that spatial cues from the environment were not used to infer the relationships. The figure 2 depicts what was asked to the participant during each learning phase. The figure 3 and 4 illustrate the associative recall phase and the relational recall phase.

Figure 2: the steps of the learning phases (identify the display stand, go to the display and open it by placing his or her hand on the button, observe the object, and go back to the black cross to state the object ‘s name out-lout). These steps were repeated 12 times for each learning (A-B and B-C). The order was randomised by the software. The individual in these pictures only serves as a representation of the participant and was not present in the environment during the learning process.

Save View full size Expand inline Collapse inline

Figure 3: during the associative recall the participant saw an object (depicted here using a square) on a display stand in the middle of the room and he/she could grab a laser pointer to indicate where he/she thought to have seen the object.

Save View full size Expand inline Collapse inline

Figure 4: during the relational recall the participant saw an object (depicted hereas a square) on a table inside a neutral environment. He or she had to indicate out-loud which object was in the same position during the learning phase.

Save View full size Expand inline Collapse inline

During the delay at the end of the learning, part of the sample was placed in the rest group in which they were offered a 20-minute nap window in a quiet room and the other part of the sample was placed in the wake group in which they were asked to continue their daily routine for 20 minutes. Afterwards, all participants were given a 20-minute delay prior to the delayed recall. The procedure is summarised in figure 5.

Figure 5: Course of the protocol for all participants

Save View full size Expand inline Collapse inline

Results

All statistical significance thresholds were set at p < .05. In this section, we first studied the group effect by comparing those who were offered a nap (rest group) to those who were asked to continue their daily routine (wake group). We then studied a group effect by comparing only participant in the rest group based on whether or not they reported having a nap (verified by actimetry).

Main results

To compare the two groups (rest and wake), we performed two repeated-measures ANOVAs. Firstly, a repeated-measures ANOVA was performed on free recall score. The condition (rest or wake) was used as a factor between subjects and type of delay (immediate/delayed) was used as a factor within subjects. There was a significant effect of delay type (F (2, 80) = 98.99, p < .001), and no significant interaction effect (F (2, 80) = 6.26, p = .015).

Secondly, another repeated-measures ANOVA was performed on the associative recall score. The condition (resting or wake) was used as a factor between subjects and type of delay (immediate/delayed) was used as a factor within subject. There was a significant effect of recognition type (F (2, 80) = 17.31, p < .001), and a significant interaction effect (F (2, 80) = 9.78, p = .002).

Additionally, following the method employed by Alger and Payne (2016), we computed delta scores (dScore) by subtracting performance of immediate recall to delayed recall for the free and associative recalls. The t-test comparisons yield similar outcomes of the ANOVAs: dScore for free recall in the rest group (M = 1.35, SD = 2.59) and in the wake group (M = 2.50, SD = 3.20) were not significantly different (p = .078, Cohen’s d = -.41); dScore for associative recall in the rest group (M = 2.41, SD = 1.88) and in the wake group (M = 1.70, SD= 2.32) were significantly different (p = .002, Cohen’s d = .71). The aforementioned results are summarised in Figure 6.

A student t-test comparing the two groups for relational recall score was not significant.

Figure 6: Performance for the rest and the wake group for the free recall, the associative recall and the relational recall.

Save View full size Expand inline Collapse inline

In the rest group, 27 participants out of 54 reported sleep. All 27 shown actimetry consistent with sleep. Using a Student's t-test, we compared participants who slept and those who did not for all performance indicators (immediate and delayed free recall, immediate and delayed associative recall, and relational recall); no comparisons were significant. Using a Spearman correlation analysis, we assessed if the time spent sleeping was correlated with performance; no correlation was significant.  We performed again the two ANOVA (Wake, Rest) while keeping only in the rest group, participant who had not slept (n = 27) for a comparison and those who had slept (n = 27) in another comparison. The significance outcomes were identical as those displayed in figure 1 suggesting that sleep did not influence significantly our result.

Discussion

In the present study, we elaborated an associative inference task (Alger & Payne, 2016; Huguet et al., 2019) that allows the evaluation of associative and relational properties of memory. One of the originalities of this study was to use the possibilities that offer virtual reality (VR) to form association between an object and a location in a space room. Another originality was to investigate the effect of a short period of rest and the possible nap that took place within this period of associative and relational performances. The main results of the present study concerned the enhancement of associative recall after a nap opportunity which is consistent with precedent research (Alger & Payne, 2016).

Most of our results are congruent with the state of art in the literature as, on the one hand, associative performances were better for the participants who had a period of rest than for the other participants. In another hand, relational performances were not different between groups as previously observed by Ellenbogen et al. (2007) who proposed that relational enhancement may depend on an offline process that needs more than 20 minutes to be perceptible. Another explanation could be found in the intentionality of learning. In this study, we did not inform the participant that he or she will be asked to recall related items. In other words, encoding was intentional for association and incidental in relation. Interestingly, a short sleep period benefits preferentially on intentionally encoded memory trace over incidentally ones (Drosopoulos et al., 2005). This limitation is inherent to the associative inference task, and has not, to our knowledge, yet been addressed in previous investigations. We therefore encourage future studies to modify the associative inference task to take this limitation into account (e.g. by making the association incidental). It would also be interesting to compare whether sleep modulation of relational performance differs according to the intentionality of learning.

One interesting and surprising result of this study is that the benefits of the rest period of associative performance did not depend on whether or not a nap took place during this period. The result did not also significantly differ for relational performance. Moreover, memory performance was not significantly influenced by the subjective time spent napping. One explanation could be that sleep had no additional effect on memory and did not alter the process of consolidation. Another explanation could be that the beneficial effect of sleep was masked by the short time window proposed. One theory of sleep-related memory improvement is that there is less interference arising from the world during sleep to disrupt memory (see Talamini et al., 2008). Protection from interference cannot satisfactorily account for the present study result as free and relational recall were not significantly improved. Therefore, it is reasonable to consider that rest has a beneficial effect on the associative performance on its own. This proposition is consistent with the research of Craig et al. (2015) who have shown that individuals who had a rest immediately after learning spatial association in a VR environment performed better in recalling certain specificities of these associations such as temporality. This benefit of awake rest was also demonstrated on associative performance itself (Craig et al., 2016). The environment used in the two mentioned studies was larger than the one used in the present research since it offered a complete segment of a realistic small town. The way in which a resting period allows the association of elements with a spatial component therefore appears to be robust. This result is also consistent with the proposition that wakeful rest can be considered as a specific state of consciousness that actively benefits the consolidation process. To our knowledge and as specified by the authors of the two studies mentioned above, the benefit of a rest period on associative performance is mainly studied with an interest in sleep. In contrary, the present study invites to consider the benefit of a rest period that possibly not contain a nap on the associative properties of memory.

Conclusion

We demonstrated that memory for associated items is enhanced by a rest period comparatively to a period of active wake. In perspective, we propose that future studies test this improvement with an even shorter duration and assess whether such a rest period could be useful in an academic learning context.

The virtual environment proposed in this study has succeeded in highlighting the consolidation process that took place during a period of rest on associative properties of memory. Virtual reality offers an unparalleled opportunity to study memory consolidation for associations containing a spatial component because it places the individual in a simulation of a credible environment. As such, it allows both an experimental rigour in the presentation of objects comparable to other computerised tasks (e.g. randomisation, brightness control) while proposing a three-dimensional environment analogous, in terms of spatiality, to ones in which the individual evolves in his or her daily life. Virtual reality is already used in research by various disciplines, such as psychology (e.g., Sauzéon et al., 2012), neuroscience (e.g., Bohil et al., 2011), human-computer interaction (Coeugnet et al., 2018) or biology (e.g. Theart et al., 2017). This availability may indicate that VR has reached sufficient maturity to be integrated into a broader spectrum of scientific research projects, even if they are of moderate scale.

The present study was conducted in the context of a larger project on the use of the virtual environment and user experience. Designing and administrating our environment and protocol were highly informative and led to several recommendations that we believe could be useful to other researchers. A first recommendation that may be obvious has emerged empirically and is intended for cognitive researchers; we believe that expertise in virtual environments and user experience should be sought before starting the development of studies that rely on VR. To contextualise this view, we need to indicate that this research is the result of spontaneous discussions that were encouraged by shared open space that enables interactions among researchers from different fields or theoretical backgrounds. Without these discussions, the protocol would not have taken into account some of the specificity of VR use. More specifically, these interdisciplinary exchanges led to the establishment of a series of constraints that guided the development of the protocol. For example, these constraints concerned: maximum duration of the environment, the addition of interaction with a plausible effect on the environment, the quality of the headset to be used in terms of framerate, the avoidance of a flicker of light, the addition of sensory multimodality, freedom of movement and the size and properties of the room, etc. These constraints aimed to improve both interactive immersion and the coherence of sensory input. A massive amount of research on VR technologies has been conducted in the past 30 years (Slater, 2018). Researchers interested in using VR for research can rely on studies on presence (e.g. Slater, 1999), cyber sickness (e.g. Serge and Moss, 2015) and usability of virtual environments (e.g. Bach & Scapin, 2003). However, clear documentation to guide psychologists and research engineers in the development of their protocol with more care is, to our knowledge, still necessary. A review specifically addressing the use of VR when investigating episodic memory have been published since we elaborated this study, it could possibly complete and extend some of the recommendations we emitted presently (Smith, 2019).

Acknowledgments

We would like to thank the Picardy regional authority and the European Regional Development Fund (FEDER) for funding support through the LEON project. We also would like to thank Damien Fernandez who participated in the pre-testing of the material.

References

• Alger S.E., Payne J. D., 2016. The differential effects of emotional salience on direct associative and relational memory during a nap. Cognitive, Affective, & Behavioral Neuroscience, 16(6), 1150–1163. doi:10.3758/s13415-016-0460-1
• Bach C., Scapin D.L., 2003. Adaptation of ergonomic criteria to human-virtual environments interactions. In Proceedings of Interact’03 (pp. 880–883). Amsterdam: IOS Press.
• Baldwin, J. M. (1893). Elements of Psychology (p.162).
• Bohil C.J., Alicea B., Biocca, F.A., 2011. Virtual reality in neuroscience research and therapy. Nature Reviews Neuroscience, 12(12), 752–762. doi:10.1038/nrn3122
• Chatburn A., Lushington K., Kohler M.J., 2014. Complex associative memory processing and sleep: A systematic review and meta-analysis of behavioural evidence and underlying EEG mechanisms. Neuroscience and Biobehavioral Reviews, 47, 646–655. doi:10.1016/j.neubiorev.2014.10.018
• Cherdieu M., Versace R., Rey A.E., Vallet G.T., Mazza, S., 2018. Sleep on your memory traces: How sleep effects can be explained by Act–In, a functional memory model. Sleep Medicine Reviews, 39, 155–163. doi:10.1016/j.smrv.2017.09.001
• Coeugnet S., Dommes A., Panëels S., Chevalier A., Vienne F., Dang N.T. Anastassova M., 2018. Helping older pedestrians navigate unknown environments through vibrotactile guidance instructions. Transportation Research Part F: Traffic Psychology and Behaviour, 58, 816–830. doi: 10.1016/j.trf.2018.07.017
• Craig M., Dewar M., Della Sala S., Wolbers T., 2015. Rest boosts the long-term retention of spatial associative and temporal order information. Hippocampus, 25(9), 1017–1027. doi:10.1002/hipo.22424
• Craig M., Dewar M., Harris M.A., Della Sala S., Wolbers T., 2016. Wakeful rest promotes the integration of spatial memories into accurate cognitive maps. Hippocampus, 26 (2), 185–193. doi:10.1002/hipo.22502
• Drosopoulos S., Wagner U., Born J., 2005. Sleep enhances explicit recollection in recognition memory. Learning & Memory, 12(1), 44–51. doi:10.1101/lm.83805
• Eichenbaum H., 2004. Hippocampus: Cognitive Processesand Neural Representations thatUnderlie Declarative Memory. Neuron, 44(1), 109–120. doi:10.1016/j.neuron.2004.08.028
• Ellenbogen J.M., Hu P.T., Payne J.D., Titone D., Walker M.P., 2007. Human relational memory requires time and sleep. Proceedings of the National Academy of Sciences, 104 (18), 7723–7728. doi:10.1073/pnas.0700094104
• Ficca G., Axelsson J., Mollicone D.J., Muto V., Vitiello M.V., 2010. Naps, cognition and performance. Sleep Medicine Reviews, 14(4), 249–258. doi:10.1016/j.smrv.2009.09.005
• Hebb D.O., 1949. The Organization of Behavior. The American Journal of Psychology, 63 (4), 633. doi:10.2307/1418888
• Huguet M., Payne J.D., Kim S.Y., Alger S.E., 2019. Overnight sleep benefits both neutral and negative direct associative and relational memory. Cognitive, Affective, & Behavioral Neuroscience, 33, 57-64 doi:10.3758/s13415-019-00746-8
• Inostroza M., Born J., 2013. Sleep for Preserving and Transforming Episodic Memory. Annual Review of Neuroscience, 36 (1), 79–102. doi:10.1146/annurev-neuro-062012-170429
• Lahl O., Wispel C., Willigens B., Pietrowsky R., 2008. An ultra short episode of sleep is sufficient to promote declarative memory performance. Journal of Sleep Research, 17 (1), 3–10. doi:10.1111/j.1365-2869.2008.00622.x
• Naus M.J., Philipp L.M., Samsi M., 2009. From paper to pixels: A comparison of paper and computer formats in psychological assessment. Computers in Human Behavior, 25 (1), 1–7. doi:10.1016/j.chb.2008.05.012
• Orban P., Rauchs G., Balteau E., Degueldre C., Luxen A., Maquet P., Peigneux P., 2006. Sleep after spatial learning promotes covert reorganization of brain activity. Proceedings of the National Academy of Sciences of the United States of America, 103 (18), 7124–7129. doi:10.1073/pnas.0510198103
• Rasch B., Born J., 2013. About sleep’s role in memory. Physiological Reviews 93 (2), 681-766 doi:10.1152/physrev.00032.2012
• Rauchs G., Orban P., Schmidt C., Albouy G., Balteau E., Degueldre C., Schnackers C, Sterpenich V, Tinguely G, Luxen A, Maquet P, Peigneux P., 2008. Sleep modulates the neural substrates of both spatial and contextual memory consolidation. PloS One, 3 (8), e2949. doi:10.1371/journal.pone.0002949
• Robin J., Olsen, R.K., 2019. Scenes facilitate associative memory and integration. Learning and Memory, 26 (7), 252–261. doi:10.1101/lm.049486.119
• Rupp T.L., Balkin, T.J., 2011. Comparison of Motionlogger Watch and Actiwatch actigraphs to polysomnography for sleep/wake estimation in healthy young adults. Behavior Research Methods, 43 (4), 1152–1160. doi:10.3758/s13428-011-0098-4
• Sauzéon H., Arvind Pala P., Larrue F., Wallet G., Déjos M., Zheng X., Guitton P., N’Kaoua B., 2012. The Use of Virtual Reality for Episodic Memory Assessment. Experimental Psychology, 59 (2), 99–108. doi:10.1027/1618-3169/a000131
• Schatz P., Browndyke J., 2002. Applications of computer-based neuropsychological assessment. Journal of Head Trauma Rehabilitation, 17(5), 395–410. doi:10.1097/00001199-200210000-00003
• Serge S.R., Moss, J.D., 2015. Simulator Sickness and the Oculus Rift: A First Look. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 59 (1), 761–765. doi: 10.1177/1541931215591236
• Slater M., 1999. Measuring Presence: A Response to the Witmer and Singer Presence Questionnaire. Presence: Teleoperators and Virtual Environments, 8 (5), 560–565. doi: 10.1162/105474699566477
• Slater M., 2018. Immersion and the illusion of presence in virtual reality. British Journal of Psychology, 109, 431–433. doi: 10.1111/bjop.12305
• Smith S.A., 2019. Virtual reality in episodic memory research: A review. Psychonomic Bulletin and Review, 26(4), 1213–1237. doi:10.3758/s13423-019-01605-w
• Talamini L.M., Nieuwenhuis I.L.C., Takashima, A. Jensen O., 2008. Sleep directly following learning benefits consolidation of spatial associative memory. Learning and Memory, 15 (4), 233–237. doi:10.1101/lm.771608
• Tamminen J., Lambon Ralph M.A., Lewis P.A., 2013. The role of sleep spindles and slow-wave activity in integrating new information in semantic memory. Journal of Neuroscience, 33 (39), 15376–15381. doi:10.1523/JNEUROSCI.5093-12.2013
• Theart R.P., Loos B., Niesler T.R., 2017. Virtual reality assisted microscopy data visualization and colocalization analysis. BMC Bioinformatics, 18 (64), 1–16. doi: 10.1186/s12859-016-1446-2
• Van der Helm E., Gujar N., Nishida M., Walker M.P., 2011. Sleep-Dependent Facilitation of Episodic Memory Details. PLoS ONE, 6 (11). doi:10.1371/journal.pone.0027421
• Versace R., Vallet G.T., Riou B., Lesourd M., Labeye É., Brunel L., 2014. Act-In: An integrated view of memory mechanisms. Journal of Cognitive Psychology. doi:10.1080/20445911.2014.892113
• Walker M.P., Brakefield T., Seidman J., Morgan A., Hobson J.A., Stickgold R., 2003. Sleep and the time course of motor skill learning. Learning and Memory, 10(4), 275–284. doi:10.1101/lm.58503
• Wamsley E.J., Stickgold R., 2018. Dreaming of a learning task is associated with enhanced memory consolidation: Replication in an overnight sleep study. Journal of Sleep Research, 28 (1) e12749. doi:10.1111/jsr.12749
• Wamsley E.J., Tucker M., Payne J.D., Benavides J.A., Stickgold R., 2010. Dreaming of a Learning Task Is Associated with Enhanced Sleep-Dependent Memory Consolidation. Current Biology, 20 (9), 850–855. doi:10.1016/j.cub.2010.03.027
• Wiedemann C., 2007. Sleep and memory: Memory consolidation…while you are sleeping. Nature Reviews Neuroscience, 8 (2), 86–87. doi:10.1038/nrn2084