Identifying Cybersickness through Heart Rate Variability alterations

- Numerous users of Head Mounted Dis-plays develop symptoms similar to motion sickness. This phenomenon is known as cybersickness. The current standard method to evaluate it is the Simulator Sickness Questionnaire (SSQ). Additionally, detection by methods such as biosignals is possible, but it requires sophisticated equipment and expertise. In order to detect early cybersickness symptoms, the availability of real-time detection by using simple equipment, such as a 2-lead ECG, would be highly useful. This contribution aims to analyze the feasibility of detecting cybersickness with a simple 2 lead ECG. A total of 13 participants played the VR game QuakeVR wearing an Oculus Rift DK2 for 15 minutes, during which a 2-lead ECG was recorded. Additionally, pre-and post-SSQ questionnaires were given. Four of 13 participants had to end the experiment pre-maturely due to cybersickness. The difference in SSQ scores is statistically signiﬁcant (p < .01), as is the ECG (p=.02), in these participants. This study shows the utility of a simple 2-lead ECG to detect cybersickness. These ﬁndings raise the possibility of real-time monitoring and prediction of cybersickness with simple devices and open the question of whether photoplethysmography could be used with the same purpose.


I. INTRODUCTION
Cybersickness is a term used to refer to the cluster of symptoms that users experience during, or after, Virtual Reality (VR) exposure (McCauley and Sharkey 1992). It is acknowledged as a physiological response to an unusual stimulus, similar to motion sickness or seasickness (Bouchard, Robillard, and Renaud 2007). The reported incidence of cybersickness amongst users of VR is varied, but at least, 60% of participants in a first VR experience will suffer its symptoms to some degree, and although most users adapt to the environment after few immersions, approximately 5 percent will never do so. The degree of intensity depends on the nature of the VR environment, and previous works have shown the incidence of mild cybersickness ranges between 60% and 90%, with 5-30% of participants having to discontinue research E-mail: augusto.garcia@kom.tu-darmstadt.de evaluations due to strong symptoms (Johnson 2005;Regan 1995). The amount of time required to cause significant sickness depends on the environment, but generally, symptoms can already be expected 5 to 15 minutes after starting (DiZio and Lackner 1997;DiZio and Lackner 2000;Lampton et al. 1994;So and Lo 1999), in some cases even under less than 5 minutes (Dennison, Wisti, and D'Zmura 2016). Although these symptoms usually disappear 15 minutes after the experience is concluded, users can remain sensitized for hours, and re-immersion causes cybersickness to reappear strongly and abruptly, although this is not always the case (DiZio and Lackner 2000). Cybersickness is regarded as a real problem with medical and safety implications, which will surely hamper its integration into the many fields it could be useful for, such as education, entertainment, engineering, and video games (Kennedy and Lilienthal 1995). Head-mounted devices such as the Oculus Rift have already been proven to cause vection and sickness (Palmisano, Mursic, and Kim 2017), but the way through which vection acts is unknown, and symptoms vary greatly from user to user. Currently, the standard method to evaluate cybersickness is the Simulator Sickness Questionnaire (SSQ) (Kennedy et al. 1993). This questionnaire, derived from a prior motion sickness questionnaire, classifies symptoms into three blocks: Nausea, Oculomotor and Disorientation, and then scores each symptom from 1 (none) to 4 (severe). An SSQ Score of 10 or higher is considered to be substantial, and a score higher than 20 is considered to be due to bad VR design (Stanney, Kennedy, and Drexler 1997). At the moment, no specific questionnaire exists to address cybersickness. However, studies have been done (Bruck and Watters 2011) on the viability of the Simulator Sickness Questionnaire (SSQ) as a correct tool to assess cybersickness. These studies suggest that, in the specific environment of virtual reality, symptoms can be structured in 4 factors that account for 78% of the alterations in physiological behavior: General cybersickness (cardiac activity, fatigue, increased nausea and salivation, burping, anxiety), vision factor (eyestrain and headache), arousal (respiration alterations) and fatigue (blurred vision and dizziness), where general cybersickness alone accounts for 38.75% of biosignal variation. Nevertheless, the design of a questionnaire specific to cybersickness remains an issue, especially when considering that it has been proven that the pathophysiology of cybersickness and simulator sickness is not identical. Curiously enough, in (Wilson 1996) and (Young, Adelstein, and Ellis 2007), the possibility whether taking an SSQ/MSQ before and after, or just after the experiment, has a measurable effect on the amount of sickness reported, was studied. Results suggest that indeed reports of motion sickness after immersion are greater when a preemptive questionnaire (such as MSSQ) is given. That is, giving a pre-test SSQ substantially increases the post-test SSQ score. This stresses the importance shown by previous research on attempting to move to more objective measurement methods, such as postural sway (Chardonnet, Mirzaei, and Merienne 2017) or biosignals. Previous studies have reported for cybersickness to increase the levels of cortisol in saliva (Kennedy, Drexler, and Kennedy 2010), cause tachygastria, correlate with sweating and respiratory rate variations (Johnson 2005) heart rate variability (Gavgani, Hodgson, and Nalivaiko 2017;Malińska et al. 2015;Nakagawa 2015;Rieder, Kristensen, and Pinho 2011) and galvanic skin response (GSR) (Gavgani et al. 2017). Other studies state that the onset of cybersickness is induced by specific factors (or triggers), but the increase in cybersickness depends on individual differences in autonomic regulation, which makes it difficult to detect it (Kiryu and Iijima 2014). A summary of the results from studies attempting to correlate vital parameters and cybersickness (Dennison, Wisti, and D'Zmura 2016;Gavgani, Hodgson, and Nalivaiko 2017;Gavgani et al. 2017;Kim et al. 2005;Nam et al. 2001;Roberts and Gallimore 2005) is presented in Table 1. It thus seems the best results are obtained when evaluating ECG, EEG, alterations in the respiratory rate, and GSR. It is generally accepted that sweating is the firstmost response to cybersickness, and the autonomic response is mostly secondary, although present (Gavgani et al. 2017). However, ECG still presents several advantages: It is much more simple to capture, and it is possible to obtain the same parameters from simpler methods, such as Photoplethysmography (PPG) (Garcia-Agundez, Dutz, and Goebel 2017) The aim of this contribution is to analyze the feasibility of detecting cybersickness with a simple 2 lead ECG and thus potentially with other solutions such as Smartphoneor wearable PPG, by finding possible correlations between SSQ scores and relevant ECG parameters.

II. METHODS Participants
A total of n=13 healthy users (2 females), median age 22, participated in our evaluation.

Materials
An Oculus Rift Developer Kit 2 was used connected to a pc with an Intel Core i7 6700k CPU, Nvidia GTX 980 Figure 1: Illustration of the Virtual Environment (phoboslab.org) graphics card and 16 GB of RAM, with which we ensured the maximum framerate (75 fps) and resolution (960x1080 per eye) could be achieved. Electrocardiography was performed using a 2-lead USBAmp Biosignal Amplifier by g.Tec (www.gtec.at). The signal was captured on Matlab (www.mathworks.com), sampled at 256 Hz and passed through a 10th order Butterworth filter to remove the effect of movement and sweating. A simple maxima detection algorithm was then applied to detect N-N intervals. N-N intervals were then normalized using a baseline taken at rest during two minutes prior to the evaluation start. Finally the average value of all N-N intervals (AVGNN) and the standard deviation of all N-N intervals (STDNN) was measured.

Procedure
Users were asked to play a VR shooter (Quakespasm Rift, a VR modification of Quake compatible with the Oculus Rift Developer Kit 2, available at www.phoboslab.org, see Figure 1) for at least 15 minutes, or until the user decided to stop, while sitting. In the game, they had to navigate through three levels and shoot at enemies. No instructions were provided regarding navigation, to make it as natural as possible, regarding the first person shooter scenario. This duration should suffice according to previous research (DiZio and Lackner 1997;DiZio and Lackner 2000;Lampton et al. 1994;So and Lo 1999). The shooter was chosen given the increased amount of rapid lateral movements and combined head rotations along several axes, which makes the event of cybersickness more likely (Bonato, Bubka, and Palmisano 2009). The ECG electrodes were attached before users filled a pre-immersion SSQ. This was done purposely to capture the baseline data. Users were asked to report the evaluators in case signs of discomfort started showing up. In case this happened, the evaluation was interrupted. After the evaluation users filled a post-immersion SSQ as well.
Ethical approval was obtained from the Ethics Committee of the TU Darmstadt. Prior to taking part in the evaluation, users were asked to fill an informed consent.

III. RESULTS & DISCUSSION
The goal of this work was to analyze if ECG data from users who experience cybersickness during a VR game differs from users who do not. This effect was analyzed by performing a statistical analysis of normalized and non-normalized ECG data, extracting the average duration of N-N intervals (AVGNN) and standard deviation of N-N intervals (STDNN), and comparing these values to the resulting scores of the SSQ in both global and specific areas (nausea, oculomotor, disorientation).

Observations during the evaluation: Interruptions and SSQ Scores
Four out of 13 users (Users 5 to 8) had to interrupt the experiment prematurely due to severe cybersickness. The time of their interruptions is presented on Table 3. About half of users (6 out of 13) reported significant symptoms of cybersickness. Across all users, the highest symptom increases were reported for eyestrain, sweating, nausea and vertigo. There was no report on fatigue, and only minor reports on headache and blurred vision occurred. These observations are based on the results presented in Table 2.

Effect of Cybersickness on ECG data and data quality
Complete results for SSQ scores and ECG data are presented in Table 3. In contrast to other studies, where tachycardia was detected in users (Gavgani et al. 2017), users in our evaluation who suffered more cybersickness presented lower heart rates, with heart rates as low as 54 beats per minute at some points. Due to sweating and extreme movement artifacts, the ECGs of users 3 and 13 had to be discarded.

SSQ Scores and ECG Data
On Average, the experiment increased the SSQ total score by 8 points. This increase was greater in the nausea symptom group, but significant for all three groups. The normalized AVGNN and STDNN remained mostly the same, but it is easy to appreciate greater changes in users who experienced severe cybersickness and had to abandon (Users 5 to 8). This phenomenon is further studied in section 3.5. These observations are based on the results presented in Table 3.

Linear regression between SSQ scores and ECG data
In general, it seems there is a higher correlation between STDNN and SSQ scores than between AVGNN and SSQ. In particular, the regression coefficients between STDNN and oculomotor (r=0.47, presented in Figure 2) or disorientation (r=0.38) are higher. The oculomotor value is slightly better than the best one obtained so far (r=.43) on (Kim et al. 2005) for the heart period, and is of particular interest, considering previous works also report higher correlations between oculomotor symtpoms and HR, even though disorientation symptoms are believed to be predominant in Cybersickness (Stanney, Kennedy, and Drexler 1997). These results are similar for both normalized and non-normalized ECG data. All linear regression coefficients are presented in Table 4.

T-tests between SSQ scores and ECG data
Across all users, differences between pre-and post SSQ scores are statistically significant in all test sections (nausea, oculomotor, disorientation and total score, p<0.03). Differences in ECG between users who stop prematurely and those who did not are statistically significant as well (p<0.05) except for normalized AVGNN values. This suggests indeed STDNN is a better parameter to evaluate the possibility of cybersickness occurring. In particular, the difference in total SSQ score between users who interrupted the evaluation and those who did not is very significant (p<0.0001). Results of all t-tests are presented in Table 5.

IV. CONCLUSION
The aim of this study was to provide insight into whether it is possible to detect Cybersickness by focusing on changes in AVGNN and STDNN in comparison with SSQ scores using a simple 2-lead ECG. We found significant differences in AVGNN and STDNN of users who report severe cybersickness in the SSQ, as well as a good correlation between Oculomotor SSQ scores and STDNN. We can thus conclude that there is a correlation between ECG parameters and certain SSQ scores. Since our contribution provides additional evidence on the possible use of ECG to detect cybersickness, we conclude using simple devices to measure biosignals instead, and not in addition to, a pre-and post-immersion SSQ questionnaire may be indeed feasible. We acknowledge that GSR is more reliable, albeit requiring specific measurement hardware. On the other hand, AVGNN and STDNN are simpler to measure, since a simple smartphone using PPG can be used for this purpose. Although it is also possible to measure GSR using a smartphone, it always requires the use of an external device, while PPG is feasible by only using the smartphone itself (Garcia-Agundez, Dutz, and Goebel 2017). In the future, we plan to evaluate the collected vital data and develop a machine learning algorithm to evaluate the possibility of automatically detecting the onset of cybersickness, and whether there is a consistent correlation between in-game events (such as the number of collisions on-screen, or the speed and acceleration of the user) and the user's physiological response. In addition, we will analyze whether PPG data correlate with ECG data during cybersickness as we demonstrated previously at rest (Garcia-Agundez, Dutz, and Goebel 2017) with the ultimate goal of detecting cybersickness with a simple smartphone.  He also provides lectures and practical courses in Serious Games and Game Technologies.