1 Introduction

Many studies have consistently found that participants report decreased cybersickness levels while repeatedly watching virtual reality (VR) content. When participants frequently experienced the same VR content with an interval of one or a few days, their level of discomfort gradually decreased (Hill and Howarth 2000; Howarth and Hodder 2008; Risi and Palmisano 2019). This alleviation has been observed regardless of the display type used (Taylor et al. 2013), exposure interval (Howarth and Hodder 2008), or participant age (Heutink et al. 2019).

In addition, the orientation of the VR scene is known to affect the severity of cybersickness. Previous results have proven that participants reported less discomfort when viewing an inverted (i.e., a virtual scene where the participant feels like he or she is hanging upside-down) or a backward-moving scene than when viewing an upright scene (Bonato et al. 2008; Bubka et al. 2007; Golding et al. 2012).

Although repetition and orientation can change the level of discomfort, the underlying mechanism of their effects has not been clearly explained. Moreover, little is known about how these factors affect human neural processing in cybersickness. While Gavgani et al. (2017) measured physiological signals during repetitive VR exposure, they mainly focused on autonomic responses (e.g., heart rate, respiratory rate, and skin conductance level). Given that our brain integrates body responses during VR interaction, it is critical to reveal how the neural system encodes and processes cybersickness.

Recently, emerging studies have shown that interoceptive signals, such as heartbeats and respiration, are also involved in human cognitive and emotional processes (Coll et al. 2021; Park and Blanke 2019). While the sense of the internal body state (i.e., interoception) usually can be measured by counting one’s heartbeats, several studies have focused on cortical processing of the interoceptive signals (Couto et al. 2015; Luft and Bhattacharya 2015; Marshall et al. 2018; Shao et al. 2011). In particular, the brain continuously communicates with both sensory and visceral organs during higher arousal responses, such as pain (Marshall et al. 2018; Shao et al. 2011). Since cybersickness also induces painful experiences to users, interoceptive cortical processing can provide a broader insight into understanding neural correlates of cybersickness. In this study, we performed a heartbeat-evoked potential (HEP) analysis to investigate changes in brain-heart communication while experiencing cybersickness.

By providing either an upright or an inverted VR roller coaster three times, we intended to induce different levels of discomfort. We recorded both subjective and objective measures to evaluate the effects of orientation and repetition on cybersickness. In particular, we focused on computing HEP amplitudes to demonstrate the interaction between the brain and heartbeats during cybersickness. Furthermore, we investigated the cortical activities that could be correlated with subjective discomfort.

2 Related works

2.1 Possible explanations for cybersickness reduction

Previous studies have tried to explain why a few methods are effective for reducing cybersickness. Several authors have claimed that participants adapt to the VR content during repetition (Hill and Howarth 2000; Howarth and Hodder 2008). That is, people feel less illusory self-motion (i.e., vection) as they become habituated with the vection-evoking scene due to the repetition. According to Regan (1995), however, participants might accumulate contextual knowledge about the VR content during the repetition. The author pointed out that repetitive exposure can help participants become familiar with the VR system and prepare them for the sickness-evoking events rather than a change in the level of vection.

Regarding the effect of VR orientation, it has been speculated that how strong the participants experience vection in a specific orientation might be associated with the discomfort (Bonato et al. 2008; Golding et al. 2012). As the human rarely experiences an unusual orientation, such as an inverted one, it is less likely to induce strong vection as well as cybersickness. Alternatively, an unnatural context might prevent immersion in VR, thereby resulting in lower cybersickness.

Based on these studies, we categorized three possible explanations for the reduction of cybersickness. First, participants can report less cybersickness when they feel less illusory self-motion to the vection-evoking scene. Second, even though they experience a similar level of vection, accumulated knowledge during VR experience helps to develop a coping strategy for discomfort. Lastly, participants’ lower immersion into the VR scene might alleviate cybersickness. In this study, we named each factor as the level of reality, predictability, and immersion. Then, we investigated which factors could be associated with the effects of orientation and repetition on cybersickness.

2.2 Measuring cybersickness

Many efforts have been undertaken to develop a reliable measurement for cybersickness. Among such efforts, the Simulator Sickness Questionnaire (SSQ) (Kennedy et al. 1993) has been the most widely used method for quantifying user discomfort. The SSQ considers various cybersickness symptoms, including nausea (SSQ nausea; SSQ-N), oculomotor (SSQ oculomotor; SSQ-O), and disorientation (SSQ disorientation; SSQ-D). As a person experiences more severe cybersickness, the SSQ score increases. Besides the SSQ, the Fast Motion sickness Scale (FMS) (Keshavarz and Hecht 2011) is also a well-known questionnaire. This index, which is relatively more concise than the SSQ, requires the verbal reporting of a single number for cybersickness (0: no sickness at all; 20: severe sickness).

Although these approaches are intuitive and easy to perform to measure cybersickness, several studies have tried to measure the symptoms in a more indirect and non-invasive manner (Dennison et al. 2016; Farmer et al. 2015; Kim et al. 2005). According to Chang et al. (2020), several physiological responses, such as body sway, electrogastrogram, electrocardiogram (ECG), and electroencephalogram (EEG), have been widely studied as promising indices.

Most previous studies have focused on an individual index for detecting cybersickness. However, recent studies have found that the communication between physiological signals is also critical for monitoring higher arousal responses (Coll et al. 2021; Park and Blanke 2019). In particular, several researchers have found that the HEP amplitude, which reflects cardio-receptive cortical processing, can be an indicator for detecting pain (Marshall et al. 2018; Shao et al. 2011). As Gavgani et al. (2017) mentioned, not only the individual autonomic response but also the central nervous system might be closely related to determining the level of cybersickness.

2.3 Previous findings in cybersickness-related EEG

Several studies have indicated that the frontal and parietal lobes can be relevant cortical areas for cybersickness. Using functional magnetic resonance imaging, Farmer et al. (2015) investigated neural correlates of nausea while experiencing visually induced motion sickness (VIMS). The results showed a positive correlation between the participants’ nausea scores and the left inferior frontal activity. In addition, Khoirunnisaa et al. (2018) performed correlation feature selection to determine the optimal channel location for identifying cybersickness-related EEG. The author concluded that the beta frequency band in the frontal area is a promising location for user discomfort.

In Arshad et al. (2015), the participants reported less motion sickness symptoms when their left parietal lobes were stimulated using transcranial direct current stimulation (tDCS). Similarly, Takeuchi et al. (2018) found reduced subjective disorientation symptoms after applying anodal tDCS to the right temporoparietal junction of VR users, indicating that this area could be directly associated with user discomfort.

A limited number of studies have investigated cybersickness-related brain waves in the temporal domain using event-related potential (ERP) analysis. In Wu et al. (2020), the participants performed a two-choice oddball task before and after the VR experience, and their brain waves were recorded during the task. Because of cybersickness, the participants showed impaired response inhibition and changes in the N2 and P3 components. In Ahn et al. (2020), the participants watched 200 trials of short VR scenes (6 s) consisting of an accelerated or constant velocity under EEG recording. The results indicated increased P3 amplitudes during the viewing of accelerated moving conditions, albeit this was only for the participants who reported more severe discomfort.

Although the aforementioned studies attempted to reveal the neurological mechanism of cybersickness in the temporal domain, they had a few limitations. Brain signals were recorded during a cognitive task or a relatively short period of VR experience (i.e., 6 s). Thus, their results might reflect the change in cognitive ability after cybersickness or motion perception during the VR experience rather than cybersickness itself. It is necessary to acquire the EEG signal while the participants are experiencing an immersive VR scene. Recording the brain waves while the participants are watching a sufficient length of VR content can better reflect the neural changes caused by cybersickness.

In this study, we conducted an HEP analysis to investigate the neural correlates of cybersickness. This analysis computes the HEP amplitude by averaging the brain waves that are time locked to specific points of the ECG signals, such as the R or T peak (Park et al. 2016). As the peak of ECG signals becomes a cortical activity event, this approach enables the provision of continuous visual information that is sufficiently long to experience immersive VR. In addition, this approach has been suggested to reveal the neurophysiological index of brain-heart communication (Coll et al. 2021; Park and Blanke 2019). It focuses on the continuously updated neural responses of the internal organs (i.e., heart), which can be related to human perceptual processes. Previous studies have found changes in the HEP amplitude of the frontal cortex in an arousing context, such as pain, anxiety, and negative emotions (Couto et al. 2015; Luft and Bhattacharya 2015; Schulz et al. 2013; Shao et al. 2011).

Based on these results, we hypothesized that experiencing cybersickness can affect the mean HEP amplitude. We investigated whether the HEP amplitude could reflect varying degrees of cybersickness by manipulating the orientation and repetition of the VR content. We also examined whether each participant’s HEP amplitude could be correlated to the subjective level of discomfort, thus showing the possibility of a new physiological marker of cybersickness.

3 Method

3.1 Participants

Twenty undergraduate students at Korea University (mean age = 24.65, SD = 2.48; eight females) participated in this experiment. All the participants were healthy and had not experienced any brain-related injuries. They had normal or corrected-to-normal vision. Before the experiment, we provided them with a written informed consent form approved by the Institutional Review Board of Korea University. Every student participated in the experiment three times at 2-day intervals. For each visit, the participants experienced four VR roller coaster rides (see details in the Sect. 3.2).

3.2 Procedure

The experiment was performed three times, with an interval of two days between each experiment. Each time, the participants were provided with a written informed consent form and received brief information about the experiment. Before the experiment began, we attached electrodes to the participant’s body to record the EEG, electrooculogram (EOG), and ECG signals (see the Sect. 3.3). After the attachment, we first recorded the participant’s resting state. The participants closed their eyes for 1 min and then opened them for another 1 min while the signals were being recorded. After the resting state session, the participants underwent a calibration session to record accurate eye trajectories. When they successfully completed the calibration, they experienced four roller coaster rides in a row. According to the pre-assigned order, half of the participants watched the upright VR scene first (first and second rides) and then watched the inverted scene (third and fourth rides). The rest of the participants watched the scene in the reverse order (i.e., they watched the inverted scene twice and then the upright scene twice). Between each ride, the participants had a 10-min break for recovery. For the SSQ score, the participants filled out the questionnaire before and after each VR experience. They verbally reported their FMS, reality, predictability, and immersion scores immediately after the end of each ride. During all experiment sessions, participants were required to maintain a fixed sitting posture using a chin rest. Figure 1 shows the procedure for the entire experiment.

Fig. 1
figure 1

Experimental procedures of each visit. The interval between each visit was two days. Participants watched a roller coaster VR four times. To control the order effect, half of the participants watched the upright VR scene for the first and second ride and then watched the inverted one for the third and fourth ride. The rest of the participants watched the scene in the reverse order. Before and after each riding, we obtained the subjective measures for the VR experience

Full size image

3.3 Apparatus

To record the EEG signal, we used a 64-channel NeuroScan SynAmps amplifier (Compumedics USA, E1 Paso, TX, USA) with a 1000 Hz sampling rate. EOG signals were recorded using four monopolar EOG electrodes. We attached two vertical EOG (VEOG) and two horizontal EOG (HEOG) electrodes on the participant’s face. The upper and lower VEOG electrodes were located on the upper and lower parts of the participant’s left eye, while the left and right HEOG electrodes were attached to each side of the participants’ temples. We used linked earlobes as the reference electrode and maintained the impedances below 10 k\(\Omega\).

To measure the ECG signal, we used a BIOPAC ECG100C amplifier (BIOPAC Systems Inc., Goleta, CA, USA) with a sampling rate of 1000 Hz. We attached two disposable ECG electrodes (\(\hbox {Kendall}^{\mathrm{TM}}\) 200 Foam Electrodes) 5 cm below the participants’ left and right collarbones. Offline analysis was conducted using AcqKnowledge 4.2 (BIOPAC Systems, Inc.).

To perform the HEP analysis, it was necessary to mark down the precise onset point of the VR content on both the EEG and ECG signals. For this purpose, we developed a customized trigger distribution system using an Arduino and in-house Unity script (Song 2018) (Fig. 2). When the VR roller coaster began, a light sensor attached to the display monitor turned on briefly, sending a 5-V signal to the distribution system through a StimTracker Quad (Cedrus Corporation, San Pedro, CA, USA). The system simultaneously distributed the onset signal to the NeuroScan and BIOPAC amplifiers, which enabled the recording of the exact start time for both physiological signals (Fig. 3).

Fig. 2
figure 2

Hardware setup for the experiment. a Schematic diagram of the overall experimental system. We recorded EEG signals during the participant’s rest state and VR experience. A red dot indicates one of the ECG electrodes of the participant’s collarbone. b Using a customized trigger system, we simultaneously sent a 5-V signal to the EEG and ECG amplifiers. c A photo of the participant during the experiment

Full size image
Fig. 3
figure 3

Schematic diagram of the trigger system. When the VR roller coaster began, a light sensor attached to the display monitor turned on briefly, sending a 5-V signal to the trigger system. This system enabled to record of the exact onset time of content to EEG and ECG signals

Full size image

3.4 Virtual reality system

The specifications of our VR system are as follows: Intel Core i7-4790K CPU @ 4.00 GHz and GeForce GTX1080 Ti AORUS Xtreme D5X 11 GB. A virtual roller coaster was adopted from “Animated Steel Coaster”Footnote 1 and was modified to implement two types of VR scenes (i.e., upright and inverted) using Unity 2018.1.8f1 (64-bit). Whereas the z value of the camera rotation was 0 in the upright condition, and that of the inverted condition was – 180 (upright: (0, – 180, 0); inverted: (0, – 180, – 180)) (Fig. 4a). The content field of view was \(80^\circ\), and the duration of each ride took approximately 3.5 min.

Fig. 4
figure 4

Illustrations of the VR system and data analysis. a In this diagram, we assumed that a participant watched an upright scene twice (\(1^{st}\) and \(2^{nd}\) ride), then watched an inverted one (\(3^{rd}\) and \(4^{th}\) ride). b While participants experienced VR rides, we recorded both ECG and EEG signals. Asterisk on the ECG indicates the R peak of cardiac signals. By averaging the cortical activity time locked to this peak, we computed HEP amplitudes for each orientation

Full size image

3.5 Questionnaires

We used two types of questionnaires to measure the subjective level of cybersickness: the SSQ and FMS. For the SSQ, we calculated four SSQ scores (i.e., SSQ total, SSQ-N, SSQ-O, and SSQ-D) according to Kennedy et al. (1993). Notably, we used relative SSQ scores (i.e., post-SSQ − pre-SSQ) for the data analysis. For the FMS, the participants were required to verbally report their discomfort level by a number (0−20).

In addition, the participants reported a subjective level of reality, predictability, and immersion scores after each ride. All the scores were marked on a six-point scale (0-5). The reality score was aimed at measuring the vividness of the VR experience. We instructed the participants to report a higher reality score if their VR riding was a more realistic experience and induced a higher level of vection. The predictability level was designed to measure whether the participants could expect what would be happened in the VR scene. A higher score indicated that the VR scene became familiar such that the participants felt like they could predict the upcoming path of the roller coaster track. The immersion level indicated the extent to which the participants were immersed in VR riding. A higher score indicated that they were highly focused on the VR content.

3.6 Data analysis

3.6.1 Statistical analysis

We used a repeated measures analysis of variance (rmANOVA). First, we performed Mardia’s test to ensure the normality of the distributions in both subjective and physiological measures. The results indicated that only the SSQ-N score violated the normality assumption. Therefore, the experimental effects of the SSQ-N were tested by a nonparametric approach using the R statistics package “nparLD” (Noguchi et al. 2012). We reported the ANOVA-type statistics of the package, which are based on a rank-based repeated-measures design, with the denominator set to infinity. In addition, we applied the Greenhouse-Geisser corrections if the assumption of sphericity was violated in the rmANOVA.

Meanwhile, we also tested the carryover effect on the subjective measures. Although there was a 10-min break between each VR ride, we checked whether the participants showed increased or decreased subjective measures owing to the four serial ridings, regardless of experimental conditions (see Online Resource 1).

All statistical analyses were performed using IBM SPSS (version 21.0; SPSS, Inc., Chicago, IL, USA) and R software (version 3.2.3; R Foundation of Statistical Computing, Vienna, Austria; R Core Team 2015).

3.6.2 Heartbeat-evoked potential

All EEG signals were analyzed using the EEGLAB Toolbox version 14.1.1b (Delorme and Makeig 2004) in MATLAB R2018b. First, we merged the EEG and ECG signals by aligning the first trigger point. We then epoched the signals from 0 to 180 s relative to the onset of the VR content. After that, we resampled the merged data to 250 Hz. The cut-off frequency of the high-pass filter was 0.5 Hz, whereas that of the low-pass filter was 100 Hz. In addition, we removed the 60-Hz line noise. Bad channels were rejected and then interpolated on the basis of the PREP pipeline (Bigdely-Shamlo et al. 2015). The thresholds for bad channel detection were as follows: robust deviation threshold = 7, high frequency noise threshold = 3.5, and correlation threshold = 0.5. Next, we performed an independent component analysis and used a SASICA plugin (Chaumon et al. 2015) to remove the EOG and ECG related components.

After the preprocessing, we conducted an offline filter for the continuous EEG and ECG data. The cut-off frequencies of the high- and low-pass filters were 1 Hz and 40 Hz, respectively. Following the method in Park et al. (2016), we used a common average reference for the EEG data. The HEPs were computed using the R-peaks of the ECG signal (Fig. 4b). The R-peaks were detected on the basis of the individual’s QRS complex using the “findpeak” function of the MATLAB signal processing toolbox. Then, the EEG data were epoched between − 200 ms and 600 ms relative to the R-peak onset. For the artifact rejection, epochs that exceeded an amplitude of \(\pm 75\) \(\upmu\)V or a maximum slope of 45 \(\upmu\)V/epoch were excluded. After the artifact rejection, the epochs were averaged into six experimental conditions (i.e., upright-day 1, upright-day 2, upright-day 3, inverted-day 1, inverted-day 2, and inverted-day 3).

We set the scalp regions and time windows for the HEPs following previous studies (Baranauskas et al. 2017; Coll et al. 2021; Shao et al. 2011). In particular, we only focused on the two areas on the scalp that consistently showed sickness-induced changes in the brain waves: fronto-central and parietal areas (Coll et al. 2021; Kim et al. 2019; Luft and Bhattacharya 2015; Shao et al. 2011). The fronto-central area was clustered with FZ, F1, F2, F3, F4, F5, F6, FCZ, FC1, FC2, FC3, and FC4. The parietal area was clustered with PZ, P1, P2, P3, P4, P5, P6, POZ, PO3, PO4, PO5, and PO6. The time window for the analysis was determined to be the average activity in the 300-448 ms post-R-peak onset (Fig. 5).

Further information for the other physiological indices (i.e., ECG and eye trajectories) is described in the Online Resource 1.

Fig. 5
figure 5

Electrode locations and time window for the HEP analysis. We targeted two cortical areas (fronto-central and parietal areas), and the time window was 300–448 ms after the onset of the ECG R-peak

Full size image

4 Results

4.1 Simulator Sickness Questionnaire

We performed a \(2\times 3\) rmANOVA with the orientation and repetition as the factors of the SSQ scores. It is worth noting that the effects on the SSQ-N score were tested using a nonparametric approach because of the violation of normality.

For the total SSQ score, we found a significant main effect of repetition (F(2,38) = 5.53, p = 0.01, \(\eta _{\text {p}}^{2}\) = 0.23). However, there was no significant effect of orientation (F(1,19) = 3.46, p = 0.08, \(\eta _{\text {p}}^{2}\) = 0.15) or the interaction between the orientation and the repetition (F(2,38) = 0.15, p = 0.86, \(\eta _{\text {p}}^{2}\) = 0.01). The participants exhibited gradually decreased cybersickness as they repeatedly experienced the VR content regardless of the orientation (Fig. 6a).

The SSQ-N score showed significant main effects of orientation (F(1,\(\infty\)) = 4.80, p = 0.03) and repetition (F(1.86,\(\infty\)) = 3.96, p = 0.02). The inverted VR ride induced a relatively lower level of SSQ-N than that of the upright VR ride. Moreover, repetitive VR rides mitigated the participant’s nausea symptoms. However, no significant interaction effect was found (F(1.52,\(\infty\)) = 0.01, p = 0.98) (Fig. 6b).

Fig. 6
figure 6

Box-and-whisker plots of the SSQ scores. a SSQ total, b SSQ nausea (SSQ-N), c SSQ oculomotor (SSQ-O), d SSQ disorientation (SSQ-D). The red box represents an upright condition, while the blue one represents an inverted condition. A dot and horizontal line in the box represent the mean and median of each experimental condition, respectively. The black cross indicates outliers. All SSQ scores except SSQ-O were reduced due to the repetition. Only SSQ-N indicated a significant orientation effect, showing a lower level of nausea in the inverted condition

Full size image

For the SSQ-O score, we did not find any significant statistical effects. However, the SSQ-D score showed a significant main effect of repetition (F(2,38) = 6.82, p = 0.00, \(\eta _{\text {p}}^{2}\) = 0.26), but no significant differences owing to the orientation (F(1,19) = 3.09, p = 0.10, \(\eta _{\text {p}}^{2}\) = 0.14) or the interaction (F(2,38) = 0.001, p = 0.99, \(\eta _{\text {p}}^{2}\) = 0.00). Although the eye-related symptoms were not affected by experimental conditions (Fig. 6c), the level of disorientation symptoms slowly decreased with the repetition (Fig. 6d).

Meanwhile, we tested the order effect on the SSQ scores to determine whether the participant’s SSQ level was affected by the order of the ride rather than by the orientation. The results indicated that there were no significant differences in the level of each SSQ score between each VR ride (see Online Resource 1, Figure S1).

4.2 Fast Motion sickness Scale

A \(2\times 3\) rmANOVA with the orientation and repetition as the factors was conducted. The results showed significant main effects of orientation (F(1,19) = 4.54, p = 0.046, \(\eta _{\text {p}}^{2}\) = 0.19) and repetition (F(2,38) = 19.53, p = 0.00, \(\eta _{\text {p}}^{2}\) = 0.51). However, the interaction effect was not significant (F(2,38) = 0.17, p = 0.85, \(\eta _{\text {p}}^{2}\) = 0.01). The FMS decreased when the participants experienced the inverted condition. In addition, the score was lowered by the repetitive VR experience (Fig. 7a). The order effect on the FMS score indicated no significant differences (see Online Resource 1, Figure S2a).

Fig. 7
figure 7

Box-and-whisker plots of the FMS and other subjective reports. a FMS, levels of b reality, c predictability, d immersion. The red box represents an upright condition, while the blue one represents an inverted condition. A dot and horizontal line in the box represent the mean and median of each experimental condition, respectively. The black cross indicates outliers. Participants reported a lower FMS in the inverted condition, and the score was decreased as they repeatedly experienced VR. The level of reality was significantly lower in the inverted condition for all three days. The level of predictability significantly decreased in the inverted condition. However, the score increased during the repetition regardless of the orientation. The level of immersion showed a similar trend as the reality score: the participants showed lower immersion in the inverted VR for all three days

Full size image

4.3 Levels of reality, predictability, and immersion

For each verbal report, we investigated the orientation and repetition effects using a \(2\times 3\) rmANOVA.

The level of reality showed a significant orientation effect (F(1,19) = 31.86, p = 0.00, \(\eta _{\text {p}}^{2}\) = 0.63) and an interaction effect (i.e., orientation \(\times\) repetition: F(2,38) = 3.64, p = 0.04, \(\eta _{\text {p}}^{2}\) = 0.16); however, the repetition effect was not significant (F(2,38) = 2.24, p = 0.12, \(\eta _{\text {p}}^{2}\) = 0.11). The participants considered VR riding to be more realistic in the upright scene. They felt stronger illusory movement when watching the usual orientation of the roller coaster. However, the level of reality at each visit was similar to one another (Fig. 7b).

The level of predictability showed that there were significant main effects of orientation (F(1,19) = 48.47, p = 0.00, \(\eta _{\text {p}}^{2}\) = 0.72), repetition (F(2,38) = 33.20, p = 0.00, \(\eta _{\text {p}}^{2}\) = 0.64), and their interaction (F(1.26,23.96) = 7.82, p = 0.01, \(\eta _{\text {p}}^{2}\) = 0.29). The participants showed a higher level of predictability in the upright one. This result indicated that the participants could better expect what would happen soon in the upright condition. In addition, regardless of the VR orientation, repetitive visits helped the participants become predictable with the VR content. A significant interaction effect indicated that the differences between the predictability scores of each orientation decreased according to the repetitive visits (Fig. 7c).

For the level of immersion, although the orientation effect was significant (F(1,19) = 15.56, p = 0.00, \(\eta _{\text {p}}^{2}\) = 0.45), the main effect of repetition (F(2,38) = 1.06, p = 0.36, \(\eta _{\text {p}}^{2}\) = 0.05) and the interaction effect (F(2,38) = 0.32, p = 0.73, \(\eta _{\text {p}}^{2}\) = 0.02) were not significant. The participants reported a lower level of immersion when watching the inverted VR scene than the upright one. It seemed harder for the participants to concentrate on the unusual orientation. These differences remained constant for three days, which indicated that the participants’ degree of immersion was not affected by the repetitive visits (Fig. 7d).

Meanwhile, the order effect on each verbal report indicated no significant differences (see Online Resource 1, Figure S2b, c, d).

4.4 Heartbeat-evoked potential

For the HEPs in a given time window (i.e., 300-448 ms after the R-peak onset), we performed a \(2\times 2\times 3\) rmANOVA with three within-subjects factors: site (fronto-central vs. parietal), orientation, and repetition. The results indicated a significant main effect of the site (F(1,19) = 40.03, p = 0.00, \(\eta _{\text {p}}^{2}\) = 0.68) and a significant interaction effect between the site and the orientation (F(1,19) = 5.32, p = 0.03, \(\eta _{\text {p}}^{2}\) = 0.22). However, other factors did not affect the HEP amplitude (orientation: F(1,19) = 1.93, p = 0.18; repetition: F(2,38) = 1.53, p = 0.23; site \(\times\) repetition: F(2,38) = 0.22, p = 0.81; orientation \(\times\) repetition: F(2,38) = 0.35, p = 0.71; site \(\times\) orientation \(\times\) repetition: F(2,38) = 0.58, p = 0.57).

Fig. 8
figure 8

The results of the HEP analysis. Topographical representations of a the grand average of all conditions and b two orientation conditions and their difference. Each dot in the topography represents the electrode locations for the analysis. c Each orientation’s mean fronto-central HEP amplitude indicated a significant suppression in the upright condition at a given time window (shaded area: 300–448 ms post-R-peak). d However, no orientation effect on ECG signals was observed, indicating that the difference of HEP amplitudes was not originated from the cardiac field artifacts

Full size image

The HEP amplitude showed distinctive reflections depending on the cortical areas. In the given time window, the grand-averaged HEP waves at the fronto-central area indicated a positive reflection, while those at the parietal area indicated a relatively negative reflection (Fig. 8a). Topographical maps of the HEP amplitude during each orientation condition showed that the difference between the two conditions indicated a suppression at the fronto-central site (Fig. 8b).

Based on the result of the significant interaction, a follow-up comparison with a Bonferroni correction was performed to investigate which cortical area showed the HEP difference according to the orientation. The results indicated that the HEP amplitude differed significantly at the fronto-central area (t(19) = −2.50, p = 0.02, Cohen’s d = 0.36). The mean HEP amplitude during the inverted VR scene was more positive than that of the upright VR scene (Fig. 8c). However, this difference was not significant at the parietal area (t(19) = 1.83, p = 0.08, Cohen’s d = 0.37). ECG amplitudes of each orientation did not indicate any difference, supporting that the HEP difference at the fronto-central area reflects changes in cortical activity time locked to heartbeats instead of cardiac field artifacts (Fig. 8d) (Park et al. 2016).

Next, we performed a Spearman’s bivariate correlation to investigate whether the participants’ HEP amplitude in each orientation correlated with their subjective discomfort. The mean HEP amplitude at the fronto-central lobe and the z-transformed SSQ scores were used for the analysis. Outliers who showed a z-transformed SSQ total score higher than 2.5 were excluded. The results showed a significant negative correlation between the HEP amplitude and the participants’ z-transformed SSQ-D scores (Spearman’s rho = −0.33, p = 0.045) Fig. 9). However, other subscales did not show any significant relationships (SSQ total: Spearman’s rho = −0.27, p = 0.11; SSQ-N: Spearman’s rho = −0.13, p = 0.45; SSQ-O: Spearman’s rho = −0.17, p = 0.31).

Fig. 9
figure 9

A scatter plot between the z-transformed SSQ-D and the fronto-central HEP amplitude in each orientation. We found a significant negative correlation (Spearman’s rho = −0.33). Shaded areas are 95% confidence intervals

Full size image

Meanwhile, we performed additional tests to ensure that we set a proper temporal window for the HEP analysis. The results showed that the statistical significance of the HEP amplitude did not depend on the specific measurement window (see Online Resource 1, Table S1).

For the ECG and eye trajectories, we did not observe any statistically significant results due to the experimental conditions. Additional data are given in Online Resource 1, Figure S3.

5 Discussion

This study aimed to investigate physiological correlates, especially the neural processing, of cybersickness by manipulating the orientation and repetition of VR. We provided both upright and inverted VR roller coaster scenes with three repetitions. According to the results, the participants experienced less cybersickness when watching the VR content repeatedly. They also reported decreased nausea while watching the inverted VR. Regarding cortical activity, we found significant suppression at the fronto-central HEP amplitudes in the upright condition. A bivariate correlation analysis revealed a negative correlation between the HEP amplitude and the level of disorientation symptoms. In this section, we provide an in-depth discussion of each result and discuss the limitations and future directions of the present study.

5.1 Subjective measures for cybersickness

In line with previous studies (Howarth and Hodder 2008; Risi and Palmisano 2019), the level of cybersickness decreased as the participants repeatedly experienced the same VR environment regardless of the orientation of the scene. These results confirm that repeated exposure is a powerful way to alleviate the subjective level of cybersickness. Several researchers have suggested the underlying mechanism of alleviation as adaptation, explaining that people could be desensitized to the vection-evoking visual stimuli owing to the repetition (Hill and Howarth 2000; Howarth and Hodder 2008; Risi and Palmisano 2019). However, according to our additional reports from the participants, there were no differences in the level of reality during the repetition. Although the participants watched the same VR roller coaster for three days, their feelings of motion (i.e., as if they were riding on a real roller coaster) did not change significantly. In addition, the participants could maintain their immersive VR experience during the repetition.

However, the only change due to repetitive exposure was that the participants became more predictable regarding the VR content for both orientations. This change could increase the possibility of accurately anticipating the upcoming path of the course. As we used a VR roller coaster that navigated a predetermined track, the participants were able to become familiarized with the track and roughly remember the driving course during the repetition. Earlier recognition of the cybersickness-evoking track could help the participants to prepare for their discomfort. According to Bos et al. (2008), predicting the forthcoming event might support our brain to synchronize sensory signals between what we have expected and actually perceived. This predictive process may minimize sensory conflict and induce a decreased level of cybersickness.

We also observed a significant orientation effect on subjective discomfort. Consistent with previous results (Bonato et al. 2008; Bubka et al. 2007; Golding et al. 2012), the participants reported lower SSQ-N and FMS scores in the inverted VR scene than those in the upright one. According to the additional reports from the participants, all indices showed significant differences depending on the orientation. The participants reported lower levels of reality, predictability, and immersion when experiencing the inverted condition. The unusual VR orientation induced a weak vection and prevented them from immersing in the VR. In terms of predictability, while the participants could better predict the upcoming event in the upright condition, they showed more severe discomfort. This result seems contrary to the repetition effect, explaining that increased predictability due to the repetition reduces cybersickness. Perhaps this might have originated from the unnatural context of the inverted condition. Compared to the normal orientation, the participants reported significantly lower levels of reality and immersion due to the lack of experience in the inverted one. As they hardly focused on the unusual situation, there might be relatively little room for the predictability to influence the degree of cybersickness. Thus, it is required to compare the predictability effect on cybersickness not between but within VR content.

5.2 Heartbeat-evoked potential approach

We found a significant orientation effect on the HEP amplitudes of the fronto-central area at 300-448 ms after the onset of the ECG R-peak. The participants showed HEP suppression at the frontal region in the upright VR condition, unlike in the inverted VR condition. However, we did not observe any changes in HEP amplitudes with repetitive exposure.

In Shao et al. (2011), the participants were subjected to painful conditions (i.e., high arousal level) using cold water. The results showed decreased mean HEP amplitudes at the right frontal region in the painful condition than in the no-pain control group. In addition, there was a negative correlation between the standardized HEP magnitude and the pain intensity rating. Given that cybersickness is also a type of higher arousal condition that evokes painful experiences from participants, it seems plausible that the upright VR condition indicates HEP suppression. Consistent with the results of Shao et al. (2011), we also found that HEP amplitudes at the fronto-central region decreased as the SSQ-D scores increased. These results suggest that the magnitude of the HEP activity at the frontal area might reflect the level of arousal caused by cybersickness.

However, we did not observe any difference in HEP amplitude owing to the repeated VR experience. While subjective discomfort gradually reduced with iteration, the mean HEP amplitude was similar in each experience. A recent study by Zhang et al. (2021) revealed unique cortical activity depending on prior knowledge of a content using event-related desynchronization. As the HEP analysis mainly focuses on brain-heart communication rather than memory-related brain activity, it might be insufficient to elucidate how accumulated knowledge by repetition can change the neural process of cybersickness. Additional approaches, such as spectral or connectivity analysis, might be useful to clarify the effect of repetition on cybersickness.

Although we observe a significant negative correlation between the HEP amplitude and the disorientation score, this does not imply a causal relationship between the two indices. Further experiments are required to demonstrate this causal relationship. First, it is necessary to apply more diverse types of VR content, including virtual riding. In addition, carefully matched groups with larger sample sizes would be supportive of generalizing the experimental results. Lastly, the VR orientation should be randomly changed during the experimental session to elucidate the causation.

Notwithstanding a few limitations, HEP analysis still has some advantages in investigating temporal cortical activity during cybersickness. This analysis uses a peak of the ECG signal as an event, thereby enabling the participants to experience a sufficient duration of VR content. Because conventional ERP studies present visual stimuli for a short period of time (only for several seconds), it is difficult to apply this technique to immersive VR experiments. However, HEP analysis allows sufficient time for the VR interaction (this study: 3.5 min), which gives it a higher ecological validity for clarifying the neural correlates during the interaction. Moreover, HEP analysis can provide insights into how the brain and heart communicate under cybersickness. Though most previous studies have focused on an individual autonomic response for identifying cybersickness, recent studies have shown that the communication between the brain and visceral organs is also related to the painful experience. For these reasons, we performed the HEP analysis to elucidate the cardio-receptive cortical processing of cybersickness under immersive VR. To the best of our knowledge, this study is the first to report on the neural correlates of cybersickness using HEP analysis. Considering the significant correlation between the HEP amplitude and the subjective cybersickness score, this approach suggests the possibility of detecting cybersickness at the cortical level.

5.3 Limitations

As we mentioned above, we did not find any differences in the HEP amplitude depending on the repetitive exposure. This might be related to the participants’ methods of coping with cybersickness. Instead of being desensitized to the VR scene, they increased the predictability of the upcoming event during the repetition, which might involve more cognitive loads. Thus, other EEG analyses, such as alpha phase adjustment or connectivity analysis, might be more suitable to clarify the neural correlates of the repetition effect (Mathewson et al. 2014; Solís-Vivanco et al. 2018; Zhang et al. 2021).

In addition, we can consider various human factors in future research. For example, a recent study by Wei et al. (2019) demonstrated that the cortical activity of motion perception could be affected by individual susceptibility to VIMS. The participants with higher susceptibility showed increased N2 amplitudes and impaired theta-band phase synchronization when watching coherent moving dots. These results suggest that we can consider individual differences to clarify the EEG dynamics of cybersickness. In future research, it would be worthwhile to take into account several human factors, such as susceptibility to cybersickness, interoceptive perception sensitivity, and vection perception sensitivity.

Though this study adopted various types of self-reported measures, we have not evaluated their psychometric properties. Further studies are required to provide psychometric attributes of each measure, such as validity and reliability.

6 Conclusion

This study provided two opposite VR directions (upright vs. inverted) three times and observed the participants’ cybersickness-related responses through subjective and physiological measures. In particular, we investigated changes in brain-heart communication during cybersickness using HEP analysis. The results showed that subjective discomfort was gradually reduced by repetition. The participants also reported lower cybersickness in the inverted condition. According to the additional reports, while repetitive exposure can induce a higher level of predictability, the VR orientation can affect the feeling of vection and immersion. For the physiological changes, we only observed significant suppression at the fronto-central HEP amplitudes in the upright condition. In addition, there was a significant negative correlation between the HEPs and the level of disorientation symptoms. These results agree with the existing evidence of reduced cybersickness depending on the orientation or repetition of VR. Furthermore, this study suggests the HEP amplitudes might reflect the neural correlates of cybersickness, showing the possibility of a new cortical marker of cybersickness.