Abstract
This study explores the effect of the experimenter’s gender/sex and its interaction with the participant’s gender/sex as potential contributors to the replicability crisis, particularly in the man-gendered domain of VR. 75 young men and women from Western France were randomly evaluated by either a man or a woman during a 13-min immersion in a first-person shooter game. Self-administered questionnaires were used to measure variables commonly assessed during VR experiments (sense of presence, cybersickness, video game experience, flow). MANOVAs, ANOVAs and post-hoc comparisons were used. Results indicate that men and women differ in their reports of cybersickness and video game experience when rated by men, whereas they report similar measures when rated by women. These findings are interpreted as consequences of the psychosocial stress triggered by the interaction between the two genders/sexes, as well as the gender conformity effect induced, particularly in women, by the presence of a man in a masculine domain. Corroborating this interpretation, the subjective measure of flow, which is not linked to video games and/or computers, does not seem to be affected by this experimental effect. Methodological precautions are highlighted, notably the brief systematic description of the experimenter, and future exploratory and confirmatory studies are outlined.
Graphical abstract
![](http://media.springernature.com/lw685/springer-static/image/art%3A10.1007%2Fs10055-024-00993-2/MediaObjects/10055_2024_993_Figa_HTML.jpg)
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
1.1 Replicability crisis
The possibility of experimentally reproducing a result is at the heart of many sciences using experimental methodology, and is the guarantee of our confidence in its operation. In the words of a well-known Popper quote: “unique events that cannot be reproduced are of no importance to science” (Popper 1959). It is because the same cause produces the same consequences (in the same circumstances) that an individual, whether scientific or not, can have confidence in the existence of the observed causal effect, whatever the context of its measurement. However, some experimental sciences, and psychology in particular, are currently undermined by a notorious inability to reproduce experimental results. For example, Baker (2015) revealed that out of 100 articles published in leading psychology journals, only a third to half of the experimental results could be replicated, as Owens (2018) later found. What's more, 70% of the 1500 researchers surveyed admitted failing to replicate a colleague’s experiment, and more than half failing to replicate their own experiment (Baker 2016). This replicability crisis extends far beyond psychology and can be observed in many other fields, such as pharmacology (Prinz et al. 2011), medicine (Begley & Ellis 2012), or even economics (Bergh et al. 2017) and the social sciences (Camerer et al. 2018). It is puzzling that the results of experimental studies that sometimes founded sub-fields of research, such as Bargh’s (1996) social priming, have failed to be replicated (Doyen et al. 2012), nor have the experimental variations that followed (Harris et al. 2013; Klein et al. 2014; Pashler et al. 2012). This inability to reproduce empirical results leads, beyond the expenditure of time, energy and money, to a decline in confidence in science on the part of the public and scientists themselves (Anvari and Lakens 2018). The causes of this crisis are many, and a full review is beyond the scope of this article. Briefly, publication bias and the publish-or-die culture leading to dubious practices, or the misuse of statistics and in particular around the p-value are causes that have been stated many times (Romero 2019; Simmons et al. 2011). Beyond the researchers themselves, the social context of science seems to be at issue too. For example, until recently, only a small proportion of neuroscience journals accepted replication studies (Martin and Clarke 2017; Yeung 2017). Another cause mentioned by Romero (2019) in his review lies in the principle of priority, which rewards the first to obtain the innovative discovery (Merton 1957; Romero 2017). Finally, another possible cause of replication difficulties lies in the prevalence of biases inherent in the experimental procedure such as the effect of the experimenter, and in particular his or her gender/sex (Chapman et al. 2018).
1.2 Distinguishing gender/sex
It is generally accepted that there's a difference between gender and sex. However, these concepts are very often confused and associated, whether in a scientific context or not. Sex refers to the biological attributes (genitalia, chromosomes, hormones…) that distinguish male and female organisms, but also intersex and hermaphrodite organisms (Hyde et al. 2019). Gender is considered a social construct and refers to behaviors “associated with membership of a sexual category”, although they are not necessarily determined by sex (Hyde et al. 2019; West and Zimmerman 1987; Westbrook and Saperstein 2015, p. 537). Gender encompasses many different psychological and social traits that define individuals as man, woman, non-binary, trans, and to which the individual conforms to a greater or lesser extent (Garcia-Sifuentes and Maney 2021; Hyde et al. 2019; Swim et al. 2020; Tannenbaum et al. 2019). It’s important to note that during social interactions, a Person’s gender determination isn’t always aligned with how the person perceives themselves, and both can change over time (Westbrook and Saperstein 2015). When studying the scientific literature it seems impossible to determine which study actually measures gender and which actually measures sex and, more importantly, which effect can be attributed to gender and which to sex. Indeed, as most research on the experimenter’s gender/sex effect is relatively old, the concepts were not as differentiated or common as they are today, even if they are still largely confused today (Cameron and Stinson 2019). For example, Chapman et al. (2018), in a recent review on the subject, systematically refer to “gender” to describe this effect (the term “sex” does not appear), whereas many empirical studies cited in their review only refer to “sex”, with the term “gender” used interchangeably with “sex”, when it appears at all (Carter et al. 2002; Fisher 2007; Rumenik et al. 1977; Stevenson and Allen 1964). This is a very common phenomenon (Westbrook and Saperstein 2015). Knowing that the vast majority of these studies give little or no indication of how gender/sex is measured, we can imagine boxes to be ticked: “sex: male/female”, or “man/woman”, or even a choice left to the researcher’s discretion, with experimental instructions “only ask when in doubt” (Westbrook and Saperstein 2015). Of course, this leaves room for a high degree of ambiguity that makes it impossible to distinguish what is actually being assessed, particularly because sex and gender are assigned at the same time of birth (Hyde et al. 2019; Westbrook and Saperstein 2015). This is not only problematic from an epistemic point of view, but also from an ethical one (Cameron and Stinson 2019). Consequently, and as recommended by Hyde et al. (2019), in this study we use the term “gender/sex” not to consider them as undifferentiated but rather as indistinguishable (at least in this experiment) and to “reflect the inseparable nature of gender and sex in practical contexts and their confounded treatment and measurement in research” (Cameron and Stinson 2019; Hyde et al. 2019).
1.3 Effect of experimenter’s gender/sex
The experimental sciences are largely imbued with a quest for positivism and objectivism that focuses almost exclusively on the standardization of experimental conditions and materials. Within this framework, the experimenter is often seen as floating above the experiment and having no impact on what he or she observes, as evidenced by the vast majority of experimental studies that make no mention of the experimenter at all. Yet it is undeniable that during an experiment, at least two humans interact, and just because one of them is a scientist does not mean that this interaction is without social effect (Carter et al. 2002; Rumenik et al. 1977). Since scientists seem to have less difficulty replicating their own studies than those of their colleagues, it’s possible that the role of the investigator, and in particular his or her gender/sex, plays a part in the replicability crisis (Baker 2016). Chapman et al. (2018) note numerous areas in which such an effect can be observed: measures of intelligence (Back and Dana 1977), creativity (Gall and Mendelsohn 1967), learning and memory (Stevenson and Allen 1964), physical performance (Rikli 1976), risk-taking and testosterone (Ronay and Hippel 2010), aggressive and prosocial behavior (Borden 1975; Hyde 2014), and of course sexuality (Clark 1952). One of the areas where this effect is most documented is pain assessment, where men seem to report less pain when the experimenter is a woman, and women more pain when assessed by men (Aslaksen et al. 2007; Carter et al. 2002; Kállai et al. 2004; Levine and De Simone 1991; Mogil 2012; Vigil et al. 2014). A possible theoretical explanation is that the stakes of an opposite gender/sex interaction are often higher than those of similar gender/sex interactions, as they may (for heterosexual individuals) lead to a sexual or marital relationship, or at least to the self-consolidation of a seductive potential (Chapman et al. 2018; Leary et al. 1994). The result would be an increase in psychosocial stress which, depending on its level, would positively or negatively influence the experience. In this sense, it has been shown that the stress of men tested by women is greater than that of men tested by men, and vice versa for women (Kirschbaum et al. 1993). Other explanations for this phenomenon could involve gender conformity mechanisms through self-categorization to social roles (Berke et al. 2012; Carli 1989; Pool et al. 2007; Terry et al. 2000; Terry and Hogg 1996). According to these points of view, it can be argued that the dual interaction triggers attention to the participant's and experimenter’s gender/sex. This would lead to more gendered behaviours, for example women reporting more pain to appear feminine and men reporting less pain to appear masculine (Pool et al. 2007). In addition, it is possible that during a short punctual meeting with a stranger, individuals adopt more gendered characteristics and attitudes in order to create a common interaction basis, as these attitudes are broadly known and shared (even by those who do not conform to them), and seem to be socially valued (Swim et al. 2020). This is somewhat reminiscent of the stereotype threat theory (Spencer et al. 2016), a theory in which individuals will conform to a stereotype about themselves when it is made salient. For example, Spencer et al. (1999) reported that women under-performed at a math test when the stereotype that women have poor mathematical abilities was made salient. In this sense, it is arguable that the perceived gender/sex of the experimenter could trigger stereotype salience (Drążkowski et al. 2017; McGlone and Aronson 2006). However, it is important to note that many studies involving stereotype threat effects were found to be difficult and / or impossible to replicate, notably the famous one cited previously (Flore et al. 2018; Flore and Wicherts 2015).
1.4 VR experimentation
The use of VR in research is developing rapidly, for example in the study of spatial cognition, neuropsychological disorders, memories or therapies, as well as perception and philosophies of mind. (Banakou et al. 2016, 2018; Cogné et al. 2017, 2018; Muratore et al. 2019; Riva et al. 2019). Indeed, VR has the advantage of combining “the best of both worlds” (Minderer et al. 2016), i.e. methodologically controlled ecological studies (Dawson and Marcotte 2017; Parsons 2015). However, VR is not a simple transposition of reality: immersion consists in superimposing the artificial information of the immersive system onto the natural information of the physical environment (Maneuvrier and Westermann 2022). This superimposition gives rise to the sense of presence, a feeling of “being there” (Heeter 1992; Sheridan 2016), which enables the participant to act as if they were actually present in the virtual environment presented to them. But immersion also triggers the emergence of negative effects, notably cybersickness (Rebenitsch and Owen 2016; Stanney et al. 2020a, b; Tian et al. 2022). Cybermalaise is a set of symptoms similar to motion sickness. It is often assumed to be caused by inconsistencies between different perceptual systems, particularly the visual system (Bos et al. 2008). An important methodological aspect is that a sense of presence is negatively associated with cybersickness (Maneuvrier et al. 2021; Weech et al. 2019), and that the former seems to have a positive impact on cognitive performance, whereas the latter would tend to decrease it (Maneuvrier et al. 2020). In addition, both cybersickness and sense of presence appear to be associated with video game experience, perhaps through a reduction in the use of visual cues or, at least, by making them more flexible (Howarth and Hodder 2008; Maneuvrier et al. 2021), or through and enhanced familiarity with ergonomics and affordances (Gibson 1977; Maneuvrier et al. 2022). The video game experience is therefore often measured in VR, although less commonly than the sense of presence or the state of flow. Flow is the famous “optimal” state of consciousness characterized by cognitive absorption, enjoyment, time distortion, forgetfulness of self and others, and generally brought about by concentration on a task related to perceived competence (Csikszentmihalyi 1990). Flow is considered to be (i) triggered by video games, and (ii) associated with the sense of presence (Cheng et al. 2014; Kang et al. 2022; Kim and Ko 2019; Rutrecht et al. 2021; Spreij et al. 2022; Wehden et al. 2021; Yang and Zhang 2022). All these measures are usually assessed in the form of self-reported questionnaires after immersion (Schwind et al. 2019; Souza et al. 2021), especially as they are suspected to have an impact on performance during VR experimentation (Maneuvrier et al. 2020). However, there is one final factor that needs to be discussed because it is repeatedly described as having an impact on the VR experience: the participant's gender/sex.
1.5 Effect of participant’s gender/sex in VR
Some authors have found that men report a greater sense of presence than women in VR (Felnhofer et al. 2012; Lachlan and Krcmar 2011; Nicovich et al. 2005), even if this effect is not systematic (Maneuvrier et al. 2020; Weech et al. 2020). Furthermore, women have long been considered more susceptible to cybersickness than men (Maneuvrier et al. 2020; Shafer et al. 2017; Stanney et al. 2003), although again this effect is not systematic (Gamito et al. 2008; Ling et al. 2013). In this context, sex- and gender-related indistinguishable effects are suggested. For example, the sense of presence and cybersickness symptoms in VR are influenced by the experience of 3D and video games, probably due to shared spatial and perceptual processes known as field (in)dependence (Boccia et al. 2016; de Castell et al. 2019; Kennedy 1975; Maneuvrier et al. 2021, 2022, 2023; Wirth et al. 2012; Witkin 1949). Now, it can be argued that women are less inclined to play video games or 3D construction games, particularly during their youth (Entertainment Software Association 2022; Evans et al. 2013; Levine et al. 2016). But it’s impossible to tell whether women play less (sensory-intensive) video games because of social factors (Bègue et al. 2017), or because they're less comfortable with sensory mismatch, for biological reasons. Indeed, some authors have highlighted possible sex differences in cybersickness sensitivity, for example because of differences in hormone secretion rates or field-of-view size (Clemes and Howarth 2005; LaViola 2000). On the other hand, Stanney et al. (2020a, b) found results supporting the existence of a gender effect, stating that the size of head-mounted screens is generally not adapted to women's interpupillary distance. Once this variable was controlled for, they found no difference in the level of cybersickness between men and women (Stanney et al. 2020a, b). In addition, women could face stereotype threat when faced with computers and games, deteriorating their experience by generating stress (Koch et al. 2008). For example, Drążkowski et al. (2017) found a stereotype threat effect on measures of field (in)dependence: men and women are often expected to show different field (in)dependence strategies (Onyekuru 2015), with men being considered less field-dependent than women. However, Drążkowski et al. (2017) suggested that these differences are caused by a stereotype threat triggered by the experimenter's gender/sex: when the experimenter was a woman they found similar levels of field (in)dependence between genders/sexes. Finally, another gender-related possibility has been suggested: men may under-report their symptoms of cybersickness with the social aim of appearing stronger (Harm et al. 2007; Rebenitsch and Owen 2016; Stanney et al. 1999), which is also interesting in our context, as very similar results of an experimenter effect have been found in the field of pain assessment (Carter et al. 2002; Kállai et al. 2004; Levine and De Simone 1991; Mogil 2012; Vigil et al. 2014). However, an interaction between the experimenter’s and the participant’s gender/sex has never, to our knowledge, been empirically explored in VR.
1.6 Hypotheses and objectives
Given the current replicability crisis, the importance of VR measures for methodologically correct virtual evaluations, the discussed effects of gender/sex on the experience of VR and the possible interaction with the experimenter’s gender/sex, we believe that an empirical exploration of these dynamics is mandatory. The aim of this study is therefore to explore whether the experimenter's gender/sex may have an effect on the self-reported variables typically administered during a VR experiment, and to assess whether this effect depends on and interacts with the participant’s gender/sex. We conducted a randomly controlled VR experiment using a virtual first-person shooter, which is usually considered a masculine-gendered type of game (Juul 2012; Yee 2017). At the end of the game, participants self-administered the most common VR measures (sense of presence, cybersickness, video game experience, flow). Although this study is largely exploratory, we use the null hypothesis testing method as well as induction based on statistical inferences. We have formulated two a priori undirected hypotheses:
H0
The experimenter's gender/sex does not influence self-reported measures of VR (sense of presence, cybersickness, video game experience, flow state).
H1
The experimenter's gender/sex influences self-reported measures of VR (sense of presence, cybersickness, video game experience, flow state). Moreover, these effects vary depending on the participant’s gender/sex.
2 Materials and methods
3 Participants
This experiment was approved by a local ethics committee (CICPPR—IMT Atlantique) and strictly followed the Declaration of Helsinki (World Medical Association 2013). An a priori power analysis (f2 = 0.2, p = 0.05) revealed that in order to obtain 80% power, 66 participants were needed (Pilllai V = 0.16, Critical F(4,61) = 2.52, Non-centrality parameter λ = 13.2). 75 young adults were then recruited locally (Author’s university, Western France) from public posters and gave their written consent. After screening for missing data (1) and outliers (4) using the objective interquartile range (IQR) on each self-reported variable, i.e. [25th percentile]—1.5 × IQR and [75th percentile] + 1.5 × IQR, 70 participants were considered for analyses. Average age was 23.9 (± 4.2), minimum age was 18 and maximum 35. Exclusions criteria were: (i) being younger than 18 year old or older than 35 year old, (ii) having a known uncorrected psychological or physiological conditions which could alter perception or the use of a visuo-manual controller. Participants were not medically screened for those criteria but trusted by the experimenters, and individuals who could use corrective lenses or glasses in the head-mounted display were included. The age exclusion criteria was chosen because it is suggested that age has an effect on cybersickness (Arns and Cerney 2005; Petri et al. 2020; Stanney et al. 2020a, b), video game experience (Entertainment Software Association 2022; Greenberg et al. 2010), and, to some extent, sense of presence (Coxon et al. 2016; Ochs et al. 2018).
3.1 Gender/sex
Due to the “inseparable nature of gender and sex in practical contexts and their confounded treatment and measurement in research” (Cameron and Stinson 2019), the gender/sex variable was measured unitary using a non-mandatory open-ended question: “What sex were you assigned at birth?”. The choice of this formulation (“sex assigned at birth” rather than, for example, “current gender identification”) was intended to simplify the evaluation of the effects of gender and sex in an indistinguishable way. Indeed, it appeared that taking into account “current gender identification” rather than gender/sex assigned at birth would require much more in-depth analyses (use of a continuum and/or different categories, evolution over time, etc.), would make the issue more complex and would not allow sex to be taken into account correctly. The current measure (the sex, and therefore the gender socially assigned at birth) allows us to take into account (i) the sex factor and (ii) the associated gender, at least in part through the social relationships imposed during childhood. Indeed, a 23-year-old responding “non-binary” to a gender question at the time of the study is likely to have been socially considered as a man or a woman for much of his or her life, depending on the sex assigned at birth (and probably still partly assigned by some social agents). Although the measurement question explicitly refers to sex, we do not defend the idea that any effect described using this variable can be considered as attributed to sex or gender, which is why we use the unitary variable “gender/sex”. Even if we participate in their confounding treatment, we consider that the differentiation between the two is well beyond the scope and capabilities of this study. The question specified that the participant was not obliged to answer the question if they did not wish to. One participant did not answer the question, and was therefore excluded from analysis for missing data. Indeed, in view of the quantitative statistical method, it was established beforehand that responses that could not be grouped into the two most widespread gender categories (man/woman) based on the open-answers would not be considered for quantitative analyses (participants were not aware of this). With the exception of this participant, all the others participants answered the gender/sex variable in a dichotomous way, either “homme” in French, translated to “man” or “masculin”, translated to “masculine” which were grouped as “men”, because none responded “mâle”, the French near equivalent to “male” (N = 27). The other participants responded with “femme” in French, translated to “women” or “féminin” translated to “feminine”, which were grouped as “women”, because none responded “femelle”, the French near equivalent of “female” (N = 42). For the sake of clarity and consistency in reporting, gender/sex modalities are indicated as “men/women”.
3.2 Experimenters
In order to counterbalance a possible co-founding effect of other individual variables (height, attractiveness, sociability…) rather than the experimenter’s gender/sex variable per se, several experimenters (10) participated in this study. The experimental protocol was highly automated, since it took place mainly in VR and/or on computer software, and the experimenters had little direct evaluation work apart from human interaction and software management. With the exception of the principal investigator, all experimenters were graduate students, and all agreed to have their data used for this experiment. Using the same gender/sex variable, half of the experimenters were grouped as “men” (5) and the other half as “women’ (5). Participants and experimenters were randomly assigned. Women experimenters collected data from 37 participants (24 women, 13 men), while men collected data from 32 participants (18 women, 14 men). For the first two experiments of each experimenter, the principal investigator (man) was present in the room to (i) reassure the student experimenters about the technical aspects of the procedure and (ii) monitor the similar conduct of the experiment by all experimenters. This experimenter was silently working on a computer and did not interact directly with the participant. However, given that a passive presence can also potentially affect an experiment, and although this presence was equally distributed between the experimenters according to the gender/sex variable, a potential effect of this passive presence will be tested statistically.
3.3 VR immersion
The VR system was the HTC-Vive Pro (1440 × 1600 resolution per eye, 98° horizontal field of view, 90 Hz refresh rate). The computer used Windows 10–64-bit as its operating system, the processor (CPU) was the Intel Core i9—9900 K 3.6 GHz and it was equipped with 32 gigabytes of RAM. The graphics processing unit (GPU) was the GeForce RX 2080. Due to a mechanical issue with the head-mounted display, we were unable to adapt the interpupillary distance to each participant. However, we did set the interpupillary distance at 62.5 mm, which corresponds to the average distance of Northern European men and women aged 16–40 according to Pointer (1999), and is therefore equally suitable (or not) for men and women, in contrast to the basic distance of the head-mounted display (67.5 mm). After a brief tutorial explaining how to use the joysticks, participants were immersed in a Western-style cartoon world (Fig. 1, part A, part B). They stood on the roof of a moving train in order to obtain a smooth, linear visual flow likely to trigger very slight cybersickness (preventing a basement effect) thanks to a slight sensory mismatch (Clifton and Palmisano 2019). The train trajectory was entirely straight to avoid too much cybersickness, and the train speed was rather slow, 0.5 units of distance per second (slow walking speed). The tracked virtual controller was transformed into a gun capable of firing one bullet per second (Fig. 1, part C). Instructions were given visually and orally by voice recording (human voice) in the virtual environment. Participants had to shoot aliens (Fig. 1, part D) who appeared to attack the train en route. The aliens appeared regularly (every 10 s) in the virtual environment and fired projectiles at the train. Participants could either shoot at the projectiles to destroy them, or shoot at the aliens to neutralize them, their only explicit objective being to protect the train. Total immersion lasted 13 min and 30 s, after which the virtual environment closed. An action shooter has been chosen because it’s one of the game genres for which the greatest gender differences exist (Bosser and Nakatsu 2006; Feng et al. 2016; Kapalo et al. 2015), which could potentially make it easier to detect differences between the genders/sexes. This environment was custom-built by the author using Unity3D and the object-oriented programming language C#. The decision not to use a full avatar, but only to represent the tracked hand and virtual gun, was taken to avoid (i) uncanny valley (Mori et al. 2012), (ii) a mismatch between virtual and real poses (Palmisano et al. 2020), given the absence of trackers on the rest of the body, and (iii) embodiment effects (Banakou et al. 2018; Tassinari et al. 2022). A video of the first-person game is available in the supplementary material.
3.4 VR measures
All the variables commonly measured in VR experiments were assessed using a computerized self-administered questionnaire, as is generally done (Grassini and Laumann 2020; Schwind et al. 2019). Sense of presence was measured using the most common questionnaire validated in French without the haptics items (Robillard et al. 2002; Witmer and Singer 1998), which contains 22 items (7 points). Because cybersickness in VR is considered to be visually induced (Bos et al. 2008; Rebenitsch and Owen 2016), cybersickness was measured using the oculomotor scale of the simulator sickness questionnaire validated in French (Bouchard et al. 2007; Kennedy et al. 1993), which contains 7 items (4 points). Video game experience was measured using a single item question “How often do you play video games?” on 10 points, where 10 was specified “Everyday” and 1 was specified “Never”. A 10-point scale was used instead of the standard 7-point Likert scale to avoid possible confusion with the number of days played per week. State of flow was assessed using a translation of various flow questionnaire items used in VR but translated into French as no validated version could be found. The items used for the flow questionnaire are given in the supplementary material.
3.5 Procedure
Participants were clearly informed of the experimental protocol, with the exception of the evaluation of the experimenter's effect. However, the experimenters all knew that the experimenter's effect would be evaluated (for pedagogical reasons). Participants were informed that they could stop the experiment at any time and without explanation, but none chose to do so. Once written consent obtained, participants were fitted with sanitary pads and the head-mounted display with a tracked controller, and told that recordings would be given in the virtual environment. To collect exploratory educational data, a virtual rod-and-frame-test (Witkin 1949; Witkin et al. 1962) was performed in the head-mounted display (fully automated, 16 trials, approx. 3 min). Next, participants completed a 13.30-min immersion in VR. Once the virtual environment was complete, participants were de-equipped with the equipment and asked by the experimenter to complete the VR measures (sense of presence, cybersickness, video game experience, state of flow) on a computer. Next, participants were invited to perform the virtual rod-and-frame-test again (same configuration). Finally, participants were thanked, the experimenter explained the nature of the research and they were invited to ask questions. They then left the laboratory room, accompanied by the experimenter.
3.6 Analyses
-
Preliminary analyses: to ensure that VR measures were, where appropriate, reliable, Macdonald’s Omega (confirmatory factor analysis estimate, analytical interval) was used.
-
Null Hypothesis testing: one MANOVA with Pillai Test was performed to test H0. The VR measures (sense of presence, cybersickkness, video game experience and flow) were used as dependent variables, and the experimenter’s gender/sex and the participant’s gender/sex were used as independent variables. In order to assess a potential effect of the passive presence of the primary interviewer, the passive presence of the primary interviewer (yes/no) was added as an independent variable. All the interactions were considered at the exception of third-degree interactions, because of the sample size. ANOVAs tables (2 × 2 between participants) were reported, along with Tukey adjusted post-hoc tests when a potential difference was outlined, and simple main effects were explored.
-
Global statistical method: JASP software was used for statistical analysis and G*Power software was used for a priori power analysis. 95% Confidence Intervals (95% CI) were systematically reported, along with p-values. Because of experimental results previously found in the literature against the null hypothesis and the a priori power analysis (p = 0.05), only p-values around p = 0.05 and below were discussed. However, and because of the exploratory nature of the study, effects were not discussed as significant or non-significant based on p-values alone, as suggested by recent epistemological debates (Amrhein et al. 2019; Wasserstein et al. 2019). Box’s M-test for homogeneity of covariance matrices and Shapiro–Wilk test for multivariate normality were used to test the statistical assumptions of the MANOVA along with a Q-Q plot exploration of the normality of residuals. Cohen’s d was used to report effect size of Post Hoc comparisons along with corrected Tuckey p-value. Pillai Trace (V) was used to report effect size of MANOVA, whereas η2 was used to report effect size of analyses of variance.
4 Results
4.1 Preliminary results
The Presence Questionnaire items showed acceptable reliability and were therefore considered a construct representing the sense of presence variable: mean = 108.5, SD = 11.7, ω = 0.82, 95% CI [0.75;0.89]. Oculomotor symptoms items from the Simulator Sickness Questionnaire showed acceptable reliability and were therefore considered a construct representing the cybersickness variable: mean = 3.04, SD = 2.7, ω = 0.75, 95% CI [0.66;0.84]. The state of flow questionnaire items showed acceptable reliability and were therefore considered as a construct representing the flow variable: mean = 66.6, SD = 7.99, ω = 0.74, 95% CI [0.65;0.82]. Since it is a single-item question, scores on the video game practice question were considered a construct representing the video game experience variable: mean = 4.81, SD = 3.04. In addition, all the statistical assumptions hypotheses tested for the MANOVAs were satisfied.
4.2 Null hypothesis testing
The MANOVA revealed a debatable but unlikely global effect of the gender/sex of the participant along with a very potential interaction effect between gender/sex of participant and gender/sex of experimenter (Table 1). Further explorations revealed four possible univariate effects on different VR variables (Table 2). In addition, it seems very unlikely that the passive presence of the main investigator had an effect on the outcome variables.
-
Sense of presence: a potential interaction between the experimenter’s gender/sex and the participant’s gender/sex was outlined (Table 2, part A), but no main simple effects or post-hoc comparisons (Fig. 2, part A).
-
Cybersickness: a potential interaction effect between the experimenter’s gender/sex of and the participant’s gender/sex was outlined (Table 2, part B). Only one post-hoc adjusted comparison was outlined, as women evaluated by men seemed to report more cybersickness than men evaluated by men (t = 2.6, Cohen’s d = 0.944, 95% CI[0.062,1.95], pTukey = 0.052) while women and men evaluated by women reported similar levels. In addition, a simple main effect of participant’s gender/sex was outlined with a moderating effect of experimenter’s gender/sex: F = 6.85, p = 0.01 (Fig. 2, Part B).
-
Video game experience: a potential main effect of the participant’s gender/sex was outlined (Table 2, part C). Post-hoc one-to-one adjusted comparisons outlined that men reported more video game experience than women (t = − 2.97, Cohen’s d = − 0.73, 95% CI[− 1.24, − 0.22], pTukey = 0.004). In addition, an interaction between the experimenter’s gender/sex and the participant’s gender/sex was outlined: F(1,66) = 4.62, p = 0.035, η2 = 0.057. Post-hoc comparisons outlined that men evaluated by men reported more video game experience than women evaluated by men (t = − 3.51, Cohen’s d = − 1.26, 95% CI[− 2.29, − 0.24], pTukey = 0.004), whereas men and women evaluated by women reported similar levels of video game experience. In addition, men evaluated by women reported more video game experience than women evaluated by men (t = 2.66, Cohen’s d = 0.98, 95% CI[− 0.048, 2], pTukey = 0.047). A last effect could be discussed as women evaluated by women seemed to report more video game experience than women evaluated by men (t = 2.48, Cohen’s d = 0.77, 95% [− 0.09, 1.64], pTukey = 0.07). Finally, a simple main effect of participant’s gender/sex was outlined: F = 12.37, p < 0.001 (Fig. 2, Part C).
-
Flow: no differences were outlined on the state of flow measure (Table 2, part D). Men and women reported similar levels of flow, regardless of whether the experimenter was a man or a woman and the interaction between the gender/sex dyad (Fig. 2, Part D).
5 Discussion
The aim of this exploratory study was to assess whether experimenter and participant gender/sex may have an effect on self-reported variables commonly measured in VR (sense of presence, cybersickness, video game experience, flow), and to evaluate a potential interaction between the two. In view of previous findings in the literature and the empirical results of this study, we chose to reject H0. Indeed, although these results suggest (i) an effect of the participant’s gender/sex, that we consider solely due to differences in video game experience, and (ii) an absence of effect of the experimenter’s gender/sex, we defend that these two variables interact to impact self-reported measures. In addition, this interaction effect seems to differ according to the variables measured: it seems to be prevalent for the video game experience and cybersickness, but very weak or non-existent for the sense of presence, and very probably non-existent for the state of flow. In line with Goodman’s (2018) suggestion, and based on previous results, a priori power-analysis, sample size and sample distribution, confidence intervals, p-values and effect sizes, we emit an 80–85% chance of verisimilitude for these exploratory results. We now turn to discuss these results and the alternative hypothesis.
5.1 Main effect of the participant gender/sex
Although an overall main effect of the participant’s gender/sex was highlighted, further comparisons revealed that this was only true for video game experience, with men reporting more video game experience than women. This fact has already been documented (Behm-Morawitz 2013; Breuer et al. 2015; de Castell et al. 2019; Entertainment Software Association 2022; Maneuvrier et al. 2022), although the differences are much greater when assessed according to the type of games played (Yee 2017). Nevertheless, gender/sex differences in video game experience, and particularly in personal use of virtual reality, should not be overlooked. Indeed, these differences could be predictors of both the sense of presence and susceptibility to cybersickness, which in turn could influence performance in VR (Clemes and Howarth 2005; De Leo et al. 2014; Gamito et al. 2010; Knight and Arns 2006; Lachlan and Krcmar 2011; Maneuvrier et al. 2020; Weech et al. 2020). On the contrary, there seems to be no or slittle evidence of differences between men and women in the experience of flow (Boyd et al. 2018; Plummer et al. 2017). Yet the present study finds none, as asserted by (Csikszentmihalyi 1990), although not detecting an effect doesn’t mean it doesn’t exist. With the same precautions, and while a few studies have suggested a gender/sex effect in the way VR users experience the sense of presence (Felnhofer et al. 2012), the absence of gender/sex differences here is consistent with recent empirical work and is rather reassuring given its potential impact on performance (Maneuvrier et al. 2020). In contrast, debates about whether women experience more cybersickness than men in VR have been going on for decades (Stanney et al. 2003, 1999; Stanney et al. 2020a, b; Stanney et al. 2020a, b) and recent empirical work has also highlighted differences between genders/sexes when it comes to cybersickness (Maneuvrier et al. 2020). In view of our experimental results, we can note that in the present study men and women reported similar levels of cybersickness… only because not all women were evaluated by men. Indeed, if some gender/sex differences in cybersickness could be partially explained by modulating human factors (Maneuvrier and Westermann 2022; Stanney et al. 2020a, b), results of the present study allow us to outline a moderating effect of the experimenter's gender/sex. However, given that the experimenter's gender/sex is rarely indicated in experiment reports, it is impossible to know whether previous studies showing a gender/sex effect were conducted by men or women (or whomever). Still, we can note that in Maneuvrier et al. (2020), an empirical study where differences were found, the only experimenter was a man, which could have triggered an interaction with the participant's gender/sex (as is the case in half the cases of the present study).
5.2 Interaction between the experimenter’s and participant’s gender/sex
Indeed, men and women appear to differ in their symptoms of cybersickness when rated by men, but not when rated by women. An important question is whether these differences stem from a bias in self-reporting of negative symptoms, or whether the differences really exist on a psychophysiological basis. Of course, the two possibilities are not mutually exclusive, and this study cannot answer that question, as other physiological or behavioral measures of negative symptoms are needed (Dennison et al. 2016; Kim et al. 2005; Nesbitt et al. 2017). However, the fact that a very similar interaction effect was found on the video game variable, which in itself has no reason to change because of the experimenter’s gender/sex, might prompt us to hypothesize and discuss an effect of self-report measures. Opposite gender/sex dynamics seem to be at play here, and we think that studies on pain analysis, which is also a negative psychophysiological phenomenon, could be of interest. According to these views, women report more pain when assessed by men in order to appear more feminine and/or to trigger protective behaviors (Levine and De Simone 1991), while men report fewer negative symptoms when assessed by women in order to appear stronger and more masculine (Aslaksen et al. 2007; Carter et al. 2002; Kállai et al. 2004; Mogil 2012; Vigil et al. 2014). However, in our study, men seemed to report more cybersickness when assessed by women. Using the psychosocial stress interpretation, it could be argued that both women and men confused their psychosocial stress caused by the opposite gender/sex interaction with the symptoms of cybersickness, reporting more negative symptoms when evaluated by the opposite gender/sex. This interpretation based on psychosocial stress theory seems sound, but it cannot explain why a similar effect was evidenced for video game experience, which is not a physiological measure. However, it seems possible to explain these interaction effects by taking into account the dynamics of gender conformity and/or stereotype threat. Indeed, VR and video games are activities socially constructed by and for men (Kuittinen et al. 2007; Paaßen et al. 2017). Firstly, because men and women play very different types of games, with men playing more “intensive games” and “action games” like the one used in the study, and women playing “casual games” (Baniqued et al. 2013; Bosser and Nakatsu 2006; Juul 2012; Kapalo et al. 2015; Kuittinen et al. 2007; Rehbein et al. 2016; Saputra et al. 2017; Yee 2017). Secondly, because video games more than often target (young) men by representing gendered behaviors and stereotypes (Beasley and Collins Standley 2002; Behm-Morawitz 2013; Dietz 1998; Dill and Thill 2007) and by under-representing female characters or, when they do, treating them as sexual objects (Behm-Morawitz 2013; Dill and Thill 2007; Dunlop 2007; Gestos et al. 2018). Furthermore, women playing online video games are often victims of bullying or harassment (Breuer et al. 2015; Kowert et al. 2017; Kowert and Quandt 2017), which has led researchers to consider an association between video games and sexism (Bègue et al. 2017; Fox and Potocki 2016; Kowert et al. 2017). From this, it is possible to consider that women participants, when interacting with a man-investigator, are confronted with the stereotype/social norm that women are not gamers (Paaßen et al. 2017), which is amplified by the “first person shooter” style of the virtual environment (Yee 2017). Women would therefore report less video game experience when evaluated by a man than they would with a woman, in order to appear more feminine and/or less deviant from gender norms. On the contrary, men would report slightly more video game experience in front of another man in order to show conformity to masculine-gender norms, which is generally socially appreciated (Swim et al. 2020), but not in the presence of women in order not to appear sexist and/or show pseudo-virtual addictive behaviors that are much more widespread among men (Fisher 1994; King et al. 2020; Park and Hwang 2009; Van Rooij et al. 2011). It should be noted here that our empirical results seem to show that men are not, or at least much less, sensitive to the experimenter's effect than women. It is therefore important to bear in mind that the differences statistically observed may well be mostly due to an induced effect in women. Similar arguments could explain the differences reported in cybersickness, with women reporting more negative effects than men in order to show their maladjustment to a video game and technology reserved for men and/or to appear more sensitive to pain, which is socially considered feminine (Keogh and Denford 2009; Lloyd et al. 2020). Indeed, in Robinson et al. (2001) study, both men and women rated men as less willing to report pain than women, and women as more sensitive to pain and less able to bear it than men. Finally, similar interpretations could be used to explain the possible (weak) interaction effect on the sense of presence: women would report a slightly lower sense of presence when evaluated by a man in order to remain consistent with a “non-video game” gender norm, especially since they were faced with a typically masculine western-style shooting game. However, the fact that the interaction effect is less important for sense of presence than for cybersickness or video game experience is interesting and could confirm our interpretation. Indeed, and although the sense of presence is not a commonly gendered concept (as the concept is mostly socially unknown, although continuously experienced), the questionnaire that we used places a strong emphasis on technical processes, with questions on “transmission delay” or “graphical display” (Robillard et al. 2002; Schwind et al. 2019; Witmer and Singer 1998), aspects that contain gender stereotypes (Clayton et al. 2009; Koch et al. 2008; López-Sáez et al. 2011; Meadows and Sekaquaptewa 2013; Spencer et al. 1999). In comparison, measures of the state of flow are entirely psychological in nature and are not linked to VR or video games and/or computers and/or gender norms (“the activity totally absorbed my attention”). Thus, no technical or video game stereotypes interfere, which could explain why we found no effect of the gender/sex of the experimenter on the flow measures reported by the participants.
5.3 Further explorations and theory evaluation
First of all, the practice of video games should probably not be considered as a single-factor variable based on frequency (Baniqued et al. 2013; Bosser and Nakatsu 2006; Juul 2012; Kapalo et al. 2015; Kuittinen et al. 2007; Maneuvrier and Westermann 2022; Rehbein et al. 2016; Yee 2017), although the aim here was to play on the associated global masculine connotation. Still, future studies assessing video games impact in VR should try to take into account the different types and media of video games, and in particular the use of recreational VR, which could prove decisive for the methodology of the tool. In addition, future studies investigating reports of cybersickness could learn by assessing susceptibility to motion sickness prior to immersion (Brown et al. 2022), even though this may have a suggestive effect. Furthermore, to find out whether the gender/sex experimenter effect lies in a bias in subjective self-reports or an alteration in phenomenology, particularly for cybersickness and sense of presence, other psycho-physiological and/or dynamic measures could be considered (Arcioni et al. 2019; Dennison et al. 2016; Grassini and Laumann 2020; Haydu et al. 2016; Kim et al. 2005; Maneuvrier et al. 2023; Nesbitt et al. 2017; Tian et al. 2022; Yang et al. 2022). In addition, and in order to test the two interpretive theories of the experimenter's gender/sex effect, namely psychosocial stress (Chapman et al. 2018) and gender conformity dynamics (Swim et al. 2020), two explorations are mandatory. First, participants’ gendered attitudes toward video games and VR and/or computers could be measured (e.g., to what extent they consider men to be better at video games than women) as well as their gender conformity attitudes (e.g., to what extent they personally conform to and value gender norms in others). In addition, stereotype threat theory could be used (either inducing stereotype activation or not) to test the interaction between experimenter and participant gender/sex, but the non-replicable results of studies supporting this theory cast doubt on its use (Flore et al. 2018; Flore and Wicherts 2015). Secondly, other variables modulating the stakes of social interaction could be manipulated (e.g. attractiveness, age, social status of the experimenter…) to test the psychosocial stress theory. In addition, researchers could play on the gendered identification of immersive schemes: does the environment resemble a male-oriented game (for example, a first-person action game) or a more neutral or feminine game? Finally, it could be very interesting to further investigate gender dynamics, for example by evaluating “actual identified gender” and “experimenter-perceived gender” on a continuum, or by evaluating similar effects with other gender/sex categories (intersex/non-binary, transgender…) and/or other sexual orientations. Indeed, the use of the term “gender/sex”, if it allows a form of humility and scientific precaution in the current framework which seems fundamental to us, will need to be pushed further at some point for an analysis less globalizing and more detailed of the phenomena at stake. The study of the experimenter’s effect and its interaction with the participant’s genre/sex is of course not sufficient to solve the reproducibility crisis, but it could contribute to it. Other (non-statistical) recommendations already discussed would be the pre-registration of empirical studies in scientific journals, as well as methodological review boards (Anvari and Lakens 2018; Lakens 2023; Munafò et al. 2017; Nosek et al. 2018), independent institutions dedicated to empirical replications and the global use of post-publication public peer-review (Ortega 2022; Townsend 2013).
5.4 Practical implications
The main recommendation to be drawn from these results is obviously to try as far as possible to alternate the gender/sex of experimenters, whether in VR studies or not. Indeed, if the data in the present study had been collected exclusively by a man, we might have concluded that there was a gender effect on participants’ cybersickness, with women reporting more negative symptoms than men. Instead, we argue here that this is an experimenter effect and that mend women do not differ intrinsically in their susceptibility to cybersickness. In the same vein, we also invite experimental researchers collecting human data to systematically describe, at least succinctly, the experimenters, including their gender/sex. Indeed, given the results of our study, it would be highly relevant to be able to know the gender/sex of experimenters who observed differences and those who did not, in order to carry out meta-analyses. In addition, we invite researchers to carry out control comparisons between men and women, but also, if possible, between experimenters. Furthermore, we defend the use of Hyde et al. (2019) “gender/sex” expression when their distinction in an empirical or theoretical study seems impossible, but also the use of open questions and, more generally, Cameron and Stinson’s (2019) guidelines for measuring gender/sex. Finally, we defend the “ditching” of statistical significance and the absolute p-value rule for exploratory studies, as we did in this study (Amrhein et al. 2019; Colquhoun 2017; Munafò et al. 2017; Simmons et al. 2011; Wasserstein et al. 2019).
5.5 Limitations
One of the main problems with this exploratory empirical study is the discrepancy between the number of men (27) and women (42) participants, which could bias the analysis in favor of the men's extreme values (even though the outliers have been objectively removed). It also alters the statistical power pre-calculated at 80%. This gender discrepancy may be explained by the difficulties encountered during the recruitment process due to the COVID-19 epidemic, as well as by the overall dominant representation of women in today’s social science French universities and, of course, by the financial and time resources available. In addition, the video game experience variable, which does not distinguish between types of video game or medium (particularly VR), may be a limit, even though this variable is a purely dependent variable used to measure the experimenter’s effect and not an explanatory variable. Similarly, the homemade translation of the flow state scale could necessitate psychometric consolidation before being re-used to other means. Another limitation is that, although we tested the potential effect of the passive presence of the principal investigator (equally and randomly distributed between experimenters and participants), it is still possible that this presence had a statistically undetected effect on the results. Similarly, although we did not find an effect of experimenter bias on flow measures, this does not prove that it does not exist at all. In addition, most of the participants and experimenters were young Western Europeans, and most of the participants were students. It should be noted that ethnicity was not taken into account for the simple reason that the use of ethnic and racial statistics is prohibited in France (Möschel 2009). Consequently, it would be risky to make inductive deductions from this sample about other populations. Finally, this exploratory study, like any exploratory study, requires confirmatory research (Munafò et al. 2017), which is often difficult given that virtual environments are generally not shared. For this reason, the author would like to make it clear that the virtual environment as well as the data used in this study will be shared freely upon request to the corresponding author for a reproducible and/or replicative and/or confirmatory study, with all the human support that will be possible. Without speculating on the replicability of the results of the present study, we consider that it is at least easily reproducible, particularly with the help of the automated tools used.
5.6 Conclusion
The main finding of this study is that an experimenter’s gender/sex interacts with a participant's gender/sex to influence the latter's responses on several self-reported measures of VR (cybersickness and video game experience, and to a lesser extent sense of presence): men and women differ when rated by men, but not by women. This result is discussed as an effect of psychosocial stress induced by interaction with the opposite gender/sex, as well as conformity to gender norms, particularly in women with regard to action VR video games strongly influenced by a masculine social construction. This interpretation also explains why this effect was absent from self-reported measures of flow, which were the only questions unrelated to VR and/or video games. This study invites researchers in the experimental sciences to better control for peri-experimental effects such as the effect of the experimenter, and in particular their gender/sex, in order to consolidate research in times of replicability and, to a greater extent, epistemological crisis.
Data availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request, as well as the virtual environments used.
References
Amrhein V, Greenland S, McShane B (2019) Scientists rise up against statistical significance. Nature 567(7748):305–307. https://doi.org/10.1038/d41586-019-00857-9
Anvari F, Lakens D (2018) The replicability crisis and public trust in psychological science. Compr Results Soc Psychol 3(3):266–286. https://doi.org/10.1080/23743603.2019.1684822
Arcioni B, Palmisano S, Apthorp D, Kim J (2019) Postural stability predicts the likelihood of cybersickness in active HMD-based virtual reality. Displays 58:3–11. https://doi.org/10.1016/j.displa.2018.07.001
Arns LL, Cerney MM (2005) The relationship between age and incidence of cybersickness among immersive environment users. IEEE Proc VR 2005 Virtual Real 2005:267–268. https://doi.org/10.1109/VR.2005.1492788
Aslaksen PM, Myrbakk IN, Høifødt RS, Flaten MA (2007) The effect of experimenter gender on autonomic and subjective responses to pain stimuli. Pain 129(3):260–268. https://doi.org/10.1016/j.pain.2006.10.011
Back R, Dana RH (1977) Examiner sex bias and Wechsler intelligence scale for children scores. J Consult Clin Psychol 45(3):500–500. https://doi.org/10.1037/0022-006X.45.3.500
Baker M (2015) Over half of psychology studies fail reproducibility test. Nature. https://doi.org/10.1038/nature.2015.18248
Baker M (2016) 1,500 scientists lift the lid on reproducibility. Nature 533(7604):7604. https://doi.org/10.1038/533452a
Banakou D, Hanumanthu PD, Slater M (2016) Virtual embodiment of white people in a black virtual body leads to a sustained reduction in their implicit racial bias. Front Hum Neurosci. https://doi.org/10.3389/fnhum.2016.00601
Banakou D, Kishore S, Slater M (2018) Virtually being einstein results in an improvement in cognitive task performance and a decrease in age bias. Front Psychol. https://doi.org/10.3389/fpsyg.2018.00917
Baniqued PL, Lee H, Voss MW, Basak C, Cosman JD, DeSouza S, Severson J, Salthouse TA, Kramer AF (2013) Selling points: what cognitive abilities are tapped by casual video games? Acta Physiol (oxf) 142(1):74–86. https://doi.org/10.1016/j.actpsy.2012.11.009
Bargh JA, Chen M, Burrows L (1996) Automaticity of social behavior: direct effects of trait construct and stereotype activation on action. J Pers Soc Psychol 71:230–244. https://doi.org/10.1037/0022-3514.71.2.230
Beasley B, Collins Standley T (2002) Shirts vs. skins: clothing as an indicator of gender role stereotyping in video games. Mass Commun Soc 5(3):279–293. https://doi.org/10.1207/S15327825MCS0503_3
Begley CG, Ellis LM (2012) Raise standards for preclinical cancer research. Nature 483(7391):7391. https://doi.org/10.1038/483531a
Bègue L, Sarda E, Gentile DA, Bry C, Roché S (2017) Video games exposure and sexism in a representative sample of adolescents. Front Psychol. https://doi.org/10.3389/fpsyg.2017.00466
Behm-Morawitz E (2013) Mirrored selves: the influence of self-presence in a virtual world on health, appearance, and well-being. Comput Hum Behav 29(1):119–128. https://doi.org/10.1016/j.chb.2012.07.023
Bergh DD, Sharp BM, Aguinis H, Li M (2017) Is there a credibility crisis in strategic management research? Evidence on the reproducibility of study findings. Strateg Organ 15(3):423–436. https://doi.org/10.1177/1476127017701076
Berke DS, Sloan CA, Parrott D, Zeichner A (2012) Effects of female gender role and gender norm conformity on aggression in men: does positive masculinity reduce the risk? Psychol Men Masc 13:367–378. https://doi.org/10.1037/a0026525
Boccia M, Piccardi L, Di Marco M, Pizzamiglio L, Guariglia C (2016) Does field independence predict visuo-spatial abilities underpinning human navigation? Behav Evid Exp Brain Res 234(10):2799–2807. https://doi.org/10.1007/s00221-016-4682-9
Borden RJ (1975) Witnessed aggression: influence of an observer’s sex and values on aggressive responding. J Pers Soc Psychol 31(3):567–573. https://doi.org/10.1037/h0076480
Bos JE, Bles W, Groen EL (2008) A theory on visually induced motion sickness. Displays 29:47–57. https://doi.org/10.1016/j.displa.2007.09.002
Bosser A-G, Nakatsu R (2006) Hardcore gamers and casual gamers playing online together. In: Proceedings of the 5th international conference on entertainment computing, pp 374–377. https://doi.org/10.1007/11872320_53
Bouchard S, Robillard G, Renaud P (2007) Revising the factor structure of the simulator sickness questionnaire. Acte De Colloque Du Ann Rev CyberTherapy Telemed 5:128–137
Boyd JM, Schary DP, Worthington AR, Jenny SE (2018) An examination of the differences in flow between individual and team athletes. Phys Cult Sport Stud Res 78(1):33–40. https://doi.org/10.2478/pcssr-2018-0011
Breuer J, Kowert R, Festl R, Quandt T (2015) Sexist games=sexist gamers? A longitudinal study on the relationship between video game use and sexist attitudes. Cyberpsychol Behav Soc Netw 18(4):197–202. https://doi.org/10.1089/cyber.2014.0492
Brown P, Spronck P, Powell W (2022) The simulator sickness questionnaire, and the erroneous zero baseline assumption. Front Virtual Real. https://doi.org/10.3389/frvir.2022.945800
Camerer CF, Dreber A, Holzmeister F, Ho T-H, Huber J, Johannesson M, Kirchler M, Nave G, Nosek BA, Pfeiffer T, Altmejd A, Buttrick N, Chan T, Chen Y, Forsell E, Gampa A, Heikensten E, Hummer L, Imai T, Wu H (2018) Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat Hum Behav 2(9):9. https://doi.org/10.1038/s41562-018-0399-z
Cameron JJ, Stinson DA (2019) Gender (mis)measurement: guidelines for respecting gender diversity in psychological research. Soc Pers Psychol Compass 13(11):e12506. https://doi.org/10.1111/spc3.12506
Carli LL (1989) Gender differences in interaction style and influence. J Pers Soc Psychol 56:565–576. https://doi.org/10.1037/0022-3514.56.4.565
Carter LE, McNeil DW, Vowles KE, Sorrell JT, Turk CL, Ries BJ, Hopko DR (2002) Effects of emotion on pain reports, tolerance and physiology. Pain Res Manag 7(1):21–30. https://doi.org/10.1155/2002/426193
Chapman CD, Benedict C, Schiöth HB (2018) Experimenter gender and replicability in science. Sci Adv 4(1):e1701427. https://doi.org/10.1126/sciadv.1701427
Cheng L-K, Chieng M-H, Chieng W-H (2014) Measuring virtual experience in a three-dimensional virtual reality interactive simulator environment: a structural equation modeling approach. Virtual Real 18(3):173–188. https://doi.org/10.1007/s10055-014-0244-2
Clark RA (1952) The projective measurement of experimentally induced levels of sexual motivation. J Exp Psychol 44(6):391–399. https://doi.org/10.1037/h0056822
Clayton KL, von Hellens LA, Nielsen SH (2009) Gender stereotypes prevail in ICT: a research review. In: Proceedings of the special interest group on management information system’s 47th annual conference on computer personnel research, pp 153–158. https://doi.org/10.1145/1542130.1542160
Clemes SA, Howarth PA (2005) The menstrual cycle and susceptibility to virtual simulation sickness. J Biol Rhythms 20(1):71–82. https://doi.org/10.1177/0748730404272567
Clifton J, Palmisano S (2019) Effects of steering locomotion and teleporting on cybersickness and presence in HMD-based virtual reality. Virtual Real. https://doi.org/10.1007/s10055-019-00407-8
Cogné M, Auriacombe S, Vasa L, Tison F, Klinger E, Sauzéon H, Joseph P-A, Kaoua NB (2018) Are visual cues helpful for virtual spatial navigation and spatial memory in patients with mild cognitive impairment or Alzheimer’s disease? Neuropsychology 32(4):385–400. https://doi.org/10.1037/neu0000435
Cogné M, Taillade M, N’Kaoua B, Tarruella A, Klinger E, Larrue F, Sauzéon H, Joseph P-A, Sorita E (2017) The contribution of virtual reality to the diagnosis of spatial navigation disorders and to the study of the role of navigational aids: a systematic literature review. Ann Phys Rehabil Med 60(3):164–176. https://doi.org/10.1016/j.rehab.2015.12.004
Colquhoun D (2017) The reproducibility of research and the misinterpretation of p-values. R Soc Open Sci 4(12):171085. https://doi.org/10.1098/rsos.171085
Coxon M, Kelly N, Page S (2016) Individual differences in virtual reality: are spatial presence and spatial ability linked? Virtual Real 20(4):203–212. https://doi.org/10.1007/s10055-016-0292-x
Csikszentmihalyi M (1990) Flow: the psychology of optimal experience, Book Club. Harper & Row, New York
Dawson DR, Marcotte TD (2017) Special issue on ecological validity and cognitive assessment. Neuropsychol Rehabil 27(5):599–602. https://doi.org/10.1080/09602011.2017.1313379
de Castell S, Larios H, Jenson J (2019) Gender, videogames and navigation in virtual space. Acta Physiol (oxf) 199:102895. https://doi.org/10.1016/j.actpsy.2019.102895
De Leo G, Diggs LA, Radici E, Mastaglio TW (2014) Measuring sense of presence and user characteristics to predict effective training in an online simulated virtual environment. Simul Healthc J Soc Simul Healthc 9(1):1–6. https://doi.org/10.1097/SIH.0b013e3182a99dd9
Dennison MS, Wisti AZ, D’Zmura M (2016) Use of physiological signals to predict cybersickness. Displays 44:42–52. https://doi.org/10.1016/j.displa.2016.07.002
Dietz TL (1998) An examination of violence and gender role portrayals in video games: implications for gender socialization and aggressive behavior. Sex Roles 38(5):425–442. https://doi.org/10.1023/A:1018709905920
Dill KE, Thill KP (2007) Video game characters and the socialization of gender roles: young people’s perceptions mirror sexist media depictions. Sex Roles 57(11):851–864. https://doi.org/10.1007/s11199-007-9278-1
Doyen S, Klein O, Pichon C-L, Cleeremans A (2012) Behavioral priming: it’s all in the mind, but whose mind? PLoS ONE 7(1):e29081. https://doi.org/10.1371/journal.pone.0029081
Drążkowski D, Szwedo J, Krajczewska A, Adamczuk A, Piątkowski K, Jadwiżyc M, Rakowski A (2017) Women are not less field independent than men-the role of stereotype threat. Int J Psychol J Int De Psychol 52(5):415–419. https://doi.org/10.1002/ijop.12238
Dunlop JC (2007) The U.S. video game industry: analyzing representation of gender and race. Int J Technol Hum Interact (IJTHI) 3(2):96–109. https://doi.org/10.4018/jthi.2007040106
Entertainment Software Association (2022) Essential facts about the computer and video game industry. https://www.theesa.com/resource/2022-essential-facts-about-the-video-game-industry/
Evans C, Richardson JTE, Waring M (2013) Field independence: reviewing the evidence. Br J Educ Psychol 83(Pt 2):210–224. https://doi.org/10.1111/bjep.12015
Felnhofer A, Kothgassner O, Beutl L, Hlavacs H, Kryspin-Exner I (2012) Is virtual reality made for men only? Exploring gender differences in the sense of presence. International Society for Presence Research Annual Conference—ISPR 2012. International Society for Presence Research Annual Conference—ISPR 2012, Philadelphia, Pennsylvania, USA. https://eprints.cs.univie.ac.at/3557/
Feng J, Spence I, Pratt J (2016) Playing an action video game reduces gender differences in spatial cognition. Psychol Sci. https://doi.org/10.1111/j.1467-9280.2007.01990.x
Fisher S (1994) Identifying video game addiction in children and adolescents. Addict Behav 19(5):545–553. https://doi.org/10.1016/0306-4603(94)90010-8
Fisher TD (2007) Sex of experimenter and social norm effects on reports of sexual behavior in young men and women. Arch Sex Behav 36(1):89–100. https://doi.org/10.1007/s10508-006-9094-7
Flore PC, Mulder J, Wicherts JM (2018) The influence of gender stereotype threat on mathematics test scores of Dutch high school students: a registered report. Compr Results Soc Psychol 3(2):140–174. https://doi.org/10.1080/23743603.2018.1559647
Flore PC, Wicherts JM (2015) Does stereotype threat influence performance of girls in stereotyped domains? A meta-analysis. J Sch Psychol 53(1):25–44. https://doi.org/10.1016/j.jsp.2014.10.002
Fox J, Potocki B (2016) Lifetime video game consumption, interpersonal aggression, hostile sexism, and rape myth acceptance: a cultivation perspective. J Interpers Violence 31(10):1912–1931. https://doi.org/10.1177/0886260515570747
Gall M, Mendelsohn GA (1967) Effects of facilitating techniques and subject-experimenter interaction on creative problem solving. J Pers Soc Psychol 5(2):211–216. https://doi.org/10.1037/h0024130
Gamito P, Oliveira J, Morais D, Baptista A, Santos N, Soares F, Saraiva T, Rosa P (2010) Training presence: the importance of virtual reality experience on the “sense of being there.” Stud Health Technol Inform 154:128–133
Gamito P, Oliveira J, Morais D, Santos N, Soares F (2008) Presence, immersion and cybersickness assessment through a test anxiety virtual environment. Ann Rev Cyberther Telemed 6:83–90
Garcia-Sifuentes Y, Maney DL (2021) Reporting and misreporting of sex differences in the biological sciences. Elife 10:e70817. https://doi.org/10.7554/eLife.70817
Gestos M, Smith-Merry J, Campbell A (2018) Representation of women in video games: a systematic review of literature in consideration of adult female wellbeing. Cyberpsychol Behav Soc Netw 21(9):535–541. https://doi.org/10.1089/cyber.2017.0376
Gibson J-J (1977) How perception really develops: a view from outside the network. In: Basic processes in reading: perception and comprehension. Erlbaum, pp 155–173
Goodman SN (2018) How sure are you of your result? Put a number on it. Nature 564(7734):7–7. https://doi.org/10.1038/d41586-018-07589-2
Grassini S, Laumann K (2020) Questionnaire measures and physiological correlates of presence: a systematic review. Front Psychol. https://doi.org/10.3389/fpsyg.2020.00349
Greenberg BS, Sherry J, Lachlan K, Lucas K, Holmstrom A (2010) Orientations to video games among gender and age groups. Simul Gaming 41(2):238–259. https://doi.org/10.1177/1046878108319930
Harm D, Taylor LC, Bloomberg J (2007) Adaptive changes in sensorimotor coordination and motion sickness following repeated exposures to virtual environments. https://www.semanticscholar.org/paper/Adaptive-Changes-in-Sensorimotor-Coordination-and-Harm-Taylor/e61f64a5c393518b4b63fab6629a0de2c6eb7f13
Harris CR, Coburn N, Rohrer D, Pashler H (2013) Two failures to replicate high-performance-goal priming effects. PLoS ONE 8(8):e72467. https://doi.org/10.1371/journal.pone.0072467
Haydu VB, Jacyntho Zacarin MR, de Souza Domingos AP, Borloti E (2016) Behavioral measures of presence in virtual environments. Revista Ces Psicologia 9(1):47–64
Heeter C (1992) Being there: the subjective experience of presence. Presence Teleoper Virtual Environ 1(2):262–271. https://doi.org/10.1162/pres.1992.1.2.262
Howarth PA, Hodder SG (2008) Characteristics of habituation to motion in a virtual environment. Displays 29(2):117–123. https://doi.org/10.1016/j.displa.2007.09.009
Hyde JS (2014) Gender similarities and differences. Annu Rev Psychol 65(1):373–398. https://doi.org/10.1146/annurev-psych-010213-115057
Hyde JS, Bigler RS, Joel D, Tate CC, van Anders SM (2019) The future of sex and gender in psychology: five challenges to the gender binary. Am Psychol 74(2):171–193. https://doi.org/10.1037/amp0000307
Juul J (2012) A casual revolution: reinventing video games and their players. The MIT Press, Cambridge
Kállai I, Barke A, Voss U (2004) The effects of experimenter characteristics on pain reports in women and men. Pain 112(1):142–147. https://doi.org/10.1016/j.pain.2004.08.008
Kang YG, Song H-D, Yun H, Jo Y (2022) The effect of virtual reality media characteristics on flow and learning transfer in job training: the moderating effect of presence. J Comput Assist Learn 38(6):1674–1685. https://doi.org/10.1111/jcal.12702
Kapalo KA, Dewar AR, Rupp MA, Szalma JL (2015) Individual differences in video gaming: defining hardcore video gamers. Proc Hum Factors Ergon Soc Annu Meet 59(1):878–881. https://doi.org/10.1177/1541931215591261
Kennedy RS (1975) Motion sickness questionnaire and field independence scores as predictors of success in naval aviation training. Aviat Space Environ Med 46(11):1349–1352
Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG (1993) Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol 3(3):203–220. https://doi.org/10.1207/s15327108ijap0303_3
Keogh E, Denford S (2009) Sex differences in perceptions of pain coping strategy usage. Eur J Pain 13(6):629–634. https://doi.org/10.1016/j.ejpain.2008.07.002
Kim D, Ko YJ (2019) The impact of virtual reality (VR) technology on sport spectators’ flow experience and satisfaction. Comput Hum Behav 93:346–356. https://doi.org/10.1016/j.chb.2018.12.040
Kim YY, Kim HJ, Kim EN, Ko HD, Kim HT (2005) Characteristic changes in the physiological components of cybersickness. Psychophysiology 42(5):616–625. https://doi.org/10.1111/j.1469-8986.2005.00349.x
King DL, Delfabbro PH, Billieux J, Potenza MN (2020) Problematic online gaming and the COVID-19 pandemic. J Behav Addict 9(2):184–186. https://doi.org/10.1556/2006.2020.00016
Kirschbaum C, Pirke KM, Hellhammer DH (1993) The ’Trier Social Stress Test’—A tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology 28(1–2):76–81. https://doi.org/10.1159/000119004
Klein RA, Ratliff KA, Vianello M, Adams RB, Bahník Š, Bernstein MJ, Bocian K, Brandt MJ, Brooks B, Brumbaugh CC, Cemalcilar Z, Chandler J, Cheong W, Davis WE, Devos T, Eisner M, Frankowska N, Furrow D, Galliani EM, Nosek BA (2014) Investigating variation in replicability. Soc Psychol 45(3):142–152. https://doi.org/10.1027/1864-9335/a000178
Knight MM, Arns LL (2006) The relationship among age and other factors on incidence of cybersickness in immersive environment users. ACM SIGGRAPH 2006 Res Posters. https://doi.org/10.1145/1179622.1179846
Koch SC, Müller SM, Sieverding M (2008) Women and computers. Effects of stereotype threat on attribution of failure. Comput Educ 51(4):1795–1803. https://doi.org/10.1016/j.compedu.2008.05.007
Kowert R, Breuer J, Quandt T (2017) The cycle of exclusion and sexism in video game content and culture. In: New perspectives on the social aspects of digital gaming. Routledge.
Kowert R, Quandt T (eds) (2017) New perspectives on the social aspects of digital gaming: multiplayer 2, 1st edn. Routledge, London
Kuittinen J, Kultima A, Niemelä J, Paavilainen J (2007) Casual games discussion. In: Proceedings of the 2007 conference on future play, pp 105–112. https://doi.org/10.1145/1328202.1328221
Lachlan K, Krcmar M (2011) Experiencing presence in video games: the role of presence tendencies, game experience, gender, and time spent in play. Commun Res Rep 28(1):27–31. https://doi.org/10.1080/08824096.2010.518924
Lakens D (2023) Is my study useless? Why researchers need methodological review boards. Nature 613(7942):9–9. https://doi.org/10.1038/d41586-022-04504-8
LaViola JJ (2000) A discussion of cybersickness in virtual environments. ACM SIGCHI Bull 32(1):47–56. https://doi.org/10.1145/333329.333344
Leary MR, Nezlek JB, Downs D, Radford-Davenport J, Martin J, McMullen A (1994) Self-presentation in everyday interactions: effects of target familiarity and gender composition. J Pers Soc Psychol 67(4):664–673. https://doi.org/10.1037/0022-3514.67.4.664
Levine FM, De Simone LL (1991) The effects of experimenter gender on pain report in male and female subjects. Pain 44(1):69–72. https://doi.org/10.1016/0304-3959(91)90149-R
Levine SC, Foley A, Lourenco S, Ehrlich S, Ratliff K (2016) Sex differences in spatial cognition: advancing the conversation. Wires Cognit Sci 7(2):127–155. https://doi.org/10.1002/wcs.1380
Ling Y, Nefs HT, Brinkman W-P, Qu C, Heynderickx I (2013) The relationship between individual characteristics and experienced presence. Comput Hum Behav 29(4):1519–1530. https://doi.org/10.1016/j.chb.2012.12.010
Lloyd EP, Paganini GA, ten Brinke L (2020) Gender stereotypes explain disparities in pain care and inform equitable policies. Policy Insights Behav Brain Sci 7(2):198–204. https://doi.org/10.1177/2372732220942894
López-Sáez M, Puertas S, Sáinz M (2011) Why don’t girls choose technological studies? Adolescents’ stereotypes and attitudes towards studies related to medicine or engineering. Span J Psychol 14(1):74–87. https://doi.org/10.5209/rev_SJOP.2011.v14.n1.6
Maneuvrier A, Ceyte H, Renaud P, Morello R, Fleury P, Decker LM (2022) Virtual reality and neuropsychological assessment: an analysis of human factors influencing performance and perceived mental effort. Virtual Real. https://doi.org/10.1007/s10055-022-00698-4
Maneuvrier A, Decker LM, Ceyte H, Fleury P, Renaud P (2020) Presence promotes performance on a virtual spatial cognition task: impact of human factors on virtual reality assessment. Front Virtual Real. https://doi.org/10.3389/frvir.2020.571713
Maneuvrier A, Decker LM, Renaud P, Ceyte G, Ceyte H (2021) Field (In)dependence flexibility following a virtual immersion is associated with cybersickness and sense of presence. Front Virtual Real 2:110. https://doi.org/10.3389/frvir.2021.706712
Maneuvrier A, Nguyen N-D-T, Renaud P (2023) Predicting VR cybersickness and its impact on visuomotor performance using head rotations and field (in)dependence. Front Virtual Real. https://doi.org/10.3389/frvir.2023.1307925
Maneuvrier A, Westermann H (2022) The phi angle: a theoretical essay on sense of presence, human factors, and performance in virtual reality. PRESENCE Virtual Augment Real. https://doi.org/10.1162/pres_a_00359
Martin GN, Clarke RM (2017) Are psychology journals anti-replication? A snapshot of editorial practices. Front Psychol. https://doi.org/10.3389/fpsyg.2017.00523
McGlone MS, Aronson J (2006) Stereotype threat, identity salience, and spatial reasoning. J Appl Dev Psychol 27(5):486–493. https://doi.org/10.1016/j.appdev.2006.06.003
Meadows LA, Sekaquaptewa D (2013) The influence of gender stereotypes on role adoption in student teams. 23.1217.1–23.1217.16. https://peer.asee.org/the-influence-of-gender-stereotypes-on-role-adoption-in-student-teams
Merton RK (1957) Priorities in scientific discovery: a chapter in the sociology of science. Am Sociol Rev 22(6):635–659. https://doi.org/10.2307/2089193
Minderer M, Harvey CD, Donato F, Moser EI (2016) Virtual reality explored. Nature 533(7603):7603. https://doi.org/10.1038/nature17899
Mogil JS (2012) Sex differences in pain and pain inhibition: multiple explanations of a controversial phenomenon. Nat Rev Neurosci 13(12):859–866. https://doi.org/10.1038/nrn3360
Mori M, MacDorman KF, Kageki N (2012) The uncanny valley [from the field]. IEEE Robot Autom Mag 19(2):98–100. https://doi.org/10.1109/MRA.2012.2192811
Möschel M (2009) Race judicata: the ban on the use of ethnic and racial statistics in France. Eur Const Law Rev 5(2):197–217. https://doi.org/10.1017/S1574019609001977
Munafò MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, Percie du Sert N, Simonsohn U, Wagenmakers E-J, Ware JJ, Ioannidis JPA (2017) A manifesto for reproducible science. Nat Hum Behav 1(1):1. https://doi.org/10.1038/s41562-016-0021
Muratore M, Tuena C, Pedroli E, Cipresso P, Riva G (2019) Virtual reality as a possible tool for the assessment of self-awareness. Front Behav Neurosci 13:62. https://doi.org/10.3389/fnbeh.2019.00062
Nesbitt K, Davis S, Blackmore K, Nalivaiko E (2017) Correlating reaction time and nausea measures with traditional measures of cybersickness. Displays 48:1–8. https://doi.org/10.1016/j.displa.2017.01.002
Nicovich SG, Boller GW, Cornwell TB (2005) Experienced presence within computer-mediated communications: initial explorations on the effects of gender with respect to empathy and immersion. J Comput-Med Commun. https://doi.org/10.1111/j.1083-6101.2005.tb00243.x
Nosek BA, Ebersole CR, DeHaven AC, Mellor DT (2018) The preregistration revolution. Proc Natl Acad Sci 115(11):2600–2606. https://doi.org/10.1073/pnas.1708274114
Ochs M, Jain S, Blache P (2018) Toward an automatic prediction of the sense of presence in virtual reality environment. In: Proceedings of the 6th international conference on human-agent interaction, pp 161–166. https://doi.org/10.1145/3284432.3284452
Onyekuru BU (2015) Field dependence-field independence cognitive style, gender, career choice and academic achievement of secondary school students in emohua local government area of rivers state. J Educ Pract 6(10):76–85
Ortega JL (2022) Classification and analysis of PubPeer comments: how a web journal club is used. J Am Soc Inf Sci 73(5):655–670. https://doi.org/10.1002/asi.24568
Owens B (2018) Replication failures in psychology not due to differences in study populations. Nature. https://doi.org/10.1038/d41586-018-07474-y
Paaßen B, Morgenroth T, Stratemeyer M (2017) What is a true gamer? The male gamer stereotype and the marginalization of women in video game culture. Sex Roles 76(7):421–435. https://doi.org/10.1007/s11199-016-0678-y
Palmisano S, Allison RS, Kim J (2020) Cybersickness in head-mounted displays is caused by differences in the user’s virtual and physical head pose. Front Virtual Real 1:24. https://doi.org/10.3389/frvir.2020.587698
Park S, Hwang HS (2009) Understanding online game addiction: connection between presence and flow. In: Jacko JA (ed) Human-computer interaction, Pt Iv: interacting in various application domains, vol 5613. Springer, Berlin, p 378
Parsons TD (2015) Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences. Front Hum Neurosci. https://doi.org/10.3389/fnhum.2015.00660
Pashler H, Coburn N, Harris CR (2012) Priming of social distance? Failure to replicate effects on social and food judgments. PLoS ONE 7(8):e42510. https://doi.org/10.1371/journal.pone.0042510
Petri K, Feuerstein K, Folster S, Bariszlovich F, Witte K (2020) Effects of age, gender, familiarity with the content, and exposure time on cybersickness in immersive head-mounted display based virtual reality. Am J Biomed Sci 12(2):107–121. https://doi.org/10.5099/aj200200107
Plummer JP, Schuster D, Keebler JR (2017) The effects of gender, flow and video game experience on combat identification training. Ergonomics 60(8):1101–1111. https://doi.org/10.1080/00140139.2017.1280187
Pointer JS (1999) The far interpupillary distance. A gender-specific variation with advancing age. Ophthalmic Physiol Opt J Br Coll Ophthal Opt (optom) 19(4):317–326. https://doi.org/10.1046/j.1475-1313.1999.00441.x
Pool GJ, Schwegler AF, Theodore BR, Fuchs PN (2007) Role of gender norms and group identification on hypothetical and experimental pain tolerance. Pain 129(1):122–129. https://doi.org/10.1016/j.pain.2006.10.008
Popper KR (1959) The logic of scientific discovery. Basic Books. Oxford, England. https://doi.org/10.1063/1.3060577
Prinz F, Schlange T, Asadullah K (2011) Believe it or not: How much can we rely on published data on potential drug targets? Nat Rev Drug Discov 10(9):9. https://doi.org/10.1038/nrd3439-c1
Rebenitsch L, Owen C (2016) Review on cybersickness in applications and visual displays. Virtual Real 20(2):101–125. https://doi.org/10.1007/s10055-016-0285-9
Rehbein F, Staudt A, Hanslmaier M, Kliem S (2016) Video game playing in the general adult population of Germany: can higher gaming time of males be explained by gender specific genre preferences? Comput Hum Behav 55:729–735. https://doi.org/10.1016/j.chb.2015.10.016
Rikli R (1976) Physical performance scores as a function of experimenter sex and experimenter bias. Res Quart Am Alliance Health Phys Educ Recreat 47(4):776–782. https://doi.org/10.1080/10671315.1976.10616740
Riva G, Wiederhold BK, Mantovani F (2019) Neuroscience of virtual reality: from virtual exposure to embodied medicine. Cyberpsychol Behav Soc Netw 22(1):82–96. https://doi.org/10.1089/cyber.2017.29099.gri
Robillard G, Bouchard S, Renaud P, Cournoyer L-G (2002) Validation canadienne-française de deux mesures importantes en réalité virtuelle: L’Immersive Tendencies Questionnaire et le Presence Questionnaire. Poster presented at the 25ième congrès de la Société Québécoise pour la Recherche en Psychologie (SQRP)
Robinson ME, Riley JL, Myers CD, Papas RK, Wise EA, Waxenberg LB, Fillingim RB (2001) Gender role expectations of pain: relationship to sex differences in pain. J Pain 2(5):251–257. https://doi.org/10.1054/jpai.2001.24551
Romero F (2017) Novelty versus replicability: virtues and vices in the reward system of science. Philos Sci 84(5):1031–1043. https://doi.org/10.1086/694005
Romero F (2019) Philosophy of science and the replicability crisis. Philos Compass 14(11):e12633. https://doi.org/10.1111/phc3.12633
Ronay R, von Hippel W (2010) The presence of an attractive woman elevates testosterone and physical risk taking in young men. Soc Psychol Personal Sci 1(1):57–64. https://doi.org/10.1177/1948550609352807
Rumenik DK, Capasso DR, Hendrick C (1977) Experimenter sex effects in behavioral research. Psychol Bull 84:852–877. https://doi.org/10.1037/0033-2909.84.5.852
Rutrecht H, Wittmann M, Khoshnoud S, Igarzábal FA (2021) Time speeds up during flow states: a study in virtual reality with the video game thumper. Timing Time Percept 9(4):353–376. https://doi.org/10.1163/22134468-bja10033
Saputra R, Iqbal BM, Komarudin (2017) Stress emotion evaluation in multiplayer online battle arena (MOBA) video game related to gaming rules using electroencephalogram (EEG). In: Proceedings of the 2017 4th international conference on biomedical and bioinformatics engineering, pp 74–77. https://doi.org/10.1145/3168776.3168797
Schwind V, Knierim P, Haas N, Henze N (2019) Using presence questionnaires in virtual reality. In: Proceedings of the 2019 CHI conference on human factors in computing systems, pp 1–12. https://doi.org/10.1145/3290605.3300590
Shafer D-M, Carbonara C-P, Kropi M-F (2017) Modern virtual reality technology: cybersickness, sense of presence, and gender. Media Psychol Rev 11(2):1
Sheridan TB (2016) Recollections on presence beginnings, and some challenges for augmented and virtual reality. Presence-Teleoper Virtual Environ 25(1):75–77. https://doi.org/10.1162/PRES_e_00247
Simmons JP, Nelson LD, Simonsohn U (2011) False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 22(11):1359–1366. https://doi.org/10.1177/0956797611417632
Souza V, Maciel A, Nedel L, Kopper R (2021) Measuring presence in virtual environments: a survey. ACM Comput Surv 54(8):163:1-163:37. https://doi.org/10.1145/3466817
Spencer SJ, Logel C, Davies PG (2016) Stereotype threat. Annu Rev Psychol 67(1):415–437. https://doi.org/10.1146/annurev-psych-073115-103235
Spencer SJ, Steele CM, Quinn DM (1999) Stereotype threat and women’s math performance. J Exp Soc Psychol 35(1):4–28. https://doi.org/10.1006/jesp.1998.1373
Spreij LA, Visser-Meily JMA, Sibbel J, Gosselt IK, Nijboer TCW (2022) Feasibility and user-experience of virtual reality in neuropsychological assessment following stroke. Neuropsychol Rehabil 32(4):499–519. https://doi.org/10.1080/09602011.2020.1831935
Stanney K, Fidopiastis C, Foster L (2020a) Virtual reality is sexist: but it does not have to be. Front Robot AI. https://doi.org/10.3389/frobt.2020.00004
Stanney K, Hale KS, Nahmens I, Kennedy RS (2003) What to expect from immersive virtual environment exposure: Influences of gender, body mass index, and past experience. Hum Factors 45(3):504–520. https://doi.org/10.1518/hfes.45.3.504.27254
Stanney K, Lawson BD, Rokers B, Dennison M, Fidopiastis C, Stoffregen T, Weech S, Fulvio JM (2020b) Identifying causes of and solutions for cybersickness in immersive technology: reformulation of a research and development agenda. Int J Hum-Comput Interact 36(19):1783–1803. https://doi.org/10.1080/10447318.2020.1828535
Stanney KM, Kennedy RS, Drexler JM, Harm DL (1999) Motion sickness and proprioceptive aftereffects following virtual environment exposure. Appl Ergon 30(1):27–38. https://doi.org/10.1016/s0003-6870(98)00039-8
Stevenson HW, Allen S (1964) Adult performance as a function of sex of experimenter and sex of subject. Psychol Sci Public Interest 68:214–216. https://doi.org/10.1037/h0040395
Swim JK, Gillis AJ, Hamaty KJ (2020) Gender bending and gender conformity: the social consequences of engaging in feminine and masculine pro-environmental behaviors. Sex Roles 82(5):363–385. https://doi.org/10.1007/s11199-019-01061-9
Tannenbaum C, Ellis RP, Eyssel F, Zou J, Schiebinger L (2019) Sex and gender analysis improves science and engineering. Nature 575(7781):7781. https://doi.org/10.1038/s41586-019-1657-6
Tassinari M, Aulbach MB, Jasinskaja-Lahti I (2022) Investigating the influence of intergroup contact in virtual reality on empathy: an exploratory study using AltspaceVR. Front Psychol. https://doi.org/10.3389/fpsyg.2021.815497
Terry DJ, Hogg MA (1996) Group norms and the attitude-behavior relationship: a role for group identification. Pers Soc Psychol Bull 22(8):776–793. https://doi.org/10.1177/0146167296228002
Terry DJ, Hogg MA, McKimmie BM (2000) Attitude-behaviour relations: the role of in-group norms and mode of behavioural decision-making. Br J Soc Psychol 39(Pt 3):337–361. https://doi.org/10.1348/014466600164534
Tian N, Lopes P, Boulic R (2022) A review of cybersickness in head-mounted displays: raising attention to individual susceptibility. Virtual Real 26(4):1409–1441. https://doi.org/10.1007/s10055-022-00638-2
Townsend F (2013) Post-publication peer review: PubPeer. Editors’ Bull 9(3):45–46. https://doi.org/10.1080/17521742.2013.865333
Van Rooij AJ, Schoenmakers TM, Vermulst AA, Van Den Eijnden RJJM, Van De Mheen D (2011) Online video game addiction: identification of addicted adolescent gamers. Addiction 106(1):205–212. https://doi.org/10.1111/j.1360-0443.2010.03104.x
Vigil JM, Rowell LN, Alcock J, Maestes R (2014) Laboratory personnel gender and cold pressor apparatus affect subjective pain reports. Pain Res Manag 19:e13–e18. https://doi.org/10.1155/2014/213950
Wasserstein RL, Schirm AL, Lazar NA (2019) Moving to a World Beyond “p < 0.05.” Am Stat 73(sup1):1–19. https://doi.org/10.1080/00031305.2019.1583913
Weech S, Kenny S, Barnett-Cowan M (2019) Presence and cybersickness in virtual reality are negatively related: a review. Front Psychol. https://doi.org/10.3389/fpsyg.2019.00158
Weech S, Kenny S, Lenizky M, Barnett-Cowan M (2020) Narrative and gaming experience interact to affect presence and cybersickness in virtual reality. Int J Hum Comput Stud 138:102398. https://doi.org/10.1016/j.ijhcs.2020.102398
Wehden L-O, Reer F, Janzik R, Tang WY, Quandt T (2021) The slippery path to total presence: how omnidirectional virtual reality treadmills influence the gaming experience. Media Commun 9(1):5–16. https://doi.org/10.17645/mac.v9i1.3170
West C, Zimmerman DH (1987) Doing gender. Gend Soc 1(2):125–151. https://doi.org/10.1177/0891243287001002002
Westbrook L, Saperstein A (2015) New categories are not enough: rethinking the measurement of sex and gender in social surveys. Gend Soc 29(4):534–560. https://doi.org/10.1177/0891243215584758
Wirth W, Hofer M, Schramm H (2012) The role of emotional involvement and trait absorption in the formation of spatial presence. Media Psychol 15(1):19–43. https://doi.org/10.1080/15213269.2011.648536
Witkin HA (1949) The nature and importance of individual differences in perception. J Pers 18:145–170. https://doi.org/10.1111/j.1467-6494.1949.tb01237.x
Witkin H-A, Dyk RB, Fattuson HF, Goodenough DR, Karp SA (1962) Psychological differentiation: studies of development. Wiley, Hoboken, pp xii, 418
Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a presence questionnaire. Presence Teleoper Virtual Environ 7(3):225–240. https://doi.org/10.1162/105474698565686
World Medical Association (2013) World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA 310(20):2191–2194. https://doi.org/10.1001/jama.2013.281053
Yang AHX, Kasabov N, Cakmak YO (2022) Machine learning methods for the study of cybersickness: a systematic review. Brain Inform 9(1):24. https://doi.org/10.1186/s40708-022-00172-6
Yang S, Zhang W (2022) Presence and flow in the context of virtual reality storytelling: what influences enjoyment in virtual environments? Cyberpsychol Behav Soc Netw 25(2):101–109. https://doi.org/10.1089/cyber.2021.0037
Yee N (2017) Beyond 50/50: breaking down the percentage of female gamers by genre. Quantic Foundry. https://quanticfoundry.com/2017/01/19/female-gamers-by-genre/
Yeung AWK (2017) Do neuroscience journals accept replications? A survey of literature. Front Hum Neurosci. https://doi.org/10.3389/fnhum.2017.00468
Acknowledgements
The author would like to thank Nathalie LeBigot for her important help with logistics, materials and beta-testing, Blandine Verjut for proofreading and Fig. 2, Jessica Mange & Thomas Pineau for proofreading, all the graduate students for their participation as experimenters and beta-testers (Alan Grysole, Laurent Lansonneur, Lou-Anna Le Guen, Luca Mourgaud, Magali Hamard, Nawël Schappacher, Trang N’Guyen, Tristan Feutren, Christine Tupaj), and all the anonym participants.
Author information
Authors and Affiliations
Contributions
The main author confirms being the sole author of the present work.
Corresponding author
Ethics declarations
Conflict of interest
The author did not receive support from any organization for the submitted work and declares no competing interests. In addition, this research involving human participants was conducted in accordance with ethical standards and the Helsinki Convention. A local ethics committee validated this experiment beforehand (CICPPR—Comité Institutionnel Consultatif pour la Protection des Personnes dans la Recherche, IMT Atlantique).
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Maneuvrier, A. Experimenter bias: exploring the interaction between participant’s and investigator’s gender/sex in VR. Virtual Reality 28, 96 (2024). https://doi.org/10.1007/s10055-024-00993-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10055-024-00993-2