Introduction

Immersive technologies such as augmented reality (AR) and virtual reality (VR) provide new modalities, applications, and interfaces for human–computer interaction (Stanney, 1995; Zheng et al., 1998; Halarnkar et al., 2012; Jung & tom Dieck, 2017; Rubio-Tamayo et al., 2017). These technologies have been employed in recreational and professional settings (Nee & Ong, 2013; Bonetti et al., 2018; Damiani et al., 2018; Neumann et al., 2018; Yung & Khoo-lattimore, 2019) and informed research and teaching practice across a diverse range of disciplinary fields (Hwa Choi, 2016; Howard, 2017; Kavanagh et al., 2017; Pan & Hamilton, 2018), in particular as it comes to behavioral research on human experiences in space and place (e.g., Schuemie et al., 2001; Riva et al., 2007; Malloy & Milling, 2010; Meyerbröker & Emmelkamp, 2010; Schultze, 2010; Bohil et al., 2011; Annerstedt et al., 2013; Diemer et al., 2014; Turner & Casey ,2014; Felnhofer et al., 2015; Serrano et al., 2016; Boletsis, 2017). Current-generation VR devices, in particular, combine affordability with high-resolution displays, low latency, sensor arrays, and graphic processing capabilities that afford increasingly immersive experiences that have only became attainable over the last few years (Anthes et al., 2016).

The dissemination of AR/VR technology in society is catalyzed by large investments of prominent commercial businesses entities and the availability of affordable AR/VR end-consumer software products and platforms that allow for the publishing and viewing of AR/VR content. In particular, 360-degree video poses comparatively low barriers to the creation of content that lends itself to recreational applications, considering the wide selection and general affordability of recreational 360-degree cameras (Tan et al., 2018; Ferdig & Kosko, 2020). In the context of research, 360-degree video has also received considerable attention, as evidenced by the growing number of research publications pertaining to technological aspects of 360-degree such as adaptive video streaming (Qian et al., 2016, 2018; Corbillon et al., 2017; Nasrabadi et al., 2017). Apparent within the body of recent research pertaining to 360-degree video is a relative scarcity of human-centric studies that employ 360-degree video content as a stimulus material (Bessa et al., 2016; Kavanagh et al., 2016; Rupp et al., 2016; Stupar-Rutenfrans et al., 2017; Aitamurto et al., 2018; Lee et al., 2018; Schöne et al., 2020; Hu et al., 2020), in particular with sizable sample sizes (Tse et al., 2017; Rupp et al., 2019; Mazumder et al., 2020), as opposed to those studies that research 360-degree video as the prime subject of interest (Hosseini & Swaminathan, 2016; Kopf, 2016; Afzal et al., 2017; Su & Grauman, 2017; Domański et al., 2017). Most importantly, best practices for the use of 360-degree video in research have yet to emerge, and will likely require domain-specific solutions and context-specific applications rather than a single one-size-fits-all solution (Roche & Gal-Petitfaux, 2017; Van Damme et al., 2019; Mabrook & Singer, 2019; Benzina et al., 2020; Zolfaghari et al., 2020), as even metrics for the assessment of the quality of 360-degree video continue to be negotiated (Tran et al., 2017; Xie et al., 2017; Van den Broeck et al., 2017; Tran et al., 2018; Frans Fela et al., 2020; Xu et al., 2020).

In this paper, we report on the institutional and methodological challenges that we encountered during the implementation of an 18-month-long collaborative experimental research project on human emotion, 360-degree video, and live pet animals at two American institutions for higher learning. Within this context, we provide suggestions to facilitate research by investigators engaging in experimental work at the nexus of VR/AR technology, live animals, and human participants, with particular consideration for the utilization of high-definition 360-degree VR video showcasing pet dogs. We expect this paper to be foremost useful to researchers who are interested in the strategic and effective use of 360-degree video and pet animals in behavioral human-subject research. Our assumption throughout this paper is that the reader has prior knowledge on and experience with institutional and practical requirements pertaining to human-subject testing but less so in regard to conducting experimental research with VR technology and live animals, as was the case for our research group at the onset of our study.

In the following section, we provide a brief overview of our experimental study, including a description of its motivation and implementation. Next, we discuss the institutional considerations related to research-integrity oversight requirements at institutions of higher education in the United States for research with live animals, in general, and pet dogs, in particular. Following that, we report on the methodological challenges related to the use of VR technology and 360-degree video stimulus materials. Also included in that section is a discussion on the current state of VR devices and projections pertaining to the technological frontier of VR content in terms of visual fidelity. In the last section, we conclude the paper with a summary of our recommendations and a brief outlook.

Brief description of experimental study

In this section, we provide a brief description of the motivation, theoretical underpinnings, and general setup of our study. The description is intended to provide a sufficient frame of reference for our discussion of the institutional and methodological challenges in the subsequent sections. For a detailed description of our experimental design, procedure, data, and analysis, we defer to a future effort.

Our study investigated the effects of short-duration exposure to depictions of human–animal interactions in 360-degree video on psychological and physiological indicators in non-clinical research subjects. The study was motivated by two areas of research. First, positively valenced interactions with pet animals have been shown to induce desirable psychological and physiological responses in humans, with positive effects reported for both live (Baun et al., 1984; Serpell, 1991; Wilson, 1991; Friedmann & Thomas, 1995; Miller et al., 2009; Abate et al., 2011; Kruger & Serpell, 2010; Berget & Grepperud, 2011; Levine et al., 2013; Vrbanac et al., 2013; Kamioka et al., 2014; Stefanini et al., 2015; Maber-Aleksandrowicz et al., 2016; Giuliani & Jacquemettaz, 2017; Charry-Sánchez et al., 2018; Hu et al., 2018) as well as robotic pets (Moyle et al., 2013; Banks et al., 2008). Second, past studies found that higher-immersion VR equipment produced greater positive effects in humans compared to less immersive technology such as a computer screen. However, it is currently unknown to which extent positive effects of human–animal interactions translate to VR, and specifically to depictions of positive human–animal interactions in 360-degree video. Our group hypothesized that a current-generation VR device showcasing 360-degree videos of human–animal interactions induces greater change in core affect activation and valence (Ekkekakis & Russell, 2013), heart rate, and blood pressure in VR-group participants than in participants who were exposed to companion animals through less immersive means (i.e., desktop video, photo books).

A 360-degree video of human–dog interactions was therefore an essential component of our research study in that the video had to be in place before any experimentation with human subjects could commence. The video showcased a single adult observer seated in a mid-sized room among a small number of dogs, with the dogs interacting with the observer and each other in a playful, good-natured manner. Considering the importance of touch and eye contact as it comes to positive physiological responses in the human–dog dyad and human–dog bonding (Hosey & Melfi, 2014; Kaminski et al., 2019; Marshall-Pescini et al., 2017; Nagasawa et al., 2015; Téglás et al., 2012; Waller et al., 2013; Meyer & Forkman, 2014; Kuhne et al., 2012b; Feuerbacher & Wynne, 2015; Kuhne et al., 2012a; Kekecs et al., 2016; McCune et al., 2014), we deliberatively put an emphasis on capturing close-proximity human–animal interactions such as petting, eye contact, and playing. Figure 1 presents a typical scene from the video that we recorded.

Fig. 1
figure 1

Screenshot from the 360-degree video that we created for our experimental study for use as stimulus materials. The video captured the interactions of a single human individual with two non-reactive dogs and their playful interaction with each other and the human observer

Full size image

All stimulus materials were derived from the single 10-min-long 360-degree high-definition video that our group created in-house. The video was recorded using a professional, high-resolution 360-degree cameraFootnote 1 that was attached to the top of a helmet worn by the single observer, approximating a first-person perspective on the surrounding space. Furthermore, we employed the Oculus RiftFootnote 2, a commercial off-the-shelf VR headset for our VR group participants.

We collected human-subject data from December 2018 through May 2019 at two dedicated lab spaces on the campuses of the research group members. During the data collection period, the two lab spaces were available to participants during regular business hours for 20 h per week. However, the weekly average number of participants completing the experiment at both study sites was 5 (Fig. 2), irrespective of persistent recruitment efforts throughout the data collection period comprised of the campus-wide distribution of flyers, in-class announcements, and social media postings. Participants were able to self-manage their appointments through an online scheduling serviceFootnote 3. Overall, 285 participants took part in the experiment.

Fig. 2
figure 2

Number of completed experiments across our two study sites by week. Our data collection took place over a 29-week period from December 2018 through June 2019. Participation numbers across the two study sites were similar, with consistent decreases in participation during weeks when courses were not in session (both locations operate under the 10-week quarter system), and an average of five participants per week

Full size image

Institutional considerations for experimental studies using animals and VR technology

In this section, we discuss the unanticipated challenges the were afforded to our group by the combination of pet dogs, 360-degree video, and overall VR-centric experimental human-subject study design. Our research study underwent a substantial preparation and approval process regarding research integrity oversight, given that our study involved both animal and human participants. Whereas our team was familiar with the institutional expectation and requirements regarding the protection of the welfare of human research subjects, commonly administered in the United States through a university’s Institutional Review Board (IRB), the institutional research oversight requirements regarding the involvement of animals were unclear to us. Below, we provide a summary of our experiences pertaining to research integrity oversight during our study.

Institutional considerations for research activities with live animals

As our group prepared for the creation of the stimulus materials for our study, it became apparent that the institutional oversight requirements for the involvement of live animals were generally more stringent than those for human participants. Even though our focus here is on the animal-research oversight regulations for institutions of higher education in the US, we expect similar regulations and procedures in countries that have implemented safeguards for the ethical use and the well-being of animals in research (Baumans, 2004; Caporale et al., 2005; Institute of Medicine (US) and National Research Council (US), 2012; Sinmez & Yasar, 2019).Footnote 4

Whereas single-member reviews (i.e., Designated Member Review; DMR) of research protocols are often sufficient for no- to minimal-risk studies with human participants, research activities with live animals usually require a review by the Institutional Animal Care and Use Committee (IACUC; Everitt & Berridge, 2017). Exemptions to full-committee review are limited to research activities where no direct contact with or manipulation of the behavior of an animal occurs (Monamy, 2017). For instance, documenting the behavior of animals at a distance and location that does not allow the observed animals to sense the presence of the researcher qualifies for DMR (Silverman et al., 2014). Considering that a core requirement for our experimental study was the creation of 360-degree video of playful interactions between a researcher and pet dogs, DMR was not an option, and our project was subject to a full IACUC review. Table 1 lists the periods of major activities related to research integrity oversight pertaining to our project.

Table 1 Overview of research-integrity activities related to institutional research oversight requirement by the Institutional Review Board (IRB) and the Institutional Animal Care and Use Committee (IACUC)
Full size table

Overall, the period of most effort pertaining to the IACUC review and approval process for our research protocol span a period of approximately three and a half months. We estimate that the time allocation for the two research group members who led the animal-related research activities to be the equivalent of 200 work hours. This number only pertains to our research group, with effort and time contributed by members of the university administration and campus community non-accounted for here since such information is not available to us.

Apart from the time necessary for the review of our application materials and study design by the IACUC committee, a considerable amount of time was required to establish the details of our research protocol as far as the use of live pet dogs in our project was concerned. Initially, we anticipated that the video recording would take place on the premises of a local animal-welfare organization (e.g., Humane Society or Animal Services), considering that these organizations would be able to supply appropriate facilities and staff support that could be leveraged for our project. As we learned after our initial consultations with members of the university’s animal research integrity office, this approach was unlikely to be attainable in a timely manner due to the requirement to have in place written agreements on animal-research oversight procedures and regulations with outside organizations (Everitt & Berridge, 2017).

First and foremost, academic institutions who receive funds under the U.S. Public Health Service (PHS) and conduct animal activities require to have on file a formal assurance with the Office of Laboratory Animal Welfare (OLAW)Footnote 5. The chief responsibility of OLAW is to monitor compliance with the Policy on Humane Care and Use of Laboratory Animals by assured institutions to ensure the humane care and use of animals in PHS-supported research, testing, and training (Silverman et al., 2014). Apart from the National Institutes of Health (NIH), PHS funding extends to the National Science Foundation (NSF), the Centers for Disease Control and Prevention (CDC), and the Food and Drug Administration (FDA). OLAW does not oversee animal activities funded by the Department of Defense (DoD), the U.S. Department of Agriculture (USDA), and neither does its oversight extend to non-profit or for-profit organizations (Everitt & Berridge, 2017). Hence, it is unlikely for any given local animal shelter or rescue to have OLAW Assurance. If researchers at an institution with OLAW Assurance choose to conduct animal activities at a secondary institution without OLAW Assurance, the activities need to be subcontracted to the secondary institution and the secondary institution be named a performance site by the prime awardee institution (Silverman et al., 2014; Everitt & Berridge, 2017; Monamy, 2017). After the review and approval of the performance site, the primary institution will work with OLAW to negotiate an assurance with the secondary institution. Such an approach is likely to require significant buy-in by the partnering organization and the explicit allocation of dedicated staff time, which might render it non-feasible within the time constraints of a shorter-duration research study such as ours. In this context, research procedures with pet animals on a university campus have to consider that university policies often prohibit the presence of pet animals other than emotional support and service animals on a university campus, in particular as it comes to indoor spaces.Footnote 6Footnote 7

Another consideration in this context is that shelter environments are often stressful for the animals and their behavior inside of the shelter environment may not be indicative of the behavior that the animals will exhibit in a different space (Palma et al., 2005). Dogs are highly sensitive to changes in their environment, which can act as a stressor on the dogs and in turn lead to non-desirable, reactive behavior such as barking and biting (Wright & Nesselrote, 1987; Lindsay, 2000). After adoption from a shelter, it is common to take multiple weeks for the dog’s typical personality to emerge, including non-desirable reactive behavior due to unforeseeable factors such as genetics, past experiences, sensory triggers, and fear responses (Lord et al., 2008). Shelter animals may also become unavailable at any time due to adoption. Thus, it may be unpredictable and difficult to identify dogs coming from a shelter environment who would behave as we intended to showcase in our video, that is, relaxed and playful. In turn, our group decided to move the capturing of the 360-degree video to a suitable room on our university campus and constrain our selection of dogs to those currently living with their private owners.

Research protocol for the use of pet dogs

For the capturing of the 360-degree video, careful thought had to be given to establishing a research protocol that would safeguard the well-being of the dogs, their owners, as well as the other individuals on campus who might cross paths with the dogs during their presence on the university grounds. To minimize the possibility of reactive behavior, before individual dogs and their owners were invited to take part in the video recording events, we devised an online screening survey to make sure that it was appropriate for the dogs to participate in the video recording. The questions that were asked followed common standards in veterinary behavior and dog daycare facilities (Hsu & Serpell, 2003; Duffy et al., 2014; Wiener & Haskell, 2016).Footnote 9 In addition to questions on the age, size, and breed of the dog, a particular focus was on the presence or absence of prior occurrences of reactive behavior towards other dogs or human strangers. The survey also inquired into the status of vaccinations and medical examinations. If the dog’s owner stated that their dog exhibited any reactive behavior in the past, the dog was excluded from further consideration. Furthermore, we excluded dogs in poor health and those without recent veterinary care or up-to-date vaccination records.

Subsequently, we invited a group of eight eligible dogs and their owners to partake in the video recording procedure. The video recording procedure was comprised of two separate 1-hour long events on two consecutive days. Both events were aimed at assessing the social compatibility of the invited dogs while introducing the dogs to each other, the researchers, the video equipment, and the study room through a structured, step-wise process. All interactions among the dogs and researchers were closely monitored by two dog-behavior specialists. Furthermore, all researchers who interacted with the dogs completed IACUC research-integrity and animal-research training requirements. For the final selection of eligible dogs for the video recording, we deferred to the observations and professional judgement of the dog behavior professionals to identify those dogs that exhibited greatest compatibility. Participation in the video recording sessions was incentivized by a modest stipend of $40 for each dog owner and event. Dogs owners assumed responsibility for transporting their dogs to the study room in a safe manner.

During both sessions, dog owners and on-leash dogs met in front of the study room, with the two dog behavior professionals providing advice on assessing and minimizing any distress among the dogs. The dogs were sequentially introduced to each other and the researchers, and, subsequently, the study room and recording equipment, first on-leash and then off-leash. During all times, the dog behavior professionals monitored for signs of stress, reactivity, aggression, and/or discomfort and provided advice on the duration of each step. Out of the group of eight eligible dogs that were invited, two dogs were ultimately selected for the video recording.

On the second day, once the dog behavior professionals confirmed that the selected dogs had gotten used to the room and exhibited calm, positive attitudes toward the environment and each other while being off-leash, the recording of the video commenced. During the capturing of the video, the camera was set to record continuously. A single researcher, a single dog behavior specialist, and the two selected dogs were collocated in the study room. The dogs remained off leash and were free to roam the room. The researcher was seated in a chair in the middle of the study room, wearing a helmet with a mounted 360-degree, virtual-reality camera. The researcher remained seated, facing in the same direction, and avoided loud speaking as well as movements of the body and head. A small treat was placed in the researcher’s shirt pocket to entice interest by the dogs. The dog behavior specialist remained standing silently in a corner of the room and periodically bounced a toy. All interactions between the researcher and the dogs were driven by the dogs’ natural behaviors and personalities. We observed the following interactions: (a) the dogs behaved in a friendly manner; (b) the researcher threw a toy and one of the dogs fetched it; (c) the researcher reached for and petted one of the dogs; and (d) the researcher placed one of the dogs on his lap and petted it. Once the video recording session concluded, the dogs and dog owners were debriefed by the researchers and guided one-by-one on separate routes towards the exits to minimize the potential for further arousal among the dogs. No adverse events occurred leading up to or during any of the sessions.

Research considerations for the use of VR technology

In this section, we discuss the requirements for use of VR technology in research. Specifically, we first summarize the space and computing requirements pertaining to current-generation VR headsets. Subsequently, we discuss technological and technical considerations for the creation of 360-degree video for VR.

Lab space and computing requirements

Our two labs were arranged so that each space could afford 360-degree tracking of the VR headset and controllers through external tracking sensors (i.e., outside-in tracking) (i.e., outside-in tracking Anthes et al., 2016).Footnote 10 Our lab spaces offer approximately 80 square feet of obstruction-free space dedicated to the VR setup, with the minimal space requirements for room-scale VR setups usually no less than 30 square feet. Three tracking sensors in a triangular arrangement (two up front, one in the back) are required for seamless 360-degree tracking of the headset.Footnote 11 The sensors can be placed on table surfaces or, for generally better coverage, mounted on walls at a recommended height of 8 feet using third-party wall mounts. Each sensor affords a conical field of coverage, with the ideal placement of each sensor such that its cone is maximized and unobstructed, with zones of partial overlap between the individual tracking cones, as illustrated in Fig. 3.

Fig. 3
figure 3

A typical arrangement of an external room-scale, outside-in VR tracking setup in a mid-sized room using three sensors (shown as black solid ellipses). For the consumer version of Oculus Rift, the horizontal coverage of each sensors is about 100 degrees horizontally and 75 degrees vertically. In the figure, the overlap among the cone-shaped horizontal coverage areas of multiple sensors is visualized by darker shades of grey. For optimal tracking of head movements, the area in which most of the VR activity occurs should be covered by all three sensors. The rectangle with the dashed border represents the suggested minimal area for room-scale tracking. It is likely that the recommended room size for room-scale tracking will remain the same to VR devices using inside-out tracking

Full size image

Some recent VR headsets have abandoned external tracking sensors in favor of inside-out tracking capabilities (Anthes et al., 2016). In inside-out positional tracking, cameras and/or sensors are integrated into the VR device itself instead of relying on external hardware. Considering that they have fewer requirements for a functional VR setup, VR devices with inside-out tracking generally require fewer steps to utilize and maintainFootnote 12. In addition, forgoing external tracking hardware is a necessity for self-contained, potentially wireless VR headsets, which are generally seen as more user friendlyFootnote 13. Having said that, inside-out tracking has the drawback that objects and equipment that are not in the field of view of the tracking hardware, often times realized through the integration of video cameras in the front of the headset, will not be accounted for.Footnote 14 Irrespective of the tracking system, current VR headsets share similar minimum and maximum room size requirements. Table 2 provides an overview of major consumer-level VR headsets and their specifications.

Table 2 Past and current major commercial VR headsets and their key specifications
Full size table

Until advances in technology miniaturization and wireless data transfer surpass the processing demands of high-fidelity, high-resolution VR headsets, greatest visual fidelity will remain within the purview of VR headset that are tethered to a desktop computer with a powerful, dedicated graphics processing unit (GPU)Footnote 15. In the context of computer equipment for VR devices, we recommend purchasing computer workstations that come pre-configured with a high-end, VR-ready GPU over separate purchases of a non-VR workstation and aftermarket GPU. We assume here that most academic institutions in the U.S. rely on computer equipment purchasing channels and vendor service contracts similar to ours (i.e., Dell). The high-end GPUs required for tethered VR headsets have significant power supply and physical space requirements in terms of both width and height compared to lower-tier graphics cards that are geared towards business tasks. In our case, the models of computer workstations that were available to us for purchase order did not entail those with a pre-configured, VR-ready GPU. That, in turn, necessitated the purchase of an aftermarket GPU, which had us content with technical troubleshooting to get the aftermarket GPU ready for use, with a list of select difficulties as follows: 1. the supplemental power supply connectors on the workstation were vendor-specific and required the identification and purchase of an additional adapter that was not included with either the workstation or aftermarket GPU; 2. fitting the aftermarket GPU in terms of width required the reconfiguration of internal cards, including the removal of a cooling element from a controller card; 3. fitting the aftermarket GPU in terms of height did not allow for the side access panel of the computer case to be re-attached and shut (in order to minimize the possibility of accidental contact with the innards of the workstation, we relocated it to a corner desk so that the exposed side of the case rested against a wall). Considering that computer manufacturers typically do not provide information on available internal physical space and types of power supply connectors, it remains unknown if any of these difficulties could have been avoided through the purchase of a different workstation model. Hence, our recommendation is to avoid aftermarket GPUs in favor of VR-ready preconfigured systems.

As an alternative to using a computer system with an internal GPU, for those otherwise underpowered desktop and notebook computers that are equipped with a high-throughput peripheral port (e.g., USB-C with Thunderbolt 3)Footnote 16, VR-ready graphic processing capabilities can be achieved through the use of an external GPU enclosure. These ports offer large-bandwidth, bi-directional video data transmission capabilities that are sufficient to drive a VR-ready GPU in an external GPU enclosure (eGPU), while only introducing a modest overall performance decrease of 15–20% as compared to an internal GPU.Footnote 17. What is gained through the use of an eGPU is that the VR setup becomes flexible since the eGPU can be moved and connected to other computer systems that support the proper connector on an as-needed basis. Having high-performance graphic processing hardware available as a shared, modular resource is beneficial for any data processing scenario that relies on raw GPU power (e.g., VR equipment; video editing and processing; 3D modeling). Furthermore, aftermarket eGPUs are likely to avoid difficulties related to connecting and supplying power to the chosen GPU and should make near-future GPU upgrades a matter of a simple swap.

360-degree video for VR

Three major parameters for the creation of 360-degree video for VR are resolution, frame rate, and monoscopy or stereoscopy, in particular as it comes to perceptions of the quality of the experience (Lambooij et al., 2007; Keshavarz & Hecht, 2012; Ling et al., 2012; Allen et al., 2016). As with static images, resolution describes the number of pixels that are contained in each individual image captured by the video sensor, which typically determines the amount of detail resolved in each image, with higher resolution equating to greater detail. Frame rate is exclusive to videos and specifies how many static images are captured in a given time period, usually per second. Modern video standards often employ 24 frames per second, which is the minimum speed for motion in a video to appear smooth and realistic. Lastly, monoscopy and stereoscopy describes how many simultaneous video images are used in the VR video for the left and right eye. Stereoscopic video refers to 360-degree video that uses two simultaneous, separate video images that provide a slightly offset perspective for each eye, which leverages human natural capacity for stereopsis (i.e., binocular vision) and depth perception. In monoscopic video, on the other hand, the same video image is shown to both eyes, resulting in less visual depth-related information being related to the brain.

Considering the benefits of stereoscopic over monoscopic imagery (IJsselsteijn et al., 2001; Henn et al., 2002; Ilgner et al., 2006; Baños et al., 2008; Faria et al., 2016; Loup-Escande et al., 2017), we used the highest resolution that was supported by our VR video cameraFootnote 18, that is, stereoscopic video at a resolution of 6K and 30 frames per second. The video camera employed a configuration of six lenses that were equally spaced around the center line of the spherical camera housing. During recording, each lens captured an individual video file which was subsequently stitched together with third-party VR video editing software as a part of the post-processing period of the six source video files. The output of the stitching procedure was a single spherical (i.e., 360-degree) output video. We found that both professional editing software programs capable of processing 360-degree video that we testedFootnote 19 required multiple iterations for the output of the stitching algorithm to be acceptable. With one software program, in particular, a single stitching seam was marked by pronounced warping and waviness, with the seam distorting the body of one of the dogs for a large part of the video. Further trial-and-error testing of the software settings eliminated such distortion and produced clean, non-distracting seams. Subsequently, we applied algorithmic motion stabilization to the video to eliminate involuntary small movements of the helmet-mounted camera during recording. As an artefact of this process, we noticed slight intermittent wandering of the perspective in the VR video when the VR headset was not moved. Given that it was reasonable to assume that participants in the VR headset group would continuously move their heads (e.g., to track the dogs or explore the surrounding VR space) rather than remain motionless for any period of time, this artefact was not further concerning to us.

A point of concern, however, was that during the piloting of different monoscopic and stereoscopic videos that we generated from the source files, a number of our research assistants reported feeling motion sickness to the extent that they were not able to finish watching the 10-min-long videos in VR. Considering previous studies that studied stereoscopic imagery (Lambooij et al., 2007; Keshavarz & Hecht, 2012; Schild et al., 2012; Allen et al., 2016) and those that utilized the same head-mounted display as us (Munafo et al., 2017; Chessa et al., 2019), certain discomfort in terms of motion sickness for the participants in the VR group was to be expected. However, one adjustment to reduce the occurrence of motion sickness among the study participants was to restrict the amount of VR exposure to 10 min. After further pilot testing with members of our research group, we found that the stereoscopic video files were causing the reactions whereas the monoscopic videos did not. This might have been a product of the video player software and computer hardware, which we were not able to resolve. Furthermore, video files with a resolution greater than 4K presented persistent problems during playback, with many files not being able to be played at all or only with prominent jitter of the video, which rendered these videos unwatchable. As a consequence, we opted for monoscopic video with a resolution of 4K at 30 frames per second to function as our final stimulus video for the experiment with human participants. It is noteworthy that throughout the duration of our data collection, no participant prematurely stopped the experiment due to motion sickness.Footnote 20

Eye-resolution VR

It is noteworthy that the displays that are used in current VR headsets have a long way to go in order to approach the visual fidelity of the human eye. For each new-generation model of a GPU, typically brought to market on an annual basis, iterative performance gains of 50% are typical over last-generation models of the same make. If one accepts that a comparison between electronic displays and the human eye is proper, an equivalent of perfect visual acuity for an adult viewer across the fovea (i.e., 20/20 vision) can reasonably be approximated to 2200 pixels per inch. At such pixel density, individual pixels would be imperceptible to the viewer wearing a VR headset, with a distance of 4 inches between the eyes and displays. Hence, the content shown on the displays would resemble life-like visual stimuli.Footnote 21

Current-generation VR headsets employ display sizes of about 3 inches square per eye and provide a field of view that covers about one-fourth of the content of a spherical 360-degree spaceFootnote 22. This has direct implication for the amount of pixels that are displayed by a VR headset at any given time. Specifically, in order to achieve a pixel density of 2200 dpi (i.e., eye-resolution VR), one would require roughly 5 million pixels per inch square or 50 million total pixels per display, that is, a resolution of 8K as far as the field of view of the headset is concerned. Conversely, and assuming that the entire spherical video is broadcasted to the VR headset, eye-resolution VR video would have to provide a resolution of 32K (i.e., four times the 8K resolution visible in the VR headset (Table 3). With VR video at 4K resolution, as in our research project, the effective viewing resolution in the VR headset was about 1K, or 1000 pixels horizontally. Considering that annual gains in graphical processing performance are about 50%, we estimate that the technological requirements for eye-resolution VR will not be met for the next 5–6 years (Table 4).

Table 3 Considering that the field of view of current head-mounted displays (HMD) equates to about 25% of the available visual information in a spherical, 360-degree VR video (i.e., the part of the video sphere that the user is currently facing) at any given time, the amount of pixels being by the HMD can be approximated based on the total resolution of the spherical video
Full size table
Table 4 Projected increases in average computational graphics processing power
Full size table

Discussion and conclusions

Over the last few years, we have witnessed a rapid proliferation of affordable off-the-shelf VR and AR devices that provide for increasingly immersive experiences of virtual environments that may not be achieved by other forms of techniques or technological interventions (Anthes et al., 2016; Heyselaar et al., 2017; Brookes et al., 2019). Having said that, even though the new generation of VR and AR devices has taken significant strides forward in terms of video and audio fidelity compared to older generations (Regan, 1995), technological advances over the next years in terms of graphical processing power and display technology are likely to bring about sophisticated VR/AR devices and content that will afford life-like virtual experiences. To what degree technological fidelity matters in the context of human perception, cognition, and emotion remains an open subject for academic research (Dinh et al., 1999; Nichols et al., 2000; Schuemie et al., 2001; Baños et al., 2000; Lin et al., 2002; Baños et al., 2004; Bowman & Mcmahan, 2007; Riva et al., 2007; Alsina-Jurnet et al., 2007; Baumgartner, 2008; Slater et al., 2009; Gorini et al., 2010; Chessa et al., 2019; Chamilothori et al., 2019). Considering the complexity of the domain, there is no question that future research and knowledge discovery will continue to profit from multi- and interdisciplinary inquiries that adopt diverse perspectives to elucidate the human experience as it relates to virtual environments, in particular as it comes to mental well-being and psychological interventions (Turner & Casey, 2014; Valmaggia et al., 2016).

There are a multitude of considerations for the strategic and effective use of VR technology in research. Producing video-based stimulus materials requires considerable effort that needs to be taken into account when planning research activities, timelines, and budgets. This holds particularly true for 360-degree video in which post-processing of the source video files is more complicated compared to conventional video editing of rectangular video. This is due to the need for the managing, stitching, and processing of a number of source files to create a final spherical video that can be viewed through a VR headset. However, a major advantage of 360-degree video is that realistic, well-controlled stimulus materials can be readily captured with a suitable VR video camera. Put differently, 360-degree video is an accessible means for the creation of VR content that captures life-like social interactions. The challenging alternative to this approach would be to design, model, and animate computer-generated landscapes and populate these with digital agents that behave in a plausibly intelligent and/or convincingly realistic manner. The latter approach was out of the question for our research study on human–pet interactions. We recommend using the maximum resolution shared by both the VR video recording device and the post-production video editing software to capture the source video. At or above a theoretical source resolution of 32K (i.e., eye-resolution VR video), gains in visual fidelity should be imperceptible to the eye. During the piloting of the spherical 360-degree videos for our study, stereoscopic videos and frame refresh rates above 30 led to heightened rates of motion sickness. In addition, none of the video rendering software that we tested with the Oculus Rift reliably supported the playback of spherical videos above 4K. Our recommended settings for the creation of 360-degree video are summarized in Table 5.

Table 5 Recommended settings for the creation of 360-degree video for VR
Full size table

In terms of the selection of a VR headset, most devices currently available to end consumers offer comparable specifications in terms of resolution, refresh rate, and field of view, and it is unlikely that iterative advances over the next few years in any one of these parameters will result in significant leaps in realism and immersiveness of the VR content. Thus, a researcher’s choice in regard to the purchase of any particular VR headset might be best determined by cost and personal preference. For optimal coverage and performance of positional tracking functionality, VR headsets that rely on external tracking sensors (i.e., outside-in tracking) are generally preferable in that they are able to sense movement of the user and that of peripheral equipment (e.g., hand controllers) that are not in the immediate coverage of the headset alone (e.g., rapid turns of the head; placing hands on the back). Similarly, if the presence of a cable connected to the VR headset is not a major concern, VR headsets that are tethered to a computer equipped with a high-performance GPU provide the highest visual fidelity and largest potential for simple near-future upgrades of the GPU (and the incremental gains in visual fidelity resulting from the upgrade) over having to replace the VR headset to achieve comparable gains.

In this paper, we compiled our experiences and the lessons that we learned during a sizable experimental study on human emotion, VR technology, and companion animals. Our project exemplifies one of many potential innovative engagements of VR technology in academic research. The main objective of this paper is to support researchers in the planning and conduction of experimental research on immersive technology, virtual-reality experiences, and human–computer interactions. To that account, we presented our experiences pertaining to the institutional challenges that presented themselves to us regarding the use of live animals in our research study as well as those that emerged during our data collection with human participants. Furthermore, we presented practical knowledge on the use of VR technology and content, in general, and contemporary, off-the-shelf VR headsets and 360-degree video, in particular. Specifically, we discussed the methodological and technological challenges pertaining to the creation of 360-degree video and the use of virtual-reality technology in research.

Our main objective for this paper was to convey insights that help mitigate potential barriers to employing VR devices and content in experimental research and propagate their use in academic practice. As with any technology, piloting of the functioning of VR setups, devices, and content is strongly warranted to make sure that the chosen configurations are compatible and working properly and that the VR viewing experience is comfortable. Furthermore, the involvement of live animals in research will likely necessitate a full-committee IACUC review, which affords considerable allocation of effort and time. Based on the experiences that we discussed and the details that we provided, other researchers should be able to anticipate and strategically plan for these activities and successfully execute them in an efficient and deterministic manner.