The rapid advances in the technology and science of presenting spatial sound in virtual, augmented, and mixed-reality environments seem to be underrepresented in recent literature. The goal of this special issue of the Virtual Reality Journal is twofold: to provide a state-of-the-art review of progress in spatial sound as applied to virtual reality (VR) and augmented reality (AR), and to stimulate further research in this emerging and important field. In this special issue, we are pleased to present papers representing a range of topics, from basic research on the perception of spatial sound to more applied papers on the use of spatial sound in real-world settings. It is the hope of the guest editors that the special issue will also encourage scientists to incorporate spatial sound into their own research and to take the next step beyond the innovative projects described here.

Virtual reality, mixed reality, and augmented reality are often technology-driven. On this point, the recent emergence of affordable head-mounted displays—as exemplified by HTC and Valve Vive, Samsung Gear VR, Sony (PlayStation 4) VR, Samsung-sponsored eye-tracking FOVE, Google Cardboard, and the Facebook-acquired Oculus Rift—signals a mass diffusion of VR-style applications such as games, programs, and visualization. Presumably, tracking, alignment, and omnidirectional capabilities often found in these visual displays will foster similar developments on personalized audio displays.

Considering the range of information that human senses can detect, multimodal systems that present users with a coordinated display are needed to increase realism and performance. In R&D of virtual and mixed-reality systems, the audio modality has been somewhat underemphasized, partly because the “original field” of virtual reality focused on the dominant (at least for some tasks) visual modality. For instance, despite its ostensible ambitions for multimodal interfaces, the Virtual Reality journal is still classified by its publisher as “Image Processing: Computer Imaging, Graphics, and Vision,” whereas Springer’s category “HCI: User Interfaces, HCI and Ergonomics” would probably be a better fit.

The peer-reviewed articles presented in this special issue represent a broad consideration of themes on spatial sound, including basic research papers which we include to help provide a baseline of established scientific data in the field and “application” papers which are not only interesting but can be used to evaluate how well systems with spatial audio perform in realistic scenarios. These papers will be especially useful for those who design systems for real-world and pragmatic applications.

Representing expanding interest in the field, the papers presented in this special issue come from innovative researchers in Asia, Europe, and North America, with a focus on recent advances in spatial “virtual” sound, including spatialized audio interfaces, perception, presence and cognition, navigation and way-finding, and applications of spatialized sound for VR, AR, mixed reality (MR), and presence. A deeper review of many of these topics can be found in a just-published anthology of augmented reality and wearable computers, edited by one of the special issue guest editors (Barfield 2016) and including contributions about spatial sound and augmented audio reality by the other two (Michael Cohen and Julián Villegas).

The first paper presented in the special issue, by Sungyoung Kim, Richard King, and Toru Kamekawa, “A Cross-Cultural Comparison of Salient Perceptual Characteristics of Height-Channels for a Virtual Auditory Environment,” describes a series of experiments showing the effect of cultural background on preferences among various spatial soundscapes displayed through speaker arrays. The paper provides data which is useful for the design of spatial sound systems, and comparative data for different user populations. Further, it is well known that user testing can be used to validate the importance of various aspects of spatial sound in virtual and mixed-reality environments. On this point, the article “Creation and Calibration Method of Acoustic Models for Historic Virtual Reality Auralizations,” by Barteld Postma and Brian Katz, details efforts to align in situ measurements and auralization simulation results of a historic church in Paris, including a methodological calibration procedure for room acoustic models using geometrical acoustical simulation software to recreate realistic auralizations. In another study to validate the usefulness of spatial sound in VR, the paper, “On the Validity of Virtual Reality-based Auditory Experiments: A Case Study about Ratings of the Overall Listening Experience” by Michael Schoeffler, Jan Lukas Gernert, Maximilian Neumayer, Susanne Westphal, and Jurgen Herre, presents a series of experiments to characterize the perception of spatial sound in real and virtual spaces, including an actual and simulated listening booth and cinema, as simulated via an Oculus Rift.

The paper by Julián Villegas, “Locating Virtual Sound Sources at Arbitrary Distances in Real-Time Binaural Reproduction,” describes the implementation of a binaural rendering system featuring sonorealistic modulation of distance. The data presented in Villegas’ paper can not only be used to analyze a rendering system, but presents basic data on the perception of spatial sound displayed via headphones.

Another paper which presents useful data on spatial sound, “Sound Localization on a Horizontal Surface: Virtual and Real Sound Source Localization,” by Jonathan Lam, Bill Kapralos, Kamen Kanev, Karen Collins, Andrew Hogue, and Michael Jenkin, details an experiment that was run to compare localization via horizontal and vertical spatial sound displays around tabletop computers, including comparison to real-world sources located at various positions within such a surface workspace.

Spatial sound in VR and AR is not only presented to increase presence, but to improve performance. In “An Experimental Study of Spatial Sound Usefulness in Searching and Navigating through AR Environments,” Dariusz Marek Rumiński describes an augmented reality authoring environment with which a proof of concept featuring spatial sound for navigation is deployed using an XML-based scene description. Finally, the paper, “Intermodal Audio-Haptic Intermodal Display: Improvement of Communication and Interpersonal Awareness for Collaborative Search Tasks” by Mehdi Ammi and Brian Katz, introduces an intermodal system (haptic and auditory) that can be used by two-person teams to collaboratively explore 2D datasets, using sonification and “tactilization” to display simultaneous parallel samples.

The above papers contribute to the basic idea of stimulating more than the visual modality in VR, AR, and MR environments, and we believe that the future for such displays is even better integration of the visual, haptic, and auditory modalities, with low-cost and accurate gustatory, and olfactory displays a future possibility.

We thank the referees for their selfless contributions, anonymously writing thoughtful reviews full of substantial suggestions and constructive criticism. All the papers in the special edition benefited from expert advice provided by the reviewers, and each paper is the product of multiple review cycles, which significantly improved the original submissions, including refinement of high-level organization, tightened reasoning, verbal expression, and even more informative figures.

Further, the guest editors thank the Virtual Reality Journal for recognizing the importance of publishing a special issue on spatial sound, and we particularly thank the journal’s Editor-in-Chief, Daniel Ballin, for his interest, valuable input, and keeping the special issue focused and on schedule. Lastly, we thank the authors for their submissions and timely responses to reviewers’ comments.