Multimedia content has the potential for significant impact on users’ emotions, their sense of presence and engagement experiencing the service, application or information being provided, in immersive environments.
The evolution of technology, user expectations and results from research activities have led to an enormous increase in the amount of content delivered in different formats, via a number of heterogeneous communication networks, to a range of devices, many of them portable and offering tremendous opportunities for immersion, user participation and personalization.
New paradigms for media production, distribution and consumption have been emerging, introducing different sensory modalities and audio-visual surround effects, for an increased sense of presence, and also enabling participation and social interaction in the media chain, thus increasing the sense of belonging and contributing to the success of the services being provided.
Capturing, producing, sharing and accessing information from users’ own perspectives and experiences, over the Internet, in social media, through on-demand services, or even in large spaces like a theatre or a football stadium, introduce many technical challenges. These include the integration of different media formats (high‐definition, 3D, panoramic and multi‐view content, etc.), the ways of constructing different narratives, human‐media interaction mechanisms, or the possibility of delivering and accessing content through different networks or distribution mechanisms.
Many of these topics have been discussed within the Workshop Series on Immersive Media Experiences (ImmersiveMe) that took place at ACM Multimedia from 2013 to 2015. This special issue is a result of initial discussions during these sessions and of contributions of authors with different backgrounds that reflect complementary perspectives on the area.
The articles belong to roughly four categories: The first set of papers deals with the challenge of sensorially-enriching media, exploiting the senses of taste and smell; the second set considers aspects on immersive virtual reality experience, namely in gaming; the third set presents user studies that enable better understanding of QoE (quality of experience) in immersive and multimodal environments; finally, the last one deals with aspects relating to distribution and transmission of immersive 3D multi-view video over the Internet, accounting for QoE.
We summarise the contributions as follows:
The first paper entitled “Virtual Ingredients for Food and Beverages to Create Immersive Taste Experiences: The sensation of taste as an electronic media,” (10.1007/s11042-015-3162-8), by Nimesha Ranasinghe, Kuan-Yi Lee, Gajan Suthokumar and Ellen Yi-Luen Do, introduces a proposal for technology that creates an additional layer of taste sensation that enables modifying the existing taste sensations (salty, sour, and bitter) of food and beverages virtually. It presents two prototypes, Spoon+ and Bottle+, that were tested through a user experiment and discusses the impact of using this technology for making a platform for creating virtual food and beverage ingredients.
Authored by Michael J. Howell, Nicolas S. Herrera, Alec G. Moore and Ryan P. McMahan, the second paper addresses “A Reproducible Olfactory Display for Exploring Olfaction in Immersive Media Experiences” (10.1007/s11042-015-2971-0). It presents three studies that use this equipment in order to analyse the effect of olfaction on users’ sense of presence. One of the conclusions shows that olfaction could significantly affect the effectiveness of a television commercial. Additionally, the paper discusses the effects of smell in training sessions. Initial results indicate that olfaction can both positively and negatively affect experience, depending upon the specific odorant used.
The third paper, by Jongkyu Shin, Gwangseok An, Joon-Sang Park, Seung Jun Baek and Kyogu Lee, and entitled “Application of Precise Indoor Position Tracking to Immersive Virtual Reality with Translational Movement Support” (10.1007/s11042-016-3520-1), proposes an application for immersive virtual reality experiences which integrates three-dimensional (3D) head-mounted displays (HMDs) with a precise indoor position tracking algorithm based on ultrasound. Results show that the system obtains accurate estimates of the positions of users and delivers a highly immersive experience.
In “Augmented Immersion: Video Cutout and Gesture-guided Embedding for Gaming Applications” (10.1007/s11042-016-3435-x), Tam V. Nguyen and Jose Sepulveda present a lightweight framework (Gesture and Appearance Cutout Embedding - GACE) that supports real-time integration of human appearance and gesture-guided control within games. It aims to augment the immersive level by allowing game players to see their personal appearance interacting with other computer game characters.
Iris Galloso, Juan F. Palacios, Claudio Feijóo and Asunción Santamaría bring us their work “On the influence of individual characteristics and personality traits on the user experience with multi-sensorial media: an experimental insight” (10.1007/s11042-016-3360-z). This paper explores the impact of individual preferences and habits, knowledge of technologies, personal characteristics and personality traits on the user experience with multi-sensorial media in the context of enriched sports media experiences. The outcomes indicate that individual preferences and previous habits are determinant factors underlying the potential of sensorially enriched media assets to deliver an enhanced user experience.
The next paper, “Modeling Immersive Media Experiences by Sensing Impact on Subjects” (10.1007/s11042-015-2980-z), by Eleni Kroupi, Philippe Hanhart, Jong-Seok Lee, Martin Rerabek and Touradj Ebrahimi, investigates various QoE-related aspects (depth perception, sensation of reality, content preference and perceived quality) when experiencing 2D and 3D content. EEG and peripheral features are used to automate the analysis. Independent and joint experiences show that decision fusion between peripheral and EEG improve performance in some cases.
“Adaptive Delivery of Immersive 3D Multi-View Video over the Internet” (10.1007/s11042-016-3475-2) by Cagri Ozcinar, Erhan Ekmekcioglu, Janco Calic and Ahmet Kondoz discusses an approach for delivering 3D Multi-View Video (MVV) over the Internet using content-aware mechanisms and adaptation methods to improve the QoE. Additional metadata is used for view reconstruction and the tests have revealed an increase in the perceptual quality of the delivered video.
This special issue covers a wide range of topics and technologies that provide the grounds for real immersive media experiences. We hope it will offer a valuable source of reference for researchers, engineers and developers working in the area, and that the readers will find these papers informative and interesting. We believe that, with the fast development of devices and technologies, the growth of the internet of things and the continuous request for new applications and experiences, immersive media systems will play a decisive role in the near future and have an impact in the economy and society.
This fine collection of papers was achieved by fruitful collaborations. The guest editors would like to thank the authors, who have prepared and revised their papers providing worthy contribution to the special issue, and the reviewers who have given their time and valuable advice to the authors. We extend our gratitude to the MTAP Editor-in-Chief Dr. Borko Furht, for having accepted our proposal, sharing and supporting our vision for this special issue, and Springer editorial staff for their technical and administrative assistance.