Keywords

1 Introduction

Collaborative Virtual Environment (CVE) supports multiple collaborators either co-located or geographically distributed to work in a shared environment to achieve a common goal. However, CVE has mostly been used in a co-located space for a collaborative task [1,2,3,4,5]. With the availability of several resources, a collaborator wants to use hardware system for remote collaboration that is readily available to them. But currently available collaborative frameworks have been designed targeting few specific VR systems for remote collaboration and scaling of those collaborative frameworks for new and developing systems are difficult [6]. CVE with the flexibility to support diverse VR systems to visualize different 3D application for remote collaboration will help collaborators to be independent of VR system. They can use multiple VR systems to utilize them to their full potential and support each other for better understanding and sharing of information during collaborative tasks.

Our CVE provides a collaborative framework that has a simple and easy interface to integrate existing, new and emerging virtual reality system for collaborative task across geographically distributed collaborators and share their virtual workspace for achieving a shared goal. This CVE, which supports remote collaboration, is independent of VR systems. It combines existing algorithms and technologies in a novel way to provide collaborators with a collaborative framework that supports heterogeneous virtual reality systems across different geographical location.

This CVE has been tested and supports multiple users across diverse VR system for remote collaboration. The dataset used for collaboration is loaded in each collaborators system, and the system only sends and receives the changed information about the data and virtual environment so that it helps to decrease the network congestion, bandwidth problem, and latency during collaboration. It provides an additional layer of security with authentication system to avoid anonymous users to connect to the collaboration server. It also supports the use of 3D video avatar or web camera for video communication and furthermore, supports audio communication while collaborating across geographically distributed collaborators.

In this paper, we present the implementation details and features of our CVE for remote collaborators, which is independent of VR and tracking systems that can be used for multiple different 3D applications quickly and efficiently. Furthermore, we present the pilot study that we collected from researchers and scientists to evaluate the effectiveness and gather feedback on the usability of the CVE.

2 Background and Related Works

Uni-CAVE [7] is an add-on package for Unity3d software, which is freely available. It supports cluster-based VR displays, device tracking, and display synchronization but does not support remote collaboration. We have used Uni-CAVE to support the use of Unity3D in CAVE and IQStation. Unity Networking (UNET) [8] is used for networking between remote collaborators to supports collaboration. It provides us with high-level scripting so that we do not have to be worried about the low-level implementation. UNET uses network manager to control the state of the networked system to make remote procedure calls (RPCs) and networking events from the server to clients, message handlers and it uses a general-purpose serializer to serialize the data when sending through the networks. Virtual Reality Peripheral Network (VRPN) [9] is used as to map and provide an interface to connect the interaction and tracking system with the collaborative framework.

There has been growing work in integrating different VR system for remote collaboration. DIVE [10] has been used to support remote collaboration across desktop, head-mounted-display or projected displays. It also supports networked communication with the use of audio, video, whiteboards, text chat and simple graphical gestures. However, in our CVE it adds more existing and emerging VR systems to supports remote collaboration like smartphones, tablets, tiled walled display to increase the scope of the collaborative framework. Furthermore, this CVE supports 3D video avatar to provide face-to-face communication for an increased sense of presence while working with remote collaborators. MASSIVE-2 [11], which uses the third-party objects, provides a teleconferencing platform across the remote network, which also supports awareness between objects and environments. This CVE is different because it provides a platform to support authentication system for increased security level while joining the collaboration, remote collaborative data visualization to support large 2D or 3D datasets and furthermore adds support of 3d video avatar for video communication. SPLINE [12], which is a scalable platform, supports large interactive network platforms. It has not been used and tested with the handheld devices like smartphones and tablets. NSPNET [13], which is a 3D networked virtual environment, is built to support large-scale military training and simulation. This system is different because our CVE focuses on providing support for remote collaborative data visualization ranging from non-immersive to immersive display systems. CALVIN [14] supports multiple remote users for architectural design and collaborative visualization using large display systems like CAVE and ImmersaDesk and CAVEvis [15] also supports the CAVE-like display for distributed real-time visualization. However, our CVE is different from these platform as it supports large interactive to small handheld VR systems for remote collaborative data visualization in real-time. NICE [16], which is a collaborative virtual environment for young children, which presented a virtual island where the remote collaborator was presented in avatars and can use CAVE, ImmersaDesks or desktop computer. However, this NICE didn’t support the use of other available semi-immersive and non-immersive VR system for collaboration which is supported by our CVE.

There has been increasing research work in the field of remote CVE for shared virtual workspace and data visualization using a heterogeneous VR system. It will help a collaborator use these tools and features available for collaborative visualization in a fully immersive display (CAVE) to support other collaborators using semi (IQStation) or low (Desktop, smartphone, and tablets) immersive display system to work collaboratively or vice versa. It will provide an effective and efficient platform to support the collaborative work across distributed users in geographically separate location.

3 Collaborative Architectural Framework

This collaborative architectural framework has been developed using Unity3D (version 5.5.2f2) [17] in C# language. Unity Networking (UNET) is used to support the networking across geographically distributed collaborators. Uni-CAVE, an add-on package, is used to support the use of Unity3D with CAVE and IQStation (Fig. 1).

Fig. 1.
figure 1

(Left) Collaborative architectural framework. (Right) Collaborators using a. CAVE, b. IQStation, c. Laptop, d. HTC Vive, e. Desktop for working in heterogeneous VR system for collaboration

Full size image

This collaborative framework comprises of a master client, which acts like other default clients but initiates, manages, maintains, and interacts during collaboration. This collaborative framework provides an added layer of an authentication system for security. Each collaborator when connecting to the collaboration server needs to provide the server name and authentication key, which is 32 bits generated using a random string, to restrict any anonymous user connect the server.

The data used for collaborative data visualization is loaded in the respective collaborator’s VR system, i.e., each collaborator will have a copy of the data being used for collaboration. The reason behind having a copy of the data in each collaborator is to reduce the load on the server. Only the changed information of the data is sent and received across the network, which will help to reduce the network congestion. The viewpoint of the collaborator is data-centric, i.e., the collaborator’s position in the virtual space is relative to the data. The master client initially has control over the shared data used for visualization. However, any client can request for control over the data while working collaboratively. Any changes made to the shared data will be cloned to all the clients connected to the collaboration. The master client is responsible for sharing the changes made to the shared data between the clients. The data used for visualization is stored as a Unity3D prefabs [18].

This collaborative framework also supports video and audio communication to help collaborators while working in geographically distributed locations. This framework supports the use of Kinect V2 camera or a web camera for video communication. Kinect V2 camera is used to generate 3D video avatar of a collaborator and is sent to the collaboration server as a point cloud. It is the responsibility of the master client to send the 3D video avatar information to the other clients connected to the server. This Kinect V2 avatar will provide collaborators with face-to-face collaboration, which will help to increase the sense of presence. Currently, we support the use of single Kinect V2 camera for each collaborator to create 3D video avatar. This framework also supports the use of web camera to support 2D video communication. Furthermore, this framework also supports audio communication using microphones. It uses.NET sockets to transport the sound data over the network. These sockets will use P2P multicast to transport the sound from one collaborator to all other connected remote collaborators.

This framework has implemented a class method, which receives information from the VRPN, which helps to connect the devices and systems and then map all the buttons of the interaction devices for respective VR system used by the collaborator. In the current implementation of this collaborative framework, this has been tested with flystick2 [19], flystick3 [19], Wiimote [20], HTC Vive Controllers, Gamepad Joysticks, and Keyboard/mouse.

4 Experimental Study

We conducted a pilot study (N = 15, #Groups = 7), with mixed experimental design, with the type of VR system as a between-subjects condition and the combination of the VR systems as the within-subjects condition to evaluate the usability of CVE with groups of participants on a collaborative task. In within-subject conditions, we compared the use of uniform VR system with heterogeneous VR system for collaboration. In addition, in the between-subject conditions, collaborators for uniform VR system were assigned desktop-to-desktop system or IQStation-to-IQStation for collaboration randomly. Similarly, for heterogeneous VR system we had CAVE to HMD, CAVE to Desktop, CAVE to IQStation, Desktop to HMD, HMD to IQStation, IQStation to Desktop system randomly assigned to participants for collaboration. We collected qualitative data through questionnaire and interviews with the individuals of the groups to evaluate the usability of the CVE.

4.1 Apparatus

HTC Vive used has a resolution of 1080 × 1200 pixels per eye. It has a refresh rate of 90 Hz and a field of view (FOV) of 110 degrees. Kinect V2 used had a resolution 1920 × 1080 (16:9) pixels, and the infrared camera had a resolution of 512 × 424 pixels. The CAVE used for the collaboration is a four-walled display system. Its dimensions is a 12 × 12 × 7.5-foot cube, which uses rear-projection to display computer graphics on three walls and front projection on the floor. IQStation uses a 3D projector to project on the screen. This Panasonic projector has a resolution of 1200 × 720 pixels. It uses a low-cost tracking system, i.e., smarttrack [21]. Wiimote is used as an input device to interact with virtual objects and environment.

4.2 Experimental Data and Task

We have used graphite billet (shown in Fig. 2, left) as a data set for collaborative immersive data visualization and analysis. We conducted an informal discussion with domain scientists to understand what different type of datasets they explore in immersive visualizations. We designed the collaborative task for participants in a group to sample the highest variation of density level of graphite, which is shown in the dataset using change in color gradient. Each collaborator is assigned to a cube randomly (shown in Fig. 2) and each collaborator has to use a cube for sampling the highest variation of density level. This cube can be manipulated and translated to determine the correct region in the dataset for sampling. The collaborator who initiated the server will have the control over the shared dataset first. This owner can manipulate the dataset for finding the correct region for highest change in density level, and this manipulation (i.e., change in position and orientation) of the dataset will be across the entire collaborator’s environment. The collaborators will work together to determine the highest change in density level and have their cube translated to that position. Collaborators can communicate with each other using Kinect 3D Video Avatar/web camera for video communication and use headphones for audio communication.

Fig. 2.
figure 2

(Left) Graphite Billet dataset rendered in the system for the experimental study. (Right) A collaborator working collaboratively and controlling red cube to sample the highest change in density value in the graphite billet (Color figure online)

Full size image

A collaborator can choose any one of the cubes available in the environment in mutual consent while working in the collaborative environment. These cubes are uneven in size to encourage collaborative task by helping each other to sample the best density variation. Therefore, each collaborator will have his/her cube for sampling the dataset. Initially one of the collaborators who initiated the collaboration server will have the control of the graphite data set.

Moreover, during the collaborative task, if anyone of the collaborators who wants to have the control of the data set for manipulating the data, he/she can easily do so by pressing a button available to them in their interaction device and then get control of the dataset. Since this task is a collaborative task, it will be completed when both collaborators agree that they have the best-sampled density information from the dataset.

5 Results and Discussion

Participants completed a post-questionnaire to provide feedback on the usability and willingness to use the provided CVE for their collaborative work with geographically distributed collaborators. The graph (see Fig. 3) shows the desire of the participants to use the provided collaborative framework in future for their collaborative tasks. It suggests that majority of the participants were willing to use this system for remote collaborative work. 11 participants were ready to use, 4 participants could use, and there were not any participants who did not want to use the system in future. This result suggests that if we provide collaborators with a collaborative framework which is independent of any VR systems and is easy to use will encourage them to incorporate the system into their research work for more effective and efficient collaborative task. Furthermore, we also collected qualitative data about the usability of the system. This data will suggest how helpful the collaborative framework to complete the collaborative task effectively and efficiently was. One of the participants commented “Interested in using as a tool for data visualization in a user facility environment.” Another participant commented that “Use of different systems allowed quicker movement for both the individual and the team.” and furthermore, another participant commented “Going from the 2D desktop environment to the 3D virtual environment made it incredibly easy and incredibly fast to look, share, and discuss the data set we were looking at. In the future, I will definitely prefer the 3D virtual environments using different virtual reality system for collaborative work.” This feedback from participants suggested that the CVE provided an effective and efficient platform for the collaborative task and they are encouraged to use this system for their work.

Fig. 3.
figure 3

Collaborators feedback on using this CVE for remote collaboration

Full size image

6 Conclusion and Future Works

We presented our collaborative virtual environment that supports geographically distributed collaborators using heterogeneous VR systems for collaborative immersive data visualizations. Our CVE provides collaborator with a framework, which is independent of VR systems thus encouraging researchers and scientist to collaborate their work without any difficulties remotely. In our pilot study, we remotely collaborated with collaborators to work on real-world datasets. This study suggested that our CVE has the potential to increase the workflow, task effectiveness, and provide a better user experience while remotely collaborating across heterogeneous VR systems.

In future, we will evaluate the performance of our CVE across broader audience using diverse VR systems. In addition, we want to assess the network congestion, bandwidth, and latency while collaborating remotely with several collaborators who are using different VR systems. Furthermore, we also want to add interaction and rendering techniques to support visualization tasks to increase the scope of the CVE.