1 Introduction

Recent developments in Extended Reality (XR) technologies, such as wearable Head Mounted Displays (HMD) for Virtual Reality (VR), Mixed Reality (MR), and Augmented Reality (AR), have yielded affordable hardware for experiencing immersive virtual environments in multidimensional space. 2D displays are limited to showing content on a screen, but XR applications can embed a user in his or her digital information, utilizing a 360-degree view, perceived depth, physical location, and movement tracking to expand what can be experienced in a digital application. Furthermore, continued development in this space has seen additional capabilities, such as haptics, spatial audio, hand tracking, improved sensors, etc., be continually added to XR platforms.

While XR provides increased potential for interacting with digital information, it is important to remember that these advancements require an expansion of our current understanding of UX design. Established patterns and guidelines are built around applications running on a 2D screen and do not address the potential additional considerations that come with spatial environments and interactions. As long as existing design lessons are not extended for the capabilities of XR, designers likely face barriers to unlock the full potential of these technologies.

Recent works from various communities, such as device manufactures, developer blogs, news sites, and academic sources, can be found that provide ideas around designing for XR platforms. These resources vary in content, in supporting evidence, and in format, such as first-hand experiences, proposed guidelines, speculative ideas, prototypes, etc. This provides a diverse wealth of information; however, it also makes it difficult to identify valid patterns that can be beneficial for designers.

Inspired by this work, we developed a set of UX guidelines for designing XR applications that aim at integrating various ideas from different XR and UX communities. We used an approach adapted from Rusu et al. [22] and Quinõnes et al. [21] to formulate, specify, correlate and refine a set of design guidelines. A total of 60 web resources, 1 peered reviewed paper, 1 book, 3 non-peered review academic sources, and 3 traditional UX sources, were analyzed and used to derive our guidelines. Our work is targeted towards aiding in the design of applications for wearable HMD XR devices, such as the Microsoft HoloLens, Magic Leap, Meta, HTC Vive, Oculus Rift, etc., that uses stereoscopic imaging, spatial audio, and head tracking to provide users with an immersive experience. This paper is not meant to present absolute rules, but rather to provide in a concise manner a reference for understanding what others have tried and discovered.

Our work makes three contributions. First, we provide a guided overview into the existing literature on UX for XR applications in academic research and in industry. Second, we propose a set of guidelines that are encompassing the body of work in a concise and structured manner. Third, based on our guidelines, we make statements about which topics have been emphasized, which topics are not well covered, and areas of opportunities for further research.

2 Background

2.1 Extended Reality

As the industries around these platforms develop, so does the terminology. Different definitions have been used in this field, but there have been distinct categories of technologies that have emerged in recent years. For the purpose of this paper, we will use the following terminology to classify Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and Extended Reality (XR). VR allows users to be fully immersed in a computer-generated environment while the real world is occluded when the device is in use [14]. AR allows for information to be superimposed around a user without blinding them to their own physical surroundings [25]. Similar to AR, MR overlays information onto the real world, but also includes the capability to understand and use the environment around the user to show or hide part of the digital content [24]. Building off Milgram et al. [17] work on classifying the reality-virtuality continuum, we updated his terminology to define XR as technologies in which real world and virtual world objects are presented within a single display. As such, XR encompasses all the previously mentioned categories.

2.2 Guidelines

Consistent with Nielsen [18], design guidelines are well-known principles for user interface design which should be followed in the development project and that can also be used to evaluate the usability of a system. According to Endsley et al. [5], in user interface design, the term guideline is almost synonymous with heuristics, and can be used by both designers and evaluators. Certainly, the most well-known design guidelines/heuristics are the ten usability heuristics from Molich and Nielsen [20]. Like psychological heuristics, usability guidelines are often used as shortcuts when data from a formal user study is not available. Guidelines can be used across the stages of the design process, guiding small and large decisions and predicting success or failure of the usability in prototypes with varying levels of fidelity.

Several authors have stated that it is necessary to create new guidelines for evaluating specific applications, for example, in [16], the authors conclude that the existing usability guidelines were too general to be applicable for evaluating the usability of mobile map applications. Thus, it is necessary to develop a new set that evaluates the specific features of that domain.

2.3 Related Work

The most recent and relevant study for our purposes is [5]. Endsley et al. [5] aimed at providing practitioners with a set of AR heuristics. They developed their heuristics by making use of affinity diagrams, expert evaluations, feedback from active AR designers, and statistical analyses. According to these authors, the literature review revealed few validated, generalized AR heuristics in use. The authors used a set of 11 heuristics, including related fields, such as: [1, 4, 6, 8, 9, 12, 14, 15, 19]. Their affinity diagramming approach sought to leverage the existing heuristics in the AR space and generating themes for immersive AR interactions.

Wheeler Atkinson et al. [1] proposed the MHET (Multiple Heuristics Evaluation Table), which describes an approach to integrate existing approaches chosen by the authors in a single table. Their approach also sought to enhance these approaches by addressing existing gaps and providing concrete examples that illustrate the application of concepts. Although providing valuable insights, this work is not directly related to ours.

Dunser et al. [4] also investigated how general HCI guidelines may relate to the domain of AR application design. To do so, they combined some known user-centered design guidelines – [18, 27, 30, 34, 37] – with the demands of AR systems to identify issues that should be considered by AR interface researchers. According to the authors, this work was an initial attempt to fill the gap that existed in the area and the presented design guidelines were just a small overview and the guidelines given were rather general and had to be further refined.

Ko et al. [14] proposed the creation of usability guidelines for the development and evaluation of smartphone applications using AR technology. The authors developed these guidelines by analyzing existing research about heuristic evaluation methods, design guidelines for AR systems, guidelines for handheld mobile device interfaces, and usability guidelines for tangible user interfaces.

Kourouthanassis et al. [15] proposed a set of interaction design guidelines for the development of Mobile Augmented Reality (MAR) applications. According to the authors, the design recommendations adopt a user-centered perspective and, thus, they focus on the necessary actions to ensure high-quality MAR user experiences.

In addition to those mentioned above, we also identified non-peer reviewed studies, such as a master’s thesis from Kalalahati [12], another master’s thesis from Bloksa [2], and a bachelor degree project from Frojdman [7], that discuss research and guidelines related to XR applications.

The reviewed work present interesting, novel and useful approaches for the purpose of this paper, and our work ultimately builds on the ideas and information introduced by the literature. With that said, these works also highlight the limited academic research and the lack of real-world validation around this topic. Additionally, observations from other communities, such as XR device manufactures, news articles, and developer’s blogs, has shown that there is a wealth of knowledge that was not being reflected in academic works. Even though not all of the information or ideas found were academically validated, they were concepts that developers were using to build and design XR applications. This suggest that non-academic sources provide additional insights and should be incorporated in the development of guidelines. Our eleven UX guidelines extend to include the wealth of insights from other communities in conjunction with academic work. As such, concepts that developers have used to build and design current XR applications are being reflected in our research that were not in prior works.

3 Research Method

Previous work in the creation of heuristics have stated the importance of using a formal process to develop an effective and efficient set of heuristics [21]. As such, we used an approach by Rusu et al. [22] and Quiñones et al. [21] as the bases for creating our set of proposed guidelines. The authors describe stages to be followed to formulate, specify, correlate and refine a new set of heuristics. These stages were adapted and applied to fit within the context of our situation and research goals.

3.1 Conducting Search for Resources

In the first stage, a search was conducted to obtain existing literature from different sources regarding design guidelines for XR applications. To achieve this, we defined a search string, following systematic literature reviews (SLR) recommendations [13], that was iteratively reworked to accurately reflect the scope of the research. The resulting search string was:

(heuristic OR principle OR guideline) AND (“virtual reality” OR “augmented reality” OR “mixed reality” OR immersive) AND (usability OR “user experience”) AND (design). The search string was then applied to search title, abstract, and keywords in ScopusFootnote 1, a database containing a variety of peer-reviewed papers published in either conferences or journals, which resulted in 350 papers found. Each paper was screened by two researchers with a third providing feedback to exclude studies that are not relevant to our topic. The criteria we used was based on the abstract explicitly mentioning dealing with HCI guidelines, heuristics or guidelines for VR, AR or MR applications and, from the abstract, being able to deduce that the focus of the paper has relevance to the creation of HCI guidelines for HMD XR design and/or evaluation. Through this process, only one paper (Endsley et al. [5]) fit in our inclusion criteria.

To broaden our resources, we conducted a search to find different information sources, such as scientific articles, thesis, previous experiments, books, and websites [21] outside of peer-reviewed databases to include in our research. We started our search for resources throughout the internet by applying multiple variations of our initial search string on the Google search engineFootnote 2. This search resulted in a large number of related resources. After a quick overview, using backward snowballing as a complement to SLRs [26], we selected resources which focused on providing information regarding UX design for different XR technologies. The collection process continued until the researchers found that the resources continually contained repeat ideas and that new concepts became difficult to come by. We ended up with an additional 60 web resources, 3 non-peer reviewed academic sources, and 1 book which brought our total to 65. The list of resources is likely not exhaustive due to limitations of the search engine/process, however, it represents a large range of resources from different communities.

3.2 Theme Extraction and Thematic Analysis

In order to extract themes from our collection of resources, we choose to conduct a thematic analysis, in which we iteratively grouped, labelled, discussed and re-labelled categories and guidelines that described and explained the various guidelines and recommendations. This process involved several meetings among the researchers.

According to Hawkins [10], researchers conducting thematic analysis look for recognizable reoccurring topics, ideas, or patterns – themes – occurring within the data that provide insight. When a researcher uncovers common themes throughout the data, those themes may indicate areas that help explain phenomena or point out areas of needed improvement. Therefore, information that supports the theme is extracted directly from the data. As the analysis continues, the researcher revisits the data to ensure the understandings extracted within the data contribute to the research questions. A theme indicates a common line of understanding occurring within the data and may contribute explicit or implicit information. Themes may be overarching, providing a general idea under which subthemes exist and contribute to understanding [3].

It should be noted that we have used an Inductive – bottom-up – approach, in which, researchers choose to locate themes inductively, and build themes directly from the data under investigation. No prior theoretical frameworks specifically guided the research project. Therefore, any and all reoccurring themes within the data are under investigation as long as themes align with the goal of the project [10]. Thus, after the thematic analysis process, we ended up with our set of guidelines, containing eleven UX guidelines for XR.

3.3 Correlating Existing Work

After the creation of our initial set of UX guidelines, we correlated our findings with related works. In this step, our goals were to identify the existing heuristics and guidelines that were already addressed in our initial set of guidelines and discover new ideas that can be incorporated into our work [10]. There were two categories in which the resources we used fell under, academic research, found in our initial search, that proposed high level design guidelines surrounding XR applications, such as [2, 4, 5, 7, 12] and traditional UX heuristics and design guidelines such as: [11, 20, 23]. The motivation behind using traditional design principles came from the belief that these resources still provide valid and useful information for designing XR applications despite the differences in medium. In order to find relevancy, two authors separately read each related resource and compared their high-level ideas to our initial set of guidelines. Each was marked as relevant or not relevant to indicate if the ideas were addressed in our work. Afterward, the results were combined. Discrepancies were resolved by looking at the paper together and coming up with a consensus through discussion. In the end, 32 of the existing guidelines were identified as not related by our guidelines. For each of them, we discussed whether to discard, to incorporate with our existing guidelines, or create a new guideline based on the idea. This decision was based on importance, similarities to other guidelines, and whether or not they fit within our scope. For example, through this process, we identified ideas from [4, 11, 20, 23] around error tolerance and prevention that were not being reflected in our work. Eventually, a new guideline was created to incorporate these concepts due to their perceived value and lack of parallels with other guidelines. Through this process, we used three new resources to correlate to traditional UX heuristics and design guidelines which brought our total used resources to 68.

3.4 Refining UX Guidelines

In addition to integrating new ideas, the correlation stage identified aspects that needed to be improved. For example, we were able to find ambiguous parts through the different interpretations and discrepancies among the researchers. Based on the insights and new ideas until this point, we refined our initial set of guidelines. This process involved several meetings among the researchers around regrouping, merging, rewording, and restructuring different aspects of our work. Additionally, based on the structure from [21], we organized our guidelines by providing a name, definition, and explanation with sub guidelines for each guideline. In the end, we had eleven guidelines.

4 UX Guidelines for Extended Reality Applications on HMD

In this section, we present eleven UX guidelines for developing XR applications. They are ordered based on their specificity to the context of HMD XR applications.

4.1 Organize the Spatial Environment to Maximize Efficiency

Definition. XR is inherently spatial. Use space as an organizational tool to create an environment that is comfortable to use and minimizes the amount of conscious thinking a user has to do to accomplish his or her goals [7, 27, 29,30,31, 40, 41, 48,49,50, 52,53,54, 57, 61, 66, 67, 71, 72, 74,75,77, 80, 81, 84].

Explanation. XR can leverage how humans interpret spatial information to free up working memory and create dynamic environments. With that said, it is easy to create uncomfortable experiences depending on how the elements are placed around the user. It is important that the environment is carefully designed to take advantage of the extra space while limiting physical movement for accomplishing a task.

Keep Visual and Physical Restrictions in Mind When Arranging Content. When positioning content, it is important to remember that users have physical limitations and a limited field of view. Place visual elements in areas where users’ can comfortably view for long periods and interactive elements in areas where interaction is relaxed [7, 27, 29, 31, 40, 41, 48, 50, 52,53,54, 57, 66, 67, 74,75,76, 80, 81, 84].

Explore How Space Can Be Utilized. XR provides users with a larger area for placing and interacting with content. Depending on the context and type of application, designers should explore how the additional space can be used to avoid cluttering of objects and information. For example, the Windows Mixed Reality Home application in Fig. 1 lets users place apps and content in a virtual house similar to a 2D desktop. It showcases how a 3D environment can be used to spread out content to help with organization, multitasking, and visual appeal [37, 52, 53, 61, 71, 72].

Group Similar Objects to Make Them Easier to Find. Placing similar objects together can use attention chaining behaviour to conceptually link application features together. This can help users find digital objects and information more efficiently and effectively [30, 53, 71, 77].

Fig. 1.
figure 1

The picture is of Windows Mixed reality home. The application lets users customize and place content throughout a 3D virtual house.

Full size image

4.2 Create Flexible Interactions and Environments

Definition. Provide users with the capability to customize the application to their personal preferences and comforts. Build in options that cater to a range of users that take into account different experience levels and physical considerations [2, 11, 20, 23, 48, 50, 52, 73].

Explanation. It is important to build in features that makes the XR application accessible to a wide range of individuals. By giving users some control over the physical placement of digital elements, they will have opportunities to improve the overall satisfaction and ease of use for themselves. This is even more important for XR as the users are exposed to more factors that affect the overall experience than 2D displays.

Build in Interactions for Both Inexperienced and Experienced Users. The applications should have options that cater to users of varying level of experiences. Create cues for interactions for novice users and options to accelerate actions for more advanced users [2, 11, 20, 23].

Let Users Shape Their Environment to Optimize Their Workflow. Customization can be used to improve efficiency. Provide users with options to personalize the environment based on their preferences. For example, users can place frequently used content in their immediate line of sight and change them depending on the situation [50, 52, 73] (Fig. 2).

Let Users Define What it Means to Be Comfortable. Factors that affect comfort like personal boundaries, physical limitations, social considerations, brightness, etc. should be flexible. The designer should identify potential aspects of their application that could negatively impact the user and build in options to adjust them in the settings [48].

Fig. 2.
figure 2

Picture from Google’s Tilt Brush VR application. Shows how the menu can be altered based on user preference by allowing parts to be grabbed and placed in different areas of the menu.

Full size image

4.3 Prioritize User’s Comfort

Definition. The XR application should keep the user safe by taking extra precautions to maintain the physical, physiological, and environmental comforts for the user throughout the experience.

Explanation. Being immersed in a 360-degree environment opens the user up to a lot of different factors that impact users’ comfort. Avoid putting users in situations where potential distresses might arise and provide options for users to adjust settings related to comfort [27,28,29, 31, 33, 34, 36, 40,41,42, 44, 46,47,50, 52,53,54, 57, 59, 66, 67, 74,75,76, 80,81,84, 86].

Respect Users’ Personal Space. People are both physically and mentally sensitive, especially around the head, to objects that are placed too close to them. Make sure objects are defaulted to a comfortable distance away from the user and allow them to dictate what happens within their personal space [27, 29, 42, 48, 50, 52, 53, 59, 66, 75].

Physiological Considerations. Due to the fact that XR provides users with an immersive experience, users might experience discomfort due to the disparity between what one feels and what one expects to feel. It is the designer responsibility to take extra precautions to reduce the chances of users experiencing motion sickness. OculusFootnote 3 and googleFootnote 4 have a set of additional guidelines for designers to help mitigate this effect [28, 31, 33, 34, 40, 44, 46,47,48,49, 53, 57, 59, 75, 76, 80, 82, 83, 86].

Environment Comfort. Certain users can experience various discomforts in certain situation like heights, small spaces, etc. Provide options for users to change or adjust the environment if they are uncomfortable [36, 40, 47, 50].

Be Mindful of Physically Draining Interactions. XR applications provide designers with the opportunity to create 3D interactions. Be cautious of the fact that 3D interactions can be physically exerting for users especially over long periods and repeated repetitions. Additionally, where content is placed also dictates the type of physical interactions that are required. For example, content placed outside a user’s field of view requires them to turn their body or head to view or interact with the object [27, 31, 40, 47,48,49, 52, 53, 57, 66, 74, 81, 83] (Fig. 3).

Fig. 3.
figure 3

Microsoft proposed allowable field of view determined by the range of motion of the neck. The picture was adapted from Microsoft blog on mixed reality comfort (https://docs.microsoft.com/en-us/windows/mixed-reality/comfort).

Full size image

4.4 Keep It Simple: Do Not Overwhelm the User

Definition. The more there is, the less the user remembers. Create simple and relevant elements in an environment that do not distract the user from what is important.

Explanation. There is a fine line between enhancing an experience through additional information and overwhelming the user. XR provides more virtual space but the user still has a limit to how much information she can consume. Designers should focus on minimalism as irrelevant information competes for user attention [2, 31, 42, 43, 47, 48, 50, 52, 53, 59, 75,76,77].

Keep Tools and Information Ready, but not Distracting. Designers can provide a lot of information, but they should be implemented in a way that does not distract users when they do not require them. Provide users with the ability to hide, minimize, or turn off elements. Additionally, the element should be visually quiet while inactive as to not take the focus away from other tasks [2, 31, 43, 48, 50, 52, 53, 77] (Fig. 4).

Don’t Obscure the User’s Vision with Virtual Elements. Minimize the density of information within a user’s field of view and be wary of using persistent heads-up displays (HUD) that takes up a large portion of a user’s view [42, 47, 52, 53, 59, 75, 76].

Fig. 4.
figure 4

Microsoft Hololens can toggle between showing and hiding the main menu through a bloom gesture.

Full size image

4.5 Design Around Hardware Capabilities and Limitations

Definition. The way users interact and explore the environment will be greatly dependent on the system they are using. Always keep the capabilities of the hardware in mind when crafting XR experiences [27, 28, 50, 83].

Explanation. Currently, there is a lack of standardization across XR devices. VR, AR, and MR offer different experiences that require different design considerations. Additionally, there is a lot of variety within these categories regarding input type, tracking limitations, device features set, etc. As such, it is important that designers understand the medium and build an experience that suits its capabilities.

Sensitive to the Capabilities of the Hardware. Applications should not include features that go beyond the limits of the hardware. The designer should be selective and choose features that comfortably stay within the capabilities of the system. For example, the Microsoft HoloLens has a field of view (FOV) of \(35^{\circ }\) while HTC Vive has a FOV of 110\(^\circ \). This difference affects how much content can be placed within a user’s immediate line of sight. As such, the size and placement of visual elements should be adjusted based on the targeted system of the application [27, 28, 83] (Fig. 5).

Use the Strengths of the Medium. It is important that designers focus on features that highlight the strength of the given hardware in their application. Showing off the advantages of the system will minimize the impact hardware limitations have on UX [50].

Fig. 5.
figure 5

Picture showcase a range of different HMD XR product. From left to right: HP windows mixed reality headset with inside-out tracked controllers, Meta 2 with built-in hand tracking, Microsoft Hololens with gesture controls, HTV Vive with base station tracking. All these devices have differences that should be considered when designing for them.

Full size image

4.6 Use Cues to Help Users Throughout Their Experience

Definition. Create signifying cues to help users to get started, provide additional information, guide user’s attention, and simplify choice within the application [2, 20, 28, 30,31,32, 36,37,38,39,40, 48,49,53, 59, 61, 65, 66, 76, 81,82,83, 85].

Explanation. The expanded capabilities of XR also comes with the greater potential of users being overwhelmed or lost. Designers should take extra measures to make sure that they provide enough guidance throughout the applications to keep frustrations to a minimum.

Use Attention Directors to Help Users Discover What They Can Do and Where They Should Go. Provide the user with directional indicators to point them to important content in the application. This is especially vital for XR as the area in which content can be placed is much larger than traditional 2D application [28, 30, 36, 39, 40, 49,50,51,52,53, 59, 65, 76, 81, 82, 85].

Simplify Choice. Use cues to explicitly inform users what they can do next especially if the steps are not obvious. This can help reduce hard cognitive thinking and ease frustrations throughout the experience. For example, Fig. 6 illustrates how The Lab uses a yellow arrow to explicitly inform the user of a possible action that she could take [30, 32, 37, 38, 48, 49, 52, 61, 66, 83].

Do not Overload the User’s Senses with Notifications. Be mindful of the fact that users can be easily overwhelmed by an excessive amount of information, especially in XR. When designing cues, not only think about delivering the information but also how it can be conveyed in an efficient and comfortable manner [2, 52].

Use Cues to Integrate Help and Tutorials in the Experience. It is important that users have access to help when they need it. Instead of just providing external documentation, think about how different cues can be used to integrate help into the XR application to create a seamless experience [2, 20].

Inform the Users of Actions That May Result in Errors. The best way to deal with an error is to prevent it from happening in the first place. Use cues to inform users of the potential dangers and consequences of an action before it is executed [20].

Fig. 6.
figure 6

Picture from Valve’s The Lab: Robot Repair. Yellow cues are placed throughout the experience to inform the user of the actions that need to take place in order to progress. (Color figure online)

Full size image

4.7 Create a Compelling XR Experience

Definition. XR allows users to be immersed in the virtual environment. Enhance their senses through visuals, audio, and narrative elements that captivate them in the experience [28, 34, 36, 42,43,44, 46, 48,49,50, 52, 53, 55, 60, 61, 65, 66, 73,74,76, 78,79,83].

Explanation. Users senses are more exposed when immersed in XR applications. This presents additional challenges for designers, but it also offers an opportunity to elevate the experience. Explore how aesthetic elements can be used to captivate the user and provide them with an application that they want to use continually.

Make the Experience Appealing. Add visual elements that make viewing and interacting with the application enjoyable [49, 50, 52, 55, 61, 65, 82, 83].

Use Audio to Enhance the Experience. In the real world, sound plays an important role in how people interact and understand the space around them. Use spatial sound to help users feel situated, suggest spatial relationships through feedback, and create ambiences to immerse users in the virtual environment [34, 36, 42, 44, 46, 48,49,50, 52, 53, 61, 62, 66, 73,74,75,76, 79,80,82].

Make a Complete Experience. The more comprehensive the environment, the more it resonates. XR provides the tools for immersion, but it is up to the designers to build in details to maintain the feeling of being in an environment the user know is not completely real [28, 35, 42, 49, 50, 60, 61, 79, 80, 82, 83].

4.8 Build upon Real World Knowledge

Definition. Help users to understand how to use the application by designing the interactions, objects, and environments around existing knowledge of the real world [27, 28, 31, 32, 36, 37, 42, 48,49,50, 52, 55, 58, 62,63,64, 66, 68,69,70, 72, 77, 79].

Explanation. People have prebuilt mental models on how objects should behave based on prior real-world knowledge and experiences. XR designers can capitalize on these familiarities by using aspects of the real world as inspirations for designing virtual environments. This can help lessen cognitive load and educate users without being explicit.

Use Real-Life Inspiration to Create Affordances in Objects. Affordance refers to the properties of an object that informs the user how it should be interacted with and used. Build in virtual elements which reflect properties from real objects that hint at how users should use certain elements. For example, Fig. 7 shows how Weightless, a demo from Leap motion, used properties of a bowling ball to help inform users on the correct way to grasp certain objects in the application [27, 31, 36, 37, 48,49,50, 52, 55, 58, 63, 66, 68,69,70, 72, 77] (Fig. 8).

Pair Actions with Outcomes That Users Expect. Keep in mind that certain visual characteristics may affect users’ expectation of how certain elements should behave. Make sure that virtual elements act in accordance with their characteristics and associated affordances [32, 37, 42, 48, 52, 62, 64, 79].

Consider the Use of 3D (Volumetric) Representation. XR applications give designers the ability to create elements that can be explored and interacted with in 3D. This can be leveraged to better mimic and build upon real world objects [37, 50, 52].

Fig. 7.
figure 7

Picture is based on leap motions’ blog Interaction Sprints at Leap Motion: Exploring the Hand-Object Boundary and Designing Physical Interactions for Objects That Don’t Exist. It shows the process of building real life affordances into the design of a grabbable object in their demo Weightless. (http://blog.leapmotion.com/interaction-sprint-exploring-the-hand-object-boundary/)

Full size image

Be Cautious of Simplified Interactions. Users’ expectations for the detail of interactions may be higher in XR due to the increased associations with real experiences that comes with immersion. For example, a real screwdriver is a tool meant for screwing and unscrewing screws, but it has physical properties that allow it to do more than its original purpose. A virtual screwdriver may not have additional interactions outside its intended purpose. It is important that designer build protections to deal with or inform the user of the limitations of the application [28].

4.9 Provide Feedback and Consistency

Definition. Use feedback to generalize perception of events and interactions. Additionally, feedback should be consistent such that users can build an understanding of what they can and cannot do within the application [27, 29, 34, 36, 37, 44, 45, 48,49,50, 52,53,54,55,56, 58, 63, 66, 74, 76, 79, 80, 83, 85].

Explanation. The more consistent feedback that is provided to the user, the better off they will be to make informed decisions. Due to the immersive nature of XR applications, the importance of feedback is amplified as users may have expectations on how certain aspect of the application should behave based on real-life experiences. For example, real objects have properties, like weight, hardness, size, etc., that governs how they behave. Despite having similar visual appearance, digital elements do not have guarantees that their behaviour will match that of real-world objects with similar properties. As such, it is important that developers build in consistent feedback to inform the user on how to interact with the application.

Use Feedback to Standardize Interactions States. Consider how elements in a digital environment should react to user interactions. All interactions should have distinct states that are conveyed through visual feedback. Furthermore, different states should have distinct properties that are consistent such that users can familiarize themselves and recognize them as they use the program [27, 29, 34, 36, 44, 45, 48,49,50, 52,53,54,55, 58, 66, 74, 76, 79, 80, 83, 85].

Use Feedback to Help Recognizes Errors and Unwanted States. Inform users of the results whenever an action is performed. For instance, the system should make it clear if an action is possible, whether an interaction was detected, if all requirements were met to perform a certain action, etc., such that users can alter their behaviour to learn from their interaction mistakes [54].

Design for Dynamic Exploration: Let the User Explore the Environment and Understand it Through Feedback. Provide users with enough information to learn how to use the application through using the application [37, 48, 52, 55, 56, 58, 63].

Fig. 8.
figure 8

Example of dynamic feedback from leap motion’s Cat Explorer. A: Menu does not appear when the user’s hand is far away B: Menu appears when the user brings their hands’ closer C: The button light up when it is pressed.

Full size image

4.10 Allow Users to Feel in Control of the Experience

Definition. The application should act and respond in a way that gives users the sense that they are in charge.

Explanation. Users are vulnerable when they are immersed in XR environments. It is important for the application to establish a feeling of trust with the user by making sure that they feel in control during the whole experience [7, 11, 20, 23, 29, 37, 47, 48, 50, 52, 53, 76, 83, 85].

Don’t Force Actions Without User’s Permission. The application should never assume what a user wants to do. Make sure express permission is given by a user before performing actions with consequences [29, 48, 83, 85].

Provide Exits for Users to Leave Unwanted States. Provide users with easy options to abandon certain actions or leave unwanted situations with minimal resistance and hassle from the application [7, 11, 20, 23].

The Application Should Be “Honest”. Make sure the behaviour of the system matches the user’s expectations. Failure to achieve this will make the user feel like they are being deceived by the application [37, 50, 52].

Let Users Control Their Movements. For HMD XR applications, it is always important that the control of the virtual world camera stays with the user. This will help her or him feel in control and immersed in the experience [7, 47, 53, 76].

4.11 Allow for Trial and Error

Definition. As much as possible, allow actions to be reversible and set up protections around potential mistakes made by users. This will help relieve user’s anxiety and promote exploration of the application [4, 11, 20, 23].

Explanation. XR designers have more opportunities to provide users with interactions that have a bigger impact on the experience. However, more control also equates to a larger possibility for mistakes. Furthermore, previous error tolerance techniques, such as dedicated back buttons or keyboard shortcuts, are not always meaningful in XR applications. It is important that designer builds in protections for when users’ inevitably make errors.

Permit Easy Reversal of Actions. Provide users with clear options to deal with actions that they may want to undo such that can freely explore the application without worries [4, 11, 20, 23].

5 Discussion

In this section, we overview several points of discussions that were identified through the creation of our UX guidelines.

Emphasis on VR Devices and Experiences. During our analysis of the literature, it was noticed that most of the resources found were based around VR experiences. This may be due to the differences in accessibility among XR devices. VR headsets, such as the HTC Vive, Oculus Rift, and the Window Mixed reality suite are cheaper on average and provide a larger range of options when compared to AR and MR devices. At the time of collecting the resources, many AR or MR manufactures, such as the Microsoft HoloLens and Magic Leap, only had developers/creators or business version of their devices available to the public. As such, the community around VR is larger and has had more time to explore the technology, which was reflected in the content of our resources. This trend suggests that there is an opportunity for further research around UX design practices specifically in the context of AR and MR.

Emphasis on Addressing Comfort. Our guidelines show that ideas around comfort were a common theme in previous works. In addition to the high occurrence, these guidelines were often more defined and consistent throughout the various resources. For example, [27, 40, 41, 53, 75, 76, 84] all explicitly discussed zones of comfort for XR based on human ergonomics. All these sources shared similarities concepts, and a few referenced each other. This pattern extends to the other sub guidelines under UX guideline 3 –Prioritize user’s comfort–. This trend was not as prominent in other guidelines which suggest that comfort has been a common problem that designers faced when building applications for HMD XR devices. Further exploring user comfort within XR application may be a valuable area for future research to be conducted.

Few Developed XR Examples. While exploring the resources, we found relatively few XR examples that were given to support the concepts being conveyed. Instead, the literature would often borrow from other domains to further explain a concept. For example, [37] discusses how space can be used to amplify mental capabilities for spatial applications. Instead of showing an example in XR, the author references how chefs, carpenters and other experts constantly rearrange items to easily track the state of different tasks or to better notice properties signalling what to do next. From the lack of XR examples, it becomes apparent that there is a need for further work around applying these concepts in actual applications.

Opportunity for Evaluation, Validation and Empirical research. By opening up our search for resources to communities outside of academic sources, we drew upon ideas and concepts that were not academically validated. Additionally, as seen in Sect. 2, related academic literature showed limited work around validation and often express the need for further research in this area. This suggests that there are lots of opportunities for these concepts to be further explored to understand better if and how these ideas can help in the development of XR applications.

6 Limitations

While our work focused on creating a reference for understanding what others have tried and discovered, we acknowledge that our research only scratches the surfaces for what is needed to fully understand how to design XR applications. One of our primary limits is the lack of evaluation. As mentioned in the previous section, a majority of the resources that were used were not validated. Additionally, our research did not include studies to evaluate our UX guidelines. Future research building on our findings is required to further explore the concepts within our guidelines and understand their effectiveness for designers and developers.

7 Conclusion

Recent developments in XR has provided hardware that allows users to experience spatial environments and interactions. While initial searches for academic resources around XR design guidelines yielded limited results, a wealth of information can be found from different communities that provide insights into this topic. To further inform the design of XR applications, we conducted an analysis of 68 different resources to derive eleven UX design guidelines for HMD devices from various communities. We presented our findings by providing each guideline with a name, definition, and explanation with additional sub guidelines. Ultimately, our guidelines are not meant to provide absolute rules for designers to follow but serve as a reference to build off from and adapt based on their intended medium and situation. With our work, we aimed at providing a resource for understanding what others have tried and discovered such that future research can challenge, build off, and extend the ideas presented in this paper.