When choosing an approach for developing MR-based navigation, at least two factors need to be considered: (1) spatial anchor and (2) path generation. A spatial anchor is a fixed coordinate system that is generated and tracked by MR, and ensures the anchored holograms are located in the precise location.Footnote 10 In an MR-based navigation system, the spatial anchor can be a local (LA, stored in the device) or a cloud (CA, stored on the cloud, e.g. Azure Spatial AnchorFootnote 11) one and the path can be predefined (PP) or generated during the navigation (GP). Therefore, four approaches are feasible in developing the system. They are compared regarding the requirements of materials, resources, and performance (Table 1). We also give examples of each approach.
Local Anchor–Predefined Path Approach
In this approach, local spatial anchors and predefined paths are used. Spatial anchor is stored on the device and cannot be shared across multiple devices. It is loaded and manually anchored by the user to a new position each time the software runs. In this case, internet connection and BIM are not required, and the anchor position changes, even slightly, between different runs or if multiple devices are used in the research. Applying only one spatial anchor would reduce the time and workload of anchoring but may increase localization errors. Therefore, the number of necessary anchors relies on the study area and research questions.
A predefined path does not mean that only one path is available. It is possible to set multiple destinations/paths during development. However, once deployed, the paths are set, and the visualization cannot be changed. For example, the path cannot be changed to avoid a passer-by. Besides, since all the holograms are locked to one anchor, the errors are cumulating, i.e., the farther the hologram is away from the anchor, the larger the misalignment would be. The anchor should be in the middle of the whole study area instead of the start point, and the study area should not be too big. It is an immediate and low-code requirement to build. Therefore, this approach is suitable for a quick assessment of the interface and elements design and fast feedback on cognitive issues. It is also convenient for the research that must be done without the internet. This workflow suits beginners and small projects or rapid prototyping. However, since it requires manual anchoring, it is unsuitable for research involving large user groups.
Example. Building with Local Spatial Anchor and a Predefined Path
This example uses a local spatial anchor and predefined path to create an indoor navigation demo, which was used to test spatial learning during navigation (Liu et al. 2021). It was built using Unity, MRTK, and HoloLens 1.
Figure 5 shows the workflow in this example. Although BIM is not mandatory, HoloLens is used to create a rough model of the study area and map the layout (Fig. 5a). It allows the researcher to put the holograms in the correct position. A floor map also helps to show the number of turns, length, etc.
The holograms should also be prepared (Fig. 5b). In this case, pictorial landmarks and arrows are used. MRTK provides basic GameObjects, such as cubes, spheres, arrows, etc. The pictorial landmarks are generated from png-format pictures. The png. files are used as the material of the basic GameObjects, and the size of GameObjects can be adapted. The GameObjects can also be attached with scripts and then made into prefabs for re-use. For example, when users are moving around, the landmarks should always face the users to remain identifiable. This function can be realized by the billboard provided by MRTK. The prefabs can be located according to the model (Fig. 5c, d) or the floor map. At the beginning of the user study, the researcher needs to set the spatial anchor manually. It is recommended that the researcher walk through the whole study area and ensure the misalignment is acceptable. The user’s view is shown in Fig. 5e.
This demo used one spatial anchor at the middle point of the path (Fig. 6). The spatial anchor is a rectangle instead of a point to allow the alignment both horizontally and vertically with the real world and make sure the direction of the whole path is correct. The anchor was designed in grey and transparent so that it does not affect the users’ navigation. This design proved effective as most users did not notice this anchor.
Local Anchor–Generated Path Approach
Like the LA-PP approach, the LA-GP approach does not require an internet connection, the spatial anchor is not constant for each user, and cannot be shared across devices. This means the visualization of the spatial anchor should also be big enough for accurate localization. This approach is also not ideal for studies with an extensive study area or large user groups. BIM is required so that the location of destination is known and can be used in path generation. This approach is more flexible and suitable for exploring interaction and dynamic situations, for example, how users can find the preferred route or avoid obstacles.
Different from the LA-PP approach, in the LA-GP approach, a path between two locations can be generated according to the users’ command by a path generation algorithm. This allows users to set their preferred destinations or avoid obstacles in real time. However, BIM is needed to generate the path, and it requires higher coding ability to integrate the path generation algorithm in the demo.
One example of the LA-GP approach is the work of Qiu (2019) using Unity, Holo Toolkit (the predecessor of MRTK), and HoloLens 1, with the aim to design an indoor navigation system that can avoid obstacles in real-time. A* search algorithm and BIM were used. The indoor space was segmented into many nodes. Once the user set a destination by hand gesture or voice control, the software generated a path. During walking, the HoloLens constantly maps the spatial environment and checks if there are obstacles (e.g., a passer-by) on the following paths. If so, the path will be re-calculated to avoid the obstacle.
Cloud Anchor–Predefined Path Approach
The resources required in the CA-PP approach are similar to those in the LA-PP approach. The BIM and path generation algorithm are not necessary. However, it requires an internet connection. The holograms are anchored to an online anchor, e.g., Azure Spatial Anchor. A cloud spatial anchor module is needed to upload and download the anchor to/from the cloud. Therefore, it needs more codes and requires higher coding capability compared to the LA-PP approach. The positions of holograms remain the same across sessions, which spares the effort of setting the spatial anchor each time. The spatial anchors can be shared by different devices and thus multiple users can collaborate. Besides, since the paths are predefined, the holograms can be locked to one spatial anchor. In this case, once the spatial anchor is set, no internet connection is necessary. However, if multiple spatial anchors are used, a stable internet connection is necessary. Therefore, the CA-PP approach is suitable for user studies that may last for a long time, involve many users and are with limited internet connection.
Figure 7 shows an example using the CA-PP approach for building MR-indoor navigation. In this demo, Unity, MRTK, and HoloLens 2 were used. The layout was measured by MR measure apps on a smartphone (Fig. 7a). The holograms (Fig. 7b) and path (Fig. 7c) are designed similarly to those in the LA-PP example. The main difference is that the spatial anchor is uploaded to the Azure platform on the first run. Afterward, the spatial anchor can be loaded in each session (Fig. 7d). Therefore, the spatial anchor is not necessarily at the middle point and can be set at the start point or endpoint. Figure 7e shows the user’s view at the start point.
Cloud Anchor–Generated Path Approach
The abovementioned three approaches can fulfill the requirements of most studies. However, the paths generated in those approaches are usually not very long to allow stable visualization, which cannot reveal cognitive issues that might only occur during more extended usage of MR-based indoor navigation (Curtsson 2021). Besides, some cognitive issues may occur during intensive interactions among different users (Liu et al 2021). The CA-GP approach has the advantage for applications in a bigger study area and with many users.
In this approach, the path generation algorithm and BIM are needed. A stable, constant internet connection is necessary for a smooth navigation experience. Since the spatial anchors are saved online, no BIM is needed. A possible solution is to provide two-player roles. The first role is for the researcher, who needs to walk around, set and upload spatial anchors to the cloud and edit the properties. The second role is for the users to set destinations and navigate themselves. Takahiro Miyaura provides an example.Footnote 13 In this project, the user can create different paths and upload them to the cloud interactively and find the path to a specific destination afterwards.