Orientation and navigation are fundamental aspects of everyday life. Estimating distances and angles, recalling and recognizing object locations and memorizing routes or landmarks are just a few examples of many basic tasks in outdoor as well as indoor environments (e.g., Keil et al. 2019; Plumert et al. 2005; Postma and De Haan 1996). To optimize the effectiveness and efficiency of these tasks, people often use geospatial media, such as maps or map-like representations. The academic fields of cartography and spatial cognition have increasingly addressed the question of how to integrate and use modern technologies—usually originating in IT- and entertainment industries—to develop proper geospatial media (e.g., Edler et al. 2019; Hruby 2019; Knust and Buchroithner 2014).
Going beyond traditional 2D print approaches, the portfolio of cartographic products was strongly influenced by digital techniques since the establishment of the computer as a mass media device and, in addition, as a tool to create other mass media (e.g., Clarke et al. 2019; Taylor and Lauriault 2007; Müller 1997). Multimedia cartography—sometimes also referred to as “cybercartography” (Taylor 2005)—lead to fundamental approaches of computer-based animated, interactive and multisensory (web) map applications (e.g., Kraak 1999; Peterson 1995; Krygier 1994). It is argued that animation techniques used in cartography have been influenced by the computer and video game industries (Edler et al. 2018a; Edler and Dickmann 2017; Ahlqvist 2011; Corbett and Wade 2005).
The development of computer-based animation techniques came along with appropriate software and hardware solutions. Higher performances of software and hardware also allowed the further development of stable and detailed 3D visualization methods and techniques. For example, autostereoscopic displays allowed to generate 3D depth effects, which was explored in several studies on visualization and user experiments in cartography (e.g., Edler and Dickmann 2015; Bröhmer et al. 2013; Buchroithner 2007). Moreover, the open availability of game engines, such as Unity and Unreal Engine, supports the creation of individual 3D landscapes that can be accessed with virtual reality (VR) headsets, in real-time and from the ego perspective—thus, creating an impression of immersion. The potentials of VR-based visualization are currently under study (e.g., Cöltekin et al. 2019; Edler et al. 2018b; Hruby et al. 2019; Kersten et al. 2018).
Closely related to 3D visualization in VR are 3D visualizations in augmented reality (AR). AR techniques allow to project static or animated objects into real environments, thus extending real physical environments. Representing an early development stage, AR visualization techniques can be based on so called mid-air displays, sometimes also referred as free-space displays (Dickmann 2013). A mid-air displays projects graphical objects on free projection surfaces, such as a hardly visible wall of fog (“fog screen”) created by an installed blower (DiVerdi et al. 2008).
A famous example of an AR application interacting a lot with space is the gaming app “Pokémon GO” (Zhao and Chen 2017). In this smartphone- or tablet-based game, users interact with audiovisually animated game characters that can be found in the real environment. In this way, the whole logic and process of the game is added like an additional information layer into the physical landscape. The smartphone or tablet is the ‘physical gateway’ to this augmentation. The camera of the device is used to record the area in front of the user and the recordings are augmented with virtual objects in real time. As demonstrated by de Almeida Pereira et al. (2017), this technique can also be used to augment physical 2D maps with 3D geographic information as height maps.
Two head-mounted display (HMD) devices representing the state of the art of current AR are the Microsoft HoloLens (version 2 recently introduced) and the HTC Vive Pro. The Microsoft HoloLens (Fig. 1) uses a pair of smart glasses. Stereoscopic images (holograms) are projected onto two small lenses in front of the eyes (Noor 2016). As these lenses are see-through (Gruenefeld et al. 2017), the projected holograms merge with the real environment. The HTC Vive Pro (Fig. 1) is a VR headset with AR capability. The environment in front of the user can be recorded with two cameras inside the HMD and rendered on two displays located in front of the eyes of the user (HTC 2019). Additionally, the camera recordings can be augmented by artificial elements acting as stereoscopic holograms.
The Potential of AR Techniques for Experiencing Space
As both the Microsoft HoloLens and the HTC Vive Pro are capable of tracking head movements, they make it possible to create an impression of permanent presence of holographic geospatial objects. Even if the user walks around in a defined area, commonly indoor area, holograms remain and adopt to the user location and viewing perspective. This permanent and adaptable holographic projection may lead to visualization approaches that bring additional advantages for the cognitive processing of the geospatial area experienced.
Empirical research of cartography, spatial cognition and experimental psychology has recently led to some recommendations for the construction of user cognition-oriented cartographic media. For example, it was reported in user experiments that an additional layer of square grids increases the performances in memory of object locations (Bestgen et al. 2017; Kuchinke et al. 2016; Edler et al. 2014) and in estimating longer linear distances in maps (Dickmann et al. 2019). The grid-based memory effect also occurs if the grid structure is physically reduced to indicated (“illusory”) lines (Dickmann et al. 2017), gets a depth offset (Edler et al. 2015) or is changed to a hexagonal pattern (Edler et al. 2018c). Other studies reported that reducing the visibility of some map areas can direct visual attention towards other map areas (Keil et al. 2018), and that the display of landmarks can improve route knowledge (Ruddle et al. 2011) and orientation (Li et al. 2014).
The above mentioned cognition-based effects on spatial performance measures in maps are promising results indicating that an extended communication of spatial information can bring advantages for the map user in terms of map perception, orientation, navigation and the formation of spatial knowledge. Similar effects will likely occur in real 3D environments augmented by holographic spatial objects. These holographic layers could offer an additional (geometrical) structure to support the cognitive processing of object locations, distance estimations and relative directions between objects.
To investigate possible effects for the perception of spatial information, new methodological challenges occur. The possibility to implement spatial models in AR applications has already been investigated and described (Wang et al., 2018). However, to take full advantage of the possibilities of AR for geospatial applications, technical limitations of the current available AR devices must be faced. These include the precise placement and stability of holograms in the three-dimensional space, a crucial quality criterion for AR applications (Harders et al. 2008). Having found stable solutions that guarantee a high spatial precision, the AR devices can become valuable methodological tools in geospatial experiments focused on fundamental questions of spatial cognition in 3D environments. In user studies, they could be used to project holographic objects in the environment. Moreover, AR devices could assist experimental investigators to arrange the spatial layout of movable real world objects used in their study. The projection of ‘virtual place markers’ can increase the precision of identical spatial object arrangements, which—from a methodological perspective—increases the comparability of acquired user data (between participants). Moreover, projected ‘virtual markers’ can support the analysis of user tasks, such as the identification/measurement of distortion errors, for example, in location memory tasks.
To exploit the possibilities of AR systems for geospatial user experiments, it is necessary to create technical methods to establish controlled procedures and to standardize the placement of holographic objects in a real 3D setting. In the following sections, we describe the functionality of current AR systems, how technical factors of these AR systems affect the targeted placement of holograms, and additional requirements for geospatial applications and experiments. To address and resolve the described discrepancy between requirements and limitations, we present a self-developed AR interface application capable of (re-)placing holograms highly accurately during runtime. Code examples are provided for transparency, a better understanding, and replicability.