Abstract
To reduce the decline of spatial cognitive skills caused by the increasing use of automated GPS navigation, the virtual global landmark (VGL) system is proposed to help people naturally improve their sense of direction. Designed to accompany a heads-up navigation system, VGL system constantly displays silhouette of global landmarks in the navigator’s vision as a notable frame of reference. This study exams how VGL system impacts incidental spatial learning, i.e., subconscious spatial knowledge acquisition. We asked 55 participants to explore a virtual environment and then draw a map of what they had explored while capturing electroencephalogram (EEG) signals and eye activity. The results suggest that, with the VGL system, participants paid more attention during exploration and performed significantly better at the map drawing task—a result that indicates substantially improved incidental spatial learning. This finding might kickstart a redesigning navigation aids, to teach users to learn a route rather than simply showing them the way.
Similar content being viewed by others
Introduction
Spatial abilities are considered to be amongst the most important capacities for the survival of all species1,2,3. Developing navigational skills is not only a benefit for independence and personal well-being but also has positive effects on the human brain4,5,6,7,8. The discovery of brain’s ‘global positioning system’ by O’Keefe’s team7 built up a direct bridge between our brain and orientation skills, which reveals that the ability to navigate could be associated with improved brain functions. This close relationship can be found when new mental maps are formed through the act of navigation, the neural connections then increase, and new neural pathways are created accordingly4,5,6,7,8. In other words, our navigation skills, i.e., our spatial learning ability, directly and positively affects the physical condition of our brain. A well-known study on the enlarged hippocampi of London cab drivers is a good example of how spatial navigation skills can biologically impact the brain’s cortical plasticity9. This is because during their working time, they are required to continuously process the surrounding streets and environment, which, in turn, expands their brain to accommodate the cognitive demands of navigating London’s streets.
Today, with the development of technology on the global positioning system (GPS), the essential demands of spatial navigation and orientation are being offloaded to automatic orientation systems. This technology undoubtedly benefits human navigation with reliable, efficient directions, especially in outdoor environments. However, as a negative effect, this automated support replaces our fundamental tendency to build up a cognitive map of our surroundings through self-active exploration with trials and errors1,7,10, in other words, it may turn off the brain’s own GPS. Following simple turn-by-turn navigation instructions can easily cause an ignorance of processing of environmental information1,11,12. Researchers have noted that people who use GPS-based navigation systems have a poorer sense of direction, which leads to a decline in spatial memory11,13. This declines in spatial memory can negatively impact the ability to construct mental maps11,12,13. One could argue that automated navigation services are damaging the development of spatial skills in humans1,14,15.
To overcome this spatial deskilling, a new generation of navigation assistance systems is being developed that incorporates some of the natural elements of the human internal navigation system. The idea is to incorporate more of one’s surroundings into the directions to positively influence spatial awareness in the brain16,17. More spatial cues while a person is moving through an environment might encourage the user to process more spatial information, which can help them retain more awareness and attention on their actual surroundings, even in unfamiliar territory. As one of the most basic external references for an environment, landmarks are easily recognized and remembered to serve as the key navigation cues18,19. Recent empirical studies have uncovered the benefits of associating landmarks with navigation instructions16,20. For instance, integrating references to landmarks into turn-by-turn verbal directions, e.g., “Please turn right at the concert hall.”, has proven to bolster spatial learning1,16,21. Similarly, displaying distant landmarks at the edge of a mobile screen to support spatial orientation positively contributes to the user’s wayfinding efficiency and awareness of spatial information for surrondings20. This approach may enhance how a user encodes spatial knowledge while using a navigation system. Overall, as a common heading reference, the application of landmarks on navigational aids could help to overcome the negative effects of relying on navigation assistance systems.
Inspired by the constructive role landmarks can play in spatial learning, we developed a system called virtual global landmark (VGL) system that incorporates global landmarks into a virtual heads-up display (see Fig. 1). The way the system works is that it constantly ensures a global landmark is visible, even when obscured by one’s surroundings. Global landmarks include things like city skylines or mountains—features that are visible from far away and serve as a broad frame of reference. In a process called landmark encoding, these landmarks do not disappear or change when the observer moves a small distance away22. Our idea is to ensure that the navigator is always aware of their own position relative to a global landmark. Learning this information not only helps people to develop a cognitive map of the environment, it is also considered to be one of the most significant strategies for the allocation of attention23 and developing spatial memory7,10. It therefore serves as a sort of proxy for a compass. Continuous neural processing of this basic point of reference could then help to increase route knowledge in an environment and in the meantime, grab more chance to motivate the user to continually survey the spatial knowledge, which contributes to spatial learning.
In this study, we examined the effects of the VGL system on spatial learning and, in particular, its impact on spatial knowledge acquisition. According to the model introduced by Siegel and White24, spatial knowledge acquisition can be described at different levels from the simple to the complex. The model also describes different types of spatial learning acquisition—landmark knowledge, which represents knowledge about objects in space; route knowledge, which refers to the connections between those landmarks in space; and survey knowledge, which is map-like knowledge that includes spatial relations between landmarks. Survey knowledge also enables complex computations like calculating short-cuts from previously unknown routes. Thus, to assess overall levels of spatial knowledge acquisition gained via the VGL system, we conducted a trial where the participants had to draw a map of a relatively unfamiliar environment. A map drawing task is a common method of evaluating the spatial knowledge gained from an environment that contains certain spatial information because it is robust to reflecting cognitive impact25,26,27.
A cohort of 55 students from the University of Technology Sydney participated in the study. Each was equipped with a head-mounted VR system, which they used to explore a medium-scale VR environment called “Sydney Park”, and a mobile brain/body imaging (MoBI)28,29,30 setup that captured brain dynamics via EEG signals while they explored. During exploration, a fixed route was predefined to control and standardize how participants explored the environment. The route was defined in a way that balanced the participant’s exposure to local and global landmarks. In this stage, the participants were divided into two groups for exploration. One group explored Sydney Park following auditory instructions, but with the VGL system equipped. The other did not use the VGL system; they simply followed the auditory instructions. The idea with providing the instructions in verbal form was to guide participants walked through the predefined route, as well as simulate navigation aids that provide turn-by-turn instructions. Adding the VGL system provided a way to assess an incidental knowledge acquisition alongside this guided navigation process. Before and during their explorations, the participants were not informed that they would subsequently be asked to draw a map of the route they had taken. We measured the fit of the sketch map’s configuration to the actual target configuration as a score against seven points of fidelity to show the outcome of spatial knowledge acquisition.
To investigate human brain spectral activity during exploration, we evaluated event-related spectral perturbations (ERSP) originating in the occipital and parietal cortices of the participants. The ERSP31,32 measured the average dynamic changes in amplitude of the broad band EEG frequency spectrum as a function of time relative to fixation events generated by the eye tracker. Many studies have demonstrated involvement by the parietal cortex in spatial navigation33,34,35,36. We then focused on brain activity in parietal regions to evaluate the brain encoding on the spatial knowledge related to navigation in exploration. And, as the visual processing area of our brains, the occipital lobe is responsible for interpreting the visual world around the body, such as the shape, color, and location of an object37,38. Thus, we were also interested to see whether brain dynamics in the occipital cortical areas would differ when participants were fixating on VGL as compared on other landmark stimuli during the exploration. We also took notes on behavioral activity and asked each of the participants to complete a Santa Barbara Sense of Direction39 (SBSOD) test together with Perspective Taking/Spatial Orientation Task40 (PTSOT) before completing the trials to control for individual differences in spatial abilities.
Based on our design of VGL system as a steady reference frame for surroundings, we hypothesized that participants using the VGL system during exploration would be more aware of the surroundings, process more spatial knowledge incidentally. Thus, in exploration phase, as measures to show whether more awareness and attention occur, we expected to find participants voluntary to spent longer time to complete the exploration and fixated longer in the environment. Being a method to visualize the outcomes of spatial knowledge processing in exploration, we expected to see an improved performance in map drawing task by participants with aid of VGL system, which would be the evidence of an increased spatial knowledge processed from exploration. Additionally, the mental demand investigated by NASA Task Load Index (NASA-TLX)41 was expected to show no increase for participants drawing with VGL information. Lastly, the increased alpha and decreased theta power over parietal and occipital cortex has been revealed by the suppressing distractions from the visual system42,43,44,45. Hence, to assess whether VGL could help attracting awareness and attention of the surroundings, we also expected to find the same phenomena in the ERSPs related to VGL stimuli around parietal and occipital sites.
Results
Learning performance in the exploration and map drawing task were assessed via one-way ANOVAs. Separate ANOVAs were computed for: (i) total time spent in exploration; (ii) average blink intervals during exploration; (iii) NASA-TLX mental demand scores during the map drawing task; (iv) total map scores for the map drawing task; and (v) single indexes scores for the map drawing task, including canonical organization scores, recalled points percentage, rotational bias, scaling bias, canonical accuracy, distance accuracy, and angle accuracy. The group conditions, VGL and (non-) NVGL, were entered as a between-subjects factor. All the above indexes for both groups were normally distributed (p > 0.05). The Spearman’s correlations result between each individual’s spatial ability and the above indexes are shown in Fig. S1 of the supplementary. Significantly correlated factors were entered as covariates to compute the ANOVAs. Additionally, gender was added as a between-subjects factor when calculating one-way ANOVAs to evaluate gender effect on results. However, there was no significant interaction between gender and group condition for any of the measures. The test results of the between-subject effects are provided in the supplementary materials.
VGL system users paid more attention during exploration
During the exploration phase of the trial, the participants determined their own walking speed. Hence, the time spent by the two groups was dependent on each participant. As presented in Fig. 2a, we found a statistically significant difference in the time spent (F1,50 = 12.08, p = 0.001, partial η2 = 0.20) with 54.83 s longer spent by the VGL group (M = 267.78, SE = 10.72) than the NVGL group (M = 212.95, SE = 11.58). To assess the impact of accuracy in navigation and response to verbal prompts on time spent during exploration, we analyzed the Spearman’s correlation between time spent and the number of wrong response to verbal prompts. The result reveals no significant correlation between these two (rs(55) = 0.16, p = 0.25). These findings suggest that participants aided by the VGL system walked more slowly and might be attracted more attention with significant more time spending while exploring. Additionally, due to the significant difference in time spent during exploration between two groups, we used this time for exploration as a covariate to compute ANOVAs for other measured indexes to control the time effect.
To find more evidence on attention during exploration, we also computed the average blink interval for each participant. As shown in Fig. 2b, the results are statistically significant (F1,48 = 14.02, p < 0.000, partial η2 = 0.23). The VGL group had a 0.86 s longer blink interval (M = 2.16, SE = 0.15) than the NVGL group (M = 1.30, SE = 0.16). The longer interval between blinks relates to a lower blink rate. By the interpretation of split attention revealed with higher blink rate46,47, the significant longer blink interval for VGL group indicates an increased attention while exploring when participants were aided by VGLs.
VGL system users scored higher for incidental spatial knowledge acquisition in the map drawing task without additional mental workload
The map drawing task was our basis for assessing incidental spatial knowledge acquisition during exploration. Assessments of the sketched maps were made using Gardony Map Drawing Analyzer (GMDA)27. Of these, the NASA Task Load Index (NASA-TLX)41 indicates mental workload. The results in Fig. 3a demonstrated no statistically significant difference in mental demand scores (F1,52 = 1.75, p = 0.19, partial η2 = 0.03), which indicates an equal level on mental workload in two groups. However, as indicated in Fig. 3b, there was a significant difference in the total map scores (F1,51 = 4.60, p = 0.04, partial η2 = 0.09) with higher map scores in the VGL group (M = 3.24, SE = 0.06) than that in the NVGL group (M = 3.05, SE = 0.06) by 0.19, reflecting an overall better preformation in sketch map for VGL group.
Figure 4 charts all the single indexes used to calculate the total map scores. No significant differences between the two groups were found for canonical organization score (F1,51 = 1.96, p = 0.17, partial η2 = 0.04), recalled points percentage (F1,51 = 0.73, p = 0.40, partial η2 = 0.01), scaling bias (F1,52 = 0.34, p = 0.56, partial η2 = 0.007), canonical accuracy (F1,51 = 3.09, p = 0.09, partial η2 = 0.06) and distance accuracy (F1,51 = 2.77, p = 0.10, partial η2 = 0.05). However, significant differences were found for rotational bias (F1,52 = 4.15, p = 0.047, partial η2 = 0.07) with 6.78 degree lower in the VGL group (M = 10.08, SE = 2.20) than that in the NVGL group (M = 16.86, SE = 2.25). As the rotational bias measures the direction of angular error of inter landmark angles27, the significant lower rotational bias for VGL group suggests a higher precision on landmark rotation. This outperformance on angle representation of VGL group is also further shown by a significant higher angle accuracy (F1,51 = 7.71, p = 0.008, partial η2 = 0.13) with 0.05 percentage higher in the VGL group (M = 0.88, SE = 0.01) than that in the NVGL group (M = 0.83, SE = 0.01).
VGL system users saw ERSP desynchronization in the theta and alpha bands of occipital-parietal clusters
Figure 5 shows the spectral fluctuations in the left and right occipital clusters and parietal midline clusters as measured via ERSPs. The readings are from −1 to 5 s, where the time point 0 indicates where the participant reached one of the virtual checkpoints (see the checkpoint map in Fig. 9 of the “Materials and methods” section). The time point 1000 ms indicates where a fixation on one kind of landmark started, and the time point 5000 ms shows where the participant reached the next position (offset in ERSPs). Only one landmark of any kind was viewed before reaching the next position (offset) (i.e., a virtual global landmark in the VGL group, a local landmark in the NVGL group). For both the occipital and parietal clusters, the non-green pixels indicate differences in the ERSPs (Fig. 5c,f,i). Within one group, the significant differences level for mean ERSP is p < 0.001. Between the two groups, these differences were statistically significant at p < 0.05.
Average ERSPs in the parietal midline cluster
The parietal midline cluster comprised 24 independent components from 18 participants for the VGL group, and 33 independent components from 19 participants for the NVGL group. The average scalp maps for these clusters along with the dipole locations are presented in Fig. 5a for the VGL group and Fig. 5b for the NVGL group. As presented in Fig. 5c, the average ERSPs for the VGL group revealed significant decreases in theta activity shortly after the onset of stimuli, and an increase in alpha power band activity, p < 0.001. By contrast, the ERSPs for the NVGL trials showed significant increases in the same frequency bands, p < 0.001. The difference in ERSPs between group conditions had statistical significance in these activity bands, p < 0.05, and demonstrates a pronounced suppression in theta activity and an increase in the alpha band of the VGL group compared to the NVGL group.
Average ERSPs in the left and right occipital clusters
The left occipital cluster included 21 independent components from 16 participants for the VGL group, and 21 independent components from 15 participants for the NVGL group. The right occipital cluster included 33 independent components from 25 participants for the VGL group, and 15 independent components from 13 participants for the NVGL group. The average scalp maps for these clusters along with the dipole locations are presented in Fig. 5d and g for the VGL group and Fig. 5e and h for the NVGL group.
As shown in Fig. 5f and i, in both the left and right cerebrum, the average ERSPs in the occipital cortex for the VGL group reveals a strong increase in alpha power band activity at the onset of the landmark stimuli (p < 0.001). The increase became stronger for a short time after viewing the virtual global landmark. The stimulus induced theta-power suppression was also apparent. In contrast, for the NVGL group, power increased significantly in the theta bands and decreased in the alpha band (p < 0.001). Theta activity was increased strongly around the stimuli of local landmarks, tapering back to slight after a while. The differences in ERSPs between conditions was statistically significant for the theta and alpha bands, p < 0.05, and showed a significant suppression of theta activity and an increase in alpha band activity for the VGL group compared to the NVGL group.
Discussion
In this work, we seek to explore the particular impact of the VGL system on spatial knowledge acquisition. To investigate this, we undertook the neural measures as well as the behavioral traits, most importantly, through a map drawing task with 55 participants. These 55 participants were divided into two groups—one equipped with the VGL system and one without—and asked to explore an unfamiliar environment. The exploration route for both groups was fixed. Once the participants completed their route, we asked them to sketch a map48 of the environment they had just explored. We hypothesized that those using the VGL system would absorb more spatial information, be more spatially aware of their environment, and draw their maps more accurately than those who did not have the aid of the VGL system.
The results of the map drawing task in Fig. 3 demonstrate significantly higher scores for the VGL system participants at the same levels of mental workload as the NVGL system participants. Here, mental workload for mapdrawing task was assessed through NASA-TLX41, which is a common subjective workload assessment tool. Consistent with our hypothesis, there was no difference between the cognitive workloads of the two groups in terms of NASA-TLX results. A possible reason for this is that the VGL group did more information processing in terms of mentally organizing and sketching their surroundings while exploring. This view is supported by the higher map scores achieved by this group. Thus, the comparable mental workload between VGL and NVGL group could imply that the VGL system did not carry more or even help to ease the workload of spatial memory though more information processing required comparing without VGL system.
Sketching maps is a common method of evaluating spatial knowledge gained from an environment and one that is robust to cognitive impacts25,26,27 as well as spatial memory49,50. The results of all the measured indexes pinpointed the main significant differences for the total map scores came from the indexes to mark the angular accuracy level of the sketch map, including angle accuracy and rotational bias. As for the measures to assess the organization related scores, including canonical organization score itself, points percentage to show successfully recalled landmarks and intersections as well as scaling bias and distance accuracy to show organization of interlandmark distances, participants in the environment with VGL and local landmarks showed an equal performance level to those in the environment with local landmarks only. This might be because, as a conspicuous and easy-to-pinpoint reference, global landmarks can help participants build a clearer directional relationship between themselves and a constant point of reference18,19. Thus, as a persistent reference for global landmark, VGL system could support participants to keep well-marking the rotational relationship among environment features by a clearer directional sense. This is why the participants aided by the VGL system were able to gain a more accurate performance on angular measures of incidental spatial knowledge in this new environment.
Further, because participants were not aware they would subsequently be required to draw a map, we can be sure they were retrieving spatial information from a mental memory map during the map drawing exercise. Notably, they formed this map from only one exposure to the route and environment. Still, participants from the VGL group drew much more precise maps, which indicates that the global landmarks helped them to process the environment’s orientation and structures more accurately.
Comparing the other behaviors of the two groups, the participants using the VGL system spent significantly more time in the exploration phase, as shown in Fig. 2a. This could simply because the VGL system participants hesitated during exploration, or which is more possible that they walked more slowly and were more willing to look around and attracted attention to their surroundings rather than simply following the instructions they were given. The blink interval readings, given in Fig. 2b, support this idea. A higher blink rate can be interpreted as a function of split attention46,47. In other words, when participants show increased interest in stimuli, the eyes tend to fixate for longer and the blink count decreases. Figure 2b reveals a significant increase in blink interval for the VGL group, which means the participants fixated for longer on their surroundings compared to the group that was not using the VGL system.
Consistent with our behavioral findings, desynchronizations in the theta and alpha power bands of the occipital and parietal sites also suggested an increased attention in visual stimuli in VGL group versus the NVGL group during guided exploration. As indicated in Fig. 5, there were significant differences in the occipital and parietal ERSPs of the VGL group versus the NVGL group in terms of theta oscillation suppression and increased alpha oscillations. We interpret the increased alpha and decreased theta power over these sites as the participants suppressing distractions from their visual system42,43,44,45. Previous studies on short-term memory also found alpha activity could be increased while brain was carrying more memory load42,51. In other words, the suppression of theta activity and the increase of alpha activity signified that more attention was being paid during neural processing. Thus, as the ERSP data for the VGL group shows, our findings with the brain dynamics are consistent with our findings for the behavior results, both demonstrating that participants using the VGL system paid more attention to spatial knowledge acquisition during the exploration phase.
In summary, the study reveals a significant improvement in incidental spatial knowledge acquisition when using the VGL system as reflected in the assessment of sketch maps. As a reference point, the VGL system encourages users to be more actively aware of their surroundings and process their environment. By enhancing user perceptions of navigation information, the VGL system can augment the navigation capabilities of users, which could be the scientific basis for a new generation of future navigation assistance systems that not only show users the way to go but also support spatial learning. In our future work, we will continue studying VGL’s effect on the efficacy of spatial learning with more physical task, e.g., assessing orientation accuracy and efficiency with VGL after a guided exploration.
Materials and methods
Participants
The experiment involved 55 participants: 24 females and 31 males (see Table 1 for demographic information). Experiments were conducted in the UTS Tech Lab. Before participating in the study, the experimental procedure was explained, and all participants provided informed consent. The Human Research Ethics Committee (HREC) of University of Technology Sydney (UTS) also reviewed the protocols and issued their approval (grant number: UTS HREC REF NO. ETH17-2095). All experiments were performed in accordance with relevant guidelines and regulations. None of the participants reported a history of any psychological disorders that could have affected the experimental results. To control for individual differences in spatial abilities, we administered a SBSOD test39 and a PTSOT test40 prior to conducting the experiment.
The VGL system setup
The virtual global landmarks were displayed as transparent, two-dimensional silhouettes of the real landmark in the VR scenario, which serve as a stable reference for the direction of specific locations without disturbing the overall environment. With the displaying of these virtual landmarks, users were encouraged to continuously compute directions from a particular location52. The silhouettes were presented in the direction of the global landmark within the participant’s sightline. Whether walking or turning, as long as the participant looked in that direction, they were able to see either see the real landmark or the silhouette if it was blocked by another object, as shown in Fig. 1. During their trials, the participants were shown one of three virtual global landmarks: a lighthouse, the Sydney Opera House, or the Sydney Tower Eye.
VR and EEG setup
Figure 6b provides an overview of the setup for participants. The Sydney Park scenario was based on VR but imitates the real environment of the Sydney Botanical Gardens. The scenario was fully immersive so as to hold participants’ attention during the full duration of the navigation experiments. We used HTC’s Vive Pro eye headset with an embedded Tobii eye tracker. The Vive Pro eye uses a dual OLED 3.5" diagonal display with a resolution of 1440 × 1600 pixels per eye (2880 × 1600 pixels combined) and a refresh rate of 90 Hz, as reported by HTC. The participant’s head position was principally tracked with embedded inertial measurement units, while an external lighthouse tracking system cleared the common tracking drift with a 60 Hz update rate. We tracked the eye activity of participants using the Tobii eye tracker at a sampling rate of 120 Hz.
The EEG data were recorded continuously using Brain Vision’s LiveAmp 64 system (Brain Products, Gilching, Germany) using 64 active electrodes mounted on an elastic cap. The sampling rate was 500 Hz with a low-pass filter of 131 Hz. The electrodes were positioned according to an extended 10–20 system53. The EEG signals were referenced to the electrode located at FCz and the impedance of all sensors was kept below 5kΩ. EEG events were created when the participants’ fixated on the surface of a defined landmark, both real and virtual. All data streams from the EEG cap, eye tracker and head-mounted display were synchronized with Lab Streaming Layer (LSL).
Sydney Park was created in Unity 2018.3.5f1 (Unity Technologies, USA). Figure 7 presents the screen views of Sydney Park scenario in Unity. The Sydney Park environment consists of eleven local landmarks (Label 1 to Label 11 in Fig. 7a,b) and three global landmarks (Label 12 to Label 14 in Fig. 7b–d, including a lighthouse, the Sydney Opera House and the Sydney Tower Eye), in combination with paths, intersections, bushes, trees, etc. Two sides of the scenario were extended with only the sea, and a lighthouse standing in the corner next to the ocean, as shown in Fig. 7b. Of the remaining two sides, one had a view of the Sydney Opera House along with Sydney Harbour Bridge behind (Fig. 7c) and the other had a view of the Sydney Tower Eye with other tall buildings as a city skyline (Fig. 7d). The background views are similar to the actual views from the Royal Botanical Gardens, whereas the inner layout of Sydney Park is unique. To control the visibility proportion of all global landmarks, we used background plants, e.g., trees, to ensure global landmarks were obscured. The visible status for all global landmarks was checked at each checkpoint (checkpoint map is shown in Fig. 9). Overall, global landmarks were blocked from view for nearly 60% of the time during the experiment.
Experiment procedure
An overview of the experimental procedure is shown in Fig. 6a.
Pre-test
Before the participants performed the exploratory and navigational tasks, we conducted a pre-test (SBSOD39, PTSOT40) to assess their individual spatial abilities. The participants were not aware of the experimental procedure to come while completing this phase. Every participant completed the SBSOD questionnaire as a subjective measure of his or her sense of direction, as well as a PTSOT to evaluate their spatial orientation ability.
Exploration phase
All participants were randomly divided into two groups in this phase, one group explored Sydney Park along with VGL system (VGL group), the other without (NVGL group). Each participant first had five minutes to walk inside a meadow area in the VR environment. Participants were given this time to explore until feel comfortable to do physical movement in the VR environment. If any uncomfortable feedback occurs, the experiment would be terminated. In the next step, participants were given an instruction based on the map of Royal Botanical Gardens and its surroundings to explain the background layout of our Sydney Park scenario as well as the global landmarks. For VGL group, during instruction about global landmarks, VGL was also explained. Except the global landmarks within the background surrounding, participants were not shown any local landmarks as well as the scenario’s inner layout. After the instruction, scenario was presented in VR and participants started walking through the Sydney Park scenario along a fixed, predefined route with the assistance of auditory instructions (see Fig. 6c and Movie S1 in the supplementary). Turn-by-turn auditory instructions were used to build a familiar navigation aid of automatic orientation environment. To control all participants from both groups travelling the same fixed route, once wrong response to the verbal prompts occurred, the instructor who monitored the experiment would immediately ask the participants to stop and instruct them back to the right direction before reaching a different path. This was intended to standardize how participants explored the environment, and in the meantime, the use of auditory instruction could help to observe and investigate the VGL system’ effects on spatial learning in a common guided-navigation environment. All target landmarks involved in the following tasks were passed just twice while navigating the fixed route.
Map drawing task and NASA-TLX questionnaire
After the exploration, each participant was given a blank 11.7 × 16.5-inch sheet of paper, a pencil and an eraser. They were then instructed to draw any information they remembered about the scenario, including landmarks, paths, etc., They were not allowed to conduct the scenario or listen to the instructions again. Following the map drawing task, each participant filled out a NASA Task Load Index questionnaire (NASA-TLX)41.
Sketch map analysis
We analyzed the sketched maps with the Gardony Map Drawing Analyzer (GMDA)27. This software package provides quantitative measures of map accuracy using novel measures, unique to GMDA, that rely on both pairwise comparisons and traditional bidimensional regression parameters54. Figure 8 show examples of sketch maps from participants. During scoring the total map score for each sketch map, we drew upon all the critical measures provided by GMDA, including canonical organization (with a square-root-corrected measure), canonical accuracy, points percentage, rotational and scaling biases, distance accuracy, and angle accuracy. The total map score can be represented by an equation as follows: :
The total map score ranges from −2 to 5 and a higher score indicates an overall better preformation in sketch map. For all the single measures, the canonical organization measures the overall configuration accuracy of the sketch map, with penalties for missing landmarks while canonical accuracy measures the configurational accuracy of only the drawn landmarks. These two measures are all range from 0 to 1 with higher scores indicating better configural accuracy and landmark recall. The points percentage records the successfully recalled points of the explored environment, including landmarks and intersections, as percentages, which ranges from 0 to 1 with higher scores indicating higher proportion of successful recalled points. The scaling bias measures the direction of scaling of the inter landmark distances, which ranges from 0 to 1 with lower scores indicating higher precision on landmark scaling. Rotational bias measures the direction of angular error of inter landmark angles, which ranges from 0 to 360 with lower scores indicating higher precision on landmark rotation; however, to equal this index to others, we took \(\frac{\mathrm{rotational bias}}{360}\) instead as shown in Eq. (1) to make it range from 0 to 1 as well. Distance and angle accuracy reflect the accuracy of inter landmark distances and angles, which all range from 0 to 1 with higher scores indicating more accurate interlandmark distance and angle representation, respectively.
EEG analysis
Pre-processing
All raw EEG data were imported into MATLAB version 2018a (MathWorks Inc., USA) for processing. We used the EEGLAB toolbox version 2020.055 to aid in the analysis. For each participant’s raw data, we first checked the data quality by eye to ensure the EEG data was consistently recording with active movements during experiment. Of the 55 participants, data of one participant from NVGL group was excluded due to poor EEG quality. The raw data for the remaining 54 participants were first bandpass filtered from 1 to 100 Hz and downsampled to 250 Hz. Then, data from each single task were merged into one large EEG dataset for the following pre-processing steps. Line noise and flatlines were removed in turn using the cleanline and clean_flatlines functions in EEGLAB. Noisy channels were rejected with the clean_channels function. All missing EEG channels were interpolated by spherical splines before re-referencing to the average of all channels. Noisy data in the time domain were removed through automatic continuous data cleaning. On average, 48.55% ± 17.79% of the data in the time domain were removed. The data were then submitted to adaptive mixed independent component analysis to obtain independent components55,56. The equivalent dipole model of each independent component was computed using a boundary element head model as implemented in EEGLAB’s DIFIT2 routines57. Last, the sphere and weights of the ICA and dipole models were copied back to the pre-processed but uncleaned EEG single-task data for further analysis. (There was no cleaning in the time domain).
Event-related spectral perturbation (ERSP)
The cleaned data for the exploration was extracted with a time window of [−1 s to 7 s]. The onset and offset events were generated based on checkpoints (see Fig. 9). For each single fixation period between two checkpoints (on the virtual global landmarks in the VGL group or the local landmarks in the NVGL group), the onset event was generated at the first checkpoint, and an offset event was defined at the second checkpoint. Bad epochs were detected and removed based on component activities using the autorej function. On average, 0.08 ± 0.40% epochs of the NVGL group condition were removed; 6.95 ± 2.89% epochs of the NVGL group condition were removed. All trials were then time-warped to time lengths of [−1 s to 6 s], and ERSPs were plotted for each independent component with the newtimef function.
Independent component clustering
The independent components from all participants were first selected with less than 15% residual variation of the equivalent dipole model, and the components with an equivalent dipole model located outside the head sphere were removed. Then, the selected components were clustered using K-means clustering in EEGLAB. To avoid the “double dipping” problem58,59, only dipole locations were included as the measure for clustering. For the ERSP analysis, we focused on the clusters of components located in or near the parietal cortex and occipital cortex. We looked for active neural dynamic differences related to spatial navigation34,60,61,62,63 or visual stimuli42,44 between the VGL and NVGL groups. We used the Talairach client tool64,65 to evaluate the nearest gray matter of the dipole locations from the targeted cluster centroid and clustered components.
Group-level ERSPs and statistics
We first computed the ERSPs at the single independent component level based on the cluster of interest, then averaged them at the participant level, and finally at the group level. The time–frequency data of all independent components from the same participant were averaged. Then, the ERSPs of all participants were averaged for the final ERSPs at the group level. Significant differences from the baseline activity are displayed in red for positive deviations, blue for negative deviations, and green for nonsignificant differences. We determined conditional differences using the newtimef function with a statistical threshold of p < 0.001 for all selected independent components. A global overview of our processing steps for the EEG analysis is provided in Fig. S2 in the Supplementary.
Statistical analysis
The statistical analyses were conducted using SPSS Statistics 26 (IBM Analytics, Armonk, USA). Data visualizations were created with the ggplot function of R66 (RStudio Inc, USA). We computed one-way ANOVAs for the between-subject factor trial by group (VGL and NVGL) for the exploration, as well as for the map drawing task and for eye activity. The dependent values for exploration analysis were the response time (time spent in exploration) and blink intervals. For the map drawing task, the dependent values included the NASA-TLX score and total sketch map score along with all single indexes, i.e., canonical organization, recalled points percentage, rotational bias, scaling bias, canonical accuracy, distance accuracy and angle accuracy. Each measure was calculated separately for the VGL and NVGL group.
For all measures, we first explored the data to check if outliers existed for the within-subject factor computations. All outliers inspected by boxplots for values greater than 1.5 box lengths from the edge of the box were removed. In addition, we ran a Shapiro–Wilk test to determine whether the mean values for both groups were normally distributed. We used Spearman's rank-order correlation to assess the relationship between individual spatial ability factors (SBSOD and PTSOT scores) and all measures. The Spearman’s correlations result between each individual’s spatial ability and the measures are shown in the Supplementary Fig. S1. With the significantly correlated factors, we then used the scores from these tests as covariates to assess how much the participants’ inherent, subjective sense of direction and orientating ability affected their completion of the navigation tasks.
References
Clemenson, G. D., Maselli, A., Fiannaca, A. J., Miller, A. & Gonzalez-Franco, M. Rethinking GPS navigation: Creating cognitive maps through auditory clues. Sci. Rep. 11, 7764. https://doi.org/10.1038/s41598-021-87148-4 (2021).
Malanchini, M. et al. Evidence for a unitary structure of spatial cognition beyond general intelligence. npj Sci. Learn. 5, 9. https://doi.org/10.1038/s41539-020-0067-8 (2020).
Morand-Ferron, J., Cole, E. & Quinn, J. Studying the evolutionary ecology of cognition in the wild: A review of practical and conceptual challenges. Biol. Rev. https://doi.org/10.1111/brv.12174 (2015).
Arnold, A. E. G. F., Protzner, A. B., Bray, S., Levy, R. M. & Iaria, G. Neural network configuration and efficiency underlies individual differences in spatial orientation ability. J. Cogn. Neurosci. 26, 380–394. https://doi.org/10.1162/jocn_a_00491%M24047389 (2014).
Epstein, R. A., Patai, E. Z., Julian, J. B. & Spiers, H. J. The cognitive map in humans: Spatial navigation and beyond. Nat. Neurosci. 20, 1504–1513. https://doi.org/10.1038/nn.4656 (2017).
Babayan, B. M. et al. A hippocampo-cerebellar centred network for the learning and execution of sequence-based navigation. Sci. Rep. 7, 17812. https://doi.org/10.1038/s41598-017-18004-7 (2017).
O’keefe, J. & Nadel, L. The Hippocampus as a Cognitive Map (Clarendon Press, 1978).
Alexander, A. S. & Nitz, D. A. Retrosplenial cortex maps the conjunction of internal and external spaces. Nat. Neurosci. 18, 1143–1151. https://doi.org/10.1038/nn.4058 (2015).
Maguire, E. A. et al. Navigation-related structural change in the hippocampi of taxi drivers. Proc. Natl. Acad. Sci. 97, 4398–4403. https://doi.org/10.1073/pnas.070039597 (2000).
Tolman, E. C. Cognitive maps in rats and men. Psychol. Rev. 55, 189 (1948).
Dahmani, L. & Bohbot, V. D. Habitual use of GPS negatively impacts spatial memory during self-guided navigation. Sci. Rep. 10, 6310. https://doi.org/10.1038/s41598-020-62877-0 (2020).
Ishikawa, T., Fujiwara, H., Imai, O. & Okabe, A. Wayfinding with a GPS-based mobile navigation system: A comparison with maps and direct experience. J. Environ. Psychol. 28, 74–82 (2008).
Gardony, A. L., Brunyé, T. T., Mahoney, C. R. & Taylor, H. A. How navigational aids impair spatial memory: Evidence for divided attention. Spat. Cogn. Comput. 13, 319–350. https://doi.org/10.1080/13875868.2013.792821 (2013).
Aporta, C. & Higgs, E. Satellite culture : Global positioning systems, Inuit wayfinding, and the need for a new account of technology. Curr. Anthropol. 46, 729–753. https://doi.org/10.1086/432651 (2005).
Münzer, S., Zimmer, H. D. & Baus, J. Navigation assistance: A trade-off between wayfinding support and configural learning support. J. Exp. Psychol. Appl. 18, 18–37. https://doi.org/10.1037/a0026553 (2012).
Gramann, K., Hoepner, P. & Karrer-Gauss, K. Modified navigation instructions for spatial navigation assistance systems lead to incidental spatial learning. Front. Psychol. https://doi.org/10.3389/fpsyg.2017.00193 (2017).
Rizzardo, C. A., Colle, H. A., McGregor, E. A. & Wylie, D. Using a single map display both for navigational planning and for turn-by-turn vehicle guidance: Configural spatial knowledge acquisition. J. Exp. Psychol. Appl. 19, 301–319. https://doi.org/10.1037/a0034622 (2013).
Steck, S. D. & Mallot, H. A. The role of global and local landmarks in virtual environment navigation. Presence Teleoper. Virtual Environ. 9, 69–83 (2000).
Credé, S., Thrash, T., Hölscher, C. & Fabrikant, S. I. The advantage of globally visible landmarks for spatial learning. J. Environ. Psychol. https://doi.org/10.1016/j.jenvp.2019.101369 (2020).
Li, R., Korda, A., Radtke, M. & Schwering, A. Visualising distant off-screen landmarks on mobile devices to support spatial orientation. J. Location Based Serv. 8, 166–178. https://doi.org/10.1080/17489725.2014.978825 (2014).
Wunderlich, A. & Gramann, K. Landmark-based navigation instructions improve incidental spatial knowledge acquisition in real-world environments. J. Environ. Psychol. 21, 101677 (2021).
Schwering, A., Li, R. & Anacta, V. The use of local and global landmarks across scales and modes of transportation in verbal route instructions. in Poster Session Presented at the 2014 Spatial Cognition Conference, Bremen, Germany.
Beck, M. R., Goldstein, R. R., van Lamsweerde, A. E. & Ericson, J. M. Attending globally or locally: Incidental learning of optimal visual attention allocation. J. Exp. Psychol. Learn. Mem. Cogn. 44, 387–398. https://doi.org/10.1037/xlm0000428 (2018).
Siegel, A. W. & White, S. H. The development of spatial representations of large-scale environments. in Advances in Child Development and Behavior. Vol. 10. (ed. Reese, H.W.). 9–55. (JAI, 1975).
Wang, J. & Schwering, A. Invariant spatial information in sketch maps—A study of survey sketch maps of urban areas. J. Spatial Inf. Sci. 15, 31–52 (2015).
Gardony, A. L., Brunyé, T. T. & Taylor, H. A. Navigational aids and spatial memory impairment: The role of divided attention. Spat. Cogn. Comput. 15, 246–284. https://doi.org/10.1080/13875868.2015.1059432 (2015).
Gardony, A. L., Taylor, H. A. & Brunyé, T. T. Gardony map drawing analyzer: Software for quantitative analysis of sketch maps. Behav. Res. Methods 48, 151–177. https://doi.org/10.3758/s13428-014-0556-x (2016).
Makeig, S., Gramann, K., Jung, T.-P., Sejnowski, T. J. & Poizner, H. Linking brain, mind and behavior. Int. J. Psychophysiol. 73, 95–100 (2009).
Gramann, K. et al. Cognition in action: Imaging brain/body dynamics in mobile humans. Rev. Neurosci. 22, 593–608. https://doi.org/10.1515/rns.2011.047 (2011).
Gramann, K., Jung, T.-P., Ferris, D. P., Lin, C.-T. & Makeig, S. Toward a new cognitive neuroscience: Modeling natural brain dynamics. Front. Hum. Neurosci. https://doi.org/10.3389/fnhum.2014.00444 (2014).
Huang, R.-S., Jung, T.-P. & Makeig, S. Analyzing event-related brain dynamics in continuous compensatory tracking tasks. in 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference. 5750–5753. (IEEE, 2005).
Du, R. & Lee, H. J. Frontal alpha asymmetry during the audio emotional experiment revealed by event-related spectral perturbation. in 2015 8th International Conference on Biomedical Engineering and Informatics (BMEI). 531–536. (IEEE, 2015).
Oess, T., Krichmar, J. L. & Röhrbein, F. A computational model for spatial navigation based on reference frames in the hippocampus, retrosplenial cortex, and posterior parietal cortex. Front. Neurorobot. https://doi.org/10.3389/fnbot.2017.00004 (2017).
Whitlock, J. R., Sutherland, R. J., Witter, M. P., Moser, M.-B. & Moser, E. I. Navigating from hippocampus to parietal cortex. Proc. Natl. Acad. Sci. 105, 14755–14762 (2008).
Rolls, E. The Parietal Cortex, Spatial Functions, and Navigation. 363–378. (2020).
Clark, B. J., Simmons, C. M., Berkowitz, L. E. & Wilber, A. A. The retrosplenial-parietal network and reference frame coordination for spatial navigation. Behav. Neurosci. 132, 416–429. https://doi.org/10.1037/bne0000260 (2018).
Grill-Spector, K. et al. A sequence of object-processing stages revealed by fMRI in the human occipital lobe. Hum. Brain Mapp. 6, 316–328. https://doi.org/10.1002/(SICI)1097-0193(1998)6:4%3c316::AID-HBM9%3e3.0.CO;2-6 (1998).
Nehmad, O. D. L. The end in sight: A look at the occipital lobe. Clinical Eye and Vision Care 10, 125–133. https://doi.org/10.1016/S0953-4431(98)00013-7 (1998).
Hegarty, M., Richardson, A. E., Montello, D. R., Lovelace, K. & Subbiah, I. Development of a self-report measure of environmental spatial ability. Intelligence 30, 425–447. https://doi.org/10.1016/S0160-2896(02)00116-2 (2002).
Hegarty, M. & Waller, D. A dissociation between mental rotation and perspective-taking spatial abilities. Intelligence 32, 175–191. https://doi.org/10.1016/j.intell.2003.12.001 (2004).
Hart, S. G. & Staveland, L. E. Development of NASA-TLX (task load index): Results of empirical and theoretical research. in Advances in Psychology. Vol. 52. 139–183. (Elsevier, 1988).
Jensen, O., Gelfand, J., Kounios, J. & Lisman, J. E. Oscillations in the alpha band (9–12 Hz) increase with memory load during retention in a short-term memory task. Cereb. Cortex 12, 877–882. https://doi.org/10.1093/cercor/12.8.877 (2002).
Benedek, M., Schickel, R. J., Jauk, E., Fink, A. & Neubauer, A. C. Alpha power increases in right parietal cortex reflects focused internal attention. Neuropsychologia 56, 393–400. https://doi.org/10.1016/j.neuropsychologia.2014.02.010 (2014).
Kelly, S. P., Lalor, E. C., Reilly, R. B. & Foxe, J. J. Increases in alpha oscillatory power reflect an active retinotopic mechanism for distracter suppression during sustained visuospatial attention. J. Neurophysiol. 95, 3844–3851. https://doi.org/10.1152/jn.01234.2005 (2006).
Friese, U. et al. Oscillatory brain activity during multisensory attention reflects activation, disinhibition, and cognitive control. Sci. Rep. 6, 32775. https://doi.org/10.1038/srep32775 (2016).
Barton, K., Smilek, D. & Ellard, C. The relationship between blink rate and navigation task performance. J. Vis. 10, 1040–1040. https://doi.org/10.1167/10.7.1040 (2010).
Holmqvist, K. et al. Eye Tracking: A Comprehensive Guide to Methods and Measures (OUP, 2011).
Billinghurst, M. & Weghorst, S. Use of sketch maps to measure cognitive maps of virtual environments. in Proceedings—Virtual Reality Annual International Symposium. 40–47. (1995).
Billinghurst, M. & Weghorst, S. The Use of Sketch Maps to Measure Cognitive Maps of Virtual Environments. (1995).
Blades, M. The reliability of data collected from sketch maps. J. Environ. Psychol. 10, 327–339. https://doi.org/10.1016/S0272-4944(05)80032-5 (1990).
Klimesch, W. EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Res. Rev. 29, 169–195 (1999).
Singh, A. K., Liu, J., Cortes, C. A. T. & Lin, C.-T. Virtual global landmark: An augmented reality technique to improve spatial navigation learning. in CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI '21 Extended Abstracts) (ACM, 2021).
Chatrian, G., Lettich, E. & Nelson, P. T. percent electrode system for topographic studies of spontaneous and evoked EEG activities. Am. J. EEG Technol. 25, 83–92 (1985).
Friedman, A. & Kohler, B. Bidimensional regression: Assessing the configural similarity and accuracy of cognitive maps and other two-dimensional data sets. Psychol. Methods 8, 468 (2003).
Delorme, A. & Makeig, S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21 (2004).
Palmer, J. A., Kreutz-Delgado, K. & Makeig, S. AMICA: An adaptive mixture of independent component analyzers with shared components. in International Conference on Independent Component Analysis and Signal Separation. 854–861. (Springer, 2012).
Oostenveld, R. & Oostendorp, T. F. Validating the boundary element method for forward and inverse EEG computations in the presence of a hole in the skull. Hum. Brain Mapp. 17, 179–192 (2002).
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. & Baker, C. I. Circular inference in neuroscience: The dangers of double dipping. J. Vis. 8, 88–88. https://doi.org/10.1167/8.6.88 (2008).
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F. & Baker, C. I. Circular analysis in systems neuroscience: The dangers of double dipping. Nat. Neurosci. 12, 535–540. https://doi.org/10.1038/nn.2303 (2009).
Tzagarakis, C., West, S. & Pellizzer, G. Brain oscillatory activity during motor preparation: Effect of directional uncertainty on beta, but not alpha, frequency band. Front. Neurosci. https://doi.org/10.3389/fnins.2015.00246 (2015).
Tzagarakis, C., Ince, N. F., Leuthold, A. C. & Pellizzer, G. Beta-band activity during motor planning reflects response uncertainty. J. Neurosci. 30, 11270–11277 (2010).
Tan, H.-R.M., Leuthold, H. & Gross, J. Gearing up for action: Attentive tracking dynamically tunes sensory and motor oscillations in the alpha and beta band. Neuroimage 82, 634–644. https://doi.org/10.1016/j.neuroimage.2013.04.120 (2013).
Nitz, D. Parietal cortex, navigation, and the construction of arbitrary reference frames for spatial information. Neurobiol. Learn. Mem. 91, 179–185 (2009).
Lancaster, J. L. et al. Automated Talairach atlas labels for functional brain mapping. Hum. Brain Mapp. 10, 120–131 (2000).
Lancaster, J. et al. Automated labeling of the human brain: A preliminary report on the development and evaluation of a forward-transform method. Hum. Brain Mapp. 5, 238–242 (1997).
Team, R. C. R: A Language and Environment for Statistical Computing. (2013).
Acknowledgements
This work was supported in part by the Australian Research Council (ARC) under discovery grant DP210101093 and DP220100803. Research was also sponsored in part by the Australia Defence Innovation Hub under Contract No. P18-650825, the US Office of Naval Research Global under Cooperative Agreement Number ONRG—NICOP—N62909-19-1-2058, and the AFOSR—DST Australian Autonomy Initiative agreement ID10134. We also thank the NSW Defence Innovation Network and the NSW State Government of Australia for financial support in part of this research through grant DINPP2019 S1-03/09 and PP21-22.03.02.
Author information
Authors and Affiliations
Contributions
All authors conceived the study, J.L. performed the experiments, analyzed the results, and wrote the manuscript. A.K.S. and C.T.L. supervised the experiment and data analysis, and refined the structures and content of the manuscript. All authors discussed the results and edited the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Supplementary Video S1.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Liu, J., Singh, A.K. & Lin, CT. Using virtual global landmark to improve incidental spatial learning. Sci Rep 12, 6744 (2022). https://doi.org/10.1038/s41598-022-10855-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-022-10855-z
- Springer Nature Limited