Introduction

The development of environmentally conscious behavior depends on many various parameters. Ernst and Theimer mention in their literature review [1] the interplay of knowledge about the natural environment and outdoor experiences. Augmented reality (AR) can be used to enrich the environment with additional information, interaction capabilities, and visualization. Azuma [2] defines three conditions for AR and most more recent agree [3] with this definition: real environment combined with virtual objects, real-time interaction, and registration in 3D [2]. Augmented reality is applied to education in many cases, but its implementation outdoors, in nature, and in an environmental education context is not common [4], and there is a need for more research in this area. Furthermore, augmented reality is mainly used for visualization, which puts users in a rather passive state.

To enable user activity, we provide an interactive environment and investigate two interaction options with a HMD (head mounted display) in a scenario outside in nature. In our paper, also in the shorter version [5], we evaluate the advantages and disadvantages of near and far interaction techniques with respect to the specific outdoor situation.

The AR modules that we developed can be adjusted and incorporated into various educational formats, which are described in “Educational Use Cases”. For our augmented reality modules (also called mixed reality by Microsoft) we used for hardware a HoloLens 2. Specifically, we investigate the different task performance of near versus far interaction in a tidy and even-leveled environment compared to a more natural environment.

In previous user studies, we have also explored how different rendering modes [6] can be implemented in outdoor AR environments and the advantages and disadvantages of using virtual holes and occlusion with regard to depth perception and merging of the real and virtual world.

Based on the three user tests conducted over the last two years, we have analyzed and summarized important points to consider when deploying an AR application in nature or when conducting user tests during and after AR application development. The locations where our user tests were conducted were all under trees - in a park, at the end of a field, and next to a quiet road.

Numerous considerations also need to be taken into account when designing and developing outdoor AR application modules. Here, we describe the developed AR modules and propose recommendations for researchers and practitioners, organized into the topics design and development (see Proposed Practices for Design and Development of Outdoor Augmented Reality”) and usage in the field (presented in the form of “Lessons Learned”, “Lessons Learned Using Augmented Reality Applications in the Natural Environment”).

The first conducted user tests focused on different rendering and occlusion modes for showing AR underground visualization on site. In our last user tests, the main interest was on the interaction techniques using AR outdoors. Relevant related work to these cases and in general to outdoor AR is presented below.

Related Works

In the field of outdoor AR for nature, there has been past work for school field trips. In a setting explained by Kamarainen et al. [7], students follow a carbon or oxygen atom through the environment and can in this way better understand the carbon cycle in ecosystems.

An important cue for depth perception in AR is occlusion [3]. Occlusion is also an essential part of a seamless AR integration when displaying hidden underground objects. Showing underground infrastructure is a well-known example of a use case for AR. Zhang et al. [8], Schall et al. [9], and the work reported by Behzadan et al. [10] use AR to represent pipes that are naturally located underground.

Older work demonstrates various techniques for visualizing and interacting with hidden objects. Kolsch et al. [11] introduce a rendering mode that is designed like a tunnel and allows users to mask out objects that would otherwise occlude hidden objects. It is divided into zones with different renderings: focus regions with fully rendered objects, regions where only wireframes are rendered and transparent regions. The user can move the focus region to interactively explore 3D information at any distance such as entire room complexes in buildings. For use in nature, such a feature seems too complicated and is not essential. Moreover, we do not want to visualize too complex data that obscures a large part of the user’s field of view - nature should still be visible.

White et al. [12] present prototypes of a mobile AR electronic field guide and techniques for displaying and inspecting computer vision-based visual search results in the form of virtual vouchers. In their work they define virtual vouchers as digital representations of botanical reference specimen in conjunction with their characteristic data. Their research focuses on head-movement controlled augmented reality for hands-free interaction and tangible augmented reality. Although their system is for outdoor use, they evaluated the prototypes indoors in controlled environments and collected feedback from lab trials by botanists. In contrast, our experiments are conducted outdoors by people interested in nature or working in nature organizations.

Kruijff et al. [13] integrate multiple data sources and components into the HYDROSYS system to enable interactive environmental data analysis and monitoring in the field. Their system includes scalable streaming of environmental sensor data and mixed reality representations that embed multivariate sensor data visualizations. There is also support for public participation. Even earlier, the usage of AR outdoors was of interest. In 2006, King et al. [14] described the ARVino system, which uses the combination of AR and GIS to visualize GIS data outdoors to understand the parameters affecting yields and quality of grapes from different vineyards. The AR system being discussed uses a mobile computer on a tripod and an umbrella. This type of hardware is nowadays replaced by more advanced devices such as an HMD used in our tests.

We have recently provided a survey of the use of AR in nature and AR applications dealing with environmental aspects (Rambach et al. [4]).

As mentioned in our work, focused on interaction techniques outdoors [5], a lot of research has been conducted in the area of interaction. Nizam et al. provide an empirical study on the main aspects and problems of multimodal interaction with AR [15]. The modality that receives the most attention is the gesture interaction technique. In a different work, Chen et al. investigate the possibility of combining two input modalities, gestures and speech, to improve the user experience in AR [16]. Their results show that both gestures and speech are effective interaction modalities for performing simple tasks in AR.

Several papers compare direct and indirect interaction, either gestures combined with speech [17] or, e. g., Lin et al. investigate direct interaction versus interaction with the use of a controller [18] Other work examines two-handed gesture interaction with a focus on rotation and scaling [19]. In contrast, our work [5] compares near and far interaction for selection and translation tasks in different environments in nature.

The effectiveness and usability of AR interactions in dependence of the distance to the virtual objects is explained in the work of Whitlock et al. [20]. The device used (AR-HMD Microsoft HoloLens 1) is combined with a Nintendo Wiimote as an additional handheld input device. Their results indicate that embodied freehand interaction is the most preferred form of interaction by users. They study distances from 2.44 to 4.88 m (8–16 feet), all of which can be considered remote interaction and do not include the near interaction also observed in our work.

Recently, Williams et al. [21] focused on understanding how users naturally manipulate virtual objects based on various interaction modalities in augmented reality. These manipulations are composed of scaling, translation, rotation, and more abstract types such as creating, destroying, and selecting. For this experiment, they use a Magic Leap One, an optical see-through AR-HMD. They discovered that when manipulating virtual objects, the use of direct manipulation techniques is more natural for translations. This is also the type of interaction that we explored in our user tests.

There are other researchers who compare different interaction metaphors (e.g., Worlds-in-Miniature), which can also be viewed as near and far interaction (see, e.g., [22]). Some previous work has even attempted to merge near and far interaction into a single interaction metaphor [23, 24]. However, none of these previous works dealt with a natural outdoor environment, which was essential for the motivation of our work.

Educational Use Cases

The outdoor AR modules can be embedded into various educational scenarios.

As extracurricular learning venues, the so-called Green Schools are highly popular for all German schools. This context offers a use case where outdoor AR modules can be integrated for topics such as forest knowledge, trees and roots (see Fig. 1). The students can creatively experience the forest and the teacher can use AR to visualize the connection between weather, climate and trees on site. These venues are often integrated in parks or zoos. In these programs students have a day outside of school and undertake trips to learn in and about nature. With the AR modules hidden information and visualization can enrich the experience and provide out-of-the-ordinary insights.

Fig. 1
figure 1

A student using AR nature application to see an underground visualization of birch tree roots

Full size image

Outdoor AR modules can also be used during guided hikes in nature, during “forest rallies” with special stations where environmental AR games are played. Different themes can be explored with geogames. A game about a wildcat in the region of Rhön is presented here.Footnote 1 Games about forest, bird protection, moorland and more are integrated in the application uRnature.Footnote 2 These apps use GPS to navigate visitors along a path and guide them to places where tasks can be solved, thus enriching their experience on site. These kind of applications or concepts can be extended with AR modules to make the connection between game and environment even stronger and to open up new opportunities for augmented learning. With AR, digital natives can be enabled to actively explore their surroundings and learn in and about nature through playing.

Another possible use case for outdoor AR modules can be during action days for innovative forms of environmental education. The role of Extracurricular Education for sustainable development (ESD) is also mentioned by the German UNESCO Commission,Footnote 3 as it needs to be less aligned with formal structures and can therefore have considerable potential for innovation. Examples are environmental education centers, museum educational departments, science centers, youth and environmental associations.

The integration of AR in these kind of use cases is not a typical area [4], but carries a lot of potential and should be explored more intensively. Therefore, recommendations based on tests in the fields can be useful for future development and for optimizing outdoor AR experiences.

Design and Implementation of AR Outdoor Modules

Next, we provide an overview of our AR application, the implemented near and far interaction techniques and the visualizations for outdoor use in nature on a tree. A user-centered design approach [25,26,27] was adopted for the development of the educational AR modules. After discussions and interviews with experts in the field of environmental education and augmented reality, personas and user task scenarios were developed and later modified during the development cycles. The main target groups for the AR modules are:

  • persons who are interested and curious about nature

  • “digital natives” who like to use new technologies while learning

  • experts in the field of environmental education who want to use innovative solutions for learning about nature and environment

According to the target group, the AR application can offer various advantages. People who are interested in nature can learn through AR in a new way. They can discover hidden things in the forest thanks to the “magical” possibilities of AR. On the other hand, people attracted to technologies can be encouraged to learn more about nature. Initial field user testing described in our earlier work [6] was valuable for the visualization of AR in the underground, which is part of the “Growing Roots” module (see “Growing Roots”). User tests for the different interaction techniques in outdoor scenarios are discussed in more detail in this paper.

Near and Far Interaction for AR Outdoor Modules

As we chose to focus on studying and examining how people interact in an outdoor scenario, we developed interactive AR components for the example tree. These components integrate near and far interaction techniques and are described in the following.

Touchable Tree

With this feature the interaction between learner and AR starts. Appearance of virtual leaves is triggered by touching a real tree stem (see Fig. 2). The leaves of different tree species fall to the ground around the tree. This virtual-real interaction strengthens the connection between the real environment and the AR visualization, as this is by definition one of the goals for using AR. Without any connection to the real world, virtual reality could be applied to visualize objects. We used a movable virtual stem to decide which real tree should be taken for the visualization. Here, the first idea was to use the surface magnetism solver [28] to position the visualization. This feature from the Mixed Reality Toolkit would be a good option for visualizing objects on walls or floors, as they would automatically appear and “stick” to the walls. In nature, most objects do not have flat and large surfaces and therefore surface magnetism does not work well. We made the decision to have the virtual tree trunk positioned manually and thus more precisely by an advanced user, who could be a teacher in an educational context or an organizer of educational field trips in the nature. After positioning the virtual tree on the desired real tree, the teacher confirms this and the virtual trunk “disappears”.

From a technical point of view, this is done by using a shader that renders the tree stem invisible. The virtual tree is no longer movable, but can be used to show virtual leaves on the ground. The user (in the educational context a student) can now touch the tree and the virtual leaves fall to the earth.

Fig. 2
figure 2

By touching the real tree virtual leaves fall to the ground (picture also used in [5])

Full size image

Collecting AR Leaves

Over 30 virtual leaves (three different types) fall to the ground (see Fig. 3) after the user touches the real tree. User can grab the leaves by directly manipulating them with their hands, which is detected by HMD HoloLens 2  [29]. We refer to this type of interaction as near interaction. The users can then bring them to a virtual bucket, which also appears next to the tree. When a leaf is placed in the bucket, a timer and counter are activated, and users can see the running time and the number of leaves collected above the bucket. This functionality of the bucket is implemented for the user tests (see “User Tests”) to measure time performance. The second way to collect the virtual leaves is by using far interaction (also called “point and commit with hands”  [30]), which is also supported by the HMD HoloLens 2. Here, a virtual ray emanating from the hand is used to select the leaf and then the object can be moved using the grasping gesture familiar from direct interaction. A detailed scenario for the user tests is explained in depth in “User Tests”.

Fig. 3
figure 3

Different types of leaves lie on the ground and can be collected in a bucket (picture also used in [5])

Full size image

Clouds and Rain

One more interactive AR module in our application, inspired by nature, shows virtual clouds a few meters away from the tree. These clouds (rendered in low-poly style) can be dragged by the users through near or far interaction. When they are dragged near the tree, virtual rain begins to fall. To visualize the rain, we combined several particle systems [31] in the Unity engine.

Visualization of Natural Underground Processes in the Forest

To take advantage of AR’s ability to make the invisible visible in nature, we created two AR visualizations that are part of the AR application for the user tests. The modules are described below.

Underground Water

After the rain is triggered by dragging the clouds, the underground water molecules are visualized by blue spheres (see Fig. 4). The spheres are animated and gradually shrink over time to disappear into the ground and tree roots. A particle system is used to create this effect.

Fig. 4
figure 4

Part of the tree root and blue water particles (picture also used in [5])

Full size image

Growing Roots

The largest hidden virtual object in our case is the tree root (see Fig. 4). We used a 3D model that extends three meters in each direction from the tree trunk. To show the growing roots, we animated a root strand using the computer graphics software Blender [32]. The animation is played in a loop so that the user can view it more than once. The animation begins when the rain starts to fall. Since there are trees of different sizes in nature, an adjustable size of the roots is necessary. We provide functions to scale the model, and users can control the size gradually with voice commands (“bigger”/“smaller”).

Indoor vs. Outdoor

Over the development cycle, we conducted and compared indoor and outdoor tests. In the beginning, for example, flat leaves were used for the collecting task. This was quite easy indoors, but rather difficult and uncomfortable outdoors. Some of the users were not able to grasp the leaves. To ensure that the leaves are easier to collect even on uneven surfaces outdoors, we added (invisible) colliders to the leaves that are a few centimeters larger with respect to the y axis. In this way, our modules can also be used in an area overgrown with grass and small branches. Besides, if the ground is wet after a rain shower, it might be uncomfortable for users to touch it through direct manipulation. With larger colliders, they can keep grasping the virtual leaves almost naturally, but they do not have to get their hands dirty.

With a focus on depth perception and visualization of underground objects like tree roots, in our recent work [6] we investigated different rendering options using virtual holes and various textures in an outdoor scenario. We found out that the quality of depth perception with a HMD is high due to the availability of stereo rendering already. Therefore, we decided to render the tree root in this study without any additional masking so that it could be seen in its full size. This would not be very convincing in an indoor environment, as there is usually less space and the roots would go outside of the building when rendered in full size. The context of nature would obviously be missing indoors.

To achieve a good visualization of the rain outdoors, we used brighter colors because outside there is more light and the raindrops merge with the background more than indoors.

An outdoor location in nature can be more dynamic than an indoor location. Depending on the weather or season, light changes may occur. Unplanned changes in the environment, such as fallen branches or overgrown grass, can also hinder the use of the application on a particular tree. Therefore, it may be necessary to choose a tree on the fly - with enough shade or enough space around the tree. To allow more flexibility, we have included features in the AR application to scale the tree trunk and roots using voice commands. In addition, the teacher can choose between different types of trees so that different virtual leaves can be selected depending on the situation on site.

More differences and recommendations for AR in the nature are given in “Recommendations for Design, Development and Usage of Augmented Reality Applications in Nature”.

Methods and Experiments

To examine near and far interaction in AR with differently sized objects (small and bigger) in two different nature settings outdoors, we applied a formative evaluation. We developed task-based scenarios in order to compare the different interaction techniques in the different natural surroundings. Our evaluation was focused on user impressions with respect to the two variants of the interaction technique, the task performance using the different interaction techniques, and usability issues. We assumed that each technique had specific advantages and disadvantages.

User Tests

Weather conditions are very important when conducting user tests outdoors. We chose days without precipitation (e.g rain and snow), so that the hardware can be used easily. Although the tests were performed in the summer, some appointments had to be postponed due to rain. To ensure that the light conditions are suitable, the tests were performed in the morning or the early afternoon. A setting under a tree has been chosen, so there was enough shadow, that an optical see through device could be used also on sunny days.

The tests started with calibrating the HMD for the current user. This way the precision for the interactions is increased. During the tests we observed the participants and took notes when they commented on the application.

Using the module “Collecting AR Leaves” (“Collecting AR Leaves”) we measured the user task performance. We chose task completion time as a metric since this is a commonly used metric for evaluation of interactions in virtual and augmented reality [33].

The experiment was performed under an oak tree where users had to collect five oak leaves and put them in a virtual bucket, which was placed close to the tree, as fast as possible. The users had to choose from three leaf types—birch, oak and maple.

  • In a preparatory stage, the users were encouraged to try out the different techniques. These techniques had been explained to them beforehand.

  • The first task was to collect the virtual oak leaves using the direct manipulation with hands (near interaction, see Fig. 5 top).

  • The second task was to collect the leaves using far interaction (see Fig. 5 down).

We recorded the times it took users to complete the both tasks. Collecting virtual leaves was selected as a task to compare the different interaction possibilities. The overall scenario described so far was chosen for two reasons. First, a previous study on the use of AR outdoors [6], which focused on depth perception, found that a good fit of the displayed and the context plays an important role—e. g., users rejected a visualization of a grid over a virtual hole to enhance depth perception in the forest because a grid is not what one would expect in nature. Leaves are part of the forest and collecting them during a walk is not an atypical action. Second, a task to collect only a specific type of tree leaves can be used in a learning or educational scenario.

Fig. 5
figure 5

The two investigated interaction techniques. Top: Direct manipulation with hands (near). Down: Point and commit with hands (far).(Pictures also used in [5])

Full size image

Each of the above mentioned tests had to be performed at two different locations by each user. First, in a tidy and even-leveled area in a park alley, under an oak tree. Second, in a more natural area with grass and branches under another oak tree. This way we also wanted to compare the times of tasks in different types of outdoor environments.

Additional task for the participants (also to be performed at both locations) was to move virtual clouds to the tree. When participants managed to do that, a virtual rain started to fall as described in “Clouds and Rain”. Additionally, water molecules appeared under the ground and a branch of the tree root began to grow. Similar scenarios can be used in an educational setting to show hidden processes in nature or, e.g., droughts caused by global warming and its connection to trees and roots. The participants had to observe and comment what they saw and after the three tasks, they had to fill in the questionnaire that is described in “Questionnaire”.

Working Hypotheses

We defined five working hypotheses (H1–H5) which are described next.

  1. H1:

    There will be a difference regarding performance between near and far interaction.

    • We expect the two interaction methods to perform differently. From first short tests we noticed, that different people had a different preferred method. To investigate this, the time performance had to be compared and in the questionnaire there were statements for subjective preference.

  2. H2:

    Older participants (age \(>55\) years) will prefer using far interaction.

    • About one third of the test users were 55 or older. We thought that the necessity to bend down in order to collect items from the ground could be especially uncomfortable or tiring over time for older people. Therefore they could prefer to collect the items from distance using the far interaction method with a ray.

  3. H3:

    Users will be faster in completing the tasks in a tidy, even-leveled area.

    • We expect that, comparing the performance in the two locations, in a flat area, the users need less time to perform the task (collecting virtual leaves) because the users can walk easier and the ground is even. Also, grasping the leaves should be easier when no or at least less real objects on the ground are interfering with them.

  4. H4:

    For more participants near interaction for collecting items would be easier.

    • We expect, that near interaction is more natural and easier for collecting objects, as this type of interaction is more similar to the real-world grasping and the users are more familiar with it than they are with the interaction using a ray from a distance.

  5. H5:

    Far interaction will be preferred over near interaction in an area being more natural.

    • In contrast to H4, here we are considering the question if the type of environment plays a role regarding the interaction technique preferred by the users. We expect that in wilder environments people would prefer to interact from distance instead of touching the real grass, leaves, branches or to walk between bushes to reach an object.

Participants

The experiments were performed with a total of 20 people (2/3 male, 1/3 female). The youngest participant, a child, was ten years old. The child was accompanied by one of its parents, who gave permission to perform the experiment. The oldest participant was 78. Overall, participants had an average age of 42 years. A more detailed age distribution is provided in Table 1.

Table 1 Participants characteristic—age (also shown in [5])
Full size table

The results obtained from our experiments are intended to be useful for an educational module in nature. Therefore, we focused on two types of participants: people who work in the field of or are interested in nature and environment (11 participants) and users who already have previous experience with AR or study/work in the area of computer science (9 participants). This way we were able to collect opinions from the first group that can represent teachers or organizers of educational excursions in the nature. Some of these participants work in nature organizations or in the area of environment and plants. The second group can be representative for student groups or digital natives which enjoy working with technical devices.

Questionnaire

All participants had to complete a questionnaire (see Fig. 6) in addition to the user tests. This helped to obtain further information and to capture the subjective experience of the participants. For the questionnaire we adopted and adjusted statements from SUS [34] and NASA Task Load Index [35]. The first part of the questionnaire was completed after the experiment at the first testing location. The remaining questions were answered at the end of the whole user test. Questionnaire metrics, that can be considered for measuring usability [33] are integrated in our statements. The used metrics comprise ease of use (S2, S6), comfort (S2, S3, S6, S7), enjoyability (S4, S8) and fatigue (S3, S7). An expected problem regarding the spatial manipulation of virtual objects is the “gorilla-arm” effect [36] when moving a lot of objects using near interaction and holding the arm high. This can cause fatigue and negatively affect user comfort.

Fig. 6
figure 6

Statements from the questionnaire for the collecting leaves task (lower rows for the natural area marked with a graphic of leaves)

Full size image

The questionnaire also comprised free text question (“You triggered the falling of the leaves by touching the real tree trunk. How did you perceive this real-virtual interaction?”) to obtain more qualitative data.

In the next section we describe the results of the user tests, that were reported in the shorter version of our paper [5] as well.

Results

With respect to task completion time and the hypotheses H1 and H3, the average time of all completed tests was calculated for the four combinations of interaction techniques and locations (see Fig. 7). Participants were faster at the more natural location, which was also the second location. There was a difference of about two seconds between the near interaction (collecting directly with the hands) and the far interaction (selecting using the ray)—users were somewhat faster when they collected the leaves directly with their hands. It should also be mentioned that some of the older participants had problems collecting the virtual leaves and could not finish the time-based tasks. These were not taken into account when calculating the results.

Fig. 7
figure 7

Results for the average task completion times for collecting five leaves (in seconds) with the different interaction techniques at the different locations (results also shown in [5])

Full size image

One further unsuspected result (H2) is that older participants preferred collecting the leaves directly a little more. In this case, the more natural interaction is to some degree stronger than the advantage of not having to bend down and walk.

Following the data from the filled questionnaires (see Table  2) and after reading and analyzing the observation notes we can state, that our assumption, that users will find the direct interaction technique easier (H4) is not correct. The results for both techniques (near and far) show minimal difference.

Table 2 Results for the statement: “Picking up the leaves was easy after a short period of familiarization” (also shown in [5]))
Full size table

Looking at the results, it can be seen that there is no significant difference between the far and near interaction in a natural area (H5). For the near interaction the mean value was 4.3 with SD = 0.78, for the far interaction the mean value was 4.05 with SD = 1.20. 3/4 of the users agree or fully agree in the natural area that both were easy—collecting the leaves with far (15 persons) and with near (16 persons) interaction.

Concerning the factors of enjoyability and fatigue, it is worth mentioning that most users agreed or fully agreed that collecting virtual leaves using near and far interaction was fun and that each interaction was not tiring.

In the free text question about how users perceive the real-virtual interaction with the tree, 65 % of the participants made positive comments, e.g. “surprisingly real”, “The interaction was intuitive as it established the connection to reality and thereby slowly introduced to the AR world”, “a great experience”. One third of the users could not see the virtual leaves falling, they saw them first when they were lying on the ground. The HMD’s small field of view could be one reason. Additional visual feedback or longer and slower falling leaves can improve the perception.

Recommendations for Design, Development and Usage of Augmented Reality Applications in Nature

General design guidelines and many important points about interaction techniques are summarized by LaViola and Kruiff [37]. We followed their directions and supplemented them with new modifications and findings from our testing and development. Recommendations for outdoor AR in the nature are described next.

Proposed Practices for Design and Development of Outdoor Augmented Reality

The design of the 3d models used in an outdoor AR application is important.

  • Adjust size of 3D models according to device capabilities

    The size of the models in terms of resolution must be chosen according to the capabilities of the used mobile device, as there is no direct power supply on site. The capability of the battery can be extended via powerbank, but a permanent usage would be not practical and comfortable over time.

  • Adjust colors and contrasts according to context and device type

    When choosing colors and contrasts, it should be noted that black and dark colors appear transparent on optical see-through devices as the one used here (HoloLens). Therefore, they should be used with this fact in mind or avoided.

  • Virtual models should be designed flexible

    Flexibility is very important when designing for a changing environment like nature. Since in nature one can encounter uneven ground, it can be recommended to model interactive 3d, but flat virtual objects with slightly larger colliders. This way objects can still be grabbed in a natural way with direct manipulation, but also with a small distance to the virtual object. Our example with the tree leaves was discussed in “Indoor vs. Outdoor”.

    Positioning and visualization of virtual objects can be adapted to the local spatial conditions. Indoors, objects can be positioned against walls. In nature there are usually no walls or other flat surfaces, and objects should be rather flexible. Designing the objects draggable enables a more comfortable and precise positioning. In our case, the virtual tree trunk used for selecting the desired real tree was designed draggable. Here we also followed the tip Reduce degree of freedom when possible [37]. The tree trunk manipulation includes changing the position but not the rotation, as a tree trunk has a typical vertical orientation and does not need to be rotated in each axis. If the positioning of the objects should be more stable and static a voice command can be used to enable or disable the draggable functionality of a virtual object.

  • Integrate real environment in AR application

    The use of voice commands can be very practical to flexibly change the size of 3D models. In this way, adaptation to the real environment can be achieved. For example, in our AR modules it is possible to adjust the size of the virtual tree to the size of the real tree on which it is visualized, using the commands “bigger” and “smaller”, as already explained in “Growing Roots”.

    The inclusion of the real environment is by definition an important aspect of AR. A meaningful link between the real world and virtual objects opens up impressive, sometimes “magical” perspectives for the use of AR. In our AR application, for example, a real tree is used as interaction start. Users touch the tree and the virtual leaves fall on the (real) ground. The real-virtual connection was also present in the user tests (the interaction technique comparison task, see “User Tests”). The users had to collect oak leaves. The participants who did not know for sure how oak leaves look like had to explore and compare the virtual leaves with the real leaves in nature.

  • Allow usage of flexible interaction techniques

    When designing AR for outdoor use, the weather and the environment play an important role. In terms of interactive AR applications, as described in this paper (see “Near and Far Interaction for AR Outdoor Modules”), flexible interaction techniques are essential. For example, if the ground is wet or muddy, it is more comfortable to grab the 3d objects without touching the ground. In our AR modules, two interaction techniques can be used for more comfort in this scenario: direct manipulation (with larger colliders) or far interaction (ray from a distance). Hereby the “tips” [37] Nonisomorphic (“magic”) techniques are useful and intuitive, as well as Use pointing techniques for selection and grasping techniques for manipulation are followed and adapted. Although it is good to have some choices for the interaction techniques, too many different ones are not recommended, as the users need to know how to use them.

  • Provide feedback

    Another aspect to be considered is the feedback. Visual feedback should support unusual interactions for users. Let us consider the example of the tree again: users touch the tree and see how the leaves fall afterwards. It is important to superimpose feedback in the user’s field of view and make animations run longer or repeatedly. Animations should not only be applied to the real tree, but two to three leaves can fall directly into the user’s view area. In addition, the user can also receive auditory feedback.

Lessons Learned Using Augmented Reality Applications in the Natural Environment

When conducting user tests outdoors or when using AR applications, in nature, there are some special aspects to be aware of in contrast to indoor tests and usage.

The realization of user tests is considerably easier and the experiments can be performed in a much more controlled fashion if the researcher is to some degree familiar with the environment and starts the user tests “prepared”. It is advisable to visit the location where the user tests will be carried out in advance and examine whether any special preparations need to be undertaken before the actual tests, experiments or, in general, the use of the AR modules take place. To ensure that the time outside is comfortable for the participants, it may be, for example, useful to provide insect repellent and water in the summer.

Organizing outdoor user tests during development and usage scenarios with the finished AR modules is more time consuming and uncertain than in a more controlled indoor environment. For example, user testing with HMD devices should only take place when the weather is suitable and there is no rain or snow forecast for the day. Therefore, a researcher should allocate more time to this phase of a study and plan carefully, if possible in the drier seasons. Bringing an umbrella is recommended, as well as a blanket or something to put your hardware or notepad on while out in nature.

When performing the individual user tests, only as many participants should be scheduled as the battery time of the hardware allows. Alternatively, a break could be incorporated to charge the devices via a portable charging option such as a power bank. This could extend the time for the user tests. The battery of the hardware should always be fully charged before starting the outdoor activities.

Some AR devices, such as the HoloLens in our case, are still very expensive. If only one device is available, the number of participants in the user tests should not be too high in order to avoid long breaks, because in outdoor tests, participants also have the additional time to travel to the site. If the AR application is used in an actual learning scenario, more than one device is recommended. Otherwise, the learning scenario should be carefully conceptualized and adapted. Mixing analog phases with AR phases can minimize the waiting time for a single device and is in fact a good choice for teaching-learning scenarios in nature—students should also be able to directly experience and benefit from nature. The AR experience should be an add-on.

If working with voice commands is implemented in the AR application, a quiet environment should be chosen or the noise level should be considered.

It is best to work with a small group only. This way, problems with a number of devices and battery life can be avoided. It can also ensure better support if users are not experienced with the novel and uncommon gesture interactions of the AR equipment.

Discussion

With this work, we present some valuable insights on the use of near and far interaction techniques in a natural context. In this way, we address the need for more research on the application of AR in nature and the environment. We also provide recommendations for researchers and adopters of AR outdoors in the nature.

The difference between the two interaction options studied—near (directly with the hands) and far (with a ray)—is not significant in terms of usability (ease of use). We found out that it seems to be a subjective decision which technology is more convenient. This is dependent on the user’s preferences. In some cases, only one of the interaction techniques worked well. Some participants moved their hands too quickly or too slowly when “pinching” the leaf, and it seemed that the calibration process at the beginning was not entirely precise. The interaction with the ray was atypical in some ways and was not intuitively understood by some participants. For these reasons, we recommend including both interaction options for future applications.

The results of our field experiments in the different environments show that, unlike expected, users were faster when interacting in the natural setting. The speed of collecting the virtual leaves was not influenced by the uneven ground covered with grass and branches. We observed that some participants moved the grass with one hand to reach the virtual leaves, but the provided larger colliders for the virtual leaves seem to be a good and efficient way to deal with the outdoor conditions regarding the floor. The fact that the users got used to the interaction techniques and were slightly faster during the second test in nature could be an additional explanation for this result. Further research in other environments and with more AR-experienced users might be useful to obtain a more comprehensive result.

The underground visualization was fascinating for the participants, which intensified the enjoyability factor. However, in a future application, the growth of tree roots should be shown on the entire root structure and not only on a branch. This way, the animation becomes immediately visible, which was not always the case in some tests, and some users had to look around first to see the virtual growing roots.

Conclusion

In this article, we explored the use of near and far interaction in AR modules with an HMD in two different types of locations in nature. In addition, we presented several educational use cases for these types of AR modules. We developed AR visualizations for the example of a tree and used them in an interactive scenario. In this way, we investigated how users interact in a natural-virtual setting and gained an understanding of usability concerns and preferences for a particular interaction technique in practice. We examined interaction techniques using virtual leaves, clouds, and a real tree as examples. For the near interaction technique we measured a slightly better task completion time (see “Results”). However, more work and different cases are needed to draw more general conclusions for outdoor use. We also found that users enjoyed the various AR tasks, and emphasized that usage in a future educational application could benefit from both near and far interaction. Based on our experience in developing and evaluating outdoor AR modules [5, 6] for the natural environment and following guidelines for designing interactions [37], we have summarized several recommendations and practices for other researchers and practitioners. These can be useful for further user tests and research in this area, as well as for early adopters of outdoor AR in environmental education.