Keywords

1 Introduction

When humans grow older, in spite of physical condition, cognition ability will also get worse [1]. Among all cognitive abilities, the decline of spatial visualization ability causes older adults to have spatial problems and low spatial awareness in real life [2, 3]. Spatial visualization ability is the mental manipulation of spatial information to determine how a given spatial configuration appears if they are being transformed. However, this ability decreases with human age, negatively affecting older adults’ spatial capabilities such as map-reading and way-finding [4, 5].

In fact, the goal of delaying the degradation of spatial visualization ability can be achieved by taking visualization training considerably [6, 7]. However, current visualization training is conducted in two-dimensional (2D) interfaces which are not suitable for one to figure out spatial information in that cognitive load will be produced, making trainers feel arduous and gain no improvement [8]. Additionally, current training is analogous to test, which is too stubborn and not intriguing to trainers. As an instructional training tool, in spite of visual feedback, other sensory interaction ought to be applied in training as possible in order to enhance trainers’ immersion and interests.

AR is a real world environment whose elements are augmented by computer-generated sensory input. It is favorable to be developed into instructional tools based on its unique features. First, AR interfaces are conducive to spatial visualization for they provide correct spatial information and enhance three-dimensional (3D) concept [9]. Second, training with AR is able to improve training interest as well as reduce the cognitive load produced by practicing cognitive tasks [10, 11]. Finally, with AR, the information about the surrounding real world of trainers becomes interactive and digitally manipulable; thus, trainers can manipulate the virtual objects to interact with the real environment, improving sensory feedback and their immersion in the training. As a result, AR is very efficacious and suitable to be adopted in visualization training system. Spatial visualization is an essential ability related to many aspects in one’s normal life. In order to postpone the degradation time of this ability for older adults, we are devoted to designing an AR based spatial visualization training system for older adults to have more opportunities to train their visualization functions in usual time. In the long run, their spatial visualization ability gets improved, and they will confront less spatial problems in real life.

2 Related Work

Traditional training of spatial visualization requests trainers to conduct considerable paper-tests. Take one of spatial visualization criterion tests, Mental Rotation, for example. Trainers have to judge if two images are the same or mirrored through true/false paper tests. Many researchers also use computers to generate the stimuli and present them on LCD monitor in order to make them have more obvious depth cues such as shadowing or light effect [12, 13]. In some studies, animation is adopted to assist trainers’ comprehension of motion process in an effort to improve their visualization ability more significantly [14]. In conclusion, existing training methods carried out in 2D medium such as paper or LCD monitors which are not favorable to illustrate spatial concepts for heavy mental load is produced in training process, causing inverse effect on trainers’ learning interest and performance.

AR is widely used in educational domains as a new kind of learning tool because people can directly realize 3D concepts in AR rather than figure out 2D information from flat formats, which is beneficial to spatial visualization ability and dwindling cognitive load. Computerized magnetic principles and electromagnetism which are invisible in normal situation are displayed in AR; instead of being depicted in book pages, students are able to comprehend the principles in AR interfaces [15]. Some notions that are hard to be implemented into the real world such as the earth revolution [16] or the landscape visualization [10] are also suitable to be taught under AR. AR can also be applied to more professional disciplines. Engineering students directly train their visualization ability with virtual mechanical objects presented in AR books called AR-Dehaes. Compared with conventional orthographic images, students are allowed to see the integrated appearance of objects in AR-Dehaes [17]. Another example is Sonic Flashlight. It is an AR device which enables direct perceptual guidance of needle injection to achieve higher accuracy and lower variability in aiming and endpoint placements by presenting a virtual slice and the target directly within the personnel’s vision area [17].

In spite of presenting virtual objects instantly, some AR systems allow participants to directly manipulate these virtual objects. Construct 3D is a mathematics and geometry instructive system implemented by AR. Students not only see the real geometry relation in augmented environment but also manipulate these virtual parts such as translating, rotating and scaling [18]. In a study of using AR to teach sun and earth relationship, participants are able to manipulate and interact with the virtual content [19], they are likely to understand the changes caused by different position and relation between sun and earth. Some researchers are devoted to creating more specific motions defined in AR environment so that participants can execute translation, scaling, rotation and other motions through fingers detection [20].

3 Method

AR spatial visualization training system consists of two parts: manual controller and training task. The design process of these two parts is illustrated individually as below. Furthermore, the integration of AR system is shown in this chapter.

3.1 Design of Manual Controller

A spatial visualization experiment with think aloud was adopted on older adults to collect mental strategies they used to handle the tasks. Next, we analyzed the manipulation and created morphological matrix to choose the best controller with specific principles such as intuition and convenience.

Manipulation Collecting

Participants. Six older adults (3 males and 3 females) ranging in age from 65 to 70 were recruited in the experiment. All of them were cognitively normal and passed Mini-Mental Status Examination [21].

Materials. Six revised tests based on Lohman’s criterion tests [22] were used in the experiment. They were Mental Rotation v.1 and v.2, 2D and 3D Paper Folding Test, Paper Form Board and Surface Development. Each test contained two questions, resulting in a total of twelve questions for one participant (Fig. 1).

Fig. 1.
figure 1

Spatial visualization tests used in experiment

Full size image

Procedures. In the setting, we prepared a projector to display testing tests and two cameras (one from the front, and the other from the back) to record the whole process. First, we explained the purpose of experiment and introduced spatial visualization ability. Besides, personal information of participants such as age, education and job was collected. Second, we introduced how to use think-aloud method in the experiment and encouraged participants to express their thought when they were answering questions. Third, participants were answering questions; meanwhile, we were recording their speaking and sketching some specific gestures. After finishing each question, participants would be given paper models which were the same objects as those in questions, so they could express their thought again and checked the answer. The total time for each person was 40 min or so.

Gesture Analysis. After the experiment was finished, lots of manipulation and gestures were collected. In order to organize these gestures, they were analyzed by Guessability and Agreement theories [23]. Guessability was used to choose the best gesture representing the motion, and agreement was adopted to select the best motion used in the training process. Hence, the basic motions of dealing with spatial visualization tests were Select, Rotate and Translate. Also, the corresponding gestures with highest guessability scores were listed under each basic motion.

Fig. 2.
figure 2

Basic motions and morphological matrix

Full size image

Idea Generation. Morphological matrix (Fig. 2) was adopted to generate ideas of manual controller. Three basic motions were on vertical row in morphological analysis matrix; the horizontal row was the form of controller (Plate, Sphere, Cube and Disk). Through brainstorming, possible ideas and interactive ways located in each block in matrix were developed. Next, ranking each idea in Likert scale (1-7 scores) was proceeded in order to find out the best manual controller which was elderly-friendly, we made a ranking table presented. By following three principles : intuition, convenience and aesthetics, the best manual controller would be chosen.

3.2 Design of Training Task

Design of a new training task follows a very analytic method. It is essential to realize what spatial training factors current spatial visualization tests were composed of. These factors are the basic units that trainers use to solve spatial related problems. After the factors are collected, we can combine some of them into a prototype and then add story or interesting elements to shape it into an integrated spatial visualization task.

Manipulation Collecting. This part was the same as the think aloud experiment conducted in the design of manual training.

Spatial Training Factors. The factors were in fact the result of mental strategies older adults had used in think aloud experiment. After classification, there were totally twelve spatial training factors which could be classified into four groups (Table 1): (1) Checking: to confirm shape, number, direction or angle of a pair of objects is same or not. (2) Relation: to compare the relation of position or to align virtual objects. (3) Manipulation: to manipulate virtual objects in mind, including rotate, fold, unfold, move and combine. (4) Sequence: to memorize the steps of any manipulation or operation.

Table 1. Spatial training factors of mental rotation v.1
Full size table

Idea Generation. In Table 1, black dots represent the current factors of tests. We added new factors (triangle) in different tests, and developed new tasks by combining different factors together. Take Mental Rotation v.1 for example. Beside checking shape, checking number, aligning and rotation, checking direction is considered as a new spatial training factor in Mental Rotation v.1. Next, integrate these five factors and shape them into a new spatial visualization training task.

3.3 Integration of AR System

Software for Development. Qualcomm AR (QCAR) was adopted as the AR environment in the training system. It was a free AR SDK which was easy for developer to conduct coding and make interaction. Training system was constructed in Unity3D with Vuforia portal. Unity3D is a game developing engine which has been widely used to visualize the scene and make game and animation. With Vuforia attached in Unity3D, QCAR is able to be implemented (Fig. 3).

Fig. 3.
figure 3

Integration of AR spatial visualization training system

Full size image

AR Marker for Manual Controller. Vuforia Target Manager is an online software for users to define patterns and form of AR markers. Since the basic motions are Rotate, Move and Select, a cube marker and two plate markers were chosen. First plate marker was used for displaying virtual objects; the second one was used for selecting objects; cube marker was used for rotating objects. If users rotate cube marker with the second plate marker pressed, the virtual objects will not rotate but move (Fig. 4).

Fig. 4.
figure 4

(Left) options of AR markers in Vuforia target manager [24]. (Right) manipulation of three basic motions.

Full size image

Training Task. Necessary items in task were constructed in Solidworks and were transformed into FBX files in 3Ds MAX. Items only in FBX extension can be imported to Unity3D. Connection was carried out by coding Target Controller scripts in C#. One Multi-Target Controller (for cube marker) and two Image-Target Controllers (for plate markers) are entailed. After inserting Target Controller scripts to the corresponding items, users are able to conduct functions by manipulating AR markers. Basically, three kinds of interaction are requested: speed adjustment, level change and incidents trigger. They are all coded in javascript and achieved by adopting box collider in Unity3D.

4 Result

4.1 AR Spatial Visualization Training System

AR spatial visualization training system was implemented in PC (Fig. 5). The system was composed of a set of manual controller including a cube marker and two plate markers, additional camera and a monitor. One thing that should be noticed was that projecting route of camera was required to be the same direction as trainer’s sight line so that virtual objects will not be the mirrored images.

After markers were projected, corresponding virtual objects would be displayed on monitor. Trainers were able to rotate cube marker to control the angle of virtual block, and pressed the second marker which presented a virtual red disk to make block fall through the hole of the virtual floor successfully. If the shape of block did not match that of hole, block would get stuck until trainers rotated the block in right angle. 20 levels were made in training tasks. These levels differed in shape of blocks and hole.

Even if older adults were allowed to use physical controller to rotate the block, they still entailed to consider which direction of block is correct, indicating the additional spatial factor: checking direction was being trained when they were receiving AR spatial visualization training (Fig. 6).

Fig. 5.
figure 5

Setting and scene of AR spatial visualization training system

Full size image
Fig. 6.
figure 6

Training process of older adults

Full size image

4.2 Feedback from Older Adults

We invited same older adults who had participated in the experiment of manual controller design to use the AR spatial visualization training system. They all agreed the positive training effect of AR training system. What they were in favor of were manual controller and game-alike task of AR training system, which enhanced their training interests and pleasure. In addition, two of them mentioned that it was even better when someone accompanied them when they were training. However, some thought the design of manual controller should be improved so that they could manipulate it more fluently. Besides, they were not very satisfied with much equipment of current system.

Most older adults did not prefer paper tests; what they worried about were their reaction time and error rate. Similar to conventional test, older adults would suffer from great pressure of the scores, which made them frustrated. Furthermore, they might not get used to focusing on text or pictures for a long time, which was one of demerits of paper tests. Since the question of tests were associated with the manipulation of objects, they would like to manipulate a real object rather than handle mentally. Nevertheless, although older adults did not agree the form of paper tests, they admitted the brain fitness of it.

5 Discussion and Conclusion

In this paper, we followed the comprehensive design methods to implement an AR based spatial visualization training system for older adults. Since spatial visualization ability is essential to older adults, we transformed the conventional spatial visualization test into game-alike task which were more suitable for them to keep training their spatial visualization ability. The implementation of controller and AR was also conducive to the improvement of training effect, their training immersion and interest.

5.1 Manual Controller and Implemented Device

Based on the opinion of older adults, cube marker was not appropriate for them to manipulate. However, since available marker forms on Vuforia target manager were limited, we might consider a transparent spherical shell to wrap cube marker. Hence, patterns on marker were able to be projected; the spherical shell could be better for older adults to conduct rotation which was the chief type of manipulation in AR training system.

Some older adults argued the inconvenience of much equipment of AR training system in PC version. For most older people who would prefer handy and small devices, it might be sensible to develop AR training system into APP in tablet version. Tablets were more ideal for older adults to take and use; additionally, they were able to conduct training at all times. In fact, older groups were occupying more and more proportion in people using mobile devices. Around 80 percent of people ranging in age 50 to 70 used tablets in normal times. Hence, developing AR training system in APP would be a favorable idea.

5.2 Training Task

Existing training task was based on mental rotation; other types of training tasks were likely to be developed in the future. We could combine more spatial training factors and integrated into different training tasks; for example, tasks derived from 3D paper folding or paper form board, making AR spatial visualization training system more comprehensive and diverse.

According to previous studies, it was favorable that the items in training corresponded to target trainers. For instance, if trainers were children, items may likely turn into other meaningful images such as animal’s patterns, cartoon pictures or letter [26, 27]. This would promote target trainers’ familiarity and indirectly improve their training immersion as well.

Two older adults mentioned it would be better if someone accompanied them when they were training. It triggers us to develop multi-player mode in the future. Especially for older adults, the feeling of companion will be a critical issue when designers developed products. If two or more people are allowed to join the training tasks, older adults will be more willing to take training and gain more happiness. Thus, we may consider to design multi-player mode of training task.

In sum, AR based spatial visualization training system was an innovative brain fitness tool for older adults. With the exceptional features of AR, we were interested to adopted this multimedia technique to develop an unconventional training system to help maintain spatial ability for older adults. Since current feedback from older adults was positive, we were motivated to improve AR training system and qualified the data of their training performance in order to testify the spatial visualization training effect of AR based training system.