Abstract
This article presents a set of classroom practices created for the discipline of Geomorphology that use an Augmented Reality installation known as SARndbox. This set was developed respecting the characteristics and according to a set of usability guidelines for applications that make use of Natural User Interfaces (NUI). The characteristics of Natural Interfaces, as well as the usability guidelines used in the development of these practices are presented.
Keywords
1 Introduction
Augmented Reality (AR) applications aim to enhance the perception and effectiveness of the user by providing additional visual information. The user stays aware about the real world, but, ideally, is not able to distinguish between information coming from the real or the virtual world [1].
The use of RA systems has been investigated since the early 1990s, and includes medicine, production processes, aeronautics, robotics, entertainment, tourism, marketing, social networking and education [2].
The Augmented Reality has also benefited from several technological advances that have occurred in recent years, which have given rise to more natural and low-cost interfaces and human-computer interaction devices. Among these devices, depth sensors attract attention by implementing advanced interfaces in various Augmented Reality applications, from advertising to serious games for physical rehabilitation [3,4,5,6,7,8].
The present work presents a set of classroom practices for Geomorphology based on an Augmented Reality installation, known as SARndbox [9]. This set was developed based on guidelines developed for systems that use Natural Interfaces, as well as the pedagogical needs intrinsic to the teaching practices of the topics covered.
This paper is organized as follows: Sect. 2 presents some applications that use Augmented Reality in Education; In Sect. 3 the concept and characteristics of Natural User Interfaces; Sect. 4 presents the SARndbox, an installation used to support classroom practice; Sect. 5 describes the guidelines that have guided the development of practices and a brief description of practices created and finally, in Sect. 6, the final considerations and future work.
2 Applications of Augmented Reality in Education
Specifically, in Education, there are applications that aim to promote teaching from elementary to higher education, in several areas of knowledge. An extensive listing of examples of the application of RA in teaching, varying from Biology, Physics, Mathematics, to Religious Education is presented in [2, 10]. Some RA systems applied to Teaching are presented in the following sections.
2.1 3D Pop-up Book
The 3D Pop-up Book is an Augmented Reality book used for teaching English with students in Thailand, which uses markers to combine characters and objects in a book (Fig. 1) [11].
The results presented in [11] corroborate with other usability assessments, which always demonstrate the great interest of users in this technology, highlighting its playful and innovative aspects, as well as its possibilities in other studies.
2.2 Augmented Chemistry Reactions
To facilitate the understanding of Chemistry, especially the spatial structure of molecules and their behavior in some reactions, Maier Klinker proposed a visualization tool to display and control molecules as well as the dynamics between them [12].
This tool uses a cube with a handle that contains imprinted patterns (markers) on its faces (Fig. 2). Students should manipulate the cube to visualize the molecules and thus rotate and be aware of their atomic structure (Fig. 3).
This system, implemented with ARToolkit [13], is representative of the first implementations of RA, which makes use of markers to introduce virtual objects into the scene.
2.3 Augmented Reality Magic Mirror
In teaching Anatomy, Meng et al. propose a system that uses an “AR mirror” [14]. In this system, the depth camera is used to track the posture of the user standing in front of a screen. A computed tomography (CT) scan of the user is generated, creating the illusion that it is possible to observe the inside of your body, as shown in Fig. 4.
Gestures are used to select different layers of the tomography and a set of photographic data can be chosen for viewing. This system is also capable of displaying 3D models of internal organs, textual information, and anatomy images. The interaction with this system dispenses markers and uses depth data, increasing, or decreasing the visibility of the hands based on their distance from a virtual interaction plan. According to the authors, this helps the user to perceive the spatial relationships between his body and the virtual interaction plane.
Most of the examples found in the literature have been used by educators to provide preconceived teaching experiences. This can lead to situations where AR only develops lower order reasoning skills, rather than encouraging integrative skills such as analysis, evaluation, and creation.
Thus, using AR applications to stimulate critical thinking should be a main goal, allowing students to interact and act actively in these systems, instead of being mere spectators.
3 Natural User Interfaces
In recent years, the technological evolution allowed the creation of a range of devices that allowed the emergence of a new mode of interface. This new paradigm is known as Natural User Interfaces (NUI). These can be defined as “interfaces designed to reuse existing skills for direct interaction with content” [15].
Historically, NUIs can be inserted into an evolving line of Human-Computer interfaces, in which three paradigms can be highlighted (Fig. 5): Command Line Interfaces (CLI), Graphical User Interfaces (GUI), and, more recently, Natural User Interfaces.
In command-line interfaces, successive lines of text (command lines) are sent by the user to the system. This interface is found on older operating systems, such as MS-DOS, early versions of Unix, and more. Generally, its implementation consists of a prompt that is able to accept the text commands and convert them into calls to system functions. This is the type often preferred by advanced users interface, it provides a concise and powerful way to control a system.
Graphical interfaces are currently the most common, especially those based on the WIMP (Windows, Icons, Menus, Pointer) paradigm. In this paradigm, the mouse, or some similar device, is used to manipulate elements on the screen (windows, buttons and menus) and thus perform tasks. From a usability perspective, graphical interfaces introduce benefits such as the ease of remembering actions and constructing models that remain coherent for several similar tasks. Visual feedback is immediate when indicating the effects of a given action. For example, when you delete a file, its icon is removed from the screen. The great advantage of this type of interface is that they make the operation of the computer more intuitive and easy to learn.
Finally, the Natural User Interfaces can be found in various systems and applications of our everyday life: touch screens on computers and smartphones; applications using voice commands; gesture commands to control televisions and games; BCI (Brain-Computer Interface) systems, where brain commands trigger the execution of tasks.
NUIs refer to interfaces between humans and computers that use unconventional devices and are based on natural elements. The word “natural” in this case is used in opposition to most computer interfaces that use “artificial” devices whose operation needs to be learned. In this type of interface, one can affirm that the interaction is based on previous user knowledge and, thus, it is easy to learn and to become experienced [15].
3.1 Characteristics of Natural Interfaces
According to Blake, there are three important fundamental concepts that greatly contribute to the usability of NUI-based systems [15].
First, NUIs are designed, are premeditated and prior efforts are employed in their design. It is necessary to ensure that the interactions in a NUI are appropriate for both the user and the content and context. The simple fact of grouping several concepts together is not enough for an interface to be considered natural. One must be aware of the role designers need to play in creating natural interactions and ensure that the same design priority is given to the design phase.
Furthermore, NUIS reuse existing skills. Application users are experts in various skills that have been acquired simply because they are human. For several years, these users have practiced communication, verbal or non-verbal, as well as interactions with the environment.
The computing power and technology have evolved to the point where you can take advantage of these skills. Nuis do this by allowing users to interact with computers through intuitive actions such as touch, gesture and speech, and have interfaces which users can first understand with metaphors drawn from real-world experiences.
Finally, NUIs have direct interaction with content, that is, the focus of interaction is on the content itself and on direct interaction with it. That does not mean controls such as buttons and check boxes are totally absent in the interface. It simply means that such controls should be considered secondary when compared to content, and that direct manipulation should be the primary interaction method
4 Augmented Reality SandBox - SARNdBox
The Augmented Reality Sandbox (SARndbox) is an installation that aims to integrate an Augmented Reality system with graphical effects and simulations to physically created topographic models that have their surface scanned by a computer in real time [9].
A SARndbox aims to integrate an augmented reality system into physically created topographic models that have their surface scanned by a computer in real time. These models are used as background for a variety of graphic effects and simulations.
The installation consists of a computer, a projector, a 3D surface reading device (in this case, the Microsoft Kinect) and a box containing material that can be manipulated, such as sand, to create topographies interactively and with low need for supervision by a specialist (Fig. 6). All software required to configure SARndbox is available for free under the GNU General Public License. For its implementation is recommended a computer with Intel Core i7 processor, clocked at at least 3 GHz, a video card with graphics processor (recommended NVidia GeForce GTX 970), and Linux Mint 64-bit operating system.
Initially, the potential use of SARndbox in the educational environment was investigated through questions inserted into a usability assessment applied to 100 participants, including students and teachers of elementary, middle, and high school.
The evaluation consisted of two sets of questions: one to obtain the user profile (age and school education) and other to evaluate efficiency issues, prior knowledge of the topic addressed in the application (contour lines), prior knowledge of AR, ergonomics (comfort feeling when using the application, ease of use, satisfaction when using) and whether they consider that the use of the tool would facilitate learning.
The answers obtained in the usability assessment indicate that there is a great public interest in using innovative teaching technologies. The adoption of these technologies can introduce many benefits. The entire evaluation is presented in [16].
5 Geomorphology Classroom Practices
Despite AR technology is widespread in the Computer Science academic environment and used in entertainment and advertising, there is still a niche to be explored in Education. Although about half of the answers affirm already knowing this technology, it can be observed, from the answers provided, that AR applications, with great visual impact, still arise a lot of fascination and interest in people.
To enhance the teaching of Geomorphology concepts, we propose some classroom practices using the SARndbox, with the indication of auxiliary tools.
The development of these classes considered some NUI-specific usability guidelines, regardless of the type of interaction: Instant Expertise, Cognitive Load, Progressive Learning and Direct Interaction [15].
The Instant Experience assumes that the designed interactions must reuse existing skills. By doing so, we take advantage of the previous investment made by the users in their competences and create experts instantly. Both in the real world and in creating interfaces, the most difficult to use a skill is the learning process. Once this ability is learned, it becomes much easier to exercise it. Users have many skills before even using any application, many of which have been exercised since childhood. Interactions that reuse existing skills enable users to become practical quickly and with little effort. To create experts instantly, J. Blake considers two ways: to leverage domain-specific skills (such as the use of tools and devices common to a given activity); and harness skills common to all humans [15]. Regarding the SARndbox, the interaction consists solely of handling the sand contained in the box. Thus, we ensured that all users could interact with the system immediately.
Cognitive Load implies designing interfaces in which most interactions use innate and simple skills. This brings two benefits: most of the interface will be easy to use and the interface will be quick to learn, even if some of the skills are completely new. If the interface uses capabilities based on natural interactions with the real world, the frequency of interactions will be much higher. In fact, there is a trade-off between Instant Experience and Cognitive Load. These guidelines may conflict when users are already endowed with a useful skill, such as using the mouse. The focus, in these cases, should keep on minimizing a cognitive load for a majority of interactions, reusing it as existing simple skills. There was no concern about this guideline for the development of classroom practices, since the interaction was quite simple and straightforward.
Progressive Learning provides a smooth learning curve from basic tasks to advanced tasks. Natural Interfaces should allow your users to learn and evolve progressively, from beginners to expert level. At the same time, the interfaces cannot make it difficult for experienced users to perform advanced tasks in the best way. This guideline implies allowing the application to be used at the same time as more complex tasks are learned. In the case of class practices created, there was a concern to establish a growing and constant learning curve of the concepts addressed, from basic concepts to the subjects that need more critical thinking and analysis.
The Direct Interaction, in its turn, states that interactions should relate directly to the content; occur at high frequency and be appropriate to the context. Since our interactions with the real world show this effect, the adoption of direct interactions results in more fluid and natural interfaces, and allow access to many features without overwhelming the user presenting them all at once. In the case of SARndbox, the means of interaction is not only direct, but also tangible, allowing free manipulation (within the limits of the box), and giving this system a highly playful appeal to users.
Moreover, in [17] is highlighted the need to provide constant feedback to the user, so that it is aware of what is happening and what to expect as a result of their interactions. In the case of SARndbox, the results of interactions (sand movement) occur immediately with each action taken by the user (Fig. 7).
The broad sense of these practices involves activities related to both basic and higher education, as well as consulting projects, public and/or private companies, among other projects that have the landform as the focus of analysis. The classroom practices proposed include:
-
types of terrain, which consists of building different landforms, working concepts related to each shape, its genesis and evolution. The landforms to be built can be hills, plateaus, flatlands, among others;
-
hydrographic basins, that is, to build in the sand a basin with its characteristics: main river, watershed, tributary and sub-tributaries rivers;
-
slope, that is, building landforms for calculus of declivity and, later, classification of the landform as flat, slightly hilly, hilly, strongly hilly, hilly terrain and steep terrain;
-
relief and the Brazilian Forest Code, that is, to reproduce relief forms with declivities within the limits of use established by the New Brazilian Forest Code (Brazil’s Law number 12.651/2012) [18];
-
rivers and the Brazilian Forest Code, which consists of constructing different types of riverbeds and applying the provisions of both the former Brazilian Forest Code and the new Brazilian Forest Code [18], showing the differences between them in terms of delimiting the Permanent Preservation Areas along the riverbanks.
The Fig. 8 illustrates some contours created for each of the classroom practices.
The application of this set of practices should be associated with a set of pedagogical guidelines, which aims to evaluate the efficiency of this approach.
Basically, the instructional aspect of SARndbox should be well understood before using it, and after the activity, the topic addressed should be discussed.
The instructor should be able to assess whether the use of the tool was more efficient than the use of conventional teaching techniques, such as lectures, animations, videos, and illustrations.
6 Conclusions and Future Work
In this paper, we present a set of classroom practices based on an Augmented Reality tool that uses as a interaction device a depth sensor created to provide Natural User Interfaces, called the SARndbox. This set of practices was created considering both characteristics required for Natural User Interfaces and specific usability guidelines for this interaction paradigm.
The great visual appeal of this tool, associated with its inherent playful aspect, mainly due to the interaction mechanism, were the main factors that motivated the investigation of the viability of its use in the classroom environment.
The practices created take into account issues addressed in both higher education in the areas of Environmental Engineering, Civil Engineering, Geography, as well as in High School courses, and can be used, with different degrees of depth, at all levels of education.
Future Works include measuring the impact of technology adoption on teaching. To do so, it is necessary to investigate ways to obtain an accurate, results-based, replicable and applicable evaluation not only for the area of Geomorphology, but also to contribute to the evaluation of other Augmented Reality devices applied to the teaching of several other areas of knowledge.
References
Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. 77(12), 1321–1329 (1994)
Bower, M., et al.: Augmented Reality in education–cases, places and potentials. Educ. Media Int. 51(1), 1–15 (2014)
Biswas, K.K., Basu, S.K.: Gesture recognition using Microsoft Kinect. In: 2011 5th International Conference on Automation, Robotics and Applications (ICARA), pp. 100–103. doi:10.1109/ICARA.2011.6144864
Da Gama, A.E.F., Chaves, T., Figueiredo, L., Teichrieb, V.: Poster: improving motor rehabilitation process through a natural interaction based system using kinect sensor. In: 2012 IEEE Symposium on 3D User Interfaces (3DUI), pp. 145–146. doi:10.1109/3DUI.2012.6184203
Gerling, K., Livingston, I., Nacke, L., Mandryk, R.: Fullbody motion-based game interaction for older adults. In: Proceedings of the 30th International Conference on Human Factors in Computing Systems, CHI 2012, Austin, Texas, USA, pp. 1873–1882
Martin, C.C., Burkert, D.C., Choi, K.R., Wieczorek, N.B., McGregor, P.M., Herrmann, R.A., Beling, P.A.: A real-time ergonomic monitoring system using the Microsoft Kinect. In: 2012 IEEE Systems and Information Design Symposium (SIEDS), pp. 50–55. doi:10.1109/SIEDS.2012.6215130
Parajuli, M., Tran, D., Ma, W., Sharma, D.: Senior health monitoring using kinect. In: 2012 Fourth International Conference on Communications and Electronics (ICCE), pp. 309–12. doi:10.1109/CCE.2012.6315918
Yeh, S.-C., Hwang, W.-Y., Huang, T.-C., Liu, W.-K., Chen, Y.-T., Hung, Y.-P.: A study for the application of body sensing in assisted rehabilitation training. In: 2012 International Symposium on Computer, Consumer and Control (IS3C), pp. 922–925. doi:10.1109/IS3C.2012.240
Reed, S.E., et al.: Shaping watersheds exhibit: an interactive, augmented reality sandbox for advancing earth science education. AGU Fall Meeting Abstracts, vol. 1 (2014)
Zagoranski, S., Divjak, S.: Use of augmented reality in education, vol. 2. IEEE (2003)
Vate-U-Lan, P.: An augmented reality 3D Pop-up Book: the development of a multimedia project for English language teaching. In: 2012 IEEE International Conference on Multimedia and Expo, Melbourne, VIC, pp. 890–895. doi:10.1109/ICME.2012.79
Maier, P., Klinker, G.: Augmented chemical reactions: 3D interaction methods for chemistry. Int. J. Online Eng. 9, 80–82 (2013)
Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: Proceedings of the 2nd IEEE and ACM International Workshop on augmented reality (IWAR 1999), San Francisco, CA, pp. 85–94 (1999). doi:10.1109/IWAR.1999.803809
Meng, M., et al.: Kinect for interactive AR anatomy learning. In: 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Adelaide, SA, pp. 277–278 (2013). doi:10.1109/ISMAR.2013.6671803
Blake, J.: Natural User Interfaces in. Net Manning Pubs Co Series. Manning Publications Company, Greenwich. ISBN 9781935182818. http://books.google.com.br/books?id=mmMCkgEACAAJ
Mezzomo, M.D., Kawamoto, A.: Dispositivo de Baixo Custo para Interfaces Naturais como Ferramenta de Apoio Didático no Ensino de Geomorfologia. In: XI Simpósio Nacional de Geomorfologia (SINAGEO), Maringá (2016). http://www.sinageo.org.br/2016/trabalhos/5/5-198-1664.html
Microsoft Corporation: Kinect for windows human interface guidelines v1.8.0 (2013). https://msdn.microsoft.com/en-us/library/jj663791.aspx
Brasil: Código Florestal Brasileiro. Lei nº 12.651, de 25 de maio de 2012. Presidência da República do Brasil (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Kawamoto, A.L.S., Mezzomo, M.D.M. (2017). Geomorphology Classroom Practices Using Augmented Reality. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human–Computer Interaction. Designing Novel Interactions. UAHCI 2017. Lecture Notes in Computer Science(), vol 10278. Springer, Cham. https://doi.org/10.1007/978-3-319-58703-5_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-58703-5_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-58702-8
Online ISBN: 978-3-319-58703-5
eBook Packages: Computer ScienceComputer Science (R0)