Advertisement

Considerations for Immersive Learning in Intelligent Tutoring Systems

Conference paper
  • 1.6k Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9744)

Abstract

Research has examined the benefits and retractors of immersing the learner in an environment. Immersive computer-based training environments are costly to construct and may not always lead to significant learning or transfer benefits over other methods. The current paper presents a brief review of presence and immersion research in computer-based learning and adaptive tutoring. The Generalized Intelligent Framework for Tutoring (GIFT) is an open source domain-independent framework for creating intelligent tutoring systems (ITS). GIFT offers flexibility, and can be interfaced with training applications ranging from highly immersive computer-based learning environments (e.g., TC3Sim, VBS2) to less immersive mediums such as PowerPoint. The capabilities of GIFT that can be used to create immersive adaptive tutoring are discussed. Additionally, the use of GIFT to run and generate experimental studies to examine the impact of immersion is highlighted. Finally, recommendations are given on how to provide more opportunities to integrate immersive environments into GIFT.

Keywords

Immersion Intelligent tutoring systems Generalized Intelligent Framework for Tutoring Presence 

1 Introduction

Presence and immersion are concepts that are tightly coupled together in the literature [1, 2]. In many cases the terms have been used interchangeably to represent similar concepts [1]. Presence has been defined as “the subjective experience of being in one place or environment, even when one is physically situated in another” [2, p. 225]. Therefore, presence is the overall feeling that an individual has of being somewhere different than they are based on what they are experiencing. Immersion has been defined in the literature as “a psychological state characterized by perceiving oneself to be enveloped by, included in, and interacting with an environment that provides a continuous stream of stimuli and experiences” [2, p. 227]. McMahan [1] clarified immersion as being when an individual feels highly engaged in or “caught up in” an environment or story, and the positive feelings towards the environment. Further, McMahan distinguished presence from immersion, as presence is a term that tends to be applied to virtual reality and virtual environments [1].

As technology continues to rapidly improve, opportunities to encourage feelings of immersion in both virtual and physical environments have become more frequent. Feelings of immersion and presence are often goals of different entertainment mediums such as videogames, movies, and theme parks. Videogames have become increasingly more interactive with not only more visually realistic environments, but also more realistic motion based user input methods as seen in the Nintendo Wii, Kinect and Playstation Move [3]. In an effort to encourage consumers to see movies in theaters, the film industry have moved in a similar direction by releasing movies that are in 3D and IMAX 3D to give the audience the impression that they are in the action instead of watching it. Additionally, DBOX has specially designed seats in select theaters that move or shake depending on what is happening on the screen. Rather than just relying on the film’s story to draw viewers in, these 3D and tactile elements provide additional sensory cues to further enhance feelings of being in the action as opposed to watching it. Theme parks have also moved toward including 3D in their rides, to further enhance the experience. While the “land” structure of theme parks such as Disney’s Magic Kingdom and Disneyland are not a relatively new idea (i.e. Fantasyland, Tomorrowland, Frontierland, etc.), there has recently been a trend toward new highly themed and conceptually tied together “lands” that make the visitor feel like they have walked into a movie such as Harry Potter or Avatar by seamlessly recreating places that have only existed in film or books. The goal of lands is not only to provide an attraction, but also an experience where the guest feels that they are transported to a different, often fictional, place. Immersion can occur in the multisensory mediums described, however, getting highly involved in a book or movie and putting oneself in the place of a character is a form of immersion as well [2]. The current strategies of 3D and large format screens, and being able to walk into a fictional place like Hogwarts Castle at a theme park are examples of approaches that are currently being used to foster these feelings of identification with characters and immersion.

There are individual differences and characteristics of individuals which may make them more or less likely to become involved with a virtual environment or other medium. Scales such as Witmer and Singer’s Presence Questionnaire [2] and the ITC-Sense of Presence Inventory [4] have been developed in order to measure and quantify these feelings of presence. Immersion and presence are goals of all the previously described entertainment mediums. However, all of these examples are of mediums that are trying to create a sense of immersion and presence that are primarily aimed at entertainment. Many times these immersive videogames, rides, and movies get positive reviews and feedback from consumers, which results in a continued push towards using these methods. In general people tend to like systems that offer more realism and fidelity, however, this may ultimately be more distracting to the task than helpful [5]. As videogames and virtual environments have become more prevalent, they have been harnessed for learning and training purposes. The assumption that is often made is that because something is more immersive it will lead to better outcomes in training and transfer to real world tasks. Immersion has been found to have positive impacts on learning due to encouragement of situated learning, and simulating the real world [6], however, it may not be the case for every domain. Training transfer may not occur from all videogames and virtual environments, and research is needed to continue examining the interactions and impacts of presence and immersion on outcomes [7]. While immersive virtual environments and videogames may provide opportunities to learn and practice skills they may not always include pedagogy. Intelligent tutoring systems (ITS) can be combined with these environments and games to improve learning outcomes, and to foster research into the influence of these environments on training transfer/learning.

2 Considerations for Immersive Learning and Tutoring

Immersive learning environments can include established videogames or virtual environments in which learners are taught to perform a task. Learners may interact with these while looking at a computer screen and typing their inputs; or they may wear a head mounted display, or look at another visual display. Rather than simply using a keyboard, the user input method may be by motion (i.e. Kinect), voice command, or game controller. Using these varying displays and input methods may impact how immersed the individual learner feels in the environment [8].

Immersive environments are designed to engage learners in their story and the action that is being performed. By adding tutoring and feedback components to these environments it requires providing instruction and feedback. However, sometimes it can be distracting for feedback to be provided to the learner while engaging with the environment. If the gameplay is stopped to provide video or auditory feedback it may take the learner out of the experience. Also, it is important for the feedback to be salient enough for the learner to understand it. Therefore, careful consideration should be given to the method in which feedback is given, how frequently it is given, and how disruptive it should be to engaging in the environment [9, 10].

It is important for there to be a match between the learning environment and the material that is being taught [8]. It has been found that using head mounted displays while engaging with a learning environment about science and botany did not lead to positive training or transfer effects [11]. However, a head mounted display may not have been the ideal environment to teach that specific material in. Another topic that may lend itself to the display method may have had differing results. For instance, land navigation has been suggested as a relevant domain that could be used for adaptive tutoring with smart glasses [12], however, teaching algebra using smart glasses may not be as beneficial. Therefore, it is important for the match between the domain and delivery method to be examined.

There may be individual differences in learners in regard to how likely they are to become involved or immersed in an environment [2]. Additionally, while the assumption is generally made that more detail or fidelity in an environment will lead to better learning outcomes, it is not necessarily true [5]. If extra non-relevant details are included it may actually lead to distraction from the important information. This distraction has been referred to as the seductive details effect, and has been found in both assessments after viewing text based, and multimedia based training/videos [13, 14]. Therefore, it is important to consider how much detail to provide and to make sure that it enhances the focus of the tutoring rather than distracting from it.

3 Intelligent Tutoring Systems and Immersive Learning

Intelligent tutoring systems and adaptive training provide excellent opportunities for both instructors and students. Materials can be presented to students, and the computer-based system can then react and adjust the student’s path based on their specific characteristics, answers to questions, and performance in assessments. Instructors have the flexibility to decide what material they would like to teach and create alternate paths that can help provide remediation on material that students are not necessarily grasping. Additionally, the performance of a student in one lesson can generally be stored and applied to their future interactions with the learning system. ITS result in highly adaptable learning content that is tied to the specific learner’s strengths, weaknesses, and needs. ITS instruction can assist in improving learner outcomes [15, 16]. While benefits of training in immersive environments alone are not always clear, integrating these environments with an ITS could help improve learning outcomes.

3.1 Challenges of Creating Immersive Tutoring in an ITS

Even though ITS have benefits, there are still challenges to instructors who want to use them, specifically in applied domains that have virtual environment training associated with them. In general, ITS often take a great deal of time to author, as their multiple paths result in the need for more content, and carefully selected rules to send students down those specific remediation paths based on performance. [17]. Additionally, due to the time it takes to create the materials and structure of an ITS they tend to be tightly linked to the subject matter or domain that they were designed for, as opposed to flexible and reusable. Finally, creating an ITS can be very expensive due to the amount of time that it takes to author it, as well as the expertise that is often needed: subject matter expert, instructional designer, computer programmer, etc. [17].

While some tutors engage with students through simple questions/answers, and more traditional assessments, it is also possible to create tutoring/feedback that is incorporated into interactive and immersive computer-based environments. In many cases to create ITS instruction, a course instructor will need to know the subject that they want to teach, have an established videogame or virtual environment, and will need to be adept at computer programming in order to successfully design the tutor. If the instructor does not have the computer programming expertise to integrate their environment, then they will need a computer programmer to do the initial integration, which could be costly. Therefore, creating an adaptive tutor with immersive content could be cost prohibitive, and difficult.

3.2 The Generalized Intelligent Framework for Tutoring (GIFT)

Many of the challenges of creating adaptive tutoring can be lessened by using a domain-independent framework that is designed for reusability. The Generalized Intelligent Framework for Tutoring (GIFT) is a free, open-source, domain independent framework for creating ITS that has been developed in order to increase the usability of ITS components, reduce the expertise level needed to author an ITS, and reduce the time it takes to create an ITS [17]. GIFT can be used by instructors and subject matter experts (SMEs) to author adaptive tutoring. At the current time, GIFT is available in two forms: downloadable (from https://www.gifttutoring.org) and cloud based (GIFT Virtual Open Campus, https://cloud.gifttutoring.org). Adaptive tutoring can be created with both versions, and student data can be extracted by instructors in order to monitor a student’s learning and progress.

One of the larger challenges of creating immersive training in an adaptive tutoring system is the need to integrate the training application/environment with the tutor. GIFT includes a number of modules that traditionally exist in ITS: learner module, pedagogical module, domain module, sensor module, and a gateway module [18]. The gateway module serves as the bridge between GIFT and external training applications. Messages can be sent between the external game and GIFT through this gateway module. This allows for a consistent method of integration of different environments with the GIFT system. While integrating a virtual environment or training application with GIFT may initially appear challenging, GIFT includes a Developers Guide (in the GIFT help files/documentation) to assist in the process. While it is unlikely that an instructor themselves will be conducting the initial integration, the guide will help streamline the process for the individual that does, reducing the amount of time required.
Fig. 1.

Screenshot of DKF authoring in the GIFT Authoring Tool

Once the integration has occurred, the course instructor will have the opportunity to author adaptive feedback that links directly to the actions that occur within the immersive environment/game. This feedback is authored by creating a Domain Knowledge File (DKF) using the GIFT Authoring Tool. See Fig. 1 for an example of a DKF in the GIFT Authoring Tool. There are three levels or states of performance that a learner can have: above expectation, at expectation, and below expectation. When selected actions or inactions occur in the environment, it can trigger a move from one of these states to the next, which then subsequently triggers authored feedback. The DKF interfaces with the training environment, and feedback can be provided through the game itself or through a feedback bar on the left side of the screen that is graphically separate from the environment. GIFT will track the learner’s state and follow the authored instructions to provide specific adjustments and feedback based on actions in the environment. Additionally, GIFT is integrated with PowerPoint, which can be used by instructors to enhance their courses with lesson materials or create interactive material using Visual Basic for Applications.

All of the components of the desired course are put together using the GIFT Authoring Tool. DKFs can be accessed, created and edited with this tool. Additionally the flow of the course is determined by the author by adding different elements including instructions, surveys, training applications, and after action reviews. See Fig. 2 for a screenshot of course authoring in the GIFT Authoring Tool.
Fig. 2.

Screenshot of course authoring in GIFT. Components of the course and their order are listed on the left side of the screen.

There are two highly immersive games/environments that are currently integrated with GIFT: the military based training game, Virtual Battlespace (VBS2 and VBS3) and the medical game, TC3Sim (VMedic). VMedic can be downloaded for free from GIFTtutoring.org, and is used to run the Explicit Feedback within Game-Based Training-1 and Explicit Feedback within Game-Based Training-4 courses in the desktop version of GIFT. These courses demonstrate an integration between an immersive game and GIFT. The development of these courses and the associated experiment has been documented in the literature [19, 20]. Additionally, an interactive excavator training course which uses an XBOX controller can be accessed in both the cloud and desktop versions of GIFT. The authored components of these courses (Course Files, DKFs, etc.) are also of interest to individuals who wish to use GIFT as they provide working examples of the technical files associated with GIFT, which can then be used as a basis for future course and experiment development. Further, instructors may be able to use these integrated virtual environments to create new GIFT courses without needing to engage in integrating an entirely new training environment with GIFT.

3.3 Considerations and Future Directions for Immersive Learning in GIFT

At current time, the immersive virtual environments that GIFT has primarily been integrated with have been computer-based games that are intended to be interacted with on a stationary computer. However, GIFT has been integrated with the psychomotor domain of adaptive marksmanship training, which provides feedback based on actions that individuals physically take as opposed to keyboard input [21, 22]. Initial work and idea generation has also begun toward moving GIFT to the “wild”, such that learners can receive feedback through mobile devices, smart glasses, and smart watches instead of at a stationary computer [12, 23].

Integrating GIFT with virtual reality systems and head mounted displays such as Oculus RIFT has not yet occurred. However, it would be an interesting future direction that would be beneficial to examining immersive training. Providing tutoring in conjunction with head mounted displays could be advantageous, but it would also lead to specific considerations about feedback. If feedback was to occur visually it would require space within the display, and may distract from the training. Feedback could be provided by audio, which could assist in feelings of immersion or presence that a learner may experience in the training environment without overtaxing the visual channel. However, it would be beneficial for research to be conducted on the most effective ways to provide adaptive feedback when training is occurring using a head mounted display. Additionally, Cave Automatic Virtual Environments (CAVE) provide large screens that surround the individual with the virtual environment, and can be used for training. Similarly, to integrate a CAVE environment with adaptive tutoring, thought would need to be put into where the feedback would be displayed, and in which modality. A portion of the screen could be set aside for feedback, but it would be necessary to confirm that the participant looks at that area of the screen and that it does not negatively impact feelings of immersion. As the area that visual feedback is displayed on would be very different in both of these instances (head mounted display and CAVE) it may be helpful to have a tool that would allow authors to select where their visual feedback will be located and superimpose it upon the image screen size that they expect their learners to view. Templates for standard size screens could be provided. This would help for course planning purposes, and also can help course authors to see any initial problems that may come up with their configuration.

It would be helpful for authors to have very clear options and precise control over the modality that feedback is presented in. Some authors may want a character present with text, others may want a character present that speaks with no text, and others may just want text without a character taking up screen space. Providing clear flexibility and options within the authoring process will be beneficial to facilitating the feedback presentation. Additionally, it would be helpful for the course author to have a scenario preview before finalizing their course, such that they can see how their authored feedback is presented in the virtual environment without needing to validate their entire course.

As more research is needed into the impacts of presence and immersion on learning, GIFT could also be used to conduct studies. GIFT offers flexibility in the presentation of materials and has assessment capabilities that would be useful to conducting this type of research. Measures of presence and immersion can be given to learners who engage with GIFT through use of it’s Survey Authoring System (SAS). One or more questionnaires such as Witmer and Singer’s Presence Inventory [2] or the ITC-Sense of Presence Inventory [4] could be entered into the GIFT’s SAS and presented to the learner. Further, scores on the scale could be stored for the individual, and if desired, adaptation to the course could occur based on answers that are provided by the learner. These surveys could be automatically presented to the user before or after entering the training environment or completing gameplay.

4 Conclusions

Game based learning and immersive training environments can provide highly engaging educational environments for learners. However, often times there could be a benefit from integrating the learning environment with intelligent tutoring such that learners can get real time adaptive feedback. GIFT provides an opportunity to integrate a virtual environment with training, while aiming to reduce the time and effort that it would traditionally take to do so. GIFT could be used to conduct immersion and presence research, or it can be used by instructors who wish to provide an adaptive tutoring component to their game based training. GIFT has begun branching out into different tasks and modes of input/feedback that would be advantageous to use with virtual environments. Those who choose to integrate ITS and immersive training, need to give consideration to a number of features of their training including display and input methods, learner characteristics, and how/where feedback will be provided. Due to GIFT’s flexibility, it can be a used for future research to further examine the influence of presence and immersion on ITS learning outcomes in varying domains.

References

  1. 1.
    McMahan, A.: Immersion, engagement and presence. Video Game Theory Reader 67, 67–86 (2003)Google Scholar
  2. 2.
    Witmer, B.G., Singer, M.J.: Measuring presence in virtual environments: a presence questionnaire. Presence: Teleoper. Virtual Environ. 7(3), 225–240 (1998)CrossRefGoogle Scholar
  3. 3.
    Greenwald, W.: Kinect vs. PlayStation Move vs. Wii: Motion-Control Showdown. PC Magazine, 6 November 2010. http://www.pcmag.com/article2/0,2817,2372244,00.asp
  4. 4.
    Lessiter, J., Freeman, J., Keogh, E., Davidoff, J.: Across-media presence questionnaire: the ITC-sense of presence inventory. Presence 10(3), 282–297 (2001)CrossRefGoogle Scholar
  5. 5.
    Andre, A.D., Wickens, C.D.: When users want what’s not best for them. Ergon. Des. Q. Hum. Fact. Appl. 3(4), 10–14 (1995)CrossRefGoogle Scholar
  6. 6.
    Dede, C.: Immersive interfaces for engagement and learning. Science 323(5910), 66–69 (2009)CrossRefGoogle Scholar
  7. 7.
    Alexander, A.L., Brunye, T., Sidman, J., Weil, S.A.: From gaming to training: a review of studies on fidelity, immersion, presence, and buy-in and their effects on transfer in PC-based simulations and games. DARWARS Training Impact Group 5, 1–14 (2005)Google Scholar
  8. 8.
    Mantovani, F.: 12 VR learning: potential and challenges for the use of 3D environments in education and training. Towards Cyberpsychol.: Mind Cogn. Soc. Internet Age 2, 207–225 (2001)Google Scholar
  9. 9.
    Kickmeier-Rust, M.D., Albert, D.: Micro-adaptivity: protecting immersion in didactically adaptive digital educational games. J. Comput. Assist. Learn. 26(2), 95–105 (2010)CrossRefGoogle Scholar
  10. 10.
    Kickmeier-Rust, M.D., Hockemeyer, C., Albert, D., Augustin, T.: Micro adaptive, non-invasive knowledge assessment in educational games. In: 2nd IEEE International Conference on Digital Games and Intelligent Toys Based Education, pp. 135–137. IEEE (2008)Google Scholar
  11. 11.
    Moreno, R., Mayer, R.E.: Learning science in virtual reality multimedia environments: role of methods and media. J. Educ. Psychol. 94(3), 598–610 (2002)CrossRefGoogle Scholar
  12. 12.
    Sottilare, R.A., LaViola, J.: Extending intelligent tutoring beyond the desktop to the psychomotor domain. In: Proceedings of Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) (2015)Google Scholar
  13. 13.
    Harp, S.F., Mayer, R.E.: How seductive details do their damage: a theory of cognitive interest in science learning. J. Educ. Psychol. 90(3), 414–434 (1998)CrossRefGoogle Scholar
  14. 14.
    Mayer, R.E., Heiser, J., Lonn, S.: Cognitive constraints on multimedia learning: when presenting more material results in less understanding. J. Educ. Psychol. 93(1), 187–198 (2001)CrossRefGoogle Scholar
  15. 15.
    Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4, 167–207 (1995)CrossRefGoogle Scholar
  16. 16.
    VanLehn, K., Lynch, C., Schulze, K., Shapiro, J.A., Shelby, R., Taylor, L., et al.: The Andes physics tutoring system: lessons learned. Int. J. Artif. Intell. Educ. 15(3), 147–204 (2005)Google Scholar
  17. 17.
    Sottilare, R.A., Brawner, K.W., Goldberg, B.S., Holden, H.K.: The Generalized Intelligent Framework for Tutoring (GIFT) (2012). https://gifttutoring.org/documents/31
  18. 18.
    Sottilare, R.A., Graesser, A.C., Hu, X., Holden, H.: Preface. In: Design Recommendations for Intelligent Tutoring Systems: Volume 1, Learner Modeling, pp. ii–xiii (2013)Google Scholar
  19. 19.
    Goldberg, B., Cannon-Bowers, J.: Experimentation with the generalized intelligent framework for tutoring (GIFT): a testbed use case. In: AIED 2013 Workshops Proceedings, volume 7, pp. 27–36 (2013)Google Scholar
  20. 20.
    Goldberg, B., Cannon-Bowers, J.: Feedback source modality effects on training outcomes in a serious game: pedagogical agents make a difference. Comput. Hum. Behav. 52, 1–11 (2015)CrossRefGoogle Scholar
  21. 21.
    Goldberg, B., Amburn, C., Brawner, K., Westphal, M.: Developing models of expert performance for support in an adaptive marksmanship trainer. In: Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) (2014)Google Scholar
  22. 22.
    Goldberg, B., Amburn, C.: The application of GIFT in a psychomotor domain of instruction: a marksmanship use case. In: Generalized Intelligent Framework for Tutoring (GIFT) Users Symposium (GIFTSym3), pp. 115–124 (2015)Google Scholar
  23. 23.
    Sottilare, R.A.: Augmented cognition on the run: considerations for the design and authoring of mobile tutoring systems. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2015. LNCS, vol. 9183, pp. 683–689. Springer, Heidelberg (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.U.S. Army Research LaboratoryOrlandoUSA

Personalised recommendations