Keywords

1 Introduction

As it has been previously mentioned [45], the underlying technical aspects of pedagogical agent designs have been analyzed in various detail regarding their outward appearances. Examples being their depiction regarding the displayed gender and its influence on learners [2, 15, 16], the inclusion of facial action movement capabilities in order to convey non-verbal cues [1, 14], ethnicity [13] or whether or not the inclusion of animations is hindering or beneficial within the learning context [4, 18, 22].

Compared to these varying degrees of investing resources to have a pedagogical agent appear more lifelike, we propose a shift in effort to reengineer the mechanisms of an agent and to enhance its capabilities to not only convey information but to include the environment, other learners than the one in front of the screen and other elements into the learning experience. Therefore, building on the described exhibition context described earlier [18, 46] we present the interactions of different technological aspects, working towards a social catalyst function due to proactive elements. Thereby we integrate works about passive aspects of pedagogical agent design like the politeness and the social conversational style, as in the wording choices of information conveyance, the amicability of the agent and whether or not it would be useful to include discussions about topics not related to the information material at hand [28, 30], the creation of a beneficial learning environment regarding communication beyond the learning material [29] as well as the possibilities of including gestures and facial mimics [9] or non-verbal input [3]. For a broader view on the topic, refer to [12].

Therefore, a systematic consideration of individual learning requirements is not yet the focus of empirical research. So, while there is ample research about the various facets of the depiction of a pedagogical agent inside an information transfer situation, there is very little about the human-computer-interface-features which an agent might be able to provide to the user. But to integrate a pedagogical agent into a learning environment and in order to have it be able to integrate active listening, visual observations and additional information just-in-time, the system has to perform pro-actively and include these aspects into the predetermined learning context.

2 Proactive Enhancements of the Agent System

Reeves and Nass [23] have already proposed that there is a deep social link between users and computer systems. They postulate that there is a human tendency to act social in a conversational situation, even if this communication is aimed at a machine. However, Lester et al. [17] formulated a persona effect which includes the observed behavior by Reeves and Nass and expand it by focusing on the anthropomorphic depiction of an agent. But if a user is acting socially towards a machine, then it appears only logical to assume that a user will in turn expect to receive socially adequate responses from an agent system. Nevertheless, human to human conversational strategies are not limited to the rather limited input devices of modern day computer systems. Instead, they are heavily dependent on the decoding capabilities of non-verbal communication attempts like for example intonation, facial expressions and the gaze of the conversation partner. Therefore a way of including the sensor-array of currently available hardware into the communication with a pedagogical agent was developed. A commonly used webcam is consequently utilized to detect a user’s gaze upon the display. Should it divert away from the presented information material, the presentation is stopped. Furthermore a microphone is used to check for ambient noise levels and checks, whether or not the verbal output of the agent can be understood by the learner in front of the screen. Hence, the pedagogical agent is upgraded to the real world capabilities of any other teacher tasked with conveying learning materials: if the student is not attentive or if it is too noisy, the agent will suspend the presentation and continue, once the environment or non-verbal cues by the learner allow it.

As previous experience has shown, incorporation of narrative elements can be an effective tool towards strengthening visitor engagement [21]. Therefore we are going to enhance the previously outlined museum installation [46] by aspects of technical and narrative learner engagement. For example, due to the added microphone to listen in on the environment, the agent system would be able to check for background noises which might hinder the experience of a specific museum exhibit (e.g. of musical instruments or the presentation inside a cinema). Furthermore, the agent might be listening for keywords uttered by nearby visitors, which could indicate a common interest for an exhibit and therefore initialize the social catalyst routine, as described in [46]. The webcam on the other hand would be capable to check for signs of confusion or other unwanted emotions which would indicate a hindered experience of the exhibitions. Using libraries of the Facial Action Coding System [14] the ‘brow lowerer’ could indicate that something has not been understood or that the visitor does not recognize the elements as described by the agent system. Therefore, the system might explain the last part in a different way or with a different visualization, until the ‘brow lowerer’ is either not recognized anymore or might even be replaced by an ‘outer brow raiser’, which might indicate, that a user has found something that was missing before. Our constructive efforts are informed by a methodology for interdisciplinary development [20] based on analyses of interdisciplinary practice [7, 18, 19].

Obviously, other forms of physiological interaction with the agent system are within the bounds of possibility for current technology. Smartwear, like the Android-wear or fitness-trackers, offer a wide range of available biophysiological information about the user. The Microsoft-Band [2] for example offers skin-conductance measurements next to continuous heart-rate monitoring and movements. These kinds of measurements were limited to expensive and specialized research hardware a few years ago, but offer new perspectives for user-centered agent designs. Changes in biophysiological feedback can signal the system not to pursue an available connection between two visitors [46] and changes in the usual physiological measurement when looking at an exhibition can be used to determine interest within a topical domain or not. Additionally, gaze tracking via webcams is already widely researched as being capable of facilitating new forms of interaction with a device [5, 7] while real eye-tracking devices tend to become more readily available [27], allowing for the implementation of interaction strategies for handicapped or incapacitated persons [19, 24].

Furthermore, narrative techniques are used to enhance the learning context, to facilitate an interaction process between the strangers visiting the exhibition, enabling the agent system to provide the previously described social catalyst capabilities while reporting the current status of software development and hardware integration.

Due to narratives being a form of information conveyance, its origins can be found in a time when writing was not available to the population [8, 25, 33]. As Gerrig [16] points out, the information processing is the same, whether the received information is fictional or real. Regarding the proactive pedagogical agent, it has to be deemed trustworthy in order to facilitate a believability, but possesses the power to convey complex information in a more accessible way [20]. Within this context, Bruner [6] states that experiences and memories are stored as stories. This in turn should lead to a better retention of conveyed information when it is already presented while taking narrative structures into account.

So by being mindful of narrative structures, like temporal cohesion, progression of storyline and the individual complex developments of the characters [48], the learning material is anchored [26]. But this means a greater complexity for the agent system itself, since it has to obey the basic narrative structures and has to include the acquired environmental information. In order to achieve this goal, the learning material has to be pre-structured according to narrative rules.

  • Narrations are recognizable patterns of characters, events and have to regard cultural storytelling techniques [35].

  • Stories itself are not considered to be interesting. Only by the inclusion of one or more relatable characters, a narration is able to function [36].

  • Since characters are pivotal, the current living situation has to be ‘threatened’ [6, 37, 38, 40, 44] but the crisis can be resolved by implementing newly experienced events.

  • Stereotypical applications of characters can be used to reduce the complexity of the narrated scenario [42].

In order to keep the development of the learning material at a lower level of complexity, the narrative technique of five acts should be implemented [43].

  1. 1.

    Characters and the basic setting is described, upcoming challenges are insinuated but kept vague.

  2. 2.

    Events are depicted in greater detail and the introduced characters are confronted with the challenges ahead.

  3. 3.

    Depicted occurrences culminate into a cataclysmic event which the characters are not able to solve with the acquired knowledge up to this point.

  4. 4.

    The characters are given time to reevaluate their attempts, their knowledge and to complement their learned lessons.

  5. 5.

    A reintroduction of the events in step three, accompanied by ways of implementing the learned and trained knowledge to succeed in the storyline.

By implementing these features, the learner is able to gather relevant information in a concrete and tangible way while maintaining an easy to memorize structure of the learning material.

3 The Agent as a Social Catalyst

Due to the necessity of relatable characters and based on the narrative structure, the agent system is then capable of integrating other visitors of the exhibition into the learning scenario. By implementing the already described steps [46] to facilitate a communication between former strangers, the proactive pedagogical agent is able to structure learning material according to the storyline. Therefore the aforementioned model [46] is augmented by the narrative elements.

  1. 1.

    Based on information about the users beforehand, the system makes an initial analysis about user interest compatibility

  2. 2.

    The initial connection to another user is facilitated by the two respective agents onscreen, depending on the proximity of the users to each other and an exhibit which is used as an anchor for the initiation

  3. 3.

    Once the agents acknowledge each other, the storyline is adapted to the specific interests and used to engage the users into a conversation by pre-structuring the knowledge elements

  4. 4.

    Users have the option to decline this event in a courteous way without offending the other person – the agents then disengage and continue to function as a traditional information system while increasing the distance to the failed interaction

  5. 5.

    If the interaction is confirmed by both parties, the agents join each other visibly on the shared screen adjacent to the exhibition and initiate the information conveyance by building on the narrative structure

4 Proactively Enhanced Museum Exhibition

In accordance with the published scenario, the two level system of a handheld mobile device and a wall mounted installation is retained [46]. The tablet is used to display information about the exhibit in conjunction with environmental information like location, chain of visited exhibits and predetermined interests of the user. Wall mounted displays transmit information via RFID or simple QR Code like information encoding and trigger adequate reactions inside the electronic educational instance (EEI) [45, 47] (Fig. 1).

Fig. 1.
figure 1

Enhanced model of the Electronic Educational Instance (EEI) [45, 47]

Full size image

Within this enhanced EEI, the additional environmental information of the sensory input is obtained, analysed and a corresponding variation of the learning material is prepared. Thereby, the proactive pedagogical agent is capable of detecting potentially hindering elements and is able to form a situational context within the exhibition. Accordingly, in conjunction with the previously entered person specific information, the learning material can be adapted to information about the users and is capable to proactively prepare the learning material for people with impairments.

The newly integrated sensory input devices is capable of gathering information about the environment and to adapt to the individual preferences and outside information. Due to the expanded social catalyst function, the visitors of the exhibition are, with the help of their information devices as well as their personal agents, enabled to learn on their own or in cooperation with other, like-minded visitors.

5 Conclusion

The presented paper discussed the pre-requirements for the development of pedagogical agents as well as the current steps being undertaken to enable the proactive functions of the agent system in the form of the aforementioned model regarding the enhanced electronic educational instance. This EEI model is currently used to integrate already established learning material and to integrate sensory input on stationary and mobile devices.