Keywords

1 Introduction

One of the effective strategies to facilitate a learner’s motivation during tutoring is to design effective pedagogical conversational agents (PCAs). Past studies in the learning sciences and on human-computer interaction (HCI) has taken multi disciplinary approaches in designing such PCAs [1, 2, 16]. These studies investigated the function of the agent, as well as the use of multiple agents [14]. Many studies also explored the effects of verbal communication strategies by designing effective interaction strategies, such as meta-cognitive suggestions [8, 9], linguistic strategies, such as politeness [23], and affective expressions, such as positive or negative triggers [7]. The use of multiple learning actors during teaching has also been studied [11]. Researchers are still collecting evidence on factors that contribute to the development and design of effective agents. Past studies have shown that visual design influences the learner’s cognitive state and behavior during learning activities, including motivation. Learners are as sensitive to the visual appearance of an online tutorial as they are to interactions with real humans. Literature on the Media Equation [18] showed that people use the same social rules in human-agent or human-computer interactions as they do with human-human interactions. Other studies have shown that even with animally designed agent [13] and the use of a schema of human personalities [6] people still attribute human characteristics to the agents. If agents were designed to encourage emotionally satisfying human-computer interactions, we could use them as an effective tutor to keep and enhance motivation during learning activities. The question rises as to what kind of visual representation could enhance the learner’s motivation to learn. In our study, we investigated the influence of the childlike designs, and how it could become an effective social cue to increase the learning motivation of the student.

1.1 The Types of Characters that Can Facilitate Motivation

Several debates discuss the design of the pedagogical agents [10]. Agents could be designed to look realistic or cartoon-like. However, even if the agent is life like and credible, a too realistic representation could distract the learners. The “uncanny valley” suggested by [17] indicates that movements and appearances that are almost realistically human-like evoke negative impressions, such as “creepiness”.

A study conducted by [5] showed that female learners tend to prefer detailed stylish agents as learning companions rather than as tutoring instructors. Their study showed that the effects of the representations maybe due to the gender of the learners. Our study only focused on the type of agents that are selected prior to the learning task; it did not focus on the psychological process of how learners felt about a 3D or 2D agent. Moreover, this study did not investigate the emotional states that were caused by the motivational behaviors of the agents during learning. We studied the emotional process during a tutoring session with a 3D or 2D agent and delved into their effects.

Japanese animation is well-known worldwide for its use of components that are childlike and cute (e.g., HelloKitty, Pokemon). These characters are often described as “kawaii”, which is an attributive adjective in modern Japanese and is often translated into English as “cute” [19]. Kawaii expresses feelings, such as “can’t bear to see, feel pity,” that are typically elicited by human infants and toddlers and young animals. In addition, the verb “moe”, which is mostly used by the Japanese subculture Otaku-culture, expresses a strong positive feeling towards a character in a Japanese animated show or game. A study indicated that, to evoke affective feelings, these characters must be designed with large eyes [4]. Based on these studies, the design of PCAs could benefit from adopting the design principles of kawaii and moe, such as the use of cartoon characters with large eyes.In this paper, we use the word “kawaii” as a collective term for “cute” and “childlike” .The deformed PCA in this study was designed with large eyes and uses moe principles to express the childlike facial characteristics of the 2D character. In the next section, we describe how we captured the affective state during an online tutoring session. We also discuss the issue of gender differences when using these PCAs and the framework we used to investigate them.

1.2 Effects of Gender in PCA Interactions

A few cultures view women as superior to men; some cultures view women as subordinate to men; other cultures fall somewhere in between. Therefore, it is important to investigate how such social status may influence learner’s affective states during interaction with a PCA. Several studies show that gender affects the interactions of the learner with the PCA [20] depending on the learner’s gender and the agent’s gender. [3] suggested that stereotypes could influence an individual’s perception of others online. They conducted a controlled experiment to assess the accuracy of interpersonal perceptions on computer-mediated conversations. Results showed that interpersonal perceptions do not differ between computer-mediated interactions and face-to-face interactions. One possibility for this result is that social cues may be intensified in computer-mediated settings. Perhaps the gender of pedagogical agents could evoke certain stereotypes from the learners. [12] examined how positive and negative comments expressed by conversational agents affected learning performance as a function of gender. Results showed that learners had more positive impressions toward male agents with positive expressions than toward female agents. These results show that, no matter what kind of influence is detected, perhaps social stereotypes in the real world are applied to the agent-learner relationship. These studies show that the learner’s gender and the agent’s gender influence the learner’s attitude towards the PCA. However, very few studies tried to investigate the effects of gender between the learner and the PCA; we investigated these effects using the integrated framework shown in Fig. 1. Social psychological literature has shown that gender effects are amplified with “cuteness” [22]. This indicates that gender differences may be significant when using a PCA with childlike characteristics.

Fig. 1.
figure 1figure 1

Interaction framework considering the learner’s gender and the PCA’s gender

Full size image

We use Russell’s two-dimensional theory of emotion as a dependent variable to discover affective states during learning activities [21], as shown in Fig. 2. Pleasure/displeasure (or valence) is a dimension of experience that refers to a hedonic tone. Activation is a dimension of experience that refers to a sense of energy. The vertical axis is a continuum ranging from sleep (at the lowest end), through drowsiness, relaxation, alertness, hyperactivity, and, finally, frenetic excitement (at the opposite end). Our study uses this model to analyze students’ affective states during learning activities. To measure the affective states of students during a learning activity with a PCA, we used emoticons (Fig. 3) to represent each emotional state in the two-dimensional model. These emoticons were presented during the online tutoring task, and learners were asked to select which emoticon best represents his/her emotional state during the task. The emoticons were selected based on a preliminary study and each represents a specific emotional state. The representations of affective states were selected through a preliminary selection task conducted with 14 participants. These participants were shown 48 random facial expressions, and they categorized the facial expressions into the emotional model shown in Fig. 2.

Fig. 2.
figure 2figure 2

Affective model based on Russell’stwo dimensions of affect

Full size image
Fig. 3.
figure 3figure 3

Emoticons used in this study

Full size image

1.3 Aim of the Study

The goal of this study is to investigate the effects of using PCAs with deformed childlike characteristics(e.g., cartooned and with big eyes). Based on the results of the previous studies, we focused on the effects of gender, which we assumed to be different based on the affective states. We also propose a new methodology on how to determine the learners’ affective states by allowing them to select emoticons based on Russell’s two-dimensional theory of emotions [21]. We attempt to answer the following questions using six different types of agents:

  1. (1)

    How do male and female learners’ emotions differ, in terms of affective sensitivity based on Russell’s two-dimensional emotion model?

  2. (2)

    How are male and female students’ affective states influenced by the degree of childlike designed into the PCAs?

Our study aims to show the implications of the use of childlike characters in inducing affective states and motivation in a web-based tutoring activity.

2 Method

We constructed a system that guides the learner in a simple web-based tutoring system. Students in an undergraduate psychology class used the system to review key terms taught in a class. They were guided by a pedagogical agent who encouraged students by providing meta-cognitive suggestions and search results about the term on the web for further understanding. Based on Russell’s emotional models [21], we collected emotional variables as dependent variables.

2.1 Participants

The participants were all undergraduate students who were taking a psychology classes part of a humanities degree program and who undertook a web-based tutoring task as part of the class work. We refer to these participants as “learners”. Their task was to read about key psychological terms and to answer a short quiz based on the literature.

2.2 Materials and Conditions

We designed the agent PCA representations based on two design principles: (1) using cartoon characters, (2) using big eyes. For comparison, we developed 3 types of avatars that were different in terms of their visual appearance (Fig. 4). To represent non- childlike PCAs for the “3D condition”, we used stylish 3D images of avatars that were created using Poser 8 (poser.smithmicro.com), which is a 3D image/animation design tool. In the 2D conditions, the PCAs were designed as cartoons using childlike design principles. In the 2D + eye condition, the cartoon PCA had big eyes. We compared the effects of the 3D condition and the basic 2D condition to study the differences between cartoon and non-cartoon agents. Likewise, we compared the effects of the regular 2D condition and the 2D + eyes condition to study the influence of using large eyes in affective states.

Fig. 4.
figure 4figure 4

Types of interfaces used in the study: left, 3D interfaces with no childlike characteristics; middle, 2D cartoon agents with childlike characteristics; right, 2D agents with more childlike characteristics, such as big eyes

Full size image

2.3 Experiment Design and Participants

We examined the following three independent variables: (1) gender of the learner (male or female), (2) gender of the agent (male or female), and (3) the type of the agent (3D, 2D, and 2D + eye). Table 1 shows the combinations. The letters indicate the type of the actor (‘H’ for human and ‘A’ for the agent) and the gender (‘F’ for female and ‘M’ for male). Each combination represents an experiment condition, as follows: (a) a female learner using a female agent H(F)/A(F), (b) a male learner using a female agent H(M)/A(F), (c) a female learner using a male agent H(F)/A(M), (d) a male learner using a male agent H(M)/A(M).

Table 1. Experiment conditionsandlabels
Full size table

The experiment was conducted in three classes. In each class, one type (3D, 2D, or 2D + eye) of PCA was used. In Experiment 1,the 3D representations were used with 153Japanese undergraduates (71 males, 82 females, mean age = 19.50 years). In Experiment 2, the basic 2D representations were used with 132 Japanese undergraduates (59 males, 73 females, mean age = 19.45 years). In Experiment 3, the 2D + eyes representations were used with 129 undergraduates (52 males, 77 females, mean age = 19.65 years).

2.4 Tutoring System

A web-based tutoring system was developed for the class, using a web server, a database, and rule-based scripts. It was managed as a member-only system, and its main purpose was to tutor key terms taught in the class by presenting descriptive content. A total of 30 different key terms (e.g., Gestalt, long-term memory, cognitive dissonance) were extracted from an introductory psychology textbook, and its explanations were entered in the system database. Each student was assigned to work on one randomly selected key term. The tutoring sessions comprised 17 short passages, and students proceeded by clicking on to the next page/trial (Fig. 5). During the task, students were encouraged to go beyond simply reading these passages and to search through the web to further understand the terms. The average time for this activity was approximately 30 min.

Fig. 5.
figure 5figure 5

Experiment procedure

Full size image

Students were restricted to using specific computers in the campus at specific time periods. Learners were asked to log in to their individual page to check their progress. On the first page, the key term assigned to the student was presented, and the student was told that learning this key term was his/her task for that week. On the next page, the passage for the key term was presented with the emoticons (Fig. 6).

Fig. 6.
figure 6figure 6

Sequence of the tutoring/affective evaluation phase

Full size image

The following equation calculates the ratio of each emoticon emo, where i is the number representing the emoticon/affective state and total is the total number of the trials.

$$ 1 = \sum\nolimits_{i = 1}^{8} {\frac{{emo_{i} }}{total}} $$
(1)

3 Results

A 2 × 2 × 8 mixed factorial analysis of variance (ANOVA) was conducted on the average scores with the PCA’s gender (female agent vs. male agent) and the learner’s gender (female learner vs. male learner) as the between-subject factor, and affective state (pleasure vs. excitement vs. arousal vs. distress vs. displeasure vs. depression vs. sleepiness vs. relaxation) as the within-subject factor. When analyzing the results of the factorial analysis, we will only focus on the gender differences of the learner and of the PCA with each affective state.

3.1 Experiment 1: 3D Characters

Figure 7 shows the results calculated by the emo index. The vertical axis represents the average ratio of each individual, and the horizontal axis indicates each affective category. The ANOVA results show that the second-order interaction was not significant (F(7,1043) = 1.740, p = .10); however, the interaction between the learner’s gender and the affective state was significant (F(7,1043) = 2.181, p < .05). The interaction between the PCA’s gender and the affective state was also significant (F(7,1043) = 2.405, p < .05). However, no simple main effects were detected in each affective state. Simple main effects only show the interactions across affective states. The results of this experiment indicated no significant effects of gender on each affective state.

Fig. 7.
figure 7figure 7

Results of Experiment 1

Full size image

3.2 Experiment 2: 2D Characters

As shown in Fig. 8, ANOVA results show that the second-order interaction was significant (F(7,896) = 4.478, p < .01).A simple interaction exists between the agent gender and the human gender in the affective states of excitement and arousal (F(1,1024) = 22.675, p < .01, F(1,1024) = 9.346, p < .01). Second-order simple main effects show several differences among the conditions. Female learners rated higher excitement when using female PCAs than when using male PCAs (p < .01), and male learners rated higher arousal when using female PCAs than when using male PCAs (p < .01). Male learners also felt more excited when using male PCAs than when using female PCAs. These results indicate that same-gender PCAs induce excitement for both male and female learners. However, males experienced arousal when interacting with PCAs of the opposite gender.

Fig. 8.
figure 8figure 8

Results of Experiment 2

Full size image

3.3 Experiment 3: 2D Characters with Large Eyes

Figure 9 shows the results of Experiment 3. ANOVA results show that the second-order interaction was significant (F(7,875) = 2.565, p < .05). A simple interaction existed between the agent gender and the human gender in the affective states of pleasure, excitement, and arousal (F(1,1000) = 4.677, p < .05, F(1,1000) = 7.680, p < .01, F(1,1000) = 4.013, p < .05). Second-order simple main effects showed several differences among the conditions. Female learners rated higher pleasure when using female PCAs than when using male PCAs (p < .01). Male learners rated higher arousal when using female PCAs than when using male PCAs (p < .01). Male learners also felt more excited when using male PCAs compared to female PCAs. These results indicate that same-gender PCAs produces pleasure for females and excitement for males. These results are consistent with the results of Experiment 2, which shows that males experienced arousal when using PCAs of the opposite gender.

Fig. 9.
figure 9figure 9

Results of Experiment 3

Full size image

Figure 10 shows the summary of the results of the positive affective state according to the gender of the learner and of the PCA. Male learners using a female PCA experienced more arousal, but this occurs only with 2D agents. On the other hand, female learners felt more excited when they used a female 2D PCA, and they felt more pleasure if the PCA had larger eyes.

Fig. 10.
figure 10figure 10

Summary of results

Full size image

4 Discussions and Conclusion

We investigated the effects of using PCAs that feature childlike characteristics in an online tutoring task. We studied the 8 types of a learner’s affective states and processes during tutoring activities. Previous studies demonstrated that affective states are influenced by social status and gender, and we investigated how affective states change based on the gender of childlike PCAs.

Results showed that male learners using a female PCA experienced more arousal, but this occurs only with 2D agents. On the other hand, female learners felt more excited when they used a female 2D PCA, and they felt more pleasure if the PCA had large eyes. These results indicate several things. The gender interaction only occurs with the 2D cartoon character PCAs. Males feel more excited when they interact with PCAs of the same gender. Excitement includes elements of pleasure; therefore, it could mean that learners felt more positive feelings toward the agent. Based on these results, designers of PCAs should consider using 2D PCAs of the same gender for learners to induce positive emotions during online tutoring. An interesting point is that the 2D + eye interface was more successful in producing positive emotions. This also suggests that using PCAs with kawaii design components have a stronger effect on female learners. To understand why such characteristics have such an effect on inducing pleasure feelings, some studies investigated the social psychological structure of feelings of cuteness to explain that people perceive things as cute based on ‘baby schema’ [15]. Baby schema is characterized as a combination of infant-like physical traits, such as (1) a small body size with a disproportionately large head, (2) large eyes, (3) a pleasantly fair complexion, (4) a small nose, (5) dimples, and (6) round and softer body features. When people perceive humans or animals with these characteristics, they perceive cuteness. In the 2D + eye condition, we used PCAs with large eyes, which is a characteristic of the baby schema, and we focused only on the eyes of the PCA. Further research can be done where we modify other parts to increase these characteristics to determine the ideal design for emotion-based learning systems using PCAs.

The results of our study indicate that using deformed characters as PCAs can be used to induce positive emotions and motivation in a web-based tutoring activity. In future work, we will study the different moods produced by using the tutoring systems from mobile devices, such as smart phones.