Two types of facilitation techniques for enhancing coordinative practice
As stated above, this study investigates the effects of learner gaze-awareness feedback and PCA-based feedback, which are two intervention paradigms that have been used to effectively facilitate coordination toward mutual understanding (Richardson and Dale 2005). These methods have recently been applied in educational contexts (Schneider and Pea 2013, 2014). This subsection first reviews and discusses studies on the utility of awareness tools, particularly visual gaze feedback via eye-tracking, in facilitating coordinative activities. Then, we review and discuss the use of the PCA (Heidig and Clarebout 2011), which is a useful tool for collaborative learning activities using Intelligent Tutoring systems (ITSs) as they facilitate metacognitive processes. Both the advantages and limitations of the PCA technology in terms of the provision of support for inter-learner coordinative activities are also examined.
Social awareness: Visual gaze feedback via eye-tracking
Studies on computer-mediated communication have shown that, in distributed learning situations such as computer-mediated environments, individuals communicating through devices exhibit low awareness of one another and may form an incorrect understanding (Sproull and Kiesler 1991). Previous studies on computer-supported collaborative work, which were conducted over the last few decades, investigated the development of awareness tools that provide rich information on the ways individuals engage in activities through their experimental manipulation of behavior (Dourish and Bellotti 1992; Schmidt 2002). Hence, several types of awareness, such as social and cognitive awareness, were defined (Janssen and Bodemer 2013). In particular, social awareness refers a group member’s awareness of the activities and the online states of others. Cognitive awareness refers to the awareness of information about group members’ knowledge and expertise. According to Janssen and Bodemer (2013), these two types of awareness are important for learners performing social and communicative activities by establishing shared common knowledge and can further enable learners to acquire a deeper understanding of the domain knowledge associated with a given collaborative task.
Mutual gaze is commonly known as eye contact and is studied in the context of social relationships and interpersonal interactions (Goodwin 1981). In one of the primary studies in the field of HCI (Buxton and Moran 1990), systems known as “video tunnels” were developed. In these tunnels, half-silvered mirrors are used to set a camera angle as though the camera originates behind the eyes of the video image of a remote viewer. Similarly, a gaze awareness display inspired by ClearBoard (Ishii et al. 1993) was developed to enable speakers to establish full gaze awareness, including facial expressions. Such devices have been used to investigate how full gaze awareness can be an efficient resource for establishing grounding beyond that provided by a view of facial expressions with real mutual gaze (Monk and Gale 2002).
With advances in sensing technology, eye-tracking sensors were used in several studies to elucidate the nature of human-human coordination so as to facilitate the communication process (Jermann et al. 2011; Richardson and Dale 2005). Richardson et al. (2007) utilized two eye-trackers to investigate the relationships between speakers engaged in live, spontaneous dialog. Their analysis revealed a recurrence between the eye movements of the speaker and listener, which was established through shared common knowledge. A study showed that the degree of gaze recurrence (the portion of time for which the gazes are aligned) in speaker-listener dyads is correlated with the establishment of common ground; thus, building common ground positively influences visual attention coordination (Richardson and Dale 2005).
Various study results show that, in search tasks with different conditions, speakers can successfully communicate and coordinate their search efforts using shared gaze (Brennan et al. 2007; Keysar et al. 2000). In these studies, direct feedback on the visual gazes of collaborative partners was provided using eye-trackers (Jermann et al. 2011), which physically indicated the direction of the other collaborative learner’s gaze on the same computer screen; hence, joint attention could be achieved. In another study, the learner sequence alignments were modeled (Khedher et al. 2017). Some well-known works (Schneider and Pea 2013, 2014) demonstrated that visibly representing a partner’s gaze during a remote computer-based learning task can facilitate social collaboration and learning. In these studies, dyads collaborating remotely in a learning task learned about a neuro-science phenomenon by employing diagrams and tracing one another’s gaze behaviors. In one experiment, the participants were provided with information on their partner’s physical eye gaze on the screen. The control group lacked this information. Subsequent analysis revealed that real-time direct mutual gaze perception enables higher-quality collaboration for students.
Considering the results of these studies, gaze awareness tools can be taken to foster joint attention and take advantage of the collaboration process, which requires success in communication, as illustrated through examples such as perspective-taking in a knowledge integration task. This type of intervention is not invasive, so it has the benefit that it ensures learners’ free engagement in social interactions. However, there are concerns that to generate effective collaboration, collaboration methods must be learned and thus require guidance, instruction, and training (Slavin 1992). These findings indicate that students who do not have experience or training in collaboration may encounter difficulty in complex explanation activities, specifically in regulating their own cognitive behaviors regarding decision making in particular circumstances. Awareness tools do not provide explicit guidance about how to foster the individual and collaborative learning process. Building on this point, the use of gaze awareness tools may be limited to only particular aspects of the collaborative process. Thus, it may be more effective to also use external support over and above what gaze awareness provides, in particular, interventions that facilitate the student’s metacognition and self-reflection. The next section describes in detail the effective use of conversational agents to enhance collaboration support at the meta-level.
Metacognitive suggestions: Pedagogical conversational agents
Previous studies in CSCL have investigated the use of external collaboration scripts for collaborative learning that are supportive of individual acquisition of knowledge. As discussed at the beginning of this paper, such methods take advantage of students building on each other’s contributions within the knowledge integration tasks. Recent studies reveal that providing these external suggestions dynamically based on detecting learner states remains a challenge. In the context of ITS development, artificial intelligence in education has a long history (Aleven et al. 2006; Koedinger et al. 1997). Many of these tutoring systems provide adaptive and individual learner support, which would be difficult to achieve using human teachers alone. Other studies investigated the effects of teaching via tutoring systems (Biswas et al. 2005), the relative effectiveness of agent-provided facilitation prompts in self-regulated learning (Azevedo and Cromley 2004), and the development of systems employing advanced detectors to elucidate the learner state and generate facilitation prompts (D’Mello et al. 2012). Learning involving one-on-one dialog with dialog-based tutoring systems was shown to be more effective than simple reading and lecture attendance (VanLehn et al. 2007). When discussing the development of such dialog-based tutoring systems, it is important to mention the work of Graesser and studies on learners using AutoTutor (Nye et al. 2014). In those studies, conversational agents that provide hints, prompts, and motivate learners to meet expectations for answers to posed questions were developed based on student dialog analysis (Graesser et al. 2005). These emerging technologies for the design of PCAs as virtual teachers have been recognized as effective learner support methods.
Moreover, in 2015, the Program for International Student Assessment (PISA) governing board, administered by the Organization for Economic Cooperation and Development (OECD), assessed collaborative problem-solving using PCAs (Greiff et al. 2017). As regards the scope of the present study, there are several works that focus on the use of PCAs in learner-learner collaborations and the conversational agents were found to leverage performance by facilitating goal achievement (Holmes 2007), prompting periodic initiation opportunities (Kumar and Rosé 2011), and collaboratively setting sub-goals (Harley et al. 2017). Several design methods were investigated, such as the provision of positive emotional feedback via both dialog and visual representations of metacognitive suggestions (Hayashi 2012), the use of multiple PCAs based on this feedback (Hayashi 2019), and the use of gaze gestures during learner-learner interactions (Hayashi 2016). In those studies, the PCA successfully facilitated learner self-explanation activities and metacognitive behaviors, such as reflections. These previous studies on PCAs have shown that such technologies are useful for providing external support for internal processes, such as self-regulation and metacognition, primarily for individual-level learning support.
Building on these considerations, this study focuses on the following three types of functions: (1) Metacognitive suggestions, (2) facilitation of knowledge integration, and (3) communication encouragement. For (1), this work draws from a past study (Hayashi 2012), and related studies such as Azevedo and Cromley (2004), which have shown that the use of indirect facilitation techniques can facilitate self-regulation and metacognition. For (2), this study offers facilitation in the form of questions to learners requiring them to give examples related to task achievement. In previous studies, Graesser et al. (2005) found a set of interaction components prevalent in normal tutoring situations, such as anchoring learning in specific examples. Additionally, as Chi et al. (1994) point out, for students to develop a deep understanding, it is important for them not only to understand each separate component but to explain to themselves the relationships within and among them. For (3), this study employs facilitation prompts to elicit the learner’s motivation-related remarks on communication, such as compelling communication between the learners. This was accomplished by providing positive back-channel feedback when the learners were using words related to the task activity. Also, previous studies show that the embodied characteristics of the agent and its role in stimulating the learners by encouragement have the ability to foster motivation towards learning (Baylor and Kim 2005). We adopted this point by implementing an embodied agent that synchronized its movements and provided positive feedback when the learners were using sophisticated words during their interactions.
Study goal and hypothesis
Combining and integrating different background knowledge across members of a group is an effective strategy for developing new knowledge. Collaborative learning is beneficial in that it offers learners the opportunity to generate explanations and be exposed to different opinions from others, which might provide the opportunity to elaborate their own internal representation of knowledge. During such tasks, learners must both coordinate with others and regulate themselves to think dialectically and construct a comprehensive perspective. However, as discussed previously, knowledge integration activities may fail for several reasons. In computer-mediated environments, students often persist with low awareness about the perspectives of others, such as the topics/opinions their partners refer to during the exercise. Moreover, most learners, especially those in the early years of college, do not have any training or knowledge of how to coordinate successfully or self-regulate their cognitive behaviors to adjust their conversations in an ideal way.
Based on these points, this study used a simple knowledge integration task and investigated the effects of using gaze awareness tools, which are interventions that can foster joint attention, and external facilitations from a PCA, which can foster metacognitive awareness. Previous studies have examined the effects of using gaze awareness tools and PCAs on some specific tasks; however, there is a gap in the literature specifically with respect to evaluations of which supportive technology within this space holds the greatest potential for impact on the collaborative process and performance on knowledge integration tasks. By probing into this space in particular, we may be able to design better online collaborative learning systems, especially for knowledge integration tasks under challenging conditions with respect to group awareness. Therefore, this study investigated (1) direct facilitation using partner gaze awareness and (2) indirect third-person facilitation via a PCA, following a 2 × 2 controlled experiment design; hence, the manner in which these two methods facilitate the collaborative process was examined. On investigating the collaborative process, a coding scheme (Meier et al. 2007) was employed, which captures the crucial collaborative coordination features: mutual understanding, dialog management, information pooling, consensus reaching, task division, time management, technical coordination, reciprocal interaction, and individual task orientation (details are provided in the Methods section). For learning performance, this study focuses on assessing how well the learners were able to gauge differences in each other’s knowledge.
Figure 1 shows the research framework of this study, with the two targeted methods highlighted. These methods facilitate the collaboration process, influencing the learning performance during the task activities. Note that good learning performance is the byproduct of a successful coordination process.
We predicted that using gaze awareness interventions would enhance joint attention. Therefore, learners can coordinate better with their partner by understanding what their partner is paying attention to during the task. More specifically, the partner’s gaze provides awareness of their focus of attention, which enables learners to perform more successfully in all the collaborative processes enumerated in the coding scheme (Meier et al. 2007). Furthermore, it is predicted that if the tool enables the learners to see where their partner is looking while they are producing their explanations, it will allow them to see if their partner is paying attention to their explanations or referring to the suggestions from the PCA’s comments. This may make it easier to plan their next conversational move and influences their turn-taking behaviors. Therefore, it is predicted that gaze awareness will influence the collaborative process of communication, such as mutual understanding and dialogue management. Also, awareness of their partner’s gaze patterns can help reduce conflict when they try to pool information and reach a consensus because the gaze patterns show where their partner’s areas of interest have been, such as their contributions or their partner’s contributions or the PCA’s comments. Therefore, it is predicted that gaze awareness will influence joint information processing, such as information pooling and consensus building. Moreover, if one can see that their partner’s gaze is biased to a particular point, one might conclude that they should work on the task more efficiently, possibly through a change in roles. It is predicted that this will influence the collaborative process, such as coordination related to task divisions and time management. Also, if the learners are successful in developing such a process during the task, this may impact the learning performance’s efficiency, deepening the understanding of each other’s individual knowledge and their integrated knowledge. Therefore, in this study, it is hypothesized that learners using gaze feedback will achieve better results in the collaborative learning process compared to those who do not use such a method (H1a-1). Moreover, if learners can achieve successful collaborations during their explanations, this may improve their understanding of each other’s different knowledge and therefore, influence learning performance. Consequently, it is expected that this will also affect the learning performance in the explanation task (H1a-2). However, H1a-2 may not produce a strong effect because gaze awareness does not directly scaffold metacognition and knowledge integration.
As mentioned in the previous section, the PCA will provide interventions such as (1) Metacognitive suggestions, (2) facilitation of knowledge integration, and (3) communication encouragement. Therefore, the PCA was expected to provide direct facilitation about coordination and metacognition, such as encouraging their activities and self-regulating their behaviors when making explanations to meet the task goal. Such direct verbal information should help the learners to think about the task goal, what to do, and what to talk about. The hops is that this process will lead them to more effectively adjust their behaviors to coordinate with each other more successfully. Moreover, the PCA’s comments are expected to elevate their level of motivation, thus encouraging task orientation during their collaborative process. Considering these points, it was hypothesized that learners receiving these suggestions from a PCA would achieve superior performance in terms of the collaborative process as compared to those who do not have access to such support (H1b-1). However, metacognitive suggestions may have limited effect on facilitating the collaborative process. This form of support lacks in providing information about the partner’s awareness, which may play an important role in establishing successful communication. In contrast, it was expected that the metacognitive suggestions would impact learning gains through better understanding of the task knowledge, which requires reflective cognitive processing (H1b-2). Therefore, PCA intervention might be more effective than the mutual gaze intervention with respect to collaborative performance, while gaze might be more effective at improving the collaborative learning process.
Upon review, each tool has its advantages and disadvantages in supporting collaborative process and performance in this knowledge integration task. Therefore, metacognitive suggestions about the collaboration process from the PCA is expected to complement the collaborative process for joint attention using gaze awareness tools. Conversely, metacognitive suggestions should provide learners with ways to think about coordinating by providing more visibility into the reasons behind their partner’s gaze. Therefore, a combination of the two methods can be expected to facilitate coordinated activity (H1c-1) and influence learning (H1c-2). Overall, H1 pertains to the synergetic use of the targeted facilitation methods. In Fig. 1, this aspect is indicated by a dotted line.
The next aspect considered in this study is the relationship between the learning process and learning performance, and how this relationship is improved when the two facilitation methods (gaze feedback and PCA) are used. As discussed previously, successfully coordinated activities are essential for completing the task considered in this study, which is for learners to understand each other’s different perspectives and to integrate these perspectives to develop new knowledge. Therefore, it can be predicted that successful coordination and explanation will yield higher performance in terms of learners’ understanding of their own and others’ knowledge (H2–1). Learners who receive both gaze feedback and PCA suggestions can exploit both facilitation methods. Accordingly, it is hypothesized that learners who employ both facilitation methods will achieve higher performance in terms of coordination and explanation than learners who do not use either or both methods (H2–2). The study and findings are reported below.