1 Introduction

Experts agree that online collaborative learning has the potentials to develop students' critical thinking skills. For a discussion to be meaningful and sustained, each individual needs to play an active role as a member of a community of inquiry. The community of inquiry framework (CoI) is a framework designed to guide meaningful online discussions; it explains the online collaborative learning process within a community of inquiry as to the dynamics of social presence (SP), cognitive presence (CP), and teaching presence (TP) (Garrison et al., 1999).

Sustained and meaningful online collaborative learning requires students to play their part in a knowledge construction discourse. Although they already have the technical readiness for learning in an online learning environment, students may not have the necessary skills for interacting meaningfully within their learning community (Junus et al., 2019). Moreover, these skills cannot be acquired trivially. By applying the CoI framework, the necessary skills can be taught to students to fulfill their roles in online collaborative learning. Although the CoI framework has been implemented in several universities in Indonesia (Pratiwi et al., 2016; Yandra et al., 2021), studies on instructing students in the CoI framework to improve their preparedness to collaborate online in Indonesia are still limited. This current study used the CoI framework for two essential purposes: the study's theoretical framework and the teaching of the framework in the online role-playing context.

The CoI is one of the areas of learning covered in the Computer-aided Instruction (CAI) course, an elective course offered by the Faculty of Computer Science Universitas Indonesia. The objective of this learning area is to prepare and equip students to play a more active role in an online discussion through SP, CP, and TP. The learning strategy chosen to deliver the CoI framework is that of role-playing, which will be followed by discussion and reflection. This strategy was selected as the learning objectives involve changing behaviours and forming interpersonal skills in an online collaborative learning environment. The following discussion and reflection are designed to allow students to articulate and internalise their learning experience using the CoI framework.

1.1 Problem statement

The CoI framework guides learners and instructors in promoting a meaningful and sustained online discussion. Education experts agree that role-playing activity has the potential to develop interpersonal skills in an online collaborative learning environment. Ideally, online role-playing on the CoI framework would be able to improve students’ participation in sustained discussion through social, teaching, and cognitive presences. Encouraging student participation in discussion forums is a challenge in online collaborative learning. Participation rates are low for various reasons, and students do not yet understand their role as group members in online collaborative learning.

The CoI framework is used to guide students in carrying out their roles to participate in text-based interactions. The CoI framework is a new concept for students. They may not have realized the importance of presenting the three components of the CoI framework. Participating in online role-playing maybe also the first experience for students, so they need to adapt to this method. In this study, online role-playing was applied to prepare, implement, and strengthen the experience through discussion and reflection. The selection of roles to be played is adjusted to the actual situation setting in an online learning environment of a faculty that will implement online collaborative learning. It is expected that students can build meaningful discussions guided by the CoI framework and gain meaningful learning experiences.

1.2 Research questions

This study investigates how students perform role-playing and project themselves in social, teaching, and cognitive presences and how students interpret their learning experiences. The following research questions guide this study:

  • RQ1: What characteristics of role-playing did the students exhibit?

  • RQ2: What are the patterns of social presence, cognitive presence, and teaching presence for each group?

  • RQ3: How are the students’ perceptions of the role-playing learning experience and their learning satisfaction?

2 Relevant literature review

2.1 Role-playing

Role-playing is an instructional technique, taking on the role of a character, the assumption of a part, or the representation of a type in a pretend situation (Collins & O’Brien, 2003). This instructional strategy is intended to stimulate critical thinking as students learn new skills. Role-playing is a teaching strategy that fits within the social family of models (Joyce et al., 2000). These strategies emphasise the social nature of learning and consider cooperative behaviour as stimulating students, both socially and intellectually (Jarvis et al., 2002). They also emphasise that the role-playing teaching strategy is advantageous for both students and instructors: raising student interest in the topic, increasing involvement in the learning process, and teaching empathy and understanding for different perspectives.

Greenberg and Eskew (1993) presented a review article on how role-playing has been used to learn about behaviour in organisations. They categorised three critical dimensions by which previous papers may be characterised: Level of Involvement (passive-active dimension), Role Being Played (self-other), and Degree of Response Specificity (free and spontaneous manner – highly restricted, specified manner).

Online role-playing has the potential to be an effective student-centered learning activity that can contribute to a successful and highly enjoyable learning experience (Bender, 2005; Erturk, 2015). The online role-playing strategy provides students with the opportunity to interact meaningfully with the learning material and each other. Instructors are benefited from gaining more excellent knowledge and understanding of each student (Bender, 2005). Hence, the instructors are better equipped to help students achieve the learning objective.

Erturk (2015) proposed three specific teaching approaches that can be incorporated into a lesson plan to implement role-playing: catering for learner needs, active learning approaches (learning by doing), and feedback to learners. Learners gain feedback from their instructors and provide feedback to one another – prompt and meaningful feedback is crucial to improving learning.

Role-playing requires careful preparation, monitoring, and evaluation to achieve the learning goals. Cherif et al. (1998) suggested four stages of role-playing, each of which can also be applied to online role-playing: preparation for the activity by the instructor; classroom preparation for the students; the role-playing; and after the enactment.

2.2 The CoI framework

Online collaborative learning is an active learning method in which students solve problems collaboratively in an online learning environment; students can define and formulate problems independently and brainstorm, negotiate, answer questions, explain and debate to construct knowledge. Garrison and Vaughan (2008) emphasised the importance of a theoretical framework in adopting a particular technique, such as online collaborative learning, to prevent distortion due to the gap between theory and practice.

The CoI is a coherent framework that provides both a means to shape practice and reflect upon or evaluate outcomes (Garrison, 2011; Garrison & Vaughan, 2008). Garrison and Arbaugh (2007) stated that the CoI framework has proved helpful in guiding research and practice in online higher education. Castellanos-Reyes (2020) highlighted that the CoI framework is one of the most extensively used frameworks in online teaching and learning by researchers across the globe.

The ability to build positive learning environments is reflected in SP. Garrison et al. (1999) categorise SP into three subcomponents: emotional (affective) expression, such as personal expressions and values; open communication; and group cohesion. TP also consists of three subcomponents: instructional design and organisation, facilitating discourse, and direct instruction. The heart of discourse, CP, is operational critical thinking comprised of a triggering event (problem identification for further inquiry), exploration, integration (synthesising and making meaning from ideas formed during exploration), and a resolution (defend the solution or apply the new skills and knowledge learned into a different context) (Garrison et al., 1999).

Alavi and Taghizadeh (2013) detailed five reasons the CoI framework was selected in their technology-assisted collaborative learning research: The CoI framework explains the process of deep and meaningful collaborative learning experiences; it guides researchers to conceptualise complex interactions between participants in online learning; it outlines the behaviours and processes needed in knowledge construction in the asynchronous learning environment; it models online collaborative learning processes; it explains online learning experiences that continue to be developed and studied. Fiock (2020) stated that the CoI framework is one of the most widely used frameworks for building communities online.

Junus et al. (2019) applied cognitive apprenticeship to teaching students the CoI framework, integrated within a Linear Algebra course. The study revealed that students equipped with the CoI framework experienced increased levels of communication skills, self-regulation, co-regulation, and learning strategies. Research on how to teach the CoI framework as an area of learning within different subject matters is currently limited. In response to this research gap, this current study examines the application of role-playing as a method to teach the CoI framework. The CoI framework is one of the CAI course areas, intending to equip students with the necessary skills to interact in online collaborative learning by implementing the CoI framework.

Research on equipping the CoI framework to improve learning readiness and critical thinking skills have been done before, such as in Boris and Hall (2005) and Santoso (2014). The methods used in these studies were relatively similar, explaining to students about the framework at the beginning of a course. Students were expected to apply their knowledge to solve problems in an online collaborative learning setting. The results showed an improvement in participants' critical thinking (Boris & Hall, 2005). On the other hand, however, the study conducted by Santoso (2014) showed that dissemination of the CoI framework was not proven to improve students' critical thinking skills significantly.

Forsythe (2020) created the community of inquiry based on the CoI Model as a course topic. One of the objectives is that the participants should interact with other class members to build their learning community. The course looks at the community of inquiry and the CoI framework from the perspectives of learners and teachers. It is expected that the knowledge and skills in future practices. The course activities take place in an online discussion forum combined with reflection.

3 Context of the study

3.1 CAI class description

The CAI course is an elective course offered to third-year Computer Science students at Universitas Indonesia. This three-credit course covers the following topics: a historical review of learning and technology; learning: from speculation to science; instructional fundamentals; pedagogy concepts and online pedagogy; objectivism learning theory and behaviourism; cognitivism; constructivism; metacognition and self-regulated learning; critical thinking; online collaborative learning theory; the CoI framework; multimedia learning; course design for online learning; authoring tools and learning management systems; contemporary issues.

The CAI class was delivered in a blended learning model. The online interaction was facilitated through a learning management system called the Student-Centered e-Learning Environment (Hasibuan & Santoso, 2005). Asynchronous online discussion is incorporated as part of class instruction, designed to actively engage students in interactions with their lecturers and fellow students. The evaluation consists of weekly reflections, mid-term tests, and a final group project. In the final project, groups of five students were asked to propose a design for online training, including the online course design and prototype development; they were expected to apply the theories they had learned, including the CoI framework.

Based on the literature review, role-playing has been recognized as one of the active learning techniques. This section elaborates on the advantages of role-playing and its procedures in an online learning class context. The advantages of applying role-playing in CAI class are the following.

  1. 1.

    Role-playing is relevant to the topic and learning objectives, and the topic can be packaged as a problem to be discussed using the role-playing method. Students can immediately apply new knowledge of the CoI framework, which also serves as a guide for meaningful interaction in an online collaborative learning environment. Students practice their skills in meaningful discussion behaviours, under the CoI framework, in real situations; thus, making it easier for them to apply knowledge about the CoI framework in collaborative learning.

  2. 2.

    Role-playing in online discussion forums provides equal opportunities for all students to participate actively – it provides opportunities for interactions between individuals in the group.

  3. 3.

    Role-playing is a new experience for students – unusual learning activities ignite enthusiasm. Students are able to experience the problem from the perspective of those in the community they are playing in, which differs from their daily roles (except for the student group). In partaking in these various roles, students become more enthusiastic in carrying out their roles, so student involvement is high.

  4. 4.

    Students are provided the opportunity to express their feelings and learning experiences to the faculty, particularly those that apply online learning. For example, students have experienced difficulties related to the learning changes caused by the COVID-19 Pandemic; they are able to present these experience under their role (such as a parent relaying their child's experience or a member of the graduate user team communicating the difficulties they experienced as a student), rather than as the student they are. Students internalise the experience from the perspective of the group context being portrayed.

  5. 5.

    Role-playing in an asynchronous online discussion environment accommodates introverted students and students hampered by face-to-face discussions that demand spontaneity and quick-thinking skills. Online discussions that utilise the role-playing method provide students with the opportunity to think and prepare for their contributions in advance.

  6. 6.

    Role-playing facilitates students in expressing their thoughts and feelings freely. The group discussion's atmosphere of discussion within the community context is not the same situation as that of a group discussion during classroom learning. In the role-playing context, participants create a discussion environment situation and adjust; this allows students to build a comfortable and trusting environment where they contribute towards attaining their common goals. Ahmadaliev et al. (2018) emphasised the importance of such activities driven by the online community.

  7. 7.

    Role-playing is followed by a discussion analysing the learning process and achievements. Thus, students are trained to observe and analyse situations in the learning process they and others have navigated.

3.2 Online role-playing and its procedures

The following procedure was applied to conduct role-playing as our learning technique. The procedure consisted of several steps: Preparation, Orientation, Implementation, and Internalisation. The steps in conducting online role-playing are shown in Table 1. The orientation before the role-playing included two aspects: a brief introduction to the CoI framework to guide the role-playing (online discussion) process and the role-playing process. A familiar problem triggered the role-playing, namely implementing collaborative online learning in an institution where they are pretended to be the stakeholders. Students did not discuss the CoI framework, but they were encouraged to remind other members to apply the components of the CoI framework as a part of TP. An in-depth discussion about the CoI framework was conducted after the role-playing.

  1. 1.

    Preparation for the activity by the instructor

    Students are evenly divided into six groups. Each group is given one of the following six roles: students, lecturers, academic secretariat and information technology (IT) division, parents of the students, association of graduate users in the field of IT, and, lastly, faculty management.

  2. 2.

    Task orientation

    Substance orientation begins with an introduction using online discussion forums:

    “The topic we cover is Community of Inquiry framework. For the discussion to be meaningful and sustainable and support learning outcomes, participants need to adopt and apply a social presence, cognitive presence, and teaching presence. To understand more deeply, you are required to read the papers that have been uploaded on the online class page. The learning strategy that we use is role-playing in the forms of online discussion. Each group has a role to play in an online discussion forum. A discussion trigger will be given to each group. During the discussion, the students are expected to apply the CoI framework to guide the discussion. Enjoy the process!”

    The substance orientation was followed by a brief explanation of the CoI framework and provided a reading assignment consisting of articles on the CoI framework. The orientation of the role-playing process was performed by explaining the formation of the groups, the roles of the individuals within a group, and the task of each group. In online role-playing, the explanation (orientation) must be explicit. More importantly, teaching the CoI framework requires modeling by lecturers. The lecturers demonstrate examples of implementing cognitive, teaching, and social presences in their online interactions with the students.

  3. 3.

    The role-playing

    The following role-playing task initiates each focus group discussion:

    “Our faculty plans to implement online collaborative learning (OCL) in every class next semester. As you know, so far, only a few classes have implemented OCL. This learning strategy is relatively new for some students and lecturers. The University Chancellor asks your group to give recommendations regarding the plan. “

    The reasons for choosing the issue of online collaborative learning are as follows: it is a relatively new strategy for some lecturers and students at universities, students have individual and existing experiences of online discussion, it can be viewed from a range of perspectives, and it can be designed to resemble real-life situations of students closely. Moreover, the CoI is a framework designed to shape the community in an online learning environment.

  4. 4.

    Internalisation

    This phase is achieved through discussions that provide students with the opportunity to articulate their experiences. Lecturers support this by providing feedback and facilitating discussion, and students also receive feedback from their peers. The steps in conducting online role-playing are shown in Table 1.

Table 1 Steps in conducting online role-playing
Full size table

In terms of the degree of response specificity, participants were free to express their ideas; they were expected to participate freely and spontaneously. Based on the role being played, those playing the role of students played their own real-life role (self-role) while the group's remaining members played other roles (Table 2). It is generally easier to play a real-life role than someone else's. The more familiar students are with the task set, the easier they find it to play.

Table 2 Roles played
Full size table

4 The method

A mixed-methods approach was utilised to analyse the qualitative and quantitative data (Buchholtz, 2019; Nieswandt & McEneaney, 2009; Ponce & Pagán-Maldonado, 2015). A coding template that describes indicators of the components of the CoI framework was used to analyse the discussion transcripts.

4.1 Participants

Participants in this study were CAI students offered by the Faculty of Computer Science at Universitas Indonesia. Before the learning, students were free to decide whether to participate in the study. No consequences or benefits were applied to the choices regarding participation. The enactment in role-playing was voluntary and occurred naturally, without instructor direction. The discussion element was conducted in a free and spontaneous manner. During the role-playing, instructor intervention was limited.

4.2 Data collection

There are two types of data collected for this study: qualitative data and quantitative data. Qualitative data are obtained from discussion transcripts, reflections, and questionnaires (open-ended questions) of the students. We extracted the transcripts in role-playing from online discussion forums in the 2nd semester of 2019/2020. We used thematic content analysis to measure elements of CoI and the characteristics of the role-playing. At the end of the semester, we asked the students to reflect on their learning experience in the role-playing session. Also, we asked them to give suggestions, criticisms, ideas, and comments. These qualitative data are used to measure the students’ opinions about the role-playing implementation.

We also used quantitative data in this study. We extracted the data from online discussion forums in the CAI classes (2nd semester 2018/2019, 1st semester 2019/2020, and 2nd semester 2019/2020). The data were grouped based on roles: for each group, the number of messages and the average number of sentences per posting were counted. These numbers indicate the level of involvement (passive-active dimension). Moreover, we also collected the questionnaires from students at the end of the 2nd semester 2019/2020 to measure their students’ learning experience and satisfaction. The description of data used in this study is shown in Table 3 below.

Table 3 Data description
Full size table

4.3 Data analysis

Information extracted from the data collected over the three consecutive semesters is as follows: the level and pattern of involvement in each group, the number of students posting, the average number of sentences for each message in each group, and the average number of sentences each group. This information is intended to describe the level of activity for each group across each semester. The qualitative data (discussion transcripts) was taken from the CAI class of the Second Semester 2019/2020. The data were analysed using the quantitative content analysis method (Rourke & Anderson, 2004), and categorisation, based on themes, was used to investigate the pattern of the presence of the components of the CoI framework.

The coding template that describes the indicators of the components of the CoI, developed by Garrison et al. (1999), was utilised. This data analysis aimed to answer RQ1 (What characteristics of role-playing did the students exhibit?) and RQ2 (What are the patterns of social presence, cognitive presence, and teaching presence for each group?). The quantitative content analysis adhered to the following steps: sampling, determination of the unit analysis, discussion of coding procedure between two coders and coding protocol exercised, coders working independently, comparison of coding results, coding decision, and analysis.

Three hundred and six (306) messages were taken as the sample. The unit analysis in this study is a message. The sentence, paragraph, and theme are not suitable as the unit analysis, as some messages are long, and they are not presented in paragraph format. The samples were coded by two coders – lecturers for the CAI course – that have implemented the CoI framework for the past three years (they are also educational researchers and authors).

Messages were coded based on the existence of the SP, TP, and CP. Following this, the CPs were further coded into their sub-components: triggering event, exploration, integration, and resolution. Before coding independently, the coders determined the coding procedure and exercised the coding protocol. For each unit analysis, the coders sought the indicator of the CoI components: SP, TP, and CP. If at least one indicator appeared for each CoI component, they labelled the message with a ‘1’; otherwise, they labelled it ‘0’. One message may contain SP, TP, and CP simultaneously. The coding results of the two coders were compared and calculated using Fleish's Kappa to determine interrater reliability. If disagreement occurred, the coding decision was discussed and agreed upon by both coders.

The thematic content analysis was intended to identify the following indicators: (1) how individuals played their role, for example, how they identified themselves as a member of their group (2) whether they conveyed ideas, obstacles in the OCL or strengths in the OCL, according to the perspective of the role they played; (3) whether they related to the situation encountered during the discussion, such as the COVID-19 pandemic; (4) whether they explicitly mentioned the CoI framework or its components; and (5) how students identified themselves based on the characters being played.

In addition, to answer the RQ 3 (How are the students’ perceptions of the role-playing learning experience and their learning satisfaction?), collected data from the reflection of learning and lecturer evaluation questionnaires were analyzed using descriptive statistics and content analysis.

5 Findings

The findings are presented to answer three research questions: (RQ1) What characteristics of role-playing did the students exhibit?; and (RQ2) What are the patterns of social presence, cognitive presence, and teaching presence for each group? (RQ3) How students’ perceptions of the role-playing learning experience and how their learning satisfaction?

5.1 Students involvement in role-playing

Before answering the research questions, we used quantitative analysis to investigate the involvement of students in asynchronous role-playing based on their online discussion transcripts. The involvement of the students in online discussion shows in Table 4. Students of the second semester 2018/2019 class had the smallest average number of posts (2.2). The other classes had similar averages of 5.87 and 5.67. The average number of posts for a group varied, as shown by the last column of Table 4.

Table 4 The number of students’ postings in role-playing discussions
Full size table

The duration of discussions ranges from five to seven days. The numbers of group members are between nine and ten students. The class of the first semester 2019/2020 had the highest average number of posts and the average number of posts per student. The comparison of the number of postings is shown in Table 4. The difference in the discussion duration and the number of group members did not correlate with the average number of posts.

The academic secretariat and IT Division has the lowest frequency of posts (the least active). Its role is non-academic and indirectly relates to the implementation of OCL. Students found identifying the tasks of the supporting staff roles difficult and providing recommendations to faculty leaders. Moreover, this group was the last group formed. Students who had not previously been placed in any of the other groups joined this one. There was no dominant member in this group.

Next, the number of sentences in a message was investigated. The number of sentences in a message generally indicated the number of ideas conveyed. Moreover, they could also reflect the fluency with which students conveyed their ideas. Table 5 below presents the average message length for each group.

Table 5 The average number of sentences per message in role-playing discussions
Full size table

It is generally easier to play one’s own role than acting in the role of the others; however, the data showed that the student group was neither the most active nor did it have the highest average message length. The existence of dominant members in a group may contribute to the level of activity of the group. The faculty management group of the First Semester 2018/2019 had two dominant members – the dominant members are the most active and the most frequently referred to or mentioned in discussions. The other groups had, at most, one dominant member. This description goes some way to answering the research question regarding the level of student involvement in role-playing activities.

5.2 The characteristics of the role-playing

Answering the research question RQ1: “What characteristics of role-playing did the students exhibit?” we used thematic content analysis based on the discussion transcripts. The discussion transcript was investigated in greater depth, based on the following themes: (1) how individuals played their role and whether they identified themselves as a member of the group; (2) whether they conveyed ideas from the perspective of their role, conveyed obstacles in OCL and conveyed OCL strengths; (3) whether they related to the situations encountered during the discussion – the COVID-19 pandemic; and (4) whether they explicitly mentioned the CoI framework or its components. The results are presented in Table 6.

Table 6 Thematic content of the discussion transcript
Full size table

Based on our thematic content analysis result, these are the characteristics of role-playing the students’ exhibit to answer RQ1:

  • When participating in group discussions, students related to the current situation and their learning experiences based on the perspective of the roles they played.

  • Very few students explicitly mentioned the CoI framework or its components. No students reminded their group of the learning objective: to improve their skills by implementing the CoI framework in online discussion. All groups placed greater focus on the challenges of implementing online collaborative learning.

  • At the end of the discussion period, all groups produced sets of agreed ideas and concepts, but these were not compiled in the form of recommendation documents. However, the two groups summarised the results of their discussions. Only one group provided an in-depth rationale behind their recommendations and the arguments to support the recommendation (resolution).

  • Students raised the subject of the COVID-19 pandemic concerning the conditions associated with the application of distance learning. Students conveyed experiences as online learners (particularly those experiences that were less enjoyable). The experiences were expressed according to their role. For instance, parents of students shared their child’s difficulties in learning in the current pandemic situation. Students presented their real experiences from the perspective of their role-play group.

  • Students conveyed expectations of distance learning (from a student perspective). The expectations were expressed in the discussions as those of the role-play group. Moreover, they explained the situation that triggered the expectation. In the joint discussion and reflection session (following the role-playing activity), students were asked to share their experiences of implementing CoI. All students applied all of the CoI components. However, only a small number of students stated that they had consciously implemented the CoI framework explicitly.

5.3 The pattern of the social, cognitive, and teaching presences

The pattern of presence of the CoI component between groups in 2019/2020 Even Semester class is the same. It is assumed that this pattern also applies in the other two classes. Therefore, the encoded and analysed transcripts only data from one semester, class of the second semester 2019/2020. A quantitative content analysis was carried out to answer the RQ2, SP, TP, and CP patterns in role-playing activities. The sample of discussion transcripts, consisting of 306 messages, was separately coded by two independent coders before comparison. We used Cohen’s kappa to measure the level of agreement between the coders (Cohen, 1960). The result is 0.9, and it indicates that the agreement between the coders was high. If disagreement occurred, the coding decision was discussed and agreed upon by both coders. The final decision is presented in Table 7.

Table 7 The frequencies of SP, TP, CP
Full size table

SP had the highest frequency, demonstrating that students presented themselves socially in the form of greetings, name mentions and maintained group cohesion. CP describes how students solved problems collaboratively. The coding results, based on the sub-indicators of CP, are shown in Table 8 and Fig. 1.

Table 8 The frequencies of the sub indicators of CP
Full size table
Fig. 1
figure 1

The frequencies of the sub indicators of CP (class of second semester 2019/2020)

Full size image

The most frequently exhibited sub-indicator was that of exploration (Fig. 1). Exploration appeared in the form of brainstorming and agreement without argument. Resolution proved the most difficult to attain. Meyer (2003) explained that integration and resolution are more demanding compared to exploration. To attain integration and resolution, students need to reflect on their experiences.

Based on our quantitative analysis, we found that the patterns of social presence, cognitive presence, and teaching presence for each group, answering RQ2, are:

  • Four groups showed a similar pattern in the presentation of particular indicators of CP: they conducted fast triggering events, extremely intensive exploration, followed by less intensive integration. Two groups demonstrated a higher intensity of integration compared to exploration. The group of lecturers displayed a process of fast triggering events and exploration before proceeding to integration and synthesising of ideas.

  • All groups tended to interpret problems rapidly and carried out considerable analysis and integration of ideas. Whether agreeing or disagreeing during discussions, students often provided arguments to support their thinking.

  • The most challenging indicator to achieve is that of the resolution, as it requires the ability to apply solutions to different contexts or defend proposed solutions. As a generalisation, groups only proposed solutions in the form of recommendations and brief background information. Only one group achieved resolution; this group proposed a detailed background-supported solution, complemented by the impact of implementing the proposed recommendations and the consequences should they not be implemented.

  • The explicit mention of CoI or its components is limited, occurring in less than five messages, and only two groups mentioned it: lecturers and academic secretariat, and IT Division. A lecturer discussed the use of CP in solving specific problems. An IT Division member argued the application of the CoI framework to improve the quality of communication in the faculty environment.

  • Although it does not explicitly discuss the CoI framework, the application of SP, TP, and CP were performed by all groups; however, the essential sub-indicators of these presences have not all been applied. For example, students still lack TP in terms of reminding each other of the learning objectives, assessing the discussion progress, and summarising the discussion results. The dominant sub-indicator of TP is the directing of the discussion through asking questions.

5.4 Students’ learning experience and satisfaction

The third research question explores the learning experience and student satisfaction. We analysed the data from the reflection of learning and lecturer evaluation questionnaires to answer the question.

5.4.1 Reflection

Students were asked to reflect on their learning experience, in a discussion board, after completing the role-playing session. 22 out of 63 students were voluntarily conveying their experiences. Based on the main ideas conveyed in the reflection, they can be grouped into three themes: obstacles faced, opinions about the application of the CoI framework, and opinions on the role-playing implementation.

5.4.2 Obstacles

Two students complained about the nested and complicated discussion board feature when the discussion went fast and long. Two students admitted that they were confused about how to play a role at the beginning because this was their first experience. One student was confused about motivating a group of friends to be active and direct the discussion without appearing dominant.

5.4.3 Students’ opinion

Thirteen students gave their opinions on the implementation of the CoI framework. All opinions are positive. Students feel the difference with their previous discussion experiences when they are not familiar with the CoI framework. It makes the discussion ‘livelier’ and more focused. Participants can avoid boredom by social presence.

Furthermore, it helps students map how they should appear in the forum. Hence, students realize what to do in the discussion and are more motivated to monitor their contribution to the discussion. Eight students gave positive comments on the learning process in general: it was fun to learn with a new perspective and a broad perspective to appreciate the learning process.

5.4.4 Lecturer evaluation by students

Lecturer Evaluation by Students is a questionnaire provided by the university to determine student perceptions of the implementation of learning at the end of the semester. The questionnaire consists of two parts. The first part asked about the students' perception of Learning Content, Teaching–Learning Process, Class Management, and Learning Evaluation with 6-Likert Scale items (1-Strongly Disagree, 2-Disagree, 3-Somewhat Disagree, 4-Somewhat Agree, 5-Agree, and 6-Strongly Agree).

All 63 students enrolled in the CAI class of 2nd Semester 2019/2020 voluntarily answered the first part of the questionnaire. The overall average score in Part 1 is 5.65 shows that students are satisfied with the learning undertaken during the semester. Students scored well on every aspect: Learning Content, Teaching–Learning Process, Class Management, and Learning Evaluation with an average score of 5.7, 5.65, 5.59, and 5.66, respectively. Table 9 shows the results of the first part of the questionnaire about student perceptions in learning implementation.

Table 9 Lecturer evaluation by students: learning implementation
Full size table

The second part is an open-ended question to provide suggestions and criticism. The results can be seen in Table 10. In contrast to the Likert-Scale items, which all students answered, only 14 students wrote their opinions in the second part of the questionnaire. When the first part (quantitative) and the second part (qualitative) of questionnaires are compared, it can be concluded that most students' comments are good and even excellent. In addition, the quantitative components which are reflected by the qualitative data, are part of the Teaching–Learning Process and Class Management.

Table 10 Lecturer evaluation by students: suggestions, criticisms, ideas, and comments
Full size table

If the students' reflections and their assessments on the Lecturer Evaluation by students, it can be concluded that both are consistent. Students had good learning experiences. In particular, reflection is related to the following components.

  • Learning Content: Lecturers encourage students to participate in the learning process actively.

  • Teaching–Learning Process: Lecturers can create a classroom atmosphere that is conducive to learning

  • Class Management: Lecturers apply the teaching contract consistently.

In conclusion, we found some interesting findings related to students' perceptions of the role-playing learning experience and their learning satisfaction. Learning with role-playing is the first experience for students. Hence, there is a small number of students feel confused for a while at the beginning. Nevertheless, they can immediately adapt after seeing how other students carry out their roles. Moreover, students have positive experiences in role-playing, seeing problems from various sides, and getting hands-on experience. It showed in the quantitative and qualitative data of Lecturer Evaluation by Students. They feel benefited by studying the CoI framework and applying it in discussions. Students think the CoI framework directs how to manage good discussions. This experience makes students more aware of their role, thus monitoring the quality of their contribution.

6 Discussion

The study revealed that that difference in the discussion duration (5 and 7 days) and the number of group members (9 or 10 students) did not correlate with the average number of students’ posting. However, the difference in student involvement in discussions might correlate with the academic load during the discussion or the difference in students’ preparedness.

Very few students explicitly mentioned the CoI framework or its components during the role-playing. No students reminded their group of the learning objective: to improve their skills by implementing the CoI framework in online discussion (teaching presence). In addition, teaching presence, which describes how students solved problems collaboratively, has the lowest frequency compared to social and cognitive presence. Students admitted that it was challenging to motivate and direct discussions. Students are not yet able to become online discussion facilitators. This is consistent with the previous studies on Linear Algebra classes (Junus et al., 2014; Junus et al., 2019). An intervention from an instructor is needed so that students apply TP or regulate the roles of group members; for example, there are leaders, facilitators, timekeepers.

In general, students had positive experiences with role-playing and thought that CoI guided them in the discussion. These acquired skills and knowledge need to be applied in online collaborative learning to other courses. These skills need to be nurtured in the following courses. In addition, students could manage online text dialogue and actively participate in reflective practice facilitated by online discussion forums and assignments.

Based on observations over the last three semesters, a set of similar challenges arise. These challenges affect the implementation of role-playing in asynchronous discussion forums. We believe that these challenges will lead to the failure of building the role-playing activities. For example, active discussion in role-playing activities usually takes time. In the beginning, some group members are unaware of the learning goals and advantages of activities. Therefore, they are oblivious to the fact that it has a significant effect on their learning. The lack of their engagement will affect badly in the role-playing processes. As a result, they will not achieve the learning objectives. These challenges were caused by the difficulty of familiarisation with the learning environment (Mclaughlan & Kirkpatrick, 2005).

Some strategies are recommended as anticipatory steps in designing role-playing activities. These strategies addressed teachers in implementing their role-playing activities. It emphasizes the role of teaching presence in learning discussion. These proposed strategies can be used as a guide to design role-playing activities. For example, teachers are advised to convey expectations and give provision of learning incentives explicitly. It will improve the students’ engagement in role-playing activities. Table 11 below explains these challenges and the proposed strategies.

Table 11 Challenges in the implementation phase and proposed strategies
Full size table

7 Conclusion

This current study investigates the use of online role-playing in learning the CoI framework. The objective is for students to be able to apply the CoI framework in an asynchronous online collaborative learning environment. The participants implemented all the components of the CoI framework, but improvements are needed in evaluating and directing the discussions. The groups’ patterns of SP, TP, and CP are similar; however, SP proved the easiest to perform, and students could feel the effects immediately. The levels of critical thinking shown by students did not reach the level of resolution.

Students performed their roles and focussed on conveying ideas based on the perspective of the role assigned to them. The focus group produced the output of the discussion but did not focus on the application of all the CoI framework sub-indicators. In theory, it is easier to play one’s own role than it is to act out the role of others. Despite this, however, the results indicated that those playing the role of student were neither the most active nor did they possess the longest average message length. Those roles with more issues to discuss tended to have longer message lengths. Observations discovered that the presence of a dominant member influences group activity – dominant members stimulate others to be active.

Based on the findings, increased intervention from the class lecturer is recommended, explicitly in directing groups to intentionally apply the CoI framework and raise awareness of the learning objectives. It is also recommended that students are given more preparation time before role-playing activities commence; in particular, students should be given more time to learn the CoI framework and provided with enough examples in its application.

This study has limitations that paved the way for future research topics. First, the instructors provided no treatment intervention during the role-playing. The study indicated that instructor intervention is required to help students apply TP. Second, another significant aspect for further consideration is group formation in online role-playing that utilises small group discussion forums. Future research topics include role-playing with increased intervention, strategies to improve the students’ preparedness to collaborate online, and how role-playing improves critical thinking, learner engagement.