Introduction

Collaborative tools such as forum and wiki have the potential to facilitate participants’ interactions and joint activities in web-based learning environments, letting students share their ideas and discuss solutions. In addition, such tools normally provide course instructors with facilities to monitor learners’ interactions and assess their contributions in a straightforward manner (Andresen 2009). Although learning forums have the potential to replace face-to-face interactions in a traditional classroom, the lack of appropriate supports in such collaborative learning activities may cause participants to fail in their joint tasks or spend much more time. In response to the demand for supporting scaffold, some researchers have provided some facilities to monitor, manage, and develop effective interactions (Calvani et al. 2010). But there is still a real demand for comprehensive supporting approaches which take into account the initial states of learning environment before implementing a supporting approach as well as the possible outcomes of the supported collaborative learning activity.

This paper proposes a new supporting approach from a holistic point of view and tries to determine the learning context that can accept the approach as well as the feasible outcomes of a supported activity. The introduced supporting approach aims at improving participants’ skills in collaborative activities, rather than forcing them to move on a straight line. It emphasizes students’ gradual progress during an activity which is monitored and guided by instructors, and leads to a higher level of activity efficiency and effectiveness. In addition, considering the learning context which can accept a new approach, this paper proposes criteria to formulate learners’ and instructors’ readiness for online collaborative learning activities. In the meantime, it addresses learners’ prior experiences and points of view toward computer-supported collaborative learning (CSCL). Besides, taking into account the possible outcomes of a supported collaborative learning activity, the present paper tries to map learning goals to the activity functionalities. The functionalities determine the purposes of instructors when they define a new learning forum, no matter the forum is expected to be an area for discussing new ideas, a shared space for recording group outcomes, or just a means of challenging less active students.

A multiple-case study is carried out to investigate the relationships between the context and the functionalities, and to find the factors which highlight a functionality, in an online and a blended learning environment which utilize learning forums for different learning purposes. Through student–instructor interactions during some academic semesters, this study aimed at exploring the relationships between participants’ readiness measured by a number of proposed criteria and the final efficiency achieved by a learning forum. It also evaluated the hypothesis that the efficiency of a learning forum depends on the quality of supporting activities taken by course instructors during the forum lifetime. The outcomes have revealed some significant relationships between the three mentioned parts, and can be referred to as a preliminary validation of the proposed approach. This paper is organized as follows. Following a quick look at some similar research in “Background” section, “Research goals” section introduces the model. It then presents the research methodology in “Proposed holistic supporting approach” section, explaining the statistical and analytical evaluations. “Research hypotheses” section discusses the results, limitations, and assumptions. “Supporting mechanism” section is devoted to the conclusion that also covers some areas for future research.

Background

Recently, almost all learning management systems have utilized collaborative tools such as learning forums to set up and manage learners’ joint activities. Since forums are regarded as the most popular CSCL activities, some researchers investigated the role played by them in knowledge construction and participants’ communications, their limitations, success factors, and outcomes. There is also a considerable amount of research which either tries to extend the capabilities of learning forums, or investigates their inherent constraint. For example, participants are recommended to interact through assertion and conflict expressions to gain more opportunities to generate new ideas (Eryilmaz et al. 2013). As another example, representation of participants’ interaction as a semantic link network on discussion transcripts is proved to facilitate active collaborations (Li et al. 2009). There are also some other research investigating forum participants’ behaviors; for example it has been suggested that successful and less successful learners take part in forum discussions differently in terms of relevance, width and depth of discussion, justification, and reasoning (Noroozi et al. 2011). Having evaluated the participants’ contributions, Häkkinen (2013) proposed two concepts of the level of discussion, and the reciprocity between participants. Such concepts were utilized in a multilevel method to capture the dependencies on individual, group and, at times, classroom levels.

Although collaborative learning tools have provided more opportunities to realize social constructive learning theories (Stammberger 2010) , they are recommended to be utilized in a suitable context, be configured appropriately, and be managed along the time and (Abel et al. 2010), so that to replace the face-to-face interaction of the traditional classroom. Essentially, enabling effective collaborative learning activities presupposes the preparation of useful collaboration mechanisms, grouping of learners in a way they can cooperate effectively, assigning of appropriate formal and informal roles for all members, and managing inter-member interactions (Anjorin et al. 2011; Abnar et al. 2012; Temdee et al. 2006; Lin et al. 2010). In addition, students, and forum managers, who are mostly course instructors in typical learning environments, are required to participate effectively in forum discussions to realize forum learning goals. It has been revealed that if group members look at the issues through the same perspectives, they can co-construct a shared world (Stahl et al. 2011). However, in some situations, a learning forum may face problems with achieving expected functionalities. Some forum content analyses have revealed that a large amount of students’ participations lacked adequacy and relevance, meaning that despite spending considerable amount of time and energy, students contributed to learning discussions inefficiently (Orooji and Taghiyareh 2012). The research has concluded that it would be much efficient if instead of just a few simple recommendations, there were some facilities to guide and manage participants’ interactions and contributions during collaborative learning activities. In response to this quest, some researchers designed approaches to help teachers monitor and assess collaborative learning processes at the group level. As an example, a knowledge map was utilized to analyze the degree of process and outcome convergence in order to provide insights into the quality of collaborative learning processes (Zheng et al. 2014). Indeed, some researchers have proposed some system-enabled and teacher-enabled approaches to provide supporting scaffolds in order to monitor and evaluate participants’ interactions, and model and organize the contents of their contributions. As an instance of a system-enabled support, learning forums were equipped with management scaffolds instruments to provide the possibility of monitoring/supporting effective interactions (Calvani et al. 2010). In this category, Karakostas and Demetriadis introduced adaptive forms of domain-specific support to help peers improve their domain knowledge (2011). However, as an organizing supporting approach, collaborative learners had the facility to utilize some prepared social and epistemic scripts, which let them develop richer iterative patterns of interaction and make discussions highly participative (Mahardale and Lee 2013). Some researchers combined sentence opener scripts with diagramming tools to visualize argument structures and promote productive forms of peer discussion. This approach guided students through a process of analyzing, interrelating and evaluating opposing positions, bringing about a positive influence on the discussion quality and students’ perception of their learning (Scheuer et al. 2014). Furthermore, as a modeling supporting approach in the category of teacher-enabled support, some patterns of teacher assistances were identified based on four dimensions of aspects, moment, receipt, and root. These patterns help teachers adapt their scripts to the students and their situations (Onrubia and Engel 2012). In this category, there are some key factors of effectiveness, recommended to be considered in addition to the structure and the model of teachers’ support scripts. Some of these factors such as teachers’ beliefs play significant roles in students’ collaborative activities and lead to different practices and patterns of technology-enhanced orchestration (Song and Looi 2012). Additionally, some modern educational softwares such as tutoring agents were employed to detect the pedagogically relevant patterns and provide feedbacks and hints in return. The patterns, tutorial actions, and tutorial strategies of the agents were parameterized to enable researchers and practitioners to create tutorial support across a wide range of applications (Scheuer and McLaren 2013).

On the other hand, it has been revealed that learning situation in general and participants’ readiness in particular, leave a significant impact on their acceptance of collaborative learning approaches. In an earlier research, five attitudes consisting of system functions, system satisfaction, collaborative activities, learners’ characteristics, and system acceptance were proposed to measure learners’ attitudes toward CSCL systems (Liawa et al. 2008). However, a newer research expressed that students’ readiness for online learning programs were mostly based on being self-directed, motivated for learning, and self-efficient in online communication (Hung et al. 2010). Furthermore, knowing about their peers’ expertise, professional history, and research history which are considered as tacit knowledge, students could communicate each other more efficiently and make better sense of learning discussions (Oztok 2013). It has been also indicated that teachers’ readiness depended on their positive attitudes to new learning paradigm, and the availability of being supported by school principals and training providers (Koo 2008). Since this paper stressed the consideration of collaborative learning approaches from a functional point of view, it was necessary to survey the research investigating the possible outcomes of learning activities, categorized as learning goals in different taxonomies. Generally, learning goals determine the objectives of learning and teaching activities, and they will be more applicable and differentiable for different students if they are specific and are declared at various levels of difficulty. There is a categorization which considers four levels of difficulty for cognitive learning goals consisting of retrieval, comprehension, analysis, and knowledge utilization, on one hand, and non-cognitive cooperative goals including motivation, affection, behavior, self-concept, and social skills, which help students accomplish their academic goals on the other (Marzano 2009). As one of the most popular definitions of learning goals, Bloom’s Taxonomy also is characterized by six similar layers of cognitive domain (knowing/head) (Bloom 1956; Anderson et al. 2000). However, in order to motivate educators to focus on all domains and create a more holistic form of education, it also defined two more domains: affective domain (feeling/heart) (Krathwohl et al. 1973), and psychomotor domain (doing/hands) (Simpson 1972).

Research goals

To position this paper against the mentioned similar studies, it can be claimed that this paper investigates supporting approaches from a holistic point of view; however, each one of the mentioned papers studied just one part. As Table 1 indicates, this paper incorporates some criteria into its proposed supporting approach, to measure participants’ readiness for the approach. The criteria proposed to asses participants’ readiness, addresses their prior experiences and points of view toward CSCL. This study introduces some functionality levels to represent the possible outcomes of the supported learning activity as well. The functionality levels proposed for learning forums describe its different possible applications in response to different cognitive and cooperative learning goals targeted by course instructors. This paper incorporates these two parts into the proposed supporting approach and investigates which learning situations enable complete approach implementation and how the approach may help a pre-targeted learning goal be realized.

Table 1 Position of this paper (marked by) against the related works
Full size table

Proposed holistic supporting approach

In the area of CSCL, the supporting approach was meant as a process to investigate the relationships between learning situations, learner–instructor interactions, and learning achievements, schematically illustrated in Fig. 1. Actually, trying to formulate the learning situations which make a CSCL effective introduced by Dillenbourg et al. (2009), this paper proposed criteria to measure Participants’ Readiness to scale the condition of learning. In addition, learning outcomes were realized through different levels of Forum Functionalities. All parts of the approach are elaborated in the following subsections. The proposed supporting approach consists of two phases: Participant Training and Participant Monitoring, both of which emphasized participants’ important need to learn the correct style of collaboration as well as the significant roles played by instructors in managing the collaborations through continuously monitoring learners’ activities and giving informative feedbacks.

Fig. 1
figure 1

Schematic representation of proposed holistic supporting approach for collaborative learning which depicts the supporting mechanism relationships with learning condition (Participant Readiness) and outcomes (Forum Functionalities)

Full size image

Research hypotheses

It was hypothesized that in order to realize collaborative learning goals, each supporting collaborative learning activity needs to be issued through a holistic point of view. This hypothesis can be explained as follows:

  1. 1

    The proposed supporting approach (Participants Training and Participants Monitoring) enhances learners’ contributions and interactions.

  2. 2

    The effectiveness of the proposed supporting approach mainly depends on the condition of learning, (Participants Readiness) which represents participants’ acceptance of the supporting approach.

  3. 3

    The Approach Appliance correlates the learning outcomes of a supported CSCL (Forum Functionalities).

Supporting mechanism

The proposed supporting approach improves participants’ cognitive and communicating skills to bring about more effective and more efficient collaborative learning activities, especially learning forums. Although learning forums have become popular collaborative tools supported in almost all learning management systems, the researchers’ preliminary investigations have revealed that participants often have less knowledge and skills to utilize forums efficiently than it is expected. Some literature provided ordinary learning forums with conceptual structures which led learners to contribute to learning forums in accordance with some predefined forms of participation. Such well-supplied forums choose appropriate metadata for learners’ posts (such as labels), or provide them with some richer iterative patterns of interactions (Calvani et al. 2010; Mahardale and Lee 2013). But the present study prefers supporting approaches which improve participants’ cognitive and communicative skills through some long-term programs, because the researchers think that if learners try to improve their participations consciously, they will be able to orchestrate their collaborative learning activities occurring at various social levels, across different contexts and media. Such orchestration skill can be considered as an important ability in the future (Dillenbourg et al. 2009).

The proposed supporting mechanism focuses on training participants to be aware of the process of collaborative learning and the acceptable patterns of interactions between peers and try to provide acceptable social and scientific participations. The early version of this approach was introduced in 2012 (Orooji and Taghiyareh 2012) and the present paper explains a completed version. Some basic guidelines were developed to provide the learners with sufficient information about the nature of scientific forums and the efficient methods of participation. The guidelines summarized in Fig. 2 demand participants to select the right discussion and make self-defined posts with sufficiently informative contents. As the figure indicates, participants need to know how to open a new discussion or to continue the previously opened discussions which let forum discussions be gradually continued and completed. Participants are required to devote time to reading peer’s posts and writing meaningful replies instead of inserting new comments without any attention to the previous tasks. The criteria addressing post-structures emphasizes, that the posts need to have a well-defined structure consisting of a precisely chosen title, a comprehensive body, and some linked references or pictures in case necessary. The criteria related to post contents recommend students to come with new ideas about the subjects, continue discussion via demonstrating or analyzing a subject, summarize the discussion in a new structure, or evaluate, assert, and conflict the peers’ posts.

Fig. 2
figure 2

The participation evaluating criteria instructions of the proposed supporting mechanism

Full size image

The supporting mechanism also considers significant roles for course instructors during CSCL activities. Instructors are required to monitor students in their collaborative learning process, rate their participations, and give informative feedbacks continuously in order to guide them and let them cooperate with each other and find appropriate solutions for their problem solving activities. Grading all posts gives active learners enthusiasm to continue their contributions, and encourages less active learners to engage in learning discussions more seriously based on the theory of social comparison. The instructors were asked to give feedback in the shortest possible time, since it has been proven that immediate feedbacks help students adopt themselves to the new style of participation in a shorter period of time. The amounts of feedback may be significant at the beginning, but it is expected to decrease in the following weeks since students gradually become accustomed to the new style. In addition, if learning management systems provide some social visualization of group activities, instructors may need less time to monitor participants’ interactions and their progress along the learning process as well as evaluating activity completeness and goal achievement.

Measures

Table 2 shows some measurements for three parts of the holistic supporting approach used by the researchers to entirely evaluate the approach and investigate the relationships between the parts. As the table indicates, each criterion is scaled as low, moderate, and high levels of influence to enable the researchers to evaluate the effects of all factors as a whole, to be mentioned in the next sections.

Table 2 Proposed measures for different subsets of the holistic approach
Full size table

The subset Participants Readiness (PR) includes criteria which measure instructors, teacher assistants, and students’ readiness. These factors, mostly based on participants’ prior experiences and points of view toward collaborative learning activities, impacts the whole process of learning, participants’ communications, and forum achievements. The factors state that when the instructor of a course and his/her assistants are familiar with collaborative learning tools such as forums, students training will be completed in an acceptable number of online or offline sessions. Besides, if students have some prior experiences in working with collaborative tools such as wikis and weblogs, they are capable to be adapted with the recommended style of participation embedded in the proposed supporting approach. But if they doubt on the efficiency and necessity of collaborative learning activities, or have insufficient familiarity or experience, they may resist against utilizing CL programs. Most often, when instructors are persuaded to utilize forums in their courses, students usually resist due to the extra load which they perceive participation in forum discussions produces.

Approach Appliance (AA) criteria represent the extent to which the supporting approach was accepted formally by instructors and assistants, and the extent to which all participants tried to improve the contents of their posts. These scaling measures indicate the level of the approach utilization by instructors and their assistants and the quality of individual participations. Level of appliance is deemed Low, if trying to imply the proposed approach failed, Moderate, if teacher assistants and course participants tried to consider the approach partially, and High, if the approach was completely applied. The quality of individual participations represents the level of the approach utilization in students’ active participations and is deemed Low, if there were insufficient number of posts, Moderate, if there were sufficient number of posts but the posts were less informative than expected, and High, if there were acceptable number of informative posts.

Forum Functionalities (FF) are measured in such three levels as Motivating, Challenging, and Recording. Instructors may utilize learning forums to make learning environment more active, or may want to encourage participants to share their ideas and knowledge, or in some cases, they may want just to monitor each member’s impacts on group progress and to record group outcomes. Actually, the functionalities formulate Bloom’s learning goals achieved through students’ engagements in learning forums, since:

  • Regarding cognitive domain, they try to understand concepts, apply their knowledge, and analyze peers’ discussions.

  • Regarding affective domain, they try to achieve metacognition skills such as awareness, and become more enthusiastic based on their social interactions.

  • And even regarding psychomotor domain, they try to imitate the recommended style of participation in learning forums and follow the instructions.

Research methodology

Combining some experimental and causal comparative research, the researchers exploited some qualitative and quantitative methods to preliminarily verify the proposed holistic approach. The research lasted some academic semesters in two educational centers.

Procedures and instructional design

In order to verify and complete the approach, it was applied in given online and blended formal courses in the two following official educational centers: Department of Electrical and Computer Engineering (ECE) and e-Learning Center in University of Tehran. The first one is a traditional educational center which has been utilizing a web-based learning environment as its supplementary service, besides its face-to-face training sessions, and consequently can be considered a blended learning system. The second one just provides online learning services with no support for traditional classrooms. All students read learning contents, uploaded their answers to the assignments defined by each course instructor, and participate in group activities through course forum discussions. Although the courses enjoyed different learning contents and activities, all the learners were requested to participate in web-based collaborative learning (WBCL) activities as a mandatory task. Considering the positive effects on the transfer of learning (Demirer and Sahin 2013), the instructors utilized learning forums and wiki’s to make it possible to collaborate in an online and observable approach. The descriptive analysis of all courses, which is mentioned in the next section, provides a comparable clear picture of the courses.

Participants

This study considers data collected during six semesters from fall 2010 to fall 2012 to highlight the effects of the proposed supporting approach in a sufficiently long duration of time with the ability of yielding more reliable outcomes. Table 3 shows a brief description of the courses as well as the number of forums, discussions, and posts. The average number of posts left by the students indicates that the e-learning students have participated in course discussions more actively than the blended ones. It also demonstrates that the learners’ discussions in this group are a bit longer than those of others. It seems that learning forums are utilized more seriously in the e-Learning Center, because the participants did not have the chance for face-to-face interactions.

Table 3 A brief description of investigated courses
Full size table

Instruments

The main instruments employed in this study were Moodle learning management system, Lucene open source search software, and a number of statistical tools for qualitative and quantitative analyses of collaborative activities taken by the students. Moodle platform is an open source learning management system, which let educational centers define courses, and assign different roles for the system users as managers, teachers, teacher assistants, and students with predefined authorities and responsibilities (Moodle 2009). Moodle provides acceptable facilities such as forums and wiki’s for social and constructive learning services. Both under-study centers utilized Moodle version 1.9.x as their virtual learning environment; the e-Learning Center used Moodle to manage all educational and learning activities; however, the blended center employed it just to announce course news, define assignments, announce grades, and mange learning discussions. All data are recorded in Moodle MySQL database via a large number of tables. This research focused on forums’, discussions’, and posts’ tables as the main data sets, accompanied by some administrative reports extracted from log tables. Apache Lucene is an open source search software which provides Java-based indexing and search technology, as well as spellchecking, hit highlighting, and advanced analysis/tokenization capabilities (Apache 2009). Lucene version 2.4.1 was used in this research to enable the researchers to analyze texts written by the course participants, find significant words, and evaluate the relevance of posts left in course discussions against the discussion learning goals. A detailed description of the statistical analysis instruments used in this study is given in the following subsection.

Multi-phase analyses

Figure 3 schematically illustrates different phases of qualitative and quantitative analyses conducted in this study. In order to understand the nature of the problem, propose the holistic approach, and determine the important factors of the triple parts of the approach, the researchers had exploited some exploratory methods at phase 0 before the causal cooperative studies. They interviewed some related experts, course instructors, teacher assistants, and technical supporters in charge. The first and the second phases provided some experimental design to investigate the impact of proposed supporting approach on contents of forum discussions and participants’ interactions. The third and fourth phases utilized descriptive and correlation analyses, respectively.

Fig. 3
figure 3

The research multi-phase method of analysis

Full size image

Phase 0: approach proposal

The researchers conducted an exploratory research to make the conceptual framework of the proposed holistic supporting approach. Through descriptively analyzing the learners’ posts in course 1 and interviewing the course instructors and those in charge of technical support, researchers tried to determine the aspects which needed further consideration. In order to review and complete the preliminary version of the proposed approach, some participation criteria were introduced and utilized in course 2. During some cyclic process of utilization, the researchers tried to analyze the course forums descriptively and make refinements.

Phase 1: impacts on contents of forum discussions

This phase investigated the extent to which the participants’ posts’ contents meet the forums’ objectives and are informative. An experimental design was conducted to compare contents of learners’ posts in course 1 and course 2. At the first step, some intelligent information retrieval tools provided by Lucene (Apache 2009) were utilized and improved to investigate the similarities between the posts’ texts and the forums’ introductory texts. And in the second step, since each participant often contributes to learning discussions through sharing his/her knowledge and ideas, consideration of all of the posts left has the potential to reveal his/her area of expertise. The researchers used the tools to find the most frequent terms of the posts left by each participant. The results allowed the calculation of the efficiency of each learning forum based on the ratio of the valuable extracted terms to all extracted ones. Worthy to mention is that the researchers faced numerous challenges due to the weak structure of the students’ posts and the insufficient tools for processing Farsi language. There was always a long phase of preprocessing in which the researchers cleaned the texts and omitted the stop words in order to prepare the texts for the analyzing tools.

Phase 2: impacts on participants’ social interaction

The impacts of the supporting approach on the students’ interactions and their active contributions were investigated using an experimental study in courses 3 and 4. Utilizing the statistical reports provided by Moodle learning management system (Moodle 2009), the students’ activities were categorized in two clusters: viewing actions and editing actions. The viewing activities include viewing discussion, searching forums, and forum subscribing; the editing activities include adding a new discussion or post, deleting a discussion or post, moving a discussion, and pruning or updating a post. In addition, course situations were evaluated and the value of students’ interactions in achieving learning goals was determined in light of the helps of instructors.

Phase 3: descriptive analysis of the proposed holistic approach

Qualitative analysis of courses 3–7 let the researchers investigate the impacts of the supporting approach on the level at which a learning forum can be used. Conducting a causal comparative method, they qualitatively evaluated the effectiveness of the proposed approach in the control and treatment conditions, where the students were completely or partially supported by the proposed approach. The students in the treatment condition were expected to perform significantly better than those in the control condition in sharing and discussing their new ideas as well as in recording the outcomes of their collaborative activity.

Phase 4: correlation analysis on triple parts

Investigating all course situations, the researchers exploited a correlation analysis on the participants’ readiness, the approach appliances, and the forum functionalities. The course situations in triple parts were described by some experts including the course instructors, assistants, and the responsible technical support. The dependence between the parts was measured by Pearson’s correlation coefficient obtained by dividing the covariance of the two variables by the product of their standard deviations.

Results

This section reports the impact of the proposed supporting approach on the quality and quantity of participants’ interactions and forum functionalities. The evaluation has been completed in some preliminary and complimentary steps. Firstly, it was revealed that the proposed supporting approach has the potential to enhance the quality of learning discussions. Secondly, analyzing instructors’ and learners’ interactions, the researchers found some important signs of social communications, which represent the impact of the proposed supporting approach on students’ relationships. And finally the role of different parameters on the forums’ successfulness was formulated. The evaluation results suggested that there are significant differences in collaborative learning activities between the online and blended learning centers. Actually, the instructors in the two centers applied collaborative learning tools for different purposes; however, the teacher assistants differed in their readiness to learn and make use of new managing approaches of CSCL activities.

Pre-evaluation (phase 0)

Through descriptive analysis of course 1, the researchers found that there was a difference between forum ideal and real usages. They found that the participants in course 1 had not received any participation instructions before, and had insufficient information about how to continue previously opened discussion or start the new ones. Sometimes they started a discussion in a nonrelated forum, left an uninformative post, or provided contents in an inappropriate format. These types of posts made it hard for other participants to follow discussions and get as much information as they expected. The conclusion drawn was that participants needed to learn how to participate and to be monitored and given informative feedbacks by their teachers. In response to this demand, the proposed supporting approach introduced important criteria for acceptable contributions, participants’ readiness, and the quality of collaborating learning outcomes. The preliminary use of the proposed approach in course 2 revealed its important impacts on the quality of the forum discussions. The instructor of course 2 believed in the outcomes of using collaborative learning activities in formal trainings; however, the students doubted on participating in the course forums due to some extra loads of writing in accordance with the requested organized manner. After some explanatory sessions, they gradually started to accept the mechanism, trying to discuss their ideas, share knowledge, and record groups’ outcomes, which led the forums to achieve different functionality levels of motivating, challenging, and recording.

Impact of proposed supporting approach on contents of forum discussions (phase 1)

Having analyzed the quality of the participants’ post contents, the researchers investigated the extent to which the posts met the forum objectives, and were informative, as reported in the researchers’ preliminary statements (Orooji and Taghiyareh 2012).

Considering the posts’ relevance to their pertinent forums, the results presented in Table 4 reveal the acceptable situations of the second course, which used the proposed approach in contrast with the first course in which the proposed approach was not applied. As the table indicates, 16 % out of all 5142 posts retrieved in the traditional approach are really relevant, while 59 % of relevant posts are retrieved in the proposed approach. In addition, the table indicates that the precision of the retrieval process at ten retrieved posts is 22 and 47 % in the traditional and proposed approaches, respectively. As a result, it was concluded that in the traditional approach, the main parts of the retrieved posts do not correspond to their pertinent forums. Actually, a large number of students left their posts in some unrelated forums, or contributed through less informative posts than they were expected. In fact, lack of appropriate training, which should have been provided for e-learning students, had a real impact on their cooperative learning experiences. Briefly, the outcome proves the claim that “collaborative learning results depend upon the extent to which groups actually engage in productive interactions” (Dillenbourg et al. 2009). This means that through proposed supporting approach, the researchers could make participants engage in group discussions more efficiently than the traditional approach which led to some weak, unexpected results.

Table 4 Evaluation result of the relevance of retrieved posts to the forums purposes
Full size table

The researcher considered learners’ posts as representations of their knowledge, and used information retrieval techniques to find most important and frequent terms.

Table 5 shows a brief description of some users’ extracted terms which are selected randomly and reviewed by an expert to decide which terms must be considered valuable, and which terms should be discarded due to being related to other similar topics. The efficiency column represents the percentage of valuable participation as the proportion of valuable terms to all terms. The table demonstrates that active users whose posts are long enough often use more informative terms in comparison to the others who participate in forum discussions through short posts. Meanwhile, as the last row indicates, when the length of a post exceeds the usual size (about 1–3 paragraphs), the text often contains many uninformative terms. The results validate the criteria mentioned in Fig. 2 and emphasize on the significance of the normal size of discussion posts.

Table 5 Brief description of some users’ most frequent topics
Full size table

It can be concluded that utilizing proposed mechanism gradually prepared students for more efficient participation. The mechanism leads to more organized and valuable threads of discussions rather than a great deal of small discussions which correlate significantly and cover similar concepts. In addition, discussion posts have more relevance to forum purposes, which means that forum introductory texts clearly explain their objectives. It was also realized that if instructors provide students with informative feedbacks in considerable short durations of time, the students try to actively adapt themselves to the recommended style of participation.

Impact of proposed approach on participants’ social interactions (phase 2)

It was hypothesized that the use of the proposed approach enhances learner’s social interactions. Actually, it was supposed that students’ communications in forum discussions represent their social relationships to some extent. So the researchers tried to evaluate the impacts of the proposed approach on the frequency and the pattern of students’ interactions in courses number 3 and 4, which include a large number of students and their communities, covered a considerable variety of people. Some of participants’ social actions and reactions, consisting of posting new comment, opening new discussions, and replying left posts were investigated in this phase. Starting with course number 3, the researchers faced with some significant problems in persuading the learners to follow the instructions. Since the same students participated in course number 4, they were prepared to meet the challenges of the proposed approach.

Firstly, the two similar learning forums of the mentioned courses were compared statistically. Table 6 shows the average and the variances for the number of viewing and editing activities. As the table indicates, applying the proposed approach in course 4 brought about a greater average number of editing activities; however, there is an opposite situation in the average number of viewing activities. It can be concluded that the students of course 4 contributed more actively in learning discussions, while the students of course 3 mostly preferred to read their peers’ posts instead of writing new posts. In addition, the greater variance numbers of course 4 indicates that there were a large number of students with much higher or much lower number of editing and viewing activities than the average. It can be concluded that the course had a less homogeneous situation than course 3. Furthermore, evaluating the final outcomes of the courses by the main instructor who taught both, revealed more satisfaction with course 4. He believed that as acceptable group outcomes, almost all students began to get accustomed to the new approach and tried to express their ideas, opinions, and results through the course discussions.

Table 6 Students participations in a similar learning forum in two courses
Full size table

Secondly, the distribution of participations’ editing actions was investigated, labeled as the number of editing activities by the number of students in Fig. 4. As the figure indicates, course 4 has a logarithmic curve similar to that of social networks, where a few students with considerable amounts of editing activities existed, while tens of students participated in the courses with just a few editing activities. Although the courses have much in common, the curve is approximately linear for course 3. It can be concluded that the approach appliance in course 4 yielded more acceptable social interactions than course 3 in which the approach faced important challenges.

Fig. 4
figure 4

The distributions of the students’ participation in two simultaneous courses

Full size image

Descriptive analysis of the proposed holistic approach (phase 3)

Descriptive analysis of course 3 and course 4 in e-Learning Center

Commencing with course number 3, the researchers prepared a chart to clearly explain the steps of the proposed approach. Although the course was studied in the online center, the participants had little information about collaborative learning tools such as forums, wiki’s, and chat rooms. So, the researchers arranged some face-to-face meetings to introduce the tools, explain how the students were asked to participate, and demonstrate how the instructor and his assistants could manage the tools. These sessions familiarized the participants with interesting capabilities of collaborative tools. Some of them became encouraged to use the tools to be adapted to the proposed approach, while others remained doubtful about its necessity and usefulness. The students’ experiences in course 3 led to a better acceptance of the approach and an acceptable quality of individual participations in course 4. So, in this course, the participants gradually started to adapt themselves to the supporting approach which led to making more efficient contributions. In addition, the better performance of the course forums enabled the instructor to ask his students to record their groups’ outcomes via their collaborative tools, meaning that the course reached a higher level of forum functionalities.

Analyzing the course situations and interviewing the assistants, the researchers found that the participants’ resistances mainly comes from their doubt about the necessity and the efficiency of the proposed approach and its effects on learners’ ordinary activities. Some of the assistants argued that the tools’ limitations and forcing students may cause some fake reactions. Some other students avoided the approach because they thought writing in an organized manner might bring about extra loads. These descriptions made some preliminary evaluations of the proposed models for participants’ readiness.

Descriptive analysis of course 5, course 6, and course 7 in blended learning center

Using a causal comparative method, the researchers descriptively compared the learning forums’ outcomes in bachelor courses 5 and 6, and master course 7. The courses were provided at the center of blended learning where the instructors applied collaborative tools to facilitate groups’ communications and research activities. Starting with course 5, the B.S. students attending the course were trained to utilize the learning forums efficiently. Although the participants had little prior experiences about collaborative learning tools, they accepted the proposed approach almost easily. The students followed the instructions mainly related to posts’ structures, making less effort to provide sufficiently informative contents, because they were asked to discuss their ideas arbitrarily with no grouping. Continuing with course 6 as the second bachelor course, the students were grouped based on their characteristics and were asked to utilize the course forums for their collaborative purposes. A few groups achieved acceptable outcomes, while the others provided attenuated solutions. Finally, the approach was evaluated in course 7, where the forum-aimed learning goals were clarified and the participants were requested to record all groups’ outcomes via the course forums or wiki’s. At first some of the students resisted the instructions due to probable extra burden of writing ideas in an organized manner instead of expressing them in face-to-face sessions. After some introductory session, they were encouraged to use the course discussions for learning interactions so that to be monitored and evaluated by the assistants. The students participated in group activities seriously, leaving long and sufficiently informative posts.

These descriptions made a preliminary verification for the mentioned hypothesis that the employment of the proposed approach enables learning forums to achieve higher levels of functionality. The course situations showed that if participants left informative posts and instructors played their expected roles, the forums would present active and efficient learning discussions. Actually, the more involved the participants were in the demanded activities in accordance with the recommendation of supporting approach, the more qualified the participations were, and the deeper the levels of success achieved in the collaborative activity. The researchers found that the instructor’s opinions and learning goals determined the usages of course forums, whether students could use forums in order to share new ideas or they could record group outcomes. For example, despite the fact that the students of the bachelor courses 5 and 6 almost accepted the approach, they were almost unsuccessful in co-creation of shared knowledge and interacting each other efficiently. However, the forums of course 7 were used to share new ideas, discuss the solutions, and record the outcomes, which means higher levels of forum functionalities than courses 5 and 6. This phase also enabled the researchers to make a preliminary comparison between students in different levels of education. Compared to the B.S. students, M.S. students made more valuable contributions. It was also found that the participants’ readiness and their adaptation to the approach brought about learning forums with higher levels of functionality.

Correlation analysis between triple parts (phase 4)

In addition, since the comparison needed some quantitative measurements, it was preferred to replace labels with numbers as in the next two figures, where 3 is High, 2 is Moderate, and 1 is Low. Firstly, the final results of applying the approach in the six mentioned courses were integrated in Fig. 5. This enabled the researchers to make comparisons. As the figure indicates, courses 2, 4, and 7, where the participants had acceptable preparation or prior experiences, earned the highest quality of students’ participations. On the other hand, courses 5 and 6 where the bachelor students were mainly demanded to share and discuss their ideas with less emphasis on recording outcomes had the worst quality of individual participations.

Fig. 5
figure 5

A comparison between the courses Participants Readiness (green bars) and Approach Appliance (blue bars) according to the proposed holistic approach

Full size image

Secondly, the courses forum functionalities were compared in Fig. 6. As the figure shows, course 2 and course 4 where the students are allowed to report their individual or group outcome just through the forums had the higher levels of functionalities. Again, course 5 and course 6 had the lowest level of functionalities since the courses instructor and her assistants mainly focused on setting up some challenging discussions. In the next session, the results were discussed in detail the relationships between the proposed factors and the outcomes.

Fig. 6
figure 6

A comparison between the forum functionalities of the six courses

Full size image

The participants’ readiness, the instructors’ supports, and the functionality levels of CSCL activities of the seven courses were compared by discussing the two research questions: (1) did the participants’ readiness influence the functionality level of a learning discussion? And (2) did the supporting approach influence the functionality level of learning discussions? The relationships between proposed factors and forum functionalities were investigated through calculating the Pearson’s correlation formula. Figure 7 shows the significant relationships as bidirectional arrows, where thicker arrows represent stronger relationships. As the figure shows, the forum functionalities depend directly on the approach appliance, implying that the extent to which students are encouraged to participate in course collaborative activities following the proposed approach which is directly related to the complex roles played by forums in learning environments. This result can also be regarded as a clear sign of the approach effectiveness.

Fig. 7
figure 7

Dependencies between Participants Readiness, Approach Appliances, and Forum Functionalities, weighted arrows show the strengths of Pearson’s correlations

Full size image

In addition, there is a strong relationship between the quality of individual participations and the forum functionalities. It means that if participants qualify their posts following the approach instructions, learning discussion will reach a higher layer of functionality. The figure also indicates that although participants’ prior experiences about collaborative learning tools highly influence forum functionalities, the effects of participants’ overall readiness can be neglected. It can be concluded that if learning forums are used as means of motivating, challenging, and recording, instructors, and assistants are recommended to manage learners’ participations based on the proposed approach, rather than being worried about the participants’ readiness which is normally out of control.

Assumptions and limitations

Some assumptions were considered in the present study as follows. Firstly, the present study has so far benefited from the manual assessments made by experts, learning environment is supposed to be promoted so that to enable the automatic calculation of the measures and let the participants mention their opinions through some formal questionnaires. Secondly, investigating the relationships between the proposed measures, the researchers considered the courses as a whole, due to the limitation imposed by the insufficient number of information resources. Thirdly, in this phase, the study investigated separated groups where group outcomes were invisible for the peers, and were evaluated independently, with no study done on visible groups.

Furthermore, the present research realized the limitations of Moodle open source learning management system (Moodle 2009), which was employed in both online and blended learning centers studied. These limitations enabled students to exploit learning forums for nonacademic purposes. Actually students of online center showed more social interactions like congratulations, condolences, and appreciations in their forum discussions since there was no face-to-face interaction between students. So the results might be different if the learning management system facilitated the students’ social communications through providing the possibility to rate peers’ posts, or express their ideas as a Like status which are supported in some new web-based learning environments such as Coursera (Coursera 2012) and Edmodo (Edmodo 2008).

Challenges

In this vast research, the researchers faced with some really serious challenges.

Since instructors typically spend a considerable amount of time on reading and evaluating learners’ posts based on the criteria mentioned in Fig. 2, it is important that the courses have acceptable numbers of students. Otherwise, instructors often prefer just to read students’ posts rather than interacting efficiently. In the e-Learning Center, while it was expected that the courses have 15–20 students [as Khan (2005) emphasized], the courses under review had much more students, which led to the large total number of notes students and instructors read and wrote. Since this situation was found to lead to shorter notes with lower grade scores read by a smaller proportion of students, (Qiu et al. 2012), the researchers began to divide the students into small groups. The researchers found that such a policy was less successful in the blended learning center than in the e-learning one.

The other challenge in the e-Learning Center stemmed from the teacher assistants’ points of view regarding the purposes of online learning discussions. While the emphasis was on utilizing the course forums for just learning purposes, some of the assistants mentioned that learners’ social reactions such as appreciations toward the peers’ useful posts would encourage the authors and the readers to achieve more informative participations. This made the researchers design some facilities letting the participants express their enthusiasms through some other sorts of reactions with no impacts on scientific discussions.

Furthermore, the course instructors in both centers followed traditional lecture-based approaches instead of using web-based facilities which could have realized the theory of social constructive learning theory. In addition, persuading the students in blended learning center to manage their collaborative activities through online learning discussions needed a considerable amount of attention, since they used to arrange face-to-face meetings to share new ideas and discuss the solutions with no chance to be monitored and assessed by the course instructors. Although the participants began to use online forums, the resulting side effects were significant, implying a sort of disturbance the decrease of which would take time; however, the acceptable final achievements of applying the approach encouraged the researchers to continue the study.

Conclusion and future works

A new supporting approach was proposed and evaluated from a procedural point of view, aiming at improving the quality of learning discussions. The introduced holistic approach declared that the context of learning which facilitates supporting mechanisms and the possible outcomes of a supported CSCL activity, need to be determined in addition to the supporting mechanism itself. The approach, therefore, includes some measurements for participants’ readiness, the level of utilizing the supporting approach, and the possible outcomes of the supported activity; the different parts of the holistic approach and their correlations were evaluated in two blended and online learning centers along four academic semesters, revealing some significant distinctions. The following paragraphs explain the parts and the relationships between them in detail.

Firstly, this paper introduced a new supporting mechanism in order to resolve the issues related to participants’ insufficient knowledge and experiences about CSCL activities. The mechanism highlighted the importance of training learners to participate in forum discussions effectively, including the instructions for right discussion, well-defined structure, and informative contributions. Actually, group members were encouraged to participate in forum discussions during the activity timeline and avoid postponing the communications and the research activities until the projects come to deadlines. The approach also required that teachers and assistants read all of learners’ posts, give informative feedbacks, evaluate students’ contributions based on the instructions, and assess their impacts on group progress. Preliminary evaluations revealed the valuable influences of the approach on students’ engagements in collaborative learning activities.

Secondly, this paper investigated the situations facilitating the utilization of the proposed supporting approach. Participants’ readiness was focused on as one of the most influencing factors, formulated through three introduced measures: participants’ backgrounds, experiences, and opinions. The evaluation results approved the relationships between the measures and the participants’ acceptances toward the proposed approach. It was revealed that if participants had sufficient experience in web-based collaborative activities with positive attitudes toward applying the collaborative tools, they were highly expected to follow the approach instructions.

Thirdly, this paper investigated the possible outcomes of a supported CSCL activity as the last link in the chain of a holistic supporting approach. Course instructors often use learning forums to engage students in course topics, providing them with the possibilities to discuss new ideas and record the group outcomes. These different functionalities which learning forums can realize were categorized into three levels in the present paper: motivating, challenging, and recording. Having investigated the relationships between the proposed supporting approach and the levels of functionality, the researchers found that if participants contributed to learning discussions through more informative posts, they would be able to challenge peers more deeply, share their ideas in more understandable and informative threads of discussions, and accumulatively record their group outcomes in more organized structures.

The present study will be continued in different dimensions. The proposed framework will be completed through adding some more detailed features, and extending the approach domain to contain other collaborative learning tools such as wiki’s. The researchers would like to provide some survey instruments to assess the impacts of the proposed approach in other online and blended courses. Collecting more data, they also would like to evaluate the model through performing more complex analyses like structural equation modeling with additional data. The interoperability between different collaborative learning tools will be investigated as well, because simultaneously utilizing two or more tools facilitates students’ engagements in collaborative activities. This part of the research has been started by using wiki’s along with forums, letting the instructors realize different learning purposes through assigning forums to discussions and wiki’s to group activity outcomes. It is worthy to mention that the facilitating learners’ communications and instructors’ assessment activities have recently begun by the researchers through designing and implementing a new web-based learning environment.