1 Introduction

Several researches have shown that students that work collaboratively develop better attitudes towards the learning process, dedicate more time to learning tasks, are more tolerant, listen more the opinion of the others and have better abilities to negotiate [1, 2]. Computer supported collaborative learning –CSCL– brings together the same characteristics and qualities of traditional collaborative learning, but includes a motivational element associated with technology, the computer. Besides, its uses allow carry out a more detailed follow up the process [3].

The most relevant and necessary to work cooperatively or collaboratively is to learn how to do it. It is not a matter of putting a group of people in the same place face to face and instructs them to cooperate or collaborate in the carrying out of an activity [3]. Here is where the main problem appears, many times it has been believed that having the technological infrastructure guarantees an effective collaboration. However, to achieve this proper collaboration, it is necessary to organize and analyze additional aspects, since, in order to have a collaborative activity that generates learning among its participants, it is necessary to go further than accepting and organizing class practices set, laboratories, and their tools. It is necessary to take into account a deeper approach to guarantee the cooperation among the work teams and a common and equal learning by means of the analysis of all the external factors that must be involved in this process [4].

For the reason explained above, in this work, these three aspects are taken into account in an integrated manner: the analysis of the characteristics of the groups of people, the way in which the activities should be designed, and the technology that will be used for the interaction, considering the elements defined in [5], also adding the necessary mechanisms that allow the evaluation, monitoring and improvement of the collaborative process. From this point of view, it is important to figure out what degree of collaboration has there been during the learning process of the group. For this reason, comes out the idea of evaluating, monitoring and improving the collaborative learning process, when a group of people works around in a common activity, with a collaborative learning environment, using several interaction devices for communication. Because of the need to collaborative learning process improve with the support of monitoring and evaluation mechanisms, is necessary to use tools to guide the improvement in this context.

Based on the above, it is proposed to generate a strategy to increase collaboration through the use of mechanisms that allow monitoring and evaluation of the collaborative learning process, mechanisms that can be used both by the participants and by the coordinator of the group activity and finally a greater collaboration between the participants.

This paper is organized as follows: an introduction, where the idea of the work and the research problem is presented, later a summary of the related works, presents the phases of the collaborative learning process improvement application and finally the conclusions drawn from the work.

2 Related Works

In recent years it has led to a large increase in the use of virtual environments (or mediated by technology) in different contexts of education. The literature has long recognized the role of Information and Communication Technology (ICT) as mediators and facilitators of the teaching and learning processes. Specifically, the communication processes that take place in virtual learning environments have been the subject of numerous investigations developed from different perspectives. Often the interaction analysis has been framed in the collaborative context and joint construction of knowledge among students who are part of this process. This analysis of the interaction is not framed in these situations because the social communication should systematically involve collaborative processes, but by the boom that have acquired the collaborative methodologies in training contexts, also by the interest shown, observation of interactive processes within these dynamics, especially measured by the possibilities of the use of such technology to support teaching and learning [6].

Due to the importance that has managed collaborative learning, it is necessary to define what makes reference and how to correct application for greater efficiency and effectiveness in activities that wish to develop in a collaborative environment and towards a common learning [7].

During the development of this paper is especially used the concepts of Learn and Collaborate and is necessary to have a general meaning for these, in order to contextualize the reader and facilitate their interpretation. The Royal Academy of the Spanish Language defines Learning as “acquire knowledge of something through study or experience” and Collaborate as “work with another or others persons in the accomplishment of a work” [8].

The computer supported collaborative learning is an emerging area of science learning responsible of how people can learn together with the help of computers. The inclusion of collaboration, computer-mediated and distance education has problematized the notion of learning and has led to new questions about how to study it. The concept of collaborative learning is in constant discussion. Different researchers continue to investigate what are its main distinctive features and possible differences with the cooperation because they are terms of constant confusion in understanding the collaborative activities. Some believe that there is such a clear distinction between the two concepts and their common characteristics are more important than their differences [9]. In this sense, Kreijns et al. [10], explain that both collaboration and cooperation, develop the possibility of an active learning, the teacher’s facilitator role, education, learning from shared experiences and responsibilities of students in the own learning.

Other researchers, however, point out differences between the two types of learning and are interested in defining cross-collaborative learning. For example, Stahl et al. [11], determined that since computer-assisted collaborative learning, two forms of group work are distinguished: cooperation and collaboration. At working cooperatively, students solve tasks individually and then combine the partial results to obtain a final product; while as in collaborative work, each member of the group is committed to a common task, which is built around the group, this task is performed by the group negotiation and only then knowledge is constructed collaboratively. Dillenbourg [12] similarly explains that cooperation and collaboration are differentiated by the degree of division of labor. In cooperation, partners divide and resolve individual subtasks and then link the partial results into a final result. While in collaboration, peers perform the task “together”, and although some division of activities is presented horizontally, there is a division in which the roles are continually exchanged, also Dillenbourg in [5], believes that the chances of collaborative interactions generated increase when appropriate initial conditions are set (set groups, establish appropriate problems, use the appropriate software, etc.) and when the teacher takes the role of facilitator making minimum educational interventions to redirect teamwork in a productive way. Thus, collaboration can be promoted in a complementary manner, either by structuring the collaborative process or retroactively regulating interactions.

2.1 Evaluation and Monitoring of the Collaborative Learning Process

In [13] studies have been made regarding how evaluation and monitoring the collaborative learning process should be done, where it is proposed that for this process to be effective, certain guidelines should be followed and some roles defined. But, the simple definition of these guidelines and roles does not guarantee that learning is performed in the most efficient way.

Hurtado [14], designs a collaborative learning activity for the Chemistry teaching, through a computational tool that allows the creation of work groups in classrooms in order to work in this activity.

Barros et al. [15] built a platform (DEGREE) for the analysis of the collaboration that is given in the groups from the analysis of the interactions. Allowing the study of the different stages that occur in an argumentative discussion. In addition to these, platforms have been built to assess the content of interactions as well as attitudes towards collaboration.

In relation to the content of the interactions, Martínez et al. [16] presents a way of evaluating interactions by capturing events and processing them to model the state of the interaction. Based on these studies, it can be determined that there are mechanisms that allow the evaluation and monitoring of a collaborative process, which only focuses on having people, activities and technological tools, without taking into account the improvement of the process and the analysis of external factors that affect the process.

Lovos [17] creates a customized environment that integrates teaching paradigms: Problem-Based Learning (PBL) and computer supported collaborative learning, which has a collaborative learning environment in virtual teaching situations, through tools that provide synchronous and asynchronous services that are very useful in teaching.

Based on these studies, it can be determined that there are mechanisms that allow the evaluation and monitoring of a collaborative process, which only focuses on having people, activities and technological tools, without taking into account the improvement of the process and the analysis of factors that affect the process and the form for increasing the collaboration in these activities.

3 Research Methodology

In order to increase collaboration in the Collaborative Learning Process, a methodology focused on the execution of an improvement plan was used to validate and refine the application of a set of phases applied to the collaborative learning context, allowing the creation of monitoring and evaluation mechanisms of the process.

3.1 Phase 1: Installation

The objective of this phase was to launch and detailed project planning (schedule, budget, and activities diagrams, among others).

Coordination, assignment of tasks and responsibilities were carried out for this purpose. This allowed the areas, the people that were the subject of research, the scope of the project and its objective to be delimited, as well as the improvement objectives, which were established from the needs of the collaborative learning process, in search of the increase of the collaboration through the use of mechanisms of monitoring and evaluation of this process.

3.2 Phase 2: Definition

The objective of this phase was to analyze the characteristics at the level of the group of people, activities, and technology, in order to make a diagnosis of how the collaborative learning process is currently being carried out.

Support activities were also carried out to achieve the objectives of the project, including conducting a theoretical research, defining sources of information and selecting the group on which the collaborative activity would be carried out, a list of the characteristics to evaluate in the selected group, a collaborative activity was designed, based on the work of Pre – Process, Process, and Post – Process activities carried out in [18], the first version of monitoring and evaluation mechanisms was defined for each activity of these stages, in addition to design of the case studies that were carried out, in such a way that it gets the data to be evaluated, the group on which the research was to be carried out and the purposes of those case studies were obtained.

3.3 Phase 3: Formulation

The objective of this phase was to develop conceptual models of how collaborative processes are currently performed, analyzing the activities, groups and tools, as well as obtaining the main diagnosis of the current collaborative learning process, which allowed to detect opportunities for improvement, roles, activities, steps and tools of monitoring and evaluation for the Process stage, based on the analysis carried out in the previous phase. For this, a case study was developed that was modeled through the use of SPEM 2.0 [19], modeling the processes that were diagnosed in an academic environment without the use of tools to support the monitoring and evaluation of the collaborative learning process. Another case study was carried out where a Moodle to monitor and evaluate through the guide in [18], in a Software development undergraduate course, adding in this tool the implementation of the first version of the proposal of monitoring and evaluation mechanisms, in each of the stages of the collaborative process. In this case study was obtained results that are found in the article Agredo et al. [20].

Taking into account the results of these case studies, improvement opportunities were defined, which were assigned a candidate support tool to solve said deficiency, and subsequently, these improvement opportunities were prioritized to implement those with higher priority and greater positive effect on the collaborative learning process.

3.4 Phase 4: Metrics

The objective of this phase was to define the use of a set of indicators and collaboration metrics through the improvement opportunities found, in such a way that the new process is appreciable and verifiable. The metrics that were taken from Collazos et al. [13], which served to analyze the results and to define the increase of the collaboration were: number of errors, solution to the problem, strategies use, keep the strategy, communicate the strategy, messages of strategy, strategy work messages, strategy coordination messages, total messages. In addition, five indicators were taken: four of them are based on Johnson & Johnson’s proposed activities [21]: strategy use, intra-group cooperation, performance, monitoring and review of success criteria.

3.5 Phase 5: Mechanisms

The objective of this phase was to develop mechanisms that allow the evaluation and monitoring of the collaborative process taking into account all aspects analyzed previously.

At this stage, other activities were also taken into account, among which are:

  • Analysis of existing mechanisms that serve as the base for the given definition.

  • Apply the mechanisms of opportunities for improvement.

  • Update implementation plan for the next iteration.

  • Perform and update implementation report.

In this project developed mechanisms that allowed the evaluation and monitoring of the collaborative process, determining that the stage where the work will do, would be the Process stage with each of its activities, stage where is centered all the existing collaborative moment between the students and executed the monitoring and evaluation of the process by the teacher, it is for this reason that after consulting experts on the activities propose of this phase and the suggested monitoring and evaluation mechanisms, doing a quantitative analysis of their opinions obtained the final activities defined for this Process phase, with their respective mechanisms and each of the necessary elements for their accomplishment, as they are: guides, documents, tools, inputs, outputs, among others.

Like a strategy to increase the collaboration, a definition about the monitoring and evaluation mechanisms, was initially made and a guide to its correct use was then generated to guide the students towards learning and provide the necessary collaboration among the participants of a collaborative activity, the mechanisms that were defined after the activities previously analyzed are: forums use, chats use, wikis use, electronic mail use, activities management use, groups and roles management use, evaluation management use. The definition of each of these mechanisms is found in the paper by Agredo et al. [22].

The set of these mechanisms they were: guides, selection of tools that facilitated tasks in the process of collaborative learning, documents that specified the importance of each of these tools, in addition to the recommendations for their correct use in favor of collaboration, Are the defined mechanisms that later allowed to be applied through a software tool and delivered to both teachers and students to be used in the development of a collaborative learning activity.

3.6 Phase 6: Improvement

In this phase, the improvement opportunities identified above were materialized through the creation of a model, is a proposal for a collaborative process improved model that would solve the shortcomings found in each of the activities, roles and tools that are part of the collaborative learning, model conformed to a Process phase activities model (See Fig. 1 where show the diagram) and a model a process with the application of the improvements found.

Fig. 1.
figure 1

Process diagrams of the process stage

Full size image

From these activities designed, for each one was generated a list of inputs, outputs, and roles. In addition to these activities, sub-tasks were generated with which a detailed specification was made that contained the description of the activity and their respective monitoring and evaluation mechanisms to achieve increased collaboration.

3.7 Phase 7: Test

Having the formal process with each of its specifications, field tests were developed in different classrooms with different groups of students and teachers in order to evaluate, validate and modify the conceptual infrastructure and the proposed improved model with the objective to compare the results obtained in the diagnosis and the current results.

This stage also involved the creation of a software application called MEPAC (Monitoring and Evaluation of the Collaborative Learning Process) that contained the strategy for increasing collaboration through the compendium of tools that allowed all these mechanisms to monitor and evaluate the process collaborative formalized previously, which served as support for the accomplishment of a collaborative activity in a group of people. These mechanisms defined guidelines for the teacher and the student to use them in order to generate greater collaboration between groups. MEPAC (See Fig. 2) also had a space for the design, management, and execution of the activity, for the deliveries administration and execution of the activity requested.

Fig. 2.
figure 2

MEPAC software tool

Full size image

After the improvement, a process feedback was made, evaluating the state of the academic course before and after the improvement, based on the metrics and indicators that were used to determine the collaboration between the students and thus guaranteeing the improvement of the collaborative learning process, obtaining results that were compared before the improvement vs the results obtained in the indicators and metrics after the improvement, from them it was observed that the values increased on the premise that this type of metrics and indicators measure us the collaboration improvement executed in the activities that the teacher has carried out.

3.8 Result Analysis

As results of the diagnostic case studies obtained that the preliminary mechanisms of monitoring and evaluation in each of the phases of the collaborative process adapted for the undergraduate academic field, is useful and moderately applicable. However, the steps on how to do it are not clearly defined, therefore, the teacher and the student should seek this “how” based on other methodologies or in their experience.

It can be said that the use of the formal procedure in its entirety is extensive and requires a high training by the teacher in designing collaborative activities, correct use of all proposed mechanisms, but it was possible to analyze that if the design is repeated and execution of the phases, the time recorded would be lower, due to the gaining of experience in those subjects. It can also be said that the application of the proposed formal procedure is simple; the terminology used is very close to the teaching environment. In addition, the percentage of students who approved the activities increased, consolidating the proposed mechanisms as useful to increase the good performance of students. The perception of the students was that the tools provided were a support for the development of their activities and for the increase of the collaboration. The perception on the part of the teachers was that it was possible to classify the improvement as useful taking into account the positive impact that was generated.

From the results obtained after the analysis of the metrics and indicators, it can be said that the implementation of a set of mechanisms that allow the monitoring and evaluation of the processes of collaborative learning allow achieving the increase of the collaboration, to the extent that they are used correctly according to the strategy proposed.

There are also other ways to increase this collaboration, for example strategies where the teacher can intervene through synchronous and asynchronous mechanisms that were not taken into account in this formal specification, mechanisms that can be used by the teacher and by the student at the time that the teacher thinks fit to look for the increase of the collaboration. Another strategy that can be taken into account is the customization of content, which analyzes and interprets the student’s actions and dynamically adapts content. Some common methods used, for example, to determine the level of knowledge or personal preferences, are the preliminary test or the definition of profiles. Depending on the results, the contents and functions of each student are configured to design a personalized training offer always looking for better learning depending on their own characteristics and in this way collaboration will increase. The basic requirements for this are a wide variety of modules in terms of contents and functions, and a high level of interactivity [23]. In addition to these strategies can be included the gamification, which is a learning technique that moves the mechanics of the games to the professional educational field in order to achieve better results, either to absorb some knowledge better, improve some skill, or reward concrete actions, among many other objectives. A series of mechanical and dynamic techniques extrapolated from games are used [24].

These strategies can be included as ways to increase collaboration, and many others that can be studied in future work, thus remaining a broad path to pursue in order to increase collaboration, taking into account that it is not an easy task, requiring the Analysis of many characteristics, technologies and people in charge of this work. In addition to being able to use this work as a basis to include in this formal specification and in the software tool MEPAC, new mechanisms and new strategies that achieve a cohesion of ways to increase collaboration.

4 Conclusions

To increase collaboration, it is not enough to deliver an activity and a software tool. To achieve true collaborative processes, it is necessary to structure the activities, analyze the type of people that make up the groups, the external factors that can affect the collaborative work and have a tool designed for the use of control, monitoring and evaluation mechanisms activity; so that collaboration is promoted and is not simply an individual work activity.

Having a software tool that contains a compendium of mechanisms that allow to evaluate and monitor collaborative learning by the teacher, is of great help so that it can generate better results of collaboration between the students who participate in the same, so that are of great benefit and allowing an active collaboration and a common and egalitarian learning.

From the final case studies, the values obtained in the surveys carried out to the students, teachers and the results after the execution of the improvement, it was possible to determine that the strategy to increase the collaboration enriched with monitoring and evaluation mechanisms in the Process phase is useful and that its application is not easy for an execution of a collaborative activity. Useful because it provides the necessary activities to improve the processes and is not easy to apply because, although it exposes a guide of its application, it requires a high level of effort evaluated in the number of hours necessary for its implementation.

Based on what has been done, it is possible to have as future work the inclusion of new functionalities in MEPAC tool that facilitate collaborative activities, which will allow a simpler degree of monitoring and evaluation of the process to take corrective actions of the process at time, besides performing an analysis of all external factors that influence the process of collaboration, so that they can be used in favor of it.