Introduction

Competency is a polysemous term which has been defined from different perspectives (European Commission 2008; Zabala and Arnau 2007), but the most widely accepted definition seems to be the concept of competency as a set of abilities, knowledge, procedures, techniques and attitudes a person owns and which are necessary to a) carry out professional tasks in a specific job in an efficient way; b) solve problems in an autonomous, free and creative way and c) collaborate in work organization and in the socio-labor environment. In the Final Report (stage I) of the Tuning project (González and Wagenaar 2003) “competences [sic] and skills are understood as including knowing and understanding (theoretical knowledge of an academic field, the capacity to know and understand), knowing how to act (practical and operational application of knowledge to certain situations), knowing how to be (values as an integral element of the way of perceiving and living with others and in a social context”.

Today, working environments – less stable than they used to be – require both students and graduates and future employees to adopt new strategies that will satisfy the demands of a global market in constant change. The reduction of the unemployment rate in the European population between 20 and 64 years of age is one of the strategic objectives of the EU for 2020 and one of the ways to achieve this is to provide the population with those competencies and abilities that are suitable for current and future jobs (European Commission 2014). Universities tend to focus on the development of formal competencies, whereas employers give primary value to all those related to instrumental, interpersonal and systemic skills (Riesco González 2008).

The specialized literature in English distinguishes these so-called soft skills from other competencies related to specific technical and methodological knowledge, or hard skills. Hard skills are highly valued in the academic world whereas soft skills are highly valued in the working environment. Hoffman (2003) highlighted the discrepancies between the knowledge acquired by students and businesses’ actual needs. Thus, if a close link between universities and businesses is to be promoted, higher education needs to include generic competencies –and soft skills specifically– as a learning objective and systematically evaluate them throughout the university learning process regardless of degree or specialization. This seems to be the only way of ensuring that these skills are learned by students and added to their competency expertise in a gradual and efficient manner.

Thus, soft skills refer to personal competencies that affect the way we interact with people and include communication, listening and negotiation skills, as well as team work, leadership and planning, reflection and critical thinking, ethics and commitment (Dash 2001; Dubrin 2004; Gorman 2000; Isaacs 1998; Nealy 2005; Schulz 1998; Waller 2007). Starting in the late 1990s, a series of studies have shown the importance of these skills for business and industry, especially in those jobs demanding a certain level of responsibility and a capacity to organize and structure work autonomously and work in teams (Aquilino et al. 2006; Cameron and Whetten 2002; González Maura and González Tirados 2008; Maes et al. 1997; Murnane 1996). But even when soft skills are essential to achieve efficient work, universities still tend to overlook them. Corominas et al. (2006) note that teachers value interpersonal relationship skills, team work, decision making and problem solving throughout students’ university education, communication and information management being the skills that are actually developed and most valued in university classrooms. However, teachers do not value leadership, initiative, direction, organization and management skills, which are precisely the most valued competencies in the working environment. Palmer Pol et al. (2009) note that both academics and employers recognize the importance of team work, but while academics focus on those competencies related to the learning process (analysis, problem solving, research), employers focus on capacities related to autonomy, the efficient use of time, document writing and the use of technology. In any case, it is necessary to develop both hard and soft competencies and, therefore, the formal university syllabus should systematically include the possibility of developing interpersonal and evaluation skills in a controlled and supervised environment (Woeran and Kronsteiner 2007).

Development and evaluation of soft skills in b-learning university environments: the Evalsoft system

The learning and evaluation of generic competencies in b-learning environments is one of the outstanding challenges in university education and teachers’ tutoring work and continuing feedback is essential when it comes to achieving students’ autonomy and cooperation (Medina 2001). However, the process does not end with the development of a particular skill; it is also necessary to take an active part in the evaluation process. Students should be informed of their competency acquisition level in order to facilitate maximum achievement (Tobón 2006). In relation to this, technology has opened the possibility of making changes in the evaluation process, which is now more automated, quick, self-regulating and collaborative (Pérez Pueyo et al. 2009; Rodríguez and Ibarra 2010).

Therefore, the learning and evaluation of generic competencies in b-learning environments is one of the outstanding challenges in university education. The aim of this study is to present the results of a comprehensive, Moodle compatible system designed for the development and evaluation of inter-personal competencies – teamwork skills specifically – in university students through virtual learning environments (VLE).

The Evalsoft system allows learning, follow up and evaluation of generic competencies in a virtual professional environment. It is an application that functions as the persistence layer of a virtual environment running on Moodle; it has been designed and developed to support those needs derived from a problem-based learning (PBL) methodology. This software allows easy and efficient management of students’ team work and teachers’ monitoring by including different skill evaluation tools based on self-evaluation, peer-evaluation and teacher evaluation.

Skill learning is associated with an active team learning environment that includes the use of CTIs. Sancho, Fuentes and Fernández-Manjón (2007, 2009) claim that the use of learning management systems (LMS) should be part of a pedagogical approach featuring training activities coordinated with the possibilities opened up by technology in an educational context.

The Evalsoft system is based on a blended learning approach that combines game and role-play dynamics with problem-based and collaborative learning strategies. Thus, the pedagogical approach is based on role-plays in which students have to carry out a series of tasks working in teams in order to solve a problem. Successful PBL strategies require planning, organization and an efficient task distribution, but group cooperation and cohesion also needs to be cared for. The former are part of what constitutes efficient leadership and the latter are related to the capacity to work in teams. Therefore, these two competencies are included in the learning process along with the specific skills applied to solve the problem. Team work skills are developed through blended or remote tasks in which the leading role in each mission is rotated.

Each team organizes itself, collaborates and self-regulates in order to solve the proposed mission under their teacher’s supervision. One of the key aspects in successful work teams is grouping. Therefore, teams are set up with 3–5 members in accordance with a diagnosis of students’ learning styles (Vermunt’s Learning Styles Inventory) (Vermunt 1992), the aim being setting up self-regulating teams. That is, groups are intra-heterogeneous in terms of autonomy level but inter-homogeneous in relation to all the groups in the classroom so as to ensure balance and effectiveness in all teams. Students are assigned a functional role with specific tasks they will be responsible for within their team.

The learning process starts with the presentation of a project or mission that students have to solve by searching for information, analyzing and summarizing it, writing, making decisions, planning, organizing work and cooperating in a virtual environment. The virtual format allows for working remotely, coordinating time zones and reconciling the working pace of all the team members. As previously mentioned, the system runs under Moodle modified with new plug-ins and with a private server in order to facilitate the management and evaluation of team work, as well as the specific skills to solve the learning problem proposed. Given that competency is evaluated by carrying out tasks and is developed throughout the learning process, evaluation needs to be anchored in this process in order to collect evidence favoring competency achievement (Villardón 2006).

Therefore, the competencies that students have to apply in action are defined, and the evaluation criteria and indicators that will allow assessment of the acquisition of competencies through students’ answers to situations taken from a real context are specified. To sum up, the Evalsoft system has been created to be developed cyclically in five stages (Fig. 1). It is a continuing process, which means that once the mission is completed, a second iteration with a new mission can be initiated.

Fig. 1
figure 1

Evalsoft system stages

Full size image

Team building and role assignment (Stage 2) are carried out based on students’ interpersonal skills scores and on the perception of satisfaction and compatibility amongst team members. A quarterly subject can be developed through two or three iterations depending on the extension and level of difficulty of the project or mission. Moodle provides a series of collaborative instruments that are used in Stage 3 to facilitate work development and help complete the mission or project: glossary and private group chat through the whole course of the mission and a private group forum and private group wiki that can be independent in each stage depending on the difficulty and differentiation of the project tasks. Each post created by each group member can be improved, extended and corrected by classmates and, thus, the mission is carried out cooperatively.

Objectives

The objective is to improve university students’ team work skills (soft skills) in a b-learning environment (Evalsoft system) designed for undergraduate courses. It is, therefore, important to check the level of reliability that soft skills evaluation can achieve in virtual contexts and, consequently, it is worth considering whether or not team work skills can be reliably developed and evaluated in b-learning environments. Furthermore, it is also important to assess whether the Evalsoft system can be an accessible, efficient and realistic tool allowing university teachers to incorporate the learning and evaluation of generic competencies students will require for their future career advancement.

Population and sample

The population was made up by students enrolled in Teacher Training, Pedagogy, Social Education, Business Management, Educational Psychology and Materials Engineering courses at the Universidad Complutense de Madrid (UCM). Sampling was non-probability and incidental. The total sample includes 893 UCM students distributed by degree as shown in Fig. 2.

Fig. 2
figure 2

Sample distribution by degree

Full size image

The largest percentage corresponds to degrees in the Education field (more than 80 %), the remaining 12 % being distributed among Politics (1.2 %), Business Management (7 %) and Materials Engineering (4 %). In terms of gender distribution, over 78 % of the sample population was female and 22 % was male. The sample was divided into two groups in order to verify achievement of the objectives of the study: an intervention group which participated by using and implementing the Evalsoft tool (73.6 % of the sample) and a non-intervention group which was given traditional education with no specific use of virtual campus except for collecting material (26.4 %).

Variables

Students’ learning context – either on-site or b-learning with Evalsoft software – is the independent variable for the analysis and study work, team work skills being the dependent variable. The agreed definition of team work in this study is “to become integrated, collaborate and cooperate with others in an active and efficient way to achieve common objectives” (Ruiz Morales 2013).

Instrument

An instrument to evaluate team work skills was designed and assessed based on a review of the specialized literature (Bloxham and Boyd 2007; Ibarra and Rodríguez 2007). The object of this instrument is for students to self-evaluate their performance on a 1–10 scale in accordance with six categories related to this skill (supporting and motivating group members, agreement on and acceptance of team rules, participative attendance at group meetings, contribution of ideas and research material on the subject, analysis and preparation, contribution to the group cooperative processes). Two contrasting criteria were set for each of the categories (semantic differential scale) in order to help students with their self-evaluation process.

Data analysis

Statistical analysis was carried out with the SPSS pack (version 19.0). The instruments’ internal consistency was assessed with Cronbach’s Alpha, and team work skill differences between on-site and b-learning contexts were studied through an analysis of covariance.

Results

The instrument shows a very good internal consistency level (George and Mallery 1995), Cronbach’s alpha reached a 0.837 value (over 0.8). We carried out a covariance analysis and checked the starting position in order to analyze changes in the team work skill across different learning contexts. The results obtained reveal the environment’s capacity to affect skill level. Table 1 shows that the group variable is statistically significant (p-value < 0,000), which means there are significant differences between the intervention group and the non-intervention group. More specifically, the estimated values for these differences is +3.207. That is, the results in the intervention group are on average 3.207 points higher than those obtained in the non-intervention group (Table 2).

Table 1 Ancova TW competence
Full size table
Table 2 Ancova TW competence
Full size table

Table 3 and Fig. 3 show that the the non-intervention group mean results are below 50 points, whereas the intervention group exceeds this threshold.

Table 3 Team work skill descriptive statistics
Full size table
Fig. 3
figure 3

Differences between Intervention and Non-intervention groups. Team work skill

Full size image

The results show positive empirical evidence for the b-learning context (intervention group) in the development of team work skills. Therefore, it is safe to state that the implementation of e-learning and evaluation tasks in a b-learning context has a direct, positive and significant effect in the learning of team work skills.

Conclusions

Although previous studies have pointed out difficulties in the evaluation of team work, commitment and leadership in b-learning environments (Herradón Diez et al. 2009), this study reveals that team work soft skills reach higher achievement levels through the Evalsoft system in a b-learning environment than in the traditional classroom context using the virtual campus only as a document repository, these results being in line with the work by Guitert et al. (2007).

In this regard, PBL articulated with e-tasks, simulated context, collaboration and communication tools (forum and wiki) seem to facilitate the learning and development of this skill. More specifically, e-tasks favor collaborative e-learning; that is, tasks related to learning by doing (planning, building, designing, proposing) in turn facilitate learning by discovery (searching, questioning, exploring) and enable e-evaluation as a form of reflection and improvement (self-evaluation, peer-evaluation) so that students experience skill learning based on their involvement and co-responsibility in team work.

Adaptation to students’ needs and interests is one of the keys to the success of this work system, characterized by an integrated use of new technologies and by the close link to students’ future career course. A detailed analysis of the constitutive elements of this system shows that the wiki and the e-forum are the key tools provided by Evalsoft in terms of the development of team work skills observed in the intervention group. Specifically, the wiki can be key to students’ team work as it enables and facilitates the planning of actions to be carried out in the resolution of the proposed mission or problem and is also useful for the co-creation of the team work final document, all this being in line with findings in similar studies (González Pareja et al. 2006; Benito et al. 2008; Echazarreta et al. 2009; Redondo et al. 2009; Santos 2009; Vaquerizo et al. 2009; Villanueva, 2009).

Similarly, the e-forums used throughout the intervention facilitate students’ interaction in the collaborative prosecution of the mission as they allow the sharing of ideas, comments, messages, information and emoticons. Along these same lines, Marcelo García and Perera Rodríguez (2007); Llorente (2008) and Ezeiza and Palacios (2009) find that academic forums favor personal communication and relationships as well as students’ interaction in a virtual environment. The fact that students are highly familiar with this work tool, which they use in daily life, is also a factor worth considering as it places this work system substantially close to students’ lives.

To sum up, the results obtained reveal that PBL and, more specifically, the Evalsoft tool facilitate the learning and development of team work skills. This is in line with the findings of previous studies by a series of authors (Hernández and Lacuesta 2007; Álvarez et al. 2009; Sancho Thomas et al. 2009; Salmerón et al. 2010; and Sancho Thomas et al., 2011). Thus, the possibility of developing soft skills in remote learning environments appears to be an achievable objective thanks to ICT tools, which allow for the creating of enriched learning environments where virtual reality brings students closer to the working world. This results in more effective training and future professional work. The advancement of remote and semi-remote learning environments needs to provide an effective answer to this requirement, one which can guarantee high quality training for future professionals. This kind of training in turn requires a specific focus on skills development, which is essential to personal and professional development.

However, it is worth noting that the data obtained in this study, though promising in terms of skills development in virtual environments, is still insufficient and requires further research specifically focused on students’ internal processes of interrelation. Moreover, it would be desirable to expand the study sample in order to obtain more solid evidence in terms of the Evalsoft tool’s efficacy.