1 Introduction

The engineering studies are often qualified as “difficult”. Before getting to the specific subjects that usually attract the students (situated at the end of the studies), the first courses provide basics skills in more general and less motivating subjects. For these reasons (the difficulty of the studies and the unattractive initial subjects) and others such as the past crisis period, the number of engineering students in Spain has decreased, despite the recent upturn due to the high labor demand and progressively increasing salaries. In these years of “crisis”, high dropout rate has been observed especially in the first courses due to their difficulty. Therefore, there is a clear need to accompany the student in this educational phase, since despite being adults; they are not prepared to pass the new difficulties. We can summarize the main objectives of the academic advisor (also known as tutor or academic supervisor) as:

  • Accompany the student in his/her formation and advise him/her academically,

  • Minimize the number of students dropping out of their studies,

  • Improve the satisfaction of the students.

For these reasons, the faculty established the following principal functions of the tutor:

  • Introduce him/herself personally at the beginning of the studies.

  • Obtain teachers’ collaboration to detect problems that can affect the students and inform about any situation concerning the student, to have in mind any academic situation to follow up.

  • Regularly follow up with the student to identify any problems, even on a family level. Regarding this situation, report to the appropriate authorities responsible for that situation. Attend students, family and teachers quickly and effectively.

  • Record and keep record of the results of interviews.

  • Question him/herself after exams, especially in cases with high fail rate.

  • Attend all academic councils to take care of all cases, as well as meetings referring to educational planning.

  • Help the students with the subjects they choose and every question they may have.

  • Study and analyze incompatibilities between subjects.

  • Collaborate in the election of the delegate for each course and the representatives of the student council.

  • Take care of complaints, claims and suggestions of the formation program as well as the services and the infrastructure.

This paper presents the experiences of an intervention carried out at Engineering La Salle Campus Barcelona-URL. In order to make the students more aware of their situation, some coaching resources are used in the university tutoring process. This way, students are able to make decisions with the right criteria, motivation, conviction and responsibility. This proposal deals with improving the acquisition of general skills in decision-making, critical capacity, self-motivation, initiative and self-knowledge. The aim of the experience is to assess the inclusion of coaching techniques in the advising services, and their evaluation for further uses using an analytic approach. The solutions and results will allow us to produce a shared system that will use specific techniques for specific problems according to their functionality in the past, suggested automatically by the system for any advisor and student.

This paper continues (Sect. 2) with a brief explanation of the current theoretical framework, concerning both the typical problems of academic supervision and the effects of coaching in the educational sector. In Sect. 3, the paper describes the design of the study performed in the Engineering studies, where coaching techniques were applied in certain cases of complex academic advising. In Sect. 4, we find the main results obtained, and finally (Sect. 5), we find the conclusions and the future lines of work.

2 Theoretical Framework

Academic tutoring is being established as a highly important [1] complementary activity to the student’s evolution. In both pre-university (where the main aim is to avoid school drop-out [2, 3]) and university levels (where problems are usually more complex) we find an increasing number of educational centers that offer this service, especially in the first courses [4]. It appears almost certain that it is in the first university courses [5] that student’s tutoring is most important.

The tutor reflects physically or digitally the information from every tutoring session, but this information is not proactive for future similar problems concerning other tutors. As we will see in next sections, in these cases the use of tutoring systems and learning analytics can improve the advising service with a more quickly identification of problems and solutions. In any case, a tutorial that effectively advises the student will generate a positive educational outcome, either causing a reduction in educational dropout rates and/or improving the student’s academic results [6]. For this reason, and given the new educational panorama, it is vital to continue with the tutoring, and we must incorporate new resources, such as coaching [7].

2.1 Coaching and Self-regulated Learning

We can define the coaching as a way of enhancing people’s consciousness. Coaching aims for people to be more aware of the reality they are living, in order to find a way of achieving one’s objectives or solving one’s problems [8]. We can find many works and experimental cases where coaching activities have been developed [8]. However, Bettinger and Baker from the National Bureau of Economic Research of Cambridge [9] carried out the most similar work within our framework. This work presents the effects of coaching among first-year university students in the fields of economics, education and sociology. Among other findings, the study quantifies a 5% reduction in dropout rates on the first year, as well as a 4% decrease in dropout after 24 months.

Whether a student engages in learning actively or chooses to disengage from the activity, completely or in part, he or she is lead by motivation, and in this context, it is a self-regulated choice. In terms of one widely cited model of self‐regulated learning proposed by Winne and Hadwin [10, 11] learners exercise agency across four loosely sequenced phases: (1) They scan their environment to identify internal factors (cognitive, motivational, affective) and external features that may influence a task. (2) They frame goals and design plans to approach them. (3) They implement actions to animate their plans, monitor the match between a plan and its actualization and modestly adjust actions as they judge appropriate. And, (4) they re‐examine aspects across these three prior phases to consider major, strategic revisions to understanding and action if progress toward goals is blocked, too slow or in some other way unsatisfactory.

Adapting this approach to a coaching strategy we are close to a GROW process [12, 13], defined by these phases:

  • G (Goal): The student defines the objective he pursues, consistent with his own values.

  • R (Reality): The student analyses, jointly with the tutor-coach, ‘his reality’.

  • O (Options): The student identifies the different ways of achieving the objective, and is advised about the benefits vs. costs of every option.

  • W (Will): The student defines a plan of action: what, when, how, where…

On the other hand, and in accordance with the assumption that learners, as agents, will regulate their learning, Winne and Nesbit [14] recommended an approach that investigates “the way learners make things”. This paradigm has three main components:

  • Instruments should be developed to gather data that trace over time which information each learner operates on and how each learner operates on each of those selections.

  • From the data obtained, temporally extended trajectories of engagement should be assembled for each learner.

  • And finally, in exploring those trajectories and their relations to other “snapshot” data (e.g., responses to self-report questionnaires, measures of aptitude or end-of-course achievement), learners should be grouped into homogeneous groups based on data rather than pinning hope on random assignment to neutralize sources of unidentified variance.

2.2 Tutoring Systems and Learning Analytics

The first computer tutoring systems to be used in school classrooms [15, 16] showed the influence of the programmed instruction movement of the time: They presented instruction in short segments or frames, asked questions frequently during instruction, and provided immediate feedback on answers [17].

These systems guided learners through each step of a solution to a problem by creating hints and feedback as needed from expert-knowledge databases. The first-generation computer tutors have been given the retronym CAI tutors (Computer-Assisted Instruction tutors); the second-generation tutors are usually called Intelligent Tutoring Systems, or ITSs [18]. More recent reviews support conventional beliefs about CAI tutoring effects. For example, a 1994 review aggregated results from 12 separate meta-analyses on computer-based instruction carried out at eight different research centers [19]. Each of the analyses yielded the conclusion that computer-based instruction improves student learning to a moderate degree.

On the other hand, a learning analytic approach provides institutions with opportunities to support student progression and to enable personalized rich learning [30]. With the increased availability of large datasets, powerful analytics engines [31], and skillfully designed visualizations of analytics results [32], institutions may be able to use the experience of the past to create supportive, insightful models of primary (and perhaps real-time) learning processes [33].

A possible way to support teachers in monitoring and guiding student activities (including their collaboration), is by enhancing digital environments with learning analytics tools [20]. The traces left by student activities can be automatically collected, analyzed, and reported back to both students and teachers for optimizing learning [21]. Learning analytics that are specifically aimed at supporting the teacher may serve a number of functions. They can be placed on a continuum of how much control or choice is left to the teacher [22]: from providing overviews of data to suggesting particular actions to even undertaking those actions (for example, by automatically sending a message to a student).

We can find few proposals that have worked with the aim of creating interactive systems that allow for feedback from tutorials [23, 24], focusing on online student management [25], but not related with coaching methods in academic fields. For these reasons, our project is the first step in a process to define the main coaching variables and student situations in order to develop a system that can merge the tutoring systems, coaching approaches and learning analytics to predict both problems and solutions in the advising services.

3 Case Study and Method

3.1 Sample

For this study, 41 first year students from academic years 2014–15 and 2015–16 have been selected, belonging to the degrees of Telecommunications and Computer Engineering form La Salle, Universitat Ramon Llull. Students were selected based on their academic results of the first control point (first semester), where they have been evaluated in a total of 7 subjects.

From the 41 students, 21 have been assigned to the experimental group, and 20 to the control group. All of them, after the first basic tutoring, were identified as potential ‘risky students’, due to both personal and academic reasons. According to the available times for tutoring, 21 experimental cases were randomly selected, to which an intensive following was made based on coaching techniques.

3.2 Procedure

To obtain the initial profile of the students in the experimental group, an interview was carried out in order to identify the academic situation and personal problems regarding some of the following situations (which we will identify for the analysis of results as IS-#, corresponding with Initial Situation):

  • IS-1: Frustration and demotivation due to bad results.

  • IS-2: Existence of a limiting belief.

    • The degree is too difficult for the student.

    • The student is unable to study for a minimum of time.

    • …

  • IS-3: Will to abandon their studies.

  • IS-4: Incapacity to know how to face poor academic results:

    • What to do?

    • What to change?

    • …

  • IS-5: Personal or family problems (non-academic).

According to the detected problems, coaching activities selected for application are (defined as CA#) [26,27,28,29]:

  • CA-1: Visualization at ten years sight. This is a technique applied to achieve a desired emotional situation by imagining specific images. For example: when students are confused and do not know why they are studying their degree, they are asked to visualize how they would like their life to be in 10 years, both at professional and personal levels. This helps defining objectives at short and medium terms, since they have visualized and felt (at an emotional level) the future they desire.

  • CA-2: Six thinking hats: Methodology that allows students to analyze the situation they are living from different points of view and not only theirs, in order to be aware of the different approaches that can be used to face their situation, creating new action alternatives.

  • CA-3: Exposition to the concepts of reactivity and proactivity: Students are shown the difference between reactivity and proactivity. In most occasions, students become aware that their attitude towards their problem is reactive, and they complain without ever looking for alternatives to solve it. By asking about ‘What one can do to improve the situation?’, or ‘What depends on oneself?’ new alternatives emerge.

  • CA-4: Use of tales as metaphors to help students discover what they need to discover. It is easier for students to become aware of what is happening if they see themselves reflected in the main character of a story. Tales can help students discern that their problems are limiting beliefs, their indecision, feelings of guilt, etc.

  • CA-5: Helping to identify a clear objective through ponderous questions: Ponderous questions can make reflect, impact and even incommode. When students are not able to find an answer that satisfies them, then the process is started and they look for a change in their current situation, identifying a well-defined objective; achievable, relevant and bounded in the time period.

  • CA-6: Question limiting beliefs with ponderous questions: Through the right questions, students are helped to identify data that justify some of their beliefs.

  • CA-7: At an emotional level, identify the somatic triggers that allow awareness of an emotionally intense situation, in order to later define an escape behavior that breaks the blocking process: Through the use of the right questions, students are helped to identify the physiological indicators that warn of an incoming blocking situation. This way, before reaching an intense level leading to a block, they can define an escape behavior.

  • CA-8: Exposition to the coherence triangle: think-do-feel. Students are shown the need of having coherence between what one thinks, does and feels.

  • CA-9: Abdominal breathing that helps relaxing and regaining self-control: In some cases, students reach a level of anxiety and stress that prevents them from thinking with clarity. It is interesting, in these cases, to teach them to breath adequately.

  • CA-10: The wheel of life adapted to students: It allows them to identify the areas in their lives they would want to improve. It is an interesting resource when the student is very confused and does not know the objective he/she desires.

With the aim of evaluating the followed method, we have accounted for two variables: academic and personal. Regarding the academic variable, we have identified he number of failed subjects at the end of the year and their variation with respect to the control point, and whether the student abandoned the course. The personal variables have been integrated in a subjective way, based on their tutor’s evaluation of the process. The degree of satisfaction perceived by the tutor does not reflect the academic results, but the comments and feelings of students and their relatives. For example, a student may have failed several subjects, or even dropped out, and at the same time have a positive evaluation of the process from his tutor, because the student has evolved and became aware of the situation, taking decisions in a mature way once the situation is assimilated.

4 Main Results

As we can observe in Table 1, while in the control group the percentage of failed subjects between the exam and the end of the year has increased, this percentage has been reduced (from 4,86 to 4,14) in the experimental group. To estimate the probability that results are significantly similar, we used the Student’s t-test, and tested a null hypothesis (Ho) that there are no differences in scores between variables. As P(T) = p two-tailed is 0.036, which is less than the threshold of 0.05, this means that there is a very high probability that the results are statically different. Comparing the homogeneity of the two groups, and therefore the possibility to continue with the experiment based on the academic results, the p two-tailed is 0.83, a value that confirms the initial academic similarity between the two working groups.

Table 1. Failed subjects
Full size table

Additionally, it was observed that while in the control Group (at the end of the course) there were 35% of drop outs (all of them from the high risk group identified at the beginning), in the experimental group the percentage of dropouts was reduced to 23,81%.

At the end of the academic year, students from the experimental group have managed to reduce the number of failed subjects significantly, as well as the percent-age of dropouts. In Table 2, we can observe the summary of problems identified at the beginning of the coaching process by the experimental group:

Table 2. Initial situation/problems detected (based on classification of Sect. 3.2)
Full size table

Depending on the distribution of problems identified and their way of repeating, it was found that everyone shows an average of approximately two problems with a specific distribution of:

  • 19% of the students show only one problem (with a 50% of predominance of IS-4),

  • 47,6% shows two (with the most repeated template of IS-1 with IS-3, three repetitions, and the combination of IS-2 and IS-4, with two repetitions),

  • 28,6% shows up to 3 problems all together (with 50% repetition of the problems IS-1, IS-2 and IS-3),

  • Only one person (representing 4,8% of the sample) showed 4 initial problems (IS-[1–4]).

Analyzing the performed coaching sessions (an average of 3,76 with a maximum of 7 sessions and a minimum of 2), the summary of implemented activities can be found in Table 3:

Table 3. Coaching activities developed (based on classification described in Sect. 3.2)
Full size table

We can summarize the situation with the experimental group like a sample of students with bad results at the intermediate point of the course (an average of 70%-failed subjects). A priori, this situation is the result of other three situations coming from the difficulty of the studies: feeling frustrated (47,62%), not knowing how to handle the complicated situation (47,62), and the desire to drop out (52,38%).

Based on such data, the most used coaching techniques (with an average of four techniques per student), always selected in function of every case are:

  • Reactivity vs. proactivity (71,42%),

  • Ten years personal situation visualization (61,94%),

  • Identification of certain goals from fundamental and powerful questions (61,94%),

  • Exposure of the coherence triangle: think-do-feel (57,14%).

Analyzing the existing individual correlation between the initial problems and the techniques implemented, there were no especially high factors found. For example, for the IS-1, the CA1 is the one that had the most use (positive correlation of 0,651); while for IS-2 we found 1 correlation with CA-6 and 0,763 for the CA-2. Another elevated amount we found is IS-3 with CA-1 (0,726 factor) but we didn’t find another interrelation with index superior to 0,5 of use.

We have focused on two aspects for analyzing the effectiveness of the method: on one hand, and probably the most objective, is the continuity of the studies and the curriculum improvement of the student (Table 4). On the other hand, the subjective appreciation of the process by the tutor (Table 5).

Table 4. Efectiveness of the method based on the dropouts.
Full size table
Table 5. Efectiveness of the method based on the subjective assessment of the advisor/coach.
Full size table

It is necessary to mention that the students that conform the sample analyzed in Table 5 were divided based on their results from the coaching process by their tutor. This way, the five students that were identified in the non-satisfactory process (first column), should not coincide with the five students that dropped out in Table 4.

These last results show us a series of interesting key concepts identified:

  • The first idea of a large percentage of students when facing problems at the be-ginning of the course is to drop out (IS-3),

  • Usually, students have more than one problem at a time, and all of them re-quire a different technique,

  • The level of monitoring and motivation at the tutorial sessions influences directly on the rates of drop out and/or success of the coaching process that has been applied.

5 Conclusions and Future Work

One of the most interesting of the evaluated aspects is the comparison between the students’ academic improvement of the control and experimental groups. If we calculate the difference between the number of failed subjects at the end of the course and on the first semester, the average result for a student in the experimental group is −0,62 (0,62 less failed subjects as the first semester, meaning there has been an improvement), while in the control group, the result is +0,3 (worsening by an average of 0,3 subjects). It can be observed that the improvement in academic performance of the experimental group with respect to the control group is of 0,92 subjects.

If we execute the same calculations excluding those students from both groups who dropped out of the degree, the results for the experimental group is −1,18, while for the control group it is −0,35. That is to say, if we only focus on the performance of those students who did not abandon their studies, there is a significant improvement in the experimental group’s performance with respect to the control group. In this case, the difference is of 0,83 subjects.

On the other hand, the use of coaching as a work tool in academic tutoring has demonstrated its usability in the effective detection of problems in students. These problems are critical when they affect the student’s performance, especially in the first courses, where the risk of dropping out is higher. As it has been demonstrated, the effective identification of problems and the adequate administration of solutions makes the student more aware of the situation and allows to remedy a complicated start.

Currently, there is work being done for correctly defining problem typologies, and for creating a system that automatically allows tutors to access solution proposals for registered problems based on coaching tools. The system must allow to facilitate and share experiences in the use of coaching that can be applied on tutor teams in heterogeneous environments such as Engineering, Architecture, Animation and Business Studies.