Introduction

Retention in Computer Science (CS) Education

Students’ dropout rate is a major issue in higher education because of the serious consequences it has on the individual, the economy and the society (Cope and William, 1975; Pascarella, 1985; Duque, 2014; Pusztai et al., 2019). Dropping out from a CS course before graduation, is known to evoke ambiguous consequences. As a worldwide phenomenon, it calls curriculum makers attention to the importance of encouraging students in order to complete their studies. Although high dropout rates at universities have been considered a social problem, not too many solutions have been suggested (Lamb and Markussen, 2011). According to Bernardo et al. (2017) from many attempts of studying this problem at least five types of paradigms have been raised so far: economic (Belloc et al., 2011; Di Pietro, 2006), psychological (e. g. Nagrecha et al., 2017), sociological (Braxton et al., 2000), organizational (Bean, 1983), and educational (Cabrera et al., 2006). The aim of the present study is to analyse the results of the students through the curriculum of the CS program by factor analysis. Analysing the subjects as factors in the CS curriculum could help faculty staff introduce an education reform.

Theories of Student Attrition

One of the most widespread models of explaining student attrition is Tinto’s exploratory model (1975, 1998). The interactional theory of student persistence in academic life emphasizes the importance of the students’ interactions with the institution—how the student is integrated (“fitted in” both academically and socially)—besides other important factors such as personal characteristics, traits, experiences, and commitment. Pascarella and Terenzini (1983) also highlighted the correlation between social and academic integration, such as peer relationships and faculty member relationships with academic success. Braxton et al. (2004) also emphasized the need for community on campus as a help of social integration to develop relationships between students. Student’s pre-college experience and personality characteristics interact with internal structures, policies, and practices of the university. These factors interact with each other and only at the end of this interactional circle will it turn out whether or not the student persists (Reason, 2009; Terenzini & Reason, 2005).

Interactional theories suggest building connections between students and their institutions. Nevertheless, according to Braxton, Hirschy, and McClendon (2004), there is not a fully successful predictive model of student engagement. Despite the fact that students have the same academic backgrounds, socioeconomic demographics, characteristics and motivations, they still drop out. It remains a question why some students retain successfully at the university.

Students’ performance after the first academic year is also a topic of significant interest. HEFCE (1997) concluded observations and found that the lack of the student’s engagement in the academic life and their unpreparedness are mainly responsible for dropout after the first highly crucial period. Cook and Leckey (1999) considered the necessity of developing new attitudes among first year students. According to their views first-year students are the most vulnerable at the early stage of their academic career. Lack of preparedness, immaturity and unsuitable study skills often lead to unsuccessfulness at university.

In Europe, at one of the largest public universities (Eötvös Loránd University, Budapest, Hungary) the overall dropout rate is 30%, and at the Faculty of Informatics the average rate was 60% between 2010 and 2016, which is similar to that of other European countries (Borzovs et al., 2015; Zwedin, 2014). Most students resign from the university in their first year. After the first semester usually on average 30% of registered students leave higher education, and this number increases to 60% by the end of the first year (Borzovs et al., 2015; Ohland et al., 2008). It has become a worldwide phenomenon and an important issue to be solved.

The Role of Mathematics in CS Education

The CS Program is considered to be one of the most difficult programs because it requires a full comprehensive understanding and improved ability in order to fulfil academic requirements (Wu et al., 2014). Acquiring programming principles and logical problem-solving skills is usually related to mathematical subjects, as well. Knowledge of logical mathematics and its application in programming are essential for complex tasks. Most students have difficulties with mathematical subjects; the correlation between success in programming courses and mathematical courses is still very high (Duran, 2016; Ali et al., 2014; Balmes, 2017).

The Association for Computing Machinery published a general report as a guidance of curriculum for CS education (Sabin et al., 2015). Every CS curriculum contains mathematics subjects for at least 12 compulsory credits. At the Faculty of Informatics, ELTE most students (51%) had problems with subjects related to mathematics (Takács and Horváth, 2017).

Programming is a complex activity which can be learned and developed by various methods. Each method is characterized by the order or emphasis on teaching this knowledge and skills (Szlávi and Zsakó, 2003). In all methods mathematics, as a basement of logical thinking is required. The methods include process-, algorithm-oriented; data oriented; specification oriented; problem-type oriented; language oriented; instruction-oriented and mathematical oriented methods. For higher education, high school maths and physics education do not provide a required basic input, so preparatory or catch-up training is required. The question is whether the lack of fundamental knowledge and/or the lack of theoretical mathematics and science in the beginning contribute to a high dropout rate.

Education Reform

Based on the above mentioned, institutions have a responsibility to look into the phenomenon and consider the application of various prevention strategies in order to interfere successfully, because attrition in CS is a well-known phenomenon, which should be analysed and interfered. This might lead to important savings for the government and students alike (Bernardo et al., 2017). Analysing the problem at a deeper level, preventive actions can be made which could generate great benefits (Aljohani, 2016; White and Sivitanides, 2015). Tinto and Cullen (1973) suggested a comprehensive model to explain dropout and it was adopted in the higher education model by Tinto (1990). Lacante et al. (2002) placed the model in a longitudinal perspective. The most basic model is still Tinto’s model [e.g. Forsman et al., 2015). However, different models are available to explain the process of dropping out [e.g. Spady’s model (1970) or Bean’s student attrition model (1980)], which was applied by Tinto, who changed some of the variables. Pre-entry variables could hold an effect on which student will be adopted easily to academic requirements. Previous school experience (interaction with pupils and teachers) can result in a higher or lower level of integration. The level of university engagement, interactions will collaborate with students’ performance and will boost their motivation into studying. All of these variables can lead to decisions for engagement towards the university or can result in dropout.

In accordance with the literature discussed above, a successful dropout prevention program should provide students with opportunities and support. In order to reduce dropout rates, we should purposefully develop prevention strategies that will have positive effects on students’ success.

Basically, these programs are designed to provide additional support, find ways to inspire young people to stay engaged in learning. Programs should also use these strategies: (1) Engage community organisations and schools as collaborators beyond a typical school day (e.g., afterschool activities) well in advance so as to help more students succeed. (2) A prevention program should contain components to strenghten the personal and social skills of students (e.g. Durlak et al., 2010; Huang & Dietel, 2011). Sandvall et al. (2019) introduced a Living-Learning Program as a model for student success and engagement. Students who had participated in the program were more likely to be engaged and had higher grade point average after the first year than students who did not participate.

Bernardo et al. (2017) examined 46 variables and four of them proved to be the most influential on students’ dropout decisions. The first most important variable was student progress, which means that student performance has a high impact on withdrawal decisions. These results are congruent with those which have been conducted by other researchers (Cabrera et al., 2006; Crawford, 2014; Moreno & Stephens, 2015; Willcoxson, 2010) and this is the reason why it is essential to introduce education reform in order to promote and develop students’ academic skills and help reduce the knowledge gap (King et al., 2015).

Another important variable was age, which appeared to be an important factor in dropping-out: older students require special adaptations to fulfil their educational needs (Shepherd & Sheu, 2014). The feeling of failure is related to insufficient academic progress, which discourages them from studying for a period (Sauvé et al., 2016; Tinto, 2015) because it can impair students’ self-esteem (Carabante et al., 2013; Fang & Galambos, 2015).

This conclusion highlights the importance of promoting student engagement and self-regulation (Trevors et al., 2016) in order to prevent dropout. Time devoted to studying is another important variable that is proved to influence their students’ persistence on the institution (Alarcon & Edwards, 2013; Moulin et al., 2013; Ruiz-Gallardo et al., 2016).

These results highlight the need for making the following steps: To address the dropout problem, a program was introduced with a preventive effort to retain students. Our project proposes a prevention and promotion program concept for CS students (Anonymised, 2017). In literature this concept is relatively new to higher education and such a program for all first-year students is not well-known. The steps of the education reform are the following:

  1. 1.

    Mentor program from 2006: Peer mentors support and encourage new first-year students to succeed at the university. Each group is provided with a peer mentor and a mentor teacher and has meetings weekly. As part of the buddy program, students share their problems with their teachers and fellow students, who then help them to cope with issues in university life.

  2. 2.

    Prevention and promotion program from 2016: A special course entitled “Preparation course for university studies and developing learning skills” became obligatory for all first-year students (nearly 400 students each year). The program is held by psychologists and peer counsellors for a group of 20 students. Developing concentration helps students stay on track and be able to achieve successful test scores. The topics and skills that are involved are the following: time-management techniques, studying techniques, avoiding procrastination, developing soft skills and developing a strong study group identity. They last 30 h. Students gain one credit for participating in the course, which is obligatory for every freshman.

  3. 3.

    In the Bologna education system students have to collect 30 credits per semester by successfully completing 8–10 subjects. Usually there are two types of subjects: lectures (theory-oriented) and practices. One of the most significant steps of the education reform was that lectures became compulsory to attend like it was already with the practice sessions.

  4. 4.

    First year students were organized in fixed composition groups of 20 students in order to promote community building.

Such changes were expected to help students develop better skills in communication, interpersonal relationships, critical thinking, and other areas essential for being successful in CS, and would result in a better preparation to completing the program. The examination of university and college student dropout is an important issue all over the world, but introducing a student prevention and promotion program for all freshmen students is not a well-known phenomenon yet. Our aim was to analyse students’ data and find the positive results in our education system and then consider further improvements in the institution. In order to invent a new prevention and promotion program we extended the course and it became compulsory for every student. The program develops the essential social skills and techniques necessary to fulfil the academic requirements and strengthen social skills to have better relationships with faculty members.

Research Question

In the present study, we introduce different steps of education reform attempting to help our students be engaged in their university studies. The education reform can have an effect and benefits on the retainment of students such as their integration into the academic environments. Therefore, using factor analysis, in our research we analysed the subjects in the CS curriculum. We wanted to see the connection between dropout and students’ success in the different subjects. The study had two objectives:

  1. 1.

    To investigate how the structure of subjects changed after the education reform and whether the dropout rate had changed. The first hypothesis was that education reform had a positive effect on student dropout, i.e. after 2016 we could retain more students.

  2. 2.

    According to the literature there are two main areas in the CS program: mathematical and computing subjects. The second hypothesis says that there are two main factors in the CS curriculum regarding student performance: mathematical and programming subjects. The present study was designed to address these important unanswered research questions.

Methods

Design

Factor Analysis for Mathematical and Programming Subjects

The IRT modeling of the subjects showed that there were some changes in the behaviour of the subjects in the years before 2016 or later (some subjects became easier, whereas others more difficult). The question is in what way the education reform changed the role of the subjects?

Using Principal Axis Factoring, we conducted an analysis of the subjects of the curriculum applying standards of simple structure. We tried other methods, too (Principal Component an with VARIMAX rotation d Maximum Likelihood) and they had the same structure. The first step was to examine whether the successfully completed subjects depended on students’ behaviour. There are two groups of students: (1) The students who enrolled in a course but did not take the exam (2) The students who enrolled and took the exam. What would happen to the subjects if only those students were taken into consideration who have actually passed and/or taken the exam (regardless of its ultimate success). The study analysed the main subjects of the Bachelor’s programme in Computer Science.

The analysis was performed using the factor analysis model of the STATA15, JASP 0.10.2.0 and IBM SPSS 26.0 software package.

Descriptive Statistics

Participants

The global sample is from a large public university in Europe. The students in the sample studied CS. All first-year students (N = 3672) are full-time BSc students in Computer Science. 2863 participants started the university before 2016; and 809 after 2016; and their average age was 19.81 Data were retrieved from the monitoring system keeping track of students’ academic performance (Table 1).

Table 1 Descriptive statistics: after the education reform the number of students who dropped out decreased by 28%
Full size table

Results

The first important finding is that there is no significant difference in exploratory factor analysis with respect to pre-2016 or post-2016 in the relationship of the subjects (Table 2):

Table 2 Main subjects of the CS curriculum
Full size table

We examined the subjects before and after 2016. We compared two groups of students: those who—although admitted—did not take any exams at all (Group 1) and those who took the exam (Group 2) and received a grade, even if it was “a fail”. (Table 3).

Table 3 Goodness of fit parameters of the complete sample
Full size table

The real difference between the analyses was in the latter case (thus there was no significant change before and after 2016). In each case, each subject achieved a communality value of 0.25 (Principal Axis Factoring procedure was selected with Varimax rotation) (Table 4).

Table 4 2016 and before, 2016 and after, sorted (some items were not comparable)
Full size table

Let us examine how the matrices (subject arrangement) develop after the post-rotation between group 1 and group 2 (Table 5).

Table 5 Result of the rotated factor scores
Full size table

It is clear that the structures of the two factor analysis significantly differ from one another although there are similarities, so it was not worth applying a structural analysis (SEM, Confirmatory/Confirmatory Factor Analysis).

What are the similarities between the two groups?

  • In both cases, the discrete mathematics lecture is on the first factor (a more difficult and more important subject with a higher dispersion index, with the ability to significantly differentiate students), and the linear algebra subject works similarly.

  • The script languages subject does not really matter: it is the least weighted item; it looks like it is a component that is essentially independent of the rest of the subjects, and is organised elsewhere.

What differences can we see between the two groups?

  • If all students are taken into account, there are essentially two clear factors: a highly mathematical direction (the first factor that causes greater difference between students, since this is the first factor that “best selects” students, as it generates greater differences) or a “programming/IT” factor.

  • If we only take students from group 2 (who took the exam), we find three factors: in this case, “basic subjects” come first; in other words, if we only look at Group 2, then there is a kind of “solid foundation” of the subjects (a mixture of mathematical and programming subjects) and more programming courses will appear on the second factor. After that comes an analytical part (analysis and C ++)—where we considered Group 2 i.e. students who took the exam–, which is followed by the programming methodology and functional programming subjects as a third factor.

It is noticeable, therefore, that the subjects are arranged according to the students’ performance in the following way: if all students are taken into account, we clearly see mathematics/informatics factors.

However, if only students who really took exams and at least risked receiving an inadequate grade were taken into consideration, the foundation subjects appeared on the primary axis, i.e. subjects considered to belong to basic knowledge and after that the ones that require professional, analytical or programming skills.

Discussion

To address the dropout problem, we invented a new perspective and introduced some education measures such as a promotion and prevention program with an aim to retain and help students to succeed. The signed areas in the model and the variables that have been included in the student program the present article studied. Our main result is that after the education reform were introduced, fewer students dropped out. According to the student monitoring system keeping track of students’ academic performance, after 2016 more students retained than before. The longitudinal and contextual character of higher education dropout makes its study more complicated, because it includes many variables, and in the meantime they interact with one another. Our result may be impressive since the number of students who dropped out has decreased by 28% (hypothesis 1 was proven). Similar education reform for the whole student population has not been applied yet. Normally, promotion and prevention programs are voluntary to attend. Therefore, in light of the results obtained, which have proven to be consistent with the literature, one can have a more accurate understanding of the complex phenomenon of drop-out. Therefore, our first hypothesis (i.e. the education reform has a positive effect on the dropout rate) has been proved. Although there might be some limitations, it is important to know that these measures are the key to promoting better persistence. Between 2010 and 2017, students’ abilities did not change significantly, despite an increase in admission scores. Our higher education reform focuses more on students, as those with lower abilities had the courage to take the exams.

A very important change took place in 2016, as the above higher education reform resulted in more students going to take the exams, thus increasing their chances of acquiring subject-based basic knowledge, eg. Basic Algebra, Basic Computer Science, Discrete Mathematics, and Basic Programming. Therefore, in our view, changes in the CS programme structure and prevention promotion program help students stay on track.

On the one hand, we analysed the impact of the education reform on students’ grades. On the other hand, the role of mathematics had been considered the major reason for CS students’ failure. A close examination of the curriculum could serve as a proof for a base impact of prevention strategies. Hypothesis two was partly proven (CS programme has two main parts such as mathematical and programming subjects). The question is that the lack of fundamental knowledge and/or the lack of theoretical mathematics and science at the beginning contribute to a high dropout rate. We can partly answer this important question. It is vital to see what kind of problems posterity CS programme has, how it can be improved, as problems get worse, and it is important to equip students with a career orientation, which is in their own interests. In this article we looked for the relationship between mathematics and programming skills and drop-outs. Our education reform helped students to stay and take exams, which resulted in a higher attendance rate at exams. We found different factor structures when we looked at the whole student population or students who took the exam. If all students are taken into account, there are essentially two clear factors: a highly mathematical direction (the first factor that causes greater difference between students, since mathematical subjects cause the most failure to students and poor performance can lead to attrition. If we took only students from group 2 (who took the exam), we found three factors: in this case, the “basic subjects” came first, then there was a kind of “solid foundation” of the subjects (we found a mixture of mathematical and programming subjects). Finally, programming courses appeared on the third factor.

According to Bernardo et al. (2017) in order to efficiently prevent students from dropping out, it is highly recommended to intervene during the previous educational stages because it is not sufficient to interfere when the drop-out occurs. The aim is to prevent a wide range of problematic educational situations such as failures and teacher-student conflicts, by promoting student academic skills.

Conclusion

A series of education measures have been introduced to prevent early university drop-out, our result implies an important call for future research. As a result of the higher education reform, more students stay at the institution and pass the exams. This fact alone affects the willingness of taking exams. The students who stay learn to make an effort for their degree instead of dropping out. According to Tinto (1988), university integration is a critical point in students’ success at the university. The measures taken at the institution try to influence all factors: help to establish a relation with the institution and the teachers and the mentoring program helps establishing a connection with fellow students. The promotion and prevention course increases self-efficacy and develops learning skills. All in all, it is worth addressing students’ progress. Our study underlines results previously achieved by Carabante et al. (2013) and Bernardo et al. (2017), who refer to these main factors in dropping out.

Our results show that students have significantly better academic results, have higher grades and more students stay and successfully complete all the compulsory subjects after the first academic year. As a result of all these changes, a larger number of students (+ 28%) have been retained at our faculty. This means that taking education reform could facilitate students to stay at university after the first year.

Attrition is a complex phenomenon, which is why introducing education reform is also a complex issue in order to answer this question more purposefully. Finally, as a consequence, we suggest the curriculum should change: the mathematical foundation could be kept as a critical subject, but it could be shifted to the second or third semester or replaced by computation-oriented subjects.

Limitations and Further Research

We are aware of some theoretical-methodological limitations of this paper. Although the above suggestions are aimed to help our students graduate, other higher education institutions might find it useful to take them into consideration.

It could also be important to better monitor students’ progress and promote the engagement of so-called non-traditional students who work besides studying (Van Doorn & Van Doorn, 2014). It is also necessary to introduce some education measures aimed at maintaining students’ engagement and augmenting their potential development at the university (Bernardo et al., 2017; Duque, 2014).