Introduction

Introductory computer science and programming courses at the undergraduate level (referred to as CS1 courses) experience failure rates near 30% in multiple student populations worldwide (Bennedsen & Caspersen, 2019). This occurs in a global context in which there is an increasing demand from industry for software engineers, computer scientists, and STEM professionals with proficiency in computational thinking and programming (CompTIA, 2021; Eurostat, 2022). Experiences in different educational systems around the world show that students’ exposure to computer science at the K-12 level influences their decision to pursue computing in higher education (Armoni & Gal-Ezer, 2014; Kurhila & Vihavainen, 2015; Webb et al., 2017). Yet despite the increasing importance of these skills in diverse professional and scientific fields, it is not compulsory in many education systems to teach computational thinking and programming at the K-12 level (Bocconi, et al., 2016). For example, a recent report documented that only 51% of high schools in the United States offer at least one foundational computer science course (Code.org, CSTA & ECEP Alliance, 2021). Students in the developing world find even more limited opportunities to learn the foundations of computer science at the K-12 level (Vegas et al., 2021). Thus, students who enroll in STEM curricula in higher education must typically learn computational thinking and programming at an accelerated pace in their first introductory course to the subject (Ahadi, et al., 2014). When this is done via traditional ‘one size fits all’ instructional approaches, students experiencing learning difficulties in introductory programming courses are rapidly overwhelmed by the pace of instruction, resulting in early attrition and high rates of course failure (Patitsas et al., 2019; Robins, 2010).

Mastery learning is a pedagogical philosophy and instructional approach that has been researched since the 1960’s (Bloom, 1968; Keller, 1968), based on the assumption that every student can master a given skill if given enough time, instruction and support. In contrast with traditional instruction, which sets a standard time period (such as a quarter or a semester) in which students are expected to learn the materials, mastery learning sets standards for mastering material that students are expected to achieve at their own pace. In the computer science education literature, it has been suggested that mastery learning permitting students to learn at their own pace can help them make sustainable progress in a CS1 course (Robins, 2010). This stands in contrast to the common occurrence of students being unable to keep up with the pace of instruction and risking course failure (Petersen et al., 2016) due to a combination of cognitive (Robins et al., 2019), meta-cognitive (Liao et al., 2019) and motivational factors (Dorn & Tew, 2015).

Despite the potential benefits, adoption of mastery learning in computers science education has remained limited, and few studies describe its implementation and evaluation. A recent review by Garner et al., (2019) reports that there is a lack of documented attempts to implement mastery learning in CS1 courses and that general guidelines and best practices on how to do so have remained unavailable. Thus, there is a pressing need to explore how to best implement mastery learning in CS1 courses, establish whether these efforts lead to improved learning, and if so, derive guidelines and recommendations that can be helpful for computer science educators interested in adopting mastery learning in their own contexts.

In this paper, we report on the development of an introductory computer programming course based on mastery learning at a research-based university in Latin America. The process was based on iterative cycles of an action research methodology over a four-year period, involving a total of 959 students. The mastery learning course was designed to fit a semester-based curriculum and academic calendar. To accomplish this, a modular instructional design approach was adopted; that is, one in which the course is broken up into multiple discrete, short duration segments of material and learning activities, known as modules (Jenkins & Walker, 2014; Dochy, et al., 1989; Goldshmid & Goldshmid, 1973). Considering mastery learning’s philosophy, students complete a module and are promoted to the next only after demonstrating mastery of the respective learning goals, through both low and high-stakes assessments. Students can retake a module they failed without necessarily failing the entire course. Consequently, multiple different module progressions are possible in the course, reflecting students’ varying prior preparation, aptitude and efforts to learn computer programming. In turn, the modular structure facilitates teachers in adjusting content and/or instruction according to students’ needs.

The following section presents the theoretical background for this work, followed by the research questions. Subsequent sections describe the design, implementation and progressive improvement of pass rates and attrition over the four year-long iterations of the course. The discussion section presents a holistic view of the challenges and difficulties experienced in the development of the course, and practical recommendations drawn from lessons learned in the process. Finally, conclusions and avenues for future research are presented.

Theoretical background

Academic failure in CS1

Failure in CS1 courses has been a topic of attention in the academic literature for the past several decades (e.g. Bennedsen & Caspersen, 2007, 2019; Watson & Li, 2014) with studies consistently reporting failure rates close to 30% on average. Bennedsen and Caspersen (2019) report a 28% average failure rate in CS1, with variations between geographic regions and by type of institution, i.e., college or university. For example, the combined rate of students aborting, skipping or failing the course in universities is 29%, while this figure for colleges is 17%. In Asia, the average failure rate is 29%, in Europe it is 31% and in North America it is 24%. It can thus be affirmed that in much of the world and across different types of institutions, approximately 20 to 30% of students fail their first programming course. Improving the academic results of CS1 courses can reduce student attrition, increase graduation rates, and increase the supply of qualified workers in scientific and technological fields to meet the growing demand. A wealth of research in computer science education has sought to explain why some students perform better than others in CS1 courses (Basnet et al., 2018; Guzdial, 2019; McCartney et al., 2017; Patitsas et al., 2019). One hypothesis for academic failure in CS1 courses relates to the cumulative nature of the materials and posits the existence of “stumbling points” in the learning path of programming: that is, there is a small number of identifiable skills and concepts that can have a major impact on a student’s progress (Ahadi & Lister, 2013). The implication of this is that students failing to master critical skills at key points in their learning will likely experience severe difficulties later in their learning path. A more general theorization of this idea is seen in Robins’ (2010) theory of learning edge momentum (LEM) as an explanation for students’ variability in learning in CS1. LEM proposes that with the tight integration of concepts and skills in the domain of computer programming, failure to learn concepts becomes self-reinforcing, thus creating momentum towards unsuccessful outcomes. Conversely, successful learning is positively reinforcing and creates momentum towards more successful outcomes.

The implication of both stumbling points and LEM for traditional one-size-fits-all instruction in CS1 courses is that in every cohort, students who fail to master fundamental concepts and skills are substantially more likely to fail to learn more advanced and/or composite knowledge. The solution to this problem lies in allowing students to progress to more complex knowledge only after they have mastered the earlier knowledge that is required. This demands instructional design and pedagogy that departs from the overarching constraint of a single fixed timeline (i.e., the monolithic rate and path of progression in a traditional course; Robins, 2010), thus allowing each student to progress according to their own ability.

Mastery learning in CS1

Bloom (1968) developed the mastery-learning approach based on the observation that providing all students with the same amount of time to learn and the same instruction resulted in a distribution of student achievement that reproduced the distribution of students’ initial aptitudes, i.e., a normal curve. Carroll (1963) argued that if individual students were provided with the time they needed to learn the material, most students should be able to eventually master it. The shift from fixing learning time to fixing a standard level of achievement is the driving feature of mastery learning (Emery et al., 2018).

Mastery learning emphasizes the need for constant tutoring, feedback and proctoring, which is difficult to scale to large cohorts (Fox, 2004). Keller, (1968) recommended a maximum cohort size of a hundred students due to practical difficulties of supervising student proctors. Furthermore, since in mastery learning can decide when to take the assessments, the approach was found to cause students to procrastinate and prioritize engaging in other academic activities. In traditional university contexts, where students are under pressure to study several subjects simultaneously, the approach might be difficult for some, especially first year students new to the independence required in the university context. These students often lack the self-regulation skills necessary to organize their time and the dedication needed to be successful in both mastery-based and traditional courses (Eyre, 2007).

Despite the relevance and potential value of mastery learning for computer science education, literature specific to this domain is scarce, and the limited efforts to adopt mastery learning in computer science have been carried out in isolation of each other. Recently, Garner and colleagues (Garner, et al., 2019) conducted a review of mastery learning in Computer Science, including literature published between 1992 and 2017. Considering twelve studies in this time frame, it was identified that motivators for implementing mastery learning include the possibility of improving teaching for diverse student cohorts, addressing the ‘learning edge momentum’ problem, guaranteeing skill mastery, and contributing to the mastery learning literature as well as other case specific reasons. Obstacles to the implementation of mastery learning include student procrastination, scaling delivery and developing assessments. Many experimental approaches have been observed in distance learning, automated assessments, personalized feedback, and each program has had to solve similar problems. Moreover, few studies focus on introductory computer science courses; only five studies were reviewed, with a duration no longer than two semesters. The review concludes that implementation of mastery learning in computer science has been based on the general education literature, and that there is still a lack of an authoritative body of scholarly knowledge to guide curricular innovation, implementation and research of mastery learning in computer science.

Course modularization for mastery learning in CS1

The notion of dividing a full-semester course into discrete modules has been discussed by many authors beginning in the 1960s (Goldshmid & Goldshmid, 1973). Modularization emerged as a methodology for curricular and instructional design in European higher education, emphasizing the possibility for students to have a high degree of customization in their curriculum (e.g., presenting students with catalogs of modules with which to configure their courses) and to develop autonomous learning skills (Dochy, et al., 1989; van Eijil, 1986; van Meel, 1993). Nonetheless, there are a variety of applications and ways of implementing modularization according to the characteristics of the target educational problem and context (Jenkins & Walker, 2014).

For the present effort, modularization of a computer programming course offers the possibility to implement mastery learning in a scalable way, within the constraints of the semester-based academic activities typical of higher education institutions. A modular design permits dividing a traditional course into modules. Each module has its own learning goals and students can work on the appropriate module according to their mastery level. From the operationalization standpoint, modules can be taught multiple times each semester, performance outcomes can be quickly tested and evaluated, and teaching can be continuously improved in terms of instructional strategies and assessment methods. With regard to the common mastery learning issue of procrastination (Fox, 2004), a modular design that prescribes dates of both formative and summative assessments can drive students to focus their attention on the course and strive to meet the required mastery standard in a predictable time frame. With this rationale and expectations, the present authors persuaded faculty leadership at a research-based Latin American university to support transformation of an introductory programming course into a modular format for mastery learning.

Research framework and questions

Figure 1 depicts the methodological framework underpinning the present study, supporting the iterative process towards implementing and improving a modular CS1 course for mastery learning. The framework comprises a four-step action research methodology as described by Cohen et al., (2013) drawing on Zuber-Skerritt, (2003). The action research process focuses on improving relevant aspects of teaching and learning as identified by Biggs, (2011) in his ‘constructive alignment’ framework, which include the course’s intended learning objectives, its teaching and learning activities, and assessment tasks (see components A–C in Fig. 1). In addition, instructional design, course content and skills, and course management are three aspects identified by Fink, (2013) which we consider central in the methodology of the present research (see components D–F in Fig. 1). Given these components of interest, the steps of the action research cycle include (1) strategic Planning, wherein the A–F components are designed, planned, and revised according to relevant learning theories, and a research agenda aligned with the educational problem is crafted, (2) Action, i.e., plan enactment, (3) Observation, evaluation and self-evaluation of the course implementation with regard to components A–F, and (4) critical and self-critical Reflection on the results of points 1–3 deriving in decision making for the next action research cycle.

Fig. 1
figure 1

Action research cycle towards implementing and improving the modular course for mastery learning

Full size image

Based on the discussion presented in the previous sections, and the methodological framework that underpins the current research, the research questions addressed in this paper are the following:

  1. 1.

    What challenges and difficulties arise in the process of implementing a CS1 modular course for mastery learning with regards to the instructional design, content and skills, teaching and learning, assessment and course management, and how are these overcome in successive action research cycles?

  2. 2.

    How do students perform in the CS1 modular course for mastery learning? How diverse are their module progressions and how do these evolve as a result of interventions conducted within the action research process?

  3. 3.

    What practical implications and recommendations emerge from the experience to support successful implementation of modular CS1 courses for mastery learning?

Educational context

The research was conducted in the engineering school of a Latin American research university. The engineering school offers undergraduate programs in five areas: Industrial, Civil, Electrical, Computer Science and Environmental Engineering. As of 2020, only 7.6% of the students enroll in the Computer Science program, however, the CS1 course is compulsory for all first year engineering students, regardless of program. The intended learning objectives of the course are that by its completion students should be able to:

  • Explain key concepts involved in procedural computer programs, namely: input and output, data types, variables, operators, control flow, function invocation and return values, random access data structures (i.e.., simple and multidimensional arrays), strings, dictionaries, file access, and recursion.

  • Model algorithms based on a procedural model of computation, specifying the sequence of steps required by use of a graphical language.

  • Model simple recursive algorithms comprising base cases and recursive steps.

  • Write procedural programs in a high-level, text-based programming language, including the above-listed concepts.

  • Write programs that involve numeric computation and data visualization capabilities based on 2D plots.

Previous version of the CS1 course

The version of the CS1 course that existed prior to the present study was based on a semester format comprising the following weekly activities: two lectures, a recitation session, and a lab assignment. Assessment was based on weekly low-stakes assessments during lecture hours, weekly graded lab assessments and homework. In addition, the course incorporated a final exam worth of 30% of the final grade. It was customary to offer students who did not pass the exam but still reached a certain grade cutoff the ability to take a recovery exam and still pass the course. Under this scheme, the failure rates in the previous version of the course fluctuated between 30 and 40% in the last three years of its implementation prior to the study.

Student cohorts under study

The student cohorts that participated in the current study are described in Table 1, including their size and gender composition. Annually, student cohorts are comparable with an average of 80% male and 20% female students. The secondary education origin of students has also remained stable, with 78% to 83% coming from private secondary education institutions. Another 8–14% of the students come from Publicly-Funded Private Schools (PFPS). A minority of the students, less than 5%, come from public schools. In the national context, 37% of high school students attend a public school, 48% attend PFPS schools and 14% attend private schools. In addition, only 33% of students that attend public schools enroll in higher education, while 76% of students who attend private schools enroll in higher education (OECD, 2015).

Table 1 Student cohorts under study
Full size table

Course design features

The course developed in this study is based on the following design features, which respond to the defined learning outcomes and a mastery learning approach:

  1. 1.

    Modular course architecture for mastery learning: The intent of the modular architecture is to permit students to follow different paces of module progression in the course according to their learning capabilities, while achieving mastery of each relevant course unit (module) before being promoted to the next module. For students to demonstrate attainment of mastery, each module includes a summative examination at the end. To make the course must be operational within the structure of a semester-based calendar in the host institution with sixteen weeks of classes per term, it was chosen to structure the course into four successive modules with equal duration of three weeks. This makes possible teaching the modules in parallel, and the students can repeat a module in the first semester and still be able to complete the course in the term. Also, students do not need to wait to resit a module if they fail, as parallel modules can start and end in the same dates.

  2. 2.

    Foster algorithm design skills before coding: According to Koulouri et al. (2014), learners' analytic capability to logically decompose problems has a positive effect on learning programming, regardless of the programming language being used in the process. In the previous version of the course, problem analysis and algorithm design skills were taught as the initial content unit in the first week of the course. Arguably, these skills were not given sufficient attention as they were taught superficially, and only evaluated by low-stakes assessments. Thus, for the modular course, it was considered that time dedicated to problem analysis and algorithm design needed to be increased, as well as the need to adopt a more rigorous assessment of these skills.

  3. 3.

    Imperative procedural programming: For decades there has been debate about the order in which programming skills should be taught (Luxton-Reilly, et al., 2018); that is, if object-oriented analysis and design should be taught first, i.e., an “objects-first” approach, or if prior to that, students must master procedural programming. Faced with these alternatives, the present authors had developed the preceding course based on an imperative-procedural approach, emphasizing top-down problem analysis, and algorithm design based on the use of variables, logical and arithmetic operators, domain of flow control structures, use of functions, arrays and dictionaries. The modular course here presented maintains this tradition, with object-oriented analysis and design skills being taught in a later course in the curriculum.

  4. 4.

    Performance-based assessment: Assessment sends a strong message to students about what counts as knowledge, insofar students' perceptions of the requirements of the assessment influence their approach to learning (Ott et al., 2016; Weurlander & Soderberg, 2012). The current authors consider that performance-based assessment can therefore foster deep learning of course topics, as students are required to produce working programs in their solutions to given problems, and in this process, integrate various programming skills. Therefore, assessment problems in the modular course are based on ‘multistructural-applying’ and ‘creating’ categories of the taxonomy proposed by Meerbaum-Salant et al., (2013).

  5. 5.

    Situated problems: A student's intrinsic motivation for computer programming can be influenced by their perceived real-world applicability of knowledge and skills (Dorn & Tew, 2015; OGrady, 2012). This is salient in educational contexts in which students have diverse interests, as is it the case in the current study's educational context. Furthermore, from a constructivist standpoint, problem understanding can be facilitated if students can anchor problems to their real-world experiences and prior knowledge (Ben-Ari, 2001). Thus, both practice and assessment tasks in the modular course involve problems in which students are encourage to ‘think as engineers’ in a particular scenario.

  6. 6.

    Pursuit of Constructive Alignment: The theory of constructive alignment (Biggs, 2011) is based on the idea that learners use their activity to construct knowledge as interpreted through their own existing schema, and that assessment tasks should be aligned to what it is intended to be learned. The instructor’s duty is to set up a learning environment that encourage students to perform learning activities that align with the intended outcomes, and to assess student performances against the latter. The relevance of constructive alignment, its implications and potential enhancement of teaching and learning have been emphasized by some authors in computer science education literature (Bayu Bati et al., 2014). The modular course here proposed aims at achieving constructive alignment among learning outcomes, teaching and learning activities and assessment tasks.

Course iterations

Iteration 1

Planning

A design of four successive Modules (M1-M4) of three weeks each was adopted in both semesters of the academic year (see Appendix A a). The teaching staff was led by a coordinator (first author of the present study), and included four lecturers. The coordinator planned the course schedule, as well as the evaluation and course material development plan. All teaching staff had to contribute to the development of problem sets for tutoring sessions, lab assignments, and exams during the semester, according to the material development plan. The course coordinator monitored these duties and oversaw the distribution of the materials in addition to their work preparing the lecture material (which included slides sets and code samples). Grading was conducted by 20 Teaching Assistants (TAs), directed by a chief TA who reported to the course coordinator. The number of lecturers and TAs was maintained from the traditional course.

An example of how the modular course can evolve throughout an academic year is shown in Fig. 2. The horizontal axis shows the division of the academic year (and semesters) into Time Blocks (TBs). A TB within a semester spans for three weeks, so five TBs are commonly allotted within sixteen weeks of classes in a semester. The vertical axis denotes the modules that are active (given) in the TBs. Initially, all students start at M1 in TB 1. As students pass and fail modules through the TBs, different modules need to be taught in parallel. By the end of TB 4, a part of the cohort may have completed all four modules, thus being the first students to pass the course, accomplishing so in 12 weeks. The rest of the students continue to complete the course in successive TBs, thus taking a varying amount of TBs to pass. From one TB to the next, lecturers can be reassigned by the course coordinator to teach a different module, as the number of students requiring attending each module varies from one TB to the next. For planning purposes, and considering classroom capacity restrictions, the number of lecturers allocated to teaching each module had to be decided before the start of every TB, as module pass rates were unknown in the first iteration of the modular course.

Fig. 2
figure 2

Modular course evolution example

Full size image

In Iteration 1, similarly to the example shown in Fig. 2, an inter-term period (TB 6) was planned to offer only the final (fourth) module of the course, and thus allow students with the last pending module to complete the course before the following semester. Enrollment in the inter-term period was made optional. It was expected that by the end of the first semester, most of the students would have reached or passed M4. In the second semester the modular course was continued for an additional five three-week TBs (i.e., TBs 7–11). Students that were not able to pass the course in the first semester were allowed to resume the course starting with the module following the highest one they had passed.

In each assessment, as well as for calculation of final module grades, the grading system followed local conventions using a continuous scale from 1.0 to 7.0, with a passing level of 4.0 (50%). Grades are absolute and not curved.

Action

The structure of all course modules across iterations, including teaching and learning activities, and assessments, is depicted in Appendix A. The first two weeks of classes each included a lecture, a tutoring session and a lab assessment. Between the lecture and the tutoring session, students were expected to dedicate self-directed study time to solving a homework assignment and solving practice problems. In the third week of the module, the students sat the summative examination of the module instead of a lab assessment. Table 2 shows assessments and their weights per each module, in the first three iterations of the course.

Table 2 Course assessments and weights in the first three course iterations
Full size table

The contents of the course modules are shown in Appendix B, along with their evolution throughout successive course iterations. In the first iteration, M1 encompassed the basics of computational thinking, including problem analysis and algorithm specification. The students were taught the latter skills through drawn flow charts. Then, computer programming based on Python 2.7 was introduced, including basic input–output, variables, operators, pseudo-random numbers and the selection statement. In M2, more advanced flow control (including loops and flow control nesting) and functions were taught. M3 introduced lists, nested lists, and file access. Finally, in M4, other data structures were taught, together with Python’s numeric computation library (NumPy), and 2D plots.

Following common recommendations in the mastery learning literature (Fox, 2004), lectures were intended as motivational with the aim to provide students with key concepts and demonstrations to get started learning on their own. Because of this, lecture time was reduced to 50% of the time that was allotted in the former course. After the lectures, students were provided with readings, videos, and problem sets to support their personal study. Then, in tutoring session, students were presented with a set of increasingly complex problems, which they could solve supported by a TA and peer collaboration. These problems were not graded. The following day, the students had to attend a graded laboratory assignment (weeks one and two) or the final module exam (week three). In the first iteration of the course, in M2 to M4, the students had a homework assignment lasting seven to ten days. These assignments were optional, and counted in the final course grade with 16.6% weight if their average surpassed the weighted average of other course assessments.

With the intent to provide students with instant feedback on their performance, it was decided to implement automatic grading in most of the lab assignments. For this, contest management system (CMS; Maggiolo & Mascellani, 2012), a platform utilized in competitive programming environments, such as the International Olympiad in Informatics, was tailored to the course’s needs. Following the intent to give assessments of a situated nature, problem statements each described a particular scenario. The problem itself was divided into subgoals of increasing difficulty, each involving a number of test cases. Examples were provided for each subgoal, showing the corresponding data inputs and outputs. Upon submitting a response, each successful test case awarded points to the student.

The final summative exam in each module required students to demonstrate mastery of the knowledge and skills seen in the course up to that time. Problem statements described a scenario and functional requirements of the solution to be implemented. Students were always asked to write a working program that solved the given problem. Grading of examinations was always conducted by the Tas following a common rubric.

Observation

Students’ performance in the first iteration of the course was well below expectations at the outset of the intervention. By the end of the first semester, in TB5, there were students still in all four course modules (see Table 3), and only a small fraction of students had passed the complete course.

Table 3 Proportion of students in each course state by the end of the first (TB5) and second (TB13) semesters in iteration 1
Full size table

In the first semester, pass rates were alarmingly low in M1 and M2, especially the first time students attempted each module (see Table 4). The first time M1 was given (i.e., in its first instance), only 37.6% of the cohort passed, thus the majority of the students had to repeat it (i.e. attempt passing the module a second time). Given the low pass rates in modules 1 and 2, only a small percent of the cohort of students were able to reach modules 3 and 4 the first time they were offered. This smaller number of (high-achieving) students can help explain the higher student success in these modules in their first offering.

Table 4 Module pass rates for students’ first, second and third attempts in their first three instances in iteration 1
Full size table

Students’ average time to pass all four course modules in the first iteration was 23.2 weeks (SD = 7.38), close to eight TBs, considering both semesters and the inter-term period. The course pass rate in the first semester was only 19.3%, but including students who passed M4 in the inter-term period, this figure increased to 28.5%. Thus before the start of the second semester, slightly below a third of the original cohort had finished the course.

At the start of the second term 46.9% of the original cohort was in M3 and M4, and 10.1% of the students had dropped the course. The number of students that passed the course increased steadily in the second semester, reaching 66.7% by the end of TB 11. As in the previous semester, M4 was offered in an extra time block (TB12) in weeks 33–35. This increased the overall course pass percentage to 75% (see Table 4). The overall percentage of students who dropped the course doubled in the second semester, reaching 21%. Nearly half of this increase (i.e., 5.7%) was due to students failing to pass M3 in the last TB it was given (TB11). Lastly, a remaining 4.4% of the cohort in M4 did not drop but was unable to pass the course by the end of the year.

Reflection

The reflection step in the first iteration involved meetings with course Tas and the teaching staff, and conducting an interview process with students. Purposeful sampling was used, selecting ten students with different levels of achievement in the course, including students who had passed the course in the first semester, students who had failed the course in the first semester, and students who dropped out of the course. Based on these information sources, different issues were noted, which provided explanation for the poor results obtained in the first iteration of the modular course. The issues are summarized in Table 5.

Table 5 Issues found in iteration 1
Full size table

One of the main issues found was that the methodology used in M1 to introduce computational thinking and algorithm design did not meet the expected results. Table 4 shows meager M1 pass rates in its first three instances. In the light of academic results, and students’ testimonies in interviews, it was considered that both pedagogy and assessment required major changes (e.g., see issues AS1, CSK1 in Table 5).

Iteration 2

Planning

In Iteration 2 the schedule of the academic year remained similar to the previous iteration, however, due to limitations of the academic calendar, it was not possible to accommodate six TBs in the second semester but only five. The teaching team remained stable except for a lecturer who was replaced by a doctoral student with prior teaching experience in a traditional CS1 course. With regard to course content, it was decided that rework of M1 was necessary, considering the need to adopt more effective ways to teach the initial skills of the course. In planning the second iteration of the course, most of the issues described in Table 5 were addressed through a decision making process in which the course coordinator consulted the teaching staff, including both professors and teaching assistants, for ways to overcome each issue (Table 6).

Table 6 Actions performed in Iteration 2 to address issues found in Iteration 1
Full size table

Action

A major part of course improvement efforts in Iteration 2 were directed at addressing the pedagogical issues in M1. To deal with these, it was decided to adopt the approach of teaching visual programming with interactive tools instead of traditional hand-drawn flowcharts, before introducing text-based programming. Two block-based visual languages were considered, including code.org (Kalelioğlu, 2015) and MIT Scratch (Meerbaum-Salant et al., 2013). Scratch was considered more convenient as it focuses strictly on visual programming, while code.org draws parallels among block-based and text-based programming; the latter in ECMAScript language, which differs from Python.

With Scratch, in less than three weeks’ time, students could learn basic problem analysis and how to generate programs with variables, input and output, operators, logical and arithmetic expressions, pseudo-random number generation, conditional and iterative flow control, and use of lists. In addition, the teaching of Python in M1 was simplified, moving basic flow control (i.e., the selection statement) to M2 (see Appendix B).

Observation

In the second iteration, students’ progress through the course was notably faster than in iteration one, with an average time to finish the course of 20.5 weeks. This is a reduction of 2.7 weeks (SD = 6.58), i.e., almost a complete TB, compared to the first iteration (M = 23.2, SD = 7.38). Correspondingly, module repetition was reduced.

By TB5 there were no students in M1, and 3% of the cohort were in M2 (see Table 7). This contrasts with Iteration 1, as in TB5 7% of the cohort were in M1 and 13% in M2. By the end of the inter-term period (TB6), 47.4% of the cohort had passed the course in iteration two, compared to only 28.5% in the first iteration. In addition, 89% of the cohort completed the course by the end of the academic year (see Table 7), compared to only 75% in the previous iteration (see Table 3).

Table 7 Proportion of students in each course state by the end of the first (TB 5) and second (TB 11) semesters in Iteration 2
Full size table

Improved results in Iteration 2 with regard to student progress, were the result of increased pass rates in M1 and M2 (see Table 8). On the other hand, no major changes in teaching and format of summative exams were introduced in M3 and M4, thus, results in these modules did not improve noticeably compared to the first iteration. However, like in the first iteration, students who passed M3 performed well in M4.

Table 8 Module pass rates for students’ first, second and third attempts in their first three instances in Iteration 2
Full size table

Table 9 shows homework and examination grades considering the first instance of each module. Greater student dedication to homework assignments was observed in Iteration 2. The percentage of students handing in their homework across the different modules ranged between 84.0% in M3 and 93.7% in M2. However, homework grades were consistently higher than examination grades in M1-M3, and the difference was verified to be statistically significant in these cases, with large effect sizes. In addition, correlations among homework and examination grades were poor, ranging between 0.14 and 0.28. This may relate to different conditions under which students develop homework assignments compared to exams; in terms of time, and the possibility to seek help from others. Notably, in M4 students did better in the exam than in the homework assignment, while the mean of homework was the lowest of all modules.

Table 9 Homework (HW) grades vs exam grades in the first instance of M1-M4 in iteration 2
Full size table

Reflection

After reflections involving the teaching team, teaching assistants and data analysis of students results in each module, we were able to see that the changes to the course in Iteration 2 succeeded at improving module pass rates at the beginning of the course, which accelerated students’ overall progress in the course. The course was improved by taking into consideration the many issues found in the past iteration. However, pedagogical and course management aspects could be improved further. Table 10 shows salient issues found in iteration two, which related to teaching and learning and course management.

Table 10 Issues found in iteration 2
Full size table

Iteration 3

Planning

Planning of the course remained similar with regards to TBs in each semester. Like in the previous iteration, only one member of the teaching staff having two years’ experience in the modular course was replaced by a young lecturer with a few semesters experience teaching traditional CS1 courses. In order to cope with issue CM3 raised in Iteration 2 (see Table 10), planning of assessment construction had to be carefully negotiated by the course coordinator with each member of the teaching staff.

Action

Table 11 summarizes the actions performed in Iteration 2 to overcome issues TL4, TL5 and CM3. In addition, the Python language was upgraded to version 3.6, as version 2.7 was scheduled to sunset in January 2020. Syntactic and semantic changes among language versions are minor, especially as the present programming course only focuses on the imperative procedural paradigm. However, this required a comprehensive update to the courses’ textbook, class material and code examples.

Table 11 Actions performed in iteration 3 to address issues found in iteration 2
Full size table

Observation

Students’ progress was further improved in the third iteration, with a mean time to complete the course of 17.7 weeks (SD = 5.60). This is an improvement of an additional 2.8 weeks over Iteration 2, due mainly to an increase in module pass rates. As shown in Table 12, by the end of TB5, 54% of the cohort had passed the course. After the inter-term period, this figure increased to 67%. By the end of the year, 94% of the cohort passed the course, and only 6% dropped out.

Table 12 Proportion of students in each course state by the end of the first (TB 5) and second (TB 11) semesters in iteration 3
Full size table

Quicker student progress in the course was due to pass rates in M1 continuing to be above 0.7 in the first two instances the module was offered, and later improvements to pass rates in M2 and M3. Lastly, M4 maintained relatively high pass rates comparable to the second iteration (see Tables 12 and 13).

Table 13 Module pass rates for students’ first, second and third attempts in their first three instances in iteration 3
Full size table

Reflection

Iteration 3 showed that positive effects of changes made in Iteration 2 could be replicated, and that further improvement was achieved in the light of greater pass rates observed in M2 and M3. As pass rates in M2 and M3 were consistently improved in several of their instances, it can be affirmed that changes to pedagogy linked to addressing TL4 and TL5 (see Table 11) had a positive influence on students’ learning and performance. However, no evidence could be elicited regarding achievement of a better alignment among laboratory assignments, homework and examinations, as a result of addressing issue CM3. A path analysis procedure was conducted as a means for establishing whether results in an assessment could predict results in later assessments. This was based on the notion that an assessment could predict performance in later assessments if students manage to build and apply schemata and transfer knowledge related to the intended learning outcomes throughout successive, aligned learning and assessment events (Robins et al., 2019). The procedure was performed with R v4, considering assessments in the first instances of modules M1 to M3. In our case, with all modules it was found that only the first lab assignment could predict performance in the second lab assignment and in the summative exam (i.e., path coefficients in the range 0.44 to 0.58, p < 0.01), although the latter to a lesser extent (i.e., path coefficients in the range 0.02 to 0.3, p < 0.05). The second lab assignment could not predict performance in homework nor in the summative exam. In turn, homework did not predict lab assignment nor examination performance (see Appendix C). This provides indication that further improvement with regard to constructive alignment can be pursued by increasing opportunities for knowledge transfer, schemata activation, and display of meta-cognitive strategies across homework, lab assessments and the summative exam.

Table 14 summarizes the issues found in Iteration 3, which relate to teaching and learning, assessment and course management. Overcoming these issues was considered essential in order to improve the course both from operational and academic standpoints.

Table 14 Issues found in iteration 3
Full size table

Iteration 4

Planning

The fourth iteration of the course began in the midst of the worldwide COVID-19 crisis. In the first week of the course, the university complied with the regulations imposed by the local health authorities and the decision to teach online throughout the year was made official. The course had to be quickly adapted to this format.

The annual calendar of the course did not undergo changes compared to the previous year, keeping the number of weeks, TBs and the inter-term period unchanged. There was a 15% increase in freshmen enrollment compared to the previous year, so a sixth lecturer was added to the teaching staff. In the second semester the teaching staff was reduced to two lecturers, as in all course iterations.

Action

With the change to online education, synchronous activities such as tutorials and lectures were streamed on platforms such as YouTube and Twitch, or conducted by using videoconference systems, such as Google Meet. The lectures were reduced from 100 to 80 min, and their focus continued to be about demonstration of skills and step-by-step problem solving by the lecturers. In order to improve students’ preparation for the lectures, the teaching team produced two to four short videos (i.e., three to ten minutes long) per week, 48 in total, to cover the fundamental contents covered in each lecture. In previous iterations of the course, there was a limited number of video capsules, used mainly in M1, but these were completely renovated. In a flipped classroom fashion, students were required to watch the video capsules before each class, and read sections of the textbook.

The course began with all six lecturers teaching M1, with groups of students assigned to each as had been done for the face-to-face format. Given that the online format allows assigning an unlimited number of students to each teacher, starting from TB2, the teaching staff that gave lectures was restricted to three or four lecturers per TB. Thus, about half of the teaching staff was dedicated to lecturing, and the other lecturers had greater dedication to developing assessments and short explanatory videos. Dedication to lecturing and content creation duties were planned by the course coordinator before the start of each TB.

With regard to assessment, changes were made in Iteration 4 (see Table 15), with the aim to reduce the number of students’ requests for regrading at the end of each module, and thus lessen the burden this placed on teaching assistants at the end of every TB (i.e., issue CM4). In M1 and M2 grades were based only on class attendance and the summative exam. Homework assignments in these two modules were eliminated. Lab assessments were administered in M1 to M3, but were not compulsory. Rather, students could score bonus points to boost their final grade in the module (but were not allowed to request regrading). In M3, homework was kept in a similar format to the previous iteration and worth 12.5% of the final grade, and regrading was allowed. In M4, an integrative homework assignment, prompting students to comprehensively apply course knowledge and skills, was introduced, accounting for 22.5% weight in the final module grade. Given that this homework assignment was expected to demand a greater effort, lab assignments were omitted in M4 altogether.

Table 15 Course assessments and weights in Iteration 4
Full size table

The modules were taught with the same knowledge and skills ordering as in the last iteration. This allowed all the assessment material from the previous year to be completely reused in the form of worked examples so that the students could improve their preparation for the assessments in tutorials and in their personal study.

Changes in Iteration 4 were mostly focused on implementing the course in online format, however, it was feasible to address the issues that emerged in the past iteration to some extent (see Table 16). In the second semester, an attempt was made to improve student engagement with the support of the academic counseling model traditionally implemented in the institution. This consists of each first year student having an assigned academic counselor, generally a full-time academic. The secretary for student affairs of the Faculty was asked to coordinate the appointment of video calls so that the students could meet their counselors, and be encouraged by them to finish the course as soon as possible. Regarding the aligned development of the assessments to facilitate knowledge and skill transfer, the criterion of the previous iteration was maintained, consisting of the same teacher having to develop homework and exams in M3 and M4 (Table 17).

Table 16 Actions performed in Iteration 4 to address issues found in Iteration 3
Full size table
Table 17 Proportion of students in each course state by the end of TB 5 in Iteration 4
Full size table

Observation

In Iteration 4 the improvement trend with regard to students’ mean time to finish the course was maintained. This was figure was reduced to 14.9 weeks (SD = 3.64), that is, an improvement of 2.84 weeks (almost a complete TB) compared to Iteration 3 (M = 17.7, SD = 5.6). In addition, 96.2% of the cohort completed the course by the end of the academic year, compared to 94.3% of students achieving this result in the previous iteration (see Tables 12 and 17).

Pass rates on M1 and M2 continued to improve (see Table 18). In Iteration 3 the same pass rate was found for M3 as the previous year in the first instance, but it worsened considerably in the second instance. In M4 there was also a worsening in the pass rate in the first instance. Nonetheless, at the end of TB5, 77% of the cohort passed the course, i.e., an increase of 23% compared to Iteration 3, due to a greater proportion of students in M4 in TBs 4 and 5, explained by better pass rates in M1 and M2.

Table 18 Module pass rates for students’ first, second and third attempts in their first three instances in iteration 4
Full size table

Reflection

The best academic results in Iteration 4 were obtained in a teaching context different from that of the previous modules. Therefore, the improvement in academic results can be attributed to both the deliberate improvements pursued, as well as other factors beyond control of the teaching staff and research agenda.

Undeniably, students in the online mode had advantages over those who previously took the classroom-based course at the time of taking assessments. During the assessments, they could access study material, assessments from previous semesters, and other resources. On the other hand, students in the face-to-face format were only allowed to consult the course’s textbook during the assessments. In spite of this, it could not be established that there was systematic cheating and plagiarism on the part of the students when taking the assessments in the online course. At the begining of the course, it was announced to the students that all assessments would be reviewed with Measure of Software Similarity (Aiken, 2020). In spite of this, in TB2, 23 students were found (8.8% of the cohort) to be suspected of cheating on their assessments due to code similarity. After investigation, it was determined that 11 students cheated in lab assignments or examinations. After these students were informed of the sanctions, which consisted of failing the course in most cases, plagiarism in the cohort appeared to decrease substantially (no new suspicions of plagiarism were detected in the following TBs).

The lifestyle of the students was also considerably affected by the pandemic and this could have positively influenced their performance in the Programming course. Under normal conditions, most students spend considerable time commuting to campus from their homes. On the other hand, the University does not have laboratories open to students permanently to facilitate their study of programming, therefore, students must have their own laptop to study on campus. During the pandemic, students spent time at home in front of their computers for most of the day, so this could have facilitated their dedication to the study of programming.

Students’ module progressions and performance

A longitudinal analysis of students’ performance and module progressions in the course was performed by consolidating academic results in a relational database, and constructing reports through a development environment based on the R programming language. A student’s module progression is defined as the sequence of modules followed by the student throughout the course’s TBs, from start to end, and is specified by the corresponding sequence of module numbers, from left to right. For example, module progression ‘112345’ indicates that the student took M1 twice (i.e., failed M1 in the first TB), and continued studying and passing the following modules until passing M4 and thus achieving course completion by the end of the fifth TB. The number 5 in TB six of this progression indicates that the student had passed the course already.

Figure 3 shows the relative frequency of students’ module progressions in each iteration of the course, only considering the first five TBs (i.e., the first semester). A total of 31 different module progressions were found across the four years. For the sake of clarity in Fig. 3, module progression labels are only provided for progressions including at least 3% of the students in the yearly cohort. A decreasing trend can be observed through the years in module progressions ending in 1, 2, and 3 (i.e. students who did not pass the course in the first semester). Conversely, an increase is observed in progressions ending in 5 (students who did pass the course).

Fig. 3
figure 3

Students’ progressions in the modular course

Full size image

Table 19 shows trends in the different progressions (module sequences) at the end of the first semester (TB5) in each year (iteration). Trends were tested for statistical significance through a chi-squared test of proportions. Looking at progressions ending in M1 (i.e. 111111), in the first year, 10% of the cohort were still in M1 by the end of TB5. In the following two years, the proportion of students in this state was reduced to a minimal fraction of the cohorts (2 to 3%), and in the last year, there were no longer students who did not pass M1 by TB5. A similar trend is observed for progressions ending in M2 (e.g. 111222, 112222), as 13% of the students in the first year were still in M2 by the end of TB5. In contrast, in the last year only 1% of the cohort had not passed M2 by the end of TB5.

Table 19 Trends in module progressions organized by module at the end of TB5 by year
Full size table

A considerable portion of the cohort in the first two years followed progressions that were still in M3 at the end of TB5 (e.g. 112333, 122333). These results were improved in the third year, as 70% of the cohort managed to pass M3 by the end of TB5. In the last year, only 5% of the cohort had failed to pass M3 by the end of TB5. Lastly, for progressions ending in M4 (e.g. 122344, 123344), the same proportion of students in the first year of the intervention as in the last. However, the proportion of students who managed to pass the full course at the end of TB5 increased steadily from year to year, reaching 80% in 2020.

Figure 4 presents the proportion of students who passed the course every year, starting at the conclusion of TB4. While this proportion increased in each iteration, a consistent trend of slow progress occurred in every second semester. Students’ progress stagnates in TBs 7–9 in the second semester, and improves in the last two TBs. Consistently with other mastery learning implementations (Fox, 2004; Garner et al., 2019), when the students are given too many opportunities to fail and retake assessments, or are not faced with deadlines to be promoted to more advanced studies, they tend to procrastinate. Procrastination in the modular course became apparent in the second semester, as most students begin thereby in M3 or M4, and have a total of five to six TBs available in the semester to pass one or two modules. Late in the second semester, procrastinating students encounter pressure to finish the course in the last few TBs ahead. Despite this, the proportion of students ending the following academic year without completing the course steadily dropped through course yearly iterations, from 25% of the cohort in 2017 to 3.8% in 2020.

Fig. 4
figure 4

Proportion of students who passed the course starting at the end of week 12 (TB4) in every course iteration

Full size image

Discussion

Challenges and difficulties

Introducing fundamental knowledge and skills

The fast-paced planning of the modular course made the first module intense, as students are expected to develop basic problem analysis, abstraction and algorithm design skills in no more than three weeks, while taking other parallel courses in the engineering curriculum. How skills are taught and assessed early in the course is of the utmost importance for students’ academic success, as early course results impact their motivation and ability to make sustained progress in later modules. Finding ways to accomplish fruitful learning and successful academic results in M1 proved to be a challenge at the outset of the project. Clearly, the use of a visual block-based programming language in Iteration 2 onwards was found to be a more effective alternative than traditional hand drawn flowcharts, to support students’ own thinking and solution modeling. This, however, was not obvious at the outset of the project and had to be realized by the teaching team after the results of the first iteration.

Managing increasing knowledge complexity and assessment difficulty

The epistemology of programming results in discernable pedagogical challenges in the modular course. As the course progresses, knowledge and skills that students are expected to master become more complex, while problem solving activity requires that students integrate and apply a growing number of skills. As a greater number of programming concepts and skills become available for formulating more sophisticated problems, creative possibilities multiply. Yet for the teaching staff, experience with the modular course design led to the realization that good assessment construction is not about presenting students with entirely new and unexpected problems in assessments (i.e., ‘gotcha problems’). Instead, assessments need to consist of problems in which students can identify and transfer schemata they have had a chance to build over their hours of prior study and practice, including with worked examples. Neglecting this results in undermining constructive alignment possibilities between teaching and learning activities, and assessment.

As suggested by Robins et al., (2019), a programming pedagogy that better adjusts to students’ cognition should consider worked examples, and problems establishing subgoals. In addition, for students to develop knowledge transfer abilities, they must be presented with examples of how a problem-solving strategy can be transferred from one problem to another. These are pedagogical challenges especially for novice teachers of programming, who as in other disciplines tend to replicate the ways in which they were taught and assessed in their own educational experience.

Student procrastination

Consistent with the literature on mastery learning and reported experiences in mastery learning (Fox, 2004), student procrastination is an issue that occurred in the present modular course. This behavior is observed in Fig. 4, where a plateau in the progress of the students remains evident in each second semester. A possible workaround to the procrastination issue in the second semester is to introduce the students incentives upfront at the start of TB7, so that they commit dedication and effort to pass the course as early as possible. A maximum number of reattempts to pass each pending module could be defined. For instance, three reattempts could be allowed for M3 and M4 in the second semester. That is, if the student fails a module for the third time, fails the course. Rules such as this will be tested in future course iterations.

Team coordination and teaching freedom

Unlike traditional courses, in which the course can be taught by a single teacher, under the modular approach, several teachers need to collaborate. As the semester progresses, modules need to be taught in parallel, and in every time block each module requires different learning activities, content and assessments. Therefore, transition from the traditional course format to the modular requires a culture wherein the teaching staff is willing to relinquish some of their individual teaching freedoms for the sake of the collective effort that underpins the modular course. In this regard, the role of the coordinator proved essential in the modular course, not only for planning and assigning teaching staff’s duties and supervising course activities in a daily basis, but also for collaborating with the teaching staff in creating course content, and assessments well-aligned with prior teaching and learning activities for each module.

Practical implications and recommendations

Planning of learning activities

The modular system presented here is based on the division of the academic term into time blocks of fixed duration. This facilitates that student finish a given module and move on to the next without waiting. However, the division of the course into four three-week modules makes the rate with which students are exposed to the content faster than that of a traditional course. In the modular course, a student who passes all modules without failing any of them can finish the course in 12 weeks, versus 15 weeks in a traditional semester course. With this accelerated pace of learning, planning of learning activities must be organized carefully so that sufficient time passes after the student is presented with basic skills in order for them to exercise the skills properly and solve enough practice problems before they are assessed. On the other hand, the modular course must offer differentiated learning activities for students repeating a module from those taking a module for the first time. Repeating students can spend less time in lecture and focus most of their time exercising on problems from previous assessments, problems they were unable to solve before, and have further opportunities to seek for help and guidance from more capable peers and the teaching staff.

Block programming before text-based programming

The process of learning computational thinking and programming demands the concurrent display of a complex mix of skills, including reading comprehension, problem analysis, abstraction, and elaborating the solution representation. According to learning edge momentum theory (Robins, 2010), mastering these basic skills is key to academic success in the course. Traditional ways in which computational thinking and algorithm design skills have been taught in CS1 courses have been based on the generation of pseudocode, flowcharts or the use of visual modeling tools. More recently, the use of block programming languages, such as MIT Scratch and code.org, has become common, especially in K12 education. Despite the fact that these languages are frequently used for recreational purposes (e.g., for creating video games and multimedia), experience in the modular course shows that they are an powerful means for introducing post-secondary students to algorithm design and programming. Through these languages, students do not have to type code, including all the syntactic subtleties involved, such as indenting code with tabulation, and correctly typing each line. Instead, students use programming blocks that are easily recognizable (i.e., visual, with definite shapes and colors), and syntactically helpful (pluggable). The fact that with block programming students do not have to type code from the very beginning in a blank file brings the benefit of offloading some of the cognitive load of the task, so that they can rather dedicate more cognition and effort to model the solution, test it, and try different ways to solve the problem. In addition, other desirable skills, such as program tracing and debugging can be developed by the students early in the course through the use of block-based programming. The introduction of block-based programming and text-based programming worked well in a spiral fashion, meaning that students can be introduced to text-based programing through the same examples they saw before with block-based programming. The intent of this is to facilitate schema building and knowledge transfer, and again, offloading some of the cognitive load, so that when they come to learn the text-based programming language they can focus on learning the syntax and the mechanics of typing code sentences, through examples whose algorithmic and computational underpinnings they are already familiar with.

Assessment construction

There has been debate about the convenience of incorporating cover stories or detailed contexts into programming problem statements (Morrison et al., 2015). The experience in the modular course studied here was that verbose problem statements in the tests and lab assignments created a greater need for complex reading comprehension, adding extraneous cognitive load and leading students to incorrectly interpret the requirements of the problem (Robins et al., 2019). In addition, reading unnecessarily lengthy problem statements can be detrimental to students’ performance in time-constrained exams. Problem statements that minimize the initial context presentation and lead the student to the problem requirements directly, stated as sub-goals, are thus preferred. Each subgoal can add more context to the problem statement if required. Subgoals can progressively increase skill complexity, and can allow incremental progress in solving the given problem. However, it is recommended that there are sub-goals that do not depend on fully completing others, so that the student has a greater opportunity to demonstrate their mastery of independent skills. Finally, if it is desired that students face less structured problems, this can be done through homework assignments, for which they can have much more time, and they can also turn to the teaching staff and their peers to resolve doubts and exchange ideas.

Constructive alignment

In the modular course, it is essential that students know how they will be assessed and that they can prepare in advance to take a summative exam successfully, as summative exams largely determine their success or failure in the modular course. For this, it is essential that students can have study material with abundant worked examples, and that tutoring sessions are focused on supporting students to solve problems at the exam’s required level of proficiency. Through these activities, students must be capable of building their computational thinking and problem solving schemata, and complementarily, assessments must be constructed in ways that foster students’ activation of that same schemata, and promote transfer of their learned abilities to the problems that are presented.

Teaching team and culture

In a CS1 course, as in other contexts where there is teaching with technology, effective teaching depends on the technological pedagogical content knowledge (Koehler & Mishra, 2009) that the teachers possess. With regard to pedagogical knowledge, teachers must have the ability to teach using the programming language, performing live coding examples that students can follow and perform simultaneously with them. Teachers must also have the ability to review what students program, give them feedback, and help them overcome what is keeping them from moving forward. In addition, teachers must know and be aware of the nature of the knowledge and skills that are required to be taught in each module of the course, since this is essential if a constructive alignment strategy is to be developed. From an organizational standpoint, teachers must be willing to work as a team and to perform under a coordinating role. Lastly, the course coordinator together with the teaching team must have the support of the Faculty leadership to have freedom to make decisions about how to innovate and continuously improve the course.

Conclusions

This paper presents the development of a modular CS1 course based on mastery learning, over a period of four years, with a total of 959 engineering freshmen in a Latin American university. The development of this intervention reduced yearly course attrition from 25 to 3.8% of the cohort, decreased the average time spent by students in the course from 23.2 (SD = 7.38) weeks to 14.9 (SD = 3.64), and improved the course pass rate in the first semester from 19.3 to 77.1%.

The modular course based on mastery learning fulfilled the goal of allowing students to move through different module progressions according to their own learning ability and effort. The course gave the students the possibility to retake modules in which they had greater learning difficulties, without failing the entire course, as had happened to 30 to 40% of the cohort in the previous traditional course. Despite procrastination observed in the second semester, in the final iteration of the intervention, over 95% of the students passed the complete course by the end of the year.

The implementation of a CS1 course in the format here presented requires a well-aligned teaching team coordinated around the project and its objectives. In addition, it demands teachers with a high level of pedagogical, technological and epistemological knowledge about the teaching of programming. In the present project, this knowledge had to be developed and systematized over the course of the four years. Some practices that emerged in the present research could help researchers and practitioners from avoid some of the traps the present authors fell into at the beginning of the project. First, the way in which the initial course content is introduced influences the motivation and academic performance of students as they move towards more complex contents and skills. This was found in the first two iterations of the course and is consistent with the theory of learning edge momentum (Robins, 2010). Second, the constructive alignment of learning goals, with teaching and learning activities and assessment must be achieved through the development of teaching materials, worked examples, and assessments in a coordinated manner by the teaching team. The teaching materials must cover all relevant problem solving strategies and schemata that students need to learn in each module. Third, assessments must focus on validating students’ possession of intended problem solving schemata and actual programming skills, both which must be clearly identifiable and in explicit connection with students’ past learning experiences. In addition, given that skills and knowledge grow progressively more complex in the course, planning of teaching and learning activities must allow sufficient time for students’ deep learning and preparation. These recommendations are consistent with recent research at the intersection of cognitive science and computer science education (Robins et al., 2019). Lastly, the support and trust of the Faculty leadership, in terms of granting creative autonomy and decision-making to the teaching team for continuous improvement of the course is key to academic success.

In the future, the present authors aim to propose a formal process for the design of constructively aligned teaching, learning activities and assessments in the context of mastery learning-based CS1 courses. Also, the present authors are developing predictive models to detect students at greater risk of failing the course in the first semester, aiming to provide these students with greater support and personalization of their learning experiences. Lastly, a mobile intelligent programming tutor will be evaluated as a means to provide students with a complement to regular teaching and learning activities in the course, for constant programming practice, anywhere and anytime.