Effect of Continuous Integration on Build Health in Undergraduate Team Projects
We present the results of an analysis of the changing patterns of build health across three cohorts of undergraduate students, in a compulsory software engineering course unit. In the course unit, student teams were asked to make changes to a large open source software system, and to maintain clean release builds as they did so. Release build health (in terms of compiling code and passing unit tests) was explicitly included in the marking scheme for the coursework. We set up a continuous integration server to keep track of student build health. Initially, this was used only by TAs in marking student work, but for later cohorts we provided access to continuous integration results to all students from the early stages of each exercise. This has provided us with data on the changing patterns of student build health, with differing access to the CI server, giving an insight into how students learn to manage build health and the effects of allowing them access to CI results. We found evidence of a clear improvement in ability to manage build health when CI facilities are made available, but that some student teams were not making use of the facilities to much effect. The improvement effect was strongest on the build health of release builds, corresponding to the area of greatest marks in the marking scheme. The CI results also proved to be very valuable for academic staff, in making the problems with student builds visible.
KeywordsContinuous integration Build health Release quality Software engineering education
We are grateful to all the University of Manchester students who have taken the course units on which the work in this paper was based, for their willingness to engage with new approaches to teaching and for their (sometimes robust) feedback on our course units. We also thank the other members of the academic teaching teams we work with for their help in designing and running these course units.
- [Bec03]Beck, K.: Test-Driven Development: By Example. Addison-Wesley Professional, Reading (2003)Google Scholar
- [BS13]Billingsley, W., Steel, J.: A comparison of two iterations of a software studio course based on continuous integration. In: Proceedings of the 18th ACM Conference on Innovation and Technology in Computer Science Education, pp. 213–218. ACM (2013)Google Scholar
- [Chr16]Christensen, H.B.: Teaching DevOps and cloud computing using a cognitive apprenticeship and story-telling approach. In: Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education, pp. 174–179. ACM (2016)Google Scholar
- [EWC+17]Eddy, B.P., et al.: A pilot study on introducing continuous integration and delivery into undergraduate software engineering courses. In: Proceedings of 30th IEEE Conference on Software Engineering Education and Training (CSEE&T), pp. 47–56. IEEE (2017)Google Scholar
- [HK18]Heckman, S., King, J.: Developing software engineering skills using real tools for automated grading. In: Proceedings of the 49th ACM Technical Symposium on Computer Science Education, pp. 794–799. ACM (2018)Google Scholar
- [HMG01]Holmboe, C., McIver, L., George, C.E.: Research agenda for computer science education. In: Proceedings of 13th Annual Workshop of the Psychology of Programming Interest Group (PPIG-13), vol. 13 (2001)Google Scholar
- [LD11]Baochuan, L., DeClue, T.: Teaching agile methodology in a software engineering capstone course. J. Comput. Sci. Coll. 26(5), 293–299 (2011)Google Scholar
- [MTU17]Matthies, C., Treffer, A., Uflacker, M.: Prof. CI: employing continuous integration services and Github workflows to teach test-driven development. In: Frontiers in Education Conference (FIE), pp. 1–8. IEEE (2017)Google Scholar
- [Ped10]Pedrazzini, S.: Exploiting the advantages of continuous integration in software engineering learning projects. In: Koli Calling, p. 35 (2010)Google Scholar
- [SB12]Sü\(\upbeta \), J.G., Billingsley, W.: Using continuous integration of code and content to teach software engineering with limited resources. In: Proceedings of the 34th International Conference on Software Engineering, pp. 1175–1184. IEEE Press (2012)Google Scholar
- [Wil01]Wilson, D.: Teaching XP: a case study. In: XP Universe (2001)Google Scholar