Onboarding inexperienced developers: struggles and perceptions regarding automated testing
Previous research found that inexperienced software engineers may tend to view automatic testing as a waste of time and as an activity completely separate from programming. This could have a negative impact on their later careers and could be a sign that improvements in software engineering education are needed when it comes to testing. At the same time, this stance could negatively influence the perception that practitioners have of recent university graduates. To explore this issue, we conducted a qualitative study and surveyed 170 and interviewed 22 practitioners about their experiences with recent graduates, focusing on software testing skills. We find that practitioners do recognize a skill gap between university graduates and industry expectations and that this perception could be engrained deeply enough already to influence hiring practices. Practitioners use different and at times costly strategies to alleviate this skill gap, such as training and mentoring efforts. We validated core findings in a survey with 698 professional software developers. Our qualitative insights can help industry, research, and educational institutions guide in-depth studies that explore the severity of the effects we have found. The coping strategies we have found can provide valuable starting points that can inform changes in how we educate the software engineers of the future.
KeywordsAutomated testing Onboarding Newcomers Software developer Required skills
- Begel, A., & Simon, B. (2008a). Novice software developers, all over again. In Proceedings of the fourth international workshop on computing education research (pp. 3–14). ACM.Google Scholar
- Begel, A., & Simon, B. (2008b). Struggles of new college graduates in their first software development job. In ACM SIGCSE Bulletin (Vol. 40, pp. 226–230). ACM.Google Scholar
- Boud, D., & Feletti, G. (1998). The challenge of problem-based learning. Hove: Psychology Press.Google Scholar
- Brechner, E. (2003). Things they would not teach me of in college: What microsoft developers learn later. In Companion of the 18th annual ACM SIGPLAN conference on object-oriented programming, systems, languages, and applications (pp. 134–136). ACM.Google Scholar
- Bruegge, B. (1992). Teaching an industry-oriented software engineering course. In C. Sledge (Ed.), Software engineering education, lecture notes in computer science (Vol. 640, pp. 63–87). Berlin: Springer. doi: 10.1007/3-540-55963-9_40.
- Carrington, D., & Kim, S. K. (2003). Teaching software design with open source software. In FIE 2003 33rd annual frontiers in education, 2003 (Vol. 3, p. S1C-9). IEEE.Google Scholar
- Dagenais, B., Ossher, H., Bellamy, R. K., Robillard, M. P., & De Vries, J. P. (2010). Moving into a new software project landscape. In Proceedings of the 32nd ACM/IEEE international conference on software engineering (Vol. 1, pp. 275–284). ACM.Google Scholar
- Deak, A., & Sindre, G. (2014). Analyzing the importance of teaching about testing from alumni survey data. Norsk informatikkonferanse (NIK) 2013.Google Scholar
- DeMarco, T., & Lister, T. (1987). Peopleware: Productive people and teams. Oxford: Dorset House.Google Scholar
- Edwards, S. H. (2003a). Rethinking computer science education from a test-first perspective. In Companion of the 18th annual ACM SIGPLAN conference on object-oriented programming, systems, languages, and applications (pp. 148–155). ACM.Google Scholar
- Edwards, S. H. (2003b). Using test-driven development in the classroom: Providing students with automatic, concrete feedback on performance. In Proceedings of the international conference on education and information systems: technologies and applications EISTA (Vol. 3).Google Scholar
- Elbaum, S., Person, S., Dokulil, J., & Jorde, M., (2007). Bug hunt: Making early software testing lessons engaging and affordable. In 29th international conference on software engineering, 2007. ICSE 2007 (pp. 688–697). IEEE.Google Scholar
- Fagerholm, F., Guinea, A.S., Münch, J., & Borenstein, J. (2014). The role of mentoring and project characteristics for onboarding in open source software projects. In Proceedings of the 8th ACM/IEEE international symposium on empirical software engineering and measurement (p. 55). ACM.Google Scholar
- Fagerholm, F., Johnson, P., Guinea, A. S., Borenstein, J., & Munch, J. (2013). Onboarding in open source software projects: A preliminary analysis. In 2013 IEEE 8th international conference on global software engineering workshops (ICGSEW) (pp. 5–10). IEEE.Google Scholar
- Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research (Vol. 3). Chicago: Aldine de Gruyter.Google Scholar
- Janzen, D. S., & Saiedian, H. (2006). Test-driven learning: Intrinsic integration of testing into the CS/SE curriculum. In In ACM SIGCSE bulletin (Vol. 38, pp. 254–258). ACM.Google Scholar
- Jones, E. L. (2001). An experiential approach to incorporating software testing into the computer science curriculum. In 31st annual frontiers in education conference, 2001 (Vol. 2, p. F3D-7). IEEE.Google Scholar
- Jones, E. L., & Chatmon, C. L. (2001). A perspective on teaching software testing. In Journal of computing sciences in colleges, consortium for computing sciences in colleges (Vol. 16, pp. 92–100).Google Scholar
- Kaner, C., & Padmanabhan, S. (2007). Practice and transfer of learning in the teaching of software testing. In 20th conference on software engineering education and training, 2007. CSEET’07 (pp. 157–166). IEEE.Google Scholar
- Lübke, D., Flohr, T., & Schneider, K. (2004). Serious insights through fun software-projects. In T. Dingsøyr (Ed.), Software process improvement, lecture notes in computer science (Vol. 3281, pp. 57–68). Berlin: Springer. doi: 10.1007/978-3-540-30181-3_6.
- Lustosa Neto, V., Coelho, R., Leite, L., Guerrero, D. S., & Mendonça, A. P. (2013). Popt: A problem-oriented programming and testing approach for novice students. In 2013 35th international conference on software engineering (ICSE) (pp. 1099–1108). IEEE.Google Scholar
- Marrero, W., & Settle, A. (2005). Testing first: Emphasizing testing in early programming courses. In ACM SIGCSE bulletin (Vol. 37, pp. 4–8). ACM.Google Scholar
- Ng, S., Murnane, T., Reed, K., Grant, D., & Chen, T. (2004). A preliminary survey on software testing practices in Australia. In Software engineering conference, 2004. Proceedings. 2004 Australian (pp. 116–125). IEEE.Google Scholar
- Pham, R., Kiesling, S., Liskin, O., Singer, L., & Schneider, K. (2014). Enablers, inhibitors, and perceptions of testing in novice software teams. In 22th international symposium on the foundations of software engineering (FSE 2014), Hong Kong, China. (to appear).Google Scholar
- Pham, R., Mörschbach, J., & Schneider, K. (2015a). Communicating software testing culture through visualizing testing activity. In 7th international workshop on social software engineering (SSE 2015). ACM, SSE 2015. doi: 10.1145/2804381.2804382.
- Pham, R., Singer, L., Liskin, O., Figueira Filho, F., & Schneider, K. (2013). Creating a shared understanding of testing culture on a social coding site. In International conference on software engineering (ICSE) (pp. 112–121). IEEE Press.Google Scholar
- Pham, R., Stoliar, Y., & Schneider, K. (2015b). Automatically recommending test code examples to inexperienced developers. In Proceedings of the 10th joint meeting of the european software engineering conference and the ACM SIGSOFT symposium on the foundations of software engineering (ESEC/FSE 2015). ACM, ESEC/FSE 2015. doi: 10.1145/2786805.2803202.
- Radermacher, A., Walia, G., & Knudson, D. (2014). Investigating the skill gap between graduating students and industry expectations. In Companion Proceedings of the 36th international conference on software engineering (pp. 291–300) ACM.Google Scholar
- Rajlich, V. (2013). Teaching developer skills in the first software engineering course. In 2013 35th international conference on software engineering (ICSE) (pp. 1109–1116). IEEE.Google Scholar
- Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.Google Scholar
- Sim, S. E., & Holt, R. C. (1998). The ramp-up problem in software projects: A case study of how software immigrants naturalize. In Proceedings of the international conference on software engineering, 1998 (pp. 361–370). IEEE.Google Scholar
- Singer, L., Figueira Filho, F., Cleary, B., Treude, C., Storey, M. A., & Schneider, K. (2013). Mutual assessment in the social programmer ecosystem: An empirical investigation of developer profile aggregators. In Proceedings of the 2013 conference on computer supported cooperative work. CSCW ’13 (pp. 103–116). NY: ACM. doi: 10.1145/2441776.2441791.
- Spacco, J., & Pugh, W. (2006). Helping students appreciate test-driven development (TDD). In Companion to the 21st ACM SIGPLAN symposium on object-oriented programming systems, languages, and applications (pp. 907–913). ACM.Google Scholar
- Steinmacher, I., Wiese, I. S., Conte, T., Gerosa, M. A., & Redmiles, D. (2014). The hard life of open source software project newcomers. In Proceedings of the 7th international workshop on cooperative and human aspects of software engineering (pp. 72–78). ACM.Google Scholar
- Stuart, H. C., Dabbish, L., Kiesler, S., Kinnaird, P., & Kang, R. (2012a). Social transparency in networked information exchange: A theoretical framework. In Proceedings of the ACM 2012 conference on computer supported cooperative work. CSCW ’12 (pp. 451–460). New York, NY: ACM. doi: 10.1145/2145204.2145275.
- van Deursen, A., Nederlof, A., & Bouwers, E. (2013). Teaching software architecture: With GitHub!. Retrieved August 24, 2015, from http://avandeursen.com/2013/12/30/teaching-software-architecture-with-github/.
- Whittaker, J. A., Arbon, J., & Carollo, J. (2012). How Google tests software. Boston: Addison-Wesley.Google Scholar
- Zagalsky, A., Feliciano, J., Storey, M. D., Zhao, Y., & Wang, W. (2015). The emergence of github as a collaborative platform for education. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing, CSCW 2015, Vancouver, BC, Canada (pp. 1906–1917), March 14–18, 2015. doi: 10.1145/2675133.2675284.