Abstract
Many software-development organizations have recently started to adopt agile practices in order to benefit from the improvements they clearly bring, in terms of production speed and software quality, for example. However, in a large, distributed organization, such as Nokia Siemens Networks, an agile transformation may be a long-term, complex process, and therefore, evaluating the benefits achieved, and measuring the success of the change, can be difficult, especially in the early phases of the transformation. This paper is based on the results of a case study on the impact introducing agile practices actually had in a large, software-development organization, part of Nokia Siemens Networks. The impact of this agile transformation was evaluated in terms of how well it succeeded in achieving the goals set by the management before the transformation started. This longitudinal study included two separate analyses, made at 6 monthly intervals, which measured what agile practices were really in use in the organization, and what effects, if any, were visible in terms of the goals which had been set by the management The 6 monthly analyses included defect data metrics and opinion surveys of the personnel directly involved in the transformation. The study showed two things: firstly, that the introduction of agile practices does have the beneficial effects that one would expect, and secondly, that, using this kind of study, it is not only possible to evaluate the impact that the adoption of agile practices has in a large, established software-development company, but also that the effects are visible at a surprisingly early stage.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Auvinen, J., Back, R., Heidenberg, J., Hirkman, P., & Milovanov, L. (2005). Improving the engineering process area at Ericsson with agile practices. A case study. TUCS Technical report No 716.
Begel, A., & Nagappan, N. (2008). Pair programming: What’s in it for me? In Proceedings of the second ACM-IEEE international symposium on empirical software engineering and measurement (ESEM ‘08), IEEE (pp. 120–128).
Benefield, G. (2008). Rolling out agile in a large enterprise. In Proceedings of the 41st annual Hawaii international conference on system sciences (p. 462).
Ceschi, M., Sillitti, A., Succi, G., & Panfilis, S. D. (2005). Project management in plan-based and agile companies. IEEE Software, 22(3), 21–27.
DeClue, T. H. (2003). Pair programming and pair trading: Effects on learning and motivation in a CS2 course. Journal of Computing Sciences in Colleges, 18(5), 49–56.
Goeschl, S., Herp, M., & Wais, C. (2010). When agile meets OO testing—A case study. In Proceedings of the 1st workshop on testing object-oriented systems (ETOOS ‘10).
Hartmann, D., & Dymond, R. (2006). Appropriate agile measurement: Using metrics and diagnostics to deliver business value. In Proceedings of agile 2006 (pp. 128–134).
Ileva, S., Ivanov, P., & Stefanova, E. (2004). Analyses of an agile methodology implementation. In Proceedings of euromicro conference (pp. 393–407). IEEE Computer Society Press.
Kettunen, P., & Laanti, M. (2007). Combining agile software projects and large-scale organizational agility. Published online 13 July 2007 in Wiley InterScience.
Korhonen, K. (2009). Migrating defect management from waterfall to agile software development in a large-scale multi-site organization: A case study. In Proceedings 10th international conference on agile processes and eXtreme Programming in software engineering (XP 2009), Italy, LNBIP 31 (pp. 73–82). Springer.
Korhonen, K. (2010). Exploring defect data, quality and engagement during agile transformation at a large multisite organization. In 11th international conference on agile processes and eXtreme Programming in software engineering XP 2010 (pp. 88–102). Springer, Berlin.
Korhonen, K. (2011). Evaluating the impact of agile adoption on the software defect management practices. Software Quality Professional, 14(1), 23–33.
Korhonen, K. (2012). Supporting agile transformation with defect management in large distributed software development organisation. Dissertation at Tampere University of Technology. Publication 1032.
Korhonen, K., & Salo, O. (2008). Exploring quality metrics to support defect management in multi site organization—A case study. In Proceedings of the 19th international symposium on software reliability engineering (ISSRE 2008), Seattle, WA (pp. 213–218). IEEE Computer Society.
Laanti, M. (2010). Agile transformation study at Nokia—One year after. Lean Enterprise Software and Systems, 65, Part 1, 3–19.
Laanti, M., Salo, O., & Abrahamsson, P. (2011). Agile methods rapidly replacing traditional methods at Nokia: A survey of opinions on agile transformation. Information and Software Technology, 53(3), 276–290.
Li, J., Moe, B., & Dybå, T. (2010). Transition from a plan-driven process to Scrum: A longitudinal case study on software quality. In Proceedings of the ACM-IEEE international symposium on empirical software engineering and measurement (ESEM 2010).
Lifshitz, G., Kroskin, A., & Dubinsky, Y. (2008). The story of transition to agile sw development. In Proceedings of XP 2008 (pp. 212–214), Ireland.
Lindvall, M., Muthig, D., Dagnino, A., Wallin, C., Stupperich, M., Kiefer, D., et al. (2004): Agile software development in large organizations. IEEE Computer 37(12), 26–34.
McDowell, C., Werner, L., Bullock, H., & Fernald, J. (2002). The effects of pair-programming on performance in an introductory programming course. In Proceedings the 33rd SIGCSE technical symposium on computer science education (pp. 38–42).
Nagappan, N., Maximilien, E. M., Bhat, T., & Williams, L. (2008). Realizing quality improvement through test driven development: Results and experiences of four industrial teams. Empirical Software Engineering, 13(3), 289–302.
Petersen, K., & Wohlin, C. (2010). The effect of moving from a plan-driven to an incremental software development approach with agile practices: An industrial case study. Empirical Software Engineering, 15, 654–693.
Poppendieck, M., & Poppendieck, T. (2007). Lean software development. Addison-Wesley.
Royce, W. (1970). Managing the development of large software systems. In Proceedings of the IEEE WESCON 26 (pp. 1–9). http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/waterfall.pdf.
Schatz, B., & Abdelshafi, I. (2005). Primavera gets agile: A successful transition to agile development. IEEE Software, 22, 36–42.
Schwaber, K., & Beedle, M. (2002). Agile software development with Scrum. Prentice Hall.
Sfetsos, P., & Stamelos, I. (2010). Empirical studies on quality in agile practices: A systematic literature review. In Proceedings of QUATIC 2010.
Sillitti, A., Ceschi, M., Russo, B., & Succi, G. (2005). Managing uncertainty in requirements: A survey in documentation-driven and agile companies. In Proceedings of the 11th IEEE international symposium on software metrics (METRICS 2005) (p. 17).
van Solingen, R., & Berghout, E. (1999). The Goal/Question/Metric method: A practical guide for quality improvement of software development. New York: McGraw-Hill Publishing Company.
Vilkki, K. (2009). Impacts of agile transformation. Flexi Newsletter 1/2009.
Yin, R. (1994). Case study research: Design and methods. Sage Publishing.
Acknowledgments
The author thanks Kai Koskimies and Maria Lahti for their comments, and Nokia Foundation for funding the manuscript writing.
Author information
Authors and Affiliations
Corresponding author
Appendix: Agile transformation survey questions
Appendix: Agile transformation survey questions
1.1 Part 1: Background questions
1.2 Part 2: Agile transformation
1.3 Part 3: Agile practices in your daily work
Rights and permissions
About this article
Cite this article
Korhonen, K. Evaluating the impact of an agile transformation: a longitudinal case study in a distributed context. Software Qual J 21, 599–624 (2013). https://doi.org/10.1007/s11219-012-9189-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11219-012-9189-4