Monitoring and Controlling Release Readiness by Learning Across Projects
Releasing software on time, with desired quality while staying within budget is crucial for success. Therefore, product managers should proactively know which release readiness attributes are not performing sufficiently well (i.e., bottleneck factors) throughout the development cycle and consequently may limit readiness of the software release. We present the Cross-project Analysis for Selection of Release Readiness attributes (CASRR) method to help project managers in (i) systematically studying and analyzing release readiness attributes across multiple projects, (ii) selection of release readiness attributes for monitoring which have previously been shown to become bottlenecks in similar projects in the past, and (iii) learning how bottleneck occurrences are influenced by project characteristics. We applied CASRR to two Open Source Software projects, and analyzed six release readiness attributes in 34 similar projects over a period of two years. Continuous integration rate, feature completion rate, and bug fixing rate are observed as the most frequent bottleneck factors. Bottleneck occurrences of the monitored release readiness attributes are significantly influenced by the maturity of a release. Furthermore, the continuous integration rate is found to be significantly influenced by the team size.
KeywordsDevelopment Cycle Similar Project Software Release Project Characteristic Release Readiness
Unable to display preview. Download preview PDF.
This work was partially supported by the Natural Sciences and Engineering Research Council of Canada, NSERC Discovery Grant 250343-12, Alberta Innovates Technology Futures and by the institutional research grant IUT20-55 of the Estonian Research Council.
- 2.Asthana, A., Olivieri, J.: Quantifying software reliability and readiness. In: Proceedings of the International Workshop Technical Committee on Communications Quality and Reliability, pp. 1–6. IEEE, Washington, DC (2009)Google Scholar
- 3.Basili, V.R., Caldiera, G., Rombach, H.D.: The goal question metric approach. In: Encyclopedia of Software Engineering. Wiley, New Jersey (1994)Google Scholar
- 4.Borland: Teaminspector 2008. http://techpubs.borland.com
- 6.Didar Al Alam, S.M., Shahnewaz, S.M., Pfahl, D., Ruhe, G.: Monitoring bottlenecks in achieving release readiness: a retrospective case study across ten oss projects. In: Proceedings of the International Symposium on Empirical Software Engineering and Measurement, pp. 60:1–60:4. ACM, New York, NY, USA (2014)Google Scholar
- 9.Gokhale, S.: Optimal software release time incorporating fault correction. In: Proceedings of the Annual NASA Goddard Software Engineering Workshop, pp. 175–184. IEEE, Washington, DC (2003)Google Scholar
- 11.Larman, C.: Agile and Iterative Development: A Manager’s Guide. Addison-Wesley Professional (2003)Google Scholar
- 13.Microsoft: Plan the release readiness review meeting. http://technet.microsoft.com/en-us/library/cc526651.aspx Accessed 2014
- 14.Pearse, T., Freeman, T., Oman, P.: Using metrics to manage the end-game of a software project. In: Proceedings of the International Symposium on Software Metrics, pp. 207–215. IEEE, Washington, DC (1999)Google Scholar
- 15.Port, D., Wilf, J.: The value of certifying software release readiness: an exploratory study of certification for a critical system at jpl. In: Proceedings of the International Symposium on Empirical Software Engineering and Measurement, pp. 373–382. IEEE, Washington, DC (2013)Google Scholar
- 17.Rothman, J.: Measurements to reduce risk in product ship decisions. http://www.universityalliance.com/info1/whitepapers Accessed 2014
- 18.Satapathy, P.R.: Evaluation of software release readiness metric [0,1] across the software development life cycle. Tech. rep., Department of Computer Science & Engineering, University of California (2007)Google Scholar
- 19.Shahnewaz, S.M.: RELREA-an analytical approach supporting continuous release readiness evaluation. Master’s thesis, University of Calgary (2014)Google Scholar
- 20.Staron, M., Meding, W., Palm, K.: Release readiness indicator for mature agile and lean software development projects. In: Wohlin, C. (ed.) Agile Processes in Software Engineering and Extreme Programming. Lecture Notes in Business Information Processing, vol. 111, pp. 93–107. Springer, Heidelberg (2012)CrossRefGoogle Scholar
- 21.Ware, M., Wilkie, F., Shapcott, M.: The use of intra-release product measures in predicting release readiness. In: Proceedings of the International Conference on Software Testing, Verification, and Validation, pp. 230–237. IEEE, Washington, DC (2008)Google Scholar
- 22.Wild, R., Brune, P.: Determining software product release readiness by the change-error correlation function: on the importance of the change-error time lag. In: Proceeding of the Annual Hawaii International Conference on System Science, pp. 5360–5367. IEEE, Washington, DC (2012)Google Scholar