Skip to main content
Log in

Monitoring the software development process using a short-run control chart

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Techniques for statistical process control (SPC), such as using a control chart, have recently garnered considerable attention in the software industry. These techniques are applied to manage a project quantitatively and meet established quality and process-performance objectives. Although many studies have demonstrated the benefits of using a control chart to monitor software development processes (SDPs), some controversy exists regarding the suitability of employing conventional control charts to monitor SDPs. One major problem is that conventional control charts require a large amount of data from a homogeneous source of variation when constructing valid control limits. However, a large dataset is typically unavailable for SDPs. Aggregating data from projects with similar attributes to acquire the required number of observations may lead to wide control limits due to mixed multiple common causes when applying a conventional control chart. To overcome these problems, this study utilizes a Q chart for short-run manufacturing processes as an alternative technique for monitoring SDPs. The Q chart, which has early detection capability, real-time charting, and fixed control limits, allows software practitioners to monitor process performance using a small amount of data in early SDP stages. To assess the performance of the Q chart for monitoring SDPs, three examples are utilized to demonstrate Q chart effectiveness. Some recommendations for practical use of Q charts for SDPs are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Alemi, F. (2004). Tukey’s control chart. Quality Management in Healthcare, 13(4), 216.

    Google Scholar 

  • Baldassarre, T., Boffoli, N., Caivano, D., & Visaggio, G. (2004). Managing software process improvement (SPI) through statistical process control (SPC). In Proceedings of the 5th International Conference on Product Focused Software Process Improvement. Kansai Science City, Japan.

  • Baldassarre, T., Caivano, D., Kitchenham, B., & Visaggio, G. (2007). Systematic review of statistical process control: An experience report. In Proceedings of the 11th International Conference on Evaluation and Assessment in Software Engineering (EASE). Keele, UK.

  • Barnard, J., & Price, A. (1994). Managing code inspection information. IEEE Software, 11(2), 59–69.

    Article  Google Scholar 

  • Bothe, D. R. (1989). A powerful new control chart for job shops. In Proceedings of the 43rd Annual Quality Congress Transaction. Toronto, Ontario, Canada.

  • Caivano, D. (2005). Continuous software process improvement through statistical process control. In Proceedings of the Ninth European Conference on Software Maintenance and Reengineering (CSMR’05). Manchester, UK.

  • Castillo, E. D., & Montgomery, D. C. (1994). Short run statistical process control: Q chart enhancements and alternative methods. Quality and Reliability Engineering International, 10(2), 87–97.

    Article  Google Scholar 

  • Castillo, E. D., Grayson, J. M., Montgomery, D. C., & Runger, G. C. (1996). A review of statistical process control techniques for short run manufacturing systems. Communications in Statistics—Theory and Methods, 25(11), 2723–2737.

    Article  MATH  Google Scholar 

  • Celano, G., Castagliola, P., Trovato, E., & Fichera, S. (2011). Shewhart and EWMA t control charts for short production runs. Quality and Reliability Engineering International, 27(3), 313–326.

    Article  Google Scholar 

  • Chang, C.-P., & Chu, C.-P. (2008). Improvement of causal analysis using multivariate statistical process control. Software Quality Journal, 16(3), 377.

    Article  Google Scholar 

  • Cullen, C. C., & Bothe, D. R. (1989). SPC for short production runs. Dayton, OH: In Proceedings of the National Aerospace and Electronic Conference.

    Google Scholar 

  • Diane, M., & Stephen, H. K. (2007). Practical statistical process control for software metrics. Software Quality Professional, 9(4), 15.

    Google Scholar 

  • Fagan, M. E. (1999). Design and code inspections to reduce errors in program development. IBM Systems Journal, 38(2.3), 258-287.

    Google Scholar 

  • Florac, W. A., & Carleton, A. D. (1999). Measuring the software process: statistical process control for software process improvement: Addison-Wesley.

  • Garjani, M., Noorossana, R., & Saghaei, A. (2010). A neural network-based control scheme for monitoring start-up processes and short runs. International Journal of Advanced Manufacturing Technology, 51(9–12), 1023–1032.

    Article  Google Scholar 

  • Jacob, A. L., & Pillai, S. K. (2003). Statistical process control to improve coding and code review. IEEE Software, 20(3), 50.

    Article  Google Scholar 

  • Jalote, P., & Saxena, A. (2002). Optimum control limits for employing statistical process control in software process. IEEE Transactions on Software Engineering, 28(12), 1126–1134.

    Article  Google Scholar 

  • Komuro, M. (2006). Experiences of applying SPC techniques to software development processes. In Proceedings of the 28th international conference on Software engineering. Shanghai, China.

  • Kulpa, M. K., & Johnson, K. A. (2008). Interpreting the CMMI: A process improvement approach. Boca Raton: CRC Press.

    Google Scholar 

  • Leu, S.-S., & Lin, Y.-C. (2008). Project performance evaluation based on statistical process control techniques. Journal of Construction Engineering & Management, 134(10), 813–819.

    Article  Google Scholar 

  • Lipke, W. (2002). Statistical process control of project performance. CrossTalk, The Journal of Defense Software Engineering, 15(3), 15–18.

    Google Scholar 

  • Lipke, W., & Vaughn, J. (2000). Statistical process control meets earned value. CrossTalk, The Journal of Defense Software Engineering, 13(6), 16–20.

    Google Scholar 

  • Manlove, D., & Kan, S. H. (2007). Practical statistical process control for software metrics. Software Quality Professional, 9(4), 15.

    Google Scholar 

  • Montgomery, D. C. (2009). Introduction to statistical quality control. Hoboken, N.J.: John Wiley.

    Google Scholar 

  • Quesenberry, C. P. (1991a). SPC Q charts for a binomial parameter p: Short or long runs. Journal of Quality Technology, 23(3), 239–246.

    Google Scholar 

  • Quesenberry, C. P. (1991b). SPC Q charts for a Poisson parameter lambda: Short or long runs. Journal of Quality Technology, 23(4), 296–303.

    MathSciNet  Google Scholar 

  • Quesenberry, C. P. (1991c). SPC Q charts for start-up processes and short or long runs. Journal of Quality Technology, 23(3), 213–224.

    Google Scholar 

  • Quesenberry, C. P. (1993). The effect of sample size on estimated limits for Xbar and X control charts. Journal of Quality Technology, 25(4), 237–247.

    Google Scholar 

  • Raczynski, B., & Curtis, B. (2008). Software data violate SPC’s underlying assumptions. IEEE Software, 25(3), 48–50.

    Article  Google Scholar 

  • Radice, R. (1998). Statistical process control for software projects. In Proceedings of the 10th Software Engineering Process Group Conference. Chicago, Illinois, USA.

  • Sargut, K. U., & Demirörs, O. (2006). Utilization of statistical process control (SPC) in emergent software organizations: Pitfalls and suggestions. Software Quality Journal, 14(2), 135.

    Article  Google Scholar 

  • Shewhart, W. A. (1926). Quality control charts. Bell System Technical Journal, 5, 593–603.

    Google Scholar 

  • Tarhan, A., & Demirörs, O. (2006). Investigating suitability of software process and metrics for statistical process control. In I. Richardson, P. Runeson, & R. Messnarz (Eds.), Software Process Improvement (Vol. 4257, pp. 88-99, Lecture Notes in Computer Science): Springer Berlin/Heidelberg.

  • CMMI Product Team (2010). CMMI® for Development, Version 1.3. Software Engineering Institute, Carnegie Mellon University.

  • Torng, C. C., & Lee, P. H. (2008). ARL performance of the Tukey’s control chart. Communications in Statistics-Simulation and Computation, 37(9), 1904–1913.

    Article  MathSciNet  MATH  Google Scholar 

  • Torng, C. C., Liao, H. N., Lee, P. H., & Wu, J. C. (2009). Performance evaluation of a Tukey’s control chart in monitoring gamma distribution and short run processes. Hong Kong: In Proceedings of the International MultiConference of Engineers and Computer Scientists.

    Google Scholar 

  • Tsai, T. R., Wu, S. J., & Lin, H. C. (2004). An alternative control chart approach based on small number of subgroups. International Journal of Information and Management Sciences, 15(4), 61–73.

    MathSciNet  MATH  Google Scholar 

  • Tukey, J. W. (1977). Exploratory data analysis. Reading, Mass: Addison-Wesley Pub. Co.

    MATH  Google Scholar 

  • Wang, Q., Jiang, N., Gou, L., Che, M. R., & Zhang, R. H. (2006). Practical experiences of cost/schedule measure through earned value management and statistical process control. In Software Process Change (Vol. 3966, pp. 348–354, Lecture Notes in Computer Science).

  • Weller, E. (2000). Practical applications of statistical process control. IEEE Software, 17(3), 48–55.

    Article  Google Scholar 

  • Weller, E., & Card, D. (2008). Applying SPC to software development where and why. IEEE Software, 25(3), 48–48, 50.

    Google Scholar 

  • Zantek, P. F., & Nestler, S. T. (2009). Performance and properties of Q-statistic monitoring schemes. Naval Research Logistics, 56(3), 279–292.

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang, L., Chen, G., & Castagliola, P. (2009). On t and EWMA t charts for monitoring changes in the process mean. Quality and Reliability Engineering International, 25(8), 933–945.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the National Science Council of the Republic of China, Taiwan, for financially supporting this research under Contract No. NSC 97-2221-E-009-111-MY3. The authors also would like to thank the Information and Communications Research Laboratories (ICL) of the Industrial Technology Research Institute (ITRI), ROC, Taiwan, for providing actual project data to enrich the application of this study.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Chih-Wei Chang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chang, CW., Tong, LI. Monitoring the software development process using a short-run control chart. Software Qual J 21, 479–499 (2013). https://doi.org/10.1007/s11219-012-9182-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-012-9182-y

Keywords

Navigation