Abstract
Techniques for statistical process control (SPC), such as using a control chart, have recently garnered considerable attention in the software industry. These techniques are applied to manage a project quantitatively and meet established quality and process-performance objectives. Although many studies have demonstrated the benefits of using a control chart to monitor software development processes (SDPs), some controversy exists regarding the suitability of employing conventional control charts to monitor SDPs. One major problem is that conventional control charts require a large amount of data from a homogeneous source of variation when constructing valid control limits. However, a large dataset is typically unavailable for SDPs. Aggregating data from projects with similar attributes to acquire the required number of observations may lead to wide control limits due to mixed multiple common causes when applying a conventional control chart. To overcome these problems, this study utilizes a Q chart for short-run manufacturing processes as an alternative technique for monitoring SDPs. The Q chart, which has early detection capability, real-time charting, and fixed control limits, allows software practitioners to monitor process performance using a small amount of data in early SDP stages. To assess the performance of the Q chart for monitoring SDPs, three examples are utilized to demonstrate Q chart effectiveness. Some recommendations for practical use of Q charts for SDPs are provided.
Similar content being viewed by others
References
Alemi, F. (2004). Tukey’s control chart. Quality Management in Healthcare, 13(4), 216.
Baldassarre, T., Boffoli, N., Caivano, D., & Visaggio, G. (2004). Managing software process improvement (SPI) through statistical process control (SPC). In Proceedings of the 5th International Conference on Product Focused Software Process Improvement. Kansai Science City, Japan.
Baldassarre, T., Caivano, D., Kitchenham, B., & Visaggio, G. (2007). Systematic review of statistical process control: An experience report. In Proceedings of the 11th International Conference on Evaluation and Assessment in Software Engineering (EASE). Keele, UK.
Barnard, J., & Price, A. (1994). Managing code inspection information. IEEE Software, 11(2), 59–69.
Bothe, D. R. (1989). A powerful new control chart for job shops. In Proceedings of the 43rd Annual Quality Congress Transaction. Toronto, Ontario, Canada.
Caivano, D. (2005). Continuous software process improvement through statistical process control. In Proceedings of the Ninth European Conference on Software Maintenance and Reengineering (CSMR’05). Manchester, UK.
Castillo, E. D., & Montgomery, D. C. (1994). Short run statistical process control: Q chart enhancements and alternative methods. Quality and Reliability Engineering International, 10(2), 87–97.
Castillo, E. D., Grayson, J. M., Montgomery, D. C., & Runger, G. C. (1996). A review of statistical process control techniques for short run manufacturing systems. Communications in Statistics—Theory and Methods, 25(11), 2723–2737.
Celano, G., Castagliola, P., Trovato, E., & Fichera, S. (2011). Shewhart and EWMA t control charts for short production runs. Quality and Reliability Engineering International, 27(3), 313–326.
Chang, C.-P., & Chu, C.-P. (2008). Improvement of causal analysis using multivariate statistical process control. Software Quality Journal, 16(3), 377.
Cullen, C. C., & Bothe, D. R. (1989). SPC for short production runs. Dayton, OH: In Proceedings of the National Aerospace and Electronic Conference.
Diane, M., & Stephen, H. K. (2007). Practical statistical process control for software metrics. Software Quality Professional, 9(4), 15.
Fagan, M. E. (1999). Design and code inspections to reduce errors in program development. IBM Systems Journal, 38(2.3), 258-287.
Florac, W. A., & Carleton, A. D. (1999). Measuring the software process: statistical process control for software process improvement: Addison-Wesley.
Garjani, M., Noorossana, R., & Saghaei, A. (2010). A neural network-based control scheme for monitoring start-up processes and short runs. International Journal of Advanced Manufacturing Technology, 51(9–12), 1023–1032.
Jacob, A. L., & Pillai, S. K. (2003). Statistical process control to improve coding and code review. IEEE Software, 20(3), 50.
Jalote, P., & Saxena, A. (2002). Optimum control limits for employing statistical process control in software process. IEEE Transactions on Software Engineering, 28(12), 1126–1134.
Komuro, M. (2006). Experiences of applying SPC techniques to software development processes. In Proceedings of the 28th international conference on Software engineering. Shanghai, China.
Kulpa, M. K., & Johnson, K. A. (2008). Interpreting the CMMI: A process improvement approach. Boca Raton: CRC Press.
Leu, S.-S., & Lin, Y.-C. (2008). Project performance evaluation based on statistical process control techniques. Journal of Construction Engineering & Management, 134(10), 813–819.
Lipke, W. (2002). Statistical process control of project performance. CrossTalk, The Journal of Defense Software Engineering, 15(3), 15–18.
Lipke, W., & Vaughn, J. (2000). Statistical process control meets earned value. CrossTalk, The Journal of Defense Software Engineering, 13(6), 16–20.
Manlove, D., & Kan, S. H. (2007). Practical statistical process control for software metrics. Software Quality Professional, 9(4), 15.
Montgomery, D. C. (2009). Introduction to statistical quality control. Hoboken, N.J.: John Wiley.
Quesenberry, C. P. (1991a). SPC Q charts for a binomial parameter p: Short or long runs. Journal of Quality Technology, 23(3), 239–246.
Quesenberry, C. P. (1991b). SPC Q charts for a Poisson parameter lambda: Short or long runs. Journal of Quality Technology, 23(4), 296–303.
Quesenberry, C. P. (1991c). SPC Q charts for start-up processes and short or long runs. Journal of Quality Technology, 23(3), 213–224.
Quesenberry, C. P. (1993). The effect of sample size on estimated limits for Xbar and X control charts. Journal of Quality Technology, 25(4), 237–247.
Raczynski, B., & Curtis, B. (2008). Software data violate SPC’s underlying assumptions. IEEE Software, 25(3), 48–50.
Radice, R. (1998). Statistical process control for software projects. In Proceedings of the 10th Software Engineering Process Group Conference. Chicago, Illinois, USA.
Sargut, K. U., & Demirörs, O. (2006). Utilization of statistical process control (SPC) in emergent software organizations: Pitfalls and suggestions. Software Quality Journal, 14(2), 135.
Shewhart, W. A. (1926). Quality control charts. Bell System Technical Journal, 5, 593–603.
Tarhan, A., & Demirörs, O. (2006). Investigating suitability of software process and metrics for statistical process control. In I. Richardson, P. Runeson, & R. Messnarz (Eds.), Software Process Improvement (Vol. 4257, pp. 88-99, Lecture Notes in Computer Science): Springer Berlin/Heidelberg.
CMMI Product Team (2010). CMMI® for Development, Version 1.3. Software Engineering Institute, Carnegie Mellon University.
Torng, C. C., & Lee, P. H. (2008). ARL performance of the Tukey’s control chart. Communications in Statistics-Simulation and Computation, 37(9), 1904–1913.
Torng, C. C., Liao, H. N., Lee, P. H., & Wu, J. C. (2009). Performance evaluation of a Tukey’s control chart in monitoring gamma distribution and short run processes. Hong Kong: In Proceedings of the International MultiConference of Engineers and Computer Scientists.
Tsai, T. R., Wu, S. J., & Lin, H. C. (2004). An alternative control chart approach based on small number of subgroups. International Journal of Information and Management Sciences, 15(4), 61–73.
Tukey, J. W. (1977). Exploratory data analysis. Reading, Mass: Addison-Wesley Pub. Co.
Wang, Q., Jiang, N., Gou, L., Che, M. R., & Zhang, R. H. (2006). Practical experiences of cost/schedule measure through earned value management and statistical process control. In Software Process Change (Vol. 3966, pp. 348–354, Lecture Notes in Computer Science).
Weller, E. (2000). Practical applications of statistical process control. IEEE Software, 17(3), 48–55.
Weller, E., & Card, D. (2008). Applying SPC to software development where and why. IEEE Software, 25(3), 48–48, 50.
Zantek, P. F., & Nestler, S. T. (2009). Performance and properties of Q-statistic monitoring schemes. Naval Research Logistics, 56(3), 279–292.
Zhang, L., Chen, G., & Castagliola, P. (2009). On t and EWMA t charts for monitoring changes in the process mean. Quality and Reliability Engineering International, 25(8), 933–945.
Acknowledgments
The authors would like to thank the National Science Council of the Republic of China, Taiwan, for financially supporting this research under Contract No. NSC 97-2221-E-009-111-MY3. The authors also would like to thank the Information and Communications Research Laboratories (ICL) of the Industrial Technology Research Institute (ITRI), ROC, Taiwan, for providing actual project data to enrich the application of this study.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chang, CW., Tong, LI. Monitoring the software development process using a short-run control chart. Software Qual J 21, 479–499 (2013). https://doi.org/10.1007/s11219-012-9182-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11219-012-9182-y