Skip to main content

Modelling the Likelihood of Software Process Improvement: An Exploratory Study

Empirical Software Engineering volume 6, pages 207–229 (2001)Cite this article

Abstract

Software process assessments have become big business worldwide. They can be a powerful tool for initiating and sustaining software process improvement (SPI). However, SPI programs sometimes fail. Moreover there still are very few systematic empirical investigations about the conditions under which SPI initiatives vary in their outcomes. In this paper we present the results of a study of factors that influence the success of SPI. The data come from a sample survey of organizations that have performed assessments based on the capability maturity model for software, and was conducted from 1 to 3 years after the assessments (sufficient time had passed for changes to have taken place). The results consist of a multivariate model of the conditions (e.g., how the improvement efforts are organized and funded) that can explain the successes and failures of SPI efforts. The model is constructed using a classification tree algorithm. It identifies the most important factors that affect the outcome of SPI efforts, and describes how those factors interact with each other to influence success or failure.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Breiman, L., Friedman, J., Olshen, R., and Stone, C. 1984. Classification and Regression Trees. Wadsworth and Brooks Cole.

  • Carmines, E., and Zeller, R. 1979. Reliability and Validity Assessment. Sage Publications.

  • Chapman, D. 1983. The impact of substitution on survey estimates In: W. Madow, I. Olkin and D. Rubin (eds.): Incomplete Data in Sample Surveys. Vol. 2. Academic Press.

  • Comrey, A. 1973. A First Course on Factor Analysis. Academic Press.

  • Cronbach, L. 1951. Coefficient alpha and the internal structure of tests. Psychomterika 16(3): 297-334.

    Google Scholar 

  • Deephouse, C., Mukhopadhyay, T., Goldenson, D., and Kellner, M. 1995–1996. Software processes and project performance. Journal of Management Information Systems 12(3).

  • Dion, R. 1999. Starting the climb towards the CMM level 2 plateau. In: K. El-Emam and N. H. Madhavji (eds.): Elements of Software Process Assessment and Improvement, IEEE CS Press.

  • El-Emam, K., and Birk, A. 2000a. Validating the ISO/IEC 15504 measures of software development process capability. Journal of Systems and Software 51(2): 119-149.

    Google Scholar 

  • El-Emam, K., and Birk, A. 2000b. Validating the ISO/IEC 15504 measures of software requirements analysis process capability. IEEE Transactions on Software Engineering 26(6): 541-566.

    Google Scholar 

  • El-Emam, K., Smith, B., and Fusaro, P. 1999. Success factors and barriers in software process improvement: An empirical study. In: R. Messnarz and C. Tully (eds.): Better Software Practice for Business Benefit: Principles and Experiences. IEEE CS Press.

  • Fowler, P., and Rifkin, S. 1990. Software Engineering Process Group Guide. Software Engineering Institute, CMU/SEI-90-TR-24.

  • Goldenson, D. R., and Herbsleb, J. 1995. After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success. Software Engineering Institute, CMU/SEI-95-TR-009.

  • Goldenson, D., El-Emam, K., Herbsleb, J., and Deephouse, C. 1999. Empirical studies of software process assessment methods. In: K. El-Emam and N. H. Madhavji (eds.): Elements of Software Process Assessment and Improvement. IEEE CS Press.

  • Herbsleb, J., and Goldenson, D. 1996. A Systematic Survey of CMM Experience and Results. Proceedings of the International Conference on Software Engineering. pp. 25-30.

  • Herbsleb, J., Carleton, A., Rozum, J., Siegel, J., and Zubrow, D. 1994. Benefits of CMM-based Software Process Improvement: Initial Results. Software Engineering Institute, CMU-SEI-94-TR-13.

  • Herbsleb, J., Zubrow, D., Goldsenson, D., Hayes, W., and Paulk, M. 1997. Software quality and the capability maturity model. Communications of the ACM 40(6): 30-40.

    Google Scholar 

  • Humphrey, W., and Sweet, W. 1987. A Method for Assessing the Software Engineering of Contractors. Software Engineering Institute, CMU-SEI-87-TR-0023.

  • Jones, C. 1999. The economics of software process improvements. In: K. El-Emam and N. H. Madhavji (eds.): Elements of Software Process Assessment and Improvement. IEEE CS Press.

  • Kim, J., and Mueller, C. 1978. Factor Analysis: Statistical Methods and Practical Issues. Sage Publications.

  • Krasner, H. 1999. The payoff for software process improvement: What it is and how to get it. In: K. El-Emam and N. H. Madhavji (eds.): Elements of Software Process Assessment and Improvement. IEEE CS Press.

  • Lanubile, F., and Visaggio, G. 1997. Evaluating predictive quality models derived from software measures: Lessons learned. Journal of Systems and Software 38: 225-234.

    Google Scholar 

  • Lawlis, P., Flowe, R., and Thordahl, J. 1995. A correlational study of the CMM and software development performance. CrossTalk 21-25.

  • Little, R., and Rubin, D. 1987. Statistical Analysis With Missing Data. John Wiley & Sons.

  • Maher, J., and Gremba, J. 1994. Organizational Barriers to PI and Technology Transition. Proceedings of the 1994 SEI Software Engineering Symposium.

  • McFeeley, B. 1996. IDEAL: A User's Guide for Software Process Improvement. Software Engineering Institute, Handbook CMU-SEI-96-HB-001.

  • Miller, M., and Goldenson, D. 1992. Software Engineering Process Groups: Results of the 1992 SEPG Workshop and a First Report on SEPG Status. Software Engineering Institute, Special Report CMU/SEI-92-SR-13.

  • Mingers, J. 1989. An empirical comparison of selection measures for decision-tree induction. Machine Learning 3: 319-342.

    Google Scholar 

  • Nunnally, J. 1978. Psychometric Theory. McGraw-Hill.

  • Puffer, J. 1999. Action planning. In: K. El-Emam and N. H. Madhavji (eds.): Elements of Software Process Assessment and Improvement. IEEE CS Press.

  • Quinlan, J. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann.

  • Sande, I. 1983. Hot-deck imputation procedures. In: W. Madow and I. Olkin (eds.): Incomplete Data in Sample Surveys. Vol. 3, Academic Press.

  • Schneidewind, N. 1994. Validating metrics for ensuring space shuttle flight software quality. IEEE Computer 50-57.

  • Spector, P. 1992. Summated Rating Scale Construction. Sage Publications.

  • Stelzer, D., and Mellis, W. 1998. Success factors of organizational change in software process improvement. Software Process: Improvement and Practice 4(4): 227-250.

    Google Scholar 

  • Weiss, S., and Kulikowski, C. 1991. Computer Systems that Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning, and Expert Systems. Morgan Kaufmann Publishers.

  • Zeller, R., and Carmines, E. 1980. Measurement in the Social Sciences. Cambridge University Press.

Download references

Author information

Authors and Affiliations

  1. National Research Council, Montreal Road, Bldg. M-50, Ottawa, Ontario, Canada, K1A OR6

    Khaled El-Emam

  2. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA, 15213-3890, USA

    Dennis Goldenson

  3. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA, 15213-3890, USA

    James McCurley

  4. Software Production Research Department, Lucent Technologies, Naperville, IL, USA

    James Herbsleb

Authors
  1. Khaled El-Emam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dennis Goldenson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James McCurley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. James Herbsleb
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and Permissions

About this article

Cite this article

El-Emam, K., Goldenson, D., McCurley, J. et al. Modelling the Likelihood of Software Process Improvement: An Exploratory Study. Empirical Software Engineering 6, 207–229 (2001). https://doi.org/10.1023/A:1011487332587

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1011487332587

  • Software process
  • software process improvement
  • software process assessment
  • subjective measurement
  • classification trees
  • survey
  • SW-CMM