Abstract
Understandinghow to implement SPI successfully is arguably the most challengingissue facing the SPI field today. The SPI literature containsmany case studies of successful companies and descriptions oftheir SPI programs. However, there has been no systematic attemptto synthesize and organize the prescriptions offered. The researchefforts to date are limited and inconclusive and without adequatetheoretical and psychometric justification.
This paper provides a synthesis of prescriptions for successfulquality management and process improvement found from an extensivereview of the quality management, organizational learning, andsoftware process improvement literature. The literature reviewwas confirmed by empirical studies among both researchers andpractitioners. The main result is an instrument for measuringthe key factors of success in SPI based on data collected from120 software organizations. The measures were found to have satisfactorypsychometric properties. Hence, managers can use the instrumentto guide SPI activities in their respective organizations andresearchers can use it to build models to relate the facilitatingfactors to both learning processes and SPI outcomes.
Similar content being viewed by others
References
Ahire, S. L., Golhar, D. Y. and Waller, M. A. 1996 Development and Validation of TQM Implementation Constructs, Decision Sciences, 27(1), 23-56.
Anastasi, A. and Urbina, S. 1997 Psychological Testing, Seventh edition, Upper Saddle River, New Jersey: Prentice-Hall.
Argyris, C. and Schön, D. A. 1978 Organizational Learning: A Theory of Action Perspective, Reading, Massachusetts: Addison-Wesley.
Argyris, C. and Schön, D. A. 1996 Organizational Learning II: Theory, Method, and Practice, Reading, Massachusetts: Addison-Wesley.
Bagozzi, R. P. 1996 Measurement in Marketing Research: Basic Principles of Questionnaire Design, in R. P. Bagozzi (Ed.), Principles of Marketing Research, Cambridge, Massachusetts: Blackwell, pp. 1-49.
Baruch, Y. 1999 Response Rate in Academic Studies—A Comparative Analysis, Human Relations, 52(4), 421-438.
Basili, V. R. 1996 The Role of Experimentation in Software Engineering: Past, Current, and Future, Proceedings of the 18th International Conference on Software Engineering (ICSE-18), Berlin, Germany, March 25–29, pp. 442-449.
Basili, V. R. and Caldiera, G. 1995 Improve Software Quality by Reusing Knowledge and Experience, Sloan Management Review, 37, Autumn, pp. 55-64.
Basili, V. R. and Rombach, H. D. 1988 The TAME Project: Towards Improvement-Oriented Software Environments, IEEE Transactions on Software Engineering, 14(6), 758-773.
Basili, V. R. and Selby, R. W. 1991 Paradigms for Experimentation and Empirical Studies in Software Engineering, Reliability Engineering and System Safety, 32(1–2), pp. 171-191.
Basili, V. R., Selby, R. W. and Hutchens, D. H. 1986 Experimentation in Software Engineering, IEEE Transactions on Software Engineering, 12(7), 733-743.
Black, S. A. and Porter, L. J. 1996 Identification of the Critical Factors of TQM, Decision Sciences, 27(1), 1-21.
Blalock, H. M. 1969 Theory Construction: From Verbal to Mathematical Formulations, Englewood Cliffs, N.J.: Prentice-Hall.
Carmines, E. G. and Zeller, R. A. 1979 Reliability and Validity Assessment, Sage University Paper series on Quantitative Applications in the Social Sciences, series no. 07-017, Newbury Park, CA: Sage.
Cattell, R. B. 1966 The Scree Test for the Number of Factors, Multivariate Behavioral Research, 1, 245-276.
Choo, C. W. 1995 The Knowing Organization: How Organizations Use Information to Construct Meaning Create Knowledge and Make Decisions, International Journal of Information Management, 16(5), 329-340.
Choo, C. W. 1998 The Knowing Organization: How Organizations Use Information to Construct Meaning, Create Knowledge, and Make Decisions, New York: Oxford University Press.
Cliff, N. R. 1988 The Eigenvalues-greater-than-one Rule and the Reliability of Components, Psychological Bulletin, 103, 276-279.
Cohen, J. 1988 Statistical Power Analysis for the Behavioral Sciences, Second Edition, Hillsdale, New Jersey: Laurence Erlbaum.
Comrey, A. 1973 A First Course on Factor Analysis, London: Academic Press.
Cronbach, L. J. 1951 Coefficient Alpha and the Internal Consistency of Tests, Psychometrica, 16, 297-334, September.
Cronbach, L. J. 1971 Test Validation, in Educational Measurement, 2nd Edition, R. L. Thorndike (ed.), American Council on Education, Washington, D.C, pp. 443-507.
Crosby, P. B. 1979 Quality is Free: The Art of Making Quality Certain, New York: McGraw-Hill.
Crosby, P. B. 1984 Quality Without Tears, New York: McGraw-Hill.
Crosby, P. B. 1996 Quality Is Still Free: Making Quality Certain in Uncertain Times, New York: McGraw-Hill.
Davis, D. 1996 Business Research for Decision Making, Fourth Edition, Belmont, California: Duxbury Press.
Deming, W. E. 1982 Quality, Productivity, and Competitive Position, Camebridge, Massachusetts: MIT Centre for Advanced Engineering Study.
Deming, W. E. 1986 Out of the Crisis, Cambridge, Massachusetts: MIT Center for Advanced Engineering Study.
Deephouse, C., Mukhopadhyay, T., Goldenson, D. R. and Kellner, M. I. 1996 Software Processes and Project Performance, Journal of Management Information Systems, 12(3), 187-205.
DeVellis, R. F. 1991 Scale Development: Theory and Applications, Newbury Park, CA: Sage.
Dybå, T. 2000a Improvisation in Small Software Organizations, IEEE Software, 17(5), Sept.–Oct.
Dybå, T. (Ed.) 2000b SPIQ—Software Process Improvement for better Quality: Methodology Handbook, Version 3.0 (Final), IDI Report 2/2000, Norwegian University of Science and Technology, Trondheim, Norway (in Norwegian).
EFQM 1999 The EFQM Excellence Model, Brussels: European Foundation for Quality Management.
El Emam 1998 The Internal Consistency of the ISO/IEC 15504 Software Process Capability Scale, Proceedings of the Fifth International Symposium on Software Metrics, IEEE Computer Society Press, pp. 72-81.
El Emam, K. and Birk, A. 2000 Validating the ISO/IEC 15504 Measure of Software Requirements Analysis Process Capability, IEEE Transactions on Software Engineering, 26(6), 541-566.
El Emam, K. and Madhavji, N. H. 1995 The Reliability of Measuring Organizational Maturity, Software Process—Improvement and Practice, 1, 3-25.
El Emam, K. and Madhavji, N. H. 1996 An Instrument for Measuring the Success of the Requirements Engineering Process in Information Systems Development, Empirical Software Engineering, 1(3), 201-240.
El Emam, K., Fusaro, P. and Smith, B. 1999 Success Factors and Barriers for Software Process Improvement, in R. Messnarz, and C. Tully (Eds.) Better Software Practice for Business Benefit: Principles and Experiences, Los Alamitos, California: IEEE Computer Society Press, pp. 355-371.
El Emam, K., Goldenson, D. R., McCurley J. and Herbsleb, J. 1998 Success or Failure? Modeling the Likelihood of Software Process Improvement, Technical Report, ISERN-98-15, International Software Engineering Research Network.
Feigenbaum, A. V. 1991 Total Quality Control, Fortieth Anniversary Edition, New York: McGraw-Hill.
Fink, A. and Kosecoff, J. 1998 How to Conduct Surveys: A Step-By-Step Guide, Second Edition, Thousand Oaks, California: Sage Publications.
French, W. L. and Bell, C. H. Jr. 1999 Organization Development: Behavioral Science Interventions for Organization Improvement, Sixth Edition, Upper Saddle River, New Jersey: Prentice-Hall.
Fusaro, P., El Emam, K. and Smith, B. 1998 The Internal Consistency of the 1987 SEI Maturity Questionnaire and the SPICE Capability Dimension, Empirical Software Engineering, 3(2), 179-201.
Garvin, D. A. 1983 Quality on the Line, Harvard Business Review, 61(5), 65-75.
Garvin, D. A. 1984 Japanese Quality Management, Columbia Journal of World Business, 19(3), 3-19.
Garvin, D. A. 1993 Building a Learning Organization, Harvard Business Review, 71(4), 78-91.
Goldenson, D. R. and Herbsleb, J. (1995) After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success, Technical Report, CMU/SEI-95-TR-009, Carnegie Mellon University, Software Engineering Institute.
Goldenson, D. R., El Emam, K., Herbsleb, J. and Deephouse, C. 1999 Empirical Studies of Software Process Assessment Methods, in K. El Emam and N. H. Madhavji (Eds.), Elements of Software Process Assessment and Improvement, Los Alamitos, California: IEEE Computer Society Press, pp. 177-218.
Guilford, J. P. 1954 Psychometric Methods, 2nd edition, New York: McGraw-Hill.
Harrison, R., Badoo, N., Barry, E., Biffl, S., Parra, A., Winter, B. and Wuest, J. 1999 Directions and Methodologies for Empirical Software Engineering Research, Empirical Software Engineering, 4(4), 405-410.
Humphrey, W. S. 1989 Managing the Software Process, Reading, Massachusetts: Addison-Wesley.
Humphrey, W. S. 1997 Managing Technical People: Innovation, Teamwork, and the Software Process, Reading, Massachusetts: Addison-Wesley.
Hunt, R. and Buzan, T. 1999 Creating a Thinking Organization: Groundrules for Success, Hampshire, England: Gower.
Hunter, J. E. and Schmidt, F. L. 1990 Methods of Meta-Analysis: Correcting Error and Bias in Research Findings, Newbury Park, California: Sage Publications.
Ishikawa, K. 1986 Guide to Quality Control, Second Edition, New York: Quality Resources.
Ishikawa, K. 1990 Introduction to Quality Control, London: Chapman & Hall.
ISO/DIS 9000 2000 Quality Management Systems—Fundamentals and Vocabulary.
ISO/DIS 9004 2000 Quality Management Systems—Guidelines for Performance Improvements.
ISO/IEC TR 15504-7 1998 Information Technology—Software Process Assessment—Part 7: Guide for Use in Process Improvement.
Jeffery, D. R. and Votta, L. G. 1999 Guest Editor's Special Section Introduction, IEEE Transactions on Software Engineering, 25(4), 435-437.
Juran, J. M. 1992 Juran on Quality by Design: The New Steps for Planning Quality into Goods and Services, New York: Free Press.
Juran, J. M. and Godfrey, A. B. (Eds.) 1999 Juran's Quality Handbook, Fifth Edition, New York: McGraw-Hill.
Kaiser, H. F. 1960 The Application of Electronic Computers to Factor Analysis, Educational and Psychological Measurements, 20, 141-151.
Kaiser, H. F. 1970 A Second Generation Little Jiffy, Psychometrika, 35, 401-417.
Kanuk, L. and Berenson, C. 1975 Mail Surveys and Response Rates: A Literature Review, Journal of Marketing Research, 12, 440-453.
Kerlinger, F. 1986 Foundations of Behavioral Research, Holt, Rinehart and Winston.
Larsen, E. A, and Kautz, K. 1997 Quality Assurance and Software Process Improvement in Norway, Software Process—Improvement and Practice, 3(2), 71-86.
Lawrence, P. R. and Lorsch, J. W. 1986 Organization and Environment: Managing Differentiation and Integration, 2nd Edition, Boston: Harvard Business School Press.
Likert, R. 1932 A Technique for the Measurement of Attitudes, Archives of Psychology, 22(140).
Likert, R. 1967 The Human Organization: Its Management and Value, New York: McGraw-Hill.
Likert, R. and Roslow, S. 1934 The Effects Upon the Reliability of Attitude Scales of Using Three, Five or Seven Alternatives, Working Paper, New York University.
Lissitz, R. W. and Green, S. B. 1975 Effects of the Number of Scale Points on Reliability: A Monte Carlo Approach, Journal of Applied Psychology, 60, 10-13, February.
March, J. G. 1991 Exploration and Exploitation in Organizational Learning, Organization Science, 2(1), 71-87.
March, J. G. 1999 The Pursuit of Organizational Intelligence, Malden, Massachusetts: Blackwell.
Miller, G. A. 1956 The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information, Psychological Review, 63, 81-97.
Neuman, W. L. 2000 Social Research Methods: Qualitative and Quantitative Approaches, Fourth Edition, Boston: Allyn and Bacon.
Nevis, E. C., DiBella, A. J. and Gould, J. M. 1995 Understanding Organizations as Learning Systems, Sloan Management Review, 37, 73-85.
Nonaka, I. 1991 The Knowledge-Creating Company, Harvard Business Review, 69(6), 96-104.
Nonaka, I. 1994 A Dynamic Theory of Organizational Knowledge Creation, Organization Science, 5(1), 14-37.
Nonaka, I. and Takeuchi, H. 1995. The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation, New York: Oxford University Press.
Novick, M. and Lewis, G. 1967 Coefficient Alpha and the Reliability of Composite Measurements, Psychometrika, 32, 1-13.
Nunnally, J. C. 1978 Psychometric Theory, Second edition, New York: McGraw-Hill.
Nunnally, J. C. and Bernstein, I. A. 1994 Psychometric Theory, Third edition, New York: McGraw-Hill.
Powell, T. C. 1995 Total Quality Management as Competitive Advantage: A Review and Empirical Study, Strategic Management Journal, 16, 15-37.
Rogers, E. M. 1995 Diffusion of Innovations, Fourth Edition, New York: The Free Press.
Sanders, M. (Ed.) 1998 The SPIRE Handbook: Better, Faster, Cheaper Software Development in Small Organisations, Dublin: Centre for Software Engineering Ltd.
Saraph, J. V., Benson, P. G. and Schroeder, R. G. 1989. An Instrument for Measuring the Critical Factors of Quality Management, Decision Sciences, 20(4), 810-829.
Schaffer, R. H. and Thomson, H. A. 1992. Successful Change Programs Begin with Results, Harvard Business Review, 70, 80-89.
Senge, P. M. 1990 The Fifth Discipline: The Art and Practice of the Learning Organization, New York: Doubleday.
Senge, P. M., Kleiner, A., Roberts, C., Ross, R. and Smith, B. 1994 The Fifth Discipline Fieldbook: Strategies and Tools for Building a Learning Organization, New York: Currency/Doubleday.
Senge, P. M., Kleiner, A., Roberts, C., Ross, R., Roth, G. and Smith, B. 1999 The Dance of Change: The Challenges of Sustaining Momentum in Learning Organizations, New York: Currency/Doubleday.
Spector, P. 1992 Summated Rating Scale Construction: An Introduction, Sage University Paper series on Quantitative Applications in the Social Sciences, series no. 07-082, Newbury Park, CA: Sage.
SPSS 1999a SPSS Base 9.0: User's Guide, Chicago, IL: SPSS Inc.
SPSS 1999b SPSS Base 9.0: Applications Guide, Chicago, IL: SPSS Inc.
Stelzer, D. and Mellis, W. 1998 Success Factors of Organizational Change in Software Process Improvement, Software Process—Improvement and Practice, 4(4), 227-250.
Stelzer, D., Mellis, W. and Herzwurm, G. 1996 Software Process Improvement via ISO 9000? Results of Two Surveys among European Software Houses, Proceedings of the 29th Hawaii International Conference on Systems Sciences, January 3–6, Wailea, Hawaii, USA.
Stevens, J. 1992 Applied Multivariate Statistics for the Social Sciences, London: Lawrence Erlbaum.
Straub, D. W. 1989 Validating Instruments in MIS Research, MIS Quarterly, 13(2), 147-169.
Stålhane, T., Borgersen, P. C. and Arnesen, K. 1997 In Search of the Customer's Quality View, Journal of Systems Software, 38, 85-93.
Saarinen, T. 1996 An Expanded Instrument for Evaluating Information System Success, Information & Management, 31, 103-118.
Taguchi, G. 1986 Introduction to Quality Engineering: Designing Quality into Products and Processes, Tokyo: Asian Productivity Organization.
Taguchi, G., Elsayed, E. A. and Hsiang, T. C. 1989 Quality Engineering in Production Systems, New York: McGraw-Hill.
Teng, J. T. C., Jeong, S. R. and Grover, V. 1998 Profiling Successful Reengineering Projects, Communications of the ACM, 41(6), 96-102.
Van de Ven, A. H. and Ferry, D. L. 1980 Measuring and Assessing Organizations, New York: John Wiley & Sons.
Winter, S. G. 1994 Organizing for Continuous Improvement: Evolutionary Theory Meets the Quality Revolution, in J. A. C. Baum and J. Singh (Eds.) The Evolutionary Dynamics of Organizations, Oxford University Press.
Yusof, S. M. and Aspinwall, E. 1999 Critical Success Factors for Total Quality Management Implementation in Small and Medium Enterprises, Total Quality Management, 10 (4&5), 803-809.
Zahran, S. 1998 Software Process Improvement: Practical Guidelines for Business Success, Harlow, England: Addison-Wesley.
Zwick, W. R. and Velicer, W. F. 1986 Comparison of Five Rules for Determining the Number of Components to Retain, Psychological Bulletin, 99, 432-442.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Dyba, T. An Instrument for Measuring the Key Factors of Success in Software Process Improvement. Empirical Software Engineering 5, 357–390 (2000). https://doi.org/10.1023/A:1009800404137
Issue Date:
DOI: https://doi.org/10.1023/A:1009800404137