Skip to main content
SpringerLink
Log in

A research framework for building SPI proposals in small organizations: the COMPETISOFT experience

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Establishing a research strategy that is suitable for undertaking research on software engineering is vital if we are to guarantee that research products are developed and validated following a systematic and coherent method. We took this into account as we carried out the COMPETISOFT research project, which investigated software process improvement (SPI) in the context of Latin American small companies. That experience has enabled us to develop a research strategy based on the integrated use of action research and case study methods. This paper introduces the proposed research strategy and provides extensive discussion of its application for: (1) developing the Methodological framework of COMPETISOFT for SPI, (2) putting this framework into practice in eight small software companies, and (3) refine the Methodological framework due to the practice feedback. The use of this research strategy allowed us to observe that it was suitable for developing, refining, improving, applying, and validating COMPETISOFT’s Methodology framework. Furthermore, having seen it applied, we believe that this strategy offers a successful integration of action research and case study, which can be useful for conducting research in other software engineering areas which address needs of small software companies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Aaen, I., Arent, J., Mathiassen, L., & Ngwenyama, O. (2001). A conceptual MAP of software process improvement. Scandinavian Journal of Information Systems, 13, 81–101.

    Google Scholar 

  • Avison, D., Lan, F., Myers, M., & Nielsen, A. (1999). Action research. Communications of the ACM, 42(1), 94–97.

    Article  Google Scholar 

  • Basili, V. (2000). Keynote on “Experimental software engineering”. In Proceedings 7th European workshop on software process technology (EWSPT 2000), Kaprun (Austria). Springer LNCS 1780, pp. 150.

  • Baskerville, R. (1997). Distinguishing action research from participative case studies. Journal of Systems and Information Technology, 1(1), 25–45.

    Article  Google Scholar 

  • Baskerville, R. (1999). Investigating information systems with action research. Communications of the Association for Information Systems, 2(19), 1–32.

    Google Scholar 

  • Baskerville, R., & Wood-Harper, A. T. (1998). Diversity in information systems action research methods. European Journal of Information Systems, 7(2), 90–107.

    Article  Google Scholar 

  • Benbasat, I., Goldstein, D., & Mead, M. (1987). The case research strategy in studies of information systems. MIS Quarterly, 11(3), 369–386.

    Article  Google Scholar 

  • Braa, K., & Vidgen, R. (1999). Interpretation, intervention, and reduction in the organizational laboratory: A framework for in-context information system research. Accounting Management and Information Technology, 9, 25–47.

    Article  Google Scholar 

  • Brereton, P., Kitchenham, B., Budgen, D., & Li, Z. (2008). Using a protocol template for case study planning. In G. Visaggio, M. T. Baldassarre, S. Linkman, & M. Turner (Eds.), Evaluation and assessment in software engineering (pp. 1–8). Bari, Italy: British Computer Society.

    Google Scholar 

  • Casey, V., & Richardson, I. (2004). A practical application of the IDEAL model. Software Process: Improvement and Practice, 9(3), 123–132.

    Article  Google Scholar 

  • Checkland, P. (1991). From framework through experience to learning: The essential nature of action research. In H. Nissen, H. Klein, & R. Hirscheim (Eds.), Information systems research: Contemporary approaches and emergent traditions (pp. 397–403). The Netherlands: North Holland.

    Google Scholar 

  • Chiasson, M., Germonprez, M., & Mathiassen, L. (2009). Pluralist action research: A review of the information systems literature. Information Systems Journal, 19(1), 31–54.

    Article  Google Scholar 

  • Coghlan, D. (2001). Insider action research projects: Implications for practicing managers. Management Learning, 32, 49–60.

    Article  Google Scholar 

  • Cruz Mendoza, R., Morales Trujillo, M., Morgado, C. M., Oktaba, H., Ibarguengoitia, G. E., Pino, F. J., et al. (2009). Supporting the software process improvement in very small entities through e-learning: The HEPALE! Project. In 2009 Mexican international conference on computer science (ENC), September 21–25, 2009, pp. 221–231. doi:10.1109/enc.2009.33

  • CYTED (2015). COMPETISOFT methodological framework on EPF composer (in Spanish). http://alarcos.esi.uclm.es/competisoft/web/completo/index.htm

  • Davison, R. (1998). An action research perspective of group support systems: How to improve meetings in Hong Kong. Ph.D. Thesis, Hong Kong: City University of Hong Kong.

  • Davison, R., Martinsons, M. G., & Kock, N. (2004). Principles of canonical action research. Information Systems Journal, 14(1), 65–86.

    Article  Google Scholar 

  • Dittrich, Y., Rönkkö, K., Eriksson, J., Hansson, C., & Lindeberg, O. (2008). Cooperative method development. Combining qualitative empirical research with method, technique and process improvement. Empirical Software Engineering, 13(3), 231–260.

    Article  Google Scholar 

  • French, W. L., & Bell, C. H. (1999). Organization development: Behavioral science interventions for organization improvement. London: Prentice-Hall.

    Google Scholar 

  • Garzás, J., Pino, F., Piattini, M., & Fernández, C. (2013). A maturity model for the Spanish software industry based on ISO standards. Computer Standards & Interfaces, 35(6), 616–628.

    Article  Google Scholar 

  • Glass, R. L., Vessey, I., & Ramesh, V. (2002). Research in software engineering: An analysis of the literature. Information and Software Technology, 44(8), 491–506.

    Article  Google Scholar 

  • Gustavsen, B. (1993). Action research and the generation of knowledge. Human Relations, 46, 1361–1365.

    Article  Google Scholar 

  • Hareton, L., & Terence, Y. (2001). A process framework for small projects. Software Process: Improvement and Practice, 6(2), 67–83.

    Article  Google Scholar 

  • Hernández, M., Florez, A., Pino, F., Garcia, F., Piattini, M., & Ibargüengoitia, G., et al. (2008). Supporting the improvement process for small software enterprises through a software tool. In Software engineering symposium during ninth Mexican international conference on computer science (ENC’08). October, 2008. SES Proceedings, Mexicali, México, pp. 1–8.

  • Höst, M., & Runeson, P. (2007) Checklists for software engineering case study research. In Proceedings of 1st international symposium on empirical software engineering & measurement (ESEM). IEEE Computer Society Press, pp. 479–482.

  • Howard, M., Vidgen, R., & Powell, P. (2004). Exploring industry dynamics in e-procurement: Sense making by collaborative investigation. In 13th European conference on information systems, Turku, Finland, pp. 1–12.

  • ISO (2011). ISO/IEC 29110. Software engineering—lifecycle profiles for very small entities (VSEs). Genova: International Organization for Standardization.

  • Iversen, J., & Mathiassen, L. (2003). Cultivation and engineering of a software metrics program. Information Systems Journal, 13(1), 3–20.

    Article  Google Scholar 

  • Iversen, J., Mathiassen, L., & Nielsen, P. (2004). Managing risk in software process improvement: An action research approach. MIS Quarterly, 28(3), 395–433.

    Google Scholar 

  • Johnson, D. L., & Brodman, J. G. (1999). Tailoring the CMM for small businesses, small organizations, and small projects. In K. El Emam & N. H. Madhavji (Eds.), Elements of software process assessment and improvement (pp. 239–259). Silver Spring, MD: IEEE CS Press.

    Google Scholar 

  • Kauppinen, M., Vartiainen, M., Kontio, J., Kujala, S., & Sulonen, R. (2004). Implementing requirements engineering processes throughout organizations: Success factors and challenges. Information and Software Technology, 46(14), 937–953.

    Article  Google Scholar 

  • Kock, N., McQueen, R., & Scott, J. (1997). Can action research be made more rigorous in a positivistic sense? The contribution of an interpretive approach. Journal of Systems and Information Technology, 1, 1–24.

    Article  Google Scholar 

  • Laporte, C., Alexandre, S., & O’Connor, R. (2008a). A software engineering lifecycle standard for very small enterprises. In EuroSPI 2008, Dublin, Ireland. Springer CCIS, pp. 129–141.

  • Laporte, C., Alexandre, S., Renault, A., & Crowder, K. V. (2008b). The development of international standards for very small enterprises. In 18th annual international symposium of INCOSE (INCOSE 2008), Amsterdam, The Netherlands, pp. 1–12.

  • Levin, M. (1993). Creating networks for rural economic development in Norway. Human Relations, 46, 193–218.

    Article  Google Scholar 

  • Luzuriaga, J., Martínez, R., & Cechich, A. (2008). Setting SPI practices in Latin America: An exploratory case study in the justice area. In 2nd international conference on theory and practice of electronic governance (ICEGOV 2008), pp. 172–177.

  • Martínez-Ruiz, T., Pino, F., León-Pavón, E., García, F., & Piattini, M. (2009). Supporting the process assessment through a flexible software environment. In Software and data technologies (third international conference, ICSOFT 2008). Springer CCIS 47, pp. 187–199.

  • Mathiassen, L. (2002). Collaborative practice research. Information Technology & People, 15(4), 321–345.

    Article  Google Scholar 

  • Mathiassen, L., Pries-Heje, J., & Ngwenyama, O. (2002). Improving software organizations: From principles to practice. Upper Saddle River, NJ: Addison-Wesley.

    Google Scholar 

  • McKay, J., & Marshall, P. (2001). The dual imperatives of action research. Information Technology & People (Special Issue on Action Research in Information Systems), 14(1), 46–59.

    Google Scholar 

  • McTaggart, R. (1991). Principles for participatory action research. Adult Education Quarterly, 41(3), 168–187.

    Article  Google Scholar 

  • Medeiros, P. S., & Travassos, G. (2011). Action research can swing the balance in experimental software engineering. In M. Zelkowitz (Ed.), Advances in computers (Vol. 86, pp. 205–276). Burlington: Academia Press.

    Google Scholar 

  • Mejía, R., López, A., & Molina, A. (2007). Experiences in developing collaborative engineering environments: An action research approach. Computers in Industry, 58(4), 329–346.

    Article  Google Scholar 

  • Montesi, M., & Lago, P. (2008). Software engineering article types: An analysis of the literature. Journal of Systems and Software, 81(10), 1694–1714.

    Article  Google Scholar 

  • Moody, D. (2000). Building links between IS research and professional practice: Improving the relevance and impact of IS research. In Proceedings of the 21st international conference on information systems, Brisbane (Australia), pp. 351–360.

  • Oktaba, H. (2006). MoProSoft®: A software process model for small enterprises. In S. Garcia (Ed.), Proceedings of the first international research workshop for process improvement in small settings (pp. 93–101). Pittsburgh: Carnegie Mellon University.

    Google Scholar 

  • Oktaba, H., Garcia, F., Piattini, M., Pino, F., Alquicira, C., & Ruiz, F. (2007). Software process improvement: The COMPETISOFT project. IEEE Computer, 40(10), 21–28.

    Article  Google Scholar 

  • Philips, P. A. (1998). Disseminating and applying the best evidence. Medical Journal of Australia (MJA), 168, 260–261.

    Google Scholar 

  • Pino, F., Garcia, F., & Piattini, M. (2008). Software process improvement in small and medium software enterprises: A systematic review. Software Quality Journal, 16(2), 237–261.

    Article  Google Scholar 

  • Pino, F., García, F., & Piattini, M. (2009). An integrated framework to guide software process improvement in small organizations. In European systems & software process improvement and innovation (EuroSPI 2009), Madrid, Spain. CCIS 42, Springer, pp. 213–224.

  • Pino, F., Ruiz, F., García, F., & Piattini, M. (2012). A software maintenance methodology for small organizations: Agile_MANTEMA. Journal of Software Maintenance and Evolution: Research and Practice, 24(8), 851–876.

    Article  Google Scholar 

  • Polo, M., Piattini, M., & Ruiz, F. (2002). Using a qualitative research method for building a software maintenance methodology. Software Practice and Experience, 32(13), 1239–1260.

    Article  MATH  Google Scholar 

  • Richardson, I. (2001). Software process matrix: A small company SPI model. Software Process: Improvement and Practice, 6(3), 157–165.

    Article  Google Scholar 

  • Robson, C. (2002). Real world research (2nd ed.). Oxford: Blackwell.

    Google Scholar 

  • Runeson, P., & Höst, M. (2009). Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering, 14(2), 131–164.

    Article  Google Scholar 

  • Saiedian, H., & Carr, N. (1997). Characterizing a software process maturity model for small organizations. ACM SIGICE Bulletin, 23(1), 2–11.

    Article  Google Scholar 

  • Seaman, C. B. (1999). Qualitative methods in empirical studies of software engineering. IEEE Transaction on Software Engineering, 25(4), 557–572.

    Article  Google Scholar 

  • Staples, M., Niazi, M., Jeffery, R., Abrahams, A., Byatt, P., & Murphy, R. (2007). An exploratory study of why organizations do not adopt CMMI. Journal of Systems and Software, 80(6), 883–895.

    Article  Google Scholar 

  • Travassos, G., Medeiros, P. S., Gomes, P., Dias, A. C., & Biolchini, J. (2008). An environment to support large scale experimentation in software engineering. In Proceedings of the 13th IEEE international conference on engineering of complex computer systems (ICECCS 2008), Belfast, Northern Ireland. CS Digital Library, pp. 193–202.

  • Wadsworth, Y. (1998). What is participatory Action Research? Action Research International (Paper 2).

  • Walsham, G. (1993). Interpreting information systems in organizations. Chichester, UK: Wiley.

    Google Scholar 

  • Wohlin, C. (2005). An analysis of the most cited articles in software engineering journals. Information and Software Technology, 47(15), 957–964.

    Article  Google Scholar 

  • Wood-Harper, T. (1985). Research methods in information systems: Using action research. In E. Mumford, R. Hirschheim, G. Fitzgerald, & A. T. Wood-Harper (Eds.), Research methods in information systems (pp. 169–191). Amsterdam, North-Holland: Elsevier Science Publishers B.V.

    Google Scholar 

  • Wouters, M. (2009). A developmental approach to performance measures—results from a longitudinal case study. European Management Journal, 27(1), 64–78.

    Article  Google Scholar 

  • Yin, R. K. (2003). Case study research: Design and methods. Newbury Park: Sage Publications.

    Google Scholar 

  • Zannier, C., Melnik, G., & Maurer, F. (2006). On the success of empirical studies in the international conference on software engineering. In International conference on software engineering (ICSE 2006), Shanghai, China, pp. 341–350.

  • Zelkowitz, M. (2009). An update to experimental models for validating computer technology. Journal of Systems and Software, 82(3), 373–376.

    Article  Google Scholar 

Download references

Acknowledgments

This work has been funded by: (1) the GEODAS-BC project (Ministerio de Economía y Competitividad and Fondo Europeo de Desarrollo Regional FEDER, TIN2012-37493-C03-01), (2) the GLOBALIA project (Consejería de Educación, Ciencia y Cultura y Fondo Europeo de Desarrollo Regional FEDER, PEII11-0291-5274), and (3) LATIn Project (Universidad del Cauca, VRI3930). Francisco J. Pino acknowledges the contribution of the University of Cauca, where he works as a Full Professor.

Author information

Authors and Affiliations

  1. IDIS Research Group, Electronic and Telecommunications Engineering Faculty, University of Cauca, Calle 5 # 4 – 70, Popayán, Colombia

    Francisco J. Pino

  2. ALARCOS Research Group, Information Systems and Technologies Department, University of Castilla–La Mancha, Paseo de la Universidad 4, 13071, Ciudad Real, Spain

    Felíx García & Mario Piattini

  3. KUALI-KAANS Research Group, Facultad de Ciencias, National Autonomous University of Mexico (UNAM), Ciudad Universitaria, 04510, Mexico, D.F., Mexico

    Hanna Oktaba

Authors
  1. Francisco J. Pino
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Felíx García
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mario Piattini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hanna Oktaba
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francisco J. Pino.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pino, F.J., García, F., Piattini, M. et al. A research framework for building SPI proposals in small organizations: the COMPETISOFT experience. Software Qual J 24, 489–518 (2016). https://doi.org/10.1007/s11219-015-9278-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-015-9278-2

Keywords

Search

Navigation