Assessing the effects of introducing a new software development process: a methodological description

  • Agneta Nilsson
  • Laura M. CastroEmail author
  • Samuel Rivas
  • Thomas Arts
Regular Paper


In this article, we report from a 22-months long action research study in which we evaluate the usefulness of a set of software development tools in an industrial setting, a small software company. We focus on how developers in the industry use and adopt these tools, what expectations they have on them, how the tools can be improved, and how the adoption process itself can be improved. We describe these change processes from a methodological perspective, how we monitored the processes, how we reviewed the outcomes, and the strategies that we applied. We show how the processes evolved, intermediate results, and the steps that were taken along the way based on the outcomes. We believe that the described study may inspire other tool-developers and/or researchers to organize similar studies to further our understanding of the complex processes involved in the adoption of software development tools in industry.


Software development process  Process improvement Process change Change assessment 


  1. 1.
    Abrahamsson, P.: Is management commitment a necessity after all in software process improvement?. In: Proceedings of the 26th Euromicro Conference 2, 246–253 (2000)Google Scholar
  2. 2.
    Abrahamsson, P.: Rethinking the concept of commitment in software process improvement. Scand. J. Inf. Syst. 13, 69–98 (2001)Google Scholar
  3. 3.
    Arts, T., Hughes, J., Johansson, J., Wiger, U.: Testing telecoms software with Quviq QuickCheck, pp. 2–10 (2006)Google Scholar
  4. 4.
    Barki, H., Hartwick, J.: Measuring user participation, user involvement, and user attitude. MIS Q. Manag. Inf. Syst. 18(1), 59–79 (1994)Google Scholar
  5. 5.
    Baronas, A.M.K., Louis, M.R.: Restoring a sense of control during implementation: How user involvement leads to system acceptance. MIS Q. Manag. Inf. Syst. 12(1), 111–123 (1988)Google Scholar
  6. 6.
    Bartunek, J., Louis, M.R.: Qualitative research methods. In: Insider/outsider Team Research. SAGE Publications, Thousand Oaks (1996)Google Scholar
  7. 7.
    Baskerville, R.L., Wood-Harper, A.T.: A critical perspective on action research as a method for information systems research. J. Inf. Technol. 11(3), 235–246 (1996)Google Scholar
  8. 8.
    Beath, C.M.: Supporting the information technology champion. MIS Q. Manag. Inf. Syst. 15(3), 355–370 (1991)CrossRefGoogle Scholar
  9. 9.
    Castro, D., Gulías, V.M., Benac-Earle, C., Fredlund, L., Rivas, S.: A case study on verifying a supervisor component using McErlang. Electron. Notes Theo. Comput. Sci. 271, 23–40 (2011)CrossRefGoogle Scholar
  10. 10.
    Claessen, K., Hughes, J.: QuickCheck: A lightweight tool for random testing of Haskell programs. In: Proceedings of ACM SIGPLAN International Conference on Functional Programming, pp. 268–279 (2000)Google Scholar
  11. 11.
    Coghlan, D., Brannick, T.: Doing Action Research in Your Own Organization. SAGE Publications, Thousand Oaks (2001)Google Scholar
  12. 12.
    Cooperrider, D.L., Whitney, D.K.: Appreciative Inquiry: A Positive Revolution in Change. Berrett-Koehler Series. Berrett-Koehler, San Francisco (2005)Google Scholar
  13. 13.
    Cooperrider, D.L., Whitney, D.K., Stavros, J.M.: Appreciative Inquiry Handbook: The First in a Series of AI Workbooks for Leaders of Change. Berrett-Koehler Series, Lakeshore Communications (2003)Google Scholar
  14. 14.
    Damm, L., Lundberg, L., Wohlin, C.: Faults-slip-through-a concept for measuring the efficiency of the test process. Softw. Process Improv. Pract. 11, 47–59 (2006)CrossRefGoogle Scholar
  15. 15.
    Dawson, P.: Organisational Change: A Processual Approach. Athenaeum Press Ltd., Newcastle (1994)Google Scholar
  16. 16.
    Dawson, P.: In at the deep end: conducting processual research on organisational change. Scand. J. Manag. 13(4), 389–405 (1997)CrossRefGoogle Scholar
  17. 17.
    Fichman, R.G., Kemerer, C.F.: The illusory diffusion of innovation: an examination of assimilation gaps. Inf. Syst. Res. 10, 255–275 (1999)CrossRefGoogle Scholar
  18. 18.
    Fredlund, L., Svensson, H.: McErlang: a model checker for a distributed functional programming language. ACM SIGPLAN Notices 42(9), 125–136 (2007)CrossRefGoogle Scholar
  19. 19.
    Gable, G.G.: Integrating case study and survey research methods: an example in information systems. Eur. J. Inf. Syst. 3, 112–126 (1994)CrossRefGoogle Scholar
  20. 20.
    Geras, A., Smith, M., Miller, J.: A prototype empirical evaluation of test driven, development, pp. 405–416 (2004)Google Scholar
  21. 21.
    Ginzberg, M.J.: Early diagnosis of mis implementation failure: promising results and unanswered questions. Manag. Sci. 27(4), 459–478 (1981)CrossRefGoogle Scholar
  22. 22.
    Ginzberg, M.J.: Key recurrent issues in the MIS implementation process. MIS Q. Manag. Inf. Syst. 5(2), 47–59 (1981)CrossRefGoogle Scholar
  23. 23.
    Hardgrave, B.C., Davis, F.D., Riemenschneider, C.K.: Investigating determinants of software developers’ intentions to follow methodologies. J. Manag. Inf. Syst. 20(1), 123–151 (2003)Google Scholar
  24. 24.
    Hughes, J.: 9th International Symposium QuickCheck Testing for Fun and Profit, Practical Aspects of Declarative Languages. In: Lecture Notes in Computer Science, vol. 4354, pp. 1–32. Springer, Berlin (2007)Google Scholar
  25. 25.
    Jacky, J.M.F., Estublier, J., Sanlaville, R.: Tool adoption issues in a very large software company. In: Proceedings of the International Workshop on AdoptionCentric Software Engineering 9, 81–89 (2003)Google Scholar
  26. 26.
    Janzen, D., Saiedian, H.: Test-driven development: concepts, taxonomy, and future direction. Computer 38(9), 43–50 (2005)CrossRefGoogle Scholar
  27. 27.
    Jarvenpaa, S.L., Ives, B.: Executive involvement and participation in the management of information technology. MIS Q. Manag. Inf. Syst. 15(2), 205–223 (1991)CrossRefGoogle Scholar
  28. 28.
    Lui, K.M., Chan, K.C.C.: Test Driven Development and Software Process Improvement in China. Proceedings of the International Conference on Extreme Programming and Agile Processes in Software Engineering, Lecture Notes in Computer Science, vol. 3092, Springer, pp. 219–222 (2004)Google Scholar
  29. 29.
    Kotter, J.P.: Leading change: why transformation efforts fail. Harv. Business Rev. 85(1), 96–103 (2007)MathSciNetGoogle Scholar
  30. 30.
    Kwon, T.H., Zmud, R.W.: Unifying the fragmented models of information systems implementation. Wiley, New York (1987)Google Scholar
  31. 31.
    Langley, A.: Strategies for theorizing from process data. Acad. Manag. Rev. 24(4), 691–710 (1999)MathSciNetGoogle Scholar
  32. 32.
    Li, H., Thompson, S.: Tool support for refactoring functional programs, pp. 199–203 (2008)Google Scholar
  33. 33.
    Marchenko, A., Abrahamsson, P., Ihme, T.: Long-term effects of test-driven development: a case study. In: Lecture Notes in Business Information Processing 31, 13–22 (2009)Google Scholar
  34. 34.
    Mathiassen, L.: Collaborative practice research. Inf. Technol. People 15(4), 321–345 (2002)CrossRefGoogle Scholar
  35. 35.
    Melis, M., Turnu, I., Cau, A., Concas, G.: Evaluating the impact of test-first programming and pair programming through software process simulation. Softw. Process Improv. Pract. 11(4), 345–360 (2006)CrossRefGoogle Scholar
  36. 36.
    Mohr, L.B.: Explaining Organizational Behavior: The Limits and Possibilities of Theory and Research. Jossey-Bass, San Francisco (1982)Google Scholar
  37. 37.
    Nandhakumar, J., Rossi, M., Talvinen, J.: The dynamics of contextual forces of ERP implementation. J. Strateg. Inf. Syst. 14(2), 221–242 (2005)CrossRefGoogle Scholar
  38. 38.
    Oram, A., Wilson, G.: Making software : what really works, and why we believe it. O’Reilly Media Inc, Sebastopol (2010)Google Scholar
  39. 39.
    Patton, M.Q.: How to Use Qualitative Methods in Evaluation. CSE Program Evaluation Kit. SAGE Publications, Thousand Oaks (1987)Google Scholar
  40. 40.
    Pettigrew, A.M.: What is a processual analysis? Scand. J. Manag. 13(4), 337–348 (1997)CrossRefGoogle Scholar
  41. 41.
    Pettigrew, A.M., Woodman, R.W., Cameron, K.S.: Studying organizational change and development: challenges for future research. Acad. Manag. J. 44(4), 697–713 (2001)CrossRefGoogle Scholar
  42. 42.
    Prescott, M.B., Conger, S.A.: Information technology innovations: a classification by IT locus of impact and research approach. Data Base Adv. Inf. Syst. 26, 20–41 (1995)CrossRefGoogle Scholar
  43. 43.
    Property-based Testing for Erlang (Seventh Framework Programme), (2007)
  44. 44.
    Riemenschneider, C.K., Hardgrave, B.C., Davis, F.D.: Explaining software developer acceptance of methodologies: a comparison of five theoretical models. IEEE Trans. Softw. Eng. 28(12), 1135–1145 (2002)CrossRefGoogle Scholar
  45. 45.
    Rogers, E.M.: Diffusion of Innovations, 5th edn. Free Press, New York (2003)Google Scholar
  46. 46.
    Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14(2), 131–164 (2009)CrossRefGoogle Scholar
  47. 47.
    Senapathi, M.: Adoption of Software Engineering Process Innovations: The Case of Agile Software Development Methodologies. Proceedings of the International Conference on Agile Processes in Software Engineering and Extreme Programming. In: Lecture Notes in Business Information Processing, vol. 48, pp. 226–231 Springer, Berlin (2010)Google Scholar
  48. 48.
    Sultan, F., Chan, L.: The adoption of new technology: the case of object-oriented computing in software companies. IEEE Trans. Eng. Manag. 47(1), 106–126 (2000)CrossRefGoogle Scholar
  49. 49.
    Susman, G.I., Evered, R.D.: An assessment of the scientific merits of action research. Adm. Sci. Q. 23(4), 582–603 (1978) Google Scholar
  50. 50.
    Umarji, M., Seaman, C.: Predicting acceptance of software process improvement. ACM SIGSOFT Softw. Eng. Notes 30, 1–6 (2005)Google Scholar
  51. 51.
    Vincent S., L., Mahapatra, R.K.: Exploring the research in information technology implementation. Inf. Manag. 32(4), 187–201 (1997)Google Scholar
  52. 52.
    Weinberg, G.M.: Quality software management: anticipating change. In: Quality Software Management. Dorset House Publishing, New York (1997)Google Scholar
  53. 53.
    Yin, R.K.: Case study research : design and methods, 3rd edn. Sage Publications, Newbury Park (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Agneta Nilsson
    • 1
  • Laura M. Castro
    • 2
    Email author
  • Samuel Rivas
    • 3
  • Thomas Arts
    • 4
  1. 1.University of GothenburgGothenburgSweden
  2. 2.University of A CoruñaA CoruñaSpain
  3. 3.Interoud SL (Formerly LambdaStream S.L)A CoruñaSpain
  4. 4.Quviq ABGothenburgSweden

Personalised recommendations