Empirical Software Engineering

, Volume 16, Issue 4, pp 514–537 | Cite as

The search for a research method for studying OSS process innovation

  • Lutz Prechelt
  • Christopher Oezbek


Medium-sized, open-participation Open Source Software (OSS) projects do not usually perform explicit software process improvement on any routine basis. It would be useful to understand how to get such a project to accept a process improvement proposal and hence to perform process innovation. We want to determine an effective and feasible qualitative research method for studying the above question. We present (narratively) a case study of how we worked towards and eventually found such a research method. The case involves four attempts at collecting suitable data about innovation episodes (direct participation (twice), polling developers for episodes, manually finding episodes in mailing list archives) and the adaptation of the Grounded Theory data analysis methodology. Direct participation allows gathering rather rich data, but does not allow for observing a sufficiently large number of innovation episodes. Polling developers for episodes did not prove to be useful. Using mailing list archives to find data to be analyzed is both feasible and effective. We also describe how the data thus found can be analyzed based on the Grounded Theory Method with suitable adjustments. By-and-large, our findings ought to apply to studying various phenomena in OSS development processes that are similarly heavyweight and infrequent. However, specific details may block this possibility and we cannot predict which details that might be. The amount of effort involved in direct participation approaches to qualitative research can easily be underestimated. Also, survey approaches are not well-suited for many process issues in OSS, because too few developers are sufficiently process-conscious. An approach based on passive observation is a viable alternative in the OSS context due to the availability of large amounts of fairly complete archival data.


Open Source Methodology Innovation introduction 



The authors would like to thank Robert Schuster, Luis Quintela-Garcia, and Alexander Roßner, who conducted three of the studies as part of their theses, and the participants of the Open Source projects without whom this work would not have been possible. We thank the reviewers for insisting on some changes that make this article a less complicated read.


  1. Arthur WB (1989) Competing technologies, increasing returns, and lock-in by historical events. Econ J 99(394):116–131. CrossRefGoogle Scholar
  2. Avison DE, Lau F, Myers MD, Nielsen PA (1999) Action research. Commun ACM 42(1):94–97. doi:10.1145/291469.291479 CrossRefGoogle Scholar
  3. Barcellini F, Détienne F, Burkhardt JM, Sack W (2008) A socio-cognitive analysis of online design discussions in an Open Source Software community. Interact Comput 20(1):141–165. doi: 10.1016/j.intcom.2007.10.004 CrossRefGoogle Scholar
  4. Berdou E (2007) Managing the bazaar: commercialization and peripheral participation in mature, community-led F/OS software projects. Doctoral dissertation, London School of Economics and Political Science, Department of Media and CommunicationsGoogle Scholar
  5. Bergquist M, Ljungberg J (2001) The power of gifts: organizing social relationships in Open Source communities. Inf Syst J 11(4):305–320. doi: 10.1046/j.1365-2575.2001.00111.x CrossRefGoogle Scholar
  6. Buzan T, Buzan B (1993) The Mind Map book. BBC Books, LondonGoogle Scholar
  7. Charmaz K (2006) Constructing grounded theory: a practical guide through qualitative analysis, 1st edn. Sage Publications Ltd.
  8. CMMI Product Team (2006) CMMI for development, version 1.2. Tech. Rep. CMU/SEI-2006-TR-008, Software Engineering InstituteGoogle Scholar
  9. Cohen MD, March JG, Olsen JP (1972) A garbage can model of organizational choice. Adm Sci Q 17(1):1–25. CrossRefGoogle Scholar
  10. Corbin JM, Strauss AL (2008) Basics of qualitative research: techniques and procedures for developing grounded theory, 3rd edn. SAGEGoogle Scholar
  11. David PA (1985) Clio and the economics of QWERTY. Am Econ Rev 75(2):332–337. Google Scholar
  12. Davison R, Martinsons MG, Kock N (2004) Principles of canonical action research. Inf Syst J 14(1):65–86. doi: 10.1111/j.1365-2575.2004.00162.x CrossRefGoogle Scholar
  13. Denning PJ, Dunham R (2006) Innovation as language action. Commun ACM 49(5):47–52. doi:10.1145/1125944.1125974 CrossRefGoogle Scholar
  14. Dietze S (2004) Modell und optimierungsansatz für Open Source Softwareentwicklungsprozesse. Doktorarbeit, Universität Potsdam.
  15. Farrand P, Hussain F, Hennessy E (2002) The efficacy of the ‘mind map’ study technique. Med Educ 36(5):426–431. doi:10.1046/j.1365-2923.2002.01205.x CrossRefGoogle Scholar
  16. Giddens A (1984) The constitution of society: outline of the theory of structuration. University of California Press, BerkeleyGoogle Scholar
  17. Glaser BG, Strauss AL (1967) The discovery of grounded theory: strategies for qualitative research. Aldine de Gruyter, New YorkGoogle Scholar
  18. Hedlund G (1986) The hypermodern MNC—a heterarchy? Hum Resour Manag 25(1):9–35. doi: 10.1002/hrm.3930250103 CrossRefGoogle Scholar
  19. Hertel G, Niedner S, Herrmann S (2003) Motivation of software developers in Open Source projects: an internet-based survey of contributors to the Linux kernel. Research Policy 32(7):1159–1177. doi: 10.1016/S0048-7333(03)00047-7, (Open Source Software development)CrossRefGoogle Scholar
  20. Highsmith J, Cockburn A (2001) Agile software development: the business of innovation. IEEE Softw 18(5):120–122Google Scholar
  21. Jeffries R, Melnik G (2007) TDD—the art of fearless programming. IEEE Softw 24(3):24–30. doi: 10.1109/MS.2007.75 CrossRefGoogle Scholar
  22. Jorgensen DL (1989) Participant observation: a methodology for human studies, applied social research methods series, vol 15. Sage, Newbury Park, CAGoogle Scholar
  23. Krafft MF (2009) A Delphi study of the influences on innovation adoption and process evolution in a large open-source project—the case of Debian. PhD thesis, University of Limerick, Ireland, 2009.10.01—version submitted to examinersGoogle Scholar
  24. Langley A (1999) Strategies for theorizing from process data. Acad Manage Rev 24(4):691–710. MathSciNetGoogle Scholar
  25. Lave J, Wenger E (1991) Situated learning: legitimate peripheral participation. Cambridge University Press.
  26. Law J (1992) Notes on the theory of the actor-network: ordering, strategy and heterogeneity. Syst Pract 5(4):379–393CrossRefGoogle Scholar
  27. Lessig L (2000) Code and other laws of cyberspace. Basic Books, New YorkGoogle Scholar
  28. Li Q, Heckman R, Crowston K, Howison J, Allen E, Eseryel UY (2008) Decision making paths in self-organizing technology-mediated distributed teams. In: Proccedings of the international conference on information systems (ICIS) 2008, Association for Information SystemsGoogle Scholar
  29. Luhmann N (1984) Soziale Systeme. Grundriß einer allgemeinen Theorie. Suhrkamp, Frankfurt am Main,
  30. Manns ML, Rising L (2004) Fearless change: patterns for introducing new ideas. Addison-Wesley.
  31. Mayring P (2002) Einführung in die qualitative Sozialforschung. Beltz, Weinheim.
  32. Muhr T (1991) Atlas/ti — a prototype for the support of text interpretation. Qual Sociol 14(4):349–371. doi: 10.1007/BF00989645 CrossRefGoogle Scholar
  33. Novak JD, Cañas AJ (2006) The theory underlying concept maps and how to construct them. IHMC CmapTools 2006-01, Florida Institute for Human and Machine CognitionGoogle Scholar
  34. Novak JD, Gowin DB (1984) Learning how to learn. Cambridge University Press, New YorkGoogle Scholar
  35. Oezbek C (2010) Introducing automated regression testing in Open Source projects. Tech. Rep. TR-B-10-01, Freie Universität Berlin, Institut für Informatik, Berlin, Germany.
  36. Oezbek C (2010) Introducing innovations into Open Source projects. Doctoral thesis, Freie Universität Berlin (to appear)Google Scholar
  37. Oezbek C, Prechelt L (2007) On understanding how to introduce an innovation to an open source project. In: Proceedings of the 29th international conference on software engineering workshops (ICSEW ’07), IEEE computer society, Washington, DC, USA, reprinted in UPGRADE. The European Journal for the Informatics Professional vol 8(6), pp 40–44Google Scholar
  38. Oezbek C, Schuster R, Prechelt L (2008) Information management as an explicit role in OSS projects: a case study. Tech. Rep. TR-B-08-05, Freie Universität Berlin, Institut für Informatik, Berlin, Germany.
  39. O’Mahony S (2003) Guarding the commons: how community managed software projects protect their work. Res Policy 32(7):1179–1198. doi: 10.1016/S0048-7333(03)00048-9, (Open Source Software development)CrossRefGoogle Scholar
  40. O’Mahony S (2005) Nonprofit foundations and their role in community-firm software collaboration. In: Feller J, Fitzgerald B, Hissam SA, Lakhani KR (eds) Perspectives on free and open source software. The MIT Press Ltd., Cambridge, MA, chap 20, pp 393–414Google Scholar
  41. Peirce CS (1883) Studies in Logic, Little, Brown, and Company, Boston, MA, chap A Theory of Probable Inference, pp 126–181Google Scholar
  42. Quintela García L (2006) Die Kontaktaufnahme mit Open Source Software-Projekten. Eine Fallstudie. Bachelor thesis, Freie Universität Berlin.
  43. Raymond ES (1998) Homesteading the Noosphere. First Monday 3(10):n/a.
  44. Rogers EM (2003) Diffusion of innovations, 5th edn. Free Press, New YorkGoogle Scholar
  45. Salinger S, Prechelt L (2008) What happens during pair programming? In: Proceedings of the 20th annual workshop of the psychology of programming interest group (PPIG ’08), Lancaster, England.
  46. Sarma A, Maccherone L, Wagstrom P, Herbsleb J (2009) Tesseract: interactive visual exploration of socio-technical relationships in software development. In: ICSE ’09: Proceedings of the 2009 IEEE 31st international conference on software engineering. IEEE Computer Society, Washington, DC, USA, pp 23–33. doi: 10.1109/ICSE.2009.5070505 Google Scholar
  47. Schuster R (2005) Effizienzsteigerung freier softwareprojekte durch informationsmanagement. Studienarbeit, Freie Universität Berlin.
  48. Scotto M, Succi G (eds) (2005) The first international conference on Open Source Systems, GenovaGoogle Scholar
  49. Shaikh M, Cornford T (2003) Version management tools: CVS to BK in the linux kernel. In: Feller J, Fitzgerald B, Hissam S, Lakhani K (eds) Taking stock of the bazaar: the 3rd Workshop on open source software engineering. IEEE Computer Society, Portland, Oregon, pp 127–132Google Scholar
  50. Siefkes C (2007) An incrementally trainable statistical approach to information extraction based on token classification and rich context models. PhD thesis, Freie Universität Berlin, BerlinGoogle Scholar
  51. Strauss AL, Corbin JM (1998) Basics of qualitative research: techniques and procedures for developing grounded theory, 2nd edn. SAGE.
  52. Suddaby R (2006a) From the editors: what grounded theory is not. Acad Manage J 49(4):633–642CrossRefGoogle Scholar
  53. Suddaby R (2006B) From the editors: what grounded theory is not. Acad Manage J 49(4):633–642CrossRefGoogle Scholar
  54. Torvalds L, Diamond D (2001) Just for fun: the story of an accidental revolutionary. HarperCollinsGoogle Scholar
  55. Wenger E (1999) Communities of practice: learning, meaning, and identity. Cambridge University Press.
  56. West J, O’Mahony S (2005) Contrasting community building in sponsored and community founded open source projects. In: 38th annual Hawaii international conference on system sciences. IEEE Computer Society, Los Alamitos, CA, USA, vol 7, p 196c. doi:10.1109/HICSS.2005.166 CrossRefGoogle Scholar
  57. West J, O’Mahony S (2008) The role of participation architecture in growing sponsored open source communities. Ind Innov 15(2):145–168, 10.1080/13662710801970142 CrossRefGoogle Scholar
  58. Whittaker JA (2000) What is software testing? and why is it so hard? IEEE Softw 17(1):70–79. doi:10.1109/52.819971 CrossRefGoogle Scholar
  59. Windeler A (2001) Unternehmungsnetzwerke: konstitution und strukturation. VS VerlagGoogle Scholar
  60. Yamauchi Y, Yokozawa M, Shinohara T, Ishida T (2000) Collaboration with lean media: how open-source software succeeds. In: CSCW ’00: proceedings of the 2000 ACM conference on computer supported cooperative work, ACM, New York, NY, USA, pp 329–338. doi:10.1145/358916.359004
  61. Yin RK (2003) Case study research: design and methods. Sage.

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Freie Universität BerlinBerlinGermany

Personalised recommendations