Exploring the effects of coordination and communication tools on the efficiency of open source projects using data envelopment analysis

  • Stefan KochEmail author


In this paper we explore possible benefits of communication and coordination tools in open source projects using an efficiency score derived from data envelopment analysis (DEA) as dependent variable. DEA is a general non-parametric method for efficiency comparisons without asking the user to define any relations between different factors or a production function. The method can account for economies or diseconomies of scale, and is able to deal with multi-input, multi-output systems in which the factors have different scales. Using two different data sets, successful and random open source projects, retrieved from, we analyze impacts on their efficiency from the usage of communication and coordination tools. The results were mixed with no clear positive effects being proven consistently: In the data set of successful projects, mostly negative influences were found. On the contrary, tool adoption showed positive relationships to efficiency in the random data set. This stresses the importance of development status as a moderating variable and might also hint at threshold values for tool benefits. In addition, adoption of tools outside the hosting platform may be more likely for successful projects.


Open source software Data envelopment analysis Communication Coordination Tools Efficiency 


  1. Albrecht AJ, Gaffney JE (1983) Software function, source lines of code, and development effort prediction: a software science validation. IEEE Trans Softw Eng 9(6):639–648 doi: 10.1109/TSE.1983.235271 CrossRefGoogle Scholar
  2. Amor JJ, Robles G, Gonzalez-Barahona JM (2006) Effort Estimation by Characterizing Developer Activity. In: Proceedings 8th International Workshop on Economics-Driven Software Engineering Research (ICSE 2006), Shanghai, ChinaGoogle Scholar
  3. Banker RD, Chang H, Kemerer CF (1994) Evidence on economies of scale in software development. Inf Softw Technol 36(5):275–282 doi: 10.1016/0950-5849(94)90083-3 CrossRefGoogle Scholar
  4. Banker RD, Charnes A, Cooper W (1984) Some models for estimating technical and scale inefficiencies in data envelopment analysis. Manage Sci 30:1078–1092zbMATHCrossRefGoogle Scholar
  5. Banker RD, Kauffman RJ (1991) Reuse and productivity in integrated computer-aided software engineering: an empirical study. MIS Q 15(3):375–401 doi: 10.2307/249649 CrossRefGoogle Scholar
  6. Banker RD, Kemerer C (1989) Scale economies in new software development. IEEE Trans Softw Eng 15(10):416–429 doi: 10.1109/TSE.1989.559768 CrossRefGoogle Scholar
  7. Banker RD, Slaughter SA (1997) A field study of scale economies in software maintenance. Manage Sci 43(12):1709–1725zbMATHCrossRefGoogle Scholar
  8. Boehm BW, Abts C, Brown AW, Chulani S, Clark BK, Horowitz E et al (2000) Software Cost Estimation with COCOMO II. Prentice Hall, Upper Saddle River, NJGoogle Scholar
  9. Brooks FP Jr (1995) The mythical man-month: essays on software engineering. Anniversary edition. Addison-Wesley, Reading, MAGoogle Scholar
  10. Carmel E, Agarwal R (2001) Tactical approaches for alleviating distance in global software development. IEEE Softw 18(2):22–29 doi: 10.1109/52.914734 CrossRefGoogle Scholar
  11. Charnes A, Cooper W, Rhodes E (1978a) A Data Envelopment Analysis Approach to Evaluation of the Program Follow Through Experiments in U.S. Public School Education (Management Science Research Report No. 432) Carnegie-Mellon University, Pittsburgh, PAGoogle Scholar
  12. Charnes A, Cooper W, Rhodes E (1978b) Measuring the efficiency of decision making units. Eur J Oper Res 2:429–444 doi: 10.1016/0377-2217(78)90138-8 zbMATHCrossRefMathSciNetGoogle Scholar
  13. Crowston K, Annabi H, Howison J (2003) Defining open source software project success. In: Proceedings of ICIS 2003, Seattle, WAGoogle Scholar
  14. Crowston K, Annabi H, Howison J, Masango C (2004) Towards a portfolio of FLOSS Project success measures. In: Collaboration, Conflict and Control: The 4th Workshop on Open Source Software Engineering (ICSE 2004), Edinburgh, ScotlandGoogle Scholar
  15. Crowston K, Howison J, Annabi H (2006) Information systems success in free and open source software development: theory and measures. Softw Process Improv Pract 11(2):123–148 doi: 10.1002/spip.259 CrossRefGoogle Scholar
  16. Demetriou N, Koch S, Neumann G (2006) The Development of the OpenACS Community. In: Lytras M, Naeve A (eds) Open source for knowledge and learning management: strategies beyond tools. Idea, Hershey, PAGoogle Scholar
  17. Diaz-Martinez, Z. and Fernandez-Menendez, J. (2008). DEA: Data Envelopment Analysis. R package version 0.1–2Google Scholar
  18. Dixon R (2003) Open source software law. Artech House, Norwood, MAGoogle Scholar
  19. Dutoit AH, Bruegge B (1998) Communication metrics for software development. IEEE Trans Softw Eng 24(8):615–628 doi: 10.1109/32.707697 CrossRefGoogle Scholar
  20. Faraj S, Sproull L (2000) Coordinating expertise in software development teams. Manage Sci 46(12):1554–1568 doi: 10.1287/mnsc.46.12.1554.12072 CrossRefGoogle Scholar
  21. Farell MJ (1957) The measurement of productive efficiency. J R Stat Soc [Ser A] 120(3):250–290 doi: 10.2307/2343100 Google Scholar
  22. Fussel SR, Kraut RE, Lerch FJ, Scherlis WL, McNally MM, Cadiz JJ (1998) Coordination, overload and team performance: effects of team communication strategies. In: Proceedings CSCW 1998, pp 275–284, Seattle, WashingtonGoogle Scholar
  23. Gopal A, Mukhopadhyay T, Krishnan MS (2002) The role of software processes and communication in offshore software development. Commun ACM 45(4):193–2000 doi: 10.1145/505999.506008 CrossRefGoogle Scholar
  24. Herbsleb JD, Grinter RE (1999) Architectures, coordination, and distance: Conway’s Law and beyond. IEEE Softw 16(5):63–70 doi: 10.1109/52.795103 CrossRefGoogle Scholar
  25. Hertel G, Niedner S, Hermann S (2003) Motivation of software developers in open source projects: an internet-based survey of contributors to the Linux kernel. Res Policy 32(7):1159–1177 doi: 10.1016/S0048-7333(03)00047-7 CrossRefGoogle Scholar
  26. Holck J, Jorgensen N (2004) Do not check in on red: control meets anarchy in two open source projects. In: Koch S (ed) Free/open source software development. Idea Group, Hershey, PAGoogle Scholar
  27. Howison J, Crowston K (2004). The perils and pitfalls of mining SourceForge. In: Proceedings of the International Workshop on Mining Software Repositories, pp 7–11, Edinburgh, ScotlandGoogle Scholar
  28. Hu Q (1997) Evaluating alternative software production functions. IEEE Trans Softw Eng 23(6):379–387 doi: 10.1109/32.601078 CrossRefGoogle Scholar
  29. Hunt F, Johnson P (2002) On the pareto distribution of sourceforge projects. Proceedings Open Source Software Development Workshop, pp 122–129, Newcastle, UKGoogle Scholar
  30. Kitchenham B (2002) The question of scale economies in software—why cannot researchers agree? Inf Softw Technol 44(1):13–24 doi: 10.1016/S0950-5849(01)00204-X CrossRefGoogle Scholar
  31. Kitchenham B, Mendes E (2004) Software productivity measurement using multiple size measures. IEEE Trans Softw Eng 30(12):1023–1035 doi: 10.1109/TSE.2004.104 CrossRefGoogle Scholar
  32. Koch S (2004) Profiling an open source project ecology and its programmers. Electron Mark 14(2):77–88 doi: 10.1080/10196780410001675031 CrossRefGoogle Scholar
  33. Koch S (2005) Effort modeling and programmer participation in open source software projects (Arbeitspapiere zum Tätigkeitsfeld Informationsverarbeitung, Informationswirtschaft und Prozessmanagement, Nr. 03/2005). Wirtschaftsuniversität Wien, Vienna, AustriaGoogle Scholar
  34. Koch S (2007) Software evolution in open source projects—a large-scale investigation. J Softw Maint Evol 19(6):361–382 doi: 10.1002/smr.348 CrossRefGoogle Scholar
  35. Koch S (2008) Measuring the efficiency of free and open source software projects using data envelopment analysis. In: Sowe SK, Stamelos I, Samoladas I (eds) Emerging free and open source software practices. IGI, Hershey, PA, pp 25–44Google Scholar
  36. Koch S, Neumann C (2008) Exploring the effects of process characteristics on product quality in open source software development. J Database Manage 19(2):31–57Google Scholar
  37. Koch S, Schneider G (2002) Effort, cooperation and coordination in an open source software project: gnome. Inf Syst J 12(1):27–42 doi: 10.1046/j.1365-2575.2002.00110.x CrossRefGoogle Scholar
  38. Kraut RE, Streeter LA (1995) Coordination in software development. Commun ACM 38(3):69–81 doi: 10.1145/203330.203345 CrossRefGoogle Scholar
  39. Krishnan MS, Kriebel CH, Kekre S, Mukhopadhyay T (2000) An empirical analysis of productivity and quality in software products. Manage Sci 46(6):745–759 doi: 10.1287/mnsc.46.6.745.11941 CrossRefGoogle Scholar
  40. Laurent LS (2004) Understanding open source and free software licensing. O’Reilly & Associates: Cambridge, MAGoogle Scholar
  41. Leon A (2000) A Guide to software configuration management. Artech House, Norwood, MAzbMATHGoogle Scholar
  42. Mayrhauser A, Wohlin C, Ohlsson M (2000) Assessing and understanding efficiency and success of software production. Empir Softw Eng 5(2):125–154 doi: 10.1023/A:1009815306795 zbMATHCrossRefGoogle Scholar
  43. Michlmayr M (2005) Software process maturity and the success of free software projects. In: Zielinski K, Szmuc T (eds) Software engineering: evolution and emerging technologies. IOS, Amsterdam, The Netherlands, pp 3–14Google Scholar
  44. Mockus A, Fielding R, Herbsleb J (2002) Two case studies of open source software development: Apache and Mozilla. ACM Trans Softw Eng Methodol 11(3):309–346 doi: 10.1145/567793.567795 CrossRefGoogle Scholar
  45. Myrtveit I, Stensrud E (1999) Benchmarking COTS projects using data envelopment analysis. In: Proceedings of 6th International Software-Metrics-Symposium, pp 269–278, Boca-Raton, FLGoogle Scholar
  46. Paulk MC, Curtis B, Chrissis MB, Weber CV (1993) Capability Maturity Model, Version 1.1. IEEE Softw 10(4):18–27 doi: 10.1109/52.219617 CrossRefGoogle Scholar
  47. Perens B (1999) The Open Source Definition. In: DiBona C et al (ed) Open sources: voices from the open source revolution. O’Reilly & Associates, Cambridge, MAGoogle Scholar
  48. Perry DE, Porter AA, Votta LG (2000) Empirical studies of software engineering: a roadmap. In: Finkelstein A (ed) The future of software engineering. ACM, New YorkGoogle Scholar
  49. R Development Core Team (2008) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, AustriaGoogle Scholar
  50. Raymond ES (1999) The Cathedral and the Bazaar. O’Reilly & Associates, Cambridge, MAGoogle Scholar
  51. Reel JS (1999) Critical success factors in software projects. IEEE Softw 16(3):18–23 doi: 10.1109/52.765782 CrossRefGoogle Scholar
  52. Robbins J (2005) Adopting Open Source Software Engineering (OSSE) practices by adopting OSSE tools. In: Feller J, Fitzgerald B, Hissam SA, Lakhani KR (eds) Perspectives on free and open source software. MIT, Cambridge, MAGoogle Scholar
  53. Rosen L (2004) Open source licensing: software freedom and intellectual property law. Prentice Hall, Englewood Cliffs, NJGoogle Scholar
  54. Rusovan S, Lawford M, Parnas DL (2005) Open source software development: future or fad? In: Feller J, Fitzgerald B, Hissam SA, Lakhani KR (eds) Perspectives on free and open source software. MIT, Cambridge, MAGoogle Scholar
  55. Scacchi W, Feller J, Fitzgerald B, Hissam S, Lakhani K (2006) Understanding free/open source software development processes. Softw Process Improv Pract 11(2):95–105 doi: 10.1002/spip.255 CrossRefGoogle Scholar
  56. Stallman RM (2002) Free software, free society: selected essays of Richard M. Stallman. GNU, Boston, MAGoogle Scholar
  57. Stewart KJ (2004) OSS project success: from internal dynamics to external impact. In Collaboration, Conflict and Control: The 4th workshop on Open Source Software Engineering (ICSE 2004), Edinburgh, ScotlandGoogle Scholar
  58. Stewart KJ, Gosain S (2006) The moderating role of development stage in affecting free/open source software project performance. Softw Process Improv Pract 11(2):177–191 doi: 10.1002/spip.258 CrossRefGoogle Scholar
  59. Stewart KJ, Ammeter AP, Maruping LM (2006) Impacts of licence choice and organisational sponsorship on user interest and development activity in open source software projects. Inf Syst Res 17(2):126–144 doi: 10.1287/isre.1060.0082 CrossRefGoogle Scholar
  60. Varian HR (2005) Intermediate microeconomics: a modern approach, 2nd edn. W. W. Norton, New York, NYGoogle Scholar
  61. Weiss D (2005) Measuring success of open source projects using web search engines. In: Proceedings of the 1st International Conference on Open Source Systems, Genoa, Italy, pp 93–99Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Institute for Information BusinessVienna University of Economics and BAViennaAustria

Personalised recommendations