Exploring the Effects of Coordination and Communication Tools on the Efficiency of Open Source Projects using Data Envelopment Analysis

  • Stefan Koch
Part of the IFIP — The International Federation for Information Processing book series (IFIPAICT, volume 234)


In this paper, we propose to explore possible benefits of communication and coordination tools in open source projects using data envelopment analysis (DEA), a general method for efficiency comparisons. DEA offers several advantages: It is a non-parametric optimization method without any need for the user to define any relations between different factors or a production function, can account for economies or diseconwhile omies of scale, and is able to deal with multi-input, multi-output systems in which the factors have different scales. Using a data set of 30 open source project retrieved from SourceForge.net, we demonstrate the application of DEA, showing that the efficiency of the projects is in general relatively high. Regarding the effects of tool employment on the efficiency of projects, the results were surprising: Most of the possible tools, and overall usage, showed a negative relationship to efficiency.


Open Source Software Development Efficiency Data Envelopment Analysis Software Repositories 


  1. [1]
    Albrecht, A.J., & Gaffney, J.E. (1983). Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation. IEEE Transactions on Software Engineering, 9(6), 639–648.CrossRefGoogle Scholar
  2. [2]
    Amor, J.J., Robles, G., & Gonzalez-Barahona, J.M. (2006). Effort Estimation by Characterizing Developer Activity. In Proceedings 8 th International Workshop on Economics-Driven Software Engineering Research (ICSE 2006), Shanghai, China.Google Scholar
  3. [3]
    Banker, R.D., Charnes, A., & Cooper, W. (1984). Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis. Management Science, 30, 1078–1092.MATHCrossRefGoogle Scholar
  4. [4]
    Banker, R.D., & Kemerer, C. (1989). Scale Economies in New Software Development. IEEE Transactions on Software Engineering, 15(10), 416–429.CrossRefGoogle Scholar
  5. [5]
    Banker, R.D., & Slaughter, S.A. (1997). A Field Study of Scale Economies in Software Maintenance. Management Science, 43(12), 1709–1725.MATHGoogle Scholar
  6. [6]
    Charnes, A., Cooper, W., & Rhodes, E. (1978a). A Data Envelopment Analysis Approach to Evaluation of the Program Follow Through Experiments in U.S. Public School Education (Management Science Research Report No. 432). Carnegie-Mellon University, Pittsburgh, PA.Google Scholar
  7. [7]
    Charnes, A., Cooper, W., & Rhodes, E. (1978b). Measuring the Efficiency of Decision Making Units. European Journal of Operational Research, 2, 429–444.MATHCrossRefMathSciNetGoogle Scholar
  8. [8]
    Cooper, W., Seiford, L., & Tone, K. (2000). Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEASolver Software, Boston, MA: Kluwer Academic Publishers.Google Scholar
  9. [9]
    Crowston, K., Annabi, H., & Howison, J. (2003). Defining Open Source Software Project Success. In Proceedings of ICIS 2003, Seattle, WA.Google Scholar
  10. [10]
    Crowston, K., Annabi, H., Howison, J., & Masango, C. (2004). Towards A Portfolio of FLOSS Project Success Measures. In Collaboration, Conflict and Control: The 4th Workshop on Open Source Software Engineering (ICSE 2004), Edinburgh, Scotland.Google Scholar
  11. [11]
    Demetriou, N., Koch, S. & Neumann, G. (2006). The Development of the OpenACS Community. In Lytras, M. & Naeve, A. (eds.) Open Source for Knowledge and Learning Management: Strategies Beyond Tools, Hershey, PA: Idea Group.Google Scholar
  12. [12]
    Farell, M.J. (1957). The Measurement of Productive Efficiency. Journal of the Royal Statistical Society, Series A 120(3), 250–290.Google Scholar
  13. [13]
    Howison, J. & Crowston, K. (2004). The perils and pitfalls of mining SourceForge. In Proceedings of the International Workshop on Mining Software Repositories, pp. 7–11, Edingburgh, Scotland, UK.Google Scholar
  14. [14]
    Kitchenham, B. (2002). The question of scale economies in software-why cannot researchers agree? Information & Software Technology, 44(1), 13–24.CrossRefGoogle Scholar
  15. [15]
    Kitchenham, B., & Mendes, E. (2004). Software Productivity Measurement Using Multiple Size Measures. IEEE Transactions on Software Engineering, 30(12), 1023–1035.CrossRefGoogle Scholar
  16. [16]
    Koch, S. (2004). Profiling an open source project ecology and its programmers. Electronic Markets, 14(2), 77–88.CrossRefGoogle Scholar
  17. [17]
    Koch, S. (2005). Effort Modeling and Programmer Participation in Open Source Software Projects (Arbeitspapiere zum Tätigkeitsfeld Informationsverarbeitung, Informationswirtschaft und Prozessmanagement, Nr. 03/2005). Wirtschaftsuniversität Wien, Vienna, Austria.Google Scholar
  18. [18]
    Koch, S. (to appear). Measuring the Efficiency of Free and Open Source Software Projects Using Data Envelopment Analysis. In Sowe, S.K., Stamelos, I. and Samoladas, I. (eds.): Emerging Free and Open Source Software Practices.Google Scholar
  19. [19]
    Michlmayr, M. (2005). Software Process Maturity and the Success of Free Software Projects. In Zielinski, K. and Szmuc, T. (eds.): Software Engineering: Evolution and Emerging Technologies, pp. 3–14, IOS Press, Amsterdam, The Netherlands.Google Scholar
  20. [20]
    Myrtveit, I., & Stensrud, E. (1999). Benchmarking COTS Projects Using Data Envelopment Analysis. In Proceedings of 6th International Software-Metrics-Symposium, pp. 269–278, Boca-Raton.Google Scholar
  21. [21]
    Stewart, K.J. (2004). OSS Project Success: From Internal Dynamics to External Impact. In Collaboration, Conflict and Control: The 4th Workshop on Open Source Software Engineering (ICSE 2004), Edinburgh, Scotland.Google Scholar
  22. [22]
    Stewart, K.J., & Ammeter, T.A. (2002). An Exploratory Study of Factors Affecting the Popularity and Vitality of Open Source Projects. In Proceedings of ICIS 2002, Barcelona, Spain.Google Scholar
  23. [23]
    Weiss, D. (2005). Measuring Success of Open Source Projects Using Web Search Engines. In Proceedings of the 1st International Conference on Open Source Systems, pp. 93–99, Genoa, Italy.Google Scholar

Copyright information

© International Federation for Information Processing 2007

Authors and Affiliations

  • Stefan Koch
    • 1
  1. 1.Institute for Information BusinessVienna University of Economics and Business AdministrationViennaAustria

Personalised recommendations