Empirical Software Engineering

, Volume 21, Issue 2, pp 642–683 | Cite as

From Aristotle to Ringelmann: a large-scale analysis of team productivity and coordination in Open Source Software projects

  • Ingo Scholtes
  • Pavlin Mavrodiev
  • Frank Schweitzer
Article

Abstract

Complex software development projects rely on the contribution of teams of developers, who are required to collaborate and coordinate their efforts. The productivity of such development teams, i.e., how their size is related to the produced output, is an important consideration for project and schedule management as well as for cost estimation. The majority of studies in empirical software engineering suggest that - due to coordination overhead - teams of collaborating developers become less productive as they grow in size. This phenomenon is commonly paraphrased as Brooks’ law of software project management, which states that “adding manpower to a software project makes it later”. Outside software engineering, the non-additive scaling of productivity in teams is often referred to as the Ringelmann effect, which is studied extensively in social psychology and organizational theory. Conversely, a recent study suggested that in Open Source Software (OSS) projects, the productivity of developers increases as the team grows in size. Attributing it to collective synergetic effects, this surprising finding was linked to the Aristotelian quote that “the whole is more than the sum of its parts”. Using a data set of 58 OSS projects with more than 580,000 commits contributed by more than 30,000 developers, in this article we provide a large-scale analysis of the relation between size and productivity of software development teams. Our findings confirm the negative relation between team size and productivity previously suggested by empirical software engineering research, thus providing quantitative evidence for the presence of a strong Ringelmann effect. Using fine-grained data on the association between developers and source code files, we investigate possible explanations for the observed relations between team size and productivity. In particular, we take a network perspective on developer-code associations in software development teams and show that the magnitude of the decrease in productivity is likely to be related to the growth dynamics of co-editing networks which can be interpreted as a first-order approximation of coordination requirements.

Keywords

Software engineering Repository mining Productivity factors Social aspects of software engineering Open source software Coordination 

References

  1. Adams P, Capiluppi A, Boldyreff C (2009) Coordination and productivity issues in free software: the role of brooks’ law. In: IEEE International Conference on software maintenance. doi:10.1109/ICSM.2009.5306308. ICSM 2009, pp 319–328
  2. Alali A, Kagdi H, Maletic J (2008) What’s a typical commit? a characterization of open source software repositories. In: The 16th IEEE International Conference on program comprehension. doi:10.1109/ICPC.2008.24. ICPC 2008, pp 182–191
  3. Albrecht AJ (1979) Measuring application development productivity. In: Proceedings of the joint SHARE, GUIDE and IBM application development symposium, pp 83–92Google Scholar
  4. Arafat O, Riehle D (2009) The commit size distribution of open source software. In: 42nd Hawaii International Conference on system sciences, HICSS’09. doi:10.1109/HICSS.2009.421, pp 1–8
  5. Banker RD, Kauffman RJ (2004) 50th anniversary article: The evolution of research on information systems: a fiftieth-year survey of the literature in management science. Manag Sci 50(3):281–298. doi:10.1287/mnsc.1040.0206 CrossRefGoogle Scholar
  6. Banker R D, Kemerer C F (1989) Scale economies in new software development. IEEE Trans Softw Eng 15(10):1199–1205CrossRefGoogle Scholar
  7. Banker R D, Slaughter S A (1997) A field study of scale economies in software maintenance. Manag Sci 43(12):1709–1725CrossRefMATHGoogle Scholar
  8. Banker R D, Chang H, Kemerer C F (1994) Evidence on economies of scale in software development. Inf Softw Technol 36(5):275–282CrossRefGoogle Scholar
  9. Blackburn J D, Scudder G D, Van Wassenhove L N (1996) Improving speed and productivity of software development: a global survey of software developers. IEEE Trans Softw Eng 22(12):875–885CrossRefGoogle Scholar
  10. Blincoe K, Valetto G, Goggins S (2012) Proximity: a measure to quantify the need for developers’ coordination. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, CSCW ’12. doi:10.1145/2145204.2145406. ACM, New York, pp 1351–1360
  11. Blincoe K, Valetto G, Damian D (2013) Do all task dependencies require coordination? the role of task properties in identifying critical coordination needs in software projects. In: Proceedings of the 2013 9th joint meeting on foundations of software engineering, ESEC/FSE 2013. ACM, New York. doi:10.1145/2491411.2491440, pp 213–223
  12. Blincoe KC (2014) Timely and efficient facilitation of coordination of software developers’ activities. PhD thesis, Drexel University, Philadelphia, p aAI3613734Google Scholar
  13. Boehm B W (1984) Software engineering economics. IEEE Trans Software Eng 10(1):4–21. doi:10.1109/TSE.1984.5010193 CrossRefGoogle Scholar
  14. Boehm BW, Clark H, Brown R, Chulani MR, Steece B (2000) Software cost estimation with Cocomo II with Cdrom, 1st edn. Prentice Hall PTR, Upper Saddle RiverGoogle Scholar
  15. Brooks FP (1975) The mythical man-month. Addison-WesleyGoogle Scholar
  16. Cataldo M, Herbsleb J (2013) Coordination breakdowns and their impact on development productivity and software failures. IEEE Trans Softw Eng 39(3):343–360. doi:10.1109/TSE.2012.32 CrossRefGoogle Scholar
  17. Cataldo M, Wagstrom PA, Herbsleb JD, Carley KM (2006) Identification of coordination requirements: implications for the design of collaboration and awareness tools. In: Proceedings of the 2006 20th anniversary conference on computer supported cooperative work, CSCW ’06. doi:10.1145/1180875.1180929. ACM, New York, pp 353–362
  18. Cataldo M, Herbsleb JD, Carley KM (2008) Socio-technical congruence: a framework for assessing the impact of technical and work dependencies on software development productivity. In: Proceedings of the second ACM-IEEE international symposium on empirical software engineering and measurement, ESEM ’08. doi:10.1145/1414004.1414008. ACM, New York, pp 2–11
  19. Chidambaram L, Tung L L (2005) Is out of sight, out of mind? An empirical study of social loafing in technology-supported groups. Inf Syst Res 16(2):149–168. doi:10.1287/isre.1050.0051 CrossRefGoogle Scholar
  20. Comstock C, Jiang Z, Davies J (2011) Economies and diseconomies of scale in software development. J Softw Maint Evol Res Pract 23(8):533–548. doi:10.1002/smr.526 CrossRefGoogle Scholar
  21. Dabbish L, Stuart C, Tsay J, Herbsleb J (2012) Social coding in github: Transparency and collaboration in an open software repository. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, CSCW ’12. doi:10.1145/2145204.2145396 . ACM, New York, pp 1277–1286
  22. Earley PC (1989) Social loafing and collectivism: a comparison of the united states and the people’s republic of china. Adm Sci Q:565–581Google Scholar
  23. German DM (2006) A study of the contributors of postgresql. In: Proceedings of the 2006 international workshop on Mining software repositories. ACM, pp 163–164Google Scholar
  24. Gousios G, Kalliamvakou E, Spinellis D (2008) Measuring developer contribution from software repository data. In: Proceedings of the 2008 international working conference on mining software repositories, MSR ’08. doi:10.1145/1370750.1370781 . ACM, New York, pp 129–132
  25. Gousios G, Vasilescu B, Serebrenik A, Zaidman A (2014) Lean ghtorrent: Github data on demand. In: Proceedings of the 11th working conference on mining software repositories. doi:10.1145/2597073.2597126 MSR 2014. ACM, New York, pp 384–387
  26. Harison E, Koski H (2008) Does open innovation foster productivity? Evidence from open source software (oss) firms. Tech. rep., ETLA discussion paperGoogle Scholar
  27. Hindle A, German DM, Holt R (2008) What do large commits tell us?: a taxonomical study of large commits. In: Proceedings of the 2008 international working conference on mining software repositories, MSR ’08. doi:10.1145/1370750.1370773. ACM, New York, pp 99–108
  28. Hofmann P, Riehle D (2009) Estimating commit sizes efficiently. In: Boldyreff C, Crowston K, Lundell B, Wasserman A (eds) Open source ecosystems: diverse communities interacting, IFIP advances in information and communication technology, vol 299. doi:10.1007/978-3-642-02032-2_11. Springer, Berlin, pp 105–115
  29. Ingham AG, Levinger G, Graves J, Peckham V (1974) The ringelmann effect: Studies of group size and group performance. J Exp Soc Psychol 10(4):371–384. doi:10.1016/0022-1031(74)90033-X CrossRefGoogle Scholar
  30. Jackson J M, Harkins S G (1985) Equity in effort: an explanation of the social loafing effect. J Pers Soc Psychol 49(5):1199CrossRefGoogle Scholar
  31. Kalliamvakou E, Gousios G, Blincoe K, Singer L, German D M, Damian D (2014) The promises and perils of mining github. In: Proceedings of the 11th working conference on mining software repositories, MSR 2014. doi:10.1145/2597073.2597074 . ACM, New York, pp 92–101
  32. Karau S J, Williams K D (1993) Social loafing: a meta-analytic review and theoretical integration. J Personal Soc Psychol 65(4):681CrossRefGoogle Scholar
  33. Karau S J, Williams K D (1995) Social loafing: research findings, implications, and future directions. Curr Dir Psychol Sci 4(5):134–140CrossRefGoogle Scholar
  34. Koenker R (1981) A note on studentizing a test for heteroskedasticity. J Econ 17(1):107–112MathSciNetCrossRefMATHGoogle Scholar
  35. Kravitz D A, Martin B (1986) Ringelmann rediscovered: the original article. J Pers Soc Psychol 50(5):936–941CrossRefGoogle Scholar
  36. Latane B, Williams K, Harkins S (1979) Many hands make light the work: The causes and consequences of social loafing. J Pers Soc Psychol 37(6):822CrossRefGoogle Scholar
  37. Lerner J, Tirole J (2002) Some simple economics of open source. J Ind Econ 50(2):197–234. doi:10.1111/1467-6451.00174 CrossRefGoogle Scholar
  38. Levenshtein V I (1966) Binary codes capable of correcting deletions, insertions and reversals. In: Soviet physics doklady, vol 10, p 707Google Scholar
  39. Lin M, Lucas H, Shmueli G (2013) Research commentary - too big to fail: large samples and the p-value problem. Inf Syst Res 24(4):906–917CrossRefGoogle Scholar
  40. Maxwell K, Van Wassenhove L, Dutta S (1996) Software development productivity of european space, military, and industrial applications. IEEE Trans Softw Eng 22(10):706–718 . doi:10.1109/32.544349 CrossRefGoogle Scholar
  41. Mockus A, Fielding RT, Herbsleb J (2000) A case study of open source software development: The apache server. In: Proceedings of the 22Nd international conference on software engineering. ICSE ’00, ACM, New York, . doi:10.1145/337180.337209, pp 263–272
  42. Mockus A, Fielding R T, Herbsleb J D (2002) Two case studies of open source software development: apache and mozilla. ACM Trans Softw Eng Methodol 11 (3):309–346. doi:10.1145/567793.567795 CrossRefGoogle Scholar
  43. Paiva E, Barbosa D, Roberto Lima J, Albuquerque A (2010) Factors that influence the productivity of software developers in a developer view. In: Sobh T, Elleithy K (eds) Innovations in computing sciences and software engineering. doi:10.1007/978-90-481-9112-3_17. Springer, Netherlands, pp 99–104
  44. Premraj R, Shepperd M, Kitchenham B, Forselius P (2005) An empirical analysis of software productivity over time. In: 11th IEEE International Symposium software metrics, 2005. doi:10.1109/METRICS.2005.8
  45. Ringelmann M (1913) Recherches sur les moteurs animes: Travail de l’homme. Annales de l’Institut National Agronomique 12(1):1–40Google Scholar
  46. Robles G, Koch S, González-Barahona JM (2004) Remote analysis and measurement of libre software systems by means of the cvsanaly tool. In: 2nd ICSE workshop on remote analysis and measurement of software systems (RAMSS), pp 51–55Google Scholar
  47. Scholtes I, Mavrodiev P, Schweitzer F (2015) From aristotle to ringelmann (dataset). doi:http://dx.doi.org/abs/10.1109/METRICS.2005.8
  48. Shepperd J (1993) Productivity loss in performance groups - a motivation analysis. Psychol Bull 113(1):67–81. doi:10.1037/0033-2909.113.1.67 CrossRefGoogle Scholar
  49. Shiue Y C, Chiu C M, Chang C C (2010) Exploring and mitigating social loafing in online communities. Comput Hum Behav 26(4):768–777. doi:10.1016/j.chb.2010.01.014. http://www.sciencedirect.com/science/article/pii/S0747563210000166 emerging and Scripted Roles in Computer-supported Collaborative LearningCrossRefGoogle Scholar
  50. Sornette D, Maillart T, Ghezzi G (2014) How much is the whole really more than the sum of its parts? 1 + 1 = 2.5: Superlinear productivity in collective group actions. PLoS ONE 9(8):e103,023. doi:10.1371/journal.pone.0103023 CrossRefGoogle Scholar
  51. Steiner I D (1972) Group process and productivity. Social psychology monographs. AcademicGoogle Scholar
  52. Stigler G J (1958) The economies of scale. JL Econ 1:54Google Scholar
  53. von Krogh G, Spaeth S, Lakhani K R (2003) Community, joining, and specialization in open source software innovation: a case study. Res Policy 32 (7):1217–1241. doi:10.1016/S0048-7333(03)00050-7. open Source Software DevelopmentCrossRefGoogle Scholar
  54. Wagner J (1995) Studies of individualism-collectivism - effects on cooperation in groups. Acad Manag J 38(1):152–172CrossRefGoogle Scholar
  55. Williams K, Karau S (1991) Social loafing and social compensation - the effects of expectations of coworker performance. J Pers Soc Psychol 61(4):570–581. doi:10.1037/0022-3514.61.4.570 CrossRefGoogle Scholar
  56. Williams K, Harkins S, Latane B (1981) Identifiability as a deterrent to social loafing - 2 cheering experiments. J Personal Soc Psychol 40(2):303–311. doi:10.1037/0022-3514.40.2.303 CrossRefGoogle Scholar
  57. Wolf T, Schroter A, Damian D, Panjer L D, Nguyen T H (2009) Mining task-based social networks to explore collaboration in software teams. IEEE Software 26(1):58–66. doi:10.1109/MS.2009.16 CrossRefGoogle Scholar
  58. Yetton P, Bottger P (1983) The relationships among group size, member ability, social decision schemes, and performance. Organ Behav Hum Perform 32(2):145–159. doi:10.1016/0030-5073(83)90144-7 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Ingo Scholtes
    • 1
  • Pavlin Mavrodiev
    • 1
  • Frank Schweitzer
    • 1
  1. 1.ETH ZürichChair of Systems DesignZurichSwitzerland

Personalised recommendations