Advertisement

ALFAA: Active Learning Fingerprint based Anti-Aliasing for correcting developer identity errors in version control systems

  • Sadika AmreenEmail author
  • Audris Mockus
  • Russell Zaretzki
  • Christopher Bogart
  • Yuxia Zhang
Article
  • 48 Downloads

Abstract

An accurate determination of developer identities is important for software engineering research and practice. Without it, even simple questions such as “how many developers does a project have?” cannot be answered. The commonly used version control data from Git is full of identity errors and the existing approaches to correct these errors are difficult to validate on large scale and cannot be easily improved. We, therefore, aim to develop a scalable, highly accurate, easy to use and easy to improve approach to correct software developer identity errors. We first amalgamate developer identities from version control systems in open source software repositories and investigate the nature and prevalence of these errors, design corrective algorithms, and estimate the impact of the errors on networks inferred from this data. We investigate these questions using a collection of over 1B Git commits with over 23M recorded author identities. By inspecting the author strings that occur most frequently, we group identity errors into categories. We then augment the author strings with three behavioral fingerprints: time-zone frequencies, the set of files modified, and a vector embedding of the commit messages. We create a manually validated set of identities for a subset of OpenStack developers using an active learning approach and use it to fit supervised learning models to predict the identities for the remaining author strings in OpenStack. We then compare these predictions with a competing commercially available effort and a leading research method. Finally, we compare network measures for file-induced author networks based on corrected and raw data. We find commits done from different environments, misspellings, organizational ids, default values, and anonymous IDs to be the major sources of errors. We also find supervised learning methods to reduce errors by several times in comparison to existing research and commercial methods and the active learning approach to be an effective way to create validated datasets. Results also indicate that correction of developer identity has a large impact on the inference of the social network. We believe that our proposed Active Learning Fingerprint Based Anti-Aliasing (ALFAA) approach will expedite research progress in the software engineering domain for applications that involve developer identities.

Keywords

Software repository mining Identity disambiguation Random forest classification Record linkage Behavioral fingerprinting Social network analysis 

Notes

Acknowledgments

This research material is based on work supported by the National Science Foundation (NSF) grants IIS-1633437 and IIS-1901102. We would like to thank our collaborators at the Open Source Supply Chains and Avoidance of Risk (OSCAR) team at the University of Tennessee and from the Institute for Software Research (ISR) at the Carnegie Mellon University for their valuable feedback on this work.

References

  1. Badashian AS, Esteki A, Gholipour A, Hindle A, Stroulia E (2014) Involvement, contribution and influence in github and stack overflow. In: Proceedings of 24th annual international conference on computer science and software engineering, pp 19–33. IBM CorpGoogle Scholar
  2. Baltes S, Diehl S (2018) Usage and attribution of stack overflow code snippets in github projects. Empir Softw Eng 24:1–37Google Scholar
  3. Bird C, Gourley A, Devanbu P, Gertz M, Swaminathan A (2006) Mining email social networks. In: Proceedings of the 2006 international workshop on mining software repositories, MSR ’06.  https://doi.org/10.1145/1137983.1138016. ACM, New York, pp 137–143
  4. Bird C, Rigby PC, Barr ET, Hamilton DJ, German DM, Devanbu P (2009) The promises and perils of mining gitGoogle Scholar
  5. Bonacich P (1987) Power and centrality: A family of measures. Am J Soc 92 (5):1170–1182.  https://doi.org/10.1086/228631 CrossRefGoogle Scholar
  6. Burt RS (1992) Structural holes. Harvard University Press, HarvardGoogle Scholar
  7. Cataldo M, Wagstrom PA, Herbsleb JD, Carley KM (2006) Identification of coordination requirements: implications for the design of collaboration and awareness tools. In: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, pp 353–362. ACMGoogle Scholar
  8. Cataldo M, Herbsleb JD, Carley KM (2008) Socio-technical congruence: a framework for assessing the impact of technical and work dependencies on software development productivity. In: Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement, pp 2–11. ACMGoogle Scholar
  9. Christen P (2006) A comparison of personal name matching: Techniques and practical issues. In: 6th IEEE international conference on data mining - workshops (ICDMW’06), pp 290–294.  https://doi.org/10.1109/ICDMW.2006.2
  10. Cohen WW, Ravikumar P, Fienberg SE (2003) A comparison of string metrics for matching names and records. In: KDD Workshop on data cleaning and object consolidationGoogle Scholar
  11. Czerwonka J, Nagappan N, Schulte W, Murphy B (2013) Codemine: Building a software development data analytics platform at microsoft. IEEE Softw 30 (4):64–71CrossRefGoogle Scholar
  12. Edberg DT, Bowman BJ (1996) User-developed applications: An empirical study of application quality and developer productivity. J Manag Inf Syst 13(1):167–185CrossRefGoogle Scholar
  13. Fellegi IP, Sunter AB (1969) A theory for record linkage. J Am Stat Assoc 64(328):1183–1210.  https://doi.org/10.1080/01621459.1969.10501049. https://www.tandfonline.com/doi/abs/10.1080/01621459.1969.10501049 CrossRefzbMATHGoogle Scholar
  14. Freeman LC (1978) Centrality in social networks conceptual clarification. Soc Netw 1(3):215–239.  https://doi.org/10.1016/0378-8733(78)90021-7. http://www.sciencedirect.com/science/article/pii/0378873378900217 CrossRefGoogle Scholar
  15. German DM (2004) Mining cvs repositories, the softchange experience. In: 1st international workshop on mining software repositories, pp 17–21. CiteseerGoogle Scholar
  16. German D, Mockus A (2003) Automating the measurement of open source projects. In: Proceedings of the 3rd workshop on open source software engineering, pp 63–67. University College Cork Cork IrelandGoogle Scholar
  17. Gharehyazie M, Posnett D, Vasilescu B, Filkov V (2015) Developer initiation and social interactions in oss: A case study of the apache software foundation. Empirical Softw Eng 20(5):1318–1353.  https://doi.org/10.1007/s10664-014-9332-x CrossRefGoogle Scholar
  18. Goeminne M, Mens T (2013) A comparison of identity merge algorithms for software repositories. Sci Comput Progr 78(8):971–986.  https://doi.org/10.1016/j.scico.2011.11.004. http://www.sciencedirect.com/science/article/pii/S0167642311002048 CrossRefGoogle Scholar
  19. Hallgren KA (2012) Computing inter-rater reliability for observational data: an overview and tutorial. Tutorials in quantitative methods for psychology 8(1):23CrossRefGoogle Scholar
  20. Jergensen C, Sarma A, Wagstrom P (2011) The onion patch: migration in open source ecosystems. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering, pp 70–80. ACMGoogle Scholar
  21. Kouters E, Vasilescu B, Serebrenik A, van den Brand MGJ (2012) Who’s who in gnome: using lsa to merge software repository identities. In: 28th IEEE international conference on software maintenance (ICSM). IEEEGoogle Scholar
  22. Lawrence S, Giles CL, Bollacker K (1999) Digital libraries and autonomous citation indexing. Computer 32(6):67–71. 10.1109/2.769447CrossRefGoogle Scholar
  23. Le Q, Mikolov T (2014) Distributed representation of sentences and documents. In: Proceedings of the 31 st international conference on machine learning. https://cs.stanford.edu/quocle/paragraph_vector.pdf, vol 32. JMLR, Beijing
  24. Ma Y, Bogart C, Amreen S, Zaretzki R, Mockus A (2019) World of code: An infrastructure for mining the universe of open source vcs data. In: Proceedings of the 2019 international conference on mining software repositoriesGoogle Scholar
  25. Martinez-Romo J, Robles G, Gonzalez-Barahona JM, Ortuṅo-Perez M (2008) Using social network analysis techniques to study collaboration between a floss community and a company. In: Russo B, Damiani E, Hissam S, Lundell B, Succi G (eds) Open source development, communities and quality. Springer, Boston, pp 171–186Google Scholar
  26. Mockus A (2009a) Amassing and indexing a large sample of version control systems: towards the census of public source code history. In: 6th IEEE working conference on mining software repositories. IEEE. papers/amassing.pdfGoogle Scholar
  27. Mockus A (2009b) Succession: Measuring transfer of code and developer productivity. In: Proceedings of the 31st international conference on software engineering, pp 67–77. IEEE Computer SocietyGoogle Scholar
  28. Mockus A (2009c) Succession: Measuring transfer of code and developer productivity. In: 2009 international conference on software engineering. papers/succession.pdf. ACM Press, VancouverGoogle Scholar
  29. Mockus A (2014) Engineering big data solutions. In: ICSE’14 FOSE, pp 85–99. http://dl.acm.org/authorize?N14216
  30. Mockus A, Herbsleb JD (2002) Expertise browser: a quantitative approach to identifying expertise. In: Proceedings of the 24th international conference on software engineering, pp 503–512. ACMGoogle Scholar
  31. Nagappan N, Murphy B, Basili V (2008) The influence of organizational structure on software quality. In: 2008 ACM/IEEE 30th international conference on software engineering, pp 521–530. IEEEGoogle Scholar
  32. Nesbitt A, Nickolls B (2017) Libraries.io open source repository and dependency metadata.  https://doi.org/10.5281/zenodo.808273
  33. Ostrouchov G, Chen WC, Schmidt D, Patel P (2012) Programming with big data in r. URL http://r-pbd.org
  34. Petersen K, Wohlin C (2011) Measuring the flow in lean software development. Software: Practice and experience 41(9):975–996Google Scholar
  35. Pinzger M, Nagappan N, Murphy B (2008) Can developer-module networks predict failures?. In: Proceedings of the 16th ACM SIGSOFT international symposium on foundations of software engineering, SIGSOFT ’08/FSE-16.  https://doi.org/10.1145/1453101.1453105. ACM, New York, pp 2–12
  36. Řehu̇řek R, Sojka P (2010) Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 workshop on new challenges for NLP frameworks. ELRA, Valletta, pp 45–50Google Scholar
  37. Robles G, Gonzalez-Barahona JM (2005) Developer identification methods for integrated data from various sources. In: Proceedings of the 2005 international workshop on mining software repositories, MSR ’05.  https://doi.org/10.1145/1082983.1083162. ACM, New York, pp 1–5CrossRefGoogle Scholar
  38. Sarawagi S, Bhamidipaty A (2002) Interactive deduplication using active learning. In: Proceedings of the 8th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’02.  https://doi.org/10.1145/775047.775087. ACM, New York, pp 269–278
  39. Sariyar M, Borg A (2010) The recordlinkage package: Detecting errors in data. The R J 2(1):61–67 . https://journal.r-project.org/archive/2010-2/RJournal_2010-_Sariyar+Borg.pdf CrossRefGoogle Scholar
  40. Smalheiser NR, Torvik VI (2011) Author name disambiguation. Annual Review of Information Science and Technology 43(1):1–43.  https://doi.org/10.1002/aris.2009.1440430113. https://onlinelibrary.wiley.com/doi/abs/10.1002/aris.2009.1440430113 CrossRefGoogle Scholar
  41. Spencer D, Warfel T (2004) Card sorting: A definitive guide. Boxes and Arrows, pp 2Google Scholar
  42. Thung F, Bissyande TF, Lo D, Jiang L (2013) Network structure of social coding in github. In: 2013 17th European conference on software maintenance and reengineering, pp 323–326. IEEEGoogle Scholar
  43. Vasilescu B, Serebrenik A, Filkov V (2015) A data set for social diversity studies of github teams. In: Proceedings of the 12th working conference on mining software repositories, pp 514–517. ACM. https://dl.acm.org/citation.cfm?id=2820601
  44. Ventura SL, Nugent R, Fuchs ER (2015) Seeing the non-starts: (some) sources of bias in past disambiguation approaches and a new public tool leveraging labeled records. ElsevierGoogle Scholar
  45. Wang DJ, Shi X, McFarland DA, Leskovec J (2012) Measurement error in network data: A re-classification. Soc Netw 34(4):396–409CrossRefGoogle Scholar
  46. Watts DJ, Strogatz SH (1998) Collective dynamics of ‘small-world’ networks. Nature 393:440–442.  https://doi.org/10.1038/30918 CrossRefGoogle Scholar
  47. Wiese IS, da Silva JT, Steinmacher I, Treude C, Gerosa MA (2016) Who is who in the mailing list? comparing six disambiguation heuristics to identify multiple addresses of a participant. In: 2016 IEEE international conference on software maintenance and evolution (ICSME), pp 345–355, DOI 10.1109/ICSME.2016.13, (to appear in print)Google Scholar
  48. Winkler WE (2006) Overview of record linkage and current research directions. Tech. rep., Bureau of the CensusGoogle Scholar
  49. Wolf T, Schröter A, Damian D, Panjer LD, Nguyen THD (2009) Mining task-based social networks to explore collaboration in software teams. IEEE Softw 26 (1):58–66. 10.1109/MS.2009.16CrossRefGoogle Scholar
  50. Xiong Y, Meng Z, Shen B, Yin W (2017) Mining developer behavior across github and stackoverflow. In: The 29th international conference on software engineering and knowledge engineering, pp 578–583.  https://doi.org/10.18293/SEKE2017-062
  51. Zhou M, Mockus A, Ma X, Zhang L, Mei H (2016) Inflow and retention in oss communities with commercial involvement: A case study of three hybrid projects. ACM Transactions on Software Engineering and Methodology (TOSEM) 25(2):13CrossRefGoogle Scholar
  52. Zhu J, Wei J (2019) An empirical study of multiple names and email addresses in oss version control repositories. In: Proceedings of 16th international conference on mining software repositories (MSR). IEEE/ACMGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2020

Authors and Affiliations

  • Sadika Amreen
    • 1
    Email author
  • Audris Mockus
    • 1
  • Russell Zaretzki
    • 1
  • Christopher Bogart
    • 2
  • Yuxia Zhang
    • 3
  1. 1.University of TennesseeKnoxvilleUSA
  2. 2.Carnegie Mellon UniversityPittsburghUSA
  3. 3.Peking UniversityBeijingChina

Personalised recommendations