Skip to main content
SpringerLink
Log in

Studying backers and hunters in bounty issue addressing process of open source projects

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Issue addressing is a vital task in the evolution of software projects. However, in practice, not all issues can be addressed on time. To facilitate the issue addressing process, monetary incentives (e.g., bounties) are used to attract developers to address issues. There are two types of core roles who are involved in this process: bounty backers, who propose bounties for an issue report via bounty platforms (e.g., Bountysource), and bounty hunters, who address the bounty issues and win the bounties. We wish to study the process of bounty issue addressing from the angle of two important roles (i.e., backers and hunters) and their related behaviors. With a better understanding of how they address bounty issues, stakeholders (e.g., operators and developers) of open source projects may have a reasonable estimation of what they can expect from backers and hunters. In this study, we investigate 2,955 bounty backers and 882 bounty hunters, and their associated 3,579 GitHub issue reports with 5,589 bounties that were proposed on Bountysource. We find that: 1) Overall, the value of a bounty is small (median bounty value of $20). Both individual and corporate backers prefer to support implementing new features rather than fixing bugs. Corporate backers tend to propose larger bounties and propose bounties more frequently than individual backers. 2) 85.0% of the bounty hunters addressed less than 3 bounty issues. The income of 56.7% of the bounty hunters is no more than $100 and only 2.7% of the hunters have earned more than $2,000. In addition, most of the regular hunters and big hunters are developers that made at least one commit before addressing a bounty issue. 3) The value of a bounty issue is not a statistically significant factor that attracts developers that have never made any commit before to address an issue. Based on our findings, we provide several suggestions for stakeholders of open source projects and hunters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://www.bountysource.com

  2. https://github.com/austinpray/asset-builder/issues?q=label%3Abounty

  3. https://www.google.com/about/appsecurity/chrome-rewards/

  4. https://bountysource.github.io/

  5. https://developer.github.com/v3/

  6. https://github.com/SAILResearch/wip-18-jiayuan-bountysource-SupportMaterials

  7. https://cran.r-project.org/web/packages/rms/rms.pdf

  8. https://github.com/webmproject/libvpx/

  9. https://medium.com/@luc.trudeau/video-compression-bounty-hunters-c8edf43d440

  10. https://github.com/OpenRA

  11. https://www.bountysource.com/issues/25924774-enable-implement-ppc64-le-linux-lj_gc64-interpreter-and-jit

  12. https://www.bountysource.com/issues/5413688-repeat-every-x-days-monthlies

  13. https://github.com/HabitRPG/habitica/issues/4173

References

  • Akobeng AK (2007) Understanding diagnostic tests 3: receiver operating characteristic curves. Acta Paediat 96(5):644–647

    Article  Google Scholar 

  • Androutsellis-Theotokis S, Spinellis D, Kechagia M, Gousios G, et al. (2011) Open source software: A survey from 10,000 feet. Found Trends Technol Inf Oper Manag 4(3–4):187–347

    Google Scholar 

  • Apple Inc (2020) Apple Security Bounty). https://developer.apple.com/security-bounty/,. (last visited: Dec 12, 2020)

  • Atiq A, Tripathi A (2016) Impact of financial benefits on open source software sustainability. In: International conference on information systems (ICIS), pp 1–10

  • Avelino G, Passos L, Hora A, Valente MT (2016) A novel approach for estimating truck factors. In: IEEE 24th international conference on program comprehension (ICPC), pp 1–10

  • Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13(1):281–305

    MathSciNet  MATH  Google Scholar 

  • Bissyandé TF, Thung F, Lo D, Jiang L, Réveillère L (2013) Popularity, interoperability, and impact of programming languages in 100,000 open source projects. In: IEEE 37th annual computer software and applications conference. IEEE, pp 303–312

  • Canfora G, Di Penta M, Oliveto R, Panichella S (2012) Who is going to mentor newcomers in open source projects?. In: Proceedings of the ACM SIGSOFT 20th international symposium on the foundations of software engineering (FSE), pp 1–11

  • Coelho J, Valente MT, Silva LL, Hora A (2018) Why we engage in floss: Answers from core developers. In: Proceedings of the 11th international workshop on cooperative and human aspects of software engineering, pp 114–121

  • Comino S, Manenti FM, Parisi ML (2007) From planning to mature: on the success of open source projects. Res Policy 36(10):1575–1586

    Article  Google Scholar 

  • Dagenais B, Ossher H, Bellamy RKE, Robillard MP, de Vries JP (2010) Moving into a new software project landscape. In: Proceedings of the 32nd ACM/IEEE international conference on software engineering - Volume 1, ICSE ’10, pp 275–284

  • Dinnie M (2019) How to prioritize feature requests for software development. https://zenkit.com/en/blog/how-to-prioritize-feature-requests-for-software-development,. (last visited: November 8, 2019)

  • Duebendorfer T, Frei S (2009) Why silent updates boost security. TIK, ETH Zurich, Tech Rep 302

  • Eghbal N (2016) Roads and bridges: The unseen labor behind our digital infrastructure. Ford Foundation

    Google Scholar 

  • Eghbal N (2019) A handy guide to financial support for open source

  • Finifter M, Akhawe D, Wagner D (2013) An empirical study of vulnerability rewards programs. In: USENIX Security Symp., pp 273–288

  • Frei S, Duebendorfer T, Plattner B (2008) Firefox (in) security update dynamics exposed. ACM SIGCOMM Comput Commun Rev 39(1):16–22

    Article  Google Scholar 

  • Frey BS, Goette L (1999) Does pay motivate volunteers? Working paper/Inst Empir Res Econ 7

  • HackerOne (2018) 118 fascinating facts from hackerone’s hacker-powered security report 2018. https://www.hackerone.com/blog/118-Fascinating-Facts-HackerOnes-Hacker-Powered-Security-Report-2018. (last visited: August 27, 2018)

  • Harhoff D, Henkel J, Von Hippel E (2003) Profiting from voluntary information spillovers: how users benefit by freely revealing their innovations. Res Pol 32(10):1753–1769

    Article  Google Scholar 

  • Hata H, Guo M, Babar MA (2017) Understanding the heterogeneity of contributors in bug bounty programs. In: Proc. of the ACM/IEEE int’l symp. on empirical software engineering and measurement, pp 223–228

  • Izquierdo JLC, Cabot J (2018) The role of foundations in open source projects. In: Proceedings of the 40th international conference on software engineering: software engineering in society, pp 3–12

  • Kanda T, Guo M, Hata H, Matsumoto K (2017) Towards understanding an open-source bounty: Analysis of Bountysource. In: Int’l conf. on software analysis, evolution and reengineering. IEEE, pp 577–578

  • Kochhar PS, Thung F, Lo D (2014) Automatic fine-grained issue report reclassification. In: 2014 19th international conference on engineering of complex computer systems. IEEE, pp 126–135

  • Krishnamurthy S, Tripathi AK (2006) Bounty programs in free/libre/open source software. In: The economics of open source software development. Elsevier, pp 165–183

  • Krishnamurthy S, Ou S, Tripathi AK (2014) Acceptance of monetary rewards in open source software development. Res Policy 43(4):632–644

    Article  Google Scholar 

  • Kuhn M, et al. (2008) Building predictive models in r using the caret package. J Stat Softw 28(5):1–26

    Article  Google Scholar 

  • Lakhani KR, Wolf RG (2003) Why hackers do what they do: Understanding motivation and effort in free/open source software projects

  • Lee A, Carver JC, Bosu A (2017) Understanding the impressions, motivations, and barriers of one time code contributors to floss projects: a survey. In: IEEE/ACM 39th international conference on software engineering (ICSE), pp 187–197

  • Maillart T, Zhao M, Grossklags J, Chuang J (2017) Given enough eyeballs, all bugs are shallow? Revisiting Eric Raymond with bug bounty programs. J Cybersec 3(2):81–90

    Article  Google Scholar 

  • Mandrekar JN (2010) Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol 5(9):1315–1316

    Article  Google Scholar 

  • Matt A (2020) Bug bounties won’t make you rich (but you should participate anyway). https://www.techrepublic.com/article/bug-bounties-wont-make-you-rich-but-you-should-participate-anyway/,. (last visited: January 21, 2020)

  • Mirko Z (2020) Full-time bug hunting:, Pros and cons of an emerging career. https://www.helpnetsecurity.com/2020/04/07/bug-hunting-career/. (April 7, 2020)

  • Mockus A, Fielding RT, Herbsleb JD (2002) Two case studies of open source software development: Apache and mozilla. ACM Trans Softw Eng Methodol (TOSEM) 11(3):309–346

    Article  Google Scholar 

  • Moore DS, Kirkland S (2007) The basic practice of statistics, vol 2. WH Freeman New York

    Google Scholar 

  • Nakasai K, Hata H, Matsumoto K (2018) Are donation badges appealing?: a case study of developer responses to eclipse bug reports. IEEE Softw 36 (3):22–27

    Article  Google Scholar 

  • Rajbahadur GK, Wang S, Kamei Y, Hassan AE (2019) Impact of discretization noise of the dependent variable on machine learning classifiers in software engineering. IEEE Trans Softw Eng

  • Robert L (2019) Bug bounties continue to rise, but market has its own 1% problem). https://www.darkreading.com/vulnerabilities---threats/vulnerability-management/bug-bounties-continue-to-rise-but-market-has-its-own-1--problem/d/d-id/1335689

  • Roberts JA, Hann I-H, Slaughter SA (2006) Understanding the motivations, participation, and performance of open source software developers: a longitudinal study of the apache projects. Manag Sci 52(7):984–999

    Article  Google Scholar 

  • Robles G, Gonzalez-Barahona JM, Herraiz I (2009) Evolution of the core team of developers in libre software projects. In: 2009 6th IEEE international working conference on mining software repositories. IEEE, pp 167–170

  • Romano J, Kromrey JD, Coraggio J, Skowronek J (2006) Appropriate statistics for ordinal level data: Should we really be using t-test and cohen’s d for evaluating group differences on the nsse and other surveys. In: Annual meeting of the Florida association of institutional research, pp 1–33

  • Shah SK (2006) Motivation, governance, and the viability of hybrid forms in open source software development. Manag Sci 52(7):1000–1014

    Article  Google Scholar 

  • Steinmacher I, Silva MAG, Gerosa MA (2014) Barriers faced by newcomers to open source projects: a systematic review. In: IFIP international conference on open source systems. Springer, pp 153–163

  • Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2016) An empirical comparison of model validation techniques for defect prediction models. IEEE Trans Softw Eng 43(1):1–18

    Article  Google Scholar 

  • Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2018) The impact of automated parameter optimization on defect prediction models. IEEE Trans Softw Eng 45(7):683–711

    Article  Google Scholar 

  • Tom R (2020) Firefox’s bug bounty in 2019 and into the future. https://blog.mozilla.org/security/2020/04/23/bug-bounty-2019-and-future/,. (last visited: April 23, 2020)

  • Vasilescu B, Posnett D, Ray B, van den Brand MG, Serebrenik A, Devanbu P, Filkov V (2015) Gender and tenure diversity in github teams. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, pp 3789–3798

  • Von Hippel E (2007) Horizontal innovation networks—by and for users. Indust Corp Change 16(2):293–315

    Article  Google Scholar 

  • Von Krogh G, Haefliger S, Spaeth S, Wallin MW (2012) Carrots and rainbows: Motivation and social practice in open source software development. MIS Quart:649–676

  • Wang S, Chen T-H, Hassan AE (2018) Understanding the factors for fast answers in technical Q&A websites. Empir Softw Eng 23(3):1552–1593

    Article  Google Scholar 

  • Weiss M (2011) Control and diversity in company-led open source projects. Open Sourc Bus Res, (April 2011)

  • Ye Y, Kishida K (2003) Toward an understanding of the motivation open source software developers. In: Proceedings of the 25th international conference on software engineering (ICSE), pp 419–429

  • Zhao M, Grossklags J, Chen K (2014) An exploratory study of white hat behaviors in a web vulnerability disclosure program. In: Proc. of the workshop on security information workers. ACM, pp 51–58

  • Zhao M, Laszka A, Grossklags J (2017) Devising effective policies for bug-bounty platforms and security vulnerability discovery. J Inf Pol 7:372–418

    Google Scholar 

  • Zhou J, Wang S, Bezemer C-P, Hassan AE (2020a) Bounties on technical Q&A sites: a case study of stack overflow bounties. Empir Softw Eng 25 (1):139–177

    Article  Google Scholar 

  • Zhou J, Wang S, Bezemer C-P, Zou Y, Hassan AE (2020b) Studying the association between bountysource bounties and the issue-addressing likelihood of github issue reports. IEEE Trans Softw Eng

  • Zhou M, Mockus A, Ma X, Zhang L, Mei H (2016) Inflow and retention in oss communities with commercial involvement: a case study of three hybrid projects. ACM Trans Softw Eng Methodol (TOSEM) 25(2): 1–29

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Software Excellence (CSE), Huawei, Canada

    Jiayuan Zhou & Haoxiang Zhang

  2. Department of Computer Science, University of Manitoba, Winnipeg, MB, R3T 2N2, Canada

    Shaowei Wang

  3. Software PErformance, Analysis, and Reliability (SPEAR) Lab, Concordia University, Montreal, QC, H3G 1M8, Canada

    Tse-Hsun (Peter) Chen

  4. Software Analysis and Intelligence Lab (SAIL), Queen’s University, Kingston, ON, Canada

    Ahmed E. Hassan

Authors
  1. Jiayuan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaowei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haoxiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tse-Hsun (Peter) Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ahmed E. Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaowei Wang.

Additional information

Communicated by: Kelly Blincoe

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is not related to Jiayuan Zhou’s and Haoxiang Zhang’s roles at Huawei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, J., Wang, S., Zhang, H. et al. Studying backers and hunters in bounty issue addressing process of open source projects. Empir Software Eng 26, 81 (2021). https://doi.org/10.1007/s10664-021-09979-z

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-021-09979-z

Keywords

Search

Navigation