Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Developer recommendation on bug commenting: a ranking approach for the developer crowd

Abstract

A bug tracking system provides a collaborative platform for the developer crowd. After a bug report is submitted, developers can make comments to supplement the details of the bug report. Due to the large number of developers and bug reports, it is hard to determine which developer (also called commenter) is able to comment on a particular bug report. We refer to the problem of recommending developers for commenting on bug reports as commenter recommendation. In this paper, we perform an empirical analysis on commenter recommendation based on five-year bug reports of four open source projects. First, we preliminarily analyze bug comments and commenters in three categories, the relationship between commenters and fixers, the data scale of comments, and the collaboration on bug commenting. Second, we design a recommendation approach via ranking developers in the crowd to reduce the manual effort of identifying commenters. In this approach, we formulize the commenter recommendation problem as a multi-label recommendation task and leverage both developer collaboration and bug content to find out appropriate commenters. Experimental results show that our approach can effectively recommend commenters; 41% to 75% of the recall value is achieved for top-10 recommendation. Our empirical analysis on bug commenting can help developers understand and improve the process of fixing bugs.

摘要

软件bug跟踪系统为开发者群体提供了协作平台。在一个bug报告提交后, 开发者可以通过评论补充bug报告的细节。鉴于开发者和bug报告数量庞大, 很难决定哪个开发者(也称评论者)具有为某个bug报告评论的能力。本文称推荐开发者评论bug报告的问题为评论者推荐。我们基于四个开源项目历时五年的bug报告进行评论者推荐的实证分析。首先, 我们初步分析了bug评论和评论者的三个方面, 即评论者和修复者的关系、评论的数据规模和bug的协同评论。其次, 为了减少手工识别评论者的成本, 我们设计了基于群体中开发者排序的评论者推荐方法。该方法中, 我们将评论者推荐问题描述为多标签推荐任务, 并基于开发者协作和bug内容两个方面找到合适的评论者。实验结果表明该方法可以有效的推荐评论者。推荐前10个候选时可以获得41%到75%的召回率。对bug评论的实证分析可以帮助开发者理解和提高bug的修复过程。

This is a preview of subscription content, log in to check access.

References

  1. 1

    Wu W, Tsai W-T, Li W. An evaluation framework for software crowdsourcing. Front Comput Sci, 2013, 7: 694–709

  2. 2

    Zimmermann T, Premraj R, Bettenburg N, et al. What makes a good bug report? IEEE Trans Softw Eng, 2010, 36: 618–643

  3. 3

    Xuan J F, Martinez M, DeMarco F, et al. Nopol: automatic repair of conditional statement bugs in java programs. IEEE Trans Softw Eng, 2017, 43: 34–55

  4. 4

    Anvik J, Hiew L, Murphy G C. Who should fix this bug? In: Proceedings of the 28th International Conference on Software Engineering. New York: ACM, 2006. 361–370

  5. 5

    Meneely A, Williams L. Socio-technical developer networks: should we trust our measurements? In: Proceedings of the 33rd International Conference on Software Engineering. New York: ACM, 2011. 281–290

  6. 6

    Zhou J, Zhang H, Lo D. Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports. In: Proceedings of the 34th International Conference on Software Engineering. Washington: IEEE, 2012. 14–24

  7. 7

    Lotufo R, Malik Z, Czarnecki K. Modelling the hurried bug report reading process to summarize bug reports. In: Proceedings of the 28th IEEE International Conference on Software Maintenance. Washington: IEEE, 2012. 430–439

  8. 8

    Gao J, Bai X, Tsai W-T, et al. Mobile application testing: a tutorial. IEEE Comput, 2014, 47: 46–55

  9. 9

    Anvik J, Murphy G C. Determining implementation expertise from bug reports. In: Proceedings of the 4th International Workshop on Mining Software Repositories. Washington: IEEE, 2007. 2

  10. 10

    Tamrawi A, Nguyen T T, Al-Kofahi J M, et al. Fuzzy set and cache-based approach for bug triaging. In: Proceedings of the 19th ACM SIGSOFT Symposium on Foundations of Software Engineering and the 13th European Software Engineering Conference. New York: ACM, 2011. 365–375

  11. 11

    Pinzger M, Nagappan N, Murphy B. Can developer-module networks predict failures? In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York: ACM, 2008. 2–12

  12. 12

    Wang X, Zhang L, Xie T, et al. An approach to detecting duplicate bug reports using natural language and execution information. In: Proceedings of the 30th International Conference on Software Engineering. New York: ACM, 2008. 461–470

  13. 13

    Sun C, Lo D, Khoo S C, et al. Towards more accurate retrieval of duplicate bug reports. In: Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering. Washington: IEEE, 2011. 253–262

  14. 14

    Shihab E, Ihara A, Kamei Y, et al. Predicting re-opened bugs: a case study on the eclipse project. In: Proceedings of the 17th Working Conference on Reverse Engineering. Washington: IEEE, 2010. 249–258

  15. 15

    Hu H, Zhang H, Xuan J, et al. Effective bug triage based on historical bug-fix information. In: Proceedings of IEEE 25th International Symposium on Software Reliability Engineering. Washington: IEEE, 2014. 122–132

  16. 16

    Xuan J, Jiang H, Hu Y, et al. Towards effective bug triage with software data reduction techniques. IEEE Trans Knowl Data Eng, 2015, 27: 264–280

  17. 17

    Hong Q, Kim S, Cheung S C, et al. Understanding a developer social network and its evolution. In: Proceedings of the 27th IEEE International Conference on Software Maintenance. Washington: IEEE, 2011. 323–332

  18. 18

    Han J, Kamber M. Data Mining: Concepts and Techniques. San Francisco: Morgan Kaufmann, 2001. 71–72

  19. 19

    Jiang H, Ren Z, Xuan J, et al. Extracting elite pairwise constraints for clustering. Neurocomputing, 2013, 99: 124–133

  20. 20

    Herlocker J L, Konstan J A, Terveen L G, et al. Evaluating collaborative filtering recommender systems. ACM Trans Inf Syst, 2004, 22: 5–53

  21. 21

    Sarwar B, Karypis G, Konstan J, et al. Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web. New York: ACM, 2001. 285–295

  22. 22

    Adomavicius G, Tuzhilin A. Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng, 2005, 17: 734–749

  23. 23

    Matter D, Kuhn A, Nierstrasz O. Assigning bug reports using a vocabulary-based expertise model of developers. In: Proceedings of the 6th IEEE International Working Conference on Mining Software Repositories. Washington: IEEE, 2009. 131–140

  24. 24

    Ochiai A. Zoogeographic studies on the soleoid fishes found in japan and its neighbouring regions. Bull Jpn Soc Sci Fish, 1957, 22: 526–530

  25. 25

    Dice L R. Measures of the amount of ecologic association between species. Ecology, 1945, 26: 297–302

  26. 26

    Bettenburg N, Premraj R, Zimmermann T, et al. Duplicate bug reports considered harmful... really? In: Proceedings of IEEE International Conference on Software Maintenance. Washington: IEEE, 2008. 337–345

  27. 27

    Wu W, Zhang W, Yang Y, et al. Drex: developer recommendation with k-nearest-neighbor search and expertise ranking. In: Proceedings of the 18th Asia Pacific Software Engineering Conference. Washington: IEEE, 2011. 389–396

  28. 28

    Xie X, Zhang W, Yang Y, et al. Dretom: developer recommendation based on topic models for bug resolution. In: Proceedings of the 8th International Conference on Predictive Models in Software Engineering. New York: ACM, 2012. 19–28

  29. 29

    Xuan J F, Cornu B, Martinez M, et al. B-refactoring: automatic test code refactoring to improve dynamic analysis. Inf Softw Tech, 2016, 76: 65–80

  30. 30

    Rastkar S, Murphy G C, Murray G. Summarizing software artifacts: a case study of bug reports. In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering. New York: ACM, 2010. 505–514

  31. 31

    Xia X, Lo D, Wang X, et al. Dual analysis for recommending developers to resolve bugs. J Softw Evol Process, 2015, 27: 195–220

  32. 32

    Bird C, Pattison D, D’Souza R, et al. Latent social structure in open source projects. In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York: ACM, 2008. 24–35

  33. 33

    Xuan J, Jiang H, Ren Z, et al. Solving the large scale next release problem with a backbone-based multilevel algorithm. IEEE Trans Softw Eng, 2012, 38: 1195–1212

  34. 34

    Jiang H, Xuan J, Ren Z. Approximate backbone based multilevel algorithm for next release problem. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation. New York: ACM, 2010. 1333–1340

  35. 35

    Mao K, Yang Y, Wang Q, et al. Developer recommendation for crowdsourced software development tasks. In: Proceedings of IEEE Symposium on Service-Oriented System Engineering. Washington: IEEE, 2015. 347–356

  36. 36

    Wang S, Zhang W, Wang Q. Fixercache: unsupervised caching active developers for diverse bug triage. In: Proceedings of ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. New York: ACM, 2014. 1–10

  37. 37

    Park J-W, Lee M-W, Kim J, et al. Costriage: a cost-aware triage algorithm for bug reporting systems. In: Proceedings of the 25th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2011. 139–144

  38. 38

    Huo D, Ding T, McMillan C, et al. An empirical study of the effects of expert knowledge on bug reports. In: Proceedings of IEEE International Conference on Software Maintenance and Evolution. Washington: IEEE, 2014. 1–10

  39. 39

    Meyer A N, Fritz T, Murphy G C, et al. Software developers’ perceptions of productivity. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York: ACM, 2014. 19–29

  40. 40

    Fritz T, Begel A, Müller S C, et al. Using psycho-physiological measures to assess task difficulty in software development. In: Proceedings of the 36th International Conference on Software Engineering. New York: ACM, 2014. 402–413

  41. 41

    Shihab E, Ihara A, Kamei Y, et al. Studying re-opened bugs in open source software. Empir Softw Eng, 2013, 18: 1005–1042

  42. 42

    Xia X, Lo D, Shihab E, et al. Automatic, high accuracy prediction of reopened bugs. Autom Softw Eng, 2015, 22: 75–109

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. 61502345, 61403057, 61370144, 61272089) and New Century Excellent Talents in University (Grant No. NCET-13-0073).

Author information

Correspondence to Jifeng Xuan or He Jiang.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Xuan, J., Jiang, H., Zhang, H. et al. Developer recommendation on bug commenting: a ranking approach for the developer crowd. Sci. China Inf. Sci. 60, 072105 (2017). https://doi.org/10.1007/s11432-015-0582-8

Download citation

Keywords

  • developer recommendation
  • bug comments
  • empirical analysis
  • recommendation for the crowd
  • collaborative filtering
  • software repositories

关键词

  • 开发者推荐
  • bug评论
  • 实证分析
  • 群体推荐
  • 协同过滤
  • 软件仓库