Developer recommendation on bug commenting: a ranking approach for the developer crowd

  • Jifeng Xuan
  • He Jiang
  • Hongyu Zhang
  • Zhilei Ren
Research Paper


A bug tracking system provides a collaborative platform for the developer crowd. After a bug report is submitted, developers can make comments to supplement the details of the bug report. Due to the large number of developers and bug reports, it is hard to determine which developer (also called commenter) is able to comment on a particular bug report. We refer to the problem of recommending developers for commenting on bug reports as commenter recommendation. In this paper, we perform an empirical analysis on commenter recommendation based on five-year bug reports of four open source projects. First, we preliminarily analyze bug comments and commenters in three categories, the relationship between commenters and fixers, the data scale of comments, and the collaboration on bug commenting. Second, we design a recommendation approach via ranking developers in the crowd to reduce the manual effort of identifying commenters. In this approach, we formulize the commenter recommendation problem as a multi-label recommendation task and leverage both developer collaboration and bug content to find out appropriate commenters. Experimental results show that our approach can effectively recommend commenters; 41% to 75% of the recall value is achieved for top-10 recommendation. Our empirical analysis on bug commenting can help developers understand and improve the process of fixing bugs.


developer recommendation bug comments empirical analysis recommendation for the crowd collaborative filtering software repositories 


软件bug跟踪系统为开发者群体提供了协作平台。在一个bug报告提交后, 开发者可以通过评论补充bug报告的细节。鉴于开发者和bug报告数量庞大, 很难决定哪个开发者(也称评论者)具有为某个bug报告评论的能力。本文称推荐开发者评论bug报告的问题为评论者推荐。我们基于四个开源项目历时五年的bug报告进行评论者推荐的实证分析。首先, 我们初步分析了bug评论和评论者的三个方面, 即评论者和修复者的关系、评论的数据规模和bug的协同评论。其次, 为了减少手工识别评论者的成本, 我们设计了基于群体中开发者排序的评论者推荐方法。该方法中, 我们将评论者推荐问题描述为多标签推荐任务, 并基于开发者协作和bug内容两个方面找到合适的评论者。实验结果表明该方法可以有效的推荐评论者。推荐前10个候选时可以获得41%到75%的召回率。对bug评论的实证分析可以帮助开发者理解和提高bug的修复过程。


开发者推荐 bug评论 实证分析 群体推荐 协同过滤 软件仓库 



This work was supported by National Natural Science Foundation of China (Grant Nos. 61502345, 61403057, 61370144, 61272089) and New Century Excellent Talents in University (Grant No. NCET-13-0073).


  1. 1.
    Wu W, Tsai W-T, Li W. An evaluation framework for software crowdsourcing. Front Comput Sci, 2013, 7: 694–709MathSciNetCrossRefGoogle Scholar
  2. 2.
    Zimmermann T, Premraj R, Bettenburg N, et al. What makes a good bug report? IEEE Trans Softw Eng, 2010, 36: 618–643CrossRefGoogle Scholar
  3. 3.
    Xuan J F, Martinez M, DeMarco F, et al. Nopol: automatic repair of conditional statement bugs in java programs. IEEE Trans Softw Eng, 2017, 43: 34–55CrossRefGoogle Scholar
  4. 4.
    Anvik J, Hiew L, Murphy G C. Who should fix this bug? In: Proceedings of the 28th International Conference on Software Engineering. New York: ACM, 2006. 361–370Google Scholar
  5. 5.
    Meneely A, Williams L. Socio-technical developer networks: should we trust our measurements? In: Proceedings of the 33rd International Conference on Software Engineering. New York: ACM, 2011. 281–290Google Scholar
  6. 6.
    Zhou J, Zhang H, Lo D. Where should the bugs be fixed? more accurate information retrieval-based bug localization based on bug reports. In: Proceedings of the 34th International Conference on Software Engineering. Washington: IEEE, 2012. 14–24Google Scholar
  7. 7.
    Lotufo R, Malik Z, Czarnecki K. Modelling the hurried bug report reading process to summarize bug reports. In: Proceedings of the 28th IEEE International Conference on Software Maintenance. Washington: IEEE, 2012. 430–439Google Scholar
  8. 8.
    Gao J, Bai X, Tsai W-T, et al. Mobile application testing: a tutorial. IEEE Comput, 2014, 47: 46–55CrossRefGoogle Scholar
  9. 9.
    Anvik J, Murphy G C. Determining implementation expertise from bug reports. In: Proceedings of the 4th International Workshop on Mining Software Repositories. Washington: IEEE, 2007. 2Google Scholar
  10. 10.
    Tamrawi A, Nguyen T T, Al-Kofahi J M, et al. Fuzzy set and cache-based approach for bug triaging. In: Proceedings of the 19th ACM SIGSOFT Symposium on Foundations of Software Engineering and the 13th European Software Engineering Conference. New York: ACM, 2011. 365–375Google Scholar
  11. 11.
    Pinzger M, Nagappan N, Murphy B. Can developer-module networks predict failures? In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York: ACM, 2008. 2–12Google Scholar
  12. 12.
    Wang X, Zhang L, Xie T, et al. An approach to detecting duplicate bug reports using natural language and execution information. In: Proceedings of the 30th International Conference on Software Engineering. New York: ACM, 2008. 461–470Google Scholar
  13. 13.
    Sun C, Lo D, Khoo S C, et al. Towards more accurate retrieval of duplicate bug reports. In: Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering. Washington: IEEE, 2011. 253–262Google Scholar
  14. 14.
    Shihab E, Ihara A, Kamei Y, et al. Predicting re-opened bugs: a case study on the eclipse project. In: Proceedings of the 17th Working Conference on Reverse Engineering. Washington: IEEE, 2010. 249–258Google Scholar
  15. 15.
    Hu H, Zhang H, Xuan J, et al. Effective bug triage based on historical bug-fix information. In: Proceedings of IEEE 25th International Symposium on Software Reliability Engineering. Washington: IEEE, 2014. 122–132Google Scholar
  16. 16.
    Xuan J, Jiang H, Hu Y, et al. Towards effective bug triage with software data reduction techniques. IEEE Trans Knowl Data Eng, 2015, 27: 264–280CrossRefGoogle Scholar
  17. 17.
    Hong Q, Kim S, Cheung S C, et al. Understanding a developer social network and its evolution. In: Proceedings of the 27th IEEE International Conference on Software Maintenance. Washington: IEEE, 2011. 323–332Google Scholar
  18. 18.
    Han J, Kamber M. Data Mining: Concepts and Techniques. San Francisco: Morgan Kaufmann, 2001. 71–72Google Scholar
  19. 19.
    Jiang H, Ren Z, Xuan J, et al. Extracting elite pairwise constraints for clustering. Neurocomputing, 2013, 99: 124–133CrossRefGoogle Scholar
  20. 20.
    Herlocker J L, Konstan J A, Terveen L G, et al. Evaluating collaborative filtering recommender systems. ACM Trans Inf Syst, 2004, 22: 5–53CrossRefGoogle Scholar
  21. 21.
    Sarwar B, Karypis G, Konstan J, et al. Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web. New York: ACM, 2001. 285–295Google Scholar
  22. 22.
    Adomavicius G, Tuzhilin A. Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng, 2005, 17: 734–749CrossRefGoogle Scholar
  23. 23.
    Matter D, Kuhn A, Nierstrasz O. Assigning bug reports using a vocabulary-based expertise model of developers. In: Proceedings of the 6th IEEE International Working Conference on Mining Software Repositories. Washington: IEEE, 2009. 131–140Google Scholar
  24. 24.
    Ochiai A. Zoogeographic studies on the soleoid fishes found in japan and its neighbouring regions. Bull Jpn Soc Sci Fish, 1957, 22: 526–530CrossRefGoogle Scholar
  25. 25.
    Dice L R. Measures of the amount of ecologic association between species. Ecology, 1945, 26: 297–302CrossRefGoogle Scholar
  26. 26.
    Bettenburg N, Premraj R, Zimmermann T, et al. Duplicate bug reports considered harmful... really? In: Proceedings of IEEE International Conference on Software Maintenance. Washington: IEEE, 2008. 337–345Google Scholar
  27. 27.
    Wu W, Zhang W, Yang Y, et al. Drex: developer recommendation with k-nearest-neighbor search and expertise ranking. In: Proceedings of the 18th Asia Pacific Software Engineering Conference. Washington: IEEE, 2011. 389–396Google Scholar
  28. 28.
    Xie X, Zhang W, Yang Y, et al. Dretom: developer recommendation based on topic models for bug resolution. In: Proceedings of the 8th International Conference on Predictive Models in Software Engineering. New York: ACM, 2012. 19–28Google Scholar
  29. 29.
    Xuan J F, Cornu B, Martinez M, et al. B-refactoring: automatic test code refactoring to improve dynamic analysis. Inf Softw Tech, 2016, 76: 65–80CrossRefGoogle Scholar
  30. 30.
    Rastkar S, Murphy G C, Murray G. Summarizing software artifacts: a case study of bug reports. In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering. New York: ACM, 2010. 505–514Google Scholar
  31. 31.
    Xia X, Lo D, Wang X, et al. Dual analysis for recommending developers to resolve bugs. J Softw Evol Process, 2015, 27: 195–220CrossRefGoogle Scholar
  32. 32.
    Bird C, Pattison D, D’Souza R, et al. Latent social structure in open source projects. In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York: ACM, 2008. 24–35Google Scholar
  33. 33.
    Xuan J, Jiang H, Ren Z, et al. Solving the large scale next release problem with a backbone-based multilevel algorithm. IEEE Trans Softw Eng, 2012, 38: 1195–1212CrossRefGoogle Scholar
  34. 34.
    Jiang H, Xuan J, Ren Z. Approximate backbone based multilevel algorithm for next release problem. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation. New York: ACM, 2010. 1333–1340CrossRefGoogle Scholar
  35. 35.
    Mao K, Yang Y, Wang Q, et al. Developer recommendation for crowdsourced software development tasks. In: Proceedings of IEEE Symposium on Service-Oriented System Engineering. Washington: IEEE, 2015. 347–356Google Scholar
  36. 36.
    Wang S, Zhang W, Wang Q. Fixercache: unsupervised caching active developers for diverse bug triage. In: Proceedings of ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. New York: ACM, 2014. 1–10Google Scholar
  37. 37.
    Park J-W, Lee M-W, Kim J, et al. Costriage: a cost-aware triage algorithm for bug reporting systems. In: Proceedings of the 25th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2011. 139–144Google Scholar
  38. 38.
    Huo D, Ding T, McMillan C, et al. An empirical study of the effects of expert knowledge on bug reports. In: Proceedings of IEEE International Conference on Software Maintenance and Evolution. Washington: IEEE, 2014. 1–10Google Scholar
  39. 39.
    Meyer A N, Fritz T, Murphy G C, et al. Software developers’ perceptions of productivity. In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York: ACM, 2014. 19–29Google Scholar
  40. 40.
    Fritz T, Begel A, Müller S C, et al. Using psycho-physiological measures to assess task difficulty in software development. In: Proceedings of the 36th International Conference on Software Engineering. New York: ACM, 2014. 402–413Google Scholar
  41. 41.
    Shihab E, Ihara A, Kamei Y, et al. Studying re-opened bugs in open source software. Empir Softw Eng, 2013, 18: 1005–1042CrossRefGoogle Scholar
  42. 42.
    Xia X, Lo D, Shihab E, et al. Automatic, high accuracy prediction of reopened bugs. Autom Softw Eng, 2015, 22: 75–109CrossRefGoogle Scholar

Copyright information

© Science China Press and Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  1. 1.State Key Laboratory of Software Engineering, School of ComputerWuhan UniversityWuhanChina
  2. 2.School of SoftwareDalian University of TechnologyDalianChina
  3. 3.Microsoft Research AsiaBeijingChina
  4. 4.School of Electrical Engineering and Computer ScienceThe University of NewcastleCallaghanAustralia

Personalised recommendations