DeepReview: Automatic Code Review Using Deep Multi-instance Learning

  • Heng-Yi Li
  • Shu-Ting Shi
  • Ferdian Thung
  • Xuan Huo
  • Bowen Xu
  • Ming LiEmail author
  • David Lo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11440)


Code review, an inspection of code changes in order to identify and fix defects before integration, is essential in Software Quality Assurance (SQA). Code review is a time-consuming task since the reviewers need to understand, analysis and provide comments manually. To alleviate the burden of reviewers, automatic code review is needed. However, this task has not been well studied before. To bridge this research gap, in this paper, we formalize automatic code review as a multi-instance learning task that each change consisting of multiple hunks is regarded as a bag, and each hunk is described as an instance. We propose a novel deep learning model named DeepReview based on Convolutional Neural Network (CNN), which is an end-to-end model that learns feature representation to predict whether one change is approved or rejected. Experimental results on open source projects show that DeepReview is effective in automatic code review tasks. In terms of F1 score and AUC, DeepReview outperforms the performance of traditional single-instance based model TFIDF-SVM and the state-of-the-art deep feature based model Deeper.


Software mining Machine learning Multi-instance learning Automatic code review 



This research was supported by National Key Research and Development Program (2017YFB1001903) and NSFC (61751306).


  1. 1.
    Ebert, F., Castor, F., Novielli, N., Serebrenik, A.: Confusion detection in code reviews. In: ICSME, pp. 549–553 (2017)Google Scholar
  2. 2.
    Gay, G., Haiduc, S., Marcus, A., Menzies, T.: On the use of relevance feedback in IR-based concept location. In: ICSM, pp. 351–360 (2009)Google Scholar
  3. 3.
    Huo, X., Li, M.: Enhancing the unified features to locate buggy files by exploiting the sequential nature of source code. In: IJCAI, pp. 1909–1915 (2017)Google Scholar
  4. 4.
    Huo, X., Li, M., Zhou, Z.H.: Learning unified features from natural and programming languages for locating buggy source code. In: IJCAI, pp. 1606–1612 (2016)Google Scholar
  5. 5.
    Jiang, T., Tan, L., Kim, S.: Personalized defect prediction. In: ASE, pp. 279–289 (2013)Google Scholar
  6. 6.
    Kim, Y.: Convolutional neural networks for sentence classification. In: EMNLP, pp. 1746–1751 (2014)Google Scholar
  7. 7.
    Kononenko, O., Baysal, O., Guerrouj, L., Cao, Y., Godfrey, M.W.: Investigating code review quality: do people and participation matter? In: ICSME, pp. 111–120 (2015)Google Scholar
  8. 8.
    Lee, S., Heo, M., Lee, C., Kim, M., Jeong, G.: Applying deep learning based automatic bug triager to industrial projects. In: ESEC/FSE, pp. 926–931 (2017)Google Scholar
  9. 9.
    Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE TSE 33(1), 2–13 (2007)Google Scholar
  10. 10.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS, pp. 3111–3119 (2013)Google Scholar
  11. 11.
    Mou, L., Li, G., Zhang, L., Wang, T., Jin, Z.: Convolutional neural networks over tree structures for programming language processing. In: AAAI, pp. 1287–1293 (2016)Google Scholar
  12. 12.
    Nam, J., Pan, S.J., Kim, S.: Transfer defect learning. In: ICSE, pp. 382–391 (2013)Google Scholar
  13. 13.
    Rigby, P.C., Bird, C.: Convergent contemporary software peer review practices. In: FSE, pp. 202–212 (2013)Google Scholar
  14. 14.
    Rigby, P.C., German, D.M., Storey, M.A.: Open source software peer review practices: a case study of the apache server. In: ICSE, pp. 541–550 (2008)Google Scholar
  15. 15.
    Tao, Y., Han, D., Kim, S.: Writing acceptable patches: an empirical study of open source project patches. In: ICSME, pp. 271–280 (2014)Google Scholar
  16. 16.
    Thongtanunam, P., Tantithamthavorn, C., Kula, R.G., Yoshida, N., Iida, H., Matsumoto, K.I.: Who should review my code? A file location-based code-reviewer recommendation approach for modern code review. In: SANER, pp. 141–150 (2015)Google Scholar
  17. 17.
    Wang, S., Liu, T., Tan, L.: Automatically learning semantic features for defect prediction. In: ICSE, pp. 297–308 (2016)Google Scholar
  18. 18.
    Wei, H.H., Li, M.: Supervised deep features for software functional clone detection by exploiting lexical and syntactical information in source code. In: IJCAI, pp. 3034–3040 (2017)Google Scholar
  19. 19.
    Xu, B., Ye, D., Xing, Z., Xia, X., Chen, G., Li, S.: Predicting semantically linkable knowledge in developer online forums via convolutional neural network. In: ASE, pp. 51–62 (2016)Google Scholar
  20. 20.
    Yang, X., Lo, D., Xia, X., Zhang, Y., Sun, J.: Deep learning for just-in-time defect prediction. In: QRS, pp. 17–26 (2015)Google Scholar
  21. 21.
    Zanjani, M.B., Kagdi, H., Bird, C.: Automatically recommending peer reviewers in modern code review. IEEE TSE 42(6), 530–543 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Heng-Yi Li
    • 1
  • Shu-Ting Shi
    • 1
  • Ferdian Thung
    • 2
  • Xuan Huo
    • 1
  • Bowen Xu
    • 2
  • Ming Li
    • 1
    Email author
  • David Lo
    • 2
  1. 1.National Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina
  2. 2.School of Information SystemsSingapore Management UniversitySingaporeSingapore

Personalised recommendations