Can Formal Methods Improve the Efficiency of Code Reviews?
Code reviews are a provenly effective technique to find defects in source code as well as to increase its quality. Industrial software production often relies on code reviews as a standard QA mechanism. Surprisingly, though, tool support for reviewing activities is rare. Existing systems help to keep track of the discussion during the review, but do not support the reviewing activity directly. In this paper we argue that such support can be provided by formal analysis tools. Specifically, we use symbolic execution to improve the program understanding subtask during a code review. Tool support is realized by an Eclipse extension called Symbolic Execution Debugger. It allows one to explore visually a symbolic execution tree for the program under inspection. For evaluation we carefully designed a controlled experiment. We provide statistical evidence that with the help of symbolic execution defects are identified in a more effective manner than with a merely code-based view. Our work suggests that there is huge potential for formal methods not only in the production of safety-critical systems, but for any kind of software and as part of a standard development process.
KeywordsCode review Symbolic execution Empirical evaluation
We thank all participants of the evaluation for their valuable time and feedback.
- 3.Humphrey, W.S.: A Discipline for Software Engineering. Addison-Wesley Longman Publishing Co., Inc, Boston (1995)Google Scholar
- 4.Humphrey, W.S.: Introduction to the Personal Software Process. Addison-Wesley Longman Publishing Co., Inc, Boston (1997)Google Scholar
- 6.Burstall, R.M.: Program proving as hand simulation with a little induction. In: Information Processing 1974. Elsevier/North-Holland (1974)Google Scholar
- 7.Katz, S., Manna, Z.: Towards automatic debugging of programs. In: Proceedings of the International Conference on Reliable Software, pp. 143–155. ACM Press, Los Angeles (1975)Google Scholar
- 10.Hentschel, M., Bubel, R., Hähnle, R.: Symbolic execution debugger (SED). In: Bonakdarpour, B., Smolka, S.A. (eds.) RV 2014. LNCS, vol. 8734, pp. 255–262. Springer, Heidelberg (2014)Google Scholar
- 12.Hentschel, M., Hähnle, R., Bubel, R.: Visualizing unbounded symbolic execution. In: Seidl, M., Tillmann, N. (eds.) TAP 2014. LNCS, vol. 8570, pp. 82–98. Springer, Heidelberg (2014)Google Scholar
- 13.Leavens, G.T., Poll, E., Clifton, C., Cheon, Y., Ruby, C., Cok, D., Müller, P., Kiniry, J., Chalin, P., Zimmerman, D.M., Dietl, W.: JML Reference Manual. Draft Revision 2344, 31, May 2013Google Scholar
- 19.McConnell, S.: Code Complete, 2nd edn. Microsoft Press, Redmond (2004)Google Scholar
- 20.Zeller, A.: Why Programs Fail–A Guide to Systematic Debugging, 2nd edn. Elsevier, San Francisco (2006)Google Scholar
- 21.Bloch, J.: Effective Java, 2nd edn. Prentice Hall, Upper Saddle River (2008)Google Scholar
- 22.Frigge, M., Hoaglin, D.C., Iglewicz, B.: Some implementations of the boxplot. Am. Stat. 43(1), 50–54 (1989)Google Scholar