Effects on Unit Tests – Preliminary Analysis

Abstract

With the rising acceptance of XP, and agile methodologies in general, a growing number of software projects develop and maintain large test suites. Tests are considered a kind of a live documentation for the production code, because tests are always kept in sync with the code as opposed to typical text-based documentation which may not be in sync with the code. However, the first and foremost tests are used to execute a program with the intent of finding errors [191]. Hence, not only the thoroughness of developed tests (which is often taken into consideration by means of code coverage measures) but also the fault detection effectiveness of developed tests play the key part in software development. As explained in Sect. 3.3.2.4, code coverage measures can be useful as indicators of the thoroughness of unit test suites [170], while mutation score is a more powerful and more effective measure of the fault finding effectiveness of test suites than statement and branch coverage , and data-flow [82 ,197]. Unfortunately, the empirical evidence on the impact of the TF practice on unit test suite characteristics is limited to code coverage. Therefore, preliminary results, presented in this section, extend the body of knowledge in software engineering by means of the analysis of the impact of the TF practice on mutation score indicator (an indicator of the fault detection effectiveness of developed unit tests).

Count what is countable, measure what is measurable and what is not measurable, make measurable

Galileo Galilei