Article

Empirical Software Engineering

, Volume 10, Issue 4, pp 405-435

Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact

  • Hyunsook DoAffiliated withDepartment of Computer Science and Engineering, University of Nebraska–Lincoln Email author 
  • , Sebastian ElbaumAffiliated withDepartment of Computer Science and Engineering, University of Nebraska–Lincoln
  • , Gregg RothermelAffiliated withDepartment of Computer Science and Engineering, University of Nebraska–Lincoln

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access

Abstract

Where the creation, understanding, and assessment of software testing and regression testing techniques are concerned, controlled experimentation is an indispensable research methodology. Obtaining the infrastructure necessary to support such experimentation, however, is difficult and expensive. As a result, progress in experimentation with testing techniques has been slow, and empirical data on the costs and effectiveness of techniques remains relatively scarce. To help address this problem, we have been designing and constructing infrastructure to support controlled experimentation with testing and regression testing techniques. This paper reports on the challenges faced by researchers experimenting with testing techniques, including those that inform the design of our infrastructure. The paper then describes the infrastructure that we are creating in response to these challenges, and that we are now making available to other researchers, and discusses the impact that this infrastructure has had and can be expected to have.

Keywords

Software testing regression testing controlled experimentation experiment infrastructure