An Analysis of Koza’s Computational Effort Statistic for Genetic Programming

Abstract

As research into the theory of genetic programming progresses, more effort is being placed on systematically comparing results to give an indication of the effectiveness of sundry modifications to traditional GP. The statistic that is commonly used to report the amount of computational effort to solve a particular problem with 99% probability is Koza’s I(M, i, z) statistic. This paper analyzes this measure from a statistical perspective. In particular, Koza’s I tends to underestimate the true computational effort, by 25% or more for commonly used GP parameters and run sizes. The magnitude of this underestimate is nonlinearly decreasing with increasing run count, leading to the possibility that published results based on few runs may in fact be unmatchable when replicated at higher resolution. Additional analysis shows that this statistic also underreports the generation at which optimal results are achieved.