Optimum decision trees — an optimal variable theorem and its related applications

Summary

The problem of translation of a decision table into an optimal sequential testing procedure (STP) under two cost criteria is considered. We formulate the STP as a decision tree and introduce its description and execution costs. As usual, optimal STPs are defined as minimum cost decision trees. We call a variable optimal when it is tested first in an optimal STP. In other words an optimal variable is one which is placed (and hence to be tested) at the root of an optimum decision tree. We introduce a notion of quasi-decisive variables and prove that under the strong equivalence assumption on quasi-decisive variables (SEA), they are optimal, and conversely, only the quasi-decisive variables are optimal whenever they exist in both costs (the optimal variable theorem). SEA holds in the general description cost and in the uniform execution cost. We show that SEA in the theorem can not be weakened in case of the general execution cost.

We describe an algorithm for the construction of an optimum decision tree which requires O(N ^log3 log N) comparison operations, where N is the size of the input table. Then we show that we can speed-up this algorithm by applying the optimal variable theorem in description cost case. In fact the number of executions of the inner-most loop of the algorithm is reduced greatly under fairly reasonable assumptions on the test data tables, resulting in a considerable reduction in the total execution time of the algorithm.

As a related topic we discuss optimization problems of quasi-decisive chains under execution cost. Our optimization algorithm requires O(N log N) operations.