Relationships Between Average Depth and Number of Nodes for Decision Trees

Abstract

This paper presents a new tool for the study of relationships between total path length or average depth and number of nodes of decision trees. In addition to algorithm, the paper also presents the results of experiments with datasets from UCI ML Repository [1].

Appendix: Transformation of Functions

Let $f$ and $g$ be two functions from a set $A$ onto $C_f$ and $C_g$ respectively, where $C_f$ and $C_g$ are finite sets of nonnegative integers. Let $B_f = \{m_f, m_f+1, \ldots , M_f\}$ and $B_g = \{n_g, n_g+1, \ldots , N_g\}$ where $m_f = \min \{m : m\in C_f\}$ and $n_g = \min \{n: n\in C_g\}$. Furthermore, $M_f$ and $N_g$ are natural numbers such that $m\le M_f$ and $n\le N_g$ for any $m\in C_f$ and $n\in C_g$, respectively.

We define two functions ${\mathcal{F}} :B_g \rightarrow B_f$ and ${\mathcal{G}} :B_f \rightarrow B_g$ as follows:

$${\mathcal{F}} (n) = \min \{f(a) : a\in A, g(a) \le n\},\; \forall n\in B_g,$$

(1)

$${\mathcal{G}}(m) = \min \{g(a) : a\in A, f(a) \le m\},\; \forall m\in B_f.$$

(2)

It is clear that both ${\mathcal{F}}$ and ${\mathcal{G}}$ are nonincreasing functions.

The following proposition states that the functions ${\mathcal{F}}$ and ${\mathcal{G}}$ can be used interchangeably and we can evaluate ${\mathcal{F}}$ using ${\mathcal{G}}$ and vice versa, i.e., it is enough to know only one function to evaluate the other.

Proposition 3

For any $n\in B_g$,

$${\mathcal{F}} (n) = \min \{m\in B_f: {\mathcal{G}} (m) \le n\},$$

and for any $m\in B_f$,

$${\mathcal{G}} (m) = \min \{n\in B_g: {\mathcal{F}} (n) \le m\}.$$

Proof

Let for some $n\in B_g$

$${\mathcal{F}} (n) = m_0.$$

(3)

Furthermore, we assume that

$$\min \{m \in B_f: {\mathcal{G}} (m) \le n\} = t.$$

(4)

From (3) it follows that

• (i) there exists $b\in A$ such that $g(b) \le n$ and $f(b) = m_0$;

• (ii) for any $a\in A$ if $g(a) \le n$ then $f(a) \ge m_0$.

From (i) it follows that ${\mathcal{G}} (m_0) \le n$. This implies $t\le m_0$. Let us assume that $t< m_0$. In this case, there exits $m_1<\,m_0$ for which ${\mathcal{G}} (m_1) \le n$. Therefore, there exists $a\in A$ such that $f(a) \le m_1$ and $g(a) \le n$, but from (ii) it follows that $f(a) \ge m_0$, which is impossible. So $t=m_0$.

Similarly, we can prove the second part of the statement.

Proposition 3 allows us to transform the function ${\mathcal{G}}$ given by a tuple $\left( {\mathcal{G}} (m_f), {\mathcal{G}} (m_f+1), \ldots , {\mathcal{G}} (M_f)\right)$ into the function ${\mathcal{F}}$ and vice versa. We know that ${\mathcal{G}} (m_f) \ge {\mathcal{G}} (m_f + 1) \ge \cdots \ge {\mathcal{G}} (M_f)$, to find the minimum $m\in B_f$ such that ${\mathcal{G}} (m) \le m$ we can use binary search which requires $O(\log |B_f|)$ comparisons of numbers. So to find the value ${\mathcal{F}} (n)$ for $n\in B_g$ it is enough to make $O(\log |B_f|)$ operations of comparison.