Time and space complexity of deterministic and nondeterministic decision trees

In this paper, we study arbitrary infinite binary information systems each of which consists of an infinite set called universe and an infinite set of two-valued functions (attributes) defined on the universe. We consider the notion of a problem over information system which is described by a finite number of attributes and a mapping corresponding a decision to each tuple of attribute values. As algorithms for problem solving, we use deterministic and nondeterministic decision trees. As time and space complexity, we study the depth and the number of nodes in the decision trees. In the worst case, with the growth of the number of attributes in the problem description, (i) the minimum depth of deterministic decision trees grows either almost as logarithm or linearly, (ii) the minimum depth of nondeterministic decision trees either is bounded from above by a constant or grows linearly, (iii) the minimum number of nodes in deterministic decision trees has either polynomial or exponential growth, and (iv) the minimum number of nodes in nondeterministic decision trees has either polynomial or exponential growth. Based on these results, we divide the set of all infinite binary information systems into five complexity classes, and study for each class issues related to time-space trade-off for decision trees.


Introduction
In this paper, we divide the set of all infinite binary information systems into five complexity classes depending on the worst case time and space complexity of deterministic an nondeterministic decision trees, and study for each class issues related to time-space trade-off for decision trees.
General information system [27] consists of a universe (a set of objects) and a set of attributes (functions with finite image) defined on the universe.An information system is called infinite, if its set of attributes is infinite.Otherwise, it is called finite.An information system is called binary if each its attribute has values from the set {0, 1}.
Any problem over an information system is described by a finite number of attributes that divide the universe into domains in which these attributes have fixed values.A decision is attached to each domain.For a given object from the universe, it is required to find the decision attached to the domain containing this object.
As algorithms solving these problems, deterministic and nondeterministic decision trees are considered.Deterministic decision trees are widely used as classifiers to predict decisions for new objects, as a means of knowledge representation, and as algorithms to solve problems of fault diagnosis, computational geometry, combinatorial optimization, etc. [5,23,28].Nondeterministic decision trees are less known.They are closely related to systems of true decision rules that cover all objects from the universe.As time complexity of a decision tree, we consider its depth -the maximum number of nodes labeled with attributes in a path from the root to a terminal node.As space complexity of a decision tree, we consider its number of nodes.
Both theoretical and experimental investigations of time complexity of nondeterministic decision trees are mainly related to decision trees for Boolean functions [1,4,17].Note that, for a Boolean function, the minimum depth of a nondeterministic decision tree is equal to its certificate complexity [6].
The most part of results on deterministic decision trees is obtained for finite information systems.The results related to infinite information systems were achieved initially in the study of deterministic linear and algebraic decision trees.In this case, the universe is a subset of n-dimensional real space and each attribute is of the kind signf , where f is a linear form with n variables for linear decision trees, and f is a polynomial with n variables for algebraic decision trees.
In [8,9,13], the lower bounds close to n log 2 t were obtained for the minimum depth of linear decision trees, where t is the number of attributes in the problem description.Lower bounds on the minimum depth of algebraic decision trees were obtained later [3,10,31,32,33] as well as lower bounds on the minimum number of nodes in the algebraic decision trees [11].
In our view, the problems of complexity of decision trees over arbitrary infinite information systems were not considered prior to [16,15] for deterministic and prior to [18,19] for nondeterministic decision trees.
We developed two approaches to the study of deterministic and nondeterministic decision trees over arbitrary information systems: local, when the decision trees solving a problem can use only attributes from the problem description, and global, when the decision trees solving a problem can use arbitrary attributes from the considered information system [2,15,19,21,22,23,24,25].
Based on the obtained results we can describe possible types of behavior of four functions h d U , h a U , L d U , L a U that characterize worst case time and space complexity of deterministic and nondeterministic decision trees over an infinite binary information system U .
The function h d U characterizes the growth in the worst case of the minimum depth of a deterministic decision tree solving a problem with the growth of the number of attributes in the problem description.The function h d U is either bounded from below by logarithm and bounded from above by logarithm to the power 1 + ε, where ε is an arbitrary positive real number, or grows linearly.
The function h a U characterizes the growth in the worst case of the minimum depth of a nondeterministic decision tree solving a problem with the growth of the number of attributes in the problem description.The function h a U is either bounded from above by a constant or grows linearly.
The function L d U characterizes the growth in the worst case of the minimum number of nodes in a deterministic decision tree solving a problem with the growth of the number of attributes in the problem description.The function L d U has either polynomial or exponential growth.
The function L a U characterizes the growth in the worst case of the minimum number of nodes in a nondeterministic decision tree solving a problem with the growth of the number of attributes in the problem description.The function L a U has either polynomial or exponential growth.
We see that each of the functions h d U , h a U , L d U , L a U has two types of behavior.Thus, the tuple (h d U , h a U , L d U , L a U ) can have (a priori) 16 types of behavior.However (and this is one of the main results of the paper), the tuple (h d U , h a U , L d U , L a U ) can have only five types of behavior.All these types are enumerated in the paper and each type is illustrated by an example.Similar result without proofs and with weaker bounds on the functions h d U , h a U , L d U , L a U was announced in [20].
There are five complexity classes of infinite binary information systems corresponding to the five possible types of the tuple ( For each class, we study joint behavior of time and space complexity of decision trees.
A pair of functions (ϕ, ψ) is called a boundary d-pair of the information system U if, for any problem over U , there exists a deterministic decision tree over U which solves this problem and for which the depth is at most ϕ(n) and the number of nodes is at most ψ(n), where n is the number of attributes in the problem description.A boundary d-pair (ϕ, ψ) of the information system U is called optimal if, for any boundary d-pair (ϕ ′ , ψ ′ ) of U , the inequalities ϕ ′ (n) ≥ ϕ(n) and ψ ′ (n) ≥ ψ(n) hold for any natural n.An information system U is called d-reachable if the pair (h d U , L d U ) is boundary (and, consequently, optimal boundary) d-pair of the system U .For nondeterministic decision trees, the notions of a boundary a-pair of an information system, an optimal a-pair, and a-reachable information system are defined in a similar way.For deterministic decision trees, the best situation is when the considered information system is d-reachable: for any boundary d-pair (ϕ, ψ) for an information system U and any natural n, ϕ(n For nondeterministic decision trees, the best situation is when the information system is a-reachable. For four out of the five complexity classes, all information systems from the class are dreachable.One class contains both information systems that are d-reachable and information systems that are not d-reachable.For each information system U that is not d-reachable, we find a nontrivial boundary d-pair which is enough close to the pair (h d U , L d U ).For two out of the five complexity classes, all information systems from the class are a-reachable.For the rest three classes, all information systems from the class are not a-reachable.For some information systems U that are not a-reachable, we find nontrivial boundary a-pairs which are enough close to (h a U , L a U ).For the rest of information systems U that are not a-reachable, the pair (n, L a U (n)) is the optimal boundary a-pair.Note that for these information systems, the function h a U is bounded from above by a constant.
The obtained results are related to time-space trade-off for deterministic and nondeterministic decision trees.For any information system U for each problem, there exists a deterministic decision tree solving this problem which depth is at most h d U (n), and there exists a deterministic decision tree solving this problem for which the number of nodes is at most where n is the number of attributes in the problem description.If an information system U is not d-reachable, then there exists a problem such that there is no a deterministic decision tree solving this problem which depth is at most h d U (n) and the number of nodes is at most L d U (n), where n is the number of attributes in the problem description.Similar situation is with nondeterministic decision trees for information systems that are not a-reachable.
Let us consider an information system U for which the function h a U is bounded from above by a natural number c, and (n, L a U (n)) is the optimal boundary a-pair.For each problem over U , there exists a nondeterministic decision tree solving this problem which depth is at most c.However, for any natural n greater than c, there is no a finite upper bound on the number of nodes in such trees for problems described by at most n attributes.
Note that a part of the obtained results can be extended to infinite k-valued information systems, k > 2, in particular, the results about five possible types of infinite binary information systems -see [20].
The rest of the paper is organized as follows: Section 2 contains main results and Sections 3-5 -proofs of these results.

Main Results
Let A be an infinite set and F be an infinite set of functions that are defined on A and have values from the set {0, 1}.The pair U = (A, F ) is called an infinite binary information system [27], elements of the set A are called objects, and functions from F are called attributes.The set A is called sometimes the universe of the information system U .
A problem over U is a tuple of the kind z = (ν, f 1 , . . ., f n ), where ν : {0, 1} n → N, N is the set of natural numbers {1, 2, . ..}, and f 1 , . . ., f n ∈ F .The problem z consists in finding the value of the function z(x) = ν(f 1 (x), . . ., f n (x)) for a given object a ∈ A. Various problems of pattern recognition, combinatorial optimization, fault diagnosis, computational geometry, etc., can be represented in this form.The value dim z = n is called the dimension of the problem z.
As algorithms for problem solving we consider decision trees.A decision tree over the information system U is a directed tree with the root in which the root and edges leaving the root are not labeled, each terminal node is labeled with a number from N, each working node (which is neither the root nor a terminal node) is labeled with an attribute from F , and each edge leaving a working node is labeled with a number from the set {0, 1}.A decision tree is called deterministic if only one edge leaves the root and edges leaving an arbitrary working node are labeled with different numbers.
Let Γ be a decision tree over U and be a directed path from the root v 0 to a terminal node v m+1 of Γ (we call such path complete).Define a subset A(ξ) of the set A. If m = 0, then A(ξ) = A. Let m > 0 and, for i = 1, . . ., m, the node v i be labeled with the attribute f j i and the edge d i be labeled with the number δ i .Then The decision tree Γ solves the problem z nondeterministically if, for any object a ∈ A, there exists a complete path ξ of Γ such that a ∈ A(ξ) and, for each a ∈ A and each complete path ξ such that a ∈ A(ξ), the terminal node of ξ is labeled with the number z(a) (in this case, we can say that Γ is a nondeterministic decision tree solving the problem z).In particular, if the decision tree Γ solves the problem z nondeterministically, then, for each complete path ξ of Γ, either the set A(ξ) is empty or the function z(x) is constant on the set A(ξ).The decision tree Γ solves the problem z deterministically if Γ is a deterministic decision tree which solves the problem z nondeterministically (in this case, we can say that Γ is a deterministic decision tree solving the problem z).
The depth of the decision tree Γ is the maximum number of working nodes in a complete path of Γ. Denote h(Γ) the depth of Γ and L(Γ) -the number of nodes in Γ.
Let P (U ) be the set of problems over U .For a problem z from P (U ), let h d U (z) be the minimum depth of a decision tree over U solving the problem z deterministically, h a U (z) be the minimum depth of a decision tree over U solving the problem z nondeterministically, L d U (z) be the minimum number of nodes in a decision tree over U solving the problem z deterministically, and L a U (z) be the minimum number of nodes in a decision tree over U solving the problem z nondeterministically.
We consider four functions defined on the set N in the following way: , where the maximum is taken among all problems z over U with dim z ≤ n.These functions describe how the minimum depth and the minimum number of nodes of deterministic and nondeterministic decision trees solving problems are growing in the worst case with the growth of problem dimension.To describe possible types of behavior of these four functions, we need to define some properties of infinite binary information systems.
Let m ∈ N. A nonempty subset B of the set A is called a (m, U )-set if B coincides with the set of solutions from A of an equation system of the kind {f 1 (x) = δ 1 , . . ., f m (x) = δ m }, where f 1 , . . ., f m are attributes from F (not necessary pairwise different), and δ 1 , . . ., δ m ∈ {0, 1}.We call such system an (m, U )-system of equations.It is clear that an (m, U )-set is also an (m + 1, U )-set.
We say that the information system U satisfies the condition of coverage if there exists m ∈ N such that any (m + 1, U )-set is a union of a finite number of (m, U )-sets.In this case, we will say that U satisfies the condition of coverage with parameter m.
We say that the information system U satisfies the condition of restricted coverage if there exist m, t ∈ N such that any (m + 1, U )-set is a union of at most t (m, U )-sets.In this case, we will say that U satisfies the condition of restricted coverage with parameters m and t.
A subset {f 1 , . . ., f m } of the set F is called independent if, for any δ 1 , . . ., δ m ∈ {0, 1}, the system of equations {f 1 (x) = δ 1 , . . ., f m (x) = δ m }, has a solution from the set A. The empty set of attributes is independent by definition.We define the parameter I(U ) which is called the independence dimension or I-dimension of the information system U (this notion is similar to the notion of independence number of family of sets [26]).If, for each m ∈ N, the set F contains an independent subset of cardinality m, then I(U ) = ∞.Otherwise, I(U ) is the maximum cardinality of an independent subset of the set F .
We now consider four statements that describe possible types of behavior of functions , and L a U (n).The next statement follows immediately from Theorem 2.1 [21] and simple fact that h d U (n) ≤ n for any n ∈ N. Proposition 1.For any infinite binary information system U , the function h d U (n) has one of the following two types of behavior: (LOG) If the system U has finite I-dimension and satisfies the condition of restricted coverage, then for any ε, 0 < ε < 1, there exists a positive constant c such that, for any n ∈ N, (LIN) If the system U has infinite I-dimension or does not satisfy the condition of restricted coverage, then for any n ∈ N, For any infinite binary information system U = (A, F ) , the function h a U (n) has one of the following two types of behavior: (CON) If the system U satisfies the condition of coverage, then there exists a positive constant c such that, for any n ∈ N, h a U (n) ≤ c. (LIN) If the system U does not satisfy the condition of coverage, then for any n ∈ N, h a U (n) = n.Proposition 3.For any infinite binary information system U , the function L d U (n) has one of the following two types of behavior: (POL) If the system U has finite I-dimension, then for any n ∈ N, ) .(EXP) If the system U has infinite I-dimension, then for any n ∈ N, For any infinite binary information system U and any n ∈ N, Let U be an infinite binary information system.Proposition 1 allows us to correspond to the function h d U (n) its type of behavior from the set {LOG, LIN}.Proposition 2 allows us to correspond to the function h a U (n) its type of behavior from the set {CON, LIN}.Propositions 3 and 4 allow us to correspond to each of the functions L d U (n) and L a U (n) its type of behavior from the set {POL, EXP}.A tuple obtained from the tuple ) by replacing functions with their types of behavior is called the type of the information system U .We now describe all possible types of infinite binary information systems. 1 is the type of some infinite binary information system.For i = 1, . . ., 5, we denote by W i the class of all infinite binary information systems which type coincides with the ith row of Table 1.We now study for each of these complexity classes joint behavior of the depth and number of nodes in decision trees solving problems.

Theorem 1. For any infinite binary information system, its type coincides with one of the rows of Table 1. Each row of Table
A pair of functions (ϕ, ψ), where ϕ : N → N ∪ {0} and ψ : N → N ∪ {0}, is called a boundary d-pair of the information system U if, for any problem z over U , there exists a decision tree Γ over U which solves the problem z deterministically and for which h is boundary (and, consequently, optimal boundary) d-pair of the system U .A pair of functions (ϕ, ψ), where ϕ : N → N∪{0} and ψ : N → N∪{0}, is called a boundary a-pair of the information system U if, for any problem z over U , there exists a decision tree Γ over U which solves the problem z nondeterministically and for which h(Γ) ≤ ϕ(n) and L(Γ) ≤ ψ(n), where n = dim z.A boundary a-pair (ϕ, ψ) of the information system U is called optimal if, for any boundary a-pair (ϕ ′ , ψ ′ ) of U , the inequalities ϕ ′ (n) ≥ ϕ(n) and ψ ′ (n) ≥ ψ(n) hold for any n ∈ N.An information system U is called a-reachable if the pair (h a U , L a U ) is boundary (and, consequently, optimal boundary) a-pair of the system U .For deterministic decision trees, the best situation is when the considered information system is d-reachable, for nondeterministic decision trees -when the information system is a-reachable.
Each information system from the classes W 2 , W 3 , W 4 , W 5 is d-reachable.The class W 1 contains both information systems that are d-reachable and information systems that are not d-reachable.For each information system U that is not d-reachable, we find a nontrivial boundary d-pair which is enough close to the pair (h d U , L d U ).Each information system from the classes W 3 , W 5 is a-reachable.Each information system from the classes W 1 , W 2 , W 4 is not a-reachable.For some information systems U which are not a-reachable, we find nontrivial boundary a-pairs that are enough close to (h a U , L a U ).For the rest of information systems U that are not a-reachable, the pair (n, L a U (n)) is the optimal boundary a-pair.Note that, for such information systems, the function h a U is bounded from above by a constant.
The obtained results are related to time-space trade-off for deterministic and nondeterministic decision trees.Details can be found in the following five theorems.

Theorem 2. (a)
The class W 1 contains both information systems that are d-reachable and information systems that are not d-reachable.Let U be an information system from W 1 which is not d-reachable.Then, for each ε, 0 < ε < 1, there exists a positive constant c such that ) is a boundary d-pair of the system U .
(b) Let U be an information system from the class W 1 .Then the system U is not areachable and, for each ε, 0 < ε < 1, there exist positive constants c 1 , c 2 , and c 3 such that (c 1 , 2 c 2 (log 2 n) 1+ε +c 3 ) is a boundary a-pair of the system U .Theorem 3. Let U be an information system from the class W 2 .Then (a) is the optimal boundary a-pair of the system U .Theorem 4. Let U be an information system from the class W 3 .Then (a) The system U is d-reachable.
(b) The system U is a-reachable.
Theorem 5.The class W 4 contains both information systems that satisfy the condition of restricted coverage and information systems that do not satisfy this condition.Let U be an information system from the class W 4 .Then (a) The system U is d-reachable.
(b) The system U is not a-reachable.If the system U does not satisfy the condition of restricted coverage, then (n, L a U (n)) is the optimal boundary a-pair of the system U .If the system U satisfies the condition of restricted coverage, then there exist positive constants c 1 and c 2 such that (c 1 , c n 2 ) is a boundary a-pair of the system U .Theorem 6.Let U be an information system from the class W 5 .Then (a) The system U is d-reachable.
(b) The system U is a-reachable.
Table 2 summarizes Theorems 1-6.The first column contains name of complexity class.The next four columns describe the type of information systems from this class.The last two columns "d-pairs" and "a-pairs" contain information about boundary d-pairs and boundary a-pairs for information systems from the considered class: "d-reachable" means that all information systems from the class are d-reachable, "a-reachable" means that all information systems from the class are a-reachable, Th. 2 (a), ..., Th. 5 (b) are links to corresponding statements Theorem 2 (a), ..., Theorem 5 (b). 3 Proofs of Propositions 2-4 In this section, we prove a number of auxiliary statements and the three mentioned propositions.
Lemma 1.Let U = (A, F ) be an infinite binary information system.Then (a) If U satisfies the condition of coverage with parameter m, then for any n ∈ N, any (n, U )-set is a union of a finite number of (m, U )-sets.
(b) If U satisfies the condition of restricted coverage with parameters m and t, then for any n ∈ N, any (n, U )-set is a union of at most t n (m, U )-sets.
Proof.(a) Let U satisfy the condition of coverage with parameter m: any (m + 1, U )-set is a union of a finite number of (m, U )-sets.We now show by induction on n that, for any n ∈ N, any (n, U )-set is a union of a finite number of (m, U )-sets.If n ≤ m + 1, then evidently, the considered statement holds.Let for some n, n ≥ m + 1, the considered statement hold.Let us show that it holds for n + 1.We consider an arbitrary (n + 1, U )-set B which is the set of solutions from A of a (n + 1, U )-system of equations Let us consider a (n, U )-system of equations and the set B ′ of solutions from A of this system.Then B ′ is a (n, U )-set and, according to the inductive hypothesis, B ′ is a union of a finite number of (m, U )-sets B 1 , . . ., B k , where for i = 1, . . ., k, the set B i is the set of solutions of an (m, U )-system of equations S i .One can show that the set B is equal to the union of the sets of solutions on A of the systems of equations S i ∪ {f n+1 (x) = δ n+1 }, i = 1, . . ., k.Each of these sets is an (m + 1, U )-set and therefore is a union of a finite number of (m, U )-sets.Thus, B is a union of a finite number of (m, U )-sets.
(b) Let U satisfy the condition of restricted coverage with parameters m and t: any (m + 1, U )-set is a union of at most t (m, U )-sets.We now show by induction on n that, for any n ∈ N, any (n, U )-set is a union of at most t n (m, U )-sets.If n ≤ m + 1, then evidently, the considered statement holds.Let for some n, n ≥ m + 1, the considered statement hold.Let us show that it holds for n + 1.We consider an arbitrary (n + 1, U )-set B which is the set of solutions from A of a (n + 1, U )-system of equations Let us consider a (n, U )-system of equations and the set B ′ of solutions from A of this system.Then B ′ is a (n, U )-set and, according to the inductive hypothesis, B ′ is a union of (m, U )-sets B 1 , . . ., B k , where k ≤ t n and, for i = 1, . . ., k, the set B i is the set of solutions from A of an (m, U )-system of equations S i .One can show that the set B is equal to the union of the sets of solutions on A of the systems of equations S i ∪ {f n+1 (x) = δ n+1 }, i = 1, . . ., k.Each of these sets is an (m + 1, U )set and therefore is a union of at most t (m, U )-sets.Thus, B is a union of at most t n+1 (m, U )-sets.
Proof of Proposition 2. (CON) Let U satisfy the condition of coverage: there exists m ∈ N such that any (m + 1, U )-set is a union of a finite number of (m, U )-sets.From Lemma 1 it follows that, for any n ∈ N, any (n, U )-set is a union of a finite number of (m, U )-sets.
Let z = (ν, f 1 , . . ., f n ) be a problem over U .We now show that h a U (z) ≤ m.Let δ = (δ 1 , . . ., δ n ) be a tuple from {0, 1} n such that the equation system has a solution from A. For each solution a ∈ A of this system, we have z(a) = ν( δ).The set of solutions of S( δ) is a union of a finite number of (m, U )-sets D 1 , . . ., D s .Each of these sets D i is the set of solutions of an (m, U )-system of equations.For the considered (m, U )-system of equations, we construct a complete path ξ i with m working nodes such that A(ξ i ) = D i and the terminal node of ξ i is labeled with the number ν( δ).Denote Σ( δ) = {ξ 1 , . . ., ξ s }.Let Σ = Σ( δ), where the union is considered among all tuples δ ∈ {0, 1} n such that the system of equations S( δ) has a solution from A. We identify initial nodes of all paths from Σ.As a result, we obtain a decision tree over the information system U which solves the problem z nondeterministically and which depth is at most m.Taking into account that z is an arbitrary problem over U , we obtain h a U (n) ≤ m for any n ∈ N. (LIN) Let U do not satisfy the condition of coverage.Assume that there exists m ∈ N such that h a U (m + 1) ≤ m.Let B be an arbitrary (m + 1, U )-set given by an (m + 1, U )-system of equations S. We show that the set B is a union of a finite number of (m, U )-sets.Let Consider the problem z = (ν, f 1 , . . ., f m+1 ) over U such that, for any σ ∈ {0, 1} m+1 , ν(σ) ∈ {1, 2} and ν(σ) = 1 if and only if σ = (δ 1 , . . ., δ m+1 ).
Let Γ be a decision tree which solves the problem z nondeterministically and for which h(Γ) ≤ m.Choose all complete paths ξ in Γ in which the terminal node is labeled with the number 1 and the set A(ξ) is nonempty.Each such path ξ describes a (m, U )-set A(ξ).The number of these paths is finite.The union of sets described by these paths is equal to B. Since B is an arbitrary (m + 1, U )-set, the information system U satisfies the condition of coverage with parameter m, but this is impossible.Thus, h a U (n) = n for any n ∈ N.
We now consider some auxiliary statements related to the space complexity of decision trees.Let Γ be a decision tree over an information system U = (A, F ) and d be an edge of Γ entering a node w.We denote by Γ(d) a subtree of Γ which root is the node w.We say that a complete path ξ of Γ is realizable if A(ξ) = ∅.Lemma 2. Let U = (A, F ) be an infinite binary information system, z = (ν, f 1 , . . ., f n ) be a problem over U , and Γ be a decision tree over U which solves the problem z deterministically and for which L(Γ) = L d U (z).Then (a) Each working node of Γ has two edges leaving this node.
(b) For each node of Γ, there exists a realizable complete path that passes through this node.
Proof.(a) It is clear that there exists at least one realizable complete path that passes through the root of Γ.Let us assume that w be a node of Γ different from the root and such that there is no a realizable complete path which passes through w.Let d be an edge entering the node w.We remove from Γ the edge d and the subtree Γ(d).As a result, we obtain a decision tree Γ ′ which solves z deterministically and for which L(Γ ′ ) < L(Γ) but this is impossible.
(b) Let us assume that in Γ there exists a working node w which has only one leaving edge d entering a node w 1 .We remove from Γ the node w and the edge d and connect the edge entering the node w to the node w 1 .As a result, we obtain a decision tree Γ ′ which solves the problem z deterministically and for which L(Γ ′ ) < L(Γ) but this is impossible.
Let U be an infinite binary information system, Γ be a decision tree over U , and d be an edge of Γ.The subtree Γ(d) is called full if there exist edges d 1 , . . ., d m in Γ(d) such that the removal of these edges and subtrees Γ(d 1 ), . . ., Γ(d m ) transforms the subtree Γ(d) into a tree G such that each terminal node of G is a terminal node of Γ, and exactly two edges labeled with the numbers 0 and 1 respectively leave each working node of G. Lemma 3. Let U = (A, F ) be an infinite binary information system, z = (ν, f 1 , . . ., f n ) be a problem over U , and Γ be a decision tree over U which solves the problem z nondeterministically and for which L(Γ) = L a U (z).Then (a) For each node of Γ, there exists a realizable complete path that passes through this node.
( Proof.(a) It is clear that there exists at least one realizable complete path that passes through the root of Γ.Let us assume that w be a node of Γ different from the root and such that there is no a realizable complete path which passes through w.Let d be an edge entering the node w.We remove from Γ the edge d and the subtree Γ(d).As a result, we obtain a decision tree Γ ′ which solves the problem z nondeterministically and for which L(Γ ′ ) < L(Γ) but this is impossible.
(b) Let w be a working node of Γ which has m leaving edges d 1 , . . ., d m labeled with the same number, m ≥ 2, and at least one of the subtrees Γ(d 1 ), . . ., Γ(d m ) is full.For the definiteness, we assume that Γ(d 1 ) is full.Remove from Γ the edges d 2 , . . ., d m and subtrees Γ(d 2 ), . . ., Γ(d m ).We now show that the obtained tree Γ ′ solves the problem z nondeterministically. Assume the contrary.Then there exists an object a ∈ A such that, for each complete path ξ with a ∈ A(ξ), the path ξ passes through one of the edges d 2 , . . ., d m but it is not true.Let ξ be a complete path such that a ∈ A(ξ).Then, according to the assumption, this path passes through the node w.Let ξ ′ be the part of this path from the root of Γ to the node w.Since the edges d 1 , . . ., d m are labeled with the same number and Γ(d 1 ) is a full subtree, we can find in Γ(d 1 ) the continuation of ξ ′ to a terminal node of Γ(d 1 ) such that the obtained complete path ξ ′′ of Γ satisfies the condition a ∈ A(ξ ′′ ).Hence Γ ′ is a decision tree which solves the problem z nondeterministically and for which L(Γ ′ ) < L(Γ) but this is impossible.
(c) The part (c) of the statement can be proven in the same way as the part (b).
We now prove a number of statements about classes of decision trees.Let Γ be a decision tree.We denote by L t (Γ) the number of terminal nodes in Γ and by L w (Γ) -the number of working nodes in Γ.It is clear that Let U be an infinite binary information systems.We denote by G d (U ) the set of all deterministic decision trees over U and by G 2 d (U ) -the set of all decision trees from G d (U ) such that each working node of the tree has two leaving edges.
Lemma 4. Let U be an infinite binary information system.Then (a) It is clear that there exists a node w of the tree Γ such that all children of w are terminal nodes.Remove children of w and edges entering these children and attach to w a number from N. We denote the obtained tree by Γ ′ .It is clear that and there be m ≥ 1 working nodes in Γ each of which has exactly one leaving edge.We add m new terminal nodes to Γ and, as a result, obtain a tree We denote by G f a (U ) the set of all decision trees Γ over U that satisfy the following conditions: (i) if a working node of Γ has m leaving edges d 1 , . . ., d m labeled with the same number and m ≥ 2, then the subtrees Γ(d 1 ), . . ., Γ(d m ) are not full, and (ii) if the root of Γ has m leaving edges d 1 , . . ., d m and m ≥ 2, then the subtrees Γ(d 1 ), . . ., Γ(d m ) are not full.One can show that Lemma 5. Let U be an infinite binary information system.
Proof.We prove the considered statement by induction on Let, for some m ≥ 1 for each decision tree Then the tree Γ contains a node v (the root or a working node) which has two leaving edges labeled with the same number (if v is a working node) or do not labeled with numbers (if v is the root).We call such edges equally labeled.Since Γ is a finite tree, there is a node w of Γ which has two leaving edges d 1 and d 2 that are equally labeled, and in the subtrees Γ(d 1 ) and Γ(d 2 ) there are no nodes with two leaving edges that are equally labeled.
It is clear that the subtree Γ(d 1 ) is not full.We add to this tree a node and an edge leaving this node and entering the root of Γ(d 1 ).As a result, we obtain a decision tree from G d (U ) \ G 2 d (U ).Using Lemma 4 we obtain L w (Γ(d 1 )) > L t (Γ(d 1 )) − 1. Remove from Γ the edge d 1 and the subtree Γ(d 1 ).Denote by Γ ′ the obtained tree.Since the subtree Γ( Let U = (A, F ) be an infinite binary information system.For f 1 , . . ., f n ∈ F we denote by N U (f 1 , . . ., f n ) the number of n-tuples (δ 1 , . . ., δ n ) ∈ {0, 1} n for which the system of equations Proposition 5. Let U = (A, F ) be an infinite binary information system.Then, for any n ∈ N, Proof.Let z = (ν, f 1 , . . ., f m ) be a problem over U and m ≤ n.Let Γ be a decision tree over U which solves the problem z deterministically, uses only attributes from the set {f 1 , . . ., f m }, and has minimum number of nodes among such decision trees.In the same way as in the proof of Lemma 2, one can prove that each working node of Γ has two edges leaving this node and, for each node of Γ, there exists a realizable complete path that passes through this node.Let ξ 1 and ξ 2 be different complete paths in Γ, a 1 ∈ A(ξ 1 ), and Taking into account that z is an arbitrary problem over U with dim z ≤ n we obtain Since any decision tree solving the problem z deterministically solves it nondeterministically we obtain and, for any δ1 , δ2 ∈ {0, 1} n , if δ1 = δ2 , then ν( δ1 ) = ν( δ2 ).Let Γ be a decision tree over U which solves the problem z nondeterministically and for which L(Γ) = L a U (z).By Lemma 3, Γ ∈ G f a (U ).Using Lemmas 4 and 5 we obtain The next statement follows directly from Lemmas 5.1 and 5.2 [23] and evident inequality N U (n) ≤ 2 n which is true for any infinite binary information system U .The proof of Lemma 5.1 from [23] is based on Theorems 4.6 and 4.7 from the same monograph that are similar to results obtained in [29,30].Proposition 6.For any infinite binary information system U , the function N U (n) has one of the following two types of behavior: (POL) If the system U has finite I-dimension, then for any n ∈ N, (EXP) If the system U has infinite I-dimension, then for any n ∈ N, We now prove Propositions 3 and 4.
Proof of Proposition 3. The statement of the proposition follows immediately from Propositions 5 and 6.
Proof of Proposition 4. The statement of the proposition follows immediately from Proposition 5.

Proof of Theorem 1
First, we prove six auxiliary statements.
Lemma 6.For any infinite binary information system, its type coincides with one of the rows of Table 1.
Proof.To prove this statement we fill Table 3.In the first column "Cover.",we have either "Yes" or "No": "Yes" if the considered information system satisfies the condition of coverage and "No" otherwise.In the second column "Restr.cover.",we also have either "Yes" or "No": "Yes" if the considered information system satisfies the condition of restricted coverage and "No" otherwise.In the third column "I-dim."we have either "Fin" or "Inf": "Fin" if the considered information system has finite I-dimension and "Inf" if the considered information system has infinite I-dimension.
If an information system does not satisfy the condition of coverage, then this information system does not satisfy the condition of restricted coverage.It means that there are only six possible tuples of values of the considered three parameters of information systems which correspond to the six rows of Table 3.The values of the considered three parameters define the types of behavior of functions , and L a U (n) according to Propositions 1-4.We see that the set of possible tuples of values in the last four columns coincides with the set of rows of Table 1.For each row of Table 1, we consider an example of infinite binary information system which type coincides with this row.
For any i ∈ N, we define two functions p i : N → {0, 1} and l i : N → {0, 1}.Let j ∈ N. Then p i (j) = 1 if and only if j = i, and l i (j) = 1 if and only if j > i.
Define an information system U 1 = (A 1 , F 1 ) as follows: Lemma 7. The information system U 1 belongs to the class , and L a U 1 (n) = 2(n + 1) for any n ∈ N.This information system satisfies the condition of coverage with parameter 3, satisfies the condition of restricted coverage with parameters 3 and 1, and has finite I-dimension equals to 1.The information system U 1 is d-reachable.
Proof.It is easy to show that N U 1 (n) = n + 1 for any n ∈ N. Using Proposition 5 we obtain Therefore each decision tree Γ over U 1 that solves the problem z deterministically has at least n + 1 terminal nodes.One can show that the number of terminal nodes in Γ is at most 2 h(Γ) .Hence n + 1 ≤ 2 h(Γ) and log ≤ m (the construction of an appropriate decision tree is based on an analog of binary search, and we use only attributes from the problem description) and Let n ≥ 2 and z = (ν, f 1 , . . ., f n ) be an arbitrary problem over U 1 and l i 1 , . . ., l im be all pairwise different attributes from the set {f 1 , . . ., f n } ordered such that i 1 < . . .< i m .Then these attributes divide the set N into m + 1 nonempty domains that are sets of solutions on N of the following systems of equations: is constant in each of the considered domains.Using these facts it is easy to show that there exists a decision tree Γ over U 1 which solves the problem z nondeterministically and for which h(Γ) = 2 if m ≥ 2. Therefore h a U 1 (n) ≤ 2. One can show that there exists a problem z over U 1 such that dim z = n and h a U 1 (z) ≥ 2. Therefore h a U 1 (n) = 2. Since the function h d U 1 has the type of behavior LOG, the information system U 1 belongs to the class W 1 -see Table 1.One can show that the information system U 1 satisfies the condition of coverage with parameter 3, satisfies the condition of restricted coverage with parameters 3 and 1, and has finite I-dimension equals to 1.
Let z = (ν, f 1 , . . ., f n ) be a problem over U 1 .We know that there exists a decision tree Γ over U 1 which solves this problem deterministically, uses only attributes from the set {f 1 , . . ., f n }, and which depth is at most ⌈log 2 (n + 1)⌉.By removal of some nodes and edges from Γ we can obtain a decision tree Γ ′ over U 1 which solves the problem z deterministically, and in which each working node has exactly two leaving edges and each complete path is realizable.It is clear that , and L a U 2 (n) = 2(n + 1) for any n ∈ N.This information system satisfies the condition of coverage with parameter 2, does not satisfy the condition of the restricted coverage, and has finite I-dimension equals to 1.
Proof.It is easy to show that N U 2 (n) = n + 1 for any n ∈ N. Using Proposition 5 we obtain Consider a problem z = (ν, p 2 t +1 , . . ., p 2 t +n ) over U 2 such that, for each δ1 , δ2 ∈ {0, 1} n with δ1 = δ2 , ν( δ1 ) = ν( δ2 ).Consider an arbitrary decision tree Γ over U 2 that solves the problem z deterministically and a complete path ξ of Γ such that 2 t + n + 1 ∈ A 2 (ξ).One can show that if the number of working nodes in ξ is less than n, then the function z(x) is not constant on the set A 2 (ξ) but this is impossible.Therefore h(Γ) ≥ n and . ., f n ) be an arbitrary problem over U 2 .Each attribute from the set {f 1 , . . ., f n } is of the kind p i or l i .We say about the number i as about the index of the considered attribute.Let j be the maximum index of an attribute from the set {f 1 , . . ., f n } and t be a number from N such that 2 t > j.Then the function z(x) is constant on the sets of solutions of equation systems {p 1 (x) = 1}, . . ., {p 2 t (x) = 1}, {l 2 t (x) = 1} on A 2 , and the union of these sets of solutions is equal to A 2 .Using these facts it is easy to show that there exists a decision tree Γ over U 2 which solves the problem z nondeterministically and for which Since the function h d U 2 has the type of behavior LIN, the function h a U 2 has the type of behavior CON, and the functions L d U 2 and L a U 2 have the type of behavior POL the information system U 2 belongs to the class W 2 -see Table 1.One can show that the information system U 2 satisfies the condition of coverage with parameter 2 and has finite I-dimension equals to 1. Using Proposition 1 we obtain that this information system does not satisfy the condition of restricted coverage.Define an information system U 3 = (A 3 , F 3 ) as follows: A 3 = N and F 3 = {p i : i ∈ N}.Lemma 9.The information system U 3 belongs to the class W 3 , , and L a U 3 (n) = 2(n + 1) for any n ∈ N.This information system does not satisfy the conditions of coverage and restricted coverage, and has finite I-dimension equals to 1.
Proof.It is easy to show that N U 3 (n) = n + 1 for any n ∈ N. Using Proposition 5 we obtain Consider a problem z = (ν, p 1 , . . ., p n ) over U 3 such that, for each δ1 , δ2 ∈ {0, 1} n with δ1 = δ2 , ν( δ1 ) = ν( δ2 ).Consider an arbitrary decision tree Γ 1 over U 3 that solves the problem z deterministically and a complete path ξ in Γ 1 in which each edge leaving a working node is labeled with the number 0. One can show that if the number of working nodes in ξ is less than n, then the function z(x) is not constant on the set A 3 (ξ) but this is impossible.Therefore h(Γ 1 ) ≥ n and Consider an arbitrary decision tree Γ 2 over U 3 that solves the problem z nondeterministically. Let ξ 1 be a complete path of Γ 2 in which at least one edge leaving a working node is labeled with the number 1. Then the set A 3 (ξ 1 ) contains at most one element from A 3 .The set A 3 is infinite, the number of complete paths in Γ 2 is finite, and the union of the sets A 3 (ξ) for all complete paths ξ in Γ is equal to A 3 .Therefore there exists a complete path ξ 0 in Γ 2 in which each edge leaving a working node is labeled with the number 0. One can show that if the number of working nodes in ξ 0 is less than n, then the function z(x) is not constant on the set A 3 (ξ 0 ) but this is impossible.Therefore Since the functions h d U 3 and h a U 3 have the type of behavior LIN and the functions L d U 3 and L a U 3 have the type of behavior POL, the information system U 3 belongs to the class W 3see Table 1.One can show that this information system has finite I-dimension equals to 1. Using Propositions 1 and 2 we obtain that the information system U 3 does not satisfy the conditions of coverage and restricted coverage.Define an information system U 4 = (A 4 , F 4 ) as follows: A 4 = N and F 4 is the set of all functions from N to {0, 1}.
Lemma 10.The information system U 4 belongs to the class , and L a U 4 (n) = 2 n+1 for any n ∈ N.This information system satisfies the condition of coverage with parameter 2, satisfies the condition of restricted coverage with parameters 2 and 1, and has infinite I-dimension.
Proof.It is easy to show that the information system U 4 has infinite I-dimension.By Proposition 6, N U 4 (n) = 2 n for any n ∈ N. Using Proposition 5 we obtain for any n ∈ N. Let n ∈ N. By Proposition 1, h d U 4 (n) = n.It is easy to show that h a U 4 (n) ≥ 1.We now show that h a U 4 (n) ≤ 1.Let z = (ν, f 1 , . . ., f n ) be an arbitrary problem over U 4 .Let (δ 1 , . . ., δ n ) ∈ {0, 1} n and the set B of solutions from A 4 of the equation system {f 1 (x) = δ 1 , . . ., f n (x) = δ n } is nonempty.Then there exists a function f ∈ F 4 such that the set B is the set of solutions from A 4 of the equation system {f (x) = 1}.Using this fact it is easy to show that there exists a decision tree Γ over U 4 which solves the problem z nondeterministically and for which h(Γ) = 1.Therefore h a U 4 (n) ≤ 1. Hence h a U 4 (n) = 1.Since the function h d U 4 has the type of behavior LIN, the function h a U 4 has the type of behavior CON, and the functions L d U 4 and L a U 4 have the type of behavior EXP, the information system U 4 belongs to the class W 4 -see Table 1.One can show that this information system satisfies the condition of coverage with parameter 2 and satisfies the condition of restricted coverage with parameters 2 and 1. Define an information system U 5 = (A 5 , F 5 ) as follows: A 5 is the set of all infinite sequences a 1 , a 2 , . .., where a i ∈ {0, 1} for any i ∈ N, and F 5 = {f i : i ∈ N}, where f i (a 1 , a 2 , . ..) = a i for any a 1 , a 2 , . . .∈ A 5 and i ∈ N.
Lemma 11.The information system U 5 belongs to the class W 5 , , and L a U 5 (n) = 2 n+1 for any n ∈ N.This information system does not satisfy the condition of coverage and the condition of restricted coverage, and has infinite I-dimension.
Proof.Let n ∈ N. One can show that the equation system {f 1 (x) = δ 1 , . . ., f n (x) = δ n } has a solution from the set A 5 for any (δ 1 , . . ., δ n ) ∈ {0, 1} n .Therefore the information system U 5 has infinite I-dimension.By Proposition 6, N U 5 (n) = 2 n for any n ∈ N. Using Proposition 5 we obtain L d U 5 (n) = L a U 5 (n) = 2 n+1 for any n ∈ N. Let n ∈ N. By Proposition 1, h d U 5 (n) = n.Consider a problem z = (ν, f 1 , . . ., f n ) over U 4 such that, for each δ1 , δ2 ∈ {0, 1} n with δ1 = δ2 , ν( δ1 ) = ν( δ2 ).Let Γ be an arbitrary decision tree over U 5 that solves the problem z nondeterministically and ξ be an arbitrary complete path of Γ.We denote by G the set of attributes attached to working nodes of ξ.One can show that, if {f 1 , . . ., f n } is not a subset of the set G, then the function z(x) is not constant on the set A 5 (ξ) but this is impossible.Therefore h(Γ) ≥ n and h a U 5 (n) ≥ n.It is clear that h a U 5 (n) ≤ n.Thus, h a U 5 (n) = n.Since the functions h d U 5 and h a U 5 have the type of behavior LIN and the functions L d U 5 and L a U 5 have the type of behavior EXP, the information system U 5 belongs to the class W 5 -see Table 1.By Proposition 2 this information system does not satisfy the condition of coverage and therefore does not satisfy the condition of restricted coverage.
Proof of Theorem 1.The statements of the theorem follow from Lemmas 6-11.

Proofs of Theorems 2-6
First, we prove seven auxiliary statements.
Lemma 12. Let U be an infinite binary information system such that h d U (n) = n for any n ∈ N. Then the information system U is d-reachable.
Proof.Let z = (ν, f 1 , . . ., f n ) be a problem over U and Γ be a decision tree that solves the problem z deterministically and satisfies the following conditions: the number of working nodes in each complete path of Γ is equal to n and these nodes in the order from the root to a terminal node are labeled with attributes f 1 , . . ., f n .Remove from Γ all nodes and edges that do not belong to realizable complete paths.Let w be a working node in the obtained tree that has only one leaving edge d entering a node v.We remove the node w and edge d and connect the edge entering w to the node v.We do the same with all working nodes with only one leaving edge.Denote by Γ ′ the obtained decision tree.It is clear that Γ ′ solves the problem z deterministically, Γ ′ ∈ G 2 d (U ), and Lemma 13.Let U be an infinite binary information system such that h a U (n) = n for any n ∈ N. Then the information system U is a-reachable.
Proof.Let z = (ν, f 1 , . . ., f n ) be a problem over U .Construct for this problem the decision tree Γ ′ as in the proof of Lemma 12.It is clear that Γ ′ solves the problem z nondeterministically.We know that L(Γ ′ ) ≤ 2N U (n).Using Proposition 5 we obtain Lemma 14.Let U be an infinite binary information system which satisfies the condition of coverage.Then the information system U is not a-reachable.
Proof.By Proposition 2, the function h a U (n) is bounded from above by a positive constant c.By Proposition 6, the function N U (n) is not bounded from above by a constant.Choose n ∈ N such that N U (n) > 2 2c .Let z = (ν, f 1 , . . ., f n ) be a problem over U such that ν( δ1 ) = ν( δ2 ) for any δ1 , δ2 ∈ {0, 1} n , δ1 = δ2 , and N U (f 1 , . . ., f n ) = N U (n).Let Γ be a decision tree over U which solves the problem z nondeterministically, for which h(Γ) ≤ h a U (n) ≤ c, and which has the minimum number of nodes among such trees.In the same way as it was done in the proof of Proposition 3, we can prove that Γ ∈ G f a (U ).It is clear that Lemma 15.Let U be an infinite binary information system which does not satisfy the condition of restricted coverage.Then (n, L a U (n)) is the optimal boundary a-pair of the information system U .
Proof.The proof of the fact that (n, L a U (n)) is a boundary a-pair of the system U is similar to the proof of Lemma 13.
We now prove that this is the optimal boundary a-pair of the system U .Let (q, r) be a boundary a-pair of the system U .It is clear that r(n) ≥ L a U (n) for any n ∈ N. Let us show that q(n) ≥ n for any n ∈ N. Assume the contrary: there exists m ∈ N such that q(m + 1) ≤ m.Let us consider an arbitrary (m + 1, U )-set B given by (m + 1, U )-system of equations {f 1 (x) = δ 1 , . . ., f m+1 (x) = δ m+1 }.
Let R + be the set of nonnegative real numbers.Define an infinite binary information system U 6 = (A 6 , F 6 ) as follows: A 6 = R + and F 6 = {p i : i ∈ N} ∪ {q i : i ∈ N}, where, for any a ∈ A 6 , p i (a) = 1 if and only if a = i, and q i (a) = 1 if and only if a ≥ i + 1  2 .Lemma 16.The information system U 6 belongs to the class W 1 .This information system is not d-reachable.
Proof.It is easy to show that N U 6 (n) = n + 1 for any n ∈ N. Using Proposition 5 we obtain L d U 6 (n) = 2(n + 1) for any n ∈ N. We now show that h d U 6 (n) ≤ ⌈log 2 (n + 1)⌉ + 1 for any n ∈ N. Let z = (ν, f 1 , . . ., f n ) be an arbitrary problem over U 6 .We describe a decision tree Γ over U 6 that solves the problem z deterministically.This tree will use attributes from a set G containing all attributes f 1 , . . ., f n and, probably, some additional attributes from the set {q i : i ∈ N}.Initially, G = {f 1 , . . ., f n }.Let p i 1 , . . ., p im be all attributes from the set {p i : i ∈ N} ∩ {f 1 , . . ., f n } ordered such that i 1 < . . .< i m .For t = 1, . . ., m − 1, if the set G does not contain any attribute q j such that i t ≤ j < i t+1 , then we add to G the attribute q it .As a result, we obtain a set of attributes G that contains at most n attributes from the set {q i : i ∈ N}.Let there are attributes q j 1 , . . ., q j k , k ≤ n.It is easy to construct a decision tree Γ ′ that finds values of these attributes on a given element a ∈ A 6 and has depth at most ⌈log 2 (n + 1)⌉.If we know the values of all attributes q j 1 , . . ., q j k , then we know values of all attributes p i 1 , . . ., p im on a with the exception of at most one attribute.If we compute the value of this attribute we will know the values of all attributes f 1 , . . ., f n on a and the value z(a).So we can transform the decision tree Γ ′ into a deterministic decision tree Γ over U 6 which solves the problem z and which depth is at most ⌈log 2 (n + 1)⌉ + 1. Therefore h d U 6 (n) ≤ ⌈log 2 (n + 1)⌉ + 1.Since the function h d U 6 has the type of behavior LOG, the information system U 6 belongs to the class W 1 -see Table 1.
Let Z be the set of integers and Z − = Z \ N. Define an information system U 7 = (A 7 , F 7 ) as follows: A 7 = Z and F 7 = {p i : i ∈ N} ∪ {l 2 i : i ∈ N} ∪ F 0 .The functions p i and l 2 i have value 0 on the set Z − , and F 0 is the set of all functions from Z to {0, 1} that are equal to 0 on the set N.
Lemma 17.The information system U 7 belongs to the class W 4 and does not satisfy the condition of restricted coverage.
Proof.Let n ∈ N. One can show that there are attributes g 1 , . . ., g n ∈ F 0 such that N U 7 (g 1 , . . ., g n ) = 2 n .Therefore the information system U 7 has infinite I-dimension.Using Proposition 3 we obtain that the function L d U 7 has the type of behavior EXP.We now show that h a U 7 (n) ≤ 1.Let z = (ν, f 1 , . . ., f n ) be an arbitrary problem over U 7 .The attributes f 1 , . . ., f n divide the set A 7 into finite number of nonempty domains in each of which these attributes have fixed values.One can show that each domain can be represented as a union of finite number of subdomains such that each subdomain is the set of solutions on A 7 of equation system of the kind {f (x) = 1}, where f ∈ F 0 , or {p i (x) = 1}, or {l 2 i (x) = 1}.Using these facts it is easy to show that there exists a decision tree Γ over U 7 which solves the problem z nondeterministically and for which h(Γ) = 1.Therefore h a U 7 (n) ≤ 1. Hence the function h a U 7 has the type of behavior CON.Taking into account that the function L d U has the type of behavior EXP, we obtain that the information system U 7 belongs to the class W 4 -see Table 1.
Let us assume that the information system U 7 satisfies the condition of restricted coverage with parameters m and t.Let r be a natural number such that t < 2 r − m + 1.Let B be the set of solutions on A 7 of the equation system {l 2 r (x) = 1, p 2 r +1 (x) = 0, . . ., p 2 r +m−1 (x) = 0, l 2 r+1 (x) = 0}.
It is clear that B = {2 r + m, 2 r + m + 1, . . ., 2 r+1 } and |B| = 2 r − m + 1.Let us assume that B is a union of at most t (U 7 , m)-sets.Let C be a (U 7 , m)-set such that C ⊆ B. One can show that |C| = 1.Therefore t ≥ 2 r − m + 1 but this is impossible.Therefore the information system U 7 does not satisfy the condition of restricted coverage.Lemma 18.Let U = (A, F ) be an information system from the class W 4 which satisfies the condition of restricted coverage.Then there exist positive constants c 1 and c 2 such that (c 1 , c n 2 ) is a boundary a-pair of the system U .Proof.Let U satisfy the condition of restricted coverage with parameters m and t: any (m + 1, U )-set is a union of at most t (m, U )-sets.From Lemma 1 it follows that, for any n ∈ N, any (n, U )-set is a union of at most t n (m, U )-sets.
Let z = (ν, f 1 , . . ., f n ) be a problem over U .We now show that there exists a decision tree Γ over U which solves the problem z nondeterministically and for which h(Γ) ≤ m and L(Γ) ≤ (m + 2)2 n t n .Let δ = (δ 1 , . . ., δ n ) be a tuple from {0, 1} n such that the equation system S( δ) = {f 1 (x) = δ 1 , . . ., f n (x) = δ n } has a solution from A. For each solution a ∈ A of this system, we have z(a) = ν( δ).The set of solutions of S( δ) is a union of (m, U )-sets D 1 , . . ., D s , where s ≤ t n .Each of these sets D i is the set of solutions of an (m, U )-system of equations.For the considered (m, U )-system of equations, we construct a complete path ξ i with m working nodes such that A(ξ i ) = D i and the terminal node of ξ i is labeled with the number ν( δ).Denote Σ( δ) = {ξ 1 , . . ., ξ s }.
Let Σ = Σ( δ), where the union is considered among all tuples δ ∈ {0, 1} n such that the set of equations S( δ) has a solution from A (the number of such tuples is at most 2 n ).We identify initial nodes of all paths from Σ.As a result, we obtain a decision tree Γ over the information system U which solves the problem z nondeterministically and for which h(Γ) ≤ m and L(Γ) ≤ (m + 2)2 n t n .Denote c 1 = m and c 2 = (m + 2)2t.Then h(Γ) ≤ c 1 and L(Γ) ≤ c n 2 .Taking into account that z is an arbitrary problem over U with dim z = n, we obtain that (c 1 , c n 2 ) is a boundary a-pair of the system U .
Proof of Theorem 2. Each information system from the class W 1 satisfies the condition of coverage and the condition of restricted coverage, and has finite I-dimension (see Table 3).
(a) The existence of both d-reachable and not d-reachable information systems in the class W 1 follows from Lemmas 7 and 16.Let U be an information system from W 1 which is not d-reachable.Then U satisfies the condition of restricted coverage and has finite I-dimension.From here and from Theorem 2.1 [21] it follows that, for each ε, 0 < ε < 1, there exists a positive constant c such that (c(log 2 n) 1+ε + 1, 2 c(log 2 n) 1+ε +2 ) is a boundary d-pair of the system U .
(b) Let U = (A, F ) be an information system from the class W 1 .Then U satisfies the condition of coverage.Using Lemma 14 we obtain that the system U is not a-reachable.We know that the system U satisfies the condition of restricted coverage and has finite Idimension.Using Theorem 2.2 [21] we obtain that, for each ε, 0 < ε < 1, there exist positive constants c 1 , c 2 such that, for each problem z over U , there exists a system ∆ of decision rules of the kind (f 1 (x) = δ 1 ) ∧ . . .∧ (f m (x) = δ m ) → z(x) = σ, where f 1 , . . ., f m ∈ F , δ 1 , . . ., δ m ∈ {0, 1}, and σ ∈ N, that satisfies the following conditions: (i) each rule is true for the problem z, (ii) for each a ∈ A, there exists a rule from ∆ that accepts a, (iii) the number of conditions in the left-hand side of each rule is at most c 1 , and (iv) the number of riles in ∆ is at most 2 c 2 (log 2 n) 1+ε +1 where n = dim z.One can transform the system of decision rules ∆ into a decision tree Γ over U which solves the problem z nondeterministically and for which h(Γ) ≤ c 1 and L(Γ) ≤ (c 1 + 2)2 c 2 (log 2 n) 1+ε +1 = 2 c 2 (log 2 n) 1+ε +c 3 , where c 3 = log 2 (c 1 +2)+1.Note that complete paths in Γ correspond to rules from the system ∆.Thus, (c 1 , 2 c 2 (log 2 n) 1+ε +c 3 ) is a boundary a-pair of the system U .
Proof of Theorem 3.Each information system from the class W 2 satisfies the condition of coverage, does not satisfy the condition of restricted coverage, and has finite I-dimension (see Table 3).
b) If a working node w of Γ has m leaving edges d 1 , . . ., d m labeled with the same number and m ≥ 2, then the subtrees Γ(d 1 ), . . ., Γ(d m ) are not full.(c) If the root r of Γ has m leaving edges d 1 , . . ., d m and m ≥ 2, then the subtrees Γ(d 1 ), . . ., Γ(d m ) are not full.

Table 1 :
Possible types of infinite binary information systems

Table 3 :
Parameters and types of infinite binary information systems