Supervised Learning

Abstract

In the previous chapters, we considered algorithms for construction of classifiers—decision trees and decision rule systems for a given decision table T. If T contains complete information (we know all possible tuples of values of attributes, and these tuples are rows of T) then depending on our aims we should construct either exact or approximate classifiers. In the last case, we can control the accuracy of approximate classifiers.

If T contains incomplete information (we do not know all possible tuples of values of attributes and corresponding decisions) then we have essentially more complicated problem known as supervised learning. For a given decision table T with conditional attributes f₁,…,f _n and the decision attribute d, we should construct a classifier which will predict values of the decision attribute for tuples of values of conditional attributes which, possible, are not rows of the table T. In this case, exact classifiers can be overfitted, i.e., have a good accuracy for T and a bad one for tuples of values of attributes that are not rows of T.

The usual way in this situation is to divide initial table T into three subtables: training subtable T₁, validation subtable T₂ and test subtable T₃. The subtable T₁ is used for construction of initial classifier. The subtable T₂ is used for pruning of this classifier: we step by step decrease the accuracy of the classifier relative to T₁ by removal of its parts (nodes of decision tree or conditions from the left-hand side of decision rules), and stop when the accuracy of obtained classifier relative to T₂ will be maximum. The subtable T₃ is used to evaluate the accuracy of classifier obtained after pruning. If the accuracy is enough good we can use this classifier to predict decisions for tuples of values of attributes that are not rows of T.

In this chapter, we consider three known approaches to the supervised learning problem: based on decision trees (see, for example, [8, 71]), based on decision rule systems (see, for example, [73]) and so-called lazy learning algorithms (we omit the construction of classifier and work directly with input tuple of attribute values and decision table T [1, 20]).

This chapter contains four sections. In Sect. 7.1, we consider classifiers based on decision trees. In Sect. 7.2, we study classifiers based on decision rules. Section 7.3 is devoted to the consideration of lazy learning algorithms. Section 7.4 contains conclusions.