Classification Through Data Mining Algorithm

Abstract

Classification is a data mining technique in the machine learning domain. Various algorithms such as K-nearest neighbor, support vector machines, random forest, logistic regression, and decision trees are used to solve the classification problem. Out of them, logistic regression and decision trees are perhaps the most used classification techniques. The study seeks to compare the performance of these two techniques in classifying observations from two different data sets. This study aims to identify cases where one would prefer a particular algorithm over the other and explore the advantages and disadvantages associated with these two algorithms. After a data discovery phase, which included checking for missing values and removing redundant predictors, logistic regression and decision tree models were built for both datasets. The models were compared based on their ROC curves, and their predictive ability was obtained from their confusion matrices.

Appendix

See Table 1.

Table 1 Complexity parameter (CP) table showing the cross-validation error (xerror) and the size of a tree obtained corresponding to different CP values with respect to the deep/parent tree

Fig. 4

Distribution of the edibility of a mushroom with respect to its cap color and cap surface

Fig. 5

Distribution of the edibility of a mushroom with respect to its cap shape and cap color

Fig. 6

Distribution of the edibility of a mushroom with respect to its odor

Fig. 7

Structure of the hotel dataset

Fig. 8

Distribution of hotel stays throughout the year

Fig. 9

Distribution of hotel stays with respect to average daily rate of a hotel

Fig. 10

Decision tree obtained for the mushroom dataset

Fig. 11

Confusion matrix obtained after applying the decision tree on the training dataset

Fig. 12

Confusion matrix obtained after applying the decision tree on the testing dataset

Fig. 13

Unpruned tree obtained with CP = 0.00001 when applied on the training data of the mushroom dataset

Fig. 14

Pruned tree obtained pruning the unpruned tree at a CP value of 0.0020747

Fig. 15

Confusion matrix obtained after applying the pruned tree on the training set of the mushroom data set

Fig. 16

Confusion matrix obtained after applying the pruned tree on the testing set of the mushroom data set

Fig. 17

ROC curve obtained showing the performance of the decision tree model on the testing set of the mushroom dataset

Fig. 18

Confusion matrix obtained after applying the logistic regression model on the testing set of the hotel dataset

Fig. 19

Decision tree obtained for the hotel dataset

Fig. 20

Confusion matrix obtained on applying the decision tree model on the training set of the hotel dataset

Fig. 21

Confusion matrix obtained on applying the decision tree model on the testing set of the hotel dataset

Fig. 22

Confusion matrix obtained on applying the decision tree which is trained on the imbalanced training set on the testing set of the hotel dataset

Fig. 23

ROC curve showing the performance of logistic regression and decision tree on the testing set of the hotel dataset