Research and Application on Ensemble Learning Methods

Abstract

As shown in previous data, diabetes has led to the increasing mortality and considerable financial expenditure in the US. It is necessary to find out how to making correct diagnosis and prescription of diabetes plays an important role in helping patients. That is why we choose the dataset of diabetic inpatients having diagnosis at hospitals in the US, and predict how different treatments and medications influence patient outcomes. We use the class attribute of readmission number to obtain the results.

Because of the large and biased dataset, we firstly remove attributes with high missing value rate, and reduce the imbalance classes of instances by over-sampling and under-sampling, then followed by the attribute selection through various methods, such as the Correlation-based feature selection, the Chi-Squared Attribute Evaluator, the Information Gain Attribute Evaluator, etc.

Three classification methods C4.5, RIPPER, and Random Forests are used to predict the classification in Weka. In addition, we also use the ensemble learning methods including bagging and boosting to improve the stability and accuracy. From the analysing results, we can see that C4.5 and Ripper perform better, and both bagging and boosting increase the accuracy rate to differing degrees because both algorithms are somewhat unstable. There is no doubt that Random Forests is the best performer among all classification methods we use, and after using boosting, we see big increases in the values of the evaluation metrics we use. The final outcome is much better than random guess.