Abstract
In biomedical studies, accuracy of classification algorithms used in disease diagnosis systems is certainly an important task and the accuracy of system is strictly related to extraction of discriminatory features from data. In this paper, we propose a new multi-class feature selection method based on Rotation Forest meta-learner algorithm. The feature selection performance of this newly proposed ensemble approach is tested on Erythemato-Squamous diseases dataset. The discrimination ability of selected features is evaluated by the use of several machine learning algorithms. In order to evaluate the performance of Rotation Forest Ensemble Feature Selection approach quantitatively, we also used various and widely utilized ensemble algorithms to compare effectiveness of resultant features. The new multi-class or ensemble feature selection algorithm exhibited promising results in eliminating redundant attributes. The Rotation Forest selection based features demonstrated accuracies between 98% and 99% in various classifiers and this is a quite high performance for Erythemato-Squamous Diseases diagnosis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Polat, K., and Gunes, S., An expert system approach based on principal component analysis and adaptive neuro-fuzzy inference system to diagnosis of diabetes disease. Digit. Signal Process. 702–710, 2007, Elsevier.
Ozcift, A., and Gulten, A., Assessing effects of preprocessing mass spectrometry data on classification performance. Eur. J. Mass Spectrom. 267–273, 2008.
Kononenko, I., Machine learning for medical diagnosis: History, state of the art and perspective. Artif. Intell. Med. 23(1):89–109, 2001.
Abeel, T., Helleputte, T., Van de Peer, Y., Dupont, P., and Saeys, Y., Robust biomarker identification for cancer diagnosis with ensemble feature selection methods. Bioinformatics (Oxford, England) 26(3):392–398, 2010.
Tsymbal, A., Pechenizkiy, M., and Cunningham, P., Diversity in search strategies for ensemble feature selection, Computer-Based Medical Systems, 2003. Proceedings. 16th IEEE Symposium, pp. 83–98, 2005.
Abdel-Aal, R., Improved classification of medical data using abductive network committees trained on different feature subsets. Comput. Methods Prog. Biomed., 141–153, 2005.
Guvenir, H. A., Demiroz, G., and Ilter, N. Learning differential diagnosis of erythemato-squamous diseases using voting feature intervals. Artif. Intell. Med., 147–165, 1998.
Opitz, D., and Maclin, R., Popular ensemble methods: An empirical study. J. Artif. Intell. Res. 169–198, 1999.
Kuncheva, L., and Whitaker, C., Measures of diversity in classifier ensembles. Mach. Learn. 181–207, 2003.
Ho, T., Random decision forests. Proceedings of the Third International Conference on Document Analysis and Recognition 278–282, 1995.
Polikar, R., Ensemble based systems in decision making. IEEE Circuits Syst. Mag. 21–45, 2006.
Efron, B., Bootstrap methods: Another look at the jackknife. Ann. Stat. 1–26, 1979.
Polikar, R., Ensemble learning. Scholarpedia 4(1):2776, 2009.
Breiman, L., Random forests. Mach. Learn. 5–32, 2001.
Melville, P., and Mooney, R., Constructing diverse classifier ensembles using artificial training examples, Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence 505–510, 2003.
Zhang, C., and Zhang, J., A variant of rotation forest for constructing ensemble classifiers. Pattern Analysis Applications 59–77, 2010.
Rodriguez, J., and Kuncheva, L., Rotation forest: A new classifier ensemble method. IEEE Trans. Pattern Anal. Mach. Intell., 1619–1630, 2006.
Martinez, A. M., and Zhu, M., Where are linear feature extraction methods applicable? IEEE Transaction on Pattern Analysis and Machine Intelligence 1934–1944, 2005.
Fodor, I. K., A survey of dimension reduction techniques. Department of Energy by the University of California, 1–7, 2002
Saeys, Y., A review of feature selection techniques in bioinformatics. Bioinformatics Review 2507–2517, 2007.
Kohavi, R., and John, G., Wrappers for feature subset selection. Artif. Intell. 273–324, 1997.
Guyon, I. (Editor), Feature extraction, foundations and applications. Stud. Fuzziness Soft Comput. 119–135, 2006.
Xiao, R., Li, W., Tian, Y., and Tang, X., Joint boosting feature selection for robust face recognition. Computer Vision and Pattern Recognition. IEEE Computer Society Conference on 2 1415–1422, 2006.
Chen, K., Chou, C., and Shih, S., Feature selection for iris recognition with AdaBoost. Intelligent Information Hiding and Multimedia Signal Processing 411–414, 2007.
Tian, B., Zheng, X., Zhang, R., and Zhao, Y., Feature selection based on AdaBoost in video surveillance system, Proceedings of the 2009 Second international Conference on intelligent Computation Technology and Automation 70–72, 2009.
Viswanathan, M., Measurement error and research design. Sage Publications, 44–60, 2005
Witten, I. H., and Ian, H., Data mining: practical machine learning tools and techniques. Morgan Kaufmann Ser. Data Manage. Syst. 153–168, 2005.
David, A., Comparison of classification accuracy using Cohen’s Weighted Kappa. Expert Syst. Appl. 825–832, 2008.
Huang, J., and Ling, C., Using AUC and accuracy in evaluating learning algorithms. IEEE Trans. Knowl. Data Eng. 299–310, 2005.
Kotsiantis, S. B., Supervised machine learning: A review of classification techniques. Informatica 249–268, 2007.
Karabatak, M., and Cevdet İnce, M., A new feature selection method based on association rules for diagnosis of erythemato-squamous diseases. Expert Syst. Appl. 12500–12505, 2009.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ozcift, A., Gulten, A. A Robust Multi-Class Feature Selection Strategy Based on Rotation Forest Ensemble Algorithm for Diagnosis of Erythemato-Squamous Diseases. J Med Syst 36, 941–949 (2012). https://doi.org/10.1007/s10916-010-9558-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10916-010-9558-0