Ranked selection of nearest discriminating features
Feature selection techniques use a search-criteria driven approach for ranked feature subset selection. Often, selecting an optimal subset of ranked features using the existing methods is intractable for high dimensional gene data classification problems.
In this paper, an approach based on the individual ability of the features to discriminate between different classes is proposed. The area of overlap measure between feature to feature inter-class and intra-class distance distributions is used to measure the discriminatory ability of each feature. Features with area of overlap below a specified threshold is selected to form the subset.
The reported method achieves higher classification accuracies with fewer numbers of features for high-dimensional micro-array gene classification problems. Experiments done on CLL-SUB-111, SMK-CAN-187, GLI-85, GLA-BRA-180 and TOX-171 databases resulted in an accuracy of 74.9±2.6, 71.2±1.7, 88.3±2.9, 68.4±5.1, and 69.6±4.4, with the corresponding selected number of features being 1, 1, 3, 37, and 89 respectively.
The area of overlap between the inter-class and intra-class distances is demonstrated as a useful technique for selection of most discriminative ranked features. Improved classification accuracy is obtained by relevant selection of most discriminative features using the proposed method.
Supplementary Material (0)
- Guyon I, Elisseeff A: An introduction to variable and feature selection. J Machine Learning Res 2003, 3:1157–1182.
- Saeys Y, Inza I, Larraage P: A review of feature selection techniques in bioinformatics. Bioinformatics 2007,23(19):2507–2517. CrossRef
- Inza I, Larranaga P, Blanco R, Cerrolaza A: Filter versus warpper gene selection approaches in dna microarray domains. Artif Intelligence Med 2004, 31:91–103. CrossRef
- Ma S, Huang J: Penalized feature selection and classification in bioinformatics. Brief Bioinfrom 2008,9(5):392–403. CrossRef
- James AP, Maan A: Improving feature selection algorithms using normalised feature histograms. IET Electron lett 2011,47(8):490–491. CrossRef
- Liu H, Motoda H: Feature selection for knowledge discovery and data mining. 1998. Boston, Kluwer Academic Publishers CrossRef
- Donoho D: Formost large underdetermined systems of linear equations, the minimal l1-norm solution is also the sparest solution. Comm Pure Appl Math 2006, 59:907–934. CrossRef
- Fan J, Samworth R, Wu Y: Ultrahigh dimensional feature selection: Beyond the linear model. J Machine Learning Res 2009, 10:2013–2038.
- Glocer K, Eads D, Theiler J: Online feature selection for pixel classification. 2005.
- Zhao Z, Liu H: Multi-scource feature selection via geometry dependent covariance analysis. J Machine Learning Res, Workshop Conference Proc Volume 4: New Challenges Feature Sel Data Min Knowledge Discovery 2008, 4:36–47.
- James AP, Dimitrijev S: Nearest Neighbor Classifier Based on Nearest Feature Decisions. Comput J 2012. doi:10.1093/comjnl/bxs001
- James A, Dimitrijev S: Inter-image outliers and their application to image classification. Pattern Recognit 2010,43(12):4101–4112. CrossRef
- Lee JA, Verleysen M: Nonlinear Dimensionality Reduction. 2007. New York, Springer CrossRef
- Thangavel K, Pethalakshmi A: Dimensionality reduction based on rough set theory: A review. Appl Soft Comput 2009,9(1):1–12. CrossRef
- Sanguinetti G: Dimensionality Reduction of Clustered Data Sets. Pattern Anal Machine Intelligence, IEEE Trans 2007,30(3):535–540. CrossRef
- Zhao Z, Wang J, Sharma S, Agarwal N, Liu H, Chang Y: An intergrative approach to identifying biologically relevant genes. 2010, pp 838–849.
- Liu H, Yu L: Toward intergrating feature selection algorithms for classification and clustering. IEEE Transactions Knowledge Data Eng 2005,17(3):1–12. CrossRef
- Li T, Zhang C, Ogihara M: A comparative study of feature selection and multiclass classification methods for tissue classification based on gene expressions. Bioinformatics 2004,20(15):2429–2437. CrossRef
- Liu H, Li J, Wong L: A comparative study on feature selection and classification methods using gene expression profiles and proteomic patterns. Genone Inform 2002, 13:51–60.
- Sikonja MR, Kononenko I: Theoritical and emperical analysis of Relief and Relief. Machine Learning 2003, 53:23–69. CrossRef
- Weston J, Elisseff A, Schoelkopf B, Tipping M: Use of the zero norm with linear models and kernel methods. J Machine Learning Res 2003, 3:1439–1461.
- Song L, Smola A, Gretton A, Brogwardt K, Bedo J: Supervised feature selection via dependence estimation. 2007.
- Efron B, Hastie T, Johnstone I, Tibshirani R: Least angle regression. Ann Stat 2004, 32:407–449. CrossRef
- Zhu J, Rosset S, Hastie T, Tibshirani R: 1-norm support vector machines. 2003.
- Cawley GC, Talbot NLC, Girolami M: Sparse multinomial logistic regression via bayesian L1 regularisation. 2007.
- Hall MA: Correlation based feature selection for discrete and numeric class machine learning. 2000.
- Liu H, Setiono R: A probabilistic approach to feature selection: a filter solution. 1996.
- Kohavi R, John G: Wrappers for Feature Subset Selection. Artif Intelligence 1997,97(1–2):273–324. CrossRef
- Caruana R, Freitag D: Greedy attribute selection. 1994.
- Das S: Filters, warppers and boosting: based hybrid for feature selection. 2001.
- Ng AY: On feature selection: learning with exponentially many irrelevant features as training examples. 1998.
- Dash M, Liu H: Feature selection for classification. Intell Data Anal 1997,1(3):131–156. CrossRef
- Narendra PM, Fukunaga K: Branch and bound algorithm for feature subset selection. IEEE Trans Comput 1977,26(9):917–922. CrossRef
- Doak J: An evaluation of feature selection methods and their application to computer security. 1992.
- Liu H, Motoda H: Feature selection for knowledge discovery and data mining. 1998. Boston, Kluwer Academic CrossRef
- Almuallim H, Dietterich TG: Learning boolean concepts in the presence of many irrelavent features. Artif Intelligence 1994,69(1–2):278–305. CrossRef
- Ben-Bassat M: Pattern recognition and reduction of dimensionality. 1982.
- Blum AL, Langley P: Selection of relevant features and examples in machine learning. Artif Intelligence 1997, 97:245–271. CrossRef
- Dash M, Liu H: Feature selection for clustering. 2000, pp 110–121.
- Di JG, Brodley CE: Feature subset selection and order itdentification for unsupervised learning. San Fransisco, Morgan Kaufmann, pp 247–254.
- Kim Y, Street W, Menczer F: Feature selection for unsupervised learning via evolutionary search. 2000.
- Jain A, Zongker D: Feature selection: evaluation, application, and small sample performance. IEEE Trans Pattern Anal Mach Intell 1997, 19:153–158. CrossRef
- Blum A, Rivest R: Training a 3-Node Neural Networks in NP-Complete. Neural Networks 1992, 5:117–127. CrossRef
- John GH, Kohavi R, Pflegler K: Irrelavent feature and the subset selection problem. 1994.
- Abe S, Thawonmas R, Kobayashi Y: Feature selection by analysing class regions approximated by ellipsoids. IEEE Trans Syst, Man Cybernetics– Part C: App Rev 1998, 28:282–287. CrossRef
- Neumann J, Schnorr C, Steidl G: Combined SVM-based feature selection and classification. Machine Learning 2005, 61:129–150. CrossRef
- Xu Z, King I, Lyu MR-T, Jin R: Discriminative semisupervised feature selection via manifold regularization. IEEE Trans. on Neural Networks 2010,21(7):1033–1047. CrossRef
- Freije WA, Castro-Vargas FE, Fang Z, Horvath S, Cloughesy T, Liau LM, Mischel PS, Nelson SF: Gene expression profiling of gliomas strongly predicts survival. Cancer Res 2004,64(18):6503–6510. CrossRef
- Sun L, Hui AM, Su Q, Vortmeyer A, Kotliarov Y, Pastorino S, James AP: Neuronal and glioma-derived stem cell factor induces angiogenesis within the brain. Cancer Cell 2006,9(4):287–300. CrossRef
- Haslinger C, Schweifer N, Stilgenbauer S, Dhner H, Lichter P, Kraut N, Stratowa C, Abseher R: Microarray gene expression profiling of B-cell chronic lymphocytic leukemia subgroups defined by genomic aberrations and VH mutation status. J Clin Oncol 2004,22(19):3937–3949. CrossRef
- Piloto S, Schilling T: Ovo1 links Wnt signaling with N-cadherin localization during neural crest migration. Development 2010,137(12):1981–1990. CrossRef
- Spira A, Beane JE, Shah V, Steiling K, Liu G, Schembri F, Gilman S, Dumas YM, Calner P, Sebastiani P, Sridhar S, Beamis J, Lamb C, Anderson T, Gerry N, Keane J, Lenburg ME, Brody JS: Airway epithelial gene expression in the diagnostic evaluation of smokers with suspect lung cancer. Nat Med 2007,13(3):361–366. CrossRef
- Peng H, Long F, Ding C: Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Machine Intell 2005,27(8):1226–1238. CrossRef
- Cover TM, Thomas JA: Elem Inf Theory. 1991. New York, Wiley CrossRef
About this Article
- Ranked selection of nearest discriminating features
- Open Access
- Available under Open Access This content is freely available online to anyone, anywhere at any time.
Human-centric Computing and Information Sciences
- Online Date
- June 2012
- Online ISSN
- Additional Links