Abstract
We introduce a deterministic sampling based feature selection technique for regularized least squares classification. The method is unsupervised and gives worst-case guarantees of the generalization power of the classification function after feature selection with respect to the classification function obtained using all features. We perform experiments on synthetic and real-world datasets, namely a subset of TechTC-300 datasets, to support our theory. Experimental results indicate that the proposed method performs better than the existing feature selection methods.
Chapter PDF
Similar content being viewed by others
References
Agarwal, D.: Shrinkage estimator generalizations of proximal support vector machines. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 173–182 (2002)
Avron, H., Sindhwani, V., Woodruff, D.: Sketching structured matrices for faster nonlinear regression. In: Advances in Neural Information Processing Systems, pp. 2994–3002 (2013)
Batson, J., Spielman, D., Srivastava, N.: Twice-ramanujan sparsifiers. In: Proceedings of the 41st Annual ACM STOC, pp. 255–262 (2009)
Bhattacharyya, C.: Second order cone programming formulations for feature selection. JMLR 5, 1417–1433 (2004)
Boutsidis, C., Magdon-Ismail, M.: Deterministic feature selection for k -means clustering. IEEE Transactions on Information Theory 59(9), 6099–6110 (2013)
Dasgupta, A., Drineas, P., Harb, B., Josifovski, V., Mahoney, M.: Feature selection methods for text classification. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 230–239 (2007)
Davidov, D., Gabrilovich, E., Markovitch, S.: Parameterized generation of labeled datasets for text categorization based on a hierarchical directory. In: Proceedings of the 27th Annual International ACM SIGIR Conference, pp. 250–257 (2004), http://techtc.cs.technion.ac.il/techtc300/techtc300.html
Demmel, J., Veselic, K.: Jacobi’s method is more accurate than qr. SIAM Journal on Matrix Analysis and Applications 13(4), 1204–1245 (1992)
Drineas, P., Mahoney, M., Muthukrishnan, S.: Sampling algorithms for l2 regression and applications. In: Proceedings of the 17th Annual ACM-SIAM SODA, pp. 1127–1136 (2006)
Fung, G., Mangasarian, O.: Proximal support vector machine classifiers. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 77–86 (2001)
Paul, S., Magdon-Ismail, M., Drineas, P.: Deterministic feature selection for linear svm with provable guarantees (2014), http://arxiv.org/abs/1406.0167
Poggio, T., Smale, S.: The mathematics of learning: Dealing with data. Notices of the AMS 50(5), 537–544 (2003)
Rifkin, R., Yeo, G., Poggio, T.: Regularized least-squares classification. Nato Science Series Sub Series III Computer and Systems Sciences 190, 131–154 (2003)
Stewart, G., Sun, J.: Matrix perturbation theory (1990)
Suykens, J., Vandewalle, J.: Least squares support vector machine classifiers. Neural Processing Letters 9(3), 293–300 (1999)
Yang, Y., Pedersen, J.: A comparative study on feature selection in text categorization. In: ICML, vol. 97, pp. 412–420 (1997)
Zhang, P., Peng, J.: SVM vs regularized least squares classification. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 1, pp. 176–179 (2004)
Zhang, T., Oles, F.: Text categorization based on regularized linear classification methods. Information Retrieval 4(1), 5–31 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Paul, S., Drineas, P. (2014). Deterministic Feature Selection for Regularized Least Squares Classification. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2014. Lecture Notes in Computer Science(), vol 8725. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44851-9_34
Download citation
DOI: https://doi.org/10.1007/978-3-662-44851-9_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44850-2
Online ISBN: 978-3-662-44851-9
eBook Packages: Computer ScienceComputer Science (R0)