Abstract
Different classification tasks require different learning schemes to be satisfactorily solved. Most real-world datasets can be modeled only by complex structures resulting from deep data exploration with a number of different classification and data transformation methods. The search through the space of complex structures must be augmented with reliable validation strategies. All these techniques were necessary to build accurate models for the five high-dimensional datasets of the NIPS 2003 Feature Selection Challenge. Several feature selection algorithms (e.g. based on variance, correlation coefficient, decision trees) and several classification schemes (e.g. nearest neighbors, Normalized RBF, Support Vector Machines) were used to build complex models which transform the data and then classify. Committees of feature selection models and ensemble classifiers were also very helpful to construct models of high generalization abilities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R. Adamczak, W. Duch, and N. Jankowski. New developments in the feature space mapping model. In Third Conference on Neural Networks and Their Applications, pages 65–70, Kule, Poland, 1997. Polish Neural Networks Society.
B.E. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classifiers. In Fifth Annual Workshop on Computational Learning Theory, pages 144–152. ACM, 1992.
C.-C. Chang, C.-W. Hsu, and C.-J. Lin. The analysis of decomposition methods for support vector machines. IEEE Transaction on Neural Networks, 4:1003–1008, 2000.
T.M. Cover and P.E. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1):21–27, 1967.
W. Duch, J. Biesiada, T. Winiarski, K. Grudziński, and K. Grabczewski. Feature selection based on information theory filters and feature elimination wrapper methods. In Proceedings of the International Conference on Neural Networks and Soft Computing (ICNNSC 2002), Advances in Soft Computing, pages 173–176, Zakopane, 2002. Physica-Verlag (Springer).
W. Duch, T. Winiarski, J. Biesiada, and A. Kachel. Feature selection and ranking filters. In Artificial Neural Networks and Neural Information Processing — ICANN/ICONIP 2003, pages 251–254, Istanbul, 2003.
K. Grabczewski. SSV criterion based discretization for Naive Bayes classifiers. In Proceedings of the 7th International Conference on Artificial Intelligence and Soft Computing, Zakopane, Poland, June 2004.
K. Grabczewski and W. Duch. A general purpose separability criterion for classification systems. In Proceedings of the 4th Conference on Neural Networks and Their Applications, pages 203–208, Zakopane, Poland, June 1999.
K. Grabczewski and W. Duch. The Separability of Split Value criterion. In Proceedings of the 5th Conference on Neural Networks and Their Applications, pages 201–208, Zakopane, Poland, June 2000.
K. Grabczewski and N. Jankowski. Transformations of symbolic data for continuous data oriented models. In Artificial Neural Networks and Neural Information Processing — ICANN/ICONIP 2003, pages 359–366. Springer, 2003.
M. Grochowski and N. Jankowski. Comparison of instances seletion algorithms II: Algorithms survey. In Artificial Intelligence and Soft Computing, pages 598–603, 2004.
N. Jankowski and K. Grabczewki. Toward optimal SVM. In The Third IASTED International Conference on Artificial Intelligence and Applications, pages 451–456. ACTA Press, 2003.
N. Jankowski and V. Kadirkamanathan. Statistical control of RBF-like networks for classification. In 7th International Conference on Artificial Neural Networks, pages 385–390, Lausanne, Switzerland, October 1997. Springer-Verlag.
T. Joachims. Advances in kernel methods — support vector learning, chapter Making large-scale SVM learning practical. MIT Press, Cambridge, MA, 1998.
T. Kohonen. Learning vector quantization for pattern recognition. Technical Report TKK-F-A601, Helsinki University of Technology, Espoo, Finland, 1986.
E. Osuna, R. Freund, and F. Girosi. Support vector machines: Training and applications. AI Memo 1602, Massachusetts Institute of Technology, 1997a.
E. Osuna, R. Freund, and F. Girosi. Training support vector machines: An application to face detection. In CVPR’97, pages 130–136, New York, NY, 1997b. IEEE.
J. C. Platt. Fast training of support vector machines using sequential minimal optimization. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods — Support Vector Learning. MIT Press, Cambridge, MA., 1998.
J.C. Platt. Using analytic QP and sparseness to speed training of support vector machines. In M.S. Kearns, S.A. Solla, and D.A. Cohn, editors, Advances in Neural Information Processing Systems, volume 11, 1999.
C. Saunders, M.O. Stitson, J. Weston, L. Bottou, B. Schoelkopf, and A. Smola. Support vector machine reference manual. Technical Report CSD-TR-98-03, Royal Holloway, University of London, Egham, UK, 1998.
S.K. Shevade, S.S. Keerthi, C. Bhattacharyya, and K.R.K. Murthy. Improvements to the SMO algorithm for SVM regression. IEEE Transactions on Neural Networks, 11:1188–1194, Sept. 2000.
V. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, New York, 1995.
V. Vapnik. Statistical Learning Theory. Wiley, New York, NY, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Grabczewski, K., Jankowski, N. (2006). Mining for Complex Models Comprising Feature Selection and Classification. In: Guyon, I., Nikravesh, M., Gunn, S., Zadeh, L.A. (eds) Feature Extraction. Studies in Fuzziness and Soft Computing, vol 207. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-35488-8_24
Download citation
DOI: https://doi.org/10.1007/978-3-540-35488-8_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35487-1
Online ISBN: 978-3-540-35488-8
eBook Packages: EngineeringEngineering (R0)