Abstract
Inspired by the success of support vector machines, to improve generalization and classification abilities, conventional pattern classification techniques have been extended to incorporate maximizing margins and mapping to a feature space. For example, perceptron algorithms [1–4], neural networks (Chapter 9), and fuzzy systems (Chapter 10) have incorporated maximizing margins and/or mapping to a feature space.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Y. Freund and R. E. Schapire.Large margin classification using the perceptron algorithm.Machine Learning, 37(3):277–296, 1999.
C. Gentile. A new approximate maximal margin classification algorithm.Journal of Machine Learning Research, 2:213–242, 2001.
Y. Li and P. M. Long. The relaxed online maximum margin algorithm. Machine Learning, 46(1–3):361–387, 2002.
K. Crammer and Y. Singer. Ultraconservative online algorithms for multiclass problems. Journal of Machine Learning Research, 3:951–991, 2003.
G. Baudat and F. Anouar. Kernel-based methods and function approximation. In Proceedings of International Joint Conference on Neural Networks (IJCNN ’01), volume 2, pages 1244–1249, Washington, DC, 2001.
A. Ruiz and P. E. López-de-Teruel. Nonlinear kernel-based statistical pattern analysis. IEEE Transactions on Neural Networks, 12(1):16–32, 2001.
B. Schölkopf, A. J. Smola, and K.-R. Müller. Kernel principal component analysis. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods: Support Vector Learning, pages 327–352. MIT Press, Cambridge, MA, 1999.
A. J. Smola, O. L. Mangasarian, and B. Schölkopf. Sparse kernel feature analysis. Technical Report 99-04, University of Wisconsin, Data Mining Institute, Madison, WI, 1999.
B. Schölkopf and A. J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, 2002.
Z. Rong and A. I. Rudnicky. A large scale clustering scheme for kernel k-means.In Proceedings of the Sixteenth International Conference on Pattern Recognition (ICPR 2002), volume 4, pages 289–292, 2002.
I. S. Dhillon, Y. Guan, and B. Kulis. Kernel k-means, spectral clustering and normalized cuts. In KDD-2004: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, pages 551–556. Association for Computing Machinery, New York, 2004.
Z. Li, S. Tang, J. Xue, and J. Jiang. Modified FCM clustering based on kernel mapping. In J. Shen, S. Pankanti, and R. Wang, editors, Proceedings of SPIE: Object Detection, Classification, and Tracking Technologies, volume 4554, pages 241–245, Wuhan, China, 2001.
D.-Q. Zhang and S.-C. Chen. Clustering incomplete data using kernel-based fuzzy c-means algorithm. Neural Processing Letters, 18(3):155–162, 2003.
T. Graepel, M. Burger, and K. Obermayer. Self-organizing maps: Generalizations and new optimization techniques. Neurocomputing, 21(1–3):173–190, 1998.
E. Maeda and H. Murase. Multi-category classification by kernel based nonlinear subspace method. In Proceedings of 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP ’99), volume 2, pages 1025–1028, 1999.
S. Takeuchi, T. Kitamura, S. Abe, and K. Fukui. Subspace based linear programming support vector machines. In Proceedings of the 2009 International Joint Conference on Neural Networks (IJCNN 2009), pages 3067–3073, Atlanta, GA, 2009.
T. Kitamura, S. Abe, and K. Fukui.Subspace based least squares support vector machines for pattern classification. In Proceedings of the 2009 International Joint Conference on Neural Networks (IJCNN 2009), pages 1640–1646, Atlanta, GA, 2009.
T. Kitamura S. Takeuchi, S. Abe, and K. Fukui. Subspace-based support vector machines. Neural Networks, 22(5–6):558–567, 2009.
C. H. Park and H. Park. Efficient nonlinear dimension reduction for clustered data using kernel functions. In Proceedings of the Third IEEE International Conference on Data Mining (ICDM 2003), pages 243–250, Melbourne, FL, 2003.
P. Zhang, J. Peng, and C. Domeniconi. Dimensionality reduction using kernel pooled local discriminant information. In Proceedings of the Third IEEE International Conference on Data Mining (ICDM 2003), pages 701–704, Melbourne, FL, 2003.
G. C. Cawley and N. L. C. Talbot. Efficient formation of a basis in a kernel feature space. In Proceedings of the Tenth European Symposium on Artificial Neural Networks (ESANN 2002), pages 1–6, Bruges, Belgium, 2002.
S. Fine and K. Scheinberg. Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research, 2:243–264, 2001.
T. Evgeniou, M. Pontil, and T. Poggio. Regularization networks and support vector machines. Advances in Computational Mathematics, 13(1):1–50, 2000.
P. Zhang and J. Peng. SVM vs regularized least squares classification. In Proceedings of the Seventeenth International Conference on Pattern Recognition (ICPR 2004), volume 1, pages 176–179, Cambridge, UK, 2004.
K. Morikawa. Pattern classification and function approximation by kernel least squares. Bachelor’s thesis, Electrical and Electronics Engineering, Kobe University, Japan, 2004 (in Japanese).
G. H. Golub and C. F. Van Loan. Matrix Computations, Third Edition. The Johns Hopkins University Press, Baltimore, MD, 1996.
W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Numerical Recipes in C: The Art of Scientific Computing, Second Edition. Cambridge University Press, Cambridge, UK, 1992.
S. Mika, B. Schölkopf, A. Smola, K.-R. Müller, M. Scholz, and G. Rätsch. Kernel PCA and de-noising in feature spaces. In M. S. Kearns, S. A. Solla, and D. A. Cohn, editors, Advances in Neural Information Processing Systems 11, pages 536–542. MIT Press, Cambridge, MA, 1999.
B. Schölkopf, S. Mika, C. J. C. Burges, P. Knirsch, K.-R. Müller, G. Rätsch, and A. J. Smola. Input space versus feature space in kernel-based methods. IEEE Transactions on Neural Networks, 10(5):1000–1017, 1999.
T. Takahashi and T. Kurita. Robust de-noising by kernel PCA. In J. R. Dorronsoro, editor, Artificial Neural Networks (ICANN 2002)–-Proceedings of International Conference, Madrid, Spain, pages 739–744. Springer-Verlag, Berlin, Germany, 2002.
J. T.-Y. Kwok and I. W.-H. Tsang. The pre-image problem in kernel methods. IEEE Transactions on Neural Networks, 15(6):1517–1525, 2004.
R. O. Duda and P. E. Hart. Pattern Classification and Scene Analysis. John Wiley & Sons, New York, 1973.
S. Mika, G. Rätsch, J. Weston, B. Schölkopf, and K.-R. Müller. Fisher discriminant analysis with kernels. In Y.-H. Hu, J. Larsen, E. Wilson, and S. Douglas, editors, Neural Networks for Signal Processing IX–-Proceedings of the 1999 IEEE Signal Processing Society Workshop, pages 41–48, 1999.
G. Baudat and F. Anouar. Generalized discriminant analysis using a kernel approach. Neural Computation, 12(10):2385–2404, 2000.
H. Li, T. Jiang, and K. Zhang. Efficient and robust feature extraction by maximum margin criterion. In S. Thrun, L. K. Saul, and B. Schölkopf, editors, Advances in Neural Information Processing Systems 16, pages 97–104. MIT Press, Cambridge, MA, 2004.
J. Yang, A. F. Frangi, J.-Y. Yang, D. Zhang, and Z. Jin. KPCA plus LDA: A complete kernel fisher discriminant framework for feature extraction and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(2):230–244, 2005.
E. Pekalska and B. Haasdonk. Kernel discriminant analysis for positive definite and indefinite kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(6):1017–1031, 2009.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2010 Springer-Verlag London Limited
About this chapter
Cite this chapter
Abe, S. (2010). Kernel-Based Methods . In: Support Vector Machines for Pattern Classification. Advances in Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-84996-098-4_6
Download citation
DOI: https://doi.org/10.1007/978-1-84996-098-4_6
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-84996-097-7
Online ISBN: 978-1-84996-098-4
eBook Packages: Computer ScienceComputer Science (R0)