Adaptive Weighted Fusion of Local Kernel Classifiers for Effective Pattern Classification

  • Shixin Yang
  • Wangmeng Zuo
  • Lei Liu
  • Yanlai Li
  • David Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6838)


The theoretical and practical virtual of local learning algorithms had been verified by the machine learning community. The selection of the proper local classifier, however, remains a challenging problem. Rather than selecting one single local classifier, in this paper, we propose to choose several local classifiers and use adaptive fusion strategy to alleviate the choice problem of the proper local classifier. Based on the fast and scalable local kernel support vector machine (FaLK-SVM), we adopt the self-adaptive weighting fusion method for combining local support vector machine classifiers (FaLK-SVMa), and provide two fusion methods, distance-based weighting (FaLK-SVMad) and rank-based weighting methods (FaLK-SVMar). Experimental results on fourteen UCI datasets and three large scale datasets show that FaLK-SVMa can chieve higher classification accuracy than FaLK-SVM.


Kernel method support vector machine local learning classifier fusion nearest neighbors 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bottou, L., Vapnik, V.: Local Learning Algorithms. Neural Computation 4(6), 888–900 (1992)CrossRefGoogle Scholar
  2. 2.
    Vapnik, V., Bottou, L.: Local Algorithms for Pattern Recognition and Dependencies Estimation. Neural Computation 5(6), 893–909 (1993)CrossRefGoogle Scholar
  3. 3.
    Vapnik, V.: Statistical Learning Theory. Wiley- Interscience, Chichester (1998)MATHGoogle Scholar
  4. 4.
    Zhang, H., Berg, A.C., Maire, M., Malik, J.: SVM- KNN: Discriminative Nearest Neighbour Classification for Visual Category Recognition. In: IEEE International Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2126–2136 (2006)Google Scholar
  5. 5.
    Blanzieri, E., Melgani, F.: Nearest Neighbor Classification of Remote Sensing Images with the Maximal Margin Principle. IEEE Trans. Geoscience and Remote Sensing 46(6), 1804–1811 (2008)CrossRefGoogle Scholar
  6. 6.
    Yang, T., Kecman, V.: Face Recognition with Adaptive Local Hyperplane Algorithm. Pattern Analysis and Applications 13(1), 79–83 (2010)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Cheng, H., Tan, P.-N., Jin, R.: Localized Support Vector Machine and Its Efficient Algorithm. In: SIAM International Conference on Data Mining (2007)Google Scholar
  8. 8.
    Segata, N., Blanzieri, E.: Fast and Scalable Local Kernel Machines. Journal of Machine Learning Research 11, 1883–1926 (2010)MathSciNetMATHGoogle Scholar
  9. 9.
    Beygelzimer, A., Kakade, S., Langford, J.: Cover Trees for Nearest Neighbor. In: 23rd International Conference on Machine Learning (ICML 2006), pp. 97–104 (2006)Google Scholar
  10. 10.
    Wang, J., Neskovic, P., Cooper, N.L.: A Minimum Sphere Covering Approach to Pattern Classification. In: International Conference on Pattern Recognition, vol. 3, pp. 433–436 (2006)Google Scholar
  11. 11.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, Heidelberg (2000)CrossRefMATHGoogle Scholar
  12. 12.
    Zakai, A., Ritov, Y.: Consistency and Localizability. Journal of Machine Learning Research 10, 827–856 (2009)MathSciNetMATHGoogle Scholar
  13. 13.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On Combining Classifiers. IEEE Trans. Pattern Analysis and Machine Intelligence. 20(3), 226–239 (1998)CrossRefGoogle Scholar
  14. 14.
    Liu, C.L.: Classifier Combination Based on Confidence Transformation. Pattern Recognition 38(1), 11–28 (2005)CrossRefMATHGoogle Scholar
  15. 15.
    Zhang, L., Zhou, W.D.: Sparse Ensembles Using Weighted Combination Methods Based on Linear Programming. Pattern Recognition 44(1), 97–106 (2011)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Chang, C.-C., Lin, C.-J.: LIBSVM: a Library for Support Vector Machines,
  17. 17.
    Segata, N.: FaLKM-lib: a Library for Fast Local Kernel Machines,
  18. 18.
    Asuncion, A., Newman, D.: UCI Repository of Machine Learning Data Sets,
  19. 19.
    Tenenbaum, J.B., de Silva, V., Langford, J.C.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science 290, 2319–2323 (2000)CrossRefGoogle Scholar
  20. 20.
    Saul, L.K., Roweis, S.T.: Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifolds. Journal of Machine Learning Research 4, 119–155 (2003)MathSciNetMATHGoogle Scholar
  21. 21.
    Roweis, S.T., Saul, L.K.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science 290, 2323–2326 (2000)CrossRefGoogle Scholar
  22. 22.
    Bengio, Y., Paiement, J.-F., Vincent, P., Delalleau, O., Roux, N.L., Ouimet, M.: Out-of-sample Extensions for LLE, Isomap, MDS, Eigenmaps, and spectral clustering. In: Advances in Neural Information Processing Systems (NIPS 2003), vol. 16 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Shixin Yang
    • 1
  • Wangmeng Zuo
    • 1
  • Lei Liu
    • 1
  • Yanlai Li
    • 1
  • David Zhang
    • 1
    • 2
  1. 1.Biocomputing Research Centre, School of Computer Science and TechnologyHarbin Institute of TechnologyHarbinChina
  2. 2.Department of ComputingThe Hong Kong Polytechnic UniversityKowloonHong Kong

Personalised recommendations