Abstract
We propose a new approach, multi-view Laplacian support vector machines (SVMs), for semi-supervised learning under the multi-view scenario. It integrates manifold regularization and multi-view regularization into the usual formulation of SVMs and is a natural extension of SVMs from supervised learning to multi-view semi-supervised learning. The function optimization problem in a reproducing kernel Hilbert space is converted to an optimization in a finite-dimensional Euclidean space. After providing a theoretical bound for the generalization performance of the proposed method, we further give a formulation of the empirical Rademacher complexity which affects the bound significantly. From this bound and the empirical Rademacher complexity, we can gain insights into the roles played by different regularization terms to the generalization performance. Experimental results on synthetic and real-world data sets are presented, which validate the effectiveness of the proposed multi-view Laplacian SVMs approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chapelle, O., Schölkopf, B., Zien, A.: Semi-supervised Learning. MIT Press, Cambridge (2006)
Culp, M., Michailidis, G.: Graph-based Semi-supervised Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 174–179 (2008)
Sun, S., Shawe-Taylor, J.: Sparse Semi-supervised Learning using Conjugate Functions. Journal of Machine Learning Research 11, 2423–2455 (2010)
Zhu, X.: Semi-supervised Learning Literature Survey. Technical Report, 1530, University of Wisconsin-Madison (2008)
Sun, S., Jin, F., Tu, W.: View Construction for Multi-view Semi-supervised Learning. In: Liu, D., Zhang, H., Polycarpou, M., Alippi, C., He, H. (eds.) ISNN 2011, Part I. LNCS, vol. 6675, pp. 595–601. Springer, Heidelberg (2011)
Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)
Joachims, T.: Transductive Inference for Text Classification using Support Vector Machines. In: Proceedings of the 16th International Conference on Machine Learning, pp. 200–209 (1999)
Bennett, K., Demiriz, A.: Semi-supervised Support Vector Machines. Advances in Neural Information Processing Systems 11, 368–374 (1999)
Fung, G., Mangasarian, O.L.: Semi-supervised Support Vector Machines for Unlabeled Data Classification. Optimization Methods and Software 15, 29–44 (2001)
Belkin, M., Niyogi, P., Sindhwani, V.: Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Exampls. Journal of Machine Learning Research 7, 2399–2434 (2006)
Sindhwani, V., Niyogi, P., Belkin, M.: A Co-regularization Approach to Semi-supervised Learning with Multiple Views. In: Proceedings of the Workshop on Learning with Multiple Views, International Conference on Machine Learning (2005)
Farquhar, J., Hardoon, D., Meng, H., Shawe-Taylor, J., Szedmak, S.: Two View Learning: SVM-2K, Theory and Practice. Advances in Neural Information Processing Systems 18, 355–362 (2006)
Tikhonov, A.N.: Regularization of Incorrectly Posed Problems. Soviet Mathematics Doklady 4, 1624–1627 (1963)
Evgeniou, T., Pontil, M., Poggio, T.: Regularization Networks and Support Vector Machines. Advances in Computational Mathematics 13, 1–50 (2000)
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)
Belkin, M., Niyogi, P.: Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation 15, 1373–1396 (2003)
Blum, A., Mitchell, T.: Combining Labeled and Unlabeled Data with Co-training. In: Proceedings of the 11th Annual Conference on Computational Learning Theory, pp. 92–100 (1998)
Aronszajn, N.: Theory of Reproducing Kernels. Transactions of the American Mathematical Society 68, 337–404 (1950)
Sindhwani, V., Rosenberg, D.: An RKHS for Multi-view Learning and Manifold Co-regularization. In: Proceedings of the 25th International Conference on Machine Learning, pp. 976–983 (2008)
Kimeldorf, G., Wahba, G.: Some Results on Tchebycheffian Spline Functions. Journal of Mathematical Analysis and Applications 33, 82–95 (1971)
Rosenberg, D.: Semi-Supervised Learning with Multiple Views. PhD dissertation, Department of Statistics, University of California, Berkeley (2008)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)
Bartlett, P., Mendelson, S.: Rademacher and Gaussian Complexities: Risk Bounds and Structural Results. Journal of Machine Learning Research 3, 463–482 (2002)
Rosenberg, D., Bartlett, P.: The Rademacher Complexity of Co-regularized Kernel Classes. In: Proceedings of the 11th International Conference on Artificial Intelligence and Statistics, pp. 396–403 (2007)
Latala, R., Oleszkiewicz, K.: On the Best Constant in the Khintchine-Kahane Inequality. Studia Mathematica 109, 101–104 (1994)
Porter, M.F.: An Algorithm for Suffix Stripping. Program 14, 130–137 (1980)
Salton, G., Buckley, C.: Term-Weighting Approaches in Automatic Text Retrieval. Information Processing and Management 24, 513–523 (1988)
Sun, S.: Semantic Features for Multi-view Semi-supervised and Active Learning of Text Classification. In: Proceedings of the IEEE International Conference on Data Mining Workshops, pp. 731–735 (2008)
Rosenberg, D., Sindhwani, V., Bartlett, P., Niyogi, P.: Multiview Point Cloud Kernels for Semisupervised Learning. IEEE Signal Processing Magazine, 145–150 (2009)
Hsu, C.W., Lin, C.J.: A Comparison of Methods for Multiclass Support Vector Machines. IEEE Transactions on Neural Networks 13, 415–425 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sun, S. (2011). Multi-view Laplacian Support Vector Machines. In: Tang, J., King, I., Chen, L., Wang, J. (eds) Advanced Data Mining and Applications. ADMA 2011. Lecture Notes in Computer Science(), vol 7121. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25856-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-25856-5_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-25855-8
Online ISBN: 978-3-642-25856-5
eBook Packages: Computer ScienceComputer Science (R0)