Abstract
We develop a methodology for solving high dimensional dependency estimation problems between pairs of data types, which is viable in the case where the output of interest has very high dimension, e.g., thousands of dimensions. This is achieved by mapping the objects into continuous or discrete spaces, using joint kernels. Known correlations between input and output can be defined by such kernels, some of which can maintain linearity in the outputs to provide simple (closed form) pre-images. We provide examples of such kernels and empirical results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Mercer, J.: Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society A 209, 415–446 (1909)
Aizerman, M.A., Braverman, E.M., Rozonoer, L.I.: Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control 25, 821–837 (1964)
Boser, B.E., Guyon, I.M., Vapnik, V.: A training algorithm for optimal margin classifiers. In: Haussler, D. (ed.) Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, Pittsburgh, PA, pp. 144–152. ACM Press, New York (1992)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
Kimeldorf, G.S., Wahba, G.: Some results on Tchebycheffian spline functions. Journal of Mathematical Analysis and Applications 33, 82–95 (1971)
Weston, J., Chapelle, O., Elisseeff, A., Schölkopf, B., Vapnik, V.: Kernel dependency estimation. Neural Processing Information Systems 15 (2002)
Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support vector machine learning for interdependent and structured output spaces. ICML (2004)
Haussler, D.: Convolution kernels on discrete structure. Technical report, UC Santa Cruz (1999)
Watkins, C.: Dynamic alignment kernels. In: Smola, A.J., Bartlett, P.L., Schölkopf, B., Schuurmans, D. (eds.) Advances in Large Margin Classifiers, pp. 39–50. MIT Press, Cambridge (2000)
Vapnik, V.N.: Statistical Learning Theory. Springer, Heidelberg (1998)
Pérez-Cruz, F., Camps, G., Soria, E., Pérez, J., Figueiras-Vidal, A.R., Artés-Rodrguez, A.: Multi-dimensional function approximation and regression estimation. In: ICANN 2002 (2002)
Weston, J., Watkins, C.: Multi-class support vector machines. Royal Holloway Technical Report CSD-TR-98-04 (1998)
Hofmann, T., Tsochantaridis, I., Altun, Y.: Learning over discrete output spaces via joint kernel functions. Kernel Methods Workshop, Neural Processing Information Systems 15 (2002)
Collins, M., Duffy, N.: Convolution kernels for natural language. Neural Processing Information Systems 14 (2001)
Micchelli, C.A., Pontil, M.: On learning vector-valued functions. Research Note RN/03/08, Dept of Computer Science, UCL (2003)
Guestrin, C., Taskar, B., Koller, D.: Max-margin markov networks. Neural Information Processing Systems 16 (2003)
Kwok, J.T., Tsang, I.W.: Finding the pre-images in kernel principal component analysis. In: 6th Annual Workshop On Kernel Machines, Whistler, Canada (2002)
Bakir, G.H., Weston, J., Schölkopf, B.: Learning to find pre-images. Advances in Neural Information Processing Systems 16 (2004)
Saunders, C., Gammerman, A., Vovk, V.: Ridge regression learning algorithm in dual variables. In: Proceedings of the Fifteenth International Conference on Machine Learning, pp. 515–521. Morgan Kaufmann Publishers Inc, San Francisco (1998)
Max Planck Institute Face Database, http://faces.kyb.tuebingen.mpg.de/
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: SIGGRAPH 1999, pp. 187–194 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Weston, J., Schölkopf, B., Bousquet, O. (2005). Joint Kernel Maps. In: Cabestany, J., Prieto, A., Sandoval, F. (eds) Computational Intelligence and Bioinspired Systems. IWANN 2005. Lecture Notes in Computer Science, vol 3512. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11494669_23
Download citation
DOI: https://doi.org/10.1007/11494669_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26208-4
Online ISBN: 978-3-540-32106-4
eBook Packages: Computer ScienceComputer Science (R0)