Abstract
We introduce a learning technique for regression between high-dimensional spaces. Standard methods typically reduce this task to many one-dimensional problems, with each output dimension considered independently. By contrast, in our approach the feature construction and the regression estimation are performed jointly, directly minimizing a loss function that we specify, subject to a rank constraint. A major advantage of this approach is that the loss is no longer chosen according to the algorithmic requirements, but can be tailored to the characteristics of the task at hand; the features will then be optimal with respect to this objective, and dependence between the outputs can be exploited.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bojanczyk, A.W., Lutoborski, A.: The Procrustes problem for orthogonal Stiefel matrices. SIAM Journal on Scientific Computing 21(4), 1291–1304 (2000)
Crampin, M., Pirani, F.A.E.: Applicable Differential Geometry. Cambridge University Press, Cambridge (1986)
Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications 20(2), 303–353 (1999)
Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. John Hopkins University Press, Baltimore (1996)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2001)
Kashima, H., Tsuda, K., Inokuchi, A.: Marginalized kernels between labeled graphs. In: ICML 2003, pp. 321–328 (2003)
Saitoh, S.: Theory of Reproducing Kernels and its Applications. Longman Scientific & Technical, Harlow, England (1988)
Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
Smola, A.J., Kondor, R.: Kernels and regularization on graphs. In: Proceedings of the Annual Conference on Computational Learning Theory and Kernel Workshop, Springer, Heidelberg (2003)
Weston, J., Chapelle, O., Elisseeff, A., Schölkopf, B., Vapnik, V.: Kernel dependency estimation. In: Advances in Neural Information Processing Systems, vol. 15, MIT Press, Cambridge (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bakır, G.H., Gretton, A., Franz, M., Schölkopf, B. (2004). Multivariate Regression via Stiefel Manifold Constraints. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds) Pattern Recognition. DAGM 2004. Lecture Notes in Computer Science, vol 3175. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28649-3_32
Download citation
DOI: https://doi.org/10.1007/978-3-540-28649-3_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22945-2
Online ISBN: 978-3-540-28649-3
eBook Packages: Springer Book Archive