Abstract
A formal definition of task relatedness to theoretically justify multi-task learning (MTL) improvements has remained quite elusive. The implementation of MTL using multi-layer perceptron (MLP) neural networks evoked the notion of related tasks sharing an underlying representation. This assumption of relatedness can sometimes hurt the training process if tasks are not truly related in that way. In this paper we present a novel single-layer perceptron (SLP) approach to selectively achieve knowledge transfer in a multi-tasking scenario by using a different notion of task relatedness. The experimental results show that the proposed scheme largely outperforms single-task learning (STL) using single layer perceptrons, working in a robust way even when not closely related tasks are present.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Argyriou, A., Evgeniou, T., Pontil, M.: Multi-Task Feature Learning. In: NIPS 2006, Vancouver, B.C., Canada, 4-7 December 2006, to appear (2006)
Baxter, J.: Learning Internal Representations. In: COLT, pp. 311–320. ACM Press, New York (1995)
Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases (1998)
Caruana, R.: Multitask Learning. Machine Learning 28, 41–75 (1997)
Duda, R., Hart, P.: Pattern Classification and Scene Analysis. Wiley Interscience, Hoboken (1973)
Evgeniou, T., Pontil, M.: Regularized Multitask Learning. In: SIGKDD, Seattle, Washington, USA, 22-25 August (2004)
Evgeniou, T., Micchelli, C.A., Pontil, M.: Learning Multiple Tasks with Kernel Methods. Journal of Machine Learning Research 6, 615–637 (2005)
Evgeniou, T., Pontil, M., Toubia, O.: A Convex Optimization Approach to Modeling Heterogeneity in Conjoint Estimation. Marketing Science (to appear)
Gibbons, J., Chakraborti, S.: Nonparametric Statistical Inference, 3rd edn. Marcel Dekker, New York (1992)
Hastie, T., Tibshirani, R., Friedma, J.H.: The Elements of Statistical Learning. Springer, New York (2003)
Heskes, T.: Empirical Bayes for learning to learn. In: Langley, P. (ed.) ICML-2000, pp. 367–374. Morgan Kaufmann Publishers, San Francisco (2000)
Bakker, B., Heskes, T.: Task clustering and gating for Bayesian multi-task learning. Journal of Machine Learning Research 4, 83–99 (2003)
Jovicich, J.: Brain Areas Specific for Attentional Load in a Motion-Tracking Task. Journal of Cognitive Neuroscience 13, 1048–1058 (2001)
Liao, X., Carin, L.: Radial Basis Function Network for Multi-task Learning. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in NIPS, vol. 18, pp. 795–802. MIT Press, Volume (2006)
Pazzani, M.: The influence of prior knowledge on concept acquisition: Experimental and computational results. Journal of Experimental Psychology: Learning, Memory and Cognition 17(3), 416–432 (1991)
Rao, C.: Linear statistical inference and its applications. John Wiley, New York (1965)
Silver, D.: Selective Transfer of Neural Network Task Knowledge. PhD Thesis, University of Western Ontario (2000)
Thrun, S., O’Sullivan, J.: Clustering learning tasks and the selective cross-task transfer of knowledge. In: Thrun, S., Pratt, L.Y. (eds.) Learning to Learn, pp. 235–257. Kluwer Academic Publishers, Dordrecht (1998)
Tomasi, D., Ernst, T., Caparelli, E., Chang, L.: Practice-induced changes of brain function during visual attention: A parametric fMRI study at 4 Tesla. NeuroImage 23(4), 1414–1421 (2004)
Yu, K., Tresp, V., Schwaighofer, A.: Learning Gaussian processes from multiple tasks. In: ICML 2005, pp. 1012–1019. ACM Press, New York (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Madrid-Sánchez, J., Lázaro-Gredilla, M., Figueiras-Vidal, A.R. (2007). A Single Layer Perceptron Approach to Selective Multi-task Learning. In: Mira, J., Álvarez, J.R. (eds) Bio-inspired Modeling of Cognitive Tasks. IWINAC 2007. Lecture Notes in Computer Science, vol 4527. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73053-8_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-73053-8_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73052-1
Online ISBN: 978-3-540-73053-8
eBook Packages: Computer ScienceComputer Science (R0)