Efficient Group Learning with Hypergraph Partition in Multi-task Learning
Recently, wide concern has been aroused in multi-task learning (MTL) area, which assumes that affinitive tasks should own similar parameter representation so that joint learning is both appropriate and reciprocal. Researchers also find that imposing similar parameter representation constraint on dissimilar tasks may be harmful to MTL. However, it’s difficult to determine which tasks are similar. Z Kang et al  proposed to simultaneously learn the groups and parameters to address this problem. But the method is inefficient and cannot scale to large data. In this paper, using the property of the parameter matrix, we describe the group learning process as permuting the parameter matrix into a block diagonal matrix, which can be modeled as a hypergraph partition problem. The optimization algorithm scales well to large data. Extensive experiments demonstrate that our method is advantageous over existing MTL methods in terms of accuracy and efficiency.
Keywordsmulti-task learning sparse matrix permutation hypergraph partitioning
Unable to display preview. Download preview PDF.
- 1.Kang, Z., Grauman, K., Sha, F.: Learning with whom to share in multi-task feature learning (2011)Google Scholar
- 2.Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. Ma-chine Learning 73(3), 243–272 (2008)Google Scholar
- 3.Bakker, B., Heskes, T.: Task clustering and gating for bayesian multitask learning. The Journal of Machine Learning Research 4, 83–99 (2003)Google Scholar
- 4.Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2272–2279. IEEE (2009)Google Scholar
- 6.Çatalyürek, Ü.V., Aykanat, C.: Patoh: partitioning tool for hypergraphs. Users guide (1999)Google Scholar
- 8.Lengauer, T.: Combinatorial algorithms for integrated circuit layout. John Wiley & Sons, Inc. (1990)Google Scholar
- 9.LeCun, Y., Cortes, C.: The mnist database of handwritten digits (1998)Google Scholar