Advertisement

Efficient Group Learning with Hypergraph Partition in Multi-task Learning

  • Quanming Yao
  • Xiubao Jiang
  • Mingming Gong
  • Xinge You
  • Yu Liu
  • Duanquan Xu
Part of the Communications in Computer and Information Science book series (CCIS, volume 321)

Abstract

Recently, wide concern has been aroused in multi-task learning (MTL) area, which assumes that affinitive tasks should own similar parameter representation so that joint learning is both appropriate and reciprocal. Researchers also find that imposing similar parameter representation constraint on dissimilar tasks may be harmful to MTL. However, it’s difficult to determine which tasks are similar. Z Kang et al [1] proposed to simultaneously learn the groups and parameters to address this problem. But the method is inefficient and cannot scale to large data. In this paper, using the property of the parameter matrix, we describe the group learning process as permuting the parameter matrix into a block diagonal matrix, which can be modeled as a hypergraph partition problem. The optimization algorithm scales well to large data. Extensive experiments demonstrate that our method is advantageous over existing MTL methods in terms of accuracy and efficiency.

Keywords

multi-task learning sparse matrix permutation hypergraph partitioning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Kang, Z., Grauman, K., Sha, F.: Learning with whom to share in multi-task feature learning (2011)Google Scholar
  2. 2.
    Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. Ma-chine Learning 73(3), 243–272 (2008)Google Scholar
  3. 3.
    Bakker, B., Heskes, T.: Task clustering and gating for bayesian multitask learning. The Journal of Machine Learning Research 4, 83–99 (2003)Google Scholar
  4. 4.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2272–2279. IEEE (2009)Google Scholar
  5. 5.
    Kolar, M., Lafferty, J., Wasserman, L.: Union support recovery in multi-task learning. Journal of Machine Learning Research 12, 2415–2435 (2011)MathSciNetGoogle Scholar
  6. 6.
    Çatalyürek, Ü.V., Aykanat, C.: Patoh: partitioning tool for hypergraphs. Users guide (1999)Google Scholar
  7. 7.
    Aykanat, C., Pinar, A., Catalyurek, U.V.: Permuting sparse rectangular matrices into block-diagonal form. SIAM Journal on Scienti c Computing 25(6), 1860–1879 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Lengauer, T.: Combinatorial algorithms for integrated circuit layout. John Wiley & Sons, Inc. (1990)Google Scholar
  9. 9.
    LeCun, Y., Cortes, C.: The mnist database of handwritten digits (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Quanming Yao
    • 1
  • Xiubao Jiang
    • 1
  • Mingming Gong
    • 1
  • Xinge You
    • 1
  • Yu Liu
    • 1
  • Duanquan Xu
    • 1
  1. 1.Department of Electronics and Information EngineeringHuazhong University of Science TechnologyWuhanChina

Personalised recommendations