Skip to main content
SpringerLink
Log in

Learning multi-tasks with inconsistent labels by using auxiliary big task

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Multi-task learning is to improve the performance of the model by transferring and exploiting common knowledge among tasks. Existing MTL works mainly focus on the scenario where label sets among multiple tasks (MTs) are usually the same, thus they can be utilized for learning across the tasks. However, the real world has more general scenarios in which each task has only a small number of training samples and their label sets are just partially overlapped or even not. Learning such MTs is more challenging because of less correlation information available among these tasks. For this, we propose a framework to learn these tasks by jointly leveraging both abundant information from a learnt auxiliary big task with sufficiently many classes to cover those of all these tasks and the information shared among those partially-overlapped tasks. In our implementation of using the same neural network architecture of the learnt auxiliary task to learn individual tasks, the key idea is to utilize available label information to adaptively prune the hidden layer neurons of the auxiliary network to construct corresponding network for each task, while accompanying a joint learning across individual tasks. Extensive experimental results demonstrate that our proposed method is significantly competitive compared to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Khattar A, Hegde S, Hebbalaguppe R. Cross-domain multi-task learning for object detection and saliency estimation. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2021, 3634–3643

  2. Al-Qaisi L, Hassonah M A, Al-Zoubi M M, Al-Zoubi A M. A review of evolutionary data clustering algorithms for image segmentation. In: Aljarah I, Faris H, Mirjalili S, eds. Evolutionary Data Clustering: Algorithms and Applications. Singapore: Springer, 2021, 201–214

  3. Chaturvedi I, Su C L, Welsch R E. Fuzzy aggregated topology evolution for cognitive multi-tasks. Cognitive Computation, 2021, 13(1): 96–107

    Article  Google Scholar 

  4. Ravanelli M, Zhong J, Pascual S, Swietojanski P, Monteiro J, Trmal J, Bengio Y. Multi-task self-supervised learning for robust speech recognition. In: Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing. 2020, 6989–6993

  5. Li C, Wang B, Zhang S, Liu Y, Song R, Cheng J, Chen X. Emotion recognition from EEG based on multi-task learning with capsule network and attention mechanism. Computers in Biology and Medicine, 2022, 143: 105303

    Article  Google Scholar 

  6. Cheng B, Zhu B, Pu S. Multi-auxiliary domain transfer learning for diagnosis of MCI conversion. Neurological Sciences, 2022, 43(3): 1721–1739

    Article  Google Scholar 

  7. Ruder S, Bingel J, Augenstein I, Søgaard A. Sluice networks: learning what to share between loosely related tasks. 2017, arXiv preprint arXiv: 1705.08142v1

  8. Gong T, Zheng X, Lu X. Remote sensing scene classification with multi-task learning. In: Wang L, Wu Y, Gong J, eds. Proceedings of the 7th China High Resolution Earth Observation Conference (CHREOC 2020). Singapore: Springer, 2022, 403–418

    Chapter  Google Scholar 

  9. Islam M M, Iqbal T. Mumu: cooperative multitask learning-based guided multimodal fusion. In: Proceedings of the 36th Conference on Artificial Intelligence. 2022, 1043–1051

  10. Long M, Cao Z, Wang J, Yu P S. Learning multiple tasks with multilinear relationship networks. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 1593–1602

  11. Ma J, Zhao Z, Yi X, Chen J, Hong L, Chi E H. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 1930–1939

  12. Feng Q, Yao J, Zhong Y, Li P, Pan Z. Learning twofold heterogeneous multi-task by sharing similar convolution kernel pairs. Knowledge-Based Systems, 2022, 252: 109396

    Article  Google Scholar 

  13. Wu S, Zhang H R, Ré C. Understanding and improving information transfer in multi-task learning. In: Proceedings of the 8th International Conference on Learning Representations. 2020, 26–30

  14. Evgeniou T, Pontil M. Regularized multi-task learning. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2004, 109–117

  15. Honorio J, Samaras D. Multi-task learning of Gaussian graphical models. In: Proceedings of the 27th International Conference on Machine Learning. 2010, 447–454

  16. Liu Q, Li X, He Z, Fan N, Yuan D, Liu W, Liang Y. Multi-task driven feature models for thermal infrared tracking. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 11604–11611

  17. Wang J, Zhang S, Wang Y, Zhu Z. Learning efficient multi-task stereo matching network with richer feature information. Neurocomputing, 2021, 421: 151–160

    Article  Google Scholar 

  18. Guo P, Deng C, Xu L, Huang X, Zhang Y. Deep multi-task augmented feature learning via hierarchical graph neural network. In: Proceedings of European Conference on Machine Learning and Knowledge Discovery in Databases. 2021, 538–553

  19. Vandenhende S, Georgoulis S, van Gansbeke W, Proesmans M, Dai D, van Gool L. Multi-task learning for dense prediction tasks: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(7): 3614–3633

    Google Scholar 

  20. Shen Z, Cui C, Huang J, Zong J, Chen M, Yin Y. Deep adaptive feature aggregation in multi-task convolutional neural networks. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 2213–2216

  21. Yadav S, Chauhan J, Sain J P, Thirunarayan K, Sheth A, Schumm J. Identifying depressive symptoms from tweets: figurative language enabled multitask learning framework. In: Proceedings of the 28th International Conference on Computational Linguistics. 2020, 696–709

  22. Sun X, Panda R, Feris R, Saenko K. Adashare: learning what to share for efficient deep multi-task learning. In: Proceedings of the 34th Conference on Neural Information Processing Systems. 2020, 8728–8740

  23. Sun T, Shao Y, Li X, Liu P, Yan H, Qiu X, Huang X. Learning sparse sharing architectures for multiple tasks. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 8936–8943

  24. Verboven S, Hafeez Chaudhary M, Berrevoets J, Verbeke W. HydaLearn: highly dynamic task weighting for multi-task learning with auxiliary tasks. 2008, arXiv preprint arXiv: 2008.11643

  25. Sanh V, Wolf T, Ruder S. A hierarchical multi-task approach for learning embeddings from semantic tasks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 6949–6956

  26. Baxter J. A Bayesian/information theoretic model of learning to learn via multiple task sampling. Machine Learning, 1997, 28(1): 7–39

    Article  MATH  Google Scholar 

  27. Ruder S, Bingel J, Augenstein I, Søgaard A. Latent multi-task architecture learning. In: In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 4822–4829

  28. Strezoski G, Noord N, Worring M. Many task learning with task routing. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 1375–1384

  29. Fernando C, Banarse D, Blundell C, Zwols Y, Ha D, Rusu A A, Pritzel A, Wierstra D. PathNet: Evolution channels gradient descent in super neural networks. 2017, arXiv preprint arXiv: 1701.08734

  30. Pironkov G, Wood S U, Dupont S. Hybrid-task learning for robust automatic speech recognition. Computer Speech & Language, 2020, 64: 101103

    Article  Google Scholar 

  31. Cao P, Shan X, Zhao D, Huang M, Zaiane O. Sparse shared structure based multi-task learning for MRI based cognitive performance prediction of Alzheimer’s disease. Pattern Recognition, 2017, 72: 219–235

    Article  Google Scholar 

  32. Lee S, Son Y. Multitask learning with single gradient step update for task balancing. Neurocomputing, 2022, 467: 442–453

    Article  Google Scholar 

  33. Søgaard A, Goldberg Y. Deep multi-task learning with low level tasks supervised at lower layers. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 231–235

  34. Fan J, Zhao T, Kuang Z, Zheng Y, Zhang J, Yu J, Peng J. HD-MTL: hierarchical deep multi-task learning for large-scale visual recognition. IEEE Transactions on Image Processing, 2017, 26(4): 1923–1938

    Article  MathSciNet  MATH  Google Scholar 

  35. Ott F, Rügamer D, Heublein L, Bischl B, Mutschler C. Joint classification and trajectory regression of online handwriting using a multi-task learning approach. In: Proceedings of 2022 IEEE/CVF Winter Conference on Applications of Computer Vision. 2022, 1244–1254

  36. Zhang C, Li Y, Du N, Fan W, Yu P S. Joint slot filling and intent detection via capsule neural networks. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019, 5259–5267

  37. Li X C, Zhan D C. FedRS: federated learning with restricted softmax for label distribution non-IID data. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 995–1005

  38. Wang Y, Zhang Z, Hao W, Song C. Attention guided multiple source and target domain adaptation. IEEE Transactions on Image Processing, 2021, 30: 892–906

    Article  Google Scholar 

  39. Xue W. Weighted feature-task-aware regularization learner for multitask learning. Pattern Analysis and Applications, 2020, 23(1): 253–263

    Article  MathSciNet  Google Scholar 

  40. Zhang J, Miao J, Zhao K, Tian Y. Multi-task feature selection with sparse regularization to extract common and task-specific features. Neurocomputing, 2019, 340: 76–89

    Article  Google Scholar 

  41. Shao W, Peng Y, Zu C, Wang M, Zhang D, The Alzheimer’s Disease Neuroimaging Initiative. Hypergraph based multi-task feature selection for multimodal classification of Alzheimer’s disease. Computerized Medical Imaging and Graphics, 2020, 80: 101663

    Article  Google Scholar 

  42. Li L, Pan X, Yang H, Liu Z, He Y, Li Z, Fan Y, Cao Z, Zhang L. Multitask deep learning for fine-grained classification and grading in breast cancer histopathological images. Multimedia Tools and Applications, 2020, 79(21–22): 14509–14528

    Article  Google Scholar 

  43. Zheng Z, Wang Y, Dai Q, Zheng H, Wang D. Metadata-driven task relation discovery for multi-task learning. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 4426–4432

  44. Yan C, Xu J, Xie J, Cai C, Lu H. Prior-aware CNN with multi-task learning for colon images analysis. In: Proceedings of the 17th IEEE International Symposium on Biomedical Imaging. 2020, 254–257

  45. Misra I, Shrivastava A, Gupta A, Hebert M. Cross-stitch networks for multi-task learning. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 3994–4003

  46. Duan R, Chen N F. Unsupervised feature adaptation using adversarial multi-task training for automatic evaluation of children’s speech. In: Proceedings of the 21st Annual Conference of the International Speech Communication Association. 2020, 3037–3041

  47. Augenstein I, Søgaard A. Multi-task learning of keyphrase boundary classification. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 341–346

  48. Rai P, Daumé III H. Infinite predictor subspace models for multitask learning. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. 2010, 613–620

  49. Zhou L, Cui Z, Xu C, Zhang Z, Wang C, Zhang T, Yang J. Pattern-structure diffusion for multi-task learning. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 4513–4522

  50. Wang Y, Luo X, Ding L, Fu S, Hu S. Multi-task non-negative matrix factorization for visual object tracking. Pattern Analysis and Applications, 2020, 23(1): 493–507

    Article  MathSciNet  Google Scholar 

  51. Jeong J Y, Jun C H. Sparse tensor decomposition for multi-task interaction selection. In: Proceedings of 2019 IEEE International Conference on Big Knowledge. 2019, 105–114

  52. Huang F, Qiu Y, Li Q, Liu S, Ni F. Predicting drug-disease associations via multi-task learning based on collective matrix factorization. Frontiers in Bioengineering and Biotechnology, 2020, 8: 218

    Article  Google Scholar 

  53. Zhang Y, Zhang Y, Wang W. Deep multi-task learning via generalized tensor trace norm. 2002, arXiv preprint arXiv: 2002.04799

  54. Chen Z, Lei H, Zhao Y, Huang Z, Xiao X, Lei Y, Tan E L, Lei B. Template-oriented multi-task sparse low-rank learning for parkinson’s diseases diagnosis. In: Proceedings of the 3rd International Workshop on PRedictive Intelligence in MEdicine. 2020, 178–187

  55. Wu X, Zhang X, Cen Y. Multi-task joint sparse and low-rank representation target detection for hyperspectral image. IEEE Geoscience and Remote Sensing Letters, 2019, 16(11): 1756–1760

    Article  Google Scholar 

  56. Zhang Y, Yang Q. A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering, 2021

  57. He K, Chen X, Xie S, Li Y, Dollár P, Girshick R. Masked autoencoders are scalable vision learners. In: Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 15979–15988

  58. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations. 2015, 1556

  59. Jaworek-Korjakowska J, Kleczek P, Gorgon M. Melanoma thickness prediction based on convolutional neural network with VGG-19 model transfer learning. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2019, 2748–2756

  60. Gao Y, Ma J, Zhao M, Liu W, Yuille A L. Nddr-CNN: layerwise feature fusing in multi-task CNNs by neural discriminative dimensionality reduction. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 3200–3209

  61. Heuer F, Mantowsky S, Bukhari S S, Schneider G. Multitask-centerNet (MCN): efficient and diverse multitask learning using an anchor free approach. In: Proceedings of 2021 IEEE/CVF International Conference on Computer Vision Workshops. 2021, 997–1005

Download references

Acknowledgements

This work was supported by the NSFC (Grant No. 61672281), and the Key Program of NSFC (No. 61732006).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, China

    Quan Feng & Songcan Chen

  2. MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, China

    Quan Feng & Songcan Chen

Authors
  1. Quan Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Songcan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Songcan Chen.

Additional information

Quan Feng received his B.S. degree in Information and Computing Science from Nanchang Hangkong University, China in 2008. In 2011, he completed his MSc degree in computer science and technique at Nanchang Hangkong University, China. He is currently a Ph.D candidate at the Key Laboratory of Model Analysis and Machine Intelligence, Ministry of Industry and Information Technology, Nanjing University of Aeronautics and Astronautics, China. His research interests include pattern recognition, machine learning and multi-task learning.

Songcan Chen received the BS degree in mathematics from Hangzhou University (now merged into Zhejiang University), China in 1983, and the MS degree in computer applications from Shanghai Jiao Tong University, China in 1985, and then worked with NUAA in January 1986. He received the PhD degree in communication and information systems from the Nanjing University of Aeronautics and Astronautics (NUAA), China in 1997. Since 1998, as a fulltime professor, he has been with the College of Computer Science & Technology, NUAA. His research interests include pattern recognition, machine learning, and neural computing. He is also an IAPR and CCAI Fellow.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, Q., Chen, S. Learning multi-tasks with inconsistent labels by using auxiliary big task. Front. Comput. Sci. 17, 175342 (2023). https://doi.org/10.1007/s11704-022-2251-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-022-2251-x

Keywords

Search

Navigation