Abstract
Leveraging data from multiple related domains to enhance the model generalization performance is critical for transfer learning in text classification. However, most existing approaches try to separate the features into shared and private spaces regardless of correlations between domains, resulting in the inadequate features sharing among certain most related domains. In this paper, we propose a generic dual-channels multi-task learning framework for multi-domain text classification, which can capture global-shared, local-shared, and private features simultaneously. Our novel framework incorporates Adversarial network and Mixture of experts into a neural network for multi-domain text classification, which is very useful for sharing more features among domains. The extensive experiments on the real-world text classification data-sets across 16 domains demonstrate our proposed approach achieves better results than five state-of-the-art techniques.
Supported by the National Natural Science Foundation of China (No. 61936012, No. 61772324) and Open Project Foundation of Intelligent Information Processing Key Laboratory of Shanxi Province (No. CICIP2018007).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sebastian, R.: Neural Transfer Learning for Natural Language. National University of Ireland, Ireland, Galway (2019)
Jindong W., Yiqiang C., Han Y.: Easy transfer learning by exploitiong intra-domain structures. arXiv preprint arXiv:1904.01376v2 (2019)
Yuhsiang L., Chianyu C., Jean L., et al.: Choosing transfer languages for cross-lingual learning. In: 57th Proceedings of ACL, pp. 3125–3135. ACL, Florence (2019)
Xilun C., Ahmed H. A., Hany H.: Multi-source cross-lingual model transfer: learning what to share. In: 57th Proceedings of ACL, pp. 3098–3112. ACL, Florence (2019)
Liu, Q., Zhang, Y., Liu, J.: Learning domain representation for multi-domain sentiment classification, In: Proceedings of the 2018 Conference on NAACL-HLT, pp. 541–550. ACL, New Orleans (2018)
Pengfei L., Xipeng Q., Xuanjing H.: Recurrent neural network for text classification with multi-task learning. In: 25th Proceedings of IJCAI, pp. 2873–2879. AAAI, California (2016)
Pengfei L., Xipeng Q., Xuanjing H.: Deep multi-task learning with shared memory. In: Proceedings of the 2016 Conference on EMNLP, pp. 118–127. ACL, Austin (2016)
Pengfei L., Xipeng Q., Xuanjing H.: Adversarial multi-task learning for text classification. In: 55th Proceedings of ACL, pp. 1–10. ACL, Vancouver (2017)
Han Z., Shanghang Z., Guanhang W., et al.: Adversarial multiple source domain adaptation. In: 32nd Conference on NIPS, pp. 1–12. Montréal (2018)
Jiang G., Darsh S., Regina B.: Multi-source domain adaptation with mixture of experts. In: Proceedings of the 2018 Conference on EMNLP, pp. 4694–4703. ACL, Brussels (2018)
Noam S., Azalia M., Krzysztof M., et al.: outrageously large neural networks: the sparsely-gated mixture-of-experts layer. In: 5th Proceedings of ICLR, pp. 1–19. Toulon (2017)
Jiaqi M., Zhe Z., Xinyang Y., et al.: Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In: 24th Proceedings of The International Conference on KDD, pp. 1930–1938. SIGKDD, London (2018)
Kim Y., Stratos K.: Domain attention with an ensemble of experts. In: 55th Proceedings of ACL, pp. 643–653. ACL, Vancouver (2017)
Sebastian, R., Parsa, G., John, G.: Knowledge adaptation: teaching to adapt. arXiv preprint arXiv:1702.02052 (2017)
Himanshu S., Manjira S., Shourya R.: Cross-domain text classification with multiple domains and disparate label sets. In: 54th Proceedings of ACL, pp. 641–1650. ACL, Berlin (2016)
Himanshu S., Arun R., Shourya R.: Multi-source iterative adaptation for cross-domain classification. 25th Proceedings of IJCAI, California, pp. 3691–3697 (2016)
Han, Z., Shanghang, Z., Guanhang, W., et al.: Multiple source domain Adaptation with adversarial learning. In: 6th Proceedings of ICLR, Vancouver, pp. 1–10 (2018)
Xilun, C., Claire, C.: Multinomial adversarial networks for multi-domain text classification. In: Proceedings of the 2018 Conference on NAACL-HLT, pp. 1226–1240. ACL, New Orleans (2018)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Yaroslav, G., Evgeniya, U.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17, 1–35 (2016)
Zhou, C., Sun, C., Liu, Z., et al.: A C-LSTM neural network for text classification. Comput. Sci. 1(4), 39–44 (2015)
Rafal J., Wojciech Z., Ilya, S.: An empirical exploration of recurrent network architectures. In: 32nd Proceedings of ICML, pp. 2342–2350. ACM, Lille (2015)
Zhouhan, L., Minwei, F., Cicero, N., et al.: A structured self-attentive sentence embedding. In: 5th Proceedings of ICLR, Toulon, pp. 1–15 (2017)
GloVe, 17 April 2020. https://nlp.stanford.edu/projects/glove/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Su, X., Li, R., Li, X. (2020). Multi-domain Transfer Learning for Text Classification. In: Zhu, X., Zhang, M., Hong, Y., He, R. (eds) Natural Language Processing and Chinese Computing. NLPCC 2020. Lecture Notes in Computer Science(), vol 12430. Springer, Cham. https://doi.org/10.1007/978-3-030-60450-9_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-60450-9_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60449-3
Online ISBN: 978-3-030-60450-9
eBook Packages: Computer ScienceComputer Science (R0)
