Skip to main content
SpringerLink
Log in

Multi-view improved sequence behavior with adaptive multi-task learning in ranking

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Click through rate (CTR) and Conversion Rate (CVR) are core tasks in e-commerce recommender systems. Sequence behavior and multi-task learning have been widely used in CTR and CVR. Based on the concept of a transformer, we develop a technique of time and space feature representation for the prediction, which can capture high-level information better. In order to formulate user’s different interests from historical sequence behavior, we design multi-task learning to improve multiple objectives simultaneously. It is difficult to turn the super parameters as the tasks increasing. In this paper, we propose an adaptive learning mixture-of-experts approach, which tackles this challenge and can learn super parameters among tasks automatically. It not only saves resources but also improves the performance with cognitive of the model. Furthermore, to enhance the flexibility, we improve the loss function with a constrained joint strategy and introduce RESNET mechanism. We design feature-cross-unit module, augment-expert module, and topK-dispatch module, which assist multi-task learning to improve better. Experiments on public dataset and our library dataset demonstrate the superiority of our model over the state-of-art method. Our method achieves + 2.29% AUC gain in the CTR task and + 1.81% AUC gain in the CVR task, which is a significant improvement and demonstrates the effectiveness of proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Zhou G, Zhu X, Song C, Fan Y, Zhu H, Ma X, Gai K (2018) Deep interest network for click-through rate prediction. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1059–1068

  2. Zhou G, Mou N, Fan Y, Pi Q, Bian W, Zhou C, Gai K (2019) Deep interest evolution network for click-through rate prediction. In: Proceedings of the AAAI conference on artificial intelligence, vol 33(01), pp 5941–5948

  3. Chen Q, Zhao H, Li W, Huang P, Ou W (2019) Behavior sequence transformer for e-commerce recommendation in alibaba. In: Proceedings of the 1st international workshop on deep learning practice for high-dimensional sparse data, pp 1–4

  4. Xiao Z, Yang L, Jiang W, Wei Y, Hu Y, Wang H (2020) Deep multi-interest network for click-through rate prediction. In: Proceedings of the 29th ACM international conference on information and knowledge management, pp 2265–2268

  5. Xu W, He H, Tan M, Li Y, Lang J, Guo D (2020) Deep interest with hierarchical attention network for click-through rate prediction. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 1905–1908

  6. Li X, Wang C, Tong B, Tan J, Zeng X, Zhuang T (2020) Deep time-aware item evolution network for click-through rate prediction. In: Proceedings of the 29th ACM international conference on information and knowledge management, pp 785–794

  7. Xu E, Yu Z, Guo B, Cui H (2021) Core interest network for click-through rate prediction. ACM Trans Knowl Discovery Data (TKDD) 15(2):1–16

    Article  Google Scholar 

  8. Ouyang W, Zhang X, Zhao L, Luo J, Zhang Y, Zou H, Du Y (2020) MiNet: mixed interest network for cross-domain click-through rate prediction. In: Proceedings of the 29th ACM international conference on information and knowledge management, pp 2669–2676

  9. Li C, Liu Z, Wu M, Xu Y, Zhao H, Huang P, Lee DL (2019) Multi-interest network with dynamic routing for recommendation at Tmall. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 2615–2623

  10. Zhou C, Bai J, Song J, Liu X, Zhao Z, Chen X, Gao J (2018) Atrank: an attention-based user behavior modeling framework for recommendation. In: Proceedings of the AAAI conference on artificial intelligence vol 32(1)

  11. Gu Y, Ding Z, Wang S, Zou L, Liu Y, Yin D (2020) Deep multifaceted transformers for multi-objective ranking in large-scale e-commerce recommender systems. In: Proceedings of the 29th ACM international conference on information and knowledge management, pp 2493–2500

  12. Xin S, Ester M, Bu J, Yao C, Li Z, Zhou X, Wang C (2019) Multi-task based sales predictions for online promotions. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 2823–2831

  13. Li P, Li R, Da Q, Zeng AX, Zhang L (2020) Improving multi-scenario learning to rank in e-commerce by exploiting task relationships in the label space. In: Proceedings of the 29th ACM international conference on information and knowledge management, pp 2605–2612

  14. Hui B, Zhang L, Zhou X et al (2022) Personalized recommendation system based on knowledge embedding and historical behavior. J Applied Intell 52(1):954–966

    Article  Google Scholar 

  15. Vandenhende S, Georgoulis S, Van Gansbeke W, Proesmans M, Dai D, Van gool L (2021) Multi-task learning for dense prediction tasks: a Survey. IEEE Trans Pattern Anal Mach Intell

  16. Ma J, Zhao Z, Yi X, Chen J, Hong L, Chi EH (2018) Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1930–1939

  17. Qin Z, Cheng Y, Zhao Z, Chen Z, Metzler D, Qin J (2020) Multitask mixture of sequential experts for user activity streams. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery and data mining, pp 3083–3091

  18. Kumthekar AA, Nath A, Chi EH, Chen J, Wei L, Hong L, Zhao z (2019) Recommending what video to watch next: a multitask ranking system

  19. Ma J, Zhao Z, Chen J, Li A, Hong L, Chi EH (2019) Snr: sub-network routing for flexible parameter sharing in multi-task learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 33(01), pp 216–223

  20. Meng W, Yang D, Xiao Y (2020) Incorporating user micro-behaviors and item knowledge into multi-task learning for session-based recommendation. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 1091–1100

  21. Tang H, Liu J, Zhao M, Gong X (2020) Progressive layered extraction (PLE): a novel multi-task learning (MTL) model for personalized recommendations. In: Fourteenth ACM conference on recommender systems, pp 269–278

  22. Ma X, Zhao L, Huang G, Wang Z, Hu Z, Zhu X, Gai K (2018) Entire space multi-task model: an effective approach for estimating post-click conversion rate. In: The 41st international ACM SIGIR conference on research and development in information retrieval, pp 1137–1140

  23. Wen H, Zhang J, Wang Y, Lv F, Bao W, Lin Q, Yang K (2020) Entire space multi-task modeling via post-click behavior decomposition for conversion rate prediction. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 2377–2386

  24. He K et al, Zhang X, Ren S (2016) Deep residual learning for image Recognition[J]. 2016 IEEE Conf Comput Vis Pattern Recognit (CVPR)

  25. Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need[J]

  26. Covington P, Adams J, Sargin E (2016) Deep neural networks for youtube recommendations. In: Proceedings of the 10th ACM conference on recommender systems, pp 191–198

  27. Wang R, Fu B, Fu G, Wang M (2017) Deep and cross network for ad click predictions. In: Proceedings of the ADKDD’17, pp 1–7

  28. Yan C, Li X, Chen Y et al (2021) JointCTR: a joint CTR prediction framework combining feature interaction and sequential behavior learning. Appl Intell

  29. Liu Z, Yuan B, Ma Y (2021) A multi-task dual attention deep recommendation model using ratings and review helpfulness. Appl Intell

  30. Zhou X, Li Y (2021) Large-scale modeling of mobile user click behaviors using deep learning[C] fifteenth ACM conference on recommender systems, pp 473–483

  31. Zhang K, Qian H, Cui Q, Qi L, Li L, Zhou J, Ma J, Chen E (2021) Multi-interactive attention network for fine-grained feature learning in CTR prediction. In: Proceedings of the 14th ACM international conference on web search and data mining. Association for computing machinery, New York, pp 984–992

  32. Li C, Liu Z, Wu M, Xu Y, Zhao H, Huang P, Kang G, Chen Q, Li W, Lee DL (2019) Multi-interest network with dynamic routing for recommendation at Tmall. In: Proceedings of the 28th ACM international conference on information and knowledge management (CIKM ’19). Association for computing machinery, New York, pp 2615–2623

  33. Lv F, Jin T, Yu C, Sun F, Lin Q, Yang K, Ng W (2019) SDM: sequential deep matching model for online large-scale recommender system. In: Proceedings of the 28th ACM international conference on information and knowledge management (CIKM ’19). Association for computing machinery, New York, pp 2635–2643

  34. Xie R, Ling C, Wang Y, Wang R, Xia F, Lin L Deep feedback network for recommendation. Proceedings of the twenty-ninth international joint conference on artificial intelligence (IJCAI-2020) pp 2519–2525

  35. Qin Z, Cheng Y, Zhao Z, Chen Z, Metzler D, Qin J (2020) Multitask mixture of sequential experts for user activity streams. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery and data mining. Association for computing machinery, New York, pp 3083–3091

  36. Liu M, Cai S, Lai Z, Qiu L, Hu Z, Ding Yi (2021) A joint learning model for click-through prediction in display advertising. Neurocomputing 445:206–219

    Article  Google Scholar 

  37. Huang W, Wu J, Song W et al (2022) Cross attention fusion for knowledge graph optimized recommendation. Appl Intell

  38. Yang C, Pan J, Gao X et al (2022) Cross-task knowledge distillation in multi-task recommendation[J]. arXiv:2202.09852

  39. Ioffe S, Szegedy C (2015) Batch Normalization: accelerating deep network training by reducing internal covariate shift[J] JMLR .org

Download references

Acknowledgements

This work is supported by the Science and Technology Innovation 2030-New Generation Artificial Intelligence major project (No.2020AAA0108703). We would also like to thank the anonymous reviewers for their helpful comments.

Author information

Authors and Affiliations

  1. Department of Computer, School of Computer and Communication Engineering, University of Science and Technology Beijing (USTB), Beijing, 100083, China

    Yingshuai Wang, Dezheng Zhang & Aziguli Wulamu

  2. Beijing Key Laboratory of Knowledge Engineering for Materials Science, University of Science and Technology Beijing (USTB), Beijing, 100083, China

    Yingshuai Wang, Dezheng Zhang & Aziguli Wulamu

Authors
  1. Yingshuai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dezheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aziguli Wulamu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingshuai Wang.

Ethics declarations

Conflict of Interests

There are no conflicts of interest regarding the publication of this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Zhang, D. & Wulamu, A. Multi-view improved sequence behavior with adaptive multi-task learning in ranking. Appl Intell 53, 13158–13177 (2023). https://doi.org/10.1007/s10489-022-04088-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04088-w

Keywords

Search

Navigation