Abstract
Modern recommender systems often use sequential neural networks to capture users’ dynamic and evolving intentions from behavior data. However, a user’s different intentions might evolve over time at different speeds. Some user intentions are relatively stable with respect to time (i.e., time-invariant), and simply feeding all behavior data to sequential neural networks might not capture these time-invariant intentions well, since the inductive bias of sequential neural networks could prefer time-varying patterns than time-invariant patterns. In this paper, we propose a novel Dual Intention Decoupling Network (DIDN) framework to model time-invariant patterns and time-varying patterns in users’ behavior data separately, thus both types of patterns could be modeled more accurately. To do so, we first introduce a self-attention based model and a tree-based clustering algorithm to model time-varying and time-invariant patterns respectively, and then combine these two models to generate the overall click-through rate prediction. In the self-attention module, we further introduce a candidate item attention mechanism to implicitly decouple a user’s mixed intentions. Experimental results on three benchmarks show that our DIDN outperforms the state-of-the-art baselines in the topk sequential recommendation task.
This work was partially supported by MoE-CMCC “Artificial Intelligence” Project No. MCM20190701.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
In this paper, we focus on the intention decoupling paradigm. Thus we omit more details about the multi-head self-attention sub-layer [15] for brevity.
- 2.
After computing the multiple interest embeddings and the corresponding multiple matching scores, we use an argmax operator to choose the final matching score.
References
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Rendle, S., Freudenthaler, C., Schmidt-Thieme, L.: Factorizing personalized markov chains for next-basket recommendation. In: Proceedings of the 19th International Conference on World Wide Web, pp. 811–820 (2010)
Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728 (2018)
Higgins, I., et al.: beta-VAE: learning basic visual concepts with a constrained variational framework (2016)
Ye, W., Wang, S., Chen, X., Wang, X., Qin, Z., Yin, D.: Time matters: sequential recommendation with complex temporal information. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1459–1468 (2020)
Ma, C., Ma, L., Zhang, Y., Sun, J., Liu, X., Coates, M.: Memory augmented graph neural networks for sequential recommendation. Proc. AAAI Conf. Artif. Intell. 34(04), 5045–5052 (2020)
Ma, J., Zhou, C., Cui, P., Yang, H., Zhu, W.: Learning disentangled representations for recommendation. arXiv preprint arXiv:1910.14238 (2019)
Ma, J., Cui, P., Kuang, K., Wang, X., Zhu, W.: Disentangled graph convolutional networks. In: International Conference on Machine Learning (PMLR), pp. 4212–4221 (2019)
Wang, X., Jin, H., Zhang, A., He, X., Xu, T., Chua, T.-S.: Disentangled graph collaborative filtering. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1001–1010 (2020)
Zheng, Y., Gao, C., Li, X., He, X., Jin, D., Li, Y.: Disentangling user interest and conformity for recommendation with causal embedding. arXiv preprint arXiv:2006.11011 (2020)
Hidasi, B., Karatzoglou, A.: Recurrent neural networks with top-k gains for session-based recommendations. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 843–852 (2018)
Wang, S., Hu, L., Wang, Y., Sheng, Q.Z., Orgun, M., Cao, L.: Modeling multi-purpose sessions for next-item recommendations via mixture-channel purpose routing networks. In: International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence (2019)
Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015)
Pi, Q., Bian, W., Zhou, G., Zhu, X., Gai, K.: Practice on long sequential user behavior modeling for click-through rate prediction. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2671–2679 (2019)
Li, J., Wang, Y., McAuley, J.: Time interval aware self-attention for sequential recommendation. In: Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 322–330 (2020)
Zhou, G., et al.: Deep interest evolution network for click-through rate prediction. Proc. AAAI Conf. Artif. Intell. 33(01), 5941–5948 (2019)
Chen, S., Moore, J.L., Turnbull, D., Joachims, T.: Playlist prediction via metric embedding. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 714–722 (2012)
McAuley, J., Targett, C., Shi, Q., Van Den Hengel, A.: Image-based recommendations on styles and substitutes. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 43–52 (2015)
Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012)
Cen, Y., Zhang, J., Zou, X., Zhou, C., Yang, H., Tang, J.: Controllable multi-interest framework for recommendation. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2942–2951 (2020)
Tan, Q., et al.: Sparse-interest network for sequential recommendation. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp. 598–606 (2021)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv preprint arXiv:1607.06450 (2016)
He, R., Kang, W.-C., McAuley, J.: Translation-based recommendation. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, pp. 161–169 (2017)
Zhang, W., Chen, T., Wang, J., Yu, Y.: Optimizing top-n collaborative filtering via dynamic negative item sampling. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 785–788 (2013)
Kang, W.-C., McAuley, J.: Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp. 197–206. IEEE (2018)
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4), 1–19 (2015)
Yang, M., Liu, F., Chen, Z., Shen, X., Hao, J., Wang, J.: CausalVAE: disentangled representation learning via neural structural causal models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9593–9602 (2021)
Lian, J., et al.: Multi-interest-aware user modeling for large-scale sequential recommendations. arXiv preprint arXiv:2102.09211 (2021)
Liu, H., Lin, H., Chen, G.: TANTP: conversational emotion recognition using tree-based attention networks with transformer pre-training. In: Karlapalem, K. (ed.) PAKDD 2021. LNCS (LNAI), vol. 12713, pp. 730–742. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-75765-6_58
Shi, S., Ma, W., Zhang, M., Zhang, Y., Ma, S.: Beyond user embedding matrix: learning to hash for modeling large-scale users in recommendation. In: SIGIR 2020: The 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2020)
Batmaz, Z., Yurekli, A., Bilge, A., Kaleli, C.: A review on deep learning for recommender systems: challenges and remedies. Artif. Intell. Rev. 52(1), 1–37 (2019)
He, R., McAuley, J.: Fusing similarity models with markov chains for sparse sequential recommendation. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 191–200. IEEE (2016)
Cho, K., van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: Encoder-decoder approaches. In: Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pp. 103–111 (2014)
Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: International conference on machine learning (PMLR), pp. 1310–1318 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, L., Chen, G. (2023). Disentangling User Intention for Sequential Recommendation with Dual Intention Decoupling Network. In: Wang, X., et al. Database Systems for Advanced Applications. DASFAA 2023. Lecture Notes in Computer Science, vol 13944. Springer, Cham. https://doi.org/10.1007/978-3-031-30672-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-30672-3_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30671-6
Online ISBN: 978-3-031-30672-3
eBook Packages: Computer ScienceComputer Science (R0)