Abstract
Streaming Recommender Systems (SRSs) commonly train recommendation models on newly received data only to address user preference drift, i.e., the changing user preferences towards items. However, this practice overlooks the long-term user preferences embedded in historical data. More importantly, the common heterogeneity in data stream greatly reduces the accuracy of streaming recommendations. The reason is that different preferences (or characteristics) of different types of users (or items) cannot be well learned by a unified model. To address these two issues, we propose a Variational and Reservoir-enhanced Sampling based Double-Wing Mixture of Experts framework, called VRS-DWMoE, to improve the accuracy of streaming recommendations. In VRS-DWMoE, we first devise variational and reservoir-enhanced sampling to wisely complement new data with historical data, and thus address the user preference drift issue while capturing long-term user preferences. After that, we propose a Double-Wing Mixture of Experts (DWMoE) model to first effectively learn heterogeneous user preferences and item characteristics, and then make recommendations based on them. Specifically, DWMoE contains two Mixture of Experts (MoE, an effective ensemble learning model) to learn user preferences and item characteristics, respectively. Moreover, the multiple experts in each MoE learn the preferences (or characteristics) of different types of users (or items) where each expert specializes in one underlying type. Extensive experiments demonstrate that VRS-DWMoE consistently outperforms the state-of-the-art SRSs.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Devooght, R., Kourtellis, N., Mantrach, A.: Dynamic matrix factorization with priors on unknown values. In: SIGKDD, pp. 189–198 (2015)
Diaz-Aviles, E., Drumond, L., Schmidt-Thieme, L., Nejdl, W.: Real-time top-n recommendation in social streams. In: RecSys, pp. 59–66 (2012)
Forbes Report. http://forbes.com/sites/benkepes/2015/06/03/30-of-servers-are-sitting-comatose-according-to-research/, Accessed 29 May 2020
Guo, L., Yin, H., Wang, Q., Chen, T., Zhou, A., Quoc Viet Hung, N.: Streaming session-based recommendation. In: SIGKDD, pp. 1569–1577 (2019)
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.S.: Neural collaborative filtering. In: WWW, pp. 173–182 (2017)
He, X., Zhang, H., Kan, M.Y., Chua, T.S.: Fast matrix factorization for online recommendation with implicit feedback. In: SIGIR, pp. 549–558 (2016)
Hou, Y., Yang, N., Wu, Y., Yu, P.S.: Explainable recommendation with fusion of aspect information. World Wide Web 22(1), 221–240 (2018). https://doi.org/10.1007/s11280-018-0558-1
Lefakis, L., Fleuret, F.: Reservoir boosting: between online and offline ensemble learning. In: NIPS, pp. 1412–1420 (2013)
Ma, J., Zhao, Z., Yi, X., et al.: Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In: SIGKDD, pp. 1930–1939 (2018)
Masoudnia, S., Ebrahimpour, R.: Mixture of experts: a literature survey. Artif. Intell. Review 42(2), 275–293 (2012). https://doi.org/10.1007/s10462-012-9338-y
McKinnon, C.D., Schoellig, A.P.: Learning multimodal models for robot dynamics online with a mixture of gaussian process experts. In: ICRA, pp. 322–328 (2017)
Papagelis, M., Rousidis, I., Plexousakis, D., et al.: Incremental collaborative filtering for highly-scalable recommendation algorithms. In: ISMIS, pp. 553–561 (2005)
Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: bayesian personalized ranking from implicit feedback. In: UAI, pp. 452–461 (2009)
Shazeer, N., Mirhoseini, A., et al.: Outrageously large neural networks: the sparsely-gated mixture-of-experts layer. In: ICLR, pp. 1–19 (2017)
Silva, J.G., Carin, L.: Active learning for online bayesian matrix factorization. In: SIGKDD, pp. 325–333 (2012)
Soares, S.G., Araújo, R.: An on-line weighted ensemble of regressor models to handle concept drifts. Eng. Appl. Artif. Intell. 37, 392–406 (2015)
Song, D., Li, Z., Jiang, M., Qin, L., Liao, L.: A novel temporal and topic-aware recommender model. World Wide Web 22(5), 2105–2127 (2018). https://doi.org/10.1007/s11280-018-0595-9
Su, X., Greiner, R., Khoshgoftaar, T.M., Zhu, X.: Hybrid collaborative filtering algorithms using a mixture of experts. In: ICWI, pp. 645–649 (2007)
Vinagre, J., Jorge, A.M., Gama, J.: Fast incremental matrix factorization for recommendation with positive-only feedback. In: UMAP, pp. 459–470 (2014)
Wang, Q., Yin, H., Hu, Z., Lian, D., et al.: Neural memory streaming recommender networks with adversarial training. In: SIGKDD, pp. 2467–2475 (2018)
Wang, S., Cao, L.: Inferring implicit rules by learning explicit and hidden item dependency. IEEE Trans. Syst. Man Cybern. Syst. 50(3), 935–946 (2020)
Wang, S., Hu, L., Wang, Y., Sheng, Q.Z., Orgun, M., Cao, L.: Modeling multi-purpose sessions for next-item recommendations via mixture-channel purpose routing networks. In: IJCAI, pp. 3771–3777 (2019)
Wang, W., Yin, H., Huang, Z., Wang, Q., Du, X., Nguyen, Q.V.H.: Streaming ranking based recommender systems. In: SIGIR, pp. 525–534 (2018)
Xu, Y., Zhu, Y., Shen, Y., Yu, J.: Leveraging app usage contexts for app recommendation: a neural approach. World Wide Web 22(6), 2721–2745 (2018). https://doi.org/10.1007/s11280-018-0543-8
Yin, J., et al.: Online collaborative filtering with implicit feedback. In: DASFAA, pp. 433–448 (2019)
Yu, Y., Gao, Y., Wang, H., Wang, R.: Joint user knowledge and matrix factorization for recommender systems. World Wide Web 21(4), 1141–1163 (2017). https://doi.org/10.1007/s11280-017-0476-7
Acknowledgements
This work was partially supported by Australian Research Council Discovery Projects DP180102378 and DP210101810.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, Y., Wang, S., Wang, Y., Liu, H., Zhang, W. (2020). Double-Wing Mixture of Experts for Streaming Recommendations. In: Huang, Z., Beek, W., Wang, H., Zhou, R., Zhang, Y. (eds) Web Information Systems Engineering – WISE 2020. WISE 2020. Lecture Notes in Computer Science(), vol 12343. Springer, Cham. https://doi.org/10.1007/978-3-030-62008-0_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-62008-0_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-62007-3
Online ISBN: 978-3-030-62008-0
eBook Packages: Computer ScienceComputer Science (R0)