Abstract
The recommendation task with a textual corpus aims to model customer preferences from both user feedback and item textual descriptions. It is highly desirable to explore a very deep neural network to capture the complicated nonlinear preferences. However, training a deeper recommender is not as effortless as simply adding layers. A deeper recommender suffers from the gradient vanishing/exploding issue and cannot be easily trained by gradient-based methods. Moreover, textual descriptions probably contain noisy word sequences. Directly extracting feature vectors from them can harm the recommender’s performance. To overcome these difficulties, we propose a new recommendation method named the HighwAy recoMmender (HAM). HAM explores a highway mechanism to make gradient-based training methods stable. A multi-head attention mechanism is devised to automatically denoise textual information. Moreover, a block coordinate descent method is devised to train a deep neural recommender. Empirical studies show that the proposed method outperforms state-of-the-art methods significantly in terms of accuracy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adomavicius G, Tuzhilin A, 2005. Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng, 17(6):734–749. https://doi.org/10.1109/TKDE.2005.99
Bansal T, Belanger D, McCallum A, 2016. Ask the GRU: multi-task learning for deep text recommendations. Proc 10th ACM Conf on Recommender Systems, p. 107–114. https://doi.org/10.1145/2959100.2959180
Bennett J, Lanning S, 2007. The Netflix prize. Proc KDD Cup and Workshop, p.35.
Cai XY, Han J, Yang L, 2018. Generative adversarial network based heterogeneous bibliographic network representation for personalized citation recommendation. 32nd AAAI Conf on Artificial Intelligence, p.5747–5754.
Chorowski JK, Bahdanau D, Serdyuk D, et al., 2015. Attention-based models for speech recognition. Proc 30th Int Conf on Neural Information Processing Systems, p.577–585.
Devooght R, Bersini H, 2016. Collaborative filtering with recurrent neural networks. https://arxiv.org/abs/1608.07400
Goodfellow I, Bengio Y, Courville A, 2016. Deep Learning. MIT Press, Cambridge, MA.
Gopalan PK, Charlin L, Blei D, 2014. Content-based recommendations with Poisson factorization. Proc Advances in Neural Information Processing Systems, p.3176–3184.
Grčar M, Mladenic D, Fortuna B, et al., 2005. Data sparsity issues in the collaborative filtering framework. Proc 7th Int Workshop on Knowledge Discovery on the Web, p.58–76.
He XN, Liao LZ, Zhang HW, et al., 2017. Neural collaborative filtering. Proc 26th Int Conf on World Wide Web, p.173–182. https://doi.org/10.1145/3038912.3052569
Hsieh CK, Yang L, Cui Y, et al., 2017. Collaborative metric learning. Proc 26th Int Conf on World Wide Web, p.193–201.
Jin M, Luo X, Zhu H, et al., 2018. Combining deep learning and topic modeling for review understanding in context-aware recommendation. Proc Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p.1605–1614.
Kim D, Park C, Oh J, et al., 2016. Convolutional matrix factorization for document context-aware recommendation. Proc 10th ACM Conf on Recommender Systems, p.233–240. https://doi.org/10.1145/2959100.2959165
Kiros R, Zhu Y, Salakhutdinov RR, et al., 2015. Skip-thought vectors. Advances in Neural Information Processing Systems, p.3294–3302.
Koren Y, 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. Proc 14th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, p.426–434. https://doi.org/10.1145/1401890.1401944
Koren Y, Bell R, Volinsky C, 2009. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37. https://doi.org/10.1109/MC.2009.263
Levy O, Goldberg Y, 2014. Neural word embedding as implicit matrix factorization. Proc 27th Int Conf on Neural Information Processing Systems, p.2177–2185.
Linden G, Smith B, York J, 2003. Amazon.com recommendations: item-to-item collaborative filtering. IEEE Int Comput, 7(1):76–80. https://doi.org/10.1109/MIC.2003.1167344
Liu CH, Jin T, Hoi SCH, et al., 2017. Collaborative topic regression for online recommender systems: an online and Bayesian approach. Mach Learn, 106(5):651–670. https://doi.org/10.1007/s10994-016-5599-z
McLaughlin MR, Herlocker JL, 2004. A collaborative filtering algorithm and evaluation metric that accurately model the user experience. Proc 27th Annual Int ACM SIGIR Conf on Research and Development in Information Retrieval, p.329–336. https://doi.org/10.1145/1008992.1009050
Mhaskar H, Liao Q, Poggio T, 2017. When and why are deep networks better than shallow ones? Proc 31st AAAI Conf on Artificial Intelligence, p.2343–2349.
Neyshabur B, Bhojanapalli S, McAllester D, et al., 2017. Exploring generalization in deep learning. Proc 30th Conf on Advances in Neural Information Processing Systems, p.5947–5956.
Paterek A, 2007. Improving regularized singular value decomposition for collaborative filtering. Proc KDD Cup and Workshop, p.5–8.
Raghu M, Poole B, Kleinberg J, et al., 2017. On the expressive power of deep neural networks. Proc 34th Int Conf on Machine Learning, p.2847–2854.
Rendle S, Freudenthaler C, Gantner Z, et al., 2009. BPR: Bayesian personalized ranking from implicit feedback. Proc 25th Conf on Uncertainty in Artificial Intelligence, p.452–461.
Ruder S, 2017. An overview of multi-task learning in deep neural networks. https://arxiv.org/abs/1706.05098
Salakhutdinov R, Mnih A, 2007. Probabilistic matrix factorization. Proc 20th Int Conf on Neural Information Processing Systems, p.1257–1264.
Salakhutdinov R, Mnih A, Hinton G, 2007. Restricted Boltzmann machines for collaborative filtering. Proc 24th Int Conf on Machine Learning, p.791–798. https://doi.org/10.1145/1273496.1273596
Shoja BM, Tabrizi N, 2019. Customer reviews analysis with deep neural networks for e-commerce recommender systems. IEEE Access, 7:119121–119130. https://doi.org/10.1109/ACCESS.2019.2937518
Srebro N, Rennie J, Jaakkola TS, 2004. Maximum-margin matrix factorization. Conf on Neural Information Processing Systems, p.1329–1336.
Srivastava RK, Greff K, Schmidhuber J, 2015. Training very deep networks. Advances in Neural Information Processing Systems, p.2377–2385.
Strub F, Mary J, 2015. Collaborative filtering with stacked denoising autoencoders and sparse inputs. NIPS Workshop on Machine Learning for e-Commerce, p.1–8.
Vaswani A, Shazeer N, Parmar N, et al., 2017. Attention is all you need. 31st Conf on Neural Information Processing Systems, p.5998–6008.
Wang C, Blei DM, 2011. Collaborative topic modeling for recommending scientific articles. Proc 17th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, p.448–456. https://doi.org/10.1145/2020408.2020480
Wang H, Wang NY, Yeung DY, 2015. Collaborative deep learning for recommender systems. Proc 21st ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, p.1235–1244. https://doi.org/10.1145/2783258.2783273
Wang H, Shi XJ, Yeung DY, 2016. Collaborative recurrent autoencoder: recommend while learning to fill in the blanks. Proc 30th Int Conf on Neural Information Processing Systems, p.415–423.
Wu Y, DuBois C, Zheng AX, et al., 2016. Collaborative denoising auto-encoders for top-N recommender systems. Proc 9th ACM Int Conf on Web Search and Data Mining, p.153–162. https://doi.org/10.1145/2835776.2835837
Author information
Authors and Affiliations
Corresponding author
Additional information
Project supported by the Key R&D Program of Zhejiang Province, China (No. 2020C01024) and the National Key R&D Program (No. 2016YFB1001503)
Contributors
Cheng-wei WANG, Teng-fei ZHOU, Chen CHEN, Tian-lei HU, and Gang CHEN discussed the idea. Cheng-wei WANG designed the research. Chen CHEN processed the data. Teng-fei ZHOU wrote the code and conducted the experiments. Cheng-wei WANG drafted the manuscript. Teng-fei ZHOU, Chen CHEN, and Tian-lei HU helped organize the manuscript. Gang CHEN revised and finalized the paper.
Compliance with ethics guidelines
Cheng-wei WANG, Teng-fei ZHOU, Chen CHEN, Tianlei HU, and Gang CHEN declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Wang, Cw., Zhou, Tf., Chen, C. et al. HAM: a deep collaborative ranking method incorporating textual information. Front Inform Technol Electron Eng 21, 1206–1216 (2020). https://doi.org/10.1631/FITEE.1900382
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.1900382