Advertisement

Frontiers of Computer Science

, Volume 11, Issue 6, pp 923–936 | Cite as

User behavior modeling for better Web search ranking

  • Yiqun Liu
  • Chao Wang
  • Min Zhang
  • Shaoping Ma
Review Article

Abstract

Modern search engines record user interactions and use them to improve search quality. In particular, user click-through has been successfully used to improve clickthrough rate (CTR), Web search ranking, and query recommendations and suggestions. Although click-through logs can provide implicit feedback of users’ click preferences, deriving accurate absolute relevance judgments is difficult because of the existence of click noises and behavior biases. Previous studies showed that user clicking behaviors are biased toward many aspects such as “position” (user’s attention decreases from top to bottom) and “trust” (Web site reputations will affect user’s judgment). To address these problems, researchers have proposed several behavior models (usually referred to as click models) to describe users? practical browsing behaviors and to obtain an unbiased estimation of result relevance. In this study, we review recent efforts to construct click models for better search ranking and propose a novel convolutional neural network architecture for building click models. Compared to traditional click models, our model not only considers user behavior assumptions as input signals but also uses the content and context information of search engine result pages. In addition, our model uses parameters from traditional click models to restrict the meaning of some outputs in our model’s hidden layer. Experimental results show that the proposed model can achieve considerable improvement over state-of-the-art click models based on the evaluation metric of click perplexity.

Keywords

user behavior click model Web search 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos. 61622208, 61732008, 61532011). It is also partly supported by Tsinghua University Initiative Scientific Research Program (2014Z21032) and the National Key Basic Research Program of China (973 Program) (2015CB358700).

Supplementary material

11704_2017_6518_MOESM1_ESM.ppt (3 mb)
Supplementary material, approximately 3078 KB.

References

  1. 1.
    Joachims T, Granka L, Pan B, Hembrooke H, Gay G. Accurately interpreting clickthrough data as implicit feedback. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2005, 154–161Google Scholar
  2. 2.
    Craswell N, Zoeter O, Taylor M, Ramsey B. An experimental comparison of click position-bias models. In: Proceedings of ACM International Conference on Web Search and Data Mining. 2008, 87–94Google Scholar
  3. 3.
    Yue Y S, Patel R, Roehrig H. Beyond position bias: examining result attractiveness as a source of presentation bias in clickthrough data. In: Proceedings of the 19th ACM International Conference onWorldWide Web. 2010, 1011–1018Google Scholar
  4. 4.
    Wang C, Liu Y Q, Zhang M, Ma S P, Zheng M H, Qian J, Zhang K. Incorporating vertical results into search click models. In: Proceedings of the 36th ACM International ACM SIGIR Conference on Research and Development in Information Retrieval. 2013, 503–512Google Scholar
  5. 5.
    Guo F, Liu C, Wang Y M. Efficient multiple-click models in web search. In: Proceedings of the 2nd ACM International Conference on Web Search and Data Mining. 2009, 124–131CrossRefGoogle Scholar
  6. 6.
    Dupret G E, Piwowarski B. A user browsing model to predict search engine click data from past observations. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2008, 331–338Google Scholar
  7. 7.
    Chapelle O, Zhang Y. A dynamic Bayesian network click model for Web search ranking. In: Proceedings of the 18th ACM International Conference on World Wide Web. 2009, 1–10Google Scholar
  8. 8.
    Liu Z Y, Liu YQ, Zhou K, ZhangM, Ma S P. Influence of vertical result in Web search examination. In: Proceedings of the 38th International ACMSIGIR Conference on Research and Development in Information Retrieval. 2015, 193–202Google Scholar
  9. 9.
    Wang C, Liu Y Q, Wang M, Zhou K, Nie J Y, Ma S P. Incorporating non-sequential behavior into click models. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2013, 283–292Google Scholar
  10. 10.
    Kaisser M, Hearst M, Lowe J. Improving search results quality by customizing summary lengths. In: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technlogies (ACLHLT’08). 2008Google Scholar
  11. 11.
    Kanungo T, Orr D. Predicting the readability of short Web summaries. In: Proceedings of the International Conference on Web Search and Web Data Mining. 2009, 325–326Google Scholar
  12. 12.
    Carbonell J, Goldstein J. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 1998, 335–336Google Scholar
  13. 13.
    Clarke C L A, Kolla M, Cormack G V, Vechtomova O, Ashkan A, Büttcher S, MacKinnon I. Novelty and diversity in information retrieval evaluation. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2008, 659–666Google Scholar
  14. 14.
    Wang H N, Zhai C X, Dong A L, Chang Y. Content-aware click modeling. In: Proceedings of the 23rd International World-Wide Web Conference. 2013, 175–176Google Scholar
  15. 15.
    Schmidhuber J. Deep learning in neural networks: an overview. Neural Networks, 2015, 61: 85–117CrossRefGoogle Scholar
  16. 16.
    Yu L, Hermann K N, Blunsom P, Pulman S. Deep learning for answer sentence selection. In: Proceedings of NIPS Deep Learning and Representation Learning Workshop. 2014, 393–402Google Scholar
  17. 17.
    Kim Y. Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014, 1746–1751CrossRefGoogle Scholar
  18. 18.
    Huang P S, He X D, Gao J F, Deng L, Acero A, Heck L. Learning deep structured semantic models for Web search using clickthrough data. In: Proceedings of the 22nd ACM International Conference on Information and Knowledge Management. 2013, 2333–2338Google Scholar
  19. 19.
    Severyn A, Moschitti A. Learning to rank short text pairs with convolutional deep neural networks. In: Proceedings of the 38th International ACMSIGIR Conference on Research and Development in Information Retrieval. 2015, 373–382Google Scholar
  20. 20.
    Liu Q, Yu F, Wu S, Wang L. A convolutional click prediction model. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 2015, 1743–1746Google Scholar
  21. 21.
    Guo F, Liu C, Kannan A, Minka T, Taylor M J, Wang Y M, Faloutsos C. Click chain model in Web search. In: Proceedings of International Conference on World Wide Web. 2009, 11–20Google Scholar
  22. 22.
    Buscher G, Van Elst L, Dengel A. Segment-level display time as implicit feedback: a comparison to eye tracking. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2009, 67–74Google Scholar
  23. 23.
    Smucker M D, Clarke C L A. Time-based calibration of effectiveness measures. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2012, 95–104Google Scholar
  24. 24.
    Fox S, Karnawat K, Mydland M, Dumais S, White T. Evaluating implicit measures to improve Web search. ACM Transactions on Information Systems, 2005, 23(2): 147–168CrossRefGoogle Scholar
  25. 25.
    White R W, Kelly D. A study on the effects of personalization and task information on implicit feedback performance. In: Proceedings of the 15th ACM International Conference on Information and Knowledge Management. 2006, 297–306Google Scholar
  26. 26.
    Agichtein E, Brill E, Dumais S. Improving Web search ranking by incorporating user behavior information. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2006, 19–26Google Scholar
  27. 27.
    Xu W H, Manavoglu E, Cantu-Paz E. Temporal click model for sponsored search. In: Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2010, 106–113Google Scholar
  28. 28.
    Wang K S, Gloy N, Li X L. Inferring search behaviors using partially observable Markov (POM) model. In: Proceedings of the 3rd ACM International Conference on Web Search and Data Mining. 2010, 211–220CrossRefGoogle Scholar
  29. 29.
    Xu D Q, Liu Y Q, Zhang M, Ma S P, Ru L Y. Incorporating revisiting behaviors into click models. In: Proceedings of the 5th ACM International Conference on Web Search and Data Mining. 2012, 303–312Google Scholar
  30. 30.
    Liu Y Q, Xie X H, Wang C, Nie J Y, Zhang M, Ma S P. Time-aware click model. ACM Transactions on Information Systems, 2016, 35(3): 24–34CrossRefGoogle Scholar
  31. 31.
    Salakhutdinov R, Hinton G. Semantic hashing. International Journal of Approximate Reasoning, 2009, 50(7): 969–978CrossRefGoogle Scholar
  32. 32.
    Socher R, Huval B, Manning C D, Ng A Y. Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. 2012, 1201–1211Google Scholar
  33. 33.
    Tur G, Deng L, Hakkani-Tür D, He X D. Towards deeper understanding: deep convex networks for semantic utterance classification. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2012, 5045–5048Google Scholar
  34. 34.
    Shen Y L, He X D, Gao J F, Deng L, Mesnil G. Learning semantic representations using convolutional neural networks for Web search. In: Proceedings of the 23rd International Conference onWorld WideWeb. 2014, 373–374Google Scholar
  35. 35.
    Zhang Y Y, Dai H J, Xu C, Feng J, Wang T F, Bian J, Wang B, Liu T Y. Sequential click prediction for sponsored search with recurrent neural networks. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014, 133–134Google Scholar
  36. 36.
    Borisov A, Markov I, de Rijke M, Serdyukov P. A neural click model for Web search. In: Proceedings of the 25th International Conference on World Wide Web. 2016, 531–541CrossRefGoogle Scholar
  37. 37.
    Kalchbrenner N, Grefenstette E, Blunsom P. A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014Google Scholar
  38. 38.
    Mikolov T, Sutskever I, Chen K, Corrado G S, Dean J. Distributed representations of words and phrases and their compositionality. In: Proceedings of the Neural Information Processing Systems Conference. 2013, 3111–3119Google Scholar
  39. 39.
    Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference onMachine Learning. 2010, 807–814Google Scholar
  40. 40.
    Wan L, Zeiler M, Zhang S X, Cun Y L, Fergus R. Regularization of neural networks using dropconnect. In: Proceedings of the 30th International Conference on Machine Learning. 2013, 1058–1066Google Scholar
  41. 41.
    Bordes A, Weston J, Usunier N. Open question answering with weakly supervised embedding models. In: Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 2014, 165–180Google Scholar
  42. 42.
    Echihabi A, Marcu D. A noisychannel approach to question answering. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics. 2003, 16–23Google Scholar
  43. 43.
    Chen D Q, Chen W Z, Wang J X, Chen Z, Yang Q. Beyond ten blue links: enabling user click modeling in federated web search. In: Proceedings of the 5th ACM International Conference on Web Search and Data Mining. 2012, 463–472Google Scholar
  44. 44.
    Järvelin K, Kekäläinen J. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 2002, 20(4): 422–446CrossRefGoogle Scholar
  45. 45.
    Yang H, Mityagin A, Svore K M, Markov S. Collecting high quality overlapping labels at low cost. In Proceedings of the 33rd International ACMSIGIR Conference on Research and Development in Information Retrieval. 2010, 459–466Google Scholar
  46. 46.
    Landis J R, Koch G G. The measurement of observer agreement for categorical data. Biometrics, 1977, 33(1): 159–174CrossRefzbMATHGoogle Scholar

Copyright information

© Higher Education Press and Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations