Information Retrieval Journal

, Volume 20, Issue 1, pp 53–80 | Cite as

Enhancing click models with mouse movement information

  • Zeyang Liu
  • Jiaxin Mao
  • Chao Wang
  • Qingyao Ai
  • Yiqun Liu
  • Jian-Yun Nie
Constructing Click Models for Search Users

Abstract

User interactions in Web search, in particular, clicks, provide valuable hints on document relevance; but the signals are very noisy. In order to better understand user click behaviors and to infer the implied relevance, various click models have been proposed, each relying on some hypotheses and involving different hidden events (e.g. examination). In almost all the existing click models, it is assumed that clicks are the only observable evidence and the examinations of documents are deduced from it. However, with an increasing number of embedded heterogeneous components (e.g. verticals) on Search Engine Result Pages, click information is not sufficient to draw a complete picture of process of user examination, especially in federated search scenario. In practice, we can also collect mouse movement information, which has proven to have a strong correlation with examination. In this paper, we propose to incorporate mouse movement information into existing click models to enhance the estimation of examination. The enhanced click models are shown to have a better ability to predict both user clicks and document relevance, than the original models. The collection of mouse movement information has been implemented in a commercial search engine, showing the feasibility of the approach in practice.

Keywords

Mouse movement Click model Search engine Federated search 

Notes

Acknowledgements

This work is supported by Tsinghua University Initiative Scientific Research Program (2014Z21032), National Key Basic Research Program (2015CB358700) and Natural Science Foundation (61472206, 61073071) of China. Part of the work has been done at the Tsinghua-NUS NExT Search Centre, which is supported by the Singapore National Search Foundation & Interactive Digital Media R&D Program Office, MDA under research Grant (WBS:R-252-300-001-490).

References

  1. Arguello, J., & Capra, R. (2014). The effects of vertical rank and border on aggregated search coherence and search behavior. In CIKM’14 (pp. 539–548). ACM.Google Scholar
  2. Awadallah, A. H., & Zitouni, I. (2014). Machine-assisted evaluation for search preference judgments. In CIKM’14 (pp. 51–60). ACM.Google Scholar
  3. Broder, A. (2002). A taxonomy of web search. In ACM SIGIR forum 2002 (Vol. 36, pp. 3–10). ACM.Google Scholar
  4. Buscher, G., Cutrell, E., & Morris, M. R. M. (2009). What do you see when you’re surfing?: Using eye tracking to predict salient regions of web pages. In CHI’09 (pp. 21–30). ACM.Google Scholar
  5. Buscher, G., Dumais, S. T., & Cutrell, E. (2010). The good, the bad, and the random: An eye-tracking study of ad quality in web search. In SIGIR’10 (pp. 42–49). ACM.Google Scholar
  6. Chapelle, O., & Zhang, Y. (2009). A dynamic Bayesian network click model for web search ranking. In WWW’09 (pp. 1–10). ACM.Google Scholar
  7. Chapelle, O., Metlzer, D., Zhang, Y., & Grinspan, P. (2009). Expected reciprocal rank for graded relevance. In: CIKM’09 (pp. 621–630). ACM.Google Scholar
  8. Chen, M. M. C., Anderson, J. R. J., & Sohn, M. H. M. (2001). What can a mouse cursor tell us more?: Correlation of eye/mouse movements on web browsing. In CHI’01 Extended Abstracts on Human Factors in Computing Systems (pp. 281–282).Google Scholar
  9. Chen, D., Chen, W., Wang, H., Chen, Z., & Yang, Q. (2012). Beyond ten blue links: Enabling user click modeling in federated web search. In WSDM’12 (pp. 463–472). ACM.Google Scholar
  10. Chuklin, A., Serdyukov, P., & de Rijke, M. (2013). Using intent information to model user behavior in diversified search. In: ECIR’13 (pp. 1–13). Springer.Google Scholar
  11. Claypool, M., Le, P., Wased, M., & Brown, D. (2001). Implicit interest indicators. In IUI’01 (pp. 33–40). ACM.Google Scholar
  12. Craswell, N., Zoeter, O., Taylor, M., & Ramsey, B. (2008). An experimental comparison of click position-bias models. In WSDM’08 (pp. 87–94). ACM.Google Scholar
  13. Diaz, F., White, R. W., Buscher, G., & Liebling, D. (2013). Robust models of mouse movement on dynamic web search results pages. In CIKM’13 (pp. 1451–1460). ACM.Google Scholar
  14. Dupret, G. E., & Piwowarski, B. (2008). A user browsing model to predict search engine click data from past observations. In SIGIR’08 (pp. 331–338). ACM.Google Scholar
  15. Friedman, J. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), 1189–1232.MathSciNetCrossRefMATHGoogle Scholar
  16. Granka, L. A., Joachims, T., & Gay, G. (2004). Eye-tracking analysis of user behavior in WWW search. In SIGIR’04 (pp. 478–479). ACM.Google Scholar
  17. Guan, Z., & Cutrell, E. (2007). An eye tracking study of the effect of target rank on web search. In CHI’07 (pp. 417–420). ACM.Google Scholar
  18. Guo, Q., & Agichtein, E. (2008). Exploring mouse movements for inferring query intent. In SIGIR’08 (pp. 707–708). ACM.Google Scholar
  19. Guo, Q., & Agichtein, E. (2012). Beyond dwell time: Estimating document relevance from cursor movements and other post-click searcher behavior. In WWW’12 (pp. 569–578). ACM.Google Scholar
  20. Guo, Q., Lagun, D., & Agichtein, E. (2012). Predicting web search success with fine-grained interaction data. In CIKM’12 (pp. 2050–2054). ACM.Google Scholar
  21. Guo, F., Liu, C., & Wang, Y. M. Y. (2009a). Efficient multiple-click models in web search. In WSDM’09 (pp. 124–131). ACM.Google Scholar
  22. Guo, F., Liu, C., Kannan, A., Minka, T., Taylor, M., Wang, Y. M., & Faloutsos, C. (2009b). Click chain model in web search. In WWW’09 (pp. 11–20)Google Scholar
  23. Hu, B., Zhang, Y., Chen, W., Wang, G., & Yang, Q. (2011). Characterizing search intent diversity into click models. In WWW’11 (pp. 17–26). ACM.Google Scholar
  24. Huang, J., White, R., & Buscher. G. (2012a). User see, user point: Gaze and cursor alignment in web search. In CHI’12 (pp. 1341–1350). ACM.Google Scholar
  25. Huang, J., White, R. R. W., & Dumais, S. (2011). No clicks, no problem: using cursor movements to understand and improve search. In CHI’11 (pp. 1225–1234). ACM.Google Scholar
  26. Huang, J., White, R. W., Buscher, G., & Wang, K. (2012b). Improving searcher models using mouse. In SIGIR’12 (pp. 195–204). ACM.Google Scholar
  27. Joachims, T., Granka, L., Pan, B., Hembrooke, H., & Gay, G. (2005). Accurately interpreting click through data as implicit feedback. In SIGIR’05 (pp. 154–161). ACM.Google Scholar
  28. Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329–354.CrossRefGoogle Scholar
  29. Lagun, D., Ageev, M., Guo, Q., & Agichtein, E. (2014). Discovering common motifs in cursor movement data for improving web search. In WSDM’14 (pp. 183–192). ACM.Google Scholar
  30. Li, L., Kim, J. Y., & Zitouni, I. (2015). Toward predicting the outcome of an A/B experiment for search relevance. In WSDM’15 (pp. 37–46). ACM.Google Scholar
  31. Liu, Y., Chen, Y., Tang, J., Sun, J., Zhang, M., Ma, S., & Zhu, X. (2015a). Different users, different opinions: Predicting search satisfaction with mouse movement information. In SIGIR’15 (pp. 493–502). ACM.Google Scholar
  32. Liu, C., Guo, F., & Faloutsos, C. (2009). BBM: Bayesian browsing model from petabyte-scale data. In KDD’09 (pp. 537–546). ACM.Google Scholar
  33. Liu, Z., Liu, Y., Zhou, K., Zhang, M., & Ma, S. (2015b). Influence of vertical result in web search examination. In SIGIR’15 (pp. 193–202). ACM.Google Scholar
  34. Liu, Y., Wang, C., Zhou K., Nie, J., Zhang, M., & Shaoping, M. (2014). From skimming to reading: A two-stage examination model for web search. In CIKM’14 (pp. 849–858). ACM.Google Scholar
  35. Liu, Y., Zhang, M., Ru, L., & Ma, S. (2006). Automatic query type identification based on click through information. In AIRS’06 (pp. 593–600). Springer.Google Scholar
  36. Matthews, B. (1975). Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein. Structure, 405(2), 442.Google Scholar
  37. Moffat, A., & Zobel, J. (2008). Rank-biased precision for measurement of retrieval effectiveness. ACM Transactions on Information Systems (TOIS), 27(1), 2.CrossRefGoogle Scholar
  38. Navalpakkam, V., Jentzsch, L.L., Sayres, R., Ravi, S., Ahmed, A., & Smola, A. (2013) Measurement and modeling of eye-mouse behavior in the presence of nonlinear page layouts. In WWW’13 (pp. 953–964).Google Scholar
  39. Richardson, M., Dominowska, E., & Ragno, R. (2007). Predicting clicks: Estimating the click-through rate for new ads. In WWW’07 (pp. 521–530). ACM.Google Scholar
  40. Rodden, K., Ruthven, I., & White, R. (2007). Exploring how mouse movements relate to eye movements on web search results pages. In ACM SIGIR Forum.Google Scholar
  41. Speicher, M., Both, A., & Gaedke, M. (2013). TellMyRelevance!: Predicting the relevance of web search results from cursor interactions. In CIKM’13 (pp. 1281–1290). ACM.Google Scholar
  42. Sushmita, S., Joho, H., Lalmas, M., & Villa, R. (2010). Factors affecting click-through behavior in aggregated search interfaces. In CIKM’10 (pp. 519–528). ACM.Google Scholar
  43. Thomas, P., Scholer, F., & Moffat, A. (2013). What users do: The eyes have it (pp. 416–427). Berlin: Springer.Google Scholar
  44. Wang, C., Liu, Y., Wang, M., Zhou, K., Nie, J. Y., & Ma, S. (2015). Incorporating non-sequential behavior into click models. In SIGIR’15 (pp. 283–292). ACM.Google Scholar
  45. Wang, C., Liu, Y., Zhang, M., Ma, S., Zheng, M., Qian, J., & Zhang, K. (2013) Incorporating vertical results into search click models. In SIGIR’13 (pp. 503–512). ACM.Google Scholar
  46. Xing, Q., Liu, Y., Nie, J.Y., Zhang, M., Ma, S., & Zhang, K. (2013). Incorporating user preferences into click models. In CIKM’13 (pp. 1301–1310). ACM.Google Scholar
  47. Zhang, Y., Chen, W., Wang, D., & Yang, Q. (2011). User-click modeling for understanding and predicting search-behavior. In KDD’11 (pp. 1388–1396). ACM.Google Scholar
  48. Zhou, K., Cummins, R., Lalmas, M., & Jose, J. M. (2013). Which vertical search engines are relevant? In WWW’2013 (pp. 1557–1568). ACM.Google Scholar
  49. Zhang, Y., Park, L. A. F., & Moffat, A. (2010). Click-based evidence for decaying weight distributions in search effectiveness metrics. Information Retrieval, 13(1), 46–69.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Zeyang Liu
    • 1
  • Jiaxin Mao
    • 2
  • Chao Wang
    • 2
  • Qingyao Ai
    • 3
  • Yiqun Liu
    • 2
  • Jian-Yun Nie
    • 4
  1. 1.Department of Computer Technology and ApplicationsQinghai UniversityQinghaiChina
  2. 2.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina
  3. 3.College of Information and Computer SciencesUniversity of Massachusetts AmherstAmherstUSA
  4. 4.Department of Computer Science and Operations ResearchUniversity of MontrealMontrealCanada

Personalised recommendations