Advertisement

Boosting learning to rank with user dynamics and continuation methods

  • Nicola Ferro
  • Claudio Lucchese
  • Maria MaistroEmail author
  • Raffaele Perego
Learning from User Interactions
  • 14 Downloads

Abstract

Learning to rank (LtR) techniques leverage assessed samples of query-document relevance to learn effective ranking functions able to exploit the noisy signals hidden in the features used to represent queries and documents. In this paper we explore how to enhance the state-of-the-art LambdaMart LtR algorithm by integrating in the training process an explicit knowledge of the underlying user-interaction model and the possibility of targeting different objective functions that can effectively drive the algorithm towards promising areas of the search space. We enrich the iterative process followed by the learning algorithm in two ways: (1) by considering complex query-based user dynamics instead than simply discounting the gain by the rank position; (2) by designing a learning path across different loss functions that can capture different signals in the training data. Our extensive experiments, conducted on publicly available datasets, show that the proposed solution permits to improve various ranking quality measures by statistically significant margins.

Keywords

Learning to rank User dynamics Continuation methods 

Notes

Acknowledgements

This paper is partially supported by the BIGDATAGRAPES (EU H2020 RIA, Grant Agreement No. 780751) and the OK-INSAID (MIUR-PON 2018, Grant Agreement No. ARS01_00917) projects. The work is also partially funded by the “DAta BenchmarK for Keyword-based Access and Retrieval” (DAKKAR) Starting Grants project sponsored by University of Padua and Fondazione Cassa di Risparmio di Padova e di Rovigo, and by AMAOS (Advanced Machine Learning for Automatic Omni-Channel Support), funded by Innovationsfonden, Denmark.

References

  1. Agichtein, E., Brill, E., & Dumais, S. (2006). Improving web search ranking by incorporating user behavior. In E. N. Efthimiadis, S. Dumais, D. Hawking, & K. Järvelin (Eds.), Proceedings of 29th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2006), ACM Press, New York, USA, pp. 19–26.Google Scholar
  2. Allan, J., Aslam, J. A., Sanderson, M., Zhai, C., & Zobel, J. (Eds.). (2009). Proceedings of 32nd annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2009), ACM Press, New York, USA.Google Scholar
  3. Allgower, E. L., & Georg, K. (1980). Numerical continuation methods an introduction. Heidelberg: Springer.zbMATHGoogle Scholar
  4. Beitzel, S. M., Jensen, E. C., Frieder, O., Grossman, D., Lewis, D. D., Chowdhury, A., & Kolcz, A. (2005). Automatic web query classification using labeled and unlabeled training data. In Proceedings of the 28th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR ’05), ACM, New York, NY, USA, pp. 581–582.  https://doi.org/10.1145/1076034.1076138.
  5. Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009). Curriculum learning. In A. Danyluk, L. Bottou, M. L. Littman (Eds.), Proceedings of 26th annual international conference on machine learning (ICML 2009), ACM Press, New York, USA, pp. 41–48.Google Scholar
  6. Broder, A. (2002). A taxonomy of web search. SIGIR Forum, 36(2), 3–10.CrossRefGoogle Scholar
  7. Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., et al. (2005). Learning to rank using gradient descent. In S. Dzeroski, L. De Raedt, & S. Wrobel (Eds.), Proceedings of 22nd international conference on machine learning (ICML 2005), ACM Press, New York, USA, pp. 89–96.Google Scholar
  8. Burges, C. J. C. (2010). From RankNet to LambdaRank to LambdaMART: An overview. Technical report, Microsoft Research, MSR-TR-2010-82.Google Scholar
  9. Burges, C. J. C., Ragno, R., & Le, Q. V. (2006). Learning to rank with nonsmooth cost functions. In B. Schölkopf, J. C. Platt, & T. Hoffman (Eds.), Proceedings of 19th international conference on neural information processing systems (NIPS 2006), MIT Press, Cambridge, MA, USA, pp. 193–200.Google Scholar
  10. Chapelle, O., Chi, M., & Zien, A. (2006). A continuation method for semi-supervised SVMs. In W. Cohen, A. Moore (Eds.), Proceedings of 23rd annual international conference on machine learning (ICML 2006), ACM Press, New York, USA, pp. 185–192.Google Scholar
  11. Chapelle, O., Joachims, T., Radlinski, F., & Yue, Y. (2012). Large-scale validation and analysis of interleaved search evaluation. ACM Transactions on Information Systems (TOIS), 30(1), 6.CrossRefGoogle Scholar
  12. Chapelle, O., Metzler, D., Zhang, Y., & Grinspan, P. (2009). Expected reciprocal rank for graded relevance. In D. W. L. Cheung, I. Y. Song, W. W. Chu, X. Hu, & J. J. Lin (Eds.), Proceedings of 18th international conference on information and knowledge management (CIKM 2009), ACM Press, New York, USA, pp. 621–630.Google Scholar
  13. Chapelle, O., & Zhang, Y. (2009). A dynamic bayesian network click model for web search ranking. In J. Quemada, G. León, Y. Maarek, & W. Nejdl (Eds.), Proceedings of 18th international conference on World Wide Web (WWW 2009), ACM Press, New York, USA, pp. 1–10.Google Scholar
  14. Chen, B., & Xiu, N. (1999). A global linear and local quadratic noninterior continuation method for nonlinear complementarity problems based on Chen–Mangasarian smoothing functions. SIAM Journal on Optimization, 9(3), 605–623.MathSciNetCrossRefGoogle Scholar
  15. Chen, X., Gupta, A. (2015). Webly supervised learning of convolutional networks. In R. Bajcsy, G. Hager, Y. Ma, K. Ikeuchi, C. Schnörr, J. Sivic, & R. Vidal (Eds.), Proceedings of 2015 IEEE international conference on computer vision (ICCV), IEEE Computer Society, Los Alamitos, CA, USA, ICCV ’15, pp. 1431–1439.Google Scholar
  16. Chuklin, A., Markov, I., & de Rijke, M. (2015). Click models for web search. San Rafael: Morgan & Claypool Publishers.CrossRefGoogle Scholar
  17. Coleman, T. F., & Wu, Z. (1996). Parallel continuation-based global optimization for molecular conformation and protein folding. Journal of Global Optimization, 8(1), 49–65.MathSciNetCrossRefGoogle Scholar
  18. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12, 2493–2537.zbMATHGoogle Scholar
  19. Craswell, N., Zoeter, O., Taylor, M., & Ramsey, B. (2008). An experimental comparisonof click position-bias models. In (Najork et al., 2008), pp. 87–94.Google Scholar
  20. Donmez, P., Svore, K. M., Burges, C. J. C. (2009). On the local optimality of LambdaRank. In (Allan et al., 2009), pp. 460–467.Google Scholar
  21. Dou, Z., Song, R., Yuan, X., & Wen, J. R. (2008). Are click-through data adequate for learning web search rankings? In J. G. Shanahan, S. Amer-Yahia, I. Manolescu, Y. Zhang, D. A. Evans, A. Kolcz, K. S. Choi, & A. Chowdhury (Eds.), Proceedings of 17th international conference on information and knowledge management (CIKM 2008), ACM Press, New York, USA, pp. 73–82.Google Scholar
  22. Ferrante, M., Ferro, N., & Maistro, M. (2014). Injecting user models and time into precision via Markov chains. In S. Geva, A. Trotman, P. Bruza, C. L. A. Clarke, & K. Järvelin (Eds.), Proceedings of 37th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2014), ACM Press, New York, USA, pp. 597–606.Google Scholar
  23. Ferro, N., Lucchese, C., Maistro, M., & Perego, R. (2017). On including the user dynamic in learning to rank. In N. Kando, T. Sakai, H. Joho, H. Li, A. P. de Vries, & R. W. White (Eds.), Proceedings of 40th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2017), ACM Press, New York, USA, pp. 1041–1044.Google Scholar
  24. Ferro, N., Lucchese, C., Maistro, M., & Perego, R. (2018). Continuation methods and curriculum learning for learning to rank. In A. Cuzzocrea, J. Allan, N. W. Paton, D. Srivastava, R. Agrawal, A. Broder, M. J. Zaki, S. Candan, A. Labrinidis, A. Schuster, & H. Wang (Eds.), Proceedings of 27th international conference on information and knowledge management (CIKM 2018), ACM Press, New York, USA, pp. 1523–1526.Google Scholar
  25. Hofmann, K., Schuth, A., Whiteson, S., & de Rijke, M. (2013). Reusing historical interaction data for faster online learning to rank for IR. In S. Leonardi, A. Panconesi, P. Ferragina, A. Gionis (Eds.), Proceedings of 6th ACM international conference on web searching and data mining (WSDM 2013), ACM Press, New York, USA, pp. 183–192.Google Scholar
  26. Hu, B., Lu, Z., Li, H., & Chen, Q. (2014). Convolutional neural network architectures for matching natural language sentences. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, & K. Q. Weinberger (Eds.), Proceedings of 27th international conference on neural information processing systems (NIPS 2014), Vol. 2, MIT Press, Cambridge, MA, USA, pp. 2042–2050.Google Scholar
  27. Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), 20(4), 422–446.CrossRefGoogle Scholar
  28. Jiang, D., Pei, J., & Li, H. (2013). Mining search and browse logs for web search: A survey. ACM Transactions on Intelligent Systems and Technology (TIST), 4(4), 57:1–57:37.Google Scholar
  29. Joachims, T. (2002). Optimizing search engines using clickthrough data. In O. Zaïane, R. Goebel, D. Hand, D. Keim, & R. Ng (Ed.), Proceedings of 8th ACM SIGKDD international conference on knowledge discovery and data mining (KDD 2002), ACM Press, New York, USA, pp. 133–142.Google Scholar
  30. Joachims, T., & Radlinski, F. (2007). Search engines that learn from implicit feedback. Computer, 40(8), 34–40.CrossRefGoogle Scholar
  31. Joachims, T., Granka, L., Pan, B., Hembrooke, H., & Gay, G. (2005). Accurately interpreting click through data as implicit feedback. In R. Baeza-Yates, N. Ziviani, G. Marchionini, A. Moffat, & J. Tait (Eds.), Proceedings 28th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2005), ACM Press, New York, USA, pp. 154–161.Google Scholar
  32. Joachims, T., Swaminathan, A., & Schnabel, T. (2017). Unbiased learning-to-rank with biased feedback. In M. de Rijke, M. Shokouhi, A. Tomkins, M. Zhang (Eds.), Proceedings of 10th ACM international conference on web searching and data mining (WSDM 2017). ACM Press, New York, USA, pp. 781–789.Google Scholar
  33. Liu, T. Y. (2009). Learning to rank for information retrieval. Foundations and Trends in Information Retrieval (FnTIR), 3(3), 225–331.CrossRefGoogle Scholar
  34. Lucchese, C., Orlando, S., Perego, R., Silvestri, F., & Tolomei, G. (2013). Discovering tasks from search engine query logs. ACM Transactions on Information System (TOIS), 31(3), 14:1–14:43.CrossRefGoogle Scholar
  35. Mehrotra, R., Awadallah, A. H., & Yilmaz, E. (2018) Learnir: Wsdm 2018 workshop on learning from user interactions. In Proceedings of the eleventh ACM international conference on web search and data mining, ACM, New York, NY, USA, WSDM ’18, pp. 797–798.  https://doi.org/10.1145/3159652.3160598.
  36. Moré, J. J., & Wu, Z. (1997). Global continuation for distance geometry problems. SIAM Journal on Optimization, 7(3), 814–836.MathSciNetCrossRefGoogle Scholar
  37. Nagamune, R. (2003). A robust solver using a continuation method for Nevanlinna–Pick interpolation with degree constraint. IEEE Transactions on Automatic Control (TAC), 48(1), 113–117.MathSciNetCrossRefGoogle Scholar
  38. Najork, M., Broder, A., & Chakrabarti, S. (Eds.). (2008). Proceeding of 1st ACM international conference on web searching and data mining (WSDM 2008), ACM Press, New York, USA.Google Scholar
  39. Norris, J. R. (1998). Markov chains. Cambridge: Cambridge University Press.zbMATHGoogle Scholar
  40. Qin, T., & Liu, T. Y. (2013). Introducing LETOR 4.0 Datasets. Information Retrieval (csIR). arXiv:1306.2597.
  41. Qu, M., Tang, J., & Han, J. (2018). Curriculum learning for heterogeneous star network embedding via deep reinforcement learning. In Y. Chang, C. Zhai, Y. Liu, & Y. Maarek (Eds.), Proceedings of 11th ACM international conference on web searching and data mining (WSDM 2018), ACM Press, New York, USA, pp. 468–476.Google Scholar
  42. Rabani, E., Reichman, D. R., Krilov, G., & Berne, B. J. (2002). The calculation of transport properties in quantum liquids using the maximum entropy numerical analytic continuation method: Application to liquid para-hydrogen. Proceedings of the National Academy of Sciences of the United States of America (PNAS), 99(3), 1129–1133.CrossRefGoogle Scholar
  43. Sakai, T., & Dou, Z. (2013). Summaries, ranked retrieval and sessions: A unified framework for information access evaluation. In G. J. F. Jones, P. Sheridan, D. Kelly, M. de Rijke, & T. Sakai (Eds.), Proceedings of 36th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2013). ACM Press, New York, USA, pp. 473–482.Google Scholar
  44. Schuth, A., Hofmann, K., Whiteson, S., & de Rijke, M. (2013). Lerot: An online learning to rank framework. In K. Balog, D. Elsweiler, E. Kanoulas, L. Kelly, & M. D. Smucker (Eds.), Proceedings of 1st workshop on living labs for information retrieval evaluation (LL 2013), ACM Press, New York, USA, pp. 23–26.Google Scholar
  45. Schuth, A., Oosterhuis, H., Whiteson, S., & de Rijke, M. (2016). Multileave gradient descent for fast online learning to rank. In P. N. Bennett, V. Josifovski, J. Neville, & F. Radlinski (Eds.), Proceedings of 9th ACM international conference on web searching and data mining (WSDM 2016). ACM Press, New York, USA, pp. 457–466.Google Scholar
  46. Serdyukov, P., Craswell, N., & Dupret, G. (2012). WSCD2012: Workshop on web search click data 2012. In E. Adar, J. Teevan, E. Agichtein, Y. Maarek (Eds.), Proceedings of 5th ACM international conference on web searching and data mining (WSDM 2012). ACM Press, New York, USA, pp. 771–772.Google Scholar
  47. Silvestri, F. (2009). Mining query logs: Turning search usage data into knowledge. Foundations and Trends in Information Retrieval (FnTIR), 4(1–2), 1–174.zbMATHGoogle Scholar
  48. Smucker, M. D., Allan, J., & Carterette, B. A. (2007). A comparison of statistical significance tests for information retrieval evaluation. In M. J. Silva, A. A. F. Laender, R. Baeza-Yates, D. L. McGuinness, B. Olstad, Ø. H. Olsen, & A. Falcão (Eds.), Proceeding of 16th international conference on information and knowledge management (CIKM 2007), ACM Press, New York, USA, pp. 623–632.Google Scholar
  49. Taylor, M., Guiver, J., Robertson, S., & Minka, T. (2008). SoftRank: Optimizing non-smooth rank metrics. In (Najork et al., 2008), pp. 77–86.Google Scholar
  50. Teodorescu, I. (2009). Maximum likelihood estimation for Markov Chains. Computation (statCO). arXiv:0905.4131.
  51. Wang, H., Langley, R., Kim, S., McCord-Snook, E., & Wang, H. (2018). Efficient exploration of gradient space for online learning to rank. In: K. Collins-Thompson, Q. Mei, B. Davison, Y. Liu, & E. Yilmaz (Eds.), Proceedings of 41th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2018), ACM Press, New York, USA, pp. 145–154.Google Scholar
  52. White, R. W., Bailey, P., & Chen, L. (2009). Predicting user interests from contextual information. In: (Allan et al., 2009), pp. 363–370.Google Scholar
  53. Wu, Q., Burges, C. J. C., Svore, K. M., & Gao, J. (2010). Adapting boosting for information retrieval measures. Information Retrieval, 13(3), 254–270.CrossRefGoogle Scholar
  54. Yilmaz, E., Shokouhi, M., Craswell, N., & Robertson, S. (2010). Expected browsing utility for web search evaluation. In J. Huang, N. Koudas, G. J. F. Jones, X. Wu, K. Collins-Thompson, A. An (Eds.), Proceedings of 19th international conference on information and knowledge management (CIKM 2010), ACM Press, New York, USA, pp. 1561–1565.Google Scholar
  55. Zhang, Y., Park, L. A. F., & Moffat, A. (2010). Click-based evidence for decaying weight distributions in search effectiveness metrics. Information Retrieval, 13(1), 46–69.CrossRefGoogle Scholar
  56. Zheng, Z., Zha, H., Zhang, T., Chapelle, O., Chen, K., & Sun, G. (2007). A general boosting method and its application to learning ranking functions for web search. In J. C. Platt, D. Koller, Y. Singer, & S. T. Roweis (Eds.) Proceedings of 20th international conference on neural information processing systems (NIPS 2007), MIT Press, Cambridge, MA, USA, pp. 1697–1704.Google Scholar
  57. Zoghi, M., Tunys, T., Ghavamzadeh, M., Kveton, B., Szepesvari, C., & Wen, Z. (2017). Online learning to rank in stochastic click models. In D. Precup, & Y. Whye Teh (Eds.), Proceedings of 34th annual international conference on machine learning (ICML 2017), ACM Press, New York, USA, pp. 4199–4208.Google Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Department of Information EngineeringUniversity of PaduaPaduaItaly
  2. 2.Department of Environmental Sciences, Informatics and StatisticsCa’ Foscari University of VeniceMestreItaly
  3. 3.Department of Computer ScienceUniversity of CopenhagenCopenhagenDenmark
  4. 4.Istituto di Scienza e Tecnologie dell’Informazione A. Faedo (ISTI)National Research Council (CNR), Area di Ricerca di PisaPisaItaly

Personalised recommendations