Information Retrieval Journal

, Volume 21, Issue 1, pp 56–80 | Cite as

Session search modeling by partially observable Markov decision process

  • Grace Hui Yang
  • Xuchu Dong
  • Jiyun Luo
  • Sicong Zhang
Article
  • 98 Downloads

Abstract

Session search, the task of document retrieval for a series of queries in a session, has been receiving increasing attention from the information retrieval research community. Session search exhibits the properties of rich user-system interactions and temporal dependency. These properties lead to our proposal of using partially observable Markov decision process to model session search. On the basis of a design choice schema for states, actions and rewards, we evaluate different combinations of these choices over the TREC 2012 and 2013 session track datasets. According to the experimental results, practical design recommendations for using PODMP in session search are discussed.

Keywords

Session search Dynamic IR modeling POMDP 

Notes

Acknowledgements

The research was supported by NSF CAREER IIS-1453721, NSF CNS-1223825, and DARPA FA8750-14-2-0226. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. Thanks to the sponsorship of the China Scholarship Council as well. Any opinions, findings, conclusions, or recommendations expressed in this paper are of the authors, and do not necessarily reflect those of the sponsors.

References

  1. Altman A., & Tennenholtz, M. (2005). Ranking systems: The pagerank axioms. In Proceedings of the 6th ACM conference on electronic commerce, EC ’05 (pp. 1–8).Google Scholar
  2. Baum, L. E., & Petrie, T. (1966). Statistical inference for probabilistic functions of finite state markov chains. The Annals of Mathematical Statistics, 37(6), 1554–1563.MathSciNetCrossRefMATHGoogle Scholar
  3. Bellman, R. (1957). Dynamic programming. Princeton: Princeton University Press.MATHGoogle Scholar
  4. Blumenthal, R., & Getoor, R. (1968). Markov processes and potential theory. New York: Academic Press.MATHGoogle Scholar
  5. Chilton, L. B., & Teevan, J. (2011). Addressing people’s information needs directly in a web search result page. In Proceedings of the 20th international conference on world wide web, WWW ’11 (pp. 27–36).Google Scholar
  6. Cormack, G. V., Smucker, M. D., & Clarke, C. L. (2011). Efficient and effective spam filtering and re-ranking for large web datasets. Information Retrieval, 14(5), 441–465.CrossRefGoogle Scholar
  7. Fox, S., Karnawat, K., Mydland, M., Dumais, S., & White, T. (2005). Evaluating implicit measures to improve web search. ACM Transactions on Information Systems (TOIS), 23(2), 147–168.CrossRefGoogle Scholar
  8. Guan, D., Zhang, S., & Yang, H. (2013). Utilizing query change for session search. In Proceedings of the 36th international ACM SIGIR conference on research and development in information retrieval, ACM, New York, SIGIR ’13 (pp. 453–462).Google Scholar
  9. Hassan, A., White, R. W., Dumais, S. T., & Wang, Y. M. (2014). Struggling or exploring?: Disambiguating long search sessions. In Proceedings of the 7th ACM international conference on web search and data mining, ACM, New York, NY, USA, WSDM ’14 (pp. 53–62). doi: 10.1145/2556195.2556221.
  10. Hofmann, K., Whiteson, S., & de Rijke, M. (2011). Balancing exploration and exploitation in learning to rank online. In Proceedings of the 33rd European conference on advances in information retrieval, ECIR’11 (pp. 251–263).Google Scholar
  11. Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 20(4), 422–446.CrossRefGoogle Scholar
  12. Jin, X., Sloan, M., & Wang, J. (2013). Interactive exploratory search for multi page search results. In Proceedings of the 22nd international conference on world wide web, WWW ’13 (pp. 655–666).Google Scholar
  13. Joachims, T. (1997). A probabilistic analysis of the Rocchio algorithm with TFIDF for text categorization. In Proceedings of the fourteenth international conference on machine learning, ICML ’97 (pp. 143–151).Google Scholar
  14. Kaelbling, L., Littman, M., & Cassandra, A. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1–2), 99–134.MathSciNetCrossRefMATHGoogle Scholar
  15. Kanoulas, E., Carterette, B., Hall, M., Clough, P., & Sanderson, M. (2012). Overview of the TREC 2012 session track. In Proceedings of the 21st text retrieval conference, TREC ’12.Google Scholar
  16. Kanoulas, E., Carterette, B., Hall, M., Clough, P., & Sanderson, M. (2013). Overview of the TREC 2013 session track. In Proceedings of the 22nd text retrieval conference, TREC ’13.Google Scholar
  17. Katehakis, M. N., & Veinott, A. F. (1987). The multi-armed bandit problem: Decomposition and computation. Mathematics of Operations Research, 12(2), 262–268.MathSciNetCrossRefMATHGoogle Scholar
  18. Kim, Y., Hassan, A., White, R. W., & Zitouni, I. (2014). Modeling dwell time to predict click-level satisfaction. In Proceedings of the 7th ACM international conference on web search and data mining, ACM, New York, NY, USA, WSDM ’14 (pp. 193–202).Google Scholar
  19. Kotov, A., Bennett, P. N., White, R. W., Dumais, S. T., & Teevan, J. (2011). Modeling and analysis of cross-session search tasks. In Proceedings of the 34th international ACM SIGIR conference on research and development in information retrieval, ACM, New York, NY, USA, SIGIR ’11 (pp. 5–14).Google Scholar
  20. Luo, J., Zhang, S., & Yang, H. (2014). Win–win search: Dual-agent stochastic game in session search. In Proceedings of the 37th international ACM SIGIR conference on research and development in information retrieval, SIGIR ’14.Google Scholar
  21. Miller, D. R. H., Leek, T., & Schwartz, R. M. (1999). A hidden Markov model information retrieval system. In Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval, SIGIR ’99 (pp. 214–221).Google Scholar
  22. Mitra, B. (2015). Exploring session context using distributed representations of queries and reformulations. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, ACM, New York, NY, USA, SIGIR ’15 (pp. 3–12). doi: 10.1145/2766462.2767702.
  23. Mitsui, M., Shah, C., & Belkin, N. J. (2016). Extracting information seeking intentions for web search sessions. In Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval, ACM, New York, NY, USA, SIGIR ’16 (pp. 841–844). doi: 10.1145/2911451.2914746.
  24. Norris, J. R. (1998). Markov chains. Cambridge: Cambridge University Press.MATHGoogle Scholar
  25. Odijk, D., White, R. W., Hassan Awadallah, A., & Dumais, S. T. (2015). Struggling and success in web search. In Proceedings of the 24th ACM international on conference on information and knowledge management, ACM, New York, NY, USA, CIKM ’15 (pp. 1551–1560). doi: 10.1145/2806416.2806488.
  26. Rabiner, L. R. (1989). A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.CrossRefGoogle Scholar
  27. Radlinski, F., Kleinberg, R., & Joachims, T. (2008). Learning diverse rankings with multi-armed bandits. In Proceedings of the 25th international conference on machine learning, ICML ’08 (pp. 784–791).Google Scholar
  28. Raman, K., Bennett, P. N., & Collins-Thompson, K. (2013). Toward whole-session relevance: Exploring intrinsic diversity in web search. In Proceedings of the 36th international ACM SIGIR conference on research and development in information retrieval, SIGIR ’13 (pp. 463–472).Google Scholar
  29. Salton, G., & Buckley, C. (1997). Improving retrieval performance by relevance feedback. In K. S. Jones & P. Wille (Eds.), Readings in information retrieval. San Francisco: Morgan Kaufmann.Google Scholar
  30. Shen, X., Tan, B., & Zhai, C. (2005). Implicit user modeling for personalized search. In Proceedings of the 14th ACM international conference on information and knowledge management, CIKM ’05 (pp. 824–831).Google Scholar
  31. Singh, M., & Croft, W. B. (2016). Iterative search using query aspects. In Proceedings of the 25th ACM international on conference on information and knowledge management, ACM, New York, NY, USA, CIKM ’16 (pp. 2037–2040). doi: 10.1145/2983323.2983903.
  32. Sondik, E. (1978). The optimal control of partially observable Markov processes over the infinite horizon: Discounted cost. Operations Research, 26(2), 282–304.MathSciNetCrossRefMATHGoogle Scholar
  33. Yuan, S., & Wang, J. (2012). Sequential selection of correlated ads by POMDPs. In Proceedings of the 21st ACM international conference on information and knowledge management, CIKM ’12 (pp. 515–524).Google Scholar
  34. Zhai, C., & Lafferty, J. (2002). Two-stage language models for information retrieval. In Proceedings of the 25th annual international ACM SIGIR conference on research and development in information retrieval, SIGIR ’02 (pp. 49–56).Google Scholar
  35. Zhang, S., Luo, J., & Yang, H. (2014). A POMDP model for content-free document re-ranking. In Proceedings of the 37th international ACM SIGIR conference on research and development in information retrieval, SIGIR ’14 (pp. 1139–1142).Google Scholar
  36. Zhuang, M., Toms, E. G., & Demartini, G. (2016). Search behaviour before and after search success. In Proceedings of the second international workshop on search as learning, SAL 2016, co-located with the 39th international ACM SIGIR conference on research and development in information retrieval, SIGIR 2016, Pisa, Italy.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Grace Hui Yang
    • 1
  • Xuchu Dong
    • 1
    • 2
  • Jiyun Luo
    • 1
  • Sicong Zhang
    • 1
  1. 1.Department of Computer ScienceGeorgetown UniversityWashingtonUSA
  2. 2.College of Computer Science and TechnologyJilin UniversityChangchunChina

Personalised recommendations