Exploring URL Hit Priors for Web Search

  • Ruihua Song
  • Guomao Xin
  • Shuming Shi
  • Ji-Rong Wen
  • Wei-Ying Ma
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3936)


URL usually contains meaningful information for measuring the relevance of a Web page to a query in Web search. Some existing works utilize URL depth priors (i.e. the probability of being a good page given the length and depth of a URL) for improving some types of Web search tasks. This paper suggests the use of the location of query terms occur in a URL for measuring how well a web page is matched with a user’s information need in web search. First, we define and estimate URL hit types, i.e. the priori probability of being a good answer given the type of query term hits in the URL. The main advantage of URL hit priors (over depth priors) is that it can achieve stable improvement for both informational and navigational queries. Second, an obstacle of exploiting such priors is that shortening and concatenation are frequently used in a URL. Our investigation shows that only 30% URL hits are recognized by an ordinary word breaking approach. Thus we combine three methods to improve matching. Finally, the priors are integrated into the probabilistic model for enhancing web document retrieval. Our experiments were conducted using 7 query sets of TREC2002, TREC2003 and TREC2004, and show that the proposed approach is stable and improve retrieval effectiveness by 4%~11% for navigational queries and 10% for informational queries.


Query Term Mean Reciprocal Rank Finding Query Navigational Query Informational Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. ACM Press, New York (1999)Google Scholar
  2. 2.
    Berger, J.: Statistical decision theory and Bayesian analysis. Springer, New York (1985)CrossRefzbMATHGoogle Scholar
  3. 3.
    Bharat, K., Henzinger, M.: Improved algorithms for topic distillation in a hyperlinked environment. In: 21st Annual International ACM SIGIR Conference, Melbourne, Australia, pp. 104–111 (August 1998)Google Scholar
  4. 4.
    Border, A.: A taxonomy of Web search. SIGIR Forum 36(2) (2002)Google Scholar
  5. 5.
    Chi, C.-H., Ding, C., Lim, A.: Word segmentation and recognition for web document framework. In: CIKM 1999 (1999)Google Scholar
  6. 6.
    Craswell, N., Robertson, S., Zaragoza, H., Taylor, M.: Relevance weight for query independent evidence. In: Proceedings of ACM SIGIR 2005, Salvador, Brazil (2005)Google Scholar
  7. 7.
    Hawking, D., Voorhees, E., Craswell, N., Bailey, P.: Overview of the TREC-8 web track. In: The Eighth Text Retrieval Conference (TREC8), NIST (2001)Google Scholar
  8. 8.
    Hu, Y., Xin, G., Song, R., Hu, G., Shi, S., Cao, Y., Li, H.: Title extraction from bodies of HTML documents and its application to Web page retrieval. In: Proceedings of SIGIR 2005, Salvador, Brazil (2005)Google Scholar
  9. 9.
    Kraaij, W., Westerveld, T., Hiemstra, D.: The importance of prior probabilities for entry page search. In: SIGIR 2002 (2001)Google Scholar
  10. 10.
    Lee, U., Liu, Z., Cho, J.: Automatic identification of user goals in Web search. In: The Proceedings of the Fourteenth Int’l World Wide Web Conference (WWW 2005), Chiba, Japan (2005)Google Scholar
  11. 11.
    Marchionini, G.: Interfaces for End-User Information Seeking. Journal of the American Society for Information Science 43(2), 156–163 (1992)CrossRefGoogle Scholar
  12. 12.
    Ogilvie, P., Callan, J.: Combining structural information and the use of priors in mixed named-page and homepage finding. In: TREC 2003 (2003)Google Scholar
  13. 13.
    Ra, D.-Y., Park, E.-K., Jang, J.-S.: Yonsi/etri at TREC-10: Utilizing web document properties. In: The Tenth Text Retrieval Conference (TREC 2001), NIST (2002)Google Scholar
  14. 14.
    Robertson, S.E., Walker, S.: Okapi/Keenbow at TREC-8. In: The Eighth Text Retrieval Conference (TREC 8), pp. 151–162 (1999)Google Scholar
  15. 15.
    Robertson, S.E., Sparck Jones, K.: Relevance weighting of search terms. Journal of the American Society of Information Science 27, 129–146 (1976)CrossRefGoogle Scholar
  16. 16.
    TREC-2004 Web Track Guidelines,
  17. 17.
    Rose, D.E., Levinson, D.: Understanding user goals in Web search. In: Proceedings of the Thirteenth Int’l World Wide Web Conference (WWW 2004), New York, USA (2004)Google Scholar
  18. 18.
    Qin, T., Liu, T.-Y., Zhang, X.-D., Chen, Z., Ma, W.-Y.: A study on relevance propagation for Web search. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2005), Salvador, Brazil (2005)Google Scholar
  19. 19.
  20. 20.
    Wen, J.-R., Song, R., Cai, D., Zhu, K., Yu, S., Ye, S., Ma, W.-Y.: Microsoft Research Asia at the Web Track of TREC 2003. In: The Twelfth Text Retrieval Conference (2003)Google Scholar
  21. 21.
    Westerveld, T., Kraaij, W., Hiemstra, D.: Retrieving web pages using content, links, URLs and anchors. In: TREC 2001 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ruihua Song
    • 1
  • Guomao Xin
    • 1
  • Shuming Shi
    • 1
  • Ji-Rong Wen
    • 1
  • Wei-Ying Ma
    • 1
  1. 1.Microsoft Research Asia, 5F, Sigma CenterBeijingP.R. China

Personalised recommendations