ReFER: Effective Relevance Feedback for Entity Ranking

  • Tereza Iofciu
  • Gianluca Demartini
  • Nick Craswell
  • Arjen P. de Vries
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6611)


Web search increasingly deals with structured data about people, places and things, their attributes and relationships. In such an environment an important sub-problem is matching a user’s unstructured free-text query to a set of relevant entities. For example, a user might request ‘Olympic host cities’. The most challenging general problem is to find relevant entities, of the correct type and characteristics, based on a free-text query that need not conform to any single ontology or category structure. This paper presents an entity ranking relevance feedback model, based on example entities specified by the user or on pseudo feedback. It employs the Wikipedia category structure, but augments that structure with ‘smooth categories’ to deal with the sparseness of the raw category information. Our experiments show the effectiveness of the proposed method, whether applied as a pseudo relevance feedback method or interactively with the user in the loop.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bailey, P., Craswell, N., De Vries, A., Soboroff, I.: Overview of the TREC 2007 Enterprise Track. In: Proceedings of TREC 2007, Gaithersburg, MD (2008)Google Scholar
  2. 2.
    Balog, K., Bron, M., de Rijke, M.: Category-based query modeling for entity search. In: Gurrin, C., He, Y., Kazai, G., Kruschwitz, U., Little, S., Roelleke, T., Rüger, S., van Rijsbergen, K. (eds.) ECIR 2010. LNCS, vol. 5993, pp. 319–331. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Bast, H., Chitea, A., Suchanek, F., Weber, I.: Ester: efficient search on text, entities, and relations. In: SIGIR 2007, pp. 671–678. ACM, New York (2007)Google Scholar
  4. 4.
    Cheng, T., Chang, K.: Entity Search Engine: Towards Agile Best-Effort Information Integration over the Web. In: CIDR 2007, pp. 108–113 (2007)Google Scholar
  5. 5.
    Cheng, T., Yan, X., Chang, K.C.-C.: Entityrank: Searching entities directly and holistically. In: Proceedings of VLDB, pp. 387–398 (2007)Google Scholar
  6. 6.
    Demartini, G., Firan, C., Iofciu, T., Krestel, R., Nejdl, W.: Why finding entities in wikipedia is difficult, sometimes. Information Retrieval 13(5), 534–567 (2010)CrossRefGoogle Scholar
  7. 7.
    Demartini, G., Missen, M.M.S., Blanco, R., Zaragoza, H.: Entity summarization of news articles. In: SIGIR, pp. 795–796 (2010)Google Scholar
  8. 8.
    Denoyer, L., Gallinari, P.: The Wikipedia XML corpus. ACM SIGIR Forum. 40(1), 64–69 (2006)CrossRefGoogle Scholar
  9. 9.
    Huang, Z., Chung, W., Ong, T., Chen, H.: A graph-based recommender system for digital library. In: Proceedings of the 2nd ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 65–73. ACM, New York (2002)Google Scholar
  10. 10.
    Kumar, R., Tomkins, A.: A characterization of online search behavior. IEEE Data Eng. Bull. (2009)Google Scholar
  11. 11.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: Bringing order to the web (1998)Google Scholar
  12. 12.
    Pehcevski, J., Vercoustre, A.-M., Thom, J.A.: Exploiting locality of wikipedia links in entity ranking. In: Macdonald, C., Ounis, I., Plachouras, V., Ruthven, I., White, R.W. (eds.) ECIR 2008. LNCS, vol. 4956, pp. 258–269. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Rode, H., Serdyukov, P., Hiemstra, D.: Combining document- and paragraph-based entity ranking. In: SIGIR, pp. 851–852 (2008)Google Scholar
  14. 14.
    Ruthven, I., Lalmas, M.: A Survey on the Use of Relevance Feedback for Information Access Systems. Knowl. Eng. Rev. 18(2), 95–145 (2003)CrossRefGoogle Scholar
  15. 15.
    Serdyukov, P., Balog, K., Thomas, P., Vries, A., Westerveld, T.: Overview of the TREC 2009 Entity Track (2009)Google Scholar
  16. 16.
    Tsikrika, T., Serdyukov, P., Rode, H., Westerveld, T., Aly, R., Hiemstra, D., de Vries, A.P.: Structured document retrieval, multimedia retrieval, and entity ranking using pF/Tijah. In: Fuhr, N., Kamps, J., Lalmas, M., Trotman, A. (eds.) INEX 2007. LNCS, vol. 4862, pp. 306–320. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  17. 17.
    Vallet, D., Zaragoza, H.: Inferring the most important types of a query: a semantic approach. In: SIGIR, pp. 857–858 (2008)Google Scholar
  18. 18.
    de Vries, A.P., Vercoustre, A.-M., Thom, J.A., Craswell, N., Lalmas, M.: Overview of the INEX 2007 entity ranking track. In: Fuhr, N., Kamps, J., Lalmas, M., Trotman, A. (eds.) INEX 2007. LNCS, vol. 4862, pp. 245–251. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  19. 19.
    Yilmaz, E., Kanoulas, E., Aslam, J.A.: A simple and efficient sampling method for estimating ap and ndcg. In: SIGIR, pp. 603–610 (2008)Google Scholar
  20. 20.
    Zaragoza, H., Rode, H., Mika, P., Atserias, J., Ciaramita, M., Attardi, G.: Ranking very many typed entities on wikipedia. In: CIKM, pp. 1015–1018 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Tereza Iofciu
    • 1
  • Gianluca Demartini
    • 1
  • Nick Craswell
    • 2
  • Arjen P. de Vries
    • 3
  1. 1.L3S Research CenterHannoverGermany
  2. 2.Microsoft RedmondUSA
  3. 3.Centrum Wiskunde & InformaticaAmsterdamThe Netherlands

Personalised recommendations