Finding Relevant Relations in Relevant Documents

  • Michael SchuhmacherEmail author
  • Benjamin Roth
  • Simone Paolo Ponzetto
  • Laura Dietz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9626)


This work studies the combination of a document retrieval and a relation extraction system for the purpose of identifying query-relevant relational facts. On the TREC Web collection, we assess extracted facts separately for correctness and relevance. Despite some TREC topics not being covered by the relation schema, we find that this approach reveals relevant facts, and in particular those not yet known in the knowledge base DBpedia. The study confirms that mention frequency, document relevance, and entity relevance are useful indicators for fact relevance. Still, the task remains an open research problem.


Query Expansion Test Collection Relation Extraction Document Retrieval Entity Relevance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was in part funded by the Deutsche Forschungsgemeinschaft within the JOIN-T project (research grant PO 1900/1-1), in part by DARPA under agreement number FA8750-13-2-0020, through the Elitepostdoc program of the BW-Stiftung, an Amazon AWS grant in education, and by the Center for Intelligent Information Retrieval. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsor. We are also thankful for the support of Amina Kadry and the helpful comments of the anonymous reviewers.


  1. 1.
    Bizer, C., Lehmann, J., Kobilarov, G., Auer, S., Becker, C., Cyganiak, R., Hellmann, S.: DBpedia — A crystallization point for the web of data. J. Web Semant. 7(3), 154–165 (2009)CrossRefGoogle Scholar
  2. 2.
    Blanco, R., Zaragoza, H.: Finding support sentences for entities. In: Proceedings of SIGIR 2010, pp. 339–346 (2010)Google Scholar
  3. 3.
    Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E.R., Mitchell, T.M.: Toward an architecture for never-ending language learning. In: Proceedings of AAAI 2010, pp. 1306–1313 (2010)Google Scholar
  4. 4.
    Dalton, J., Dietz, L., Allan, J.: Entity query feature expansion using knowledge base links. In: Proceedings of SIGIR-2014, pp. 365–374 (2014)Google Scholar
  5. 5.
    Fader, A., Soderland, S., Etzioni, O.: Identifying relations for open information extraction. In: Proceedings of EMNLP 2011, pp. 1535–1545 (2011)Google Scholar
  6. 6.
    Gabrilovich, E., Ringgaard, M., Subramanya, A.: FACC1: Freebase annotation of ClueWeb corpora, Version 1 (2013)Google Scholar
  7. 7.
    Roth, B., Barth, T., Chrupała, G., Gropp, M., Klakow, D.: Relationfactory: A fast, modular and effective system for knowledge base population. In: Proceedings of EACL 2014, p. 89 (2014)Google Scholar
  8. 8.
    Schuhmacher, M., Dietz, L., Ponzetto, S.P.: Ranking entities for web queries through text and knowledge. In: Proceedings of CIKM 2015 (2015)Google Scholar
  9. 9.
    Voskarides, N., Meij, E., Tsagkias, M., de Rijke, M., Weerkamp, W.: Learning to explain entity relationships in knowledge graphs. In: Proceedings of ACL 2015, pp. 564–574 (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Michael Schuhmacher
    • 1
    Email author
  • Benjamin Roth
    • 2
  • Simone Paolo Ponzetto
    • 1
  • Laura Dietz
    • 1
  1. 1.Data and Web Science GroupUniversity of MannheimMannheimGermany
  2. 2.College of Information and Computer ScienceUniversity of MassachusettsAmherstUSA

Personalised recommendations