Cold-Start Knowledge Base Population Using Ontology-Based Information Extraction with Conditional Random Fields

  • Hendrik ter Horst
  • Matthias Hartung
  • Philipp CimianoEmail author
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11078)


In this tutorial we discuss how Conditional Random Fields can be applied to knowledge base population tasks. We are in particular interested in the cold-start setting which assumes as given an ontology that models classes and properties relevant for the domain of interest, and an empty knowledge base that needs to be populated from unstructured text. More specifically, cold-start knowledge base population consists in predicting semantic structures from an input document that instantiate classes and properties as defined in the ontology. Considering knowledge base population as structure prediction, we frame the task as a statistical inference problem which aims at predicting the most likely assignment to a set of ontologically grounded output variables given an input document. In order to model the conditional distribution of these output variables given the input variables derived from the text, we follow the approach adopted in Conditional Random Fields. We decompose the cold-start knowledge base population task into the specific problems of entity recognition, entity linking and slot-filling, and show how they can be modeled using Conditional Random Fields.


Cold-start knowledge base population Ontology-based information extraction Slot filling Conditional random fields 



This work has been funded by the Federal Ministry of Education and Research (BMBF, Germany) in the PSINK project (project number 031L0028A).


  1. 1.
    Banko, M., Cafarella, M., Soderland, S., Broadhead, M., Etzioni, O.: Open information extraction from the web. In: Proceedings of IJCAI, pp. 2670–2676 (2007)Google Scholar
  2. 2.
    Brazda, N., et al.: SCIO: an ontology to support the formalization of pre-clinical spinal cord injury experiments. In: Proceedings of the 3rd JOWO Workshops: Ontologies and Data in the Life Sciences (2017)Google Scholar
  3. 3.
    Freitag, D.: Machine learning for information extraction in informal domains. Mach. Learn. 39(2–3), 169–202 (2000)CrossRefGoogle Scholar
  4. 4.
    Hartung, M., ter Horst, H., Grimm, F., Diekmann, T., Klinger, R., Cimiano, P.: SANTO: a web-based annotation tool for ontology-driven slot filling. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (System Demonstrations), Association for Computational Linguistics (2018). in pressGoogle Scholar
  5. 5.
    Hartung, M., Klinger, R., Zwick, M., Cimiano, P.: Towards gene recognition from rare and ambiguous abbreviations using a filtering approach. Proc. BioNLP 2014, 118–127 (2014)Google Scholar
  6. 6.
    Hoffart, J., et al.: Robust disambiguation of named entities in text. In: Proceedings of EMNLP, pp. 782–792 (2011)Google Scholar
  7. 7.
    ter Horst, H., Hartung, M., Cimiano, P.: Joint entity recognition and linking in technical domains using undirected probabilistic graphical models. In: Gracia, J., Bond, F., McCrae, J.P., Buitelaar, P., Chiarcos, C., Hellmann, S. (eds.) LDK 2017. LNCS (LNAI), vol. 10318, pp. 166–180. Springer, Cham (2017). Scholar
  8. 8.
    ter Horst, H., Hartung, M., Klinger, R., Brazda, N., Müller, H.W., Cimiano, P.: Assessing the impact of single and pairwise slot constraints in a factor graph model for template-based information extraction. In: Silberztein, M., Atigui, F., Kornyshova, E., Métais, E., Meziane, F. (eds.) NLDB 2018. LNCS, vol. 10859, pp. 179–190. Springer, Cham (2018). Scholar
  9. 9.
    Koller, D., Friedman, N.: Probabilistic Graphical Models. Principles and Techniques. MIT Press, Cambridge (2009)zbMATHGoogle Scholar
  10. 10.
    Kschischang, F.R., Frey, B.J., Loeliger, H.A.: Factor graphs and sum product algorithm. IEEE Trans. Inf. Theor. 47(2), 498–519 (2001)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields probabilistic models for segmenting and labeling sequence data. In: Proceedings of ICML, pp. 282–289 (2001)Google Scholar
  12. 12.
    Leaman, R., Lu, Z.: TaggerOne Joint named entity recognition and normalization with Semi-Markov Models. Bioinformatics 32, 2839–46 (2016)CrossRefGoogle Scholar
  13. 13.
    Leaman, R., Dogan, R.I., Lu, Z.: DNorm disease name normalization with pairwise learning to rank. Bioinformatics 29, 2909–2917 (2013)CrossRefGoogle Scholar
  14. 14.
    Min, B., Freedman, M., Meltzer, T.: Probabilistic inference for cold startknowledge base population with prior world knowledge. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics vol. 1, Long Papers, pp. 601–612. Association for Computational Linguistics, Valencia, Spain (April 2017)Google Scholar
  15. 15.
    Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: Proceedings of ACL, pp. 1003–1011 (2009)Google Scholar
  16. 16.
    Nadeau, D., Sekine, S.: A survey of named entity recognition and classification. Lingvisticae Invest. 30(1), 3–26 (2007)CrossRefGoogle Scholar
  17. 17.
    Piskorski, J., Yangarber, R.: Information extraction: past, present and future. In: Poibeau, T., Saggion, H., Piskorski, J., Yangarber, R. (eds.) Multi-source Multilingual Information Extraction and Summarization Theory and Applications of Natural Language Processing. Springer, Heidelberg (2013). Scholar
  18. 18.
    Poon, H., Domingos, P.: Machine reading: a killer app for statistical relational AI. In: Proceedings of StarAI, pp. 76–81 (2010)Google Scholar
  19. 19.
    Ratinov, L., Roth, D., Downey, D., Anderson, M.: Local and global algorithms for disambiguation to wikipedia. In: Proceedings of ACL:HLT, pp. 1375–1384 (2011)Google Scholar
  20. 20.
    Resnik, P., Hardisty, E.: Gibbs sampling for the uninitiated. Maryland Univ College Park Inst for Advanced Computer Studies, Technical report (2010)Google Scholar
  21. 21.
    Röder, M., Usbeck, R., Ngomo, A.C.N.: Gerbil-benchmarking named entity recognition and linking consistently. Semantic Web J. (2018),
  22. 22.
    Smith, N.A.: Linguistic Structure Prediction. Morgan and Claypool, San Rafael (2011)Google Scholar
  23. 23.
    Sutton, C., McCallum, A.: An introduction to conditional random fields. Foundations and Trends® in Machine Learning 4(4), 267–373 (2012)CrossRefGoogle Scholar
  24. 24.
    Wei, C.H., et al.: Overview of the biocreative V chemical disease relation (CDR) task. In: Proceedings of the BioCreative V Evaluation Workshop, pp. 154–166 (2015)Google Scholar
  25. 25.
    Wick, M., Rohanimanesh, K., Culotta, A., McCallum, A.: SampleRank learning preferences from atomic gradients. In: Proceedings of the NIPS Workshop on Advances in Ranking, pp. 1–5 (2009)Google Scholar
  26. 26.
    Wimalasuriya, D.C., Dou, D.: Ontology-based information extraction: an introduction and a survey of current approaches. J. Inf. Sci. 36(3), 306–323 (2010)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Hendrik ter Horst
    • 1
  • Matthias Hartung
    • 1
  • Philipp Cimiano
    • 1
    Email author
  1. 1.CITECBielefeld UniversityBielefeldGermany

Personalised recommendations