The VLDB Journal

, Volume 26, Issue 1, pp 81–105

Incremental knowledge base construction using DeepDive

  • Christopher De Sa
  • Alex Ratner
  • Christopher Ré
  • Jaeho Shin
  • Feiran Wang
  • Sen Wu
  • Ce Zhang
Special Issue Paper

DOI: 10.1007/s00778-016-0437-2

Cite this article as:
De Sa, C., Ratner, A., Ré, C. et al. The VLDB Journal (2017) 26: 81. doi:10.1007/s00778-016-0437-2
  • 255 Downloads

Abstract

Populating a database with information from unstructured sources—also known as knowledge base construction (KBC)—is a long-standing problem in industry and research that encompasses problems of extraction, cleaning, and integration. In this work, we describe DeepDive, a system that combines database and machine learning ideas to help develop KBC systems, and we present techniques to make the KBC process more efficient. We observe that the KBC process is iterative, and we develop techniques to incrementally produce inference results for KBC systems. We propose two methods for incremental inference, based, respectively, on sampling and variational techniques. We also study the trade-off space of these methods and develop a simple rule-based optimizer. DeepDive includes all of these contributions, and we evaluate DeepDive on five KBC systems, showing that it can speed up KBC inference tasks by up to two orders of magnitude with negligible impact on quality.

Keywords

Knowledge base construction Incremental Performance 

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Christopher De Sa
    • 1
  • Alex Ratner
    • 1
  • Christopher Ré
    • 1
  • Jaeho Shin
    • 1
  • Feiran Wang
    • 1
  • Sen Wu
    • 1
  • Ce Zhang
    • 1
  1. 1.Stanford UniversityStanfordUSA

Personalised recommendations