The VLDB Journal

, Volume 20, Issue 4, pp 495–520

Dynamic constraints for record matching

  • Wenfei Fan
  • Hong Gao
  • Xibei Jia
  • Jianzhong Li
  • Shuai Ma
Regular Paper

DOI: 10.1007/s00778-010-0206-6

Cite this article as:
Fan, W., Gao, H., Jia, X. et al. The VLDB Journal (2011) 20: 495. doi:10.1007/s00778-010-0206-6
  • 241 Downloads

Abstract

This paper investigates constraints for matching records from unreliable data sources. (a) We introduce a class of matching dependencies (mds) for specifying the semantics of unreliable data. As opposed to static constraints for schema design, mds are developed for record matching, and are defined in terms of similarity predicates and a dynamic semantics. (b) We identify a special case of mds, referred to as relative candidate keys (rcks), to determine what attributes to compare and how to compare them when matching records across possibly different relations. (c) We propose a mechanism for inferring mds, a departure from traditional implication analysis, such that when we cannot match records by comparing attributes that contain errors, we may still find matches by using other, more reliable attributes. Moreover, we develop a sound and complete system for inferring mds. (d) We provide a quadratic-time algorithm for inferring mds and an effective algorithm for deducing a set of high-quality rcks from mds. (e) We experimentally verify that the algorithms help matching tools efficiently identify keys at compile time for matching, blocking or windowing and in addition, that the md-based techniques effectively improve the quality and efficiency of various record matching methods.

Keywords

Deduction Inference Matching dependencies Record matching Data cleaning 

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Wenfei Fan
    • 1
    • 3
  • Hong Gao
    • 3
  • Xibei Jia
    • 2
  • Jianzhong Li
    • 3
  • Shuai Ma
    • 2
  1. 1.University of EdinburghEdinburghUK
  2. 2.School of InformaticsUniversity of EdinburghEdinburghUK
  3. 3.Harbin Institute of TechnologyHarbinChina

Personalised recommendations