Information Integration of Partially Labeled Data

  • Steffen Rendle
  • Lars Schmidt-Thieme
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


A central task when integrating data from different sources is to detect identical items. For example, price comparison websites have to identify offers for identical products. This task is known, among others, as record linkage, object identification, or duplicate detection.

In this work, we examine problem settings where some relations between items are given in advance — for example by EAN article codes in an e-commerce scenario or by manually labeled parts. To represent and solve these problems we bring in ideas of semi-supervised and constrained clustering in terms of pairwise must-link and cannot-link constraints. We show that extending object identification by pairwise constraints results in an expressive framework that subsumes many variants of the integration problem like traditional object identification, matching, iterative problems or an active learning setting.

For solving these integration tasks, we propose an extension to current object identification models that assures consistent solutions to problems with constraints. Our evaluation shows that additionally taking the labeled data into account dramatically increases the quality of state-of-the-art object identification systems.


Information Integration Record Linkage Consistent Solution Pairwise Constraint Levenshtein Distance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. BASU, S. and BILENKO, M. and MOONEY, R. J. (2004): A Probabilistic Framework for Semi-Supervised Clustering. In: Proceedings of the 10th International Conference on Knowledge Discovery and Data Mining (KDD-2004).Google Scholar
  2. BILENKO, M. and MOONEY, R. J. (2003): Adaptive Duplicate Detection Using Learn-able String Similarity Measures. In: Proceedings of the 9th International Conference on Knowledge Discovery and Data Mining (KDD-2004).Google Scholar
  3. COHEN, W. W. and RICHMAN, J. (2002): Learning to Match and Cluster Large High-Dimensional Data Sets for Data Integration. In: Proceedings of the 8th International Conference on Knowledge Discovery and Data Mining (KDD-2002).Google Scholar
  4. MCCALLUM, A. K., NIGAM K. and UNGAR L. (2000): Efficient Clustering of High-Dimensional Data Sets with Application to Reference Matching. In: Proceedings of the 6th International Conference On Knowledge Discovery and Data Mining (KDD-2000).Google Scholar
  5. NEILING, M. (2005): Identification of Real-World Objects in Multiple Databases. In: Pro-ceedings of GfKl Conference 2005.Google Scholar
  6. RENDLE, S. and SCHMIDT-THIEME, L. (2006): Object Identification with Constraints. In: Proceedings of 6th IEEE International Conference on Data Mining (ICDM-2006).Google Scholar
  7. WINKLER W. E. (1999): The State of Record Linkage and Current Research Problems. Tech-nical report, Statistical Research Division, U.S. Census Bureau.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Steffen Rendle
    • 1
  • Lars Schmidt-Thieme
    • 1
  1. 1.Information Systems and Machine Learning LabUniversity of HildesheimGermany

Personalised recommendations