Encyclopedia of Machine Learning and Data Mining

Living Edition
| Editors: Claude Sammut, Geoffrey I. Webb

Record Linkage

Living reference work entry
DOI: https://doi.org/10.1007/978-1-4899-7502-7_712-1


Many data mining and machine learning projects require information from various data sources to be integrated and linked before they can be used for further analysis. A crucial task of such data integration is to identify which records refer to the same real-world entities across databases when no common entity identifiers are available and when records can contain errors and variations. This process of record linkage therefore has to rely upon the attributes that are available in the databases to be linked. For databases that contain personal information, for example, of customers, taxpayers, or patients, these are commonly their names, addresses, phone numbers, and dates of birth.To improve the scalability of the linkage process, blocking or indexing techniques are commonly applied to limit the comparison of records to pairs or groups that likely correspond to the same entity. Records are compared using a variety of comparison functions, most commonly approximate string comparators that account for typographical errors and variations in textual attributes. The compared records are then classified into matches, non-matches, and potential matches, depending upon the decision model used. If training data in the form of true matches and non-matches are available, supervised classification techniques can be employed. However, in many practical record linkage applications, no ground truth data are available, and therefore unsupervised approaches are required. An approach known as probabilistic record linkage is commonly employed. In this article we provide an overview of record linkage with an emphasis on the classification aspects of this process.

This is a preview of subscription content, log in to check access

Recommended Reading

  1. Bhattacharya I, Getoor L (2007) Collective entity resolution in relational data. ACM Trans Knowl Discov Data 1(1), 5-es, pp 1–35Google Scholar
  2. Bloothooft G, Christen P, Mandemakers K, Schraagen M (2015) Population reconstruction. Springer, ChamCrossRefGoogle Scholar
  3. Christen P (2012) Data matching – concepts and techniques for record linkage, entity resolution, and duplicate detection. Data-centric systems and applications. Springer, Berlin/New YorkGoogle Scholar
  4. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B 19:380–393MathSciNetMATHGoogle Scholar
  5. Dunn H (1946) Record linkage. Am J Publ Health 36(12):1412CrossRefGoogle Scholar
  6. Fellegi IP, Sunter AB (1969) A theory for record linkage. J Am Stat Assoc 64(328):1183–1210CrossRefMATHGoogle Scholar
  7. Ferrante A, Boyd J (2012) A transparent and transportable methodology for evaluating data linkage software. J Biomed Inf 45(1):165–172CrossRefGoogle Scholar
  8. Herzog TN, Scheuren FJ, Winkler WE (2007) Data quality and record linkage techniques. Springer, New York/LondonMATHGoogle Scholar
  9. Herzog TN, Scheuren FJ, Winkler WE (2010) Record linkage. Wiley Interdiscip Rev Comput Stat 2(5): 535–543CrossRefMATHGoogle Scholar
  10. Kum HC, Krishnamurthy A, Machanavajjhala A, Ahalt SC (2014) Social genome: putting big data to work for population informatics. IEEE Comput 47(1):56–63CrossRefGoogle Scholar
  11. Lahiri P, Larsen M (2005) Regression analysis with linked data. J Am Stat Assoc 100:222–230MathSciNetCrossRefMATHGoogle Scholar
  12. Larsen MD, Rubin DB (2001) Iterative automated record linkage using mixture models. J Am Stat Assoc 96(453):32–41MathSciNetCrossRefGoogle Scholar
  13. Li P, Dong XL, Maurino A, Srivastava D (2011) Linking temporal records. The VLDB conference was in Seattle, WA. In: Proceedings of the VLDB endowment, Seattle, vol 4, issue 11Google Scholar
  14. Newcombe H, Kennedy J, Axford S, James A (1959) Automatic linkage of vital records. Science 130(3381):954–959CrossRefGoogle Scholar
  15. On BW, Koudas N, Lee D, Srivastava D (2007) Group linkage. In: IEEE international conference on data engineering, Istanbul, pp 496–505Google Scholar
  16. Ramadan B, Christen P, Liang H, Gayler RW (2015) Dynamic sorted neighborhood indexing for real time entity resolution. ACM J Data Inf Qual 6(4):15Google Scholar
  17. Vatsalan D, Christen P, Verykios VS (2013) A taxonomy of privacy-preserving record linkage techniques. Elsevier Inf Syst 38(6):946–969CrossRefGoogle Scholar
  18. Winkler WE (1988) Using the EM algorithm for weight computation in the Fellegi-Sunter model of record linkage. The American Statistical Association that is located in Alexandria, VA publishes the proceedings. In: Proceedings of the section on survey research methods, New Orleans, Washington, pp 667–671Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Research School of Computer ScienceThe Australian National UniversityCanberraAustralia
  2. 2.US Census BureauSuitlandUSA