Skip to main content
SpringerLink
Log in
Book cover

Australasian Database Conference

ADC 2015: Databases Theory and Applications pp 328–332Cite as

SEMI: A Scalable Entity Matching System Based on MapReduce

  • Conference paper
  • First Online:

  • 1506 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9093))

Abstract

MapReduce framework provides a new platform for data integration on distributed environment. We demonstrate a MapReduce-based entity resolution framework which efficiently solves the matching problem for structured, semi-structured and unstructured entities. We propose a random-based data representation method for reducing network transmission; we implement our design on MapReduce and design two solutions for reducing redundant comparisons. Our demo provides an easy-to-use platform for entity matching and performance analysis. We also compare the performance of our algorithm with the state-of-the-art blocking-based methods.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baraglia, R., De Francisci Morales, G., Lucchese, C.: Document similarity self-join with mapreduce. In: Proc. of ICDM, pp. 731–736 (2010)

    Google Scholar 

  2. Kolb, L., Thor, A., Rahm, E.: Dedoop: efficient deduplication with hadoop. Proc. of VLDB 5(12), 1878–1881 (2012)

    Article  Google Scholar 

  3. Ravichandran, D., Pantel, P., Hovy, E.: Randomized algorithms and nlp: using locality sensitive hash function for high speed noun clustering. In: Proc. of ACL, pp. 622–629 (2005)

    Google Scholar 

  4. Toutanova, K., Manning, C.D.: Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. In: Proc. of EMNLP, pp. 63–70 (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Data Science and Engineering, East China Normal University, Shanghai, China

    Pingfu Chao, Yuming Li, Junhua Fang, Xiaofeng He & Rong Zhang

  2. Shanghai Key Laboratory of Trustworthy Computing, East China Normal University, Shanghai, China

    Pingfu Chao, Yuming Li, Zhu Gao, Junhua Fang, Xiaofeng He & Rong Zhang

Authors
  1. Pingfu Chao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhu Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junhua Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaofeng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rong Zhang .

Editor information

Editors and Affiliations

  1. University of Queensland, Brisbane, Queensland, Australia

    Mohamed A. Sharaf

  2. Monash University, Clayton, Australia

    Muhammad Aamir Cheema

  3. The University of Melbourne, Melbourne, Australia

    Jianzhong Qi

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Chao, P., Li, Y., Gao, Z., Fang, J., He, X., Zhang, R. (2015). SEMI: A Scalable Entity Matching System Based on MapReduce. In: Sharaf, M., Cheema, M., Qi, J. (eds) Databases Theory and Applications. ADC 2015. Lecture Notes in Computer Science(), vol 9093. Springer, Cham. https://doi.org/10.1007/978-3-319-19548-3_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-19548-3_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-19547-6

  • Online ISBN: 978-3-319-19548-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation