Advertisement

Exact and Efficient Proximity Graph Computation

  • Michail Kazimianec
  • Nikolaus Augsten
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6295)

Abstract

Graph Proximity Cleansing (GPC) is a string clustering algorithm that automatically detects cluster borders and has been successfully used for string cleansing. For each potential cluster a so-called proximity graph is computed, and the cluster border is detected based on the proximity graph. Unfortunately, the computation of the proximity graph is expensive and the state-of-the-art GPC algorithms only approximate the proximity graph using a sampling technique.

In this paper we propose two efficient algorithms for the exact computation of proximity graphs. The first algorithm, PG-DS, is based on a divide-skip technique for merging inverted lists, the second algorithm, PG-SM, uses a sort-merge join strategy to compute the proximity graph. While the state-of-the-art solutions only approximate the correct proximity graph, our algorithms are exact. We experimentally evaluate our solution on large real world datasets and show that our algorithms are faster than the sampling-based approximation algorithms, even for very small sample sizes.

Keywords

Exact Algorithm Normalize Mutual Information Inverted List Approximate String Match Proximity Graph 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Mazeika, A., Böhlen, M.H.: Cleansing databases of misspelled proper nouns. In: CleanDB (2006)Google Scholar
  2. 2.
    Li, C., Lu, J., Lu, Y.: Efficient merging and filtering algorithms for approximate string searches. In: ICDE 2008: Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, pp. 257–266. IEEE Computer Society, Los Alamitos (2008)CrossRefGoogle Scholar
  3. 3.
    Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge Univ. Press, Cambridge (2008)zbMATHGoogle Scholar
  4. 4.
    Gravano, L., Ipeirotis, P.G., Jagadish, H.V., Koudas, N., Muthukrishnan, S., Pietarinen, L., Srivastava, D.: Using q-grams in a dbms for approximate string processing. IEEE Data Eng. Bull. 24(4), 28–34 (2001)Google Scholar
  5. 5.
    Li, C., Wang, B., Yang, X.: Vgram: Improving performance of approximate queries on string collections using variable-length grams. In: VLDB (2007)Google Scholar
  6. 6.
    Levenshtein, V.: Binary Codes Capable of Correcting Deletions, Insertions and Reversals. Soviet Physics Doklady 10, 707 (1966)MathSciNetGoogle Scholar
  7. 7.
    Landau, G.M., Vishkin, U.: Fast parallel and serial approximate string matching. J. Algorithms 10(2), 157–169 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Waterman, M.S., Smith, T.F., Beyer, W.A.: Some biological sequence metrics. Advances in Mathematics 20(3), 367–387 (1976)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Bilenko, M., Mooney, R., Cohen, W., Ravikumar, P., Fienberg, S.: Adaptive name matching in information integration. In: IEEE Intelligent Systems (2003)Google Scholar
  10. 10.
    Navarro, G.: A guided tour to approximate string matching. ACM Comput. Surv. 33(1), 31–88 (2001)CrossRefGoogle Scholar
  11. 11.
    Chaudhuri, S., Ganti, V., Motwani, R.: Robust identification of fuzzy duplicates. In: International Conference on Data Engineering, pp. 865–876 (2005)Google Scholar
  12. 12.
    Xiao, C., Wang, W., Lin, X., Yu, J.X.: Efficient similarity joins for near duplicate detection. In: WWW 2008: Proceeding of the 17th International Conference on World Wide Web, pp. 131–140. ACM, New York (2008)CrossRefGoogle Scholar
  13. 13.
    Gravano, L., Ipeirotis, P.G., Jagadish, H.V., Koudas, N., Muthukrishnan, S., Srivastava, D.: Approximate string joins in a database (almost) for free. In: VLDB 2001: Proceedings of the 27th International Conference on Very Large Data Bases, pp. 491–500. Morgan Kaufmann Publishers Inc., San Francisco (2001)Google Scholar
  14. 14.
    Augsten, N., Böhlen, M.H., Dyreson, C.E., Gamper, J.: Approximate joins for data-centric XML. In: ICDE, pp. 814–823. IEEE, Los Alamitos (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Michail Kazimianec
    • 1
  • Nikolaus Augsten
    • 1
  1. 1.Faculty of Computer ScienceFree University of Bozen-BolzanoBozenItaly

Personalised recommendations