The Similarity-Aware Relational Intersect Database Operator

  • Wadha J. Al Marri
  • Qutaibah Malluhi
  • Mourad Ouzzani
  • Mingjie Tang
  • Walid G. Aref
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8821)


Identifying similarities in large datasets is an essential operation in many applications such as bioinformatics, pattern recognition, and data integration. To make the underlying database system similarity-aware, the core relational operators have to be extended. Several similarity-aware relational operators have been proposed that introduce similarity processing at the database engine level, e.g., similarity joins and similarity group-by. This paper extends the semantics of the set intersection operator to operate over similar values. The paper describes the semantics of the similarity-based set intersection operator, and develops an efficient query processing algorithm for evaluating it. The proposed operator is implemented inside an open-source database system, namely PostgreSQL. Several queries from the TPC-H benchmark are extended to include similarity-based set intersetion predicates. Performance results demonstrate up to three orders of magnitude speedup in performance over equivalent queries that only employ regular operators.


Database Operator Query Processing Algorithm Output Tuples Equivalent Query Relational Intersect 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Narayanan, M., Karp, R.M.: Gapped local similarity search with provable guarantees. In: Jonassen, I., Kim, J. (eds.) WABI 2004. LNCS (LNBI), vol. 3240, pp. 74–86. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Wang, J., Li, G., Feng, J.: Fast-join: An efficient method for fuzzy token matching based string similarity join. In: ICDE (2011)Google Scholar
  3. 3.
    Schallehn, E., Sattler, K.U., Saake, G.: Efficient similarity-based operations for data integration. Data and Knowledge Engineering 48(3) (2004)Google Scholar
  4. 4.
    Mills, P.: Efficient statistical classification of satellite measurements. International Journal of Remote Sensing 32(21) (2011)Google Scholar
  5. 5.
    Silva, Y.N., Aref, W.G., Ali, M.H.: The similarity join database operator. In: ICDE (2010)Google Scholar
  6. 6.
    Silva, Y.N., Aref, W.G., Ali, M.H.: Similarity group-by. In: ICDE (2009)Google Scholar
  7. 7.
    Silva, Y.N., Aref, W.G., Larson, P., Pearson, S., Ali, M.H.: Similarity queries: their conceptual evaluation, transformations, and processing. VLDB J. 22(3) (2013)Google Scholar
  8. 8.
    Marri, W.J.A.: Similarity-aware set operators. Master’s thesis, Qatar University (2009)Google Scholar
  9. 9.
    Wang, J., Li, G., Fe, J.: Fast-join: An efficient method for fuzzy token matching based string similarity join. In: ICDE (2011)Google Scholar
  10. 10.
    Schallehn, E., Sattler, K., Saake, G.: Advanced grouping and aggregation for data integration. In: CIKM (2001)Google Scholar
  11. 11.
    Yu, C., Cui, B., Wang, S., Su, J.: Efficient index-based knn join processing for high-dimensional data. Journal of Information and Software Technology 49(4) (2007)Google Scholar
  12. 12.
    Hjaltason, G., Samet, H.: Incremental distance join algorithms for spatial databases. In: SIGMOD (1998)Google Scholar
  13. 13.
    Arasu, A., Ganti, V., Kaushik, R.: Efficient exact set-similarity joins. In: VLDB (2006)Google Scholar
  14. 14.
    Böhm, C., Krebs, F.: The k-nearest neighbour join: Turbo charging the kdd process. Knowledge and Information Systems 6(6) (2004)Google Scholar
  15. 15.
    Gao, L., Wang, M., Wang, X.S., Padmanabhan, S.: Expressing and optimizing similarity-based queries in sql. In: Atzeni, P., Chu, W., Lu, H., Zhou, S., Ling, T.-W. (eds.) ER 2004. LNCS, vol. 3288, pp. 464–478. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Barioni, M.C.N., Razente, H.L., Traina Jr., C., Traina, A.J.M.: Querying complex objects by similarity in sql. In: SBBD (2005)Google Scholar
  17. 17.
    Barioni, M.C.N., Razente, H.L., Traina, A.J.M., Traina Jr., C.: Siren: A similarity retrieval engine for complex data. In: VLDB (2006)Google Scholar
  18. 18.
    Silva, Y.N., Aly, A.M., Aref, W.G., Larson, P.Å.: Simdb: a similarity-aware database system. In: SIGMOD (2010)Google Scholar
  19. 19.
    PostgreSQL Global Development Group: Postgresql (2014),
  20. 20.
    TPCH: Tpc-h version 2.15.0 (2014),
  21. 21.
    Intel Berkeley Research lab: Intel lab data (2014),

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Wadha J. Al Marri
    • 1
  • Qutaibah Malluhi
    • 1
  • Mourad Ouzzani
    • 2
  • Mingjie Tang
    • 3
  • Walid G. Aref
    • 3
  1. 1.Qatar UniversityDohaQatar
  2. 2.Qatar Computing Research InstituteDohaQatar
  3. 3.Purdue UniversityWest LafayetteUSA

Personalised recommendations