Advertisement

Optimal Dimension Order: A Generic Technique for the Similarity Join

  • Christian Böhm
  • Florian Krebs
  • Hans-Peter Kriegel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2454)

Abstract

The similarity join is an important database primitive which has been successfully applied to speed up applications such as similarity search, data analysis and data mining. The similarity join combines two point sets of a multidimensional vector space such that the result contains all point pairs where the distance does not exceed a given Parameter ∈. Although the similarity join is clearly CPU bound, most previous publications propose strategies that primarily improve the I/O performance. Only little effort has been taken to address CPU aspects. In this Paper, we show that most of the computational overhead is dedicated to the final distance computations between the feature vectors. Consequently, we propose a generic technique to reduce the response time of a large number of basic algorithms for the similarity join. It is applicable for index based join algorithms as well as for most join algorithms based on hashing or sorting. Our technique, called Optimal Dimension Order, is able to avoid and accelerate distance calculations between feature vectors by a careful order of the dimensions. The order is determined according to a probability model. In the experimental evaluation, we show that our technique yields high performance improvements for various underlying similarity join algorithms such as the R-tree similarity join, the breadth- first-R-tree join, the Multipage Index Join, and the ∈-Grid-Order.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Ankerst M., Breunig M.M., Kriegel H.-P., Sander J.: OPTICS: Ordering Points To Identify the Clustering Structure, ACM SIGMOD Int. Conf. on Management of Data, 1999.Google Scholar
  2. 2.
    Agrawal R., Lin K., Sawhney H., Shim K.: Fast Similarity Search in the Presence of Noise, Scaling, and Translation in Time-Series Databases, Int. Conf. on Very Large Data Bases (VLDB), 1995.Google Scholar
  3. 3.
    Arge L., Procopiuc O., Ramaswamy S., Suel T., Vitter J.S.: Scalable Sweeping-Based Spatial Join, Int. Conf. on Very Large Databases (VLDB), 1998.Google Scholar
  4. 4.
    Böhm C., Braunmüller B., Breunig M.M., Kriegel H.-P.: Fast Clustering Based on High-Dimensional Similarity Joins, Int. Conf. on Information Knowledge Management (CIKM), 2000.Google Scholar
  5. 5.
    Berchtold S., Böhm C., Jagadish H.V., Kriegel H.-P., Sander J.: Independent Quantization: An Index Compression Technique for High Dimensional Spaces, IEEE Int. Conf. on Data Engineering (ICDE), 2000.Google Scholar
  6. 6.
    Berchtold S., Böhm C., Keim D., Kriegel H.-P.: A Cost Model For Neurest Neighbor Search in High-Dimensional Data Space, ACM Symposium on Principles of Database Systems (PODS), 1997.Google Scholar
  7. 7.
    Böhm C., Braunmüller B., Krebs F., Kriegel H.-P.: Epsilon Grid Order: An Algorithm for the Similarity Join on Massive High-Dimensional Data, ACM SIGMOD Int. Conf. on Management of Data, 2001.Google Scholar
  8. 8.
    Böhm C., Kriegel H.-P.: A Cost Model und Index Architecture for the Similarity Join, IEEE Int. Conf. on Data Engineering (ICDE), 2001.Google Scholar
  9. 9.
    Brinkho. T., Kriegel H.-P., Seeger B.: Efficient Processing of Spatial Joins Using R-trees, ACM SIGMOD Int. Conf. on Management of Data, 1993.Google Scholar
  10. 10.
    Brinkho. T., Kriegel H.-P., Seeger B.: Parallel Processing of Spatial Joins Sing R-trees, IEEE Int. Conf. on Data Engineering (ICDE), 1996.Google Scholar
  11. 11.
    Huang Y.-W., Jing N., Rundensteiner E. A.: Spatial Joins Using R-trees: Breadth-First Traversal with Global Optimizations, Int. Conf. on Very Large Databases (VLDB), 1997.Google Scholar
  12. 12.
    Koudas N., Sevcik C.: Size Separation Spatial Join, ACM SIGMOD Int. Conf. on Managern. of Data, 1997.Google Scholar
  13. 13.
    Koudas N., Sevcik C.: High Dimensional Similarity Joins: Algorithms and Performance Evaluation, IEEE Int. Conf. on Data Engineering (ICDE), Best Paper Award, 1998.Google Scholar
  14. 14.
    Lo M.-L., Ravishankar C.V.: Spatial Joins Using Seeded Trees, ACM SIGMOD Int. Conf., 1994.Google Scholar
  15. 15.
    Lo M.-L., Ravishankar C.V.: Spatial Hash Joins, ACM SIGMOD Int. Conf, 1996.Google Scholar
  16. 16.
    Patel J.M., DeWitt D.J., Partition Based Spatial-Merge Join, ACM SIGMOD Int. Conf., 1996.Google Scholar
  17. 17.
    Preparata F.P., Shamos M.I.: ‘Computational Geometry’, Chapter 5 (‘Proximity: Fundamental Algorithms’), Springer Verlag New York, 1985.Google Scholar
  18. 18.
    Sander J., Ester M., Kriegel H.-P., Xu X.: Density-Based Clustering in Spatial Databases: The Algorithm GDBSCAN and its Applications, Data Mining and Knowledge Discovery, Vol. 2, No. 2, 1998.Google Scholar
  19. 19.
    Shim K., Srikant R., Agrawal R.: High-Dimensional Similarity Joins, Int. Conf. on Data Engineering, 1997.Google Scholar
  20. 20.
    Ullman J.D.: Database and Knowledge-Base Systems, Vol. II, Computer Science Press, Rockville MD, 1989Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Christian Böhm
    • 1
  • Florian Krebs
    • 2
  • Hans-Peter Kriegel
    • 2
  1. 1.University for Health Informatics and TechnologyInnsbruck
  2. 2.University of MunichMunich

Personalised recommendations