Abstract
We show that the complexity of the recently introduced medoid-shift algorithm in clustering N points is O(N 2), with a small constant, if the underlying distance is Euclidean. This makes medoid shift considerably faster than mean shift, contrarily to what previously believed. We then exploit kernel methods to extend both mean shift and the improved medoid shift to a large family of distances, with complexity bounded by the effective rank of the resulting kernel matrix, and with explicit regularization constraints. Finally, we show that, under certain conditions, medoid shift fails to cluster data points belonging to the same mode, resulting in over-fragmentation. We propose remedies for this problem, by introducing a novel, simple and extremely efficient clustering algorithm, called quick shift, that explicitly trades off under- and over-fragmentation. Like medoid shift, quick shift operates in non-Euclidean spaces in a straightforward manner. We also show that the accelerated medoid shift can be used to initialize mean shift for increased efficiency. We illustrate our algorithms to clustering data on manifolds, image segmentation, and the automatic discovery of visual categories.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Bach, F.R., Jordan, M.I.: Kernel independent componet analysis. Journal of Machine Learninig Research 3(1) (2002)
Carreira-Perpiñán, M.: Fast nonparametric clustering with gaussian blurring mean-shift. In: Proc. ICML (2006)
Cheng, Y.: Mean shift, mode seeking, and clustering. PAMI 17(8) (1995)
Choi, H., Choi, S.: Robust kernel isomap. Pattern Recognition (2006)
Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. PAMI 24(5) (2002)
Comaniciu, D., Ramesh, V., Meer, P.: The variable bandwidth mean shift and data-driven scale selection. In: Proc. ICCV (2001)
Csurka, G., Dance, C.R., Dan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Proc. ECCV (2004)
Fine, S., Scheinberg, K.: Efficient SVM training using low-rank kernel representations. Journal of Machine Learninig Research (2001)
Fukunaga, K., Hostler, L.D.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. 21(1) (1975)
Grauman, K., Darrell, T.: Unsupervised learning of categories from sets of partially matching image features. In: Proc. CVPR (2006)
Hein, M., Bousquet, O.: Hilbertian metrics and positive definite kernels on probability measures. In: Proc. AISTAT (2005)
Knuth, D.: The Art of Computer Programming: Seminumerical Algorithms, 3rd edn., vol. 2 (1998)
Koontz, W.L.G., Narendra, P., Fukunaga, K.: A graph-theoretic approach to nonparametric cluster analyisis. IEEE Trans. on Computers c-25(9) (1976)
Lowe, D.: Implementation of the scale invariant feature transform (2007), http://www.cs.ubc.ca/~lowe/keypoints/
Mika, S., Rätsch, G., Weston, J., Schölkopf, B., Müller, K.-R.: Fisher discriminant analysis with kernels. In: Proc. IEEE Neural Networks for Signal Processing Workshop (1999)
Paris, S., Durand, F.: A topological approach to hierarchical segmentation using mean shift. In: Proc. CVPR (2007)
Sain, S.R.: Multivariate locally adaptive density estimation. Comp. Stat. and Data Analysis, 39 (2002)
Schölkopf, B.: The kernel trick for distances. In: Proc. NIPS (2001)
Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
Sheikh, Y.A., Khan, E.A., Kanade, T.: Mode-seeking by medoidshifts. In: Proc. CVPR (2007)
Subbarao, R., Meer, P.: Nonlinear mean shift for clustering over analytic manifolds. In: Proc. CVPR (2006)
Sundaramoorthy, G., Yezzi, A., Mennucci, A.: Sobolev active contours. Int. J. Comput. Vision 73(3) (2007)
Yang, C., Duraiswami, R., Gumerov, N.A., Davis, L.: Improved fast Gauss transform and efficient kernel density estimation. In: Proc. ICCV (2003)
Yuan, X., Li, S.Z.: Half quadric analysis for mean shift: with extension to a sequential data mode-seeking method. In: Proc. CVPR (2007)
Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object categories: A comprehensive study IJCV (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Vedaldi, A., Soatto, S. (2008). Quick Shift and Kernel Methods for Mode Seeking. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5305. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88693-8_52
Download citation
DOI: https://doi.org/10.1007/978-3-540-88693-8_52
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88692-1
Online ISBN: 978-3-540-88693-8
eBook Packages: Computer ScienceComputer Science (R0)