Consensus Clustering Using kNN Mode Seeking

  • Jonas Nordhaug MyhreEmail author
  • Karl Øyvind Mikalsen
  • Sigurd Løkse
  • Robert Jenssen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9127)


In this paper we present a novel clustering approach which combines two modern strategies, namely consensus clustering, and two stage clustering as represented by the mean shift spectral clustering algorithm. We introduce the recent kNN mode seeking algorithm in the consensus clustering framework, and the information theoretic kNN Cauchy Schwarz divergence as foundation for spectral clustering. In combining these frameworks, two well known problematic issues are directly bypassed; the kernel bandwidth choice of the kernel density based mean shift and the computational complexity of the mean shift iterations. We demonstrate experiments on both real and synthetic data as a proof of concept for our contributions.


Kernel Density Estimate Spectral Cluster Cluster Scheme True Label Consensus Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agersborg, J., Jenssen, R.: Mean shift spectral clustering using kernel entropy component analysis. In: Proceedings of IEEE Workshop on Machine Learning for Signal Processing, Reims, France, September 21-24, 2014Google Scholar
  2. 2.
    Bache, K., Lichman, M.: UCI machine learning repository (2013).
  3. 3.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)Google Scholar
  4. 4.
    Cheng, Y.: Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 17(8), 790–799 (1995)CrossRefGoogle Scholar
  5. 5.
    Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(5), 603–619 (2002)CrossRefGoogle Scholar
  6. 6.
    Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(5), 564–577 (2003)CrossRefGoogle Scholar
  7. 7.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons, New York (2001)zbMATHGoogle Scholar
  8. 8.
    Duin, R.P.W., Fred, A.L.N., Loog, M., Pękalska, E.: Mode seeking clustering by knn and mean shift evaluated. In: Gimel’farb, G., Hancock, E., Imiya, A., Kuijper, A., Kudo, M., Omachi, S., Windeatt, T., Yamada, K. (eds.) SSPR & SPR 2012. LNCS, vol. 7626, pp. 51–59. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  9. 9.
    Fred, A.: Finding consistent clusters in data partitions. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 309–318. Springer, Heidelberg (2001) CrossRefGoogle Scholar
  10. 10.
    Fred, A.L.N., Jain, A.K.: Data clustering using evidence accumulation. In: Proceedings of the 16th International Conference on Pattern Recognition, vol. 4, pp. 276–280. IEEE (2002)Google Scholar
  11. 11.
    Fred, A., Jain, A.K.: Evidence accumulation clustering based on the k-means algorithm. In: Caelli, T., Amin, A., Duin, R.P.W., de Ridder, D., Kamel, M. (eds.) SSPR&SPR 2002. LNCS, vol. 2396, pp. 442–451. Springer, Heidelberg (2002) CrossRefGoogle Scholar
  12. 12.
    Fred, A.L.N., Jain, A.K.: Combining multiple clusterings using evidence accumulation. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(6), 835–850 (2005)CrossRefGoogle Scholar
  13. 13.
    Fukunaga, K., Hostetler, L.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Trans. Inf. Theor. 21(1), 32–40 (2006)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Georgescu, B., Shimshoni, I., Meer, P.: Mean shift based clustering in high dimensions: a texture classification example. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, pp. 456–463. IEEE (2003)Google Scholar
  15. 15.
    Jenssen, R.: Kernel Entropy Component Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(5), 847–860 (2010)CrossRefGoogle Scholar
  16. 16.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  17. 17.
    Ng, A.Y., Jordan, M.I., Weiss, Y., et al.: On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems 2, 849–856 (2002)Google Scholar
  18. 18.
    Ozertem, U., Erdogmus, D., Jenssen, R.: Mean shift spectral clustering. Pattern Recognition 41(6), 1924–1938 (2008)zbMATHCrossRefGoogle Scholar
  19. 19.
    Principe, J.C., Xu, D., Fisher, J.: Information theoretic learning. Unsupervised Adaptive Filtering 1, 265–319 (2000)Google Scholar
  20. 20.
    Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., Moore, R.: Real-Time Human Pose Recognition in Parts from Single Depth Images. Communications of the ACM 56(1), 116–124 (2013)CrossRefGoogle Scholar
  21. 21.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Academic Press, San Diego (2009)Google Scholar
  22. 22.
    Vikjord, V.V., Jenssen, R.: Information theoretic clustering using a k-nearest neighbors approach. Pattern Recognition 47(9), 3070–3081 (2014)CrossRefGoogle Scholar
  23. 23.
    von Luxburg, U.: A Tutorial on Spectral Clustering. Statistics and Computing 17(4), 395–416 (2007)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Jonas Nordhaug Myhre
    • 1
    Email author
  • Karl Øyvind Mikalsen
    • 1
  • Sigurd Løkse
    • 1
  • Robert Jenssen
    • 1
  1. 1.Machine Learning @ UiT LabUiT - The Arctic University of NorwayTromsøNorway

Personalised recommendations