Spectral Clustering Using PCKID – A Probabilistic Cluster Kernel for Incomplete Data

  • Sigurd LøkseEmail author
  • Filippo M. Bianchi
  • Arnt-Børre Salberg
  • Robert Jenssen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10269)


In this paper, we propose PCKID, a novel, robust, kernel function for spectral clustering, specifically designed to handle incomplete data. By combining posterior distributions of Gaussian Mixture Models for incomplete data on different scales, we are able to learn a kernel for incomplete data that does not depend on any critical hyperparameters, unlike the commonly used RBF kernel. To evaluate our method, we perform experiments on two real datasets. PCKID outperforms the baseline methods for all fractions of missing values and in some cases outperforms the baseline methods with up to 25% points.


Missing data Robustness Kernel methods Spectral clustering 



This work was partially funded by the Norwegian Research Council FRIPRO grant no. 239844 on developing the Next Generation Learning Machines.


  1. 1.
    Chechik, G., Heitz, G., Elidan, G., Abbeel, P., Koller, D.: Max-margin classification of data with absent features. JMLR 9, 1–21 (2008)zbMATHGoogle Scholar
  2. 2.
    Dhillon, I.S., Guan, Y., Kulis, B.: Kernel k-means: spectral clustering and normalized cuts. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 551–556. ACM (2004)Google Scholar
  3. 3.
    Dixon, J.K.: Pattern recognition with partly missing data. IEEE Trans. Syst. Man Cybern. 9(10), 617–621 (1979)CrossRefGoogle Scholar
  4. 4.
    Filippone, M., Camastra, F., Masulli, F., Rovetta, S.: A survey of kernel and spectral methods for clustering. Pattern Recogn. 41(1), 176–190 (2008)CrossRefzbMATHGoogle Scholar
  5. 5.
    Ghahramani, Z., Jordan, M.I.: Supervised learning from incomplete data via an EM approach. In: Advances in Neural Information Processing Systems, pp. 120–120 (1994)Google Scholar
  6. 6.
    Izquierdo-Verdiguier, E., Jenssen, R., Gómez-Chova, L., Camps-Valls, G.: Spectral clustering with the probabilistic cluster kernel. Neurocomputing 149, 1299–1304 (2015)CrossRefGoogle Scholar
  7. 7.
    Jain, A.K.: Data clustering: 50 years beyond k-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)CrossRefGoogle Scholar
  8. 8.
    Jenssen, R.: Kernel entropy component analysis. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 847–860 (2010)CrossRefGoogle Scholar
  9. 9.
    Kuhn, H.W.: The hungarian method for the assignment problem. Naval Res. logistics Q. 2(1–2), 83–97 (1955)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Lin, T.I., Lee, J.C., Ho, H.J.: On fast supervised learning for normal mixture models with missing information. Pattern Recogn. 39(6), 1177–1187 (2006)CrossRefzbMATHGoogle Scholar
  11. 11.
    Marlin, B.M.: Missing data problems in machine learning. Ph.D. thesis, University of Toronto (2008)Google Scholar
  12. 12.
    Mojirsheibani, M., Montazeri, Z.: Statistical classification with missing covariates. J. Roy. Stat. Soci. Ser. B (Statistical Methodology) 69(5), 839–857 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Monti, S., Tamayo, P., Mesirov, J., Golub, T.: Consensus clustering: a resampling-based method for class discovery and visualization of gene expression microarray data. Mach. Learn. 52(1–2), 91–118 (2003)CrossRefzbMATHGoogle Scholar
  14. 14.
    Ng, A.Y., Jordan, M.I., Weiss, Y., et al.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems, pp. 849–856 (2001)Google Scholar
  15. 15.
    Nie, F., Zeng, Z., Tsang, I.W., Xu, D., Zhang, C.: Spectral embedded clustering: a framework for in-sample and out-of-sample spectral clustering. IEEE Trans. Neural Netw. 22(11), 1796–1808 (2011)CrossRefGoogle Scholar
  16. 16.
    Salberg, A.B.: Land cover classification of cloud-contaminated multitemporal high-resolution images. IEEE Trans. Geosci. Remote Sens. 49(1), 377–387 (2011)CrossRefGoogle Scholar
  17. 17.
    Salberg, A.B., Jenssen, R.: Land-cover classification of partly missing data using support vector machines. Int. J. Remote Sens. 33(14), 4471–4481 (2012)CrossRefGoogle Scholar
  18. 18.
    Schölkopf, B., Smola, A., Müller, K.-R.: Kernel principal component analysis. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 583–588. Springer, Heidelberg (1997). doi: 10.1007/BFb0020217 Google Scholar
  19. 19.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)CrossRefGoogle Scholar
  20. 20.
    Stella, X.Y., Shi, J.: Multiclass spectral clustering. In: ICCV, pp. 313–319 (2003)Google Scholar
  21. 21.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Academic Press, 4th edn. (2008)Google Scholar
  22. 22.
    Von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Yang, Y., Xu, D., Nie, F., Yan, S., Zhuang, Y.: Image clustering using local discriminant models and global integration. IEEE Trans. Image Process. 19(10), 2761–2773 (2010)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Zimek, A., Gaudet, M., Campello, R.J., Sander, J.: Subsampling for efficient and effective unsupervised outlier detection ensembles. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 428–436. ACM (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Machine Learning GroupUiT – The Arctic University of NorwayTromsøNorway
  2. 2.Norwegian Computing CenterOsloNorway

Personalised recommendations