Spectral Clustering Using PCKID – A Probabilistic Cluster Kernel for Incomplete Data

  • Sigurd Løkse
  • Filippo M. Bianchi
  • Arnt-Børre Salberg
  • Robert Jenssen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10269)

Abstract

In this paper, we propose PCKID, a novel, robust, kernel function for spectral clustering, specifically designed to handle incomplete data. By combining posterior distributions of Gaussian Mixture Models for incomplete data on different scales, we are able to learn a kernel for incomplete data that does not depend on any critical hyperparameters, unlike the commonly used RBF kernel. To evaluate our method, we perform experiments on two real datasets. PCKID outperforms the baseline methods for all fractions of missing values and in some cases outperforms the baseline methods with up to 25% points.

Keywords

Missing data Robustness Kernel methods Spectral clustering 

References

  1. 1.
    Chechik, G., Heitz, G., Elidan, G., Abbeel, P., Koller, D.: Max-margin classification of data with absent features. JMLR 9, 1–21 (2008)MATHGoogle Scholar
  2. 2.
    Dhillon, I.S., Guan, Y., Kulis, B.: Kernel k-means: spectral clustering and normalized cuts. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 551–556. ACM (2004)Google Scholar
  3. 3.
    Dixon, J.K.: Pattern recognition with partly missing data. IEEE Trans. Syst. Man Cybern. 9(10), 617–621 (1979)CrossRefGoogle Scholar
  4. 4.
    Filippone, M., Camastra, F., Masulli, F., Rovetta, S.: A survey of kernel and spectral methods for clustering. Pattern Recogn. 41(1), 176–190 (2008)CrossRefMATHGoogle Scholar
  5. 5.
    Ghahramani, Z., Jordan, M.I.: Supervised learning from incomplete data via an EM approach. In: Advances in Neural Information Processing Systems, pp. 120–120 (1994)Google Scholar
  6. 6.
    Izquierdo-Verdiguier, E., Jenssen, R., Gómez-Chova, L., Camps-Valls, G.: Spectral clustering with the probabilistic cluster kernel. Neurocomputing 149, 1299–1304 (2015)CrossRefGoogle Scholar
  7. 7.
    Jain, A.K.: Data clustering: 50 years beyond k-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)CrossRefGoogle Scholar
  8. 8.
    Jenssen, R.: Kernel entropy component analysis. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 847–860 (2010)CrossRefGoogle Scholar
  9. 9.
    Kuhn, H.W.: The hungarian method for the assignment problem. Naval Res. logistics Q. 2(1–2), 83–97 (1955)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Lin, T.I., Lee, J.C., Ho, H.J.: On fast supervised learning for normal mixture models with missing information. Pattern Recogn. 39(6), 1177–1187 (2006)CrossRefMATHGoogle Scholar
  11. 11.
    Marlin, B.M.: Missing data problems in machine learning. Ph.D. thesis, University of Toronto (2008)Google Scholar
  12. 12.
    Mojirsheibani, M., Montazeri, Z.: Statistical classification with missing covariates. J. Roy. Stat. Soci. Ser. B (Statistical Methodology) 69(5), 839–857 (2007)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Monti, S., Tamayo, P., Mesirov, J., Golub, T.: Consensus clustering: a resampling-based method for class discovery and visualization of gene expression microarray data. Mach. Learn. 52(1–2), 91–118 (2003)CrossRefMATHGoogle Scholar
  14. 14.
    Ng, A.Y., Jordan, M.I., Weiss, Y., et al.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems, pp. 849–856 (2001)Google Scholar
  15. 15.
    Nie, F., Zeng, Z., Tsang, I.W., Xu, D., Zhang, C.: Spectral embedded clustering: a framework for in-sample and out-of-sample spectral clustering. IEEE Trans. Neural Netw. 22(11), 1796–1808 (2011)CrossRefGoogle Scholar
  16. 16.
    Salberg, A.B.: Land cover classification of cloud-contaminated multitemporal high-resolution images. IEEE Trans. Geosci. Remote Sens. 49(1), 377–387 (2011)CrossRefGoogle Scholar
  17. 17.
    Salberg, A.B., Jenssen, R.: Land-cover classification of partly missing data using support vector machines. Int. J. Remote Sens. 33(14), 4471–4481 (2012)CrossRefGoogle Scholar
  18. 18.
    Schölkopf, B., Smola, A., Müller, K.-R.: Kernel principal component analysis. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 583–588. Springer, Heidelberg (1997). doi:10.1007/BFb0020217 Google Scholar
  19. 19.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)CrossRefGoogle Scholar
  20. 20.
    Stella, X.Y., Shi, J.: Multiclass spectral clustering. In: ICCV, pp. 313–319 (2003)Google Scholar
  21. 21.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Academic Press, 4th edn. (2008)Google Scholar
  22. 22.
    Von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Yang, Y., Xu, D., Nie, F., Yan, S., Zhuang, Y.: Image clustering using local discriminant models and global integration. IEEE Trans. Image Process. 19(10), 2761–2773 (2010)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Zimek, A., Gaudet, M., Campello, R.J., Sander, J.: Subsampling for efficient and effective unsupervised outlier detection ensembles. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 428–436. ACM (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Machine Learning GroupUiT – The Arctic University of NorwayTromsøNorway
  2. 2.Norwegian Computing CenterOsloNorway

Personalised recommendations