Skip to main content
Download book PDF
View expanded cover

Joint European Conference on Machine Learning and Knowledge Discovery in Databases

ECML PKDD 2014: Machine Learning and Knowledge Discovery in Databases pp 501–516Cite as

Interactive Knowledge-Based Kernel PCA

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 8725)

Abstract

Data understanding is an iterative process in which domain experts combine their knowledge with the data at hand to explore and confirm hypotheses. One important set of tools for exploring hypotheses about data are visualizations. Often, however, traditional, unsupervised dimensionality reduction algorithms are used for visualization. These tools allow for interaction, i.e., exploring different visualizations, only by means of manipulating some technical parameters of the algorithm. Therefore, instead of being able to intuitively interact with the visualization, domain experts have to learn and argue about these technical parameters. In this paper we propose a knowledge-based kernel PCA approach that allows for intuitive interaction with data visualizations. Each embedding direction is given by a non-convex quadratic optimization problem over an ellipsoid and has a globally optimal solution in the kernel feature space. A solution can be found in polynomial time using the algorithm presented in this paper. To facilitate direct feedback, i.e., updating the whole embedding with a sufficiently high frame-rate during interaction, we reduce the computational complexity further by incremental up- and down-dating. Our empirical evaluation demonstrates the flexibility and utility of this approach.

Keywords

  • Interactive visualization
  • kernel methods
  • dimensionality reduction

Parts of this work have been presented in workshops[18, 19].

Download conference paper PDF

References

  1. Andrews, C., Endert, A., North, C.: Space to Think: Large High-resolution Displays for Sensemaking. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 55–64. ACM (2010)

    Google Scholar 

  2. Arbenz, P.: Lecture Notes on Solving Large Scale Eigenvalue Problems, pp. 77–93. ETH Zürich (2012)

    Google Scholar 

  3. Bunch, J.R., Nielsen, C.P., Sorensen, D.: Rank-one modification of the symmetric eigenproblem. Numerische Mathematik 31 (1978)

    Google Scholar 

  4. Callahan, E., Koenemann, J.: A comparative usability evaluation of user interfaces for online product catalog. In: Proceedings of the 2nd ACM Conference on Electronic Commerce (EC), pp. 197–206. ACM (2000)

    Google Scholar 

  5. Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning. MIT Press (2006)

    Google Scholar 

  6. Cox, T.F., Cox, M.A.A.: Multidimensional Scaling. Chapman and Hall/CRC (2000)

    Google Scholar 

  7. Dinuzzo, F., Schölkopf, B.: The representer theorem for Hilbert spaces: A necessary and sufficient condition. In: Proceedings of the Conference on Neural Information Processing Systems (NIPS), pp. 189–196 (2012)

    Google Scholar 

  8. Endert, A., Han, C., Maiti, D., House, L., Leman, S., North, C.: Observation-level interaction with statistical models for visual analytics. In: IEEE VAST, pp. 121–130. IEEE (2011)

    Google Scholar 

  9. Forsythe, G.E., Golub, G.H.: On the Stationary Values of a Second-Degree Polynomial on the Unit Sphere. Journal of the Society for Industrial and Applied Mathematics 13(4), 1050–1068 (1965)

    CrossRef  MATH  MathSciNet  Google Scholar 

  10. Gander, W.: Least Squares with a Quadratic Constraint. Numerische Mathematik 36, 291–308 (1981)

    CrossRef  MATH  MathSciNet  Google Scholar 

  11. Gander, W., Golub, G., von Matt, U.: A constrained eigenvalue problem. Linear Algebra and its Applications 114-115, 815–839 (1989)

    Google Scholar 

  12. Ham, J., Lee, D.D., Mika, S., Schölkopf, B.: A Kernel View of the Dimensionality Reduction of Manifolds. In: Proceedings of the 21st International Conference on Machine Learning (2004)

    Google Scholar 

  13. Izenman, A.J.: Linear Discriminant Analysis. Springer (2008)

    Google Scholar 

  14. Jeong, D.H., Ziemkiewicz, C., Fisher, B.D., Ribarsky, W., Chang, R.: iPCA: An Interactive System for PCA-based Visual Analytics. Comput. Graph. Forum 28(3), 767–774 (2009)

    CrossRef  Google Scholar 

  15. Jolliffe, I.T.: Principal Component Analysis. Springer (1986)

    Google Scholar 

  16. Leman, S., House, L., Maiti, D., Endert, A., North, C.: Visual to Parametric Interaction (V2PI). PLoS One 8, e50474 (2013)

    Google Scholar 

  17. Li, R.C.: Solving secular equations stably and efficiently (1993)

    Google Scholar 

  18. Paurat, D., Gärtner, T.: Invis: A tool for interactive visual data analysis. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds.) ECML PKDD 2013, Part III. LNCS, vol. 8190, pp. 672–676. Springer, Heidelberg (2013)

    CrossRef  Google Scholar 

  19. Paurat, D., Oglic, D., Gärtner, T.: Supervised PCA for Interactive Data Analysis. In: Proceedings of the Conference on Neural Information Processing Systems (NIPS) 2nd Workshop on Spectral Learning (2013)

    Google Scholar 

  20. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)

    CrossRef  Google Scholar 

  21. Schölkopf, B., Herbrich, R., Smola, A.J.: A generalized representer theorem. In: Helmbold, D., Williamson, B. (eds.) COLT/EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 416–426. Springer, Heidelberg (2001)

    Google Scholar 

  22. Shearer, C.: The CRISP-DM model: The new blueprint for data mining. Journal of Data Warehousing 5(4), 13–22 (2000)

    Google Scholar 

  23. Tenenbaum, J.B., Silva, V.D., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)

    CrossRef  Google Scholar 

  24. Tukey, J.W.: Mathematics and the picturing of data. In: Proceedings of the International Congress of Mathematicians, vol. 2, pp. 523–531 (1975)

    Google Scholar 

  25. Walder, C., Henao, R., Mørup, M., Hansen, L.K.: Semi-Supervised Kernel PCA. Computing Research Repository (CoRR) abs/1008.1398 (2010)

    Google Scholar 

  26. Weinberger, K.Q., Saul, L.K.: Unsupervised Learning of Image Manifolds by Semidefinite Programming. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 988–995 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Bonn, Bonn, Germany

    Dino Oglic, Daniel Paurat & Thomas Gärtner

  2. Fraunhofer IAIS, Sankt Augustin, Germany

    Thomas Gärtner

Authors
  1. Dino Oglic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Paurat
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Gärtner
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Applied Sciences,Department of Computer and Decision Engineering, Université Libre de Bruxelles, Av. F. Roosevelt, CP 165/15, 1050, Brussels, Belgium

    Toon Calders

  2. Dipartimento di Informatica, Università degli Studi “Aldo Moro”, via Orabona 4, 70125, Bari, Italy

    Floriana Esposito

  3. Department of Computer Science, Universität Paderborn, Warburger Str. 100, 33098, Paderborn, Germany

    Eyke Hüllermeier

  4. Dipartimento di Informatica, Università degli Studi di Torino, Corso Svizzera 185, 10149, Torino, Italy

    Rosa Meo

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Oglic, D., Paurat, D., Gärtner, T. (2014). Interactive Knowledge-Based Kernel PCA. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2014. Lecture Notes in Computer Science(), vol 8725. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44851-9_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-44851-9_32

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-44850-2

  • Online ISBN: 978-3-662-44851-9

  • eBook Packages: Computer ScienceComputer Science (R0)