Advertisement

Deep Archetypal Analysis

  • Sebastian Mathias KellerEmail author
  • Maxim Samarin
  • Mario Wieser
  • Volker Roth
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11824)

Abstract

Deep Archetypal Analysis (DeepAA) generates latent representations of high-dimensional datasets in terms of intuitively understandable basic entities called archetypes. The proposed method extends linear Archetypal Analysis (AA), an unsupervised method to represent multivariate data points as convex combinations of extremal data points. Unlike the original formulation, Deep AA is generative and capable of handling side information. In addition, our model provides the ability for data-driven representation learning which reduces the dependence on expert knowledge. We empirically demonstrate the applicability of our approach by exploring the chemical space of small organic molecules. In doing so, we employ the archetype constraint to learn two different latent archetype representations for the same dataset, with respect to two chemical properties. This type of supervised exploration marks a distinct starting point and let us steer de novo molecular design.

Notes

Acknowledgements

S. Keller is partially supported by the Swiss National Science Foundation project CR32I2 159682. M. Samarin is supported by the Swiss National Science Foundation grant 407540 167333 as part of the Swiss National Research Programme NRP 75 “Big Data”. M. Wieser is partially supported by the NCCR MARVEL, funded by the Swiss National Science Foundation and SNSF grant 51MRP0158328 (SystemsX.ch).

Supplementary material

480714_1_En_12_MOESM1_ESM.zip (1.2 mb)
Supplementary material 1 (zip 1271 KB)

References

  1. 1.
    Alemi, A.A., Fischer, I., Dillon, J.V., Murphy, K.: Deep variational information bottleneck. CoRR abs/1612.00410 (2016). http://arxiv.org/abs/1612.00410
  2. 2.
    Atkins, P., de Paula, J.: Atkins’ Physical Chemistry. OUP, Oxford (2010)Google Scholar
  3. 3.
    Bauckhage, C., Manshaei, K.: Kernel archetypal analysis for clustering web search frequency time series. In: 22nd International Conference on Pattern Recognition, pp. 1544–1549, August 2014.  https://doi.org/10.1109/ICPR.2014.274
  4. 4.
    Bauckhage, C., Kersting, K., Hoppe, F., Thurau, C.: Archetypal analysis as an autoencoder. In: Workshop New Challenges in Neural Computation 2015, pp. 8–16, October 2015. https://www.techfak.uni-bielefeld.de/~fschleif/mlr/mlr_03_2015.pdf
  5. 5.
    Bauckhage, C., Thurau, C.: Making archetypal analysis practical. In: Denzler, J., Notni, G., Süße, H. (eds.) DAGM 2009. LNCS, vol. 5748, pp. 272–281. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-03798-6_28CrossRefGoogle Scholar
  6. 6.
    Cabeza, L.F., et al.: Lithium in thermal energy storage: a state-of-the-art review. Renew. Sustain. Energy Rev. 42, 1106–1112 (2015)CrossRefGoogle Scholar
  7. 7.
    Canhasi, E., Kononenko, I.: Weighted hierarchical archetypal analysis for multi-document summarization. Comput. Speech Lang. 37 (2015).  https://doi.org/10.1016/j.csl.2015.11.004
  8. 8.
  9. 9.
    van Dijk, D., Burkhardt, D., Amodio, M., Tong, A., Wolf, G., Krishnaswamy, S.: Finding archetypal spaces for data using neural networks. arXiv preprint arXiv:1901.09078 (2019)
  10. 10.
    Chan, B.H.P., Mitchell, D., Cram, L.: Archetypal analysis of galaxy spectra. Mon. Not. Roy. Astron. Soc. 338 (2003).  https://doi.org/10.1046/j.1365-8711.2003.06099.x
  11. 11.
    Hart, Y., et al.: Inferring biological tasks using pareto analysis of high-dimensional data. Nat. Methods 12(3), 233 (2015)CrossRefGoogle Scholar
  12. 12.
    Hou, X., Shen, L., Sun, K., Qiu, G.: Deep feature consistent variational autoencoder. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1133–1141. IEEE (2017)Google Scholar
  13. 13.
    Huggins, P., Pachter, L., Sturmfels, B.: Toward the human genotope. Bull. Math. Biol. 69(8), 2723–2735 (2007).  https://doi.org/10.1007/s11538-007-9244-7MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Jang, E., Gu, S., Poole, B.: Categorical reparameterization with Gumbel-Softmax. In: International Conference on Learning Representations (ICLR) (2017)Google Scholar
  15. 15.
    Kaufmann, D., Keller, S., Roth, V.: Copula archetypal analysis. In: Gall, J., Gehler, P., Leibe, B. (eds.) GCPR 2015. LNCS, vol. 9358, pp. 117–128. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24947-6_10CrossRefGoogle Scholar
  16. 16.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. abs/1412.6980 (2014)Google Scholar
  17. 17.
    Kingma, D.P., Mohamed, S., Rezende, D.J., Welling, M.: Semi-supervised learning with deep generative models. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 8–13 December 2014, Montreal, pp. 3581–3589 (2014)Google Scholar
  18. 18.
    Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. CoRR abs/1312.6114 (2013)Google Scholar
  19. 19.
    Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV), December 2015Google Scholar
  20. 20.
    Mørup, M., Hansen, L.K.: Archetypal analysis for machine learning and data mining. Neurocomputing 80, 54–63 (2012)CrossRefGoogle Scholar
  21. 21.
    Parbhoo, S., Wieser, M., Roth, V.: Causal deep information bottleneck. arXiv e-prints arXiv:1807.02326, July 2018
  22. 22.
    Prabhakaran, S., Raman, S., Vogt, J.E., Roth, V.: Automatic model selection in archetype analysis. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds.) DAGM/OAGM 2012. LNCS, vol. 7476, pp. 458–467. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-32717-9_46CrossRefGoogle Scholar
  23. 23.
    Ramakrishnan, R., Dral, P.O., Rupp, M., von Lilienfeld, O.A.: Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1 (2014)Google Scholar
  24. 24.
    Rezende, D., Mohamed, S.: Variational inference with normalizing flows. In: Bach, F., Blei, D. (eds.) Proceedings of the 32nd International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 37, pp. 1530–1538. PMLR, Lille, 07–09 July 2015Google Scholar
  25. 25.
    Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic backpropagation and approximate inference in deep generative models 32(2), 1278–1286 (2014)Google Scholar
  26. 26.
    Ruddigkeit, L., van Deursen, R., Blum, L.C., Reymond, J.L.: Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. J. Chem. Inf. Model. 52(11), 2864–2875 (2012).  https://doi.org/10.1021/ci300415d. pMID: 23088335CrossRefGoogle Scholar
  27. 27.
    Seth, S., Eugster, M.J.A.: Probabilistic archetypal analysis. Mach. Learn. 102(1), 85–113 (2016).  https://doi.org/10.1007/s10994-015-5498-8MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Shoval, O., et al.: Evolutionary trade-offs, pareto optimality, and thegeometry of phenotype space. Science 336(6085), 1157–1160 (2012).  https://doi.org/10.1126/science.1217405. http://science.sciencemag.org/content/336/6085/1157CrossRefGoogle Scholar
  29. 29.
    Steinbeck, C., Han, Y.Q., Kuhn, S., Horlacher, O., Luttmann, E., Willighagen, E.: The Chemistry Development Kit (CDK): an open-source Java library for chemo- and bioinformatics. J. Chem. Inf. Comput. Sci. 43(2), 493–500 (2003)CrossRefGoogle Scholar
  30. 30.
    Tinoco, I.: Physical Chemistry: Principles and Applications in Biological Sciences. No. S. 229-313 in Physical Chemistry: Principles and Applications in Biological Sciences. Prentice Hall, Englewood Cliffs (2002)Google Scholar
  31. 31.
    Tishby, N., Pereira, F.C., Bialek, W.: The information bottleneck method. arXiv preprint arXiv:physics/0004057 (2000)
  32. 32.
    Wieczorek, A., Wieser, M., Murezzan, D., Roth, V.: Learning sparse latent representations with the deep copula information bottleneck. In: International Conference on Learning Representations (ICLR) (2018)Google Scholar
  33. 33.
    Wynen, D., Schmid, C., Mairal, J.: Unsupervised learning of artistic styles with archetypal style analysis. In: Advances in Neural Information Processing Systems, pp. 6584–6593 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of BaselBaselSwitzerland

Personalised recommendations