Skip to main content

Subspace Diffusion Generative Models

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13683))

Included in the following conference series:

Abstract

Score-based models generate samples by mapping noise to data (and vice versa) via a high-dimensional diffusion process. We question whether it is necessary to run this entire process at high dimensionality and incur all the inconveniences thereof. Instead, we restrict the diffusion via projections onto subspaces as the data distribution evolves toward noise. When applied to state-of-the-art models, our framework simultaneously improves sample quality—reaching an FID of 2.17 on unconditional CIFAR-10—and reduces the computational cost of inference for the same number of denoising steps. Our framework is fully compatible with continuous-time diffusion and retains its flexible capabilities, including exact log-likelihoods and controllable generation. Code is available at https://github.com/bjing2016/subspace-diffusion.

B. Jing and G. Corso—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In CLD-SGM, one must marginalize over the momentum variables; and in LSGM one must marginalize over the latent variable of VAE.

  2. 2.

    See the supplementary material for experiments on more generic synthetic data.

  3. 3.

    It is via this choice of subspace that subspace diffusion superficially resembles the cascading models discussed in Sect. 2.

  4. 4.

    That is, an \(N \times N\) subspace has dimensionality \(3N^2\).

  5. 5.

    The difference is minimal as the variance of the perturbation kernel dominates either term for reasonable divergence thresholds.

References

  1. Anderson, B.D.: Reverse-time diffusion equation models. Stochastic Processes and their Applications (1982)

    Google Scholar 

  2. Bao, F., Li, C., Zhu, J., Zhang, B.: Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models. ArXiv preprint (2022)

    Google Scholar 

  3. Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis. In: Advances in Neural Information Processing Systems (2021)

    Google Scholar 

  4. Dockhorn, T., Vahdat, A., Kreis, K.: Score-based generative modeling with critically-damped langevin diffusion. In: International Conference on Learning Representations (2022)

    Google Scholar 

  5. Du, Y., Mordatch, I.: Implicit generation and generalization in energy-based models. In: Advances in Neural Information Processing Systems (2019)

    Google Scholar 

  6. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: Advances in Neural Information Processing Systems (2020)

    Google Scholar 

  7. Ho, J., Saharia, C., Chan, W., Fleet, D.J., Norouzi, M., Salimans, T.: Cascaded diffusion models for high fidelity image generation. ArXiv preprint (2021)

    Google Scholar 

  8. Jolicoeur-Martineau, A., Li, K., Piché-Taillefer, R., Kachman, T., Mitliagkas, I.: Gotta go fast when generating data with score-based models. ArXiv preprint (2021)

    Google Scholar 

  9. Kingma, D.P., Salimans, T., Poole, B., Ho, J.: Variational diffusion models. In: Advances in Neural Information Processing Systems (2021)

    Google Scholar 

  10. Kong, Z., Ping, W.: On fast sampling of diffusion probabilistic models. In: ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models (2021)

    Google Scholar 

  11. Lam, M.W., Wang, J., Su, D., Yu, D.: Bddm: Bilateral denoising diffusion models for fast and high-quality speech synthesis. In: International Conference on Learning Representations (2021)

    Google Scholar 

  12. Menick, J., Kalchbrenner, N.: Generating high fidelity images with subscale pixel networks and multidimensional upscaling. In: International Conference on Learning Representations (2019)

    Google Scholar 

  13. Nichol, A., Dhariwal, P.: Improved denoising diffusion probabilistic models. In: International Conference on Machine Learning (2021)

    Google Scholar 

  14. Razavi, A., van den Oord, A., Vinyals, O.: Generating diverse high-fidelity images with vq-vae-2. In: Advances in Neural Information Processing Systems (2019)

    Google Scholar 

  15. Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D.J., Norouzi, M.: Image super-resolution via iterative refinement. ArXiv preprint (2021)

    Google Scholar 

  16. Salimans, T., Ho, J.: Progressive distillation for fast sampling of diffusion models. In: International Conference on Learning Representations (2022)

    Google Scholar 

  17. San-Roman, R., Nachmani, E., Wolf, L.: Noise estimation for generative diffusion models. ArXiv preprint (2021)

    Google Scholar 

  18. Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. In: International Conference on Learning Representations (2021)

    Google Scholar 

  19. Song, Y., Ermon, S.: Generative modeling by estimating gradients of the data distribution. In: Advances in Neural Information Processing Systems (2019)

    Google Scholar 

  20. Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., Poole, B.: Score-based generative modeling through stochastic differential equations. In: International Conference on Learning Representations (2021)

    Google Scholar 

  21. Vahdat, A., Kreis, K., Kautz, J.: Score-based generative modeling in latent space. In: Advances in Neural Information Processing Systems (2021)

    Google Scholar 

  22. Watson, D., Ho, J., Norouzi, M., Chan, W.: Learning to efficiently sample from diffusion probabilistic models. ArXiv preprint (2021)

    Google Scholar 

  23. Xu, K., Zhang, M., Li, J., Du, S.S., Kawarabayashi, K.i., Jegelka, S.: How neural networks extrapolate: From feedforward to graph neural networks. In: International Conference on Learning Representations (2021)

    Google Scholar 

Download references

Acknowledgements

We thank Yilun Du, Xiang Fu, Jason Yim, Shangyuan Tong, Yilun Xu, Felix Faltings, and Saro Passaro for helpful feedback and discussions. Bowen Jing acknowledges support from the Department of Energy Computational Science Graduate Fellowship.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Bowen Jing .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 9526 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jing, B., Corso, G., Berlinghieri, R., Jaakkola, T. (2022). Subspace Diffusion Generative Models. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13683. Springer, Cham. https://doi.org/10.1007/978-3-031-20050-2_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20050-2_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20049-6

  • Online ISBN: 978-3-031-20050-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics