Skip to main content

CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Abstract

Cryo-electron microscopy (cryo-EM) has become a tool of fundamental importance in structural biology, helping us understand the basic building blocks of life. The algorithmic challenge of cryo-EM is to jointly estimate the unknown 3D poses and the 3D electron scattering potential of a biomolecule from millions of extremely noisy 2D images. Existing reconstruction algorithms, however, cannot easily keep pace with the rapidly growing size of cryo-EM datasets due to their high computational and memory cost. We introduce cryoAI, an ab initio reconstruction algorithm for homogeneous conformations that uses direct gradient-based optimization of particle poses and the electron scattering potential from single-particle cryo-EM data. CryoAI combines a learned encoder that predicts the poses of each particle image with a physics-based decoder to aggregate each particle image into an implicit representation of the scattering potential volume. This volume is stored in the Fourier domain for computational efficiency and leverages a modern coordinate network architecture for memory efficiency. Combined with a symmetric loss function, this framework achieves results of a quality on par with state-of-the-art cryo-EM solvers for both simulated and experimental data, one order of magnitude faster for large datasets and with significantly lower memory requirements than existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Akçakaya, M., Yaman, B., Chung, H., Ye, J.C.: Unsupervised deep learning methods for biological image reconstruction and enhancement: an overview from a signal processing perspective. IEEE Signal Process. Mag. 39, 28–44 (2022)

    Article  Google Scholar 

  2. Atzmon, M., Lipman, Y.: SAL: sign agnostic learning of shapes from raw data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2565–2574 (2020)

    Google Scholar 

  3. Baker, T.S., Cheng, R.H.: A model-based approach for determining orientations of biological macromolecules imaged by cryoelectron microscopy. J. Struct. Biol. 116, 120–130 (1996)

    Article  Google Scholar 

  4. Baldwin, P.R., et al.: Big data in cryoEM: automated collection, processing and accessibility of EM data. Curr. Opin. Microbiol. 43, 1–8 (2018)

    Article  Google Scholar 

  5. Bendory, T., Bartesaghi, A., Singer, A.: Single-particle cryo-electron microscopy: mathematical theory, computational challenges, and opportunities. IEEE Signal Process. Mag. 37, 58–76 (2020)

    Article  Google Scholar 

  6. Bepler, T., Kelley, K., Noble, A.J., Berger, B.: Topaz-denoise: general deep denoising models for cryoEM and cryoET. Nat. Commun. 11, 1–12 (2020)

    Article  Google Scholar 

  7. Bracewell, R.N.: Strip integration in radio astronomy. Aust. J. Phys. 9, 198–217 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  8. Candes, E., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489–509 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chen, M., Ludtke, S.J.: Deep learning-based mixed-dimensional Gaussian mixture model for characterizing variability in cryo-EM. Nat. Methods 18, 930–936 (2021)

    Article  Google Scholar 

  10. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc.: Ser. B (Methodol.) 39, 1–22 (1977)

    MathSciNet  MATH  Google Scholar 

  11. Donnat, C., Levy, A., Poitevin, F., Miolane, N.: Deep generative modeling for volume reconstruction in cryo-electron microscopy. arXiv: 2201.02867 (2022)

  12. Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  13. Genova, K., Cole, F., Vlasic, D., Sarna, A., Freeman, W.T., Funkhouser, T.: Learning shape templates with structured implicit functions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7154–7164 (2019)

    Google Scholar 

  14. Gershman, S., Goodman, N.: Amortized inference in probabilistic reasoning. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 36 (2014)

    Google Scholar 

  15. Goddard, T.D., et al.: UCSF chimeraX: meeting modern challenges in visualization and analysis. Protein Sci. 27, 14–25 (2018)

    Article  Google Scholar 

  16. Greenberg, I., Shkolnisky, Y.: Common lines modeling for reference free ab-initio reconstruction in cryo-EM. J. Struct. Biol. 200, 106–117 (2017)

    Article  Google Scholar 

  17. Gupta, H., McCann, M.T., Donati, L., Unser, M.: CryoGAN: a new reconstruction paradigm for single-particle cryo-EM via deep adversarial learning. IEEE Transactions on Computational Imaging 7, 759–774 (2021)

    Article  MathSciNet  Google Scholar 

  18. Hertle, A.: On the problem of well-posedness for the radon transform. In: Herman, G.T., Natterer, F. (eds.) Mathematical Aspects of Computerized Tomography. Lecture Notes in Medical Informatics, vol. 8, pp. 36–44. Springer, Heidelberg (1981). https://doi.org/10.1007/978-3-642-93157-4_5

    Chapter  Google Scholar 

  19. Iudin, A., Korir, P., Salavert-Torres, J., Kleywegt, G., Patwardhan, A.: EMPIAR: a public archive for raw electron microscopy image data. Nat. Methods 13, 387–388 (2016)

    Article  Google Scholar 

  20. Kingma, D.P., Welling, M.: An introduction to variational autoencoders. arXiv preprint arXiv:1906.02691 (2019)

  21. Kühlbrandt, W.: The resolution revolution. Science 343, 1443–1444 (2014)

    Article  Google Scholar 

  22. Lawler, E.L., Wood, D.E.: Branch-and-bound methods: a survey. Oper. Res. 14, 699–719 (1966)

    Article  MathSciNet  MATH  Google Scholar 

  23. Lian, R., Huang, B., Wang, L., Liu, Q., Lin, Y., Ling, H.: End-to-end orientation estimation from 2D cryo-EM images. Acta Crystallogr. Sect. D: Struct. Biol. 78, 174–186 (2022)

    Article  Google Scholar 

  24. Mallick, S., Agarwal, S., Kriegman, D., Belongie, S., Carragher, B., Potter, C.: Structure and view estimation for tomographic reconstruction: a Bayesian approach. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 2253–2260 (2006)

    Google Scholar 

  25. Michalkiewicz, M., Pontes, J.K., Jack, D., Baktashmotlagh, M., Eriksson, A.: Implicit surface representations as layers in neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4743–4752 (2019)

    Google Scholar 

  26. Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 405–421. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_24

    Chapter  Google Scholar 

  27. Miolane, N., Poitevin, F., Li, Y.T., Holmes, S.: Estimation of orientation and camera parameters from cryo-electron microscopy images with variational autoencoders and generative adversarial networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 4174–4183. IEEE (2020)

    Google Scholar 

  28. Müller, C., Schlauderer, G., Reinstein, J., Schulz, G.E.: Adenylate kinase motions during catalysis: an energetic counterweight balancing substrate binding. Structure 4, 147–156 (1996)

    Article  Google Scholar 

  29. Namba, K., Makino, F.: Recent progress and future perspective of electron cryomicroscopy for structural life sciences. Microscopy 71, i3–i14 (2022)

    Article  Google Scholar 

  30. Nashed, Y.S.G., et al.: CryoPoseNet: end-to-end simultaneous learning of single-particle orientation and 3D map reconstruction from cryo-electron microscopy data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 4066–4076 (2021)

    Google Scholar 

  31. Nogales, E.: The development of cryo-EM into a mainstream structural biology technique. Nat. Methods 13, 24–27 (2016)

    Article  Google Scholar 

  32. Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 165–174 (2019)

    Google Scholar 

  33. Penczek, P.A., Grassucci, R.A., Frank, J.: The ribosome at improved resolution: new techniques for merging and orientation refinement in 3D cryo-electron microscopy of biological particles. Ultramicroscopy 53, 251–270 (1994)

    Article  Google Scholar 

  34. Plaschka, C., Lin, P.C., Nagai, K.: Structure of a pre-catalytic spliceosome. Nature 546, 617–621 (2017)

    Article  Google Scholar 

  35. Pragier, G., Shkolnisky, Y.: A common lines approach for ab-initio modeling of cyclically-symmetric molecules. Inverse Prob. 35, 124005 (2019)

    Google Scholar 

  36. Punjani, A., Fleet, D.J.: 3D flexible refinement: structure and motion of flexible proteins from cryo-EM. BioRxiv (2021)

    Google Scholar 

  37. Punjani, A., Rubinstein, J.L., Fleet, D.J., Brubaker, M.A.: cryoSPARC: algorithms for rapid unsupervised cryo-EM structure determination. Nat. Methods 14, 290–296 (2017)

    Article  Google Scholar 

  38. Ravi, N., et al.: Accelerating 3D deep learning with PyTorch3D. arXiv: 2007.08501 (2020)

  39. Renaud, J.P., et al.: Cryo-EM in drug discovery: achievements, limitations and prospects. Nat. Rev. Drug Discov. 17, 471–492 (2018)

    Article  Google Scholar 

  40. Rohou, A., Grigorieff, N.: CTFFIND4: fast and accurate defocus estimation from electron micrographs. J. Struct. Biol. 192, 216–221 (2015)

    Article  Google Scholar 

  41. Rosenbaum, D., et al.: Inferring a continuous distribution of atom coordinates from cryo-EM images using VAEs. arXiv:2106.14108 (2021)

  42. Rosenthal, P.B., Henderson, R.: Optimal determination of particle orientation, absolute hand, and contrast loss in single-particle electron cryomicroscopy. J. Mol. Biol. 333, 721–745 (2003)

    Article  Google Scholar 

  43. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60, 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  44. Scheres, S.H.: RELION: implementation of a Bayesian approach to cryo-EM structure determination. J. Struct. Biol. 180, 519–530 (2012)

    Article  Google Scholar 

  45. Sigworth, F.J.: A maximum-likelihood approach to single-particle image refinement. J. Struct. Biol. 122, 328–339 (1998)

    Article  Google Scholar 

  46. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  47. Singer, A., Coifman, R.R., Sigworth, F.J., Chester, D.W., Shkolnisky, Y.: Detecting consistent common lines in cryo-EM by voting. J. Struct. Biol. 169, 312–322 (2010)

    Article  Google Scholar 

  48. Sitzmann, V., Martel, J., Bergman, A., Lindell, D., Wetzstein, G.: Implicit neural representations with periodic activation functions. Adv. Neural. Inf. Process. Syst. 33, 7462–7473 (2020)

    Google Scholar 

  49. Sitzmann, V., Zollhöfer, M., Wetzstein, G.: Scene representation networks: continuous 3D-structure-aware neural scene representations. Adv. Neural. Inf. Process. Syst. 32 (2019)

    Google Scholar 

  50. Ullrich, K., Berg, R.V.D., Brubaker, M., Fleet, D., Welling, M.: Differentiable probabilistic models of scientific imaging with the Fourier slice theorem. arXiv preprint arXiv:1906.07582 (2019)

  51. Vainshtein, B., Goncharov, A.: Determination of the spatial orientation of arbitrarily arranged identical particles of unknown structure from their projections. In: Soviet Physics Doklady, vol. 31, p. 278 (1986)

    Google Scholar 

  52. Vaswani, A., et al.: Attention is all you need. Adv. Neural. Inf. Process. Syst. 30 (2017)

    Google Scholar 

  53. Vulović, M., et al.: Image formation modeling in cryo-electron microscopy. J. Struct. Biol. 183, 19–32 (2013)

    Article  Google Scholar 

  54. Walls, A.C., Park, Y.J., Tortorici, M.A., Wall, A., McGuire, A.T., Veesler, D.: Structure, function, and antigenicity of the SARS-CoV-2 spike glycoprotein. Cell 181, 281–292 (2020)

    Article  Google Scholar 

  55. Wang, L., Singer, A., Wen, Z.: Orientation determination of cryo-EM images using least unsquared deviations. SIAM J. Imag. Sci. 6, 2450–2483 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  56. Wong, W., et al.: Cryo-EM structure of the plasmodium falciparum 80s ribosome bound to the anti-protozoan drug emetine. Elife 3, e03080 (2014)

    Google Scholar 

  57. Zehni, M., Donati, L., Soubies, E., Zhao, Z.J., Unser, M.: Joint angular refinement and reconstruction for single-particle cryo-EM. IEEE Trans. Image Process. 29, 6151–6163 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  58. Zhong, E.: cryoDRGN-empiar (2022). https://github.com/zhonge/cryodrgn_empiar

  59. Zhong, E.D., Bepler, T., Berger, B., Davis, J.H.: CryoDRGN: reconstruction of heterogeneous cryo-EM structures using neural networks. Nat. Methods 18, 176–185 (2021)

    Article  Google Scholar 

  60. Zhong, E.D., Bepler, T., Davis, J.H., Berger, B.: Reconstructing continuous distributions of 3D protein structure from cryo-EM images. arXiv:1909.05215 (2019)

  61. Zhong, E.D., Lerer, A., Davis, J.H., Berger, B.: CryoDRGN2: ab initio neural reconstruction of 3D protein structures from real cryo-EM images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4066–4075 (2021)

    Google Scholar 

  62. Zhong, E.D., Lerer, A., Davis, J.H., Berger, B.: Exploring generative atomic models in cryo-EM reconstruction. arXiv:2107.01331 (2021)

  63. Zhou, Y., Barnes, C., Lu, J., Yang, J., Li, H.: On the continuity of rotation representations in neural networks. arXiv: 1812.07035 (2020)

  64. Zivanov, J., et al.: New tools for automated high-resolution cryo-EM structure determination in RELION-3. Elife 7, e42166 (2018)

    Google Scholar 

Download references

Acknowledgment

We thank Wah Chiu for numerous discussions that helped shape this project. This work was supported by the U.S. Department of Energy, under DOE Contract No. DE-AC02-76SF00515. N.M. acknowledges support from the National Institutes of Health (NIH), grant No. 1R01GM144965-01. We acknowledge the use of the computational resources at the SLAC Shared Scientific Data Facility (SDF).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Axel Levy .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 7532 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Levy, A. et al. (2022). CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13681. Springer, Cham. https://doi.org/10.1007/978-3-031-19803-8_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19803-8_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19802-1

  • Online ISBN: 978-3-031-19803-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics