ssEMnet: Serial-Section Electron Microscopy Image Registration Using a Spatial Transformer Network with Learned Features

  • Inwan Yoo
  • David G. C. Hildebrand
  • Willie F. Tobin
  • Wei-Chung Allen Lee
  • Won-Ki Jeong
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10553)


The alignment of serial-section electron microscopy (ssEM) images is critical for efforts in neuroscience that seek to reconstruct neuronal circuits. However, each ssEM plane contains densely packed structures that vary from one section to the next, which makes matching features across images a challenge. Advances in deep learning has resulted in unprecedented performance in similar computer vision problems, but to our knowledge, they have not been successfully applied to ssEM image co-registration. In this paper, we introduce a novel deep network model that combines a spatial transformer for image deformation and a convolutional autoencoder for unsupervised feature learning for robust ssEM image alignment. This results in improved accuracy and robustness while requiring substantially less user intervention than conventional methods. We evaluate our method by comparing registration quality across several datasets.



This work is partially supported by the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education (NRF-2017R1D1A1A09000841) and the Software Convergence Technology Development Program through the Ministry of Science, ICT and Future Planning (S0503-17-1007).


  1. 1.
    Arganda-Carreras, I., Sorzano, C.O., Marabini, R., Carazo, J.M., Ortiz-de Solorzano, C., Kybic, J.: Consistent and elastic registration of histological sections using vector-spline regularization. In: International Workshop on Computer Vision Approaches to Medical Image Analysis, pp. 85–95. Springer, New York (2006)Google Scholar
  2. 2.
    Cardona, A., Saalfeld, S., Schindelin, J., Arganda-Carreras, I., Preibisch, S., Longair, M., Tomancak, P., Hartenstein, V., Douglas, R.J.: TrakEM2 software for neural circuit reconstruction. PloS One 7(6), e38011 (2012)CrossRefGoogle Scholar
  3. 3.
    Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)Google Scholar
  4. 4.
    Hayworth, K.J., Morgan, J.L., Schalek, R., Berger, D.R., Hildebrand, D.G.C., Lichtman, J.W.: Imaging ATUM ultrathin section libraries with WaferMapper: a multi-scale approach to EM reconstruction of neural circuits. Front. Neural Circuits 8, 68 (2014)CrossRefGoogle Scholar
  5. 5.
    Helmstaedter, M.: Cellular-resolution connectomics: challenges of dense neural circuit reconstruction. Nat. Methods 10(6), 501–507 (2013)CrossRefGoogle Scholar
  6. 6.
    Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: Advances in Neural Information Processing Systems, pp. 2017–2025 (2015)Google Scholar
  7. 7.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization arXiv preprint arXiv:1412.6980 (2014)
  8. 8.
    Morgan, J.L., Berger, D.R., Wetzel, A.W., Lichtman, J.W.: The fuzzy logic of network connectivity in mouse visual thalamus. Cell 1, 192–206 (2017)Google Scholar
  9. 9.
    Saalfeld, S., Fetter, R., Cardona, A., Tomancak, P.: Elastic volume reconstruction from series of ultra-thin microscopy sections. Nat. Methods 9(7), 717–720 (2012)CrossRefGoogle Scholar
  10. 10.
    Wu, G., Kim, M.J., Wang, Q., Munsell, B., Shen, D.: Scalable high performance image registration framework by unsupervised deep feature representations learning (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Inwan Yoo
    • 1
  • David G. C. Hildebrand
    • 2
  • Willie F. Tobin
    • 3
  • Wei-Chung Allen Lee
    • 3
  • Won-Ki Jeong
    • 1
  1. 1.Ulsan National Institute of Science and TechnologyUlsanSouth Korea
  2. 2.The Rockefeller UniversityNew YorkUSA
  3. 3.Harvard Medical SchoolBostonUSA

Personalised recommendations