ssEMnet: Serial-Section Electron Microscopy Image Registration Using a Spatial Transformer Network with Learned Features
The alignment of serial-section electron microscopy (ssEM) images is critical for efforts in neuroscience that seek to reconstruct neuronal circuits. However, each ssEM plane contains densely packed structures that vary from one section to the next, which makes matching features across images a challenge. Advances in deep learning has resulted in unprecedented performance in similar computer vision problems, but to our knowledge, they have not been successfully applied to ssEM image co-registration. In this paper, we introduce a novel deep network model that combines a spatial transformer for image deformation and a convolutional autoencoder for unsupervised feature learning for robust ssEM image alignment. This results in improved accuracy and robustness while requiring substantially less user intervention than conventional methods. We evaluate our method by comparing registration quality across several datasets.
This work is partially supported by the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education (NRF-2017R1D1A1A09000841) and the Software Convergence Technology Development Program through the Ministry of Science, ICT and Future Planning (S0503-17-1007).
- 1.Arganda-Carreras, I., Sorzano, C.O., Marabini, R., Carazo, J.M., Ortiz-de Solorzano, C., Kybic, J.: Consistent and elastic registration of histological sections using vector-spline regularization. In: International Workshop on Computer Vision Approaches to Medical Image Analysis, pp. 85–95. Springer, New York (2006)Google Scholar
- 3.Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)Google Scholar
- 6.Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. In: Advances in Neural Information Processing Systems, pp. 2017–2025 (2015)Google Scholar
- 7.Kingma, D., Ba, J.: Adam: a method for stochastic optimization arXiv preprint arXiv:1412.6980 (2014)
- 8.Morgan, J.L., Berger, D.R., Wetzel, A.W., Lichtman, J.W.: The fuzzy logic of network connectivity in mouse visual thalamus. Cell 1, 192–206 (2017)Google Scholar
- 10.Wu, G., Kim, M.J., Wang, Q., Munsell, B., Shen, D.: Scalable high performance image registration framework by unsupervised deep feature representations learning (2015)Google Scholar