Skip to main content

Advertisement

SpringerLink
Log in

Semi-Automated Neuron Boundary Detection and Nonbranching Process Segmentation in Electron Microscopy Images

  • Original Article
  • Published:
Neuroinformatics Aims and scope Submit manuscript

Abstract

Neuroscientists are developing new imaging techniques and generating large volumes of data in an effort to understand the complex structure of the nervous system. The complexity and size of this data makes human interpretation a labor-intensive task. To aid in the analysis, new segmentation techniques for identifying neurons in these feature rich datasets are required. This paper presents a method for neuron boundary detection and nonbranching process segmentation in electron microscopy images and visualizing them in three dimensions. It combines both automated segmentation techniques with a graphical user interface for correction of mistakes in the automated process. The automated process first uses machine learning and image processing techniques to identify neuron membranes that deliniate the cells in each two-dimensional section. To segment nonbranching processes, the cell regions in each two-dimensional section are connected in 3D using correlation of regions between sections. The combination of this method with a graphical user interface specially designed for this purpose, enables users to quickly segment cellular processes in large volumes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

References

  • Allen, B. A., & Levinthal, C. (1990). CARTOS II semi-automated nerve tracing: Three-dimensional reconstruction from serial section micrographs. Computerized Medical Imaging and Graphics, 14(5), 319–329.

    Article  PubMed  CAS  Google Scholar 

  • Anderson, J. R., Jones, B. W., Watt, C. B., Shaw, M. V., Yang, J. H., Demill, D., et al. (2011). Exploring the retinal connectome. Molecular Vision, 17, 355–379.

    PubMed  Google Scholar 

  • Anderson, J. R., Jones, B. W., Yang, J.-H., Shaw, M. V., Watt, C. B., Koshevoy, P., et al. (2009). A computational framework for ultrastructural mapping of neural circuitry. PLoS Biology, 7(3), e74.

    Article  Google Scholar 

  • Andres, B., Köthe, U., Helmstaedter, M., Denk, W., & Hamprecht, F. A. (2008). Segmentation of SBFSEM volume data of neural tissue by hierarchical classification. In G. Rigoll (Ed.), Pattern recognition. LNCS (Vol. 5096, pp. 142–152). Berlin: Springer.

    Chapter  Google Scholar 

  • Bertalmío, M., Sapiro, G., & Randall, G. (2000). Morphing active contours. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 733–737.

    Article  Google Scholar 

  • Betzig, E., Patterson, G. H., Sougrat, R., Lindwasser, O. W., Olenych, S., Bonifacino, J. S., et al. (2006). Imaging intracellular fluorescent proteins at nanometer resolution. Science, 313(5793), 1642–1645.

    Article  PubMed  CAS  Google Scholar 

  • Briggman, K. L., & Denk, W. (2006a). Towards neural circuit reconstruction with volume electron microscopy techniques. Current Opinion in Neurobiology, 16(5), 562–570.

    Article  PubMed  CAS  Google Scholar 

  • Briggman, K. L., & Denk, W. (2006b). Towards neural circuit reconstruction with volume electron microscopy techniques. Current Opinion in Neurobiology, 16(5), 562–570.

    Article  PubMed  CAS  Google Scholar 

  • Cardona, A., Saalfeld, S., Preibisch, S., Schmid, B., Cheng, A., Pulokas, J., et al. (2010). An integrated micro- and macroarchitectural analysis of the Drosophila brain by computer-assisted serial section electron microscopy. PLoS Biol, 8(10), e1000502.

    Article  Google Scholar 

  • Chklovskii, D. B., Vitaladevuni, S., & Scheffer, L. K. (2010). Semi-automated reconstruction of neural circuits using electron microscopy. Current Opinion in Neurobiology, 20(5), 667–675.

    Article  PubMed  CAS  Google Scholar 

  • Cottrell, G. W. (1990). Extracting features from faces using compression networks: Face, identity, emotion and gender recognition using holons (pp. 328–337). San Mateo: Morgan Kaufmann.

    Google Scholar 

  • Deerinck, T. J., Bushong, E. A., Thor, A., & Ellisman, M. H. (2010). NCMIR methods for 3D EM: A new protocol for preparation of biological specimens for serial block face scanning electron microscopy. In Microscopy (pp 6–8).

  • Denk, W., & Horstmann, H. (2004). Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure. PLoS Biol, 2(11), 1900–1909.

    Article  CAS  Google Scholar 

  • Denk, W., Strickler, J. H., & Webb, W. W. (1990). Two-photon laser scanning microscopy. Science, 248, 73–76.

    Article  PubMed  CAS  Google Scholar 

  • Egner, A., & Hell, S. W. (2005). Fluorescence microscopy with super-resolved optical sections. Trends in Cell Biology, 15(4), 207–215.

    Article  PubMed  CAS  Google Scholar 

  • Fiala, J. C., & Harris, K. M. (2001). Extending unbiased stereology of brain ultrastructure to three-dimensional volumes. Journal of the American Medical Informatics Association, 8(1), 1–16.

    Article  PubMed  CAS  Google Scholar 

  • Fiala, J. C., & Harris, K. M. (2002). Computer-based alignment and reconstruction of serial sections. Microscopy and Analysis, 87, 5–8.

    Google Scholar 

  • Fiala, J. C., & Harris, K. M. (2010). Synapseweb. http://synapses.clm.utexas.edu/tools/reconstruct/reconstruct.stm.

  • Franken, E., Almsick, M., Rongen, P., Florack, L. M. J., & Haar Romeny, B. M. (2006). An efficient method for tensor voting using steerable filters. In ECCV06 (pp. IV:228–IV:240).

  • Funke, J., Andres, B., Hamprecht, F. A., Cardona, A., & Cook, M. (2011). Multi-hypothesis crf-segmentation of neural tissue in anisotropic em volumes. CoRR, abs/1109.2449.

  • Gonzalez, R. C., & Woods, R. E. (1992). Digital image processing. Boston: Addison-Wesley Longman.

    Google Scholar 

  • Gonzalez-Hernandez, M., Pablo, L. E., Armas-Dominguez, K., Rodriguez de la Vega, R., Ferreras, A., & Gonzalez de la Rosa, M. (2009). Structure-function relationship depends on glaucoma severity. British Journal of Ophthalmology, 93(9), 1195–1199.

    Article  PubMed  CAS  Google Scholar 

  • Haykin, S. (1999). Neural networks—A comprehensive foundation (2nd ed.). New York: Prentice-Hall.

    Google Scholar 

  • Ibanez, L., Schroeder, W., Ng, L., & Cates, J. (2005). The ITK software guide (2nd ed.). Kitware, Inc. ISBN 1-930934-15-7. http://www.itk.org/ItkSoftwareGuide.pdf.

  • Jain, V., Bollmann, B., Richardson, M., Berger, D. R., Helmstaedter, M. N., Briggman, K. L., et al. (2010). Boundary learning by optimization with topological constraints. In 2010 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2488–2495).

  • Jain, V., Murray, J. F., Roth, F., Turaga, S., Zhigulin, V., Briggman, K. L., et al. (2007). Supervised learning of image restoration with convolutional networks. In IEEE 11th international conference on computer vision (pp. 1–8).

  • Jain, V., Sebastian Seung, H., & Turaga, S. C. (2010). Machines that learn to segment images: A crucial technology for connectomics. Current Opinion in Neurobiology, 20(5), 653–666.

    Article  PubMed  CAS  Google Scholar 

  • Jeong, W.-K., Beyer, J., Hadwiger, M., Blue, R., Law, C., Vazquez-Reina, A., et al. (2010). Ssecrett and neurotrace: Interactive visualization and analysis tools for large-scale neuroscience data sets. IEEE Computer Graphics and Applications, 30, 58–70.

    Article  PubMed  Google Scholar 

  • Jeong, W.-K., Beyer, J., Hadwiger, M., Vazquez, A., Pfister, H., & Whitaker, R. T. (2009). Scalable and interactive segmentation and visualization of neural processes in EM datasets. IEEE Transactions on Visualization and Computer Graphics, 15, 1505–1514.

    Article  PubMed  Google Scholar 

  • Jin, Y., Hoskins, R., & Horvitz, H. R. (1994). Control of type-D GABAergic neuron differentiation by C. elegans UNC-30 homeodomain protein. Nature, 372(6508), 780–783.

    Article  PubMed  CAS  Google Scholar 

  • Jones, B. W., & Marc, R. E. (2005). Retinal remodeling during retinal degeneration. Experimental Eye Research, 81, 123–137.

    Article  PubMed  CAS  Google Scholar 

  • Jones, B. W., Watt, C. B., Frederick, J. M., Baehr, W., Chen, C. K., Levine, E. M., et al. (2003). Retinal remodeling triggered by photoreceptor degenerations. Journal of Comparative Neurology, 464, 1–16.

    Article  PubMed  Google Scholar 

  • Jones, B. W., Watt, C. B., & Marc, R. E. (2005). Retinal remodelling. Clinical and Experimental Optometry, 88, 282–291.

    Article  PubMed  Google Scholar 

  • Jurrus, E., Hardy, M., Tasdizen, T., Fletcher, P. T., Koshevoy, P., Chien, C.-B. et al. (2009). Axon tracking in serial block-face scanning electron microscopy. Medical Image Analysis, 13(1), 180–188.

    Article  PubMed  Google Scholar 

  • Jurrus, E., Paiva, A. R. C., Watanabe, S., Anderson, J. R., Jones, B. W., Whitaker, R. T., et al. (2010). Detection of neuron membranes in electron microscopy images using a serial neural network architecture. Medical Image Analysis, 14(6), 770–783. doi:10.1016/j.media.2010.06.002.

    Article  Google Scholar 

  • Jurrus, E., Paiva, A. R. C., Watanabe, S., Whitaker, R., Jorgensen, E. M., & Tasdizen, T. (2009). Serial neural network classifier for membrane detection using a filter bank. In Proc. workshop on microscopic image analysis with applications in biology.

  • Jurrus, E., Whitaker, R. T., Jones, B., Marc, R., & Tasdizen, T. (2008). An optimal-path approach for neural circuit reconstruction. In Proceedings of the 5th IEEE international symposium on biomedical imaging: From nano to macro (pp. 1609–1612).

  • Kaynig, V., Fuchs, T., & Buhmann, J. M. (2010). Neuron geometry extraction by perceptual grouping in ssTEM images. In IEEE Computer Society conference on computer vision and pattern recognition (pp. 2902–2909).

  • Kremer, J. R., Mastronarde, D. N., & McIntosh, J. R. (1996). Computer visualization of three-dimensional image data using imod. Journal of Structural Biology, 116(1), 71–76.

    Article  PubMed  CAS  Google Scholar 

  • Kumar, R., Vázquez-Reina, A., & Pfister, H. (2010). Radon-like features and their application to connectomics. In 2010 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW) (pp. 86–193). doi:10.1109/CVPRW.2010.5543594. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5543594&isnumber=5543135.

  • Leng, Z., Korenberg, J. R., Roysam, B., & Tasdizen, T. (2011). A rapid 2-D centerline extraction method based on tensor voting. In 2011 IEEE international symposium on biomedical imaging: From nano to macro (pp. 1000–1003). doi:10.1109/ISBI.2011.5872570. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5872570&isnumber=5872340.

  • Macke, J. H., Maack, N., Gupta, R., Denk, W., Schölkopf, B., & Borst, A. (2008). Contour-propagation algorithms for semi-automated reconstruction of neural processes. Journal of Neuroscience Methods, 167, 349–357.

    Article  PubMed  Google Scholar 

  • Marc, R. E., Jones, B. W., Anderson, J. R., Kinard, K., Marshak, D. W., Wilson, J. H., et al. (2007). Neural reprogramming in retinal degeneration. Investigative Ophthalmology & Visual Science, 48, 3364–3371.

    Article  Google Scholar 

  • Marc, R. E., Jones, B. W., Watt, C. B., & Strettoi, E. (2003). Neural remodeling in retinal degeneration. Progress in Retinal and Eye Research, 22, 607–655.

    Article  PubMed  Google Scholar 

  • Marc, R. E., Jones, B. W., Watt, C. B., Vazquez-Chona, F., Vaughan, D. K., & Organisciak, D. T. (2008). Extreme retinal remodeling triggered by light damage: Implications for age related macular degeneration. Molecular Vission, 14, 782–806.

    Google Scholar 

  • Martone, M. E., Tran, J., Wong, W. W., Sargis, J., Fong, L., Larson, S., Lamont, S. P., et al. (2008). The cell centered database project: An update on building community resources for managing and sharing 3D imaging data. Journal of Structural Biology, 161(3), 220–231. The 4th International Conference on Electron Tomography.

    Article  PubMed  Google Scholar 

  • Medioni, G., Lee, M.-S., & Tang, C.-K., (2000). Computational framework for segmentation and grouping. New York: Elsevier.

    Google Scholar 

  • Minsky, M. (1961). Microscopy apparatus. U.S. Patent number 301467.

  • Mishchenko, Y. (2008). Automation of 3D reconstruction of neural tissue from large volume of conventional serial section transmission electron micrographs. Journal of Neuroscience Methods.

  • Mishchenko, Y., Hu, T., Spacek, J., Mendenhall, J., Harris, K. M., & Chklovskii, D. B. (2010). Ultrastructural analysis of hippocampal neuropil from the connectomics perspective. Neuron, 67, 1009–1020.

    Article  PubMed  CAS  Google Scholar 

  • Nokia (2012). Qt: Cross-platform application and UI framework. http://qt.nokia.com.

  • Paiva, A. R. C., Jurrus, E., & Tasdizen, T. (2010). Using sequential context for image analysis. In 2010 20th international conference on pattern recognition (ICPR) (pp. 2800–2803).

  • Peng, Y. W., Hao, Y., Petters, R. M., & Wong, F. (2000). Ectopic synaptogenesis in the mammalian retina caused by rod photoreceptor-specific mutations. Nature Neuroscience, 3, 1121–1127.

    Article  PubMed  CAS  Google Scholar 

  • Pizer, S. M., Johnston, R. E., Ericksen, J. P., Yankaskas, B. C., & Muller, K. E. (1990). Contrast-limited adaptive histogram equalization: Speed and effectiveness. In Proceedings of the first conference on visualization in biomedical computing, 1990 (pp. 337–345).

  • Pomerleau, D. (1993). Knowledge-based training of artificial neural networks for autonomous robot driving. In J. Connell, & S. Mahadevan (Eds.), Robot learning (pp. 19–43). Dordrecht: Kluwer Academic.

    Chapter  Google Scholar 

  • Principe, J. C., Euliano, N. R., & Lefebvre, W. C. (2000). Neural and adaptive systems: Fundamentals through simulations. New York: Wiley.

    Google Scholar 

  • Rabi, G., & Lu, S. W. (1998). Visual speech recognition by recurrent neural networks. Journal of Electronic Imaging, 7(1), 61–69. doi:10.1117/1.482627.

    Article  Google Scholar 

  • Rand, W. M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66(336), 846–850.

    Article  Google Scholar 

  • Rueckert, D., Sonoda, L. I., Hayes, C., Hill, D. L. G., Leach, M. O., & Hawkes, D. J. (1999). Nonrigid registration using free-form deformations: Application to breast MR images. IEEE Transactions on Medical Imaging, 18(8), 712–721.

    Article  PubMed  CAS  Google Scholar 

  • Rust, M. J., Bates, M., & Zhuang, X. (2006). Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (storm). Nature Methods, 3(10), 793–796.

    Article  PubMed  CAS  Google Scholar 

  • Saalfeld, S., Cardona, A., Hartenstein, V., & Tomančák, P. (2010). As-rigid-as-possible mosaicking and serial section registration of large ssTEM datasets. Bioinformatics, 26(12), i57–i63.

    Article  Google Scholar 

  • Schroeder, W., Martin, K., & Lorensen, B. (2010). The VTK user’s guide (11th ed.). Kitware, Inc. ISBN 1-930934-19-X. http://www.vtk.org.

  • Sommer, C., Straehle, C., Köthe, U., & Hamprecht, F. A. (2011). ilastik: Interactive learning and segmentation toolkit. In 2011 IEEE international symposium on biomedical imaging: From nano to macro (pp. 230–233). doi:10.1109/ISBI.2011.5872394. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5872394&isnumber=5872340.

  • Tasdizen, T., Whitaker, R., Marc, R., & Jones, B. (2005). Enhancement of cell boundaries in transmission electron microscopy images. In ICIP (pp. 642–645).

  • Tasdizen, T., Koshevoy, P., Grimm, B. C., Anderson, J. R., Jones, B. W., & Watt, C. B. (2010). Automatic mosaicking and volume assembly for high-throughput serial-section transmission electron microscopy. Journal of Neuroscience Methods, 193(1), 132–144.

    Article  PubMed  Google Scholar 

  • Turaga, S. C., Briggman, K. L., Helmstaedter, M., Denk, W., & Seung, H. S. (2009). Maximin affinity learning of image segmentation. CoRR, abs/0911.5372.

  • Turaga, S. C., Murray, J. F., Jain, V., Roth, F., Helmstaedter, M., & Briggman, K. (2010). Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Computation, 22(2), 511–538.

    Article  PubMed  Google Scholar 

  • Varshney, L. R., Chen, B. L., Paniagua, E., Hall, D. H., Chklovskii, D. B. (2011). Structural properties of the Caenorhabditis elegans neuronal network. PLoS Computational Biology, 7(2), e1001066. doi:10.1371/journal.pcbi.1001066.

    Article  Google Scholar 

  • Vazquez, L., Sapiro, G., & Randall, G. (1998). Segmenting neurons in electronic microscopy via geometric tracing. In Proc. of ICIP (pp. 814–818).

  • Vazquez-Reina, A., Miller, E., & Pfister, H. (2009). Multiphase geometric couplings for the segmentation of neural processes. IEEE Computer Society conference on computer vision and pattern recognition (pp. 2020–2027).

  • Venkatataju, K. U., Paiva, A., Jurrus, E., & Tasdizen, T. (2009). Automatic markup of neural cell membranes using boosted decision stumps. In Proceedings of the 6th IEEE international symposium on biomedical imaging (pp. 1039–1042).

  • Visage Imaging (2012). Amira. http://www.amira.com.

  • Vu, N., & Manjunath, B. S. (2008). Graph cut segmentation of neuronal structures from transmission electron micrographs. In 15th IEEE international conference on image processing, 2008. ICIP 2008 (pp. 725–728).

  • Watanabe, K., Takeishi, H., Hayakawa, T., & Sasaki, H. (2010). Three-dimensional organization of the perivascular glial limiting membrane and its relationship with the vasculature: A scanning electron microscope study. Okajimas Folia Anatomica Japonica, 87(3), 109–121.

    Article  PubMed  Google Scholar 

  • Wells, G., Venaille, C., & Torras, C. (1996). Promising research: Vision-based robot positioning using neural networks. Image and Vision Computing, 14(10), 715–732.

    Article  Google Scholar 

  • White, J. Q., Nicholas, T. J., Gritton, J., Truong, L., Davidson, E. R., & Jorgensen, E. M. (2007). The sensory circuitry for sexual attraction in C. elegans males. Current Biology, 17(21), 1847–1857.

    Article  PubMed  CAS  Google Scholar 

  • White, J. G., Southgate, E., Thomson, J. N., & Brenner, F. R. S. (1986). The structure of the nervous system of the nematode Caenorhabditis elegans. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 314, 1–340.

    Article  PubMed  CAS  Google Scholar 

  • Xiao, Y. P., Wang, Y., & Felleman, D. J. (2003). A spatially organized representation of colour in macaque cortical area v2. Nature, 421(6922), 535–539.

    Article  PubMed  CAS  Google Scholar 

  • Yang, H.-F., & Choe, Y. (2009). Cell tracking and segmentation in electron microscopy images using graph cuts. In IEEE international symposium on biomedical imaging: From nano to macro, 2009. ISBI ’09 (pp. 306–309).

Download references

Acknowledgements

This work was supported by NIH R01 EB005832 (TT), HHMI (EMJ), NIH NINDS 5R37NS34307-15 (EMJ) and 1R01NS075314 (MHE, TT) as well as NIH NCRR for support of the National Center for Microscopy and Imaging Research at UCSD, 5P41RR004050 (MHE). We are grateful to Nikita Thomas, Nels B. Jorgensen, Jeremy B. Thompson, and Blake Paulin for their help in imaging the c. elegans VNC and Eric Bushong and Thomas Deerinck for their work in preparing the examples from the mouse cerebellum.

Author information

Authors and Affiliations

  1. Scientific Computing and Imaging Institute, University of Utah, 72 S Central Campus Drive, Salt Lake City, UT, 84112, USA

    Elizabeth Jurrus, Antonio R. C. Paiva & Tolga Tasdizen

  2. School of Computing, University of Utah, Salt Lake City, UT, USA

    Elizabeth Jurrus

  3. Department of Biology, University of Utah, Salt Lake City, UT, USA

    Shigeki Watanabe & Erik M. Jorgensen

  4. National Center for Microscopy and Imaging Research, University of California, San Diego, CA, USA

    Richard J. Giuly & Mark H. Ellisman

  5. Department of Electrical Engineering, University of Utah, Salt Lake City, UT, USA

    Tolga Tasdizen

Authors
  1. Elizabeth Jurrus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shigeki Watanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard J. Giuly
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Antonio R. C. Paiva
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mark H. Ellisman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Erik M. Jorgensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tolga Tasdizen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elizabeth Jurrus.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jurrus, E., Watanabe, S., Giuly, R.J. et al. Semi-Automated Neuron Boundary Detection and Nonbranching Process Segmentation in Electron Microscopy Images. Neuroinform 11, 5–29 (2013). https://doi.org/10.1007/s12021-012-9149-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12021-012-9149-y

Keywords

Search

Navigation