Neuroinformatics

, Volume 11, Issue 1, pp 5–29 | Cite as

Semi-Automated Neuron Boundary Detection and Nonbranching Process Segmentation in Electron Microscopy Images

  • Elizabeth Jurrus
  • Shigeki Watanabe
  • Richard J. Giuly
  • Antonio R. C. Paiva
  • Mark H. Ellisman
  • Erik M. Jorgensen
  • Tolga Tasdizen
Original Article
First Online:

Abstract

Neuroscientists are developing new imaging techniques and generating large volumes of data in an effort to understand the complex structure of the nervous system. The complexity and size of this data makes human interpretation a labor-intensive task. To aid in the analysis, new segmentation techniques for identifying neurons in these feature rich datasets are required. This paper presents a method for neuron boundary detection and nonbranching process segmentation in electron microscopy images and visualizing them in three dimensions. It combines both automated segmentation techniques with a graphical user interface for correction of mistakes in the automated process. The automated process first uses machine learning and image processing techniques to identify neuron membranes that deliniate the cells in each two-dimensional section. To segment nonbranching processes, the cell regions in each two-dimensional section are connected in 3D using correlation of regions between sections. The combination of this method with a graphical user interface specially designed for this purpose, enables users to quickly segment cellular processes in large volumes.

Keywords

Machine learning Membrane detection Artificial neural networks Filter bank Contour completion Neural circuit reconstruction Connectomics 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This work was supported by NIH R01 EB005832 (TT), HHMI (EMJ), NIH NINDS 5R37NS34307-15 (EMJ) and 1R01NS075314 (MHE, TT) as well as NIH NCRR for support of the National Center for Microscopy and Imaging Research at UCSD, 5P41RR004050 (MHE). We are grateful to Nikita Thomas, Nels B. Jorgensen, Jeremy B. Thompson, and Blake Paulin for their help in imaging the c. elegans VNC and Eric Bushong and Thomas Deerinck for their work in preparing the examples from the mouse cerebellum.

References

  1. Allen, B. A., & Levinthal, C. (1990). CARTOS II semi-automated nerve tracing: Three-dimensional reconstruction from serial section micrographs. Computerized Medical Imaging and Graphics, 14(5), 319–329.PubMedCrossRefGoogle Scholar
  2. Anderson, J. R., Jones, B. W., Watt, C. B., Shaw, M. V., Yang, J. H., Demill, D., et al. (2011). Exploring the retinal connectome. Molecular Vision, 17, 355–379.PubMedGoogle Scholar
  3. Anderson, J. R., Jones, B. W., Yang, J.-H., Shaw, M. V., Watt, C. B., Koshevoy, P., et al. (2009). A computational framework for ultrastructural mapping of neural circuitry. PLoS Biology, 7(3), e74.CrossRefGoogle Scholar
  4. Andres, B., Köthe, U., Helmstaedter, M., Denk, W., & Hamprecht, F. A. (2008). Segmentation of SBFSEM volume data of neural tissue by hierarchical classification. In G. Rigoll (Ed.), Pattern recognition. LNCS (Vol. 5096, pp. 142–152). Berlin: Springer.CrossRefGoogle Scholar
  5. Bertalmío, M., Sapiro, G., & Randall, G. (2000). Morphing active contours. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 733–737.CrossRefGoogle Scholar
  6. Betzig, E., Patterson, G. H., Sougrat, R., Lindwasser, O. W., Olenych, S., Bonifacino, J. S., et al. (2006). Imaging intracellular fluorescent proteins at nanometer resolution. Science, 313(5793), 1642–1645.PubMedCrossRefGoogle Scholar
  7. Briggman, K. L., & Denk, W. (2006a). Towards neural circuit reconstruction with volume electron microscopy techniques. Current Opinion in Neurobiology, 16(5), 562–570.PubMedCrossRefGoogle Scholar
  8. Briggman, K. L., & Denk, W. (2006b). Towards neural circuit reconstruction with volume electron microscopy techniques. Current Opinion in Neurobiology, 16(5), 562–570.PubMedCrossRefGoogle Scholar
  9. Cardona, A., Saalfeld, S., Preibisch, S., Schmid, B., Cheng, A., Pulokas, J., et al. (2010). An integrated micro- and macroarchitectural analysis of the Drosophila brain by computer-assisted serial section electron microscopy. PLoS Biol, 8(10), e1000502.CrossRefGoogle Scholar
  10. Chklovskii, D. B., Vitaladevuni, S., & Scheffer, L. K. (2010). Semi-automated reconstruction of neural circuits using electron microscopy. Current Opinion in Neurobiology, 20(5), 667–675.PubMedCrossRefGoogle Scholar
  11. Cottrell, G. W. (1990). Extracting features from faces using compression networks: Face, identity, emotion and gender recognition using holons (pp. 328–337). San Mateo: Morgan Kaufmann.Google Scholar
  12. Deerinck, T. J., Bushong, E. A., Thor, A., & Ellisman, M. H. (2010). NCMIR methods for 3D EM: A new protocol for preparation of biological specimens for serial block face scanning electron microscopy. In Microscopy (pp 6–8).Google Scholar
  13. Denk, W., & Horstmann, H. (2004). Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure. PLoS Biol, 2(11), 1900–1909.CrossRefGoogle Scholar
  14. Denk, W., Strickler, J. H., & Webb, W. W. (1990). Two-photon laser scanning microscopy. Science, 248, 73–76.PubMedCrossRefGoogle Scholar
  15. Egner, A., & Hell, S. W. (2005). Fluorescence microscopy with super-resolved optical sections. Trends in Cell Biology, 15(4), 207–215.PubMedCrossRefGoogle Scholar
  16. Fiala, J. C., & Harris, K. M. (2001). Extending unbiased stereology of brain ultrastructure to three-dimensional volumes. Journal of the American Medical Informatics Association, 8(1), 1–16.PubMedCrossRefGoogle Scholar
  17. Fiala, J. C., & Harris, K. M. (2002). Computer-based alignment and reconstruction of serial sections. Microscopy and Analysis, 87, 5–8.Google Scholar
  18. Fiala, J. C., & Harris, K. M. (2010). Synapseweb. http://synapses.clm.utexas.edu/tools/reconstruct/reconstruct.stm.
  19. Franken, E., Almsick, M., Rongen, P., Florack, L. M. J., & Haar Romeny, B. M. (2006). An efficient method for tensor voting using steerable filters. In ECCV06 (pp. IV:228–IV:240).Google Scholar
  20. Funke, J., Andres, B., Hamprecht, F. A., Cardona, A., & Cook, M. (2011). Multi-hypothesis crf-segmentation of neural tissue in anisotropic em volumes. CoRR, abs/1109.2449.Google Scholar
  21. Gonzalez, R. C., & Woods, R. E. (1992). Digital image processing. Boston: Addison-Wesley Longman.Google Scholar
  22. Gonzalez-Hernandez, M., Pablo, L. E., Armas-Dominguez, K., Rodriguez de la Vega, R., Ferreras, A., & Gonzalez de la Rosa, M. (2009). Structure-function relationship depends on glaucoma severity. British Journal of Ophthalmology, 93(9), 1195–1199.PubMedCrossRefGoogle Scholar
  23. Haykin, S. (1999). Neural networks—A comprehensive foundation (2nd ed.). New York: Prentice-Hall.Google Scholar
  24. Ibanez, L., Schroeder, W., Ng, L., & Cates, J. (2005). The ITK software guide (2nd ed.). Kitware, Inc. ISBN 1-930934-15-7. http://www.itk.org/ItkSoftwareGuide.pdf.
  25. Jain, V., Bollmann, B., Richardson, M., Berger, D. R., Helmstaedter, M. N., Briggman, K. L., et al. (2010). Boundary learning by optimization with topological constraints. In 2010 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2488–2495).Google Scholar
  26. Jain, V., Murray, J. F., Roth, F., Turaga, S., Zhigulin, V., Briggman, K. L., et al. (2007). Supervised learning of image restoration with convolutional networks. In IEEE 11th international conference on computer vision (pp. 1–8).Google Scholar
  27. Jain, V., Sebastian Seung, H., & Turaga, S. C. (2010). Machines that learn to segment images: A crucial technology for connectomics. Current Opinion in Neurobiology, 20(5), 653–666.PubMedCrossRefGoogle Scholar
  28. Jeong, W.-K., Beyer, J., Hadwiger, M., Blue, R., Law, C., Vazquez-Reina, A., et al. (2010). Ssecrett and neurotrace: Interactive visualization and analysis tools for large-scale neuroscience data sets. IEEE Computer Graphics and Applications, 30, 58–70.PubMedCrossRefGoogle Scholar
  29. Jeong, W.-K., Beyer, J., Hadwiger, M., Vazquez, A., Pfister, H., & Whitaker, R. T. (2009). Scalable and interactive segmentation and visualization of neural processes in EM datasets. IEEE Transactions on Visualization and Computer Graphics, 15, 1505–1514.PubMedCrossRefGoogle Scholar
  30. Jin, Y., Hoskins, R., & Horvitz, H. R. (1994). Control of type-D GABAergic neuron differentiation by C. elegans UNC-30 homeodomain protein. Nature, 372(6508), 780–783.PubMedCrossRefGoogle Scholar
  31. Jones, B. W., & Marc, R. E. (2005). Retinal remodeling during retinal degeneration. Experimental Eye Research, 81, 123–137.PubMedCrossRefGoogle Scholar
  32. Jones, B. W., Watt, C. B., Frederick, J. M., Baehr, W., Chen, C. K., Levine, E. M., et al. (2003). Retinal remodeling triggered by photoreceptor degenerations. Journal of Comparative Neurology, 464, 1–16.PubMedCrossRefGoogle Scholar
  33. Jones, B. W., Watt, C. B., & Marc, R. E. (2005). Retinal remodelling. Clinical and Experimental Optometry, 88, 282–291.PubMedCrossRefGoogle Scholar
  34. Jurrus, E., Hardy, M., Tasdizen, T., Fletcher, P. T., Koshevoy, P., Chien, C.-B. et al. (2009). Axon tracking in serial block-face scanning electron microscopy. Medical Image Analysis, 13(1), 180–188.PubMedCrossRefGoogle Scholar
  35. Jurrus, E., Paiva, A. R. C., Watanabe, S., Anderson, J. R., Jones, B. W., Whitaker, R. T., et al. (2010). Detection of neuron membranes in electron microscopy images using a serial neural network architecture. Medical Image Analysis, 14(6), 770–783. doi: 10.1016/j.media.2010.06.002.CrossRefGoogle Scholar
  36. Jurrus, E., Paiva, A. R. C., Watanabe, S., Whitaker, R., Jorgensen, E. M., & Tasdizen, T. (2009). Serial neural network classifier for membrane detection using a filter bank. In Proc. workshop on microscopic image analysis with applications in biology.Google Scholar
  37. Jurrus, E., Whitaker, R. T., Jones, B., Marc, R., & Tasdizen, T. (2008). An optimal-path approach for neural circuit reconstruction. In Proceedings of the 5th IEEE international symposium on biomedical imaging: From nano to macro (pp. 1609–1612).Google Scholar
  38. Kaynig, V., Fuchs, T., & Buhmann, J. M. (2010). Neuron geometry extraction by perceptual grouping in ssTEM images. In IEEE Computer Society conference on computer vision and pattern recognition (pp. 2902–2909).Google Scholar
  39. Kremer, J. R., Mastronarde, D. N., & McIntosh, J. R. (1996). Computer visualization of three-dimensional image data using imod. Journal of Structural Biology, 116(1), 71–76.PubMedCrossRefGoogle Scholar
  40. Kumar, R., Vázquez-Reina, A., & Pfister, H. (2010). Radon-like features and their application to connectomics. In 2010 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW) (pp. 86–193). doi: 10.1109/CVPRW.2010.5543594. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5543594&isnumber=5543135.
  41. Leng, Z., Korenberg, J. R., Roysam, B., & Tasdizen, T. (2011). A rapid 2-D centerline extraction method based on tensor voting. In 2011 IEEE international symposium on biomedical imaging: From nano to macro (pp. 1000–1003). doi: 10.1109/ISBI.2011.5872570. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5872570&isnumber=5872340.
  42. Macke, J. H., Maack, N., Gupta, R., Denk, W., Schölkopf, B., & Borst, A. (2008). Contour-propagation algorithms for semi-automated reconstruction of neural processes. Journal of Neuroscience Methods, 167, 349–357.PubMedCrossRefGoogle Scholar
  43. Marc, R. E., Jones, B. W., Anderson, J. R., Kinard, K., Marshak, D. W., Wilson, J. H., et al. (2007). Neural reprogramming in retinal degeneration. Investigative Ophthalmology & Visual Science, 48, 3364–3371.CrossRefGoogle Scholar
  44. Marc, R. E., Jones, B. W., Watt, C. B., & Strettoi, E. (2003). Neural remodeling in retinal degeneration. Progress in Retinal and Eye Research, 22, 607–655.PubMedCrossRefGoogle Scholar
  45. Marc, R. E., Jones, B. W., Watt, C. B., Vazquez-Chona, F., Vaughan, D. K., & Organisciak, D. T. (2008). Extreme retinal remodeling triggered by light damage: Implications for age related macular degeneration. Molecular Vission, 14, 782–806.Google Scholar
  46. Martone, M. E., Tran, J., Wong, W. W., Sargis, J., Fong, L., Larson, S., Lamont, S. P., et al. (2008). The cell centered database project: An update on building community resources for managing and sharing 3D imaging data. Journal of Structural Biology, 161(3), 220–231. The 4th International Conference on Electron Tomography.PubMedCrossRefGoogle Scholar
  47. Medioni, G., Lee, M.-S., & Tang, C.-K., (2000). Computational framework for segmentation and grouping. New York: Elsevier.Google Scholar
  48. Minsky, M. (1961). Microscopy apparatus. U.S. Patent number 301467.Google Scholar
  49. Mishchenko, Y. (2008). Automation of 3D reconstruction of neural tissue from large volume of conventional serial section transmission electron micrographs. Journal of Neuroscience Methods.Google Scholar
  50. Mishchenko, Y., Hu, T., Spacek, J., Mendenhall, J., Harris, K. M., & Chklovskii, D. B. (2010). Ultrastructural analysis of hippocampal neuropil from the connectomics perspective. Neuron, 67, 1009–1020.PubMedCrossRefGoogle Scholar
  51. Nokia (2012). Qt: Cross-platform application and UI framework. http://qt.nokia.com.
  52. Paiva, A. R. C., Jurrus, E., & Tasdizen, T. (2010). Using sequential context for image analysis. In 2010 20th international conference on pattern recognition (ICPR) (pp. 2800–2803).Google Scholar
  53. Peng, Y. W., Hao, Y., Petters, R. M., & Wong, F. (2000). Ectopic synaptogenesis in the mammalian retina caused by rod photoreceptor-specific mutations. Nature Neuroscience, 3, 1121–1127.PubMedCrossRefGoogle Scholar
  54. Pizer, S. M., Johnston, R. E., Ericksen, J. P., Yankaskas, B. C., & Muller, K. E. (1990). Contrast-limited adaptive histogram equalization: Speed and effectiveness. In Proceedings of the first conference on visualization in biomedical computing, 1990 (pp. 337–345).Google Scholar
  55. Pomerleau, D. (1993). Knowledge-based training of artificial neural networks for autonomous robot driving. In J. Connell, & S. Mahadevan (Eds.), Robot learning (pp. 19–43). Dordrecht: Kluwer Academic.CrossRefGoogle Scholar
  56. Principe, J. C., Euliano, N. R., & Lefebvre, W. C. (2000). Neural and adaptive systems: Fundamentals through simulations. New York: Wiley.Google Scholar
  57. Rabi, G., & Lu, S. W. (1998). Visual speech recognition by recurrent neural networks. Journal of Electronic Imaging, 7(1), 61–69. doi: 10.1117/1.482627.CrossRefGoogle Scholar
  58. Rand, W. M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66(336), 846–850.CrossRefGoogle Scholar
  59. Rueckert, D., Sonoda, L. I., Hayes, C., Hill, D. L. G., Leach, M. O., & Hawkes, D. J. (1999). Nonrigid registration using free-form deformations: Application to breast MR images. IEEE Transactions on Medical Imaging, 18(8), 712–721.PubMedCrossRefGoogle Scholar
  60. Rust, M. J., Bates, M., & Zhuang, X. (2006). Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (storm). Nature Methods, 3(10), 793–796.PubMedCrossRefGoogle Scholar
  61. Saalfeld, S., Cardona, A., Hartenstein, V., & Tomančák, P. (2010). As-rigid-as-possible mosaicking and serial section registration of large ssTEM datasets. Bioinformatics, 26(12), i57–i63.CrossRefGoogle Scholar
  62. Schroeder, W., Martin, K., & Lorensen, B. (2010). The VTK user’s guide (11th ed.). Kitware, Inc. ISBN 1-930934-19-X. http://www.vtk.org.
  63. Sommer, C., Straehle, C., Köthe, U., & Hamprecht, F. A. (2011). ilastik: Interactive learning and segmentation toolkit. In 2011 IEEE international symposium on biomedical imaging: From nano to macro (pp. 230–233). doi: 10.1109/ISBI.2011.5872394. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5872394&isnumber=5872340.
  64. Tasdizen, T., Whitaker, R., Marc, R., & Jones, B. (2005). Enhancement of cell boundaries in transmission electron microscopy images. In ICIP (pp. 642–645).Google Scholar
  65. Tasdizen, T., Koshevoy, P., Grimm, B. C., Anderson, J. R., Jones, B. W., & Watt, C. B. (2010). Automatic mosaicking and volume assembly for high-throughput serial-section transmission electron microscopy. Journal of Neuroscience Methods, 193(1), 132–144.PubMedCrossRefGoogle Scholar
  66. Turaga, S. C., Briggman, K. L., Helmstaedter, M., Denk, W., & Seung, H. S. (2009). Maximin affinity learning of image segmentation. CoRR, abs/0911.5372.Google Scholar
  67. Turaga, S. C., Murray, J. F., Jain, V., Roth, F., Helmstaedter, M., & Briggman, K. (2010). Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Computation, 22(2), 511–538.PubMedCrossRefGoogle Scholar
  68. Varshney, L. R., Chen, B. L., Paniagua, E., Hall, D. H., Chklovskii, D. B. (2011). Structural properties of the Caenorhabditis elegans neuronal network. PLoS Computational Biology, 7(2), e1001066. doi: 10.1371/journal.pcbi.1001066.CrossRefGoogle Scholar
  69. Vazquez, L., Sapiro, G., & Randall, G. (1998). Segmenting neurons in electronic microscopy via geometric tracing. In Proc. of ICIP (pp. 814–818).Google Scholar
  70. Vazquez-Reina, A., Miller, E., & Pfister, H. (2009). Multiphase geometric couplings for the segmentation of neural processes. IEEE Computer Society conference on computer vision and pattern recognition (pp. 2020–2027).Google Scholar
  71. Venkatataju, K. U., Paiva, A., Jurrus, E., & Tasdizen, T. (2009). Automatic markup of neural cell membranes using boosted decision stumps. In Proceedings of the 6th IEEE international symposium on biomedical imaging (pp. 1039–1042).Google Scholar
  72. Visage Imaging (2012). Amira. http://www.amira.com.
  73. Vu, N., & Manjunath, B. S. (2008). Graph cut segmentation of neuronal structures from transmission electron micrographs. In 15th IEEE international conference on image processing, 2008. ICIP 2008 (pp. 725–728).Google Scholar
  74. Watanabe, K., Takeishi, H., Hayakawa, T., & Sasaki, H. (2010). Three-dimensional organization of the perivascular glial limiting membrane and its relationship with the vasculature: A scanning electron microscope study. Okajimas Folia Anatomica Japonica, 87(3), 109–121.PubMedCrossRefGoogle Scholar
  75. Wells, G., Venaille, C., & Torras, C. (1996). Promising research: Vision-based robot positioning using neural networks. Image and Vision Computing, 14(10), 715–732.CrossRefGoogle Scholar
  76. White, J. Q., Nicholas, T. J., Gritton, J., Truong, L., Davidson, E. R., & Jorgensen, E. M. (2007). The sensory circuitry for sexual attraction in C. elegans males. Current Biology, 17(21), 1847–1857.PubMedCrossRefGoogle Scholar
  77. White, J. G., Southgate, E., Thomson, J. N., & Brenner, F. R. S. (1986). The structure of the nervous system of the nematode Caenorhabditis elegans. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 314, 1–340.PubMedCrossRefGoogle Scholar
  78. Xiao, Y. P., Wang, Y., & Felleman, D. J. (2003). A spatially organized representation of colour in macaque cortical area v2. Nature, 421(6922), 535–539.PubMedCrossRefGoogle Scholar
  79. Yang, H.-F., & Choe, Y. (2009). Cell tracking and segmentation in electron microscopy images using graph cuts. In IEEE international symposium on biomedical imaging: From nano to macro, 2009. ISBI ’09 (pp. 306–309).Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Elizabeth Jurrus
    • 1
    • 2
  • Shigeki Watanabe
    • 3
  • Richard J. Giuly
    • 4
  • Antonio R. C. Paiva
    • 1
  • Mark H. Ellisman
    • 4
  • Erik M. Jorgensen
    • 3
  • Tolga Tasdizen
    • 1
    • 5
  1. 1.Scientific Computing and Imaging InstituteUniversity of UtahSalt Lake CityUSA
  2. 2.School of ComputingUniversity of UtahSalt Lake CityUSA
  3. 3.Department of BiologyUniversity of UtahSalt Lake CityUSA
  4. 4.National Center for Microscopy and Imaging ResearchUniversity of CaliforniaSan DiegoUSA
  5. 5.Department of Electrical EngineeringUniversity of UtahSalt Lake CityUSA

Personalised recommendations