A unified deep-learning network to accurately segment insulin granules of different animal models imaged under different electron microscopy methodologies
Insulin is important for body metabolism regulation and glucose homeostasis, and its dysregulation often leads to metabolic syndrome (MS) and diabetes. Insulin is normally stored in large dense-core vesicles (LDCVs) in pancreatic beta cells, and significant reductions in the number, size, gray level and density of insulin granules confer diabetes both in mice (Xue et al., 2012) and humans (Masini et al., 2012). Due to the difficulty of obtaining human islet samples, many works use mice as the animal model. However, the architecture of normal islets in humans differs significantly from that of rodents (Cabrera et al., 2006). Beta cells in the mouse islet core are surrounded by the mantle comprising of alpha and delta cells, whereas alpha, beta and delta cells are intermingled in human islets. The structural differences suggest a possible difference in islet function. In this sense, non-human primates such as rhesus monkeys are a better model, as their islets share a similar architecture with humans (Cabrera et al., 2006). The quantitative nature of the insulin granules within monkey islet beta cells, and whether they change during metabolic dysregulation remain to be explored. Under the electron microscope (EM), insulin granules are usually spherical organelles containing an electron dense-core separated from the surrounding membrane by a halo, with a size of ranged from 100–800 nm (MacDonald et al., 2006) in mouse beta cells. This number is estimated as ~10,000 per beta cell. Because thin-section EM do not necessarily provide the correct spatial coordination of granules within one beta cell, recent years have witnessed the emergence of volumetric electron microscopy techniques such as electron tomography and focused ion beam scanning electron microscopy (FIB-SEM) (Briggman and Bock, 2012).
For the first time, we collected three-dimensional images of pancreatic beta cells in wild type (WT) and MS rhesus monkeys with a FIB-SEM and manually annotated granules from a relatively small number of images. Because the morphological and structural natures of insulin granules are important for their optimal function, quantitative and automatic analysis of insulin granules in islets is important. Manually segmenting densely distributed LDCVs is a labor-intensive task due to the big datasets brought by saturated and continuous sampling in the lateral and axial axes. Although several semi-automated segmentation methods for rodent LDCVs have been proposed (Diaz et al., 2010), they are built on time-consuming and human-designed features that cannot adapt to micrographs of different magnification and are extremely prone to errors for images with low signal-to-noise ratios. Nevertheless, the machine learning field has witnessed a flourishing of “deep-learning” algorithms. Since AlexNet outperformed all other algorithms by a large margin in the ImageNet contest in 2012, a variety of deep-learning methods for image segmentation have been widely used, including the standard convolutional networks (CNN) (Van Valen et al., 2016), and fully convolutional networks (FCN) (Long, 2014). Recently, multi-scale features, dilated convolutions, context encoding, conditional random fields (CRFs) are incorporated to FCNs to improve spatial resolution, bringing more novel and complicated network structures such as Tiramisu (Jegou et al., 2017), Deeplab (Chen et al., 2018). However, different from natural images, we only have a small electron micrograph dataset annotated, in which insulin granules only occupy a small portion of the image. In order to prevent overfitting, we take concise deep-learning networks as the starting point, such as U-Net (Ronneberger et al., 2015).
The whole image processing can be divided into three parts (Fig. S1). First, we used histogram equalization for image pre-processing, which homogenized the uneven illumination (Fig. S2), and proved to significantly speed up the convergence of the network. Next, we fed the MFCN network with pre-processed images for the binary segmentation maps. Finally, we used a simple watershed based edge detection method for the instance segmentation of the binary maps. Based on the results of the final step, we could quantitatively extract spatio-temporal information for each granule, such as the coordinates of the boundary, areas, mean gray level values and perimeters.
Side-by-side, we compared the performance of our method with other previously published methods, including the random forest (Smith and Frank, 2016), the standard convolutional networks (Van Valen et al., 2016), and the U-Net (Ronneberger, 2015). For an objective and fair comparison, we adopted two sets of evaluation metrics. One was used for measuring classification accuracy of each pixel, including pixel accuracy, mean accuracy and mean region intersection over union (mean IU) (Long, 2014). The other was used for evaluating each segmented granule, including true positive (TP) false positive (FP), true negative (TN) and false negative (FN) (detailed in Supplementary Materials).
From Figures. S3 and S4, we could infer that the random forest algorithm performed the worst, despite the high pixel accuracy it achieved (96%, Table S2). This discrepancy was because the insulin granules occupied only a small part of the whole EM image. The standard CNN network outperformed the random forest algorithm in several aspects but their results were variable among different images (Table S3). As it only perceives local semantic information from fixed and small size image patches, the standard CNN may not be suitable for detecting granules of different sizes and shapes. In addition, it needed more time than other methods, as it was computationally intense to calculate many redundant, overlapping patches for segmentation. U-Net, a typical variant of FCN for biomedical segmentation, is characterized by a U-shaped architecture containing symmetrical down-sampling and up-sampling blocks. Better than the random forest and CNN algorithm, it still achieved only ~68% precision in detecting insulin granules from WT and MS islets (Table S3). Relatively, MFCN reached ~87% precision in detecting granules (Table S3). Besides, in detecting insulin granules of low signal-to-noise contrast, U-Net segmented granules were more irregular, non-continuous and fragmented than MFCN (Figs. S3 and S4). Apart from that, many dark regions within the nucleus were incorrectly detected as granules by U-Net but not the MFCN, which led to high error rate of the former (Fig. S5).
Compared with U-Net, MFCN trimmed off several redundant skip-layer connections, added multi-branch down-sampling, and combined multi-scale contextures to produce the final output. Benefiting from these features and the combination of receptive fields of different sizes, we have demonstrated the robustness of MFCN in detecting granules with diversified sizes and shapes while rejecting dark non-granular structures, which is superior to other algorithms tested (Tables S2 and S3). Having established the robustness and superiority of the current configuration of MFCN, we tested whether the trained network could be used to segment insulin granules of different species and EM images using different microscopes. We used the same MFCN network to detect insulin granules isolated from mouse islets and imaged under the STEM tomography and thin-slice TEM (Fig. S6). Without any fine-tuning, our network, trained on the FIB-SEM dataset, readily resolved insulin granules of different signal-to-noise ratios in the tomography data and the granules of various intensities from single-slice TEM data. As a result, the trained MFCN is insensitive to image resolution, light intensity, type of electron microscope, and animal species, and thus confers robustness and transferability.
In summary, we have developed a novel deep learning framework to auto-segment insulin granules from EM images from WT and MS rhesus monkey beta cells. We show that the proposed MFCN has outperformed other algorithms in resolving insulin granules of distinct shapes and sizes, and offers good transferability in handling of data from different electron microscopes. Therefore, MFCN can be represented as a significant step toward fully automated segmentation and quantification of insulin granules from EM images. We believe that the MFCN and its underlying principles could be used for other classification problems in biological or medical image analysis in general. Applying this network to analyze the morphology and spatial distributions of insulin granules in beta cells of MS rhesus monkeys has already provided some insights. First, we confirmed that the morphology of insulin granules in rhesus monkeys is similar to that of humans or rodents. Second, as the number of granules per unit area of cytoplasm increased in MS monkeys compared with the control, this possibly reflects a compensatory increase in insulin synthesis at the early stage of diabetes in non-human primates as well. Third, the sizes and shapes of dense-cores changed in the beta cells of MS monkeys, as there were more empty granules or rod-like dense-core granules in the diseased animal. As the dense-core is produced by the co-crystallization of Zinc and insulin, these changes suggest that defects in insulin synthesis, packaging or crystallization may manifest at the early stage of disease development when there is an absolute increase in insulin granules. Finally, the defective docked granules found in MS beta-cells are consistent with the down-regulation of SNARE proteins and defective docking of insulin granules in the beta-cells from diabetic rodents and patients (Ostenson et al., 2006). All these findings may help to address this theory and prove insights into the diabetes progression process in humans.
We thank the Non-human Primate Research Center of Peking University for the samples of monkey pancreases, Dr. Lusheng Gu and Dr. Weixing Li for data analysis (Center for Biological Instrument Development, CBID), and Dr. Jianguo Zhang and Dr. Tongxin Niu for FIB-SEM imaging and imaging analysis (Center for Biological Imaging, CBI), Core Facility for Protein Research, Institute of Biophysics, Chinese Academy of Sciences. This work was supported by grants from the National Key R&D Program of China (Grant Nos. 2017YFA0504700 and 2016YFA0500400), the National Natural Science Foundation of China (Grant Nos. 31570839, 31661143041, 61472395, 31327901, 31521062 and 31730054), the Beijing Natural Science Foundation (L172003) and Joint Program between Chinese Academy of Sciences and Peking University.
Xiaoya Zhang, Xiaohong Peng, Chengsheng Han, Wenzhen Zhu, Lisi Wei, Yulin Zhang, Yi Wang, Xiuqin Zhang, Hao Tang, Jianshe Zhang, Xiaojun Xu, Fengping Feng, Erlin Yao, Yanhong Xue, Guangming Tan, Tao Xu and Liangyi Chen declare that they have no conflict of interest. All institutional and national guidelines for the care and use of laboratory animals were followed.
- Jegou S, Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: fully convolutional DenseNets for semantic segmentation. In 2017 IEEE conference on computer vision and pattern recognition workshops (Cvprw), pp 1175–1183Google Scholar
- Long J, Shelhamer E, Darrell T (2014) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651Google Scholar
- Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In medical image computing and computer-assisted intervention—MICCAI 2015, pp 234–241Google Scholar
- Szegedy C, Liu W, Jia YQ, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. IEEE Conf Comput Vis Pattern Recognit 2015:1–9Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.