Abstract
In the medical field, semantic segmentation has recently been dominated by deep-learning based image processing methods. Convolutional Neural Network approaches analyze image patches, draw complex features and latent representations and take advantage of these to label image pixels and voxels. In this paper, we investigate the usefulness of Recurrent Neural Network (RNN) for segmentation of OCT images, in which the intensity of elements of each A-mode depend on the path projected light takes through anatomical tissues to reach that point. The idea of this work is to reformulate this sequential voxel labeling/segmentation problem as language processing. Instead of treating images as patches, we regard them as a set of pixel column sequences and thus tackle the task of image segmentation, in this case pixel sequence labeling, as a natural language processing alike problem. Anatomical consistency, i.e. expected sequence of voxels representing retinal layers of eye’s anatomy along each OCT ray, serves as a fixed and learnable grammar. We show the effectiveness of this approach on a layer segmentation task for retinal Optical Coherence Tomography (OCT) data. Due to the inherent directionality of the modality, certain properties and artifacts such as varying signal strength and shadowing form a consistent pattern along increasing imaging depth. The retinal layer structure lends itself to our approach due to the fixed order of layers along the imaging direction. We investigate the influence of different model choices including simple RNNS, LSTMs and GRU structures on the outcome of this layer segmentation approach. Experimental results show that the potential of this idea that is on par with state of the art works while being flexible to changes in the data structure.
The author was not affiliated with Ludwig-Maximilians-University Munich at the time of submission.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aafaq, N., Mian, A., Liu, W., Gilani, S.Z., Shah, M.: Video description: a survey of methods, datasets, and evaluation metrics. ACM Comput. Surv. (CSUR) 52(6), 1–37 (2019)
Chiu, S.J., Allingham, M.J., Mettu, P.S., Cousins, S.W., Izatt, J.A., Farsiu, S.: Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema. Biomed. Opt. Express 6(4), 1172–1194 (2015)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
He, Y., et al.: Topology guaranteed segmentation of the human retina from OCT using convolutional neural networks. arXiv e-prints arXiv:1803.05120, March 2018
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Jurafsky, D., Martin, J.H.: Speech and Language Processing -. Pearson Education, Amsterdam (2014)
Kiaee, F., Fahimi, H., Rabbani, H.: Intra-retinal layer segmentation of optical coherence tomography using 3d fully convolutional networks. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 2795–2799. IEEE (2018)
Kugelman, J., et al.: Automatic choroidal segmentation in oct images using supervised deep learning methods. Sci. Rep. 9(1), 1–13 (2019)
Li, B., Zhao, Z., Liu, T., Wang, P., Du, X.: Weighted neural bag-of-n-grams model: new baselines for text classification. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 1591–1600. The COLING 2016 Organizing Committee, Osaka, Japan, December 2016
Minaee, S., Boykov, Y., Porikli, F., Plaza, A., Kehtarnavaz, N., Terzopoulos, D.: Image segmentation using deep learning: a survey. arXiv e-prints arXiv:2001.05566, January 2020
Orlando, J.I., et al.: U2-net: a Bayesian u-net model with epistemic uncertainty feedback for photoreceptor layer segmentation in pathological OCT scans. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1441–1445. IEEE (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Roy, A.G., et al.: ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomed. Opt. Express 8(8), 3627 (2017). https://doi.org/10.1364/BOE.8.003627. http://arxiv.org/abs/1704.02161. https://www.osapublishing.org/abstract.cfm?URI=boe-8-8-3627
Trucco, E., MacGillivray, T., Xu, Y.: Computational Retinal Image Analysis - Tools, Applications and Perspectives. Elsevier, Amsterdam (2019)
Visin, F., et al.: Reseg: a recurrent neural network-based model for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 41–48 (2016)
Visin, F., Kastner, K., Cho, K., Matteucci, M., Courville, A., Bengio, Y.: Renet: a recurrent neural network based alternative to convolutional networks. arXiv preprint arXiv:1505.00393 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Tran, A., Weiss, J., Albarqouni, S., Faghi Roohi, S., Navab, N. (2020). Retinal Layer Segmentation Reformulated as OCT Language Processing. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12265. Springer, Cham. https://doi.org/10.1007/978-3-030-59722-1_67
Download citation
DOI: https://doi.org/10.1007/978-3-030-59722-1_67
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59721-4
Online ISBN: 978-3-030-59722-1
eBook Packages: Computer ScienceComputer Science (R0)
