Abstract
Style transfer is a field with growing interest and use cases in deep learning. Recent work has shown Generative Adversarial Networks (GANs) can be used to create realistic images of virtually stained slide images in digital pathology with clinically validated interpretability. Digital pathology images are typically of extremely high resolution, making tilewise analysis necessary for deep learning applications. It has been shown that image generators with instance normalization can cause a tiling artifact when a large image is reconstructed from the tilewise analysis. We introduce a novel perceptual embedding consistency loss significantly reducing the tiling artifact created in the reconstructed whole slide image (WSI). We validate our results by comparing virtually stained slide images with consecutive real stained tissue slide images. We also demonstrate that our model is more robust to contrast, color and brightness perturbations by running comparative sensitivity analysis tests.
S. Albarqouni and E. Klaiman—Shared senior authorship.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bayramoglu, N., Kaakinen, M., Eklund, L., Heikkila, J.: Towards virtual h&e staining of hyperspectral lung histology images using conditional generative adversarial networks. In: ICCV, pp. 64–71 (2017)
Burlingame, E.A., Margolin, A., Gray, J.W., Chang, Y.H.: Shift: speedy histopathological-to-immunofluorescent translation of whole slide images using conditional generative adversarial networks. In: Medical Imaging 2018: Digital Pathology, vol. 10581, p. 1058105. SPIE (2018)
Chang, H., Lu, J., Yu, F., Finkelstein, A.: Pairedcyclegan: asymmetric style transfer for applying and removing makeup. In: CVPR, pp. 40–48 (2018)
Christiansen, E.M., et al.: In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173(3), 792–803 (2018)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: CVPR, pp. 2414–2423 (2016)
Giacomelli, M.G., et al.: Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging. PLoS ONE 11(8), e0159337 (2016)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Lahiani, A., Gildenblat, J., Klaman, I., Albarqouni, S., Navab, N., Klaiman, E.: Virtualization of tissue staining in digital pathology using an unsupervised deep learning approach. arXiv preprint arXiv:1810.06415 (2018)
Lahiani, A., Klaiman, E., Grimm, O.: Enabling histopathological annotations onimmunofluorescent images through virtualization of hematoxylin and eosin. JPI 9, 1 (2018)
Ounkomol, C., Seshamani, S., Maleckar, M.M., Collman, F., Johnson, G.R.: Label-free prediction of three-dimensional fluorescence images from transmitted-light microscopy. Nat. Methods 15(11), 917 (2018)
Rana, A., Yauney, G., Lowe, A., Shah, P.: Computational histological staining and destaining of prostate core biopsy RGB images with generative adversarial neural networks. In: ICMLA, pp. 828–834. IEEE (2018)
Rivenson, Y., Wang, H., Wei, Z., Zhang, Y., Gunaydin, H., Ozcan, A.: Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue. arXiv preprint arXiv:1803.11293 (2018)
Royer, A., et al.: XGAN: unsupervised image-to-image translation for many-to-many mappings. arXiv preprint arXiv:1711.05139 (2017)
Sampat, M.P., Wang, Z., Gupta, S., Bovik, A.C., Markey, M.K.: Complex wavelet structural similarity: a new image similarity index. IEEE Trans. Image Process. 18(11), 2385–2401 (2009)
Shaban, M.T., Baur, C., Navab, N., Albarqouni, S.: StainGan: stain style transfer for digital histological images. arXiv preprint arXiv:1804.01601 (2018)
Tao, Y.K., et al.: Assessment of breast pathologies using nonlinear microscopy. PNAS 111(43), 15304–15309 (2014)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Instance normalization: the missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: ICCV. pp. 2223–2232 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Lahiani, A., Navab, N., Albarqouni, S., Klaiman, E. (2019). Perceptual Embedding Consistency for Seamless Reconstruction of Tilewise Style Transfer. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11764. Springer, Cham. https://doi.org/10.1007/978-3-030-32239-7_63
Download citation
DOI: https://doi.org/10.1007/978-3-030-32239-7_63
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32238-0
Online ISBN: 978-3-030-32239-7
eBook Packages: Computer ScienceComputer Science (R0)
