Abstract
Formalin-Fixed Paraffin-Embedded (FFPE) and Fresh Frozen (FF) are two major types of histopathological Whole Slide Images (WSIs). FFPE provides high-quality images, however the acquisition process usually takes 12 to 48 h, while FF with relatively low-quality images takes less than 15 min to acquire. In this work, we focus on the task of translating FF to FFPE style (FF2FFPE), to synthesize FFPE-style images from FF images. However, WSIs with giga-pixels impose heavy constraints on computation and time resources. To address these issues, we propose the fastFF2FFPE for translating FF into FFPE-style efficiently. Specifically, we decompose FF images into low- and high-frequency components based on the Laplacian Pyramid, wherein the low-frequency component at low resolution is transformed into FFPE-style with low computational cost, and the high-frequency component is used for providing details. We further employ contrastive learning to encourage similarities between original and output patches. We conduct FF2FFPE translation experiments on The Cancer Genome Atlas (TCGA) Glioblastoma Multiforme (GBM) and Lung Squamous Cell Carcinoma (LUSC) datasets, and verify the efficacy of our model on Microsatellite Instability prediction in gastrointestinal cancer. The code and models are released at https://github.com/hellodfan/fastFF2FFPE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)
Brown, R.W.: Histologic Preparations: Common Problems and their Solutions. College of American Pathologists (2009)
Burt, P.J., Adelson, E.H.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 3(4), 532–540 (1983)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning (ICML), pp. 1597–1607. PMLR (2020)
Chen, X., He, K.: Exploring simple siamese representation learning. In: CVPR, pp. 15750–15758 (2021)
Cong, C., et al.: Semi-supervised adversarial learning for stain normalisation in histopathology images. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12908, pp. 581–591. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87237-3_56
Denton, E., Chintala, S., Szlam, A., Fergus, R.: Deep generative image models using a laplacian pyramid of adversarial networks. In: NeurIPS, vol. 28, pp. 1486–1494 (2015)
Falahkheirkhah, K., Guo, T., Hwang, M., et al.: A generative adversarial approach to facilitate archival-quality histopathologic diagnoses from frozen tissue sections. Lab. Investig. 102(5), 554–559 (2022)
Fan, L., Sowmya, A., Meijering, E., Song, Y.: Learning visual features by colorization for slide-consistent survival prediction from whole slide images. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12908, pp. 592–601. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87237-3_57
Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets. In: NeurIPS, vol. 27 (2014)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: CVPR, pp. 9729–9738 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: NeurIPS, vol. 30 (2017)
Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: ICCV, pp. 1501–1510 (2017)
Jaafar, H.: Intra-operative frozen section consultation: concepts, applications and limitations. Malaysian J. Med. Sci. MJMS 13(1), 4 (2006)
Kandoth, C., McLellan, M.D., Vandin, F., et al.: Mutational landscape and significance across 12 major cancer types. Nature 502(7471), 333–339 (2013)
Kather, J.N., Pearson, A.T., Halama, N., et al.: Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 25(7), 1054–1056 (2019)
Khosla, P., Teterwak, P., Wang, C., et al.: Supervised contrastive learning. In: NeurIPS, vol. 33, pp. 18661–18673 (2020)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: International Conference for Learning Representations (ICLR) (2015)
Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep Laplacian pyramid networks for fast and accurate super-resolution. In: CVPR, pp. 624–632 (2017)
Liang, J., Zeng, H., Zhang, L.: High-resolution photorealistic image translation in real-time: a Laplacian pyramid translation network. In: CVPR, pp. 9392–9400 (2021)
Liu, X., Zhang, F., Hou, Z., et al.: Self-supervised learning: generative or contrastive. IEEE Trans. Knowl. Data Eng. (2021). https://doi.org/10.1109/TKDE.2021.3090866
Macenko, M., Niethammer, M., Marron, J.S., et al.: A method for normalizing histology slides for quantitative analysis. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI), pp. 1107–1110. IEEE (2009)
Mao, X., Li, Q., Xie, H., et al.: Least squares generative adversarial networks. In: ICCV, pp. 2794–2802 (2017)
Ozyoruk, K.B., Can, S., Gokceler, G.I., et al.: Deep learning-based frozen section to FFPE translation. arXiv preprint arXiv:2107.11786 (2021)
Pang, Y., Lin, J., Qin, T., Chen, Z.: Image-to-image translation: methods and applications. IEEE Trans. Multim. (2021). https://doi.org/10.1109/TMM.2021.3109419
Park, T., Efros, A.A., Zhang, R., Zhu, J.-Y.: Contrastive learning for unpaired image-to-image translation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12354, pp. 319–345. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58545-7_19
Paszke, A., Gross, S., Massa, F., et al.: PyTorch: an imperative style, high-performance deep learning library. In: NeurIPS, pp. 8024–8035 (2019)
Rolls, G.O., Farmer, N.J., Hall, J.B.: Artifacts in histological and cytological preparations. Leica Microsystems (2008)
Sandler, M., Howard, A., Zhu, M., et al.: MobileNetv2: Inverted residuals and linear bottlenecks. In: CVPR, pp. 4510–4520 (2018)
Shaban, M.T., Baur, C., Navab, N., Albarqouni, S.: StainGAN: stain style transfer for digital histological images. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI), pp. 953–956. IEEE (2019)
Taqi, S.A., Sami, S.A., Sami, L.B., Zaki, S.A.: A review of artifacts in histopathology. J. Oral Maxillof. Pathol. 22(2), 279 (2018)
Wang, T.C., Liu, M.Y., Zhu, J.Y., et al.: High-resolution image synthesis and semantic manipulation with conditional GANs. In: CVPR, pp. 8798–8807 (2018)
Wolterink, J.M., Dinkla, A.M., Savenije, M.H.F., Seevinck, P.R., van den Berg, C.A.T., Išgum, I.: Deep MR to CT synthesis using unpaired data. In: Tsaftaris, S.A., Gooya, A., Frangi, A.F., Prince, J.L. (eds.) SASHIMI 2017. LNCS, vol. 10557, pp. 14–23. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68127-6_2
Yang, H., et al.: Unpaired brain MR-to-CT synthesis using a structure-constrained CycleGAN. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 174–182. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_20
Yi, X., Walia, E., Babyn, P.: Generative adversarial network in medical imaging: a review. Med. Image Anal. 58, 101552 (2019)
Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: Mixup: beyond empirical risk minimization. In: International Conference for Learning Representations (ICLR) (2018)
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imag. 39(6), 1856–1867 (2020)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired Image-to-Image translation using cycle-consistent adversarial networks. In: ICCV, pp. 2223–2232 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fan, L., Sowmya, A., Meijering, E., Song, Y. (2022). Fast FF-to-FFPE Whole Slide Image Translation via Laplacian Pyramid and Contrastive Learning. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13432. Springer, Cham. https://doi.org/10.1007/978-3-031-16434-7_40
Download citation
DOI: https://doi.org/10.1007/978-3-031-16434-7_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16433-0
Online ISBN: 978-3-031-16434-7
eBook Packages: Computer ScienceComputer Science (R0)
