Abstract
Fourier Ptychographic Microscopy (FPM) is an imaging procedure that overcomes the traditional limit on Space-Bandwidth Product (SBP) of conventional microscopes through computational means. It utilizes multiple images captured using a low numerical aperture (NA) objective and enables high-resolution phase imaging through frequency domain stitching. Existing FPM reconstruction methods can be broadly categorized into two approaches: iterative optimization based methods, which are based on the physics of the forward imaging model, and data-driven methods which commonly employ a feed-forward deep learning framework. We propose a hybrid model-driven residual network that combines the knowledge of the forward imaging system with a deep data-driven network. Our proposed architecture, LWGNet, unrolls traditional Wirtinger flow optimization algorithm into a novel neural network design that enhances the gradient images through complex convolutional blocks. Unlike other conventional unrolling techniques, LWGNet uses fewer stages while performing at par or even better than existing traditional and deep learning techniques, particularly, for low-cost and low dynamic range CMOS sensors. This improvement in performance for low-bit depth and low-cost sensors has the potential to bring down the cost of FPM imaging setup significantly. Finally, we show consistently improved performance on our collected real data (We have made the code avaiable at: https://github.com/at3e/LWGNet.git).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aidukas, T., Eckert, R., Harvey, A.R., Waller, L., Konda, P.C.: Low-cost, sub-micron resolution, wide-field computational microscopy using opensource hardware. Sci. Rep. 9(1), 1–12 (2019)
Bian, L., Suo, J., Zheng, G., Guo, K., Chen, F., Dai, Q.: Fourier ptychographic reconstruction using wirtinger flow optimization. Opt. Exp. 23(4), 4856–4866 (2015). https://doi.org/10.1364/OE.23.004856, http://www.opticsexpress.org/abstract.cfm?URI=oe-23-4-4856
Bian, L., et al.: Motion-corrected Fourier ptychography. Biomed. Opt. Exp. 7(11), 4543–4553 (2016)
Boominathan, L., Maniparambil, M., Gupta, H., Baburajan, R., Mitra, K.: Phase retrieval for Fourier ptychography under varying amount of measurements. arXiv preprint arXiv:1805.03593 (2018)
Bostan, E., Soltanolkotabi, M., Ren, D., Waller, L.: Accelerated wirtinger flow for multiplexed Fourier ptychographic microscopy. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 3823–3827. IEEE (2018)
Breslauer, D.N., Maamari, R.N., Switz, N.A., Lam, W.A., Fletcher, D.A.: Mobile phone based clinical microscopy for global health applications. PLoS ONE 4(7), e6320 (2009)
Jiang, S., Guo, K., Liao, J., Zheng, G.: Solving Fourier ptychographic imaging problems via neural network modeling and tensorflow. Biomed. Opt. Exp. 9(7), 3306–3319 (2018). https://doi.org/10.1364/BOE.9.003306, http://www.osapublishing.org/boe/abstract.cfm?URI=boe-9-7-3306
Kappeler, A., Ghosh, S., Holloway, J., Cossairt, O., Katsaggelos, A.: PtychNet: CNN based Fourier ptychography. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 1712–1716. IEEE (2017)
Kellman, M., Bostan, E., Chen, M., Waller, L.: Data-driven design for fourier ptychographic microscopy. In: 2019 IEEE International Conference on Computational Photography (ICCP), pp. 1–8. IEEE (2019)
Kim, J., Henley, B.M., Kim, C.H., Lester, H.A., Yang, C.: Incubator embedded cell culture imaging system (emsight) based on Fourier ptychographic microscopy. Biomed. Opt. Exp. 7(8), 3097–3110 (2016)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network (2017)
Lee, K.C., Lee, K., Jung, J., Lee, S.H., Kim, D., Lee, S.A.: A smartphone-based Fourier ptychographic microscope using the display screen for illumination. ACS Phot. 8(5), 1307–1315 (2021)
Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
Nguyen, T., Xue, Y., Li, Y., Tian, L., Nehmetallah, G.: Deep learning approach for fourier ptychography microscopy. Opt. Exp. 26(20), 26470–26484 (2018). https://doi.org/10.1364/OE.26.026470,http://www.opticsexpress.org/abstract.cfm?URI=oe-26-20-26470
Ou, X., Horstmeyer, R., Yang, C., Zheng, G.: Quantitative phase imaging via fourier ptychographic microscopy. Opt. Lett. 38(22), 4845–4848 (2013). https://doi.org/10.1364/OL.38.004845,http://ol.osa.org/abstract.cfm?URI=ol-38-22-4845
Paszke, A., et al.: Pytorch: An imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32, pp. 8024–8035. Curran Associates, Inc. (2019). http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
Dee, F.R., Leaven, T.: Department of Pathology: Iowa Virtual Slidebox, http://www.path.uiowa.edu/virtualslidebox/
Smith, Z., et al.: Cell-phone-based platform for biomedical device development and education applications. PloS one 6(3), e17150 (2011)
Song, P., Jiang, S., Zhang, H., Huang, X., Zhang, Y., Zheng, G.: Full-field Fourier ptychography (Ffp): spatially varying pupil modeling and its application for rapid field-dependent aberration metrology. APL Photon. 4(5), 050802 (2019)
Tian, L., Li, X., Ramchandran, K., Waller, L.: Multiplexed coded illumination for Fourier ptychography with an led array microscope. Biomed. Opt. Exp. 5(7), 2376–2389 (2014). https://doi.org/10.1364/BOE.5.002376,http://www.osapublishing.org/boe/abstract.cfm?URI=boe-5-7-2376
Tian, L., Liu, Z., Yeh, L.H., Chen, M., Zhong, J., Waller, L.: Computational illumination for high-speed in vitro fourier ptychographic microscopy. Optica 2(10), 904–911 (2015). http://www.osapublishing.org/optica/abstract.cfm?URI=optica-2-10-904, https://www.laurawaller.com/opensource/
Trabelsi, C., et al.: Deep complex networks. In: ICLR (2018)
Zheng, G., Horstmeyer, R., Yang, C.: Wide-field, high-resolution Fourier ptychographic microscopy. Nat. Photon. 7(9), 739–745 (2013)
Acknowledgement
We gratefully acknowledge the funding from DST IMPRINT-2 (IMP/2018/001168).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Saha, A., Khan, S.S., Sehrawat, S., Prabhu, S.S., Bhattacharya, S., Mitra, K. (2022). LWGNet - Learned Wirtinger Gradients for Fourier Ptychographic Phase Retrieval. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13667. Springer, Cham. https://doi.org/10.1007/978-3-031-20071-7_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-20071-7_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20070-0
Online ISBN: 978-3-031-20071-7
eBook Packages: Computer ScienceComputer Science (R0)