Skip to main content

Batch skeleton extraction from ESPI fringe patterns using pix2pix conditional generative adversarial network

Abstract

The key to measurement by electronic speckle pattern interferometry (ESPI) is to obtain accurate phase information from the ESPI fringe patterns. We propose a fast batch skeleton extraction method for ESPI fringe patterns using the pix2pix conditional generative adversarial network (pix2pix cGAN). The network is trained by ESPI fringe patterns and complete skeleton images, and the trained network can quickly extract skeletons; it took 11.7 s to extract the skeletons of 200 experimental ESPI fringe patterns. Compared to the fringe skeleton method, cycle GAN method, and U-net method, our method can obtain accurate, complete, and smooth skeletons faster. In addition, for some broken ESPI fringe patterns, the traditional fringe skeleton method will fail, whereas complete skeletons can be obtained through the trained network.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Karaalioglu, C., Skarlatos, Y.: Measurement of thin film thickness by electronic speckle pattern interferometry. Opt. Commun. 234, 269–276 (2004)

    ADS  Article  Google Scholar 

  2. Dong, C.Z., Li, K., Jiang, Y.X., Arola, D., Zhang, D.S.: Evaluation of thermal expansion coefficient of carbon fiber reinforced composites using electronic speckle interferometry. Opt. Express 26, 531–543 (2018)

    ADS  Article  Google Scholar 

  3. Kumar, M., Agarwal, R., Bhutani, R., Shakher, C.: Measurement of strain distribution in cortical bone around miniscrew implants used for orthodontic anchorage using digital speckle pattern interferometry. Opt. Eng. 55, 054101 (2016)

    ADS  Article  Google Scholar 

  4. Pokharna, H., Schajer, G.S.: Quasi single-frame measurements with phase-stepped ESPI. Opt. Lasers Eng. 121, 181–188 (2019)

    Article  Google Scholar 

  5. Chen, F., Luo, W.D., Dale, M., Petniunas, A., Harwood, P., Brown, G.M.: High-speed ESPI and related techniques: overview and its application in the automotive industry. Opt. Lasers Eng. 40, 459–485 (2003)

    Article  Google Scholar 

  6. Sun, P.: The separation of out-of-plane displacement from in-plane components by fringe carrier method based on large image-shearing ESPI. Opt. Commun. 275, 305–310 (2007)

    ADS  Article  Google Scholar 

  7. Dai, X.J., Shao, X.X., Geng, Z.C., Yang, F.J., Jiang, Y., He, X.Y.: Vibration measurement based on electronic speckle pattern interferometry and radial basis function. Opt. Commun. 355, 33–43 (2015)

    ADS  Article  Google Scholar 

  8. Mohamed, M.A., Manurung, Y.H.P., Lakkonen, M.: Analysis of residual stress on FSW AA 6061 using hole-drilling with ESPI for HFMI treated condition. Mater. Sci. Forum 4328, 344–347 (2017)

    Article  Google Scholar 

  9. Pedrini, G., Martínez-García, V., Weidmann, P., Wenzelburger, M., Killinger, A., Weber, U., Schmauder, S., Gadow, R., Osten, W.: Residual stress analysis of ceramic coating by laser ablation and digital holography. Exp. Mech. 56, 683–701 (2016)

    Article  Google Scholar 

  10. Chen, H.N., Chen, J.J., Su, R.K.L.: Detection of crack evolution in plain concrete by electronic speckle pattern interferometry. Key Eng. Mater. 4466, 92–96 (2017)

    Article  Google Scholar 

  11. Wang, G., Li, Y.J., Zhou, H.C.: Application of the radial basis function interpolation to phase extraction from a single electronic speckle pattern interferometric fringe. Appl. Opt. 50, 3110–3117 (2011)

    ADS  Article  Google Scholar 

  12. Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafooran, M., Laak, J.A.W.M., Ginnekn, B., Sánchez, C.I.: A Survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)

    Article  Google Scholar 

  13. Wang, S., Kang, B., Ma, J.L., Zeng, X.J., Xiao, M.M., Guo, J., Cai, M.J., Yang, J.Y., Li, Y.D., Meng, X.F., Xu, B.: A deep learning algorithm using CT images to screen for Corona virus disease (COVID-19). Eur. Radiol. 31, 6096–6104 (2021)

    Article  Google Scholar 

  14. Petrellis, N.: Measurement of fish morphological features through image processing and deep learning techniques. Appl. Sci. 11, 4416–4416 (2021)

    Article  Google Scholar 

  15. Ballesteros, D.M., Rodriguez-Ortega, Y., Renza, D., Arce, G.: Deep4SNet: deep learning for fake speech classification. Expert Syst. Appl. 184, 115465 (2021)

    Article  Google Scholar 

  16. Hu, H.C., Chang, S.Y., Wang, C.H., Li, K.J., Cho, H.Y., Chen, Y.T., Lu, C.J., Tsai, T.P., Lee, O.K.S.: Deep learning application for vocal fold disease prediction through voice recognition: preliminary development study. J. Med. Internet Res. 23, e25247–e25247 (2021)

    Article  Google Scholar 

  17. Hao, F.G., Tang, C., Xu, M., Lei, Z.K.: Batch denoising of ESPI fringe patterns based on convolutional neural network. Appl. Opt. 58, 3338–3346 (2019)

    ADS  Article  Google Scholar 

  18. Kando, D., Tomioka, S., Miyamoto, N., Ueda, R.: Phase extraction from single interferogram including closed-fringe using deep learning. Appl. Sci. 17, 1–13 (2019)

    Google Scholar 

  19. Lin, B.W., Fu, S.J., Zhang, C.M., Wang, F.L., Li, Y.: Optical fringe patterns filtering based on multi-stage convolution neural network. Opt. Laser. Eng. 126, 105853 (2020)

    Article  Google Scholar 

  20. Anantrasirichai, N., Biggs, J., Albino, F.: Application of machine learning to classification of volcanic deformation in routinely generated InSAR data. J. Geophys. Res. 123, 6592–6606 (2018)

    ADS  Google Scholar 

  21. Li, B.Y., Tang, C., Zheng, T.Y., Lei, Z.K.: Fully automated extraction of the fringe skeletons in dynamic electronic speckle pattern interferometry using a U-Net convolutional neural network. Opt. Eng. 58, 023105 (2019)

    ADS  Google Scholar 

  22. Kots, M.V., Chukanov, V.S.: U-Net adaptation for multiple instance learning. J. Phys. Conf. Ser. 1236, 012061 (2019)

    Article  Google Scholar 

  23. Liu, C.X., Tang, C., Xu, M., Hao, F.G., Lei, Z.K.: Skeleton extraction and inpainting from poor, broken ESPI fringe with an M-net convolutional neural network. Appl. Opt. 59, 5300–5308 (2020)

    ADS  Article  Google Scholar 

  24. Abdelmotaal, H., Abdou, A.A., Omar, A.F., EISebaity, D.M., Abdelazeem, K.: Pix2pix conditional generative adversarial networks for scheimpflug camera color-coded corneal tomography image generation. Transl. Vis. Sci. Technol. 10, 21 (2021)

    Article  Google Scholar 

  25. Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., Bharath, A.A.: Generative adversarial networks: an overview. IEEE Signal Process. Mag. 35, 53–65 (2017)

    ADS  Article  Google Scholar 

  26. Beke, A., Kumbasar, T.: Learning with type-2 fuzzy activation functions to improve the performance of deep neural networks. Eng. Appl. Artif. Intell. 85, 372–384 (2019)

    Article  Google Scholar 

  27. Wu, S., Li, G.Q., Deng, L., Liu, L., Wu, D., Xie, Y., Shi, L.P.: L1-norm batch normalization for efficient training of deep neural networks. IEEE Trans. Neural Netw. Learn. Syst. 30, 2043–2051 (2019)

    Article  Google Scholar 

  28. Isola, P., Zhu, J.Y., Zhou, T.H., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: IEEE Conference on Computer Vision & Pattern Recognition, pp. 21–26 (2017)

  29. Saeed, K., Tabedzki, M., Rybnik, M., Adamski, M.: K3M: a universal algorithm for image keletonization and a review of thinning techniques. Int. J. Appl. Mater. Comput. Pol. 20, 317–335 (2010)

    MATH  Google Scholar 

  30. Chen, M.M., Tang, C., Xu, M., Lei, Z.K.: Binarization of ESPI fringe patterns based on local entropy. Opt. Express 27, 32378–32391 (2019)

    ADS  Article  Google Scholar 

  31. Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inform. Process. Manag. 45, 427–437 (2009)

    Article  Google Scholar 

  32. Setiadi, D.R.I.M.: PSNR vs SSIM: imperceptibility quality assessment for image steganography. Multimed. Tools Appl. 80, 8423–8444 (2021)

    Article  Google Scholar 

Download references

Funding

This project was funded by the National Natural Science Foundation of China (Grant no 62175059).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gaofu Men.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Zhang, Z., Zhu, Q. et al. Batch skeleton extraction from ESPI fringe patterns using pix2pix conditional generative adversarial network. Opt Rev 29, 97–105 (2022). https://doi.org/10.1007/s10043-022-00728-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10043-022-00728-1

Keywords

  • ESPI
  • Fringe pattern
  • Skeleton extraction
  • Deep learning