Skip to main content
SpringerLink
Log in

Defect Enhancement and Image Noise Reduction Analysis Using Partial Least Square-Generative Adversarial Networks (PLS-GANs) in Thermographic Nondestructive Evaluation

  • Published:
Journal of Nondestructive Evaluation Aims and scope Submit manuscript

Abstract

With the recent developments in artificial intelligence, deep learning (convolutional neural networks) has been investigated for the quality management of materials by infrared thermography (IRT). In this work, we present an approach to efficiently process thermal data in order to improve the defect detection performance of IRT. A defect-enhanced algorithm based on Generative Adversarial Networks (GANs) has been designed and implemented qualitatively analysis and improve defects visibility based on data augmentation from a deep learning approach. We implemented a data dimension reduction method based on Partial Least Square Thermography (PLST) merged with GAN Networks (PLS-GANs) to achieve interpretable feature extraction and visualization, and compare result with thermal data of Pulsed Thermography in order to evaluate the efficacy of the proposed algorithm. By applying PLS-GANs, a small dataset of thermal data can be able to enlarge the diversity of data in order to improve the performance of the detection model. The experimental results were empirically illustrated over the benchmark specimens: Carbon Fiber Reinforced Polymers (CFRPs). Consequently, the experimental detection results on the CFRPs demonstrated its feasibility of the PLST-GANs method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

References

  1. Ibarra-Castanedo, C., Tarpani, J.R., Maldague, X.: Nondestructive testing with thermography. Eur. J. Phys. 34, S91–S109 (2013)

    Article  Google Scholar 

  2. Gao, B., Bai, L., Woo, W.L., Tian, G.Y., Cheng, Y.: Automatic defect identification of eddy current pulsed thermography using single channel blind source separation. IEEE Trans. Instrum. Meas. 63(4), 913–922 (2013)

    Article  Google Scholar 

  3. Shepard, S.M.: Advances in pulsed thermography. In: Thermosense XXIII. International Society for Optics and Photonics, vol. 4360, pp. 511–515 (2001)

  4. Nixon, M., Aguado, A.: Feature Extraction and Image Processing for Computer Vision. Academic Press, New York (2019)

    Google Scholar 

  5. Rajic, N.: Principal component thermography for flaw contrast enhancement and flaw depth characterisation in composite structures. Compos. Struct. 58(4), 521–528 (2002)

    Article  Google Scholar 

  6. Maldague, X., Galmiche, F., Ziadi, A.: Advances in pulsed phase thermography. Infrared Phys. Technol. 43(3–5), 175–181 (2002)

    Article  Google Scholar 

  7. Balageas, D.L., Roche, J.M., Leroy, F.H., et al.: The thermographic signal reconstruction method: a powerful tool for the enhancement of transient thermographic images. Biocybernet. Biomed. Eng. 35(1), 1–9 (2015)

    Article  Google Scholar 

  8. Deng, L., Yu, D.: Deep learning: methods and applications. Found. Trends Signal Process. 7(3–4), 197–387 (2014)

    Article  MathSciNet  Google Scholar 

  9. Fang, Q., Nguyen, B.D., Castanedo, C.I., et al.: Automatic defect detection in infrared thermography by deep learning algorithm. In: Thermosense: Thermal Infrared Applications XLII. International Society for Optics and Photonics, vol. 11409, p. 114090T, (2020)

  10. Fang, Q., Ibarra-Castanedo, C., Maldague, X.: Automatic defects segmentation and identification by deep learning algorithm with pulsed thermography: synthetic and experimental data. Big Data Cognit. Comput. 5(1), 9 (2021)

    Article  Google Scholar 

  11. Hu, J., Xu, W., Gao, B., et al.: Pattern deep region learning for crack detection in thermography diagnosis system. Metals 8(8), 612 (2018)

    Article  Google Scholar 

  12. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

  13. Wang, J., Yu, L., Zhang, W., et al.: Irgan: A minimax game for unifying generative and discriminative information retrieval models. In: Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval., pp. 515–524 (2017)

  14. Liu, K., Li, Y., Yang, J., et al.: Generative principal component thermography for enhanced defect detection and analysis. IEEE Trans. Instrum. Meas. 69(10), 8261–8269 (2020)

    Article  Google Scholar 

  15. Lopez, F., Ibarra-Castanedo, C., de Paulo, N.V., et al.: Optimization of pulsed thermography inspection by partial least-squares regression. NDT E Int. 66, 128–138 (2014)

    Article  Google Scholar 

  16. Liu, B., Zhang, H., Fernandes, H., et al.: Experimental evaluation of pulsed thermography, lock-in thermography and vibrothermography on foreign object defect (FOD) in CFRP. Sensors 16(5), 743 (2016)

    Article  Google Scholar 

  17. Adler, J., Lunz, S.: Banach wasserstein gan. https://arxiv.org/abs/1806.06621 (2018)

  18. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. https://arxiv.org/abs/1511.06434 (2015)

  19. Tavolga, W.N.: Signal/noise ratio and the critical band in fishes. J. Acoust. Soc. Am. 55(6), 1323–1333 (1974)

    Article  Google Scholar 

  20. Nielsen, C., Okoniewski, M.M.: GAN data augmentation through active learning inspired sample acquisition. In: CVPR Workshops, pp. 109–112 (2019)

  21. Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321, 321–331 (2018)

    Article  Google Scholar 

  22. Ibarra-Castanedo, C., Maldague, X.P.V.: Infrared thermography. Handbook of Technical Diagnostics, pp. 175–220. Springer, Berlin, Heidelberg (2013)

    Book  Google Scholar 

  23. C. Ibarra-Castanedo, S. Sfarra, M. Klein, and X. Maldague, “Solar loading thermography: Time-lapsed thermographic survey and advanced thermographic signal processing for the inspection of civil engineering and cultural heritage structures,” Infrared Phys. Technol., vol. 82 pp. 56–74, May. 2017.

  24. Vinzi, V.E., Chin, W.W., Henseler, J., et al.: Handbook of partial least squares[M]. Springer, Berlin (2010)

    Book  Google Scholar 

  25. Larsen, D.R., Speckman, P.L.: Multivariate regression trees for analysis of abundance data. Biometrics 60(2), 543–549 (2004)

    Article  MathSciNet  Google Scholar 

  26. S. Wold, H. Martens and H. Wold, “The Multivariate Calibration Problem in Chemistry Solved by the PLS Method,” in Conference Matrix Pencils, Heidelberg, Germany, 1984.

  27. Martens, H., Naes, T.: Multivariate Calibration, Chichester. John Wiley & Sons, UK (1989)

    MATH  Google Scholar 

  28. Alma, Ö.G.: Comparison of robust regression methods in linear regression. Int. J. Contemp. Math. Sciences 6(9), 409–421 (2011)

    MathSciNet  Google Scholar 

  29. A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” https://arxiv.org/abs/1511.06434. Nov. 2015.

  30. Q. Xuan, Z. Chen, Y. Liu, H. Huang, G. Bao, D. Zhang, “Multiview generative adversarial network and its application in pearl classification,” IEEE Trans. Ind. Electron., vol. 66(10), pp. 8244–8252, Otc. 2019.

  31. A.Antoniou, A. Storkey, and H. Edwards, “Data augmentation generative adversarial networks,” https://arxiv.org/abs/1711.04340. 2017.

  32. Su J. GAN-QP: A novel GAN framework without gradient vanishing and lipschitz constraint. https://arxiv.org/abs/1811.07296, 2018.

  33. Gao, F., Yang, Y., Wang, J., et al.: A deep convolutional generative adversarial network (DCGANs)-based semi-supervised method for object recognition in synthetic aperture radar (SAR) images. Remote Sensing 10(6), 846 (2018)

    Article  Google Scholar 

  34. Wang, Q., Zhou, X., Wang, C., et al.: WGAN-based synthetic minority over-sampling technique: Improving semantic fine-grained classification for lung nodules in CT images. IEEE Access 7, 18450–18463 (2019)

    Article  Google Scholar 

  35. Andoni, A., Indyk, P., Krauthgamer, R.: Earth mover distance over high-dimensional spaces. SODA. 8, 343–352 (2008)

    MathSciNet  MATH  Google Scholar 

  36. Rippl, T., Munk, A., Sturm, A.: Limit laws of the empirical Wasserstein distance: Gaussian distributions. J. Multivar. Anal. 151, 90–109 (2016)

    Article  MathSciNet  Google Scholar 

  37. Ioffe, A.D.: Regular points of Lipschitz functions. Trans. Am. Math. Soc. 251, 61–69 (1979)

    Article  MathSciNet  Google Scholar 

  38. M. Arjovsky, S. Chintala, and L. Bottou, ‘‘Wasserstein generative adversarial networks,’’ in Proc. Int. Conf. Mach. Learn., vol. 70, Aug. 2017, pp. 214–223.

  39. Han S, Sung Y. Dimension-wise importance sampling weight clipping for sample-efficient reinforcement learning. In: International Conference on Machine Learning. PMLR, 2019: 2586–2595.

  40. Lopez F, Nicolau V, Maldague X, et al. Multivariate infrared signal processing by partial least-squares thermography. In: ISEM Conference. 2013.

  41. Wold H. Nonlinear iterative partial least squares (NIPALS) modelling: some current developments. Multivariate analysis–III. Academic Press, 1973: 383–407.

  42. Chai, T., Draxler, R.R.: Root mean square error (RMSE) or mean absolute error (MAE)–Arguments against avoiding RMSE in the literature. Geoscientific model development 7(3), 1247–1250 (2014)

    Article  Google Scholar 

  43. Meng Q, Catchpoole D, Skillicom D, et al. Relational autoencoder for feature extraction. In: 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, 2017: 364–371.

  44. Bojanowski P, Joulin A, Lopez-Paz D, et al. Optimizing the latent space of generative networks. https://arxiv.org/abs/1707.05776, 2017.

  45. Ibarra-Castanedo, C., Piau, J.-M., Guilbert, S., Avdelidis, N.P., Genest, M., Bendada, A., et al.: Comparative study of active thermography techniques for the nondestructive evaluation of honeycomb structures. Res. Nondestruct. Eval. 20(1), 1–31 (2009)

    Article  Google Scholar 

  46. Matworks E. Compute peak signal-to-noise ratio (PSNR) between images.

  47. Chattopadhay A, Sarkar A, Howlader P, et al. Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018: 839–847.

  48. Hsiao, T.Y., Chang, Y.C., Chou, H.H., et al.: Filter-based deep-compression with global average pooling for convolutional networks. J. Syst. Architect. 95, 9–18 (2019)

    Article  Google Scholar 

  49. van der Maaten, L., Hinton, G.E.: Visualizing high-dimensional data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    MATH  Google Scholar 

Download references

Acknowledgements

This research is conducted under the Tier One—Multipolar Infrared Vision Canada Research Chair (MIVIM) in the Department of Electrical and Computer Engineering at Laval University. This research received funding support from NSERC DG program, the Canada Foundation for Innovation and the Canada Research Chair program.

Author information

Authors and Affiliations

  1. Computer Vision and Systems Laboratory, Department of Electrical and Computer Engineering, Université Laval, 1065, av. de la Médecine, Quebec, QC, G1V 0A6, Canada

    Qiang Fang, Clemente Ibarra‐Castanedo & Xavier Maldague

  2. School of Physics and Electronics, Central South University, 932 Lushan South Road, Changsha, 410083, Hunan, China

    Duan Yuxia

  3. Escuela de Ingeniería Eléctrica y Electrónica, Universidad del Valle, Cali, VA, 760032, Colombia

    Jorge Erazo-Aux

  4. Facultad de Ingeniería, Institución Universitaria Antonio José Camacho, Cali, VA, 760046, Colombia

    Jorge Erazo-Aux

  5. GeoTECH Group, CINTECX, Universidade de Vigo, 36310, Vigo, Spain

    Iván Garrido

Authors
  1. Qiang Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Clemente Ibarra‐Castanedo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Duan Yuxia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jorge Erazo-Aux
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Iván Garrido
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xavier Maldague
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, QF, YXD, and XM; methodology, QF, YXD, JE-A; software, QF; validation, QF, and XM; formal analysis YXD; investigation, QF; resources, XM, YXD; data curation, QF, IG, JE-A; writing—original draft preparation, QF; writing—review and editing, QF, CI‐C; visualization, QF, IG; supervision, XM, CI‐C; project administration, XM, CI‐C; funding acquisition, XM All authors have read and agreed to the published version of the manuscript.

Corresponding authors

Correspondence to Qiang Fang or Xavier Maldague.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fang, Q., Ibarra‐Castanedo, C., Yuxia, D. et al. Defect Enhancement and Image Noise Reduction Analysis Using Partial Least Square-Generative Adversarial Networks (PLS-GANs) in Thermographic Nondestructive Evaluation. J Nondestruct Eval 40, 92 (2021). https://doi.org/10.1007/s10921-021-00827-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10921-021-00827-0

Keywords

Search

Navigation