Introduction

The past few decades have witnessed a great deal of research attention devoted to crack detection on industrial components such as the undercarriage of the plane and the bogie of the train. In general, most of the traditional crack detection methods are based on destructive testing which may cause secondary defects. The performance of the destructive testing methods is unsatisfactory especially when detecting the subsurface cracks that could grow in a persistent yet imperceptible way. To effectively detect both the surface and subsurface cracks, the nondestructive testing (NDT) techniques have been successfully applied to a variety of industries such as aerospace, mechanical engineering, and transportation [44, 51].

Recently, a large number of NDT methods have been developed using various testing methods (e.g. the ultrasonic testing, the laser shearography testing and the electromagnetic testing methods). Owing to their fast detection speed and high detection accuracy, electromagnetic testing methods have attracted particular research attention among the aforementioned testing methods. Typical electromagnetic testing methods include the pulsed eddy current array method [49], the pulsed eddy current thermography imaging method [11], and the magnetic optical imaging (MOI) method [7]. Up to now, electromagnetic testing methods have been widely studied and employed in a variety of practical systems [6, 10]. For instance, the pulsed eddy current thermography imaging method has been employed in [6] to detect the surface crack for carbon fiber reinforced plastic material. In [10], the MOI method has been applied to detect both the surface and subsurface cracks in aircraft skins.

In electromagnetic NDT, the magnetic field of the crack area is different from that of the non-crack area. It should be mentioned that the magnetic field of the specimen mainly includes four parts, which are the magnetic field produced by the exciting circuit, the induced magnetic field (produced by the eddy current), the magnetic domain, and the magnetizing field (produced by the rust in the material and some other factors). In this context, the obtained electromagnetic NDT images contain the real cracks and noises. To accurately detect the cracks in the electromagnetic NDT images, a seemingly natural idea is to apply image segmentation methods to segment the cracks [8, 23, 43].

As a powerful image processing technique, image segmentation has been widely utilized in many real-world applications, e.g. crack detection, fault diagnosis, medical imaging, and face recognition [45, 48]. Among the popular image segmentation techniques, the threshold-based image segmentation technique has been widely used owing to its simplicity, fast processing speed and relatively small storage space. It is difficult to segment the cracks using traditional threshold-based image processing methods because of the influence of the noises in the electromagnetic images [5]. In recent years, a large number of machine learning algorithms have been developed and successfully exploited in a variety of research areas such as object detection, path planning, recommender system and image processing [1,2,3, 8, 9, 16, 22, 23, 29,30,31, 40, 43, 53]. Due to their strong abilities in feature extraction, deep learning techniques (which are a powerful family of machine learning algorithms) have been successfully applied to NDT [4]. Although deep learning algorithms have achieved a great success in NDT, the performance of the deep learning algorithms is highly related to the quantity and quality of the training data. To be specific, a large amount of data is required for training an effective deep learning model.

In deep learning, generative adversarial networks (GANs) have received enormous attention from various computer science communities (e.g. computer vision, signal processing, and image processing) because of their strong abilities in data generation [32, 39, 41]. In recent years, GANs have shown competitive performance in crack detection and fault diagnosis, see in [12, 13, 38]. In [13], a deep convolutional GAN has been utilized to automatically detect the cracks in fabrics. An improved least-square GAN has been put forward in [38] for rail crack detection. In [12], the GAN-based method has been developed for detecting the subsurface cracks in the steel material. Notice that most current GAN-based crack detection methods have been applied to detect the cracks on natural optical images.

In the context of electromagnetic crack detection, there appear to be two challenging issues: (1) the obtained electromagnetic image is not natural optical image, and the contrast ratio of the electromagnetic image is low, which makes it difficult to accurately distinguish the real cracks and noises; and (2) the actual position of the crack is hard to be identified. In this paper, it would make practical sense to develop an advanced GAN for crack detection of the electromagnetic images so as to: (1) reduce the influence of the noises in electromagnetic images; and (2) enhance the contrast ratio of the electromagnetic images for image segmentation.

Motivated by the above discussions, in this paper, we endeavor to propose an improved GAN-based crack detection method to handle the electromagnetic NDT crack detection problem. The main contributions of this paper are highlighted as follows: (1) a GAN-based method is put forward for crack detection of the magnetic optical images for the first time; (2) a novel loss function of the GAN is designed to reconstruct the magnetic optical images, where the designed penalty terms are introduced in the loss function to enhance the contrast ratio of the images; and (3) the proposed GAN-based crack detection method is successfully applied to a real-world NDT dataset with promising results.

The rest of this paper is organized as follows. The background of electromagnetic NDT is introduced in “Background of electromagnetic NDT”. In “The improved generative adversarial network”, the proposed GAN-based image reconstruction method is discussed. Experiment results and discussions are presented in “Experiment results”. Finally, conclusions are drawn in “Conclusion”.

Background of electromagnetic NDT

Mechanism of electromagnetic NDT

In electromagnetic NDT, the crack information of the conductive specimen is highly related to the magnetic field induced by the eddy current. The process of the electromagnetic testing approach is shown in Fig. 1. It is known that the path of the eddy current will be changed due to the existing of cracks in the specimen, which leads to the change of the distribution and magnitude of the induced magnetic field “\(B_2\)” as shown in Fig. 1.

Fig. 1
figure 1

The schematic diagram of the electromagnetic testing method

Full size image

It should be pointed out that the magnitude of the magnetic field at the edge of the crack area is bigger than that at other areas. In the electromagnetic NDT images, the area with bigger pixel values can be recognized as the crack area with high probability. The diagram of the magnetic field around the crack area is shown in Fig. 2.

Fig. 2
figure 2

Magnetic field around the crack area

Full size image

In recent years, many electromagnetic-based imaging NDT methods (e.g. the magnetic optical imaging (MOI) method and the eddy current pulsed thermography method) have been proposed due to the rapid development of sensors for detecting the electromagnetic signal. As a popular electromagnetic NDT technique, the eddy current thermography method uses multiple probes to detect the magnetic field, which is shown in Fig. 3.

Fig. 3
figure 3

The eddy current thermography method for detecting the magnetic field

Full size image

Characteristics of the electromagnetic images

The characteristics of electromagnetic NDT images are summarized as follows:

  1. (1)

    The amount of data is small. In the experimental situation, the NDT methods are employed to detect the cracks in the specimens. Although the detection accuracy of the NDT method is satisfactory, the computational cost of the NDT method is high and the cost of the NDT method is expensive. As a result, the number of the obtained electromagnetic images is limited.

  2. (2)

    The shape of the cracks is unpredictable. Generally, the magnetic field of the specimen is affected by various factors (including the magnetic field produced by the exciting circuit and the induced magnetic field). In this context, the detected magnetic field of the specimen is different between two separate NDT tests, which leads to the variation of the crack shape.

  3. (3)

    The existence of noise in electromagnetic images affect the performance of crack detection. In electromagnetic NDT, noises would lead to the so-called “fake crack” areas. It is difficult to distinguish the real crack area and the “fake crack” area because the pixel values of the magnetic field of the real crack and “fake crack” areas are very similar. The influence of the noise in crack detection is shown in Fig. 4.

Fig. 4
figure 4

The influence of the noise in crack detection

Full size image

According to the characteristics of electromagnetic images, it is challenging to employ traditional image segmentation methods to accurately distinguish the crack area and the “fake crack” area. Therefore, it is of practical importance to deploy an advanced image segmentation method. Owing to their strong abilities in feature extraction and data generation, GANs have shown competitive performance in crack detection. Motivated by the above discussions, a novel GAN-based method is developed in this paper to reconstruct the electromagnetic image for image segmentation so as to detect the cracks.

The improved generative adversarial network

In this section, the framework of the improved GAN and the developed GAN-based crack detection approach are presented. The designed loss function of the improved GAN is also introduced.

The proposed GAN-based approach

The original GAN consists of two competing networks: a generator and a discriminator. The generator aims to generate fake samples as real as possible. The purpose of the discriminator is to distinguish the real samples and the generated fake samples. The framework of the original GAN is shown in Fig. 5.

Fig. 5
figure 5

The structure of the GAN

Full size image

In the original GAN, the input of the generator is random noise, and the input of the discriminator includes two parts (which are the output of the generator and the real data). In this paper, the training data of the GAN includes the electromagnetic images (which is also called as the reference images) and the preprocessed images.

The framework of the proposed GAN-based crack detection approach is depicted in Fig. 6. In this paper, each electromagnetic image is divided into a number of images with a smaller size than that of the raw image, which contributes to the improvement of the learning process of the GAN. The output images of the generator are then combined together to reconstruct the electromagnetic image.

Fig. 6
figure 6

The framework of the developed GAN-based crack detection method

Full size image

The modified loss function of the GAN

In the proposed GAN-based crack detection method, we aim to apply the improved GAN to (1) distinguish the crack area and “fake crack” area; and (2) reduce the influence of noises. To achieve the aforementioned two objectives, a new loss function is put forward where two designed penalty terms are added in the loss function to enhance the contrast ratio of the generated image.

The relationship between the training image and the generated output image is shown as follows:

$$\begin{aligned} X=G(I) \end{aligned}$$
(1)

where \(G(\cdot )\) stands for the generator, I denotes the training image, and X represents the output image of the generator.

The training process of a GAN can be treated as a minimax two-player game. The loss function of the original GAN for image processing is shown as follows:

$$\begin{aligned} L_{a} ={\mathbb {E}}_{S \sim {\mathbb {P}}_{t}}[\log D(S)] +{\mathbb {E}}_{I \sim {\mathbb {P}}_{c}}[\log (1-D(G(I)))] \end{aligned}$$
(2)

where \({\mathbb {P}}_{c}\) is the pixel value distribution of the training image; \({\mathbb {P}}_{t}\) is the pixel value distribution of the real image; S is the input of the discriminator; \(G(\cdot )\) represents the generator and \(D(\cdot )\) is the discriminator. The pixel value of the output image obtained by the generator is in the range of (0, 1).

To enhance the contrast ratio of the reconstructed electromagnetic images, two designed penalty terms are introduced in the loss function of the GAN. By employing the penalty terms, the noise in the raw electromagnetic images is removed, which contributes to the crack detection. The loss function of the improved GAN is shown as follows:

$$\begin{aligned} \begin{aligned} L_{i} = L_{a}+{\mathbb {E}}_{I \sim {\mathbb {P}}_{c}}\left[ \lambda _1\Vert {\mathbf {1}}-I\Vert _F\Vert G(I)\Vert _F +\lambda _2\frac{\Vert I\Vert _F}{\Vert G(I)\Vert _F}\right] \end{aligned}\nonumber \\ \end{aligned}$$
(3)

where \(\lambda _1\) and \(\lambda _2\) are penalty factors; \(\Vert \cdot \Vert _F\) is the Frobenius norm defined by \(\Vert A\Vert _F=\sqrt{\sum _{i,j}A^{2}_{i,j}}\); \({\mathbb {E}}_{I \sim {\mathbb {P}}_{c}}[\Vert {\mathbf {1}}-I\Vert _F\Vert G(I)\Vert _F]\) is utilized to shrink the points with small pixel values; and

$$\begin{aligned} {\mathbb {E}}_{I \sim {\mathbb {P}}_{c}}\left[ \frac{\Vert I\Vert _F}{\Vert G(I)\Vert _F}\right] \end{aligned}$$

is designed to enlarge the points with large pixel values.

The purpose of the improved GAN is to solve the following minimax problem

$$\begin{aligned} \min _{G} \max _{D}L_{i} \end{aligned}$$

where the training process of such improved GAN is described in Fig. 7. It should be pointed out that the generator is trained once at each iteration in this work.

Fig. 7
figure 7

The flowchart of the algorithm G: generator of GAN; D: discriminator of GAN

Full size image

Experiment results

In this paper, the proposed GAN-based method is pre-trained to deal with the magnetic optical images which are captured through the MOI test. In this section, the hardware of the MOI test is discussed, and data preprocessing is presented. In addition, experiment results are presented to demonstrate the superiority of the proposed GAN-based crack detection method.

Experiment setup

Served as a popular NDT technique, the MOI detection mechanism includes an optical system and an excitation system. The hardware of the MOI detection mechanism is depicted in Fig. 8.

Fig. 8
figure 8

Hardware of the MOI detection mechanism

Full size image

In the MOI test, the crack information is influenced by the magnetic field. The optical system is utilized to detect the magnetic field that is distributed on the surface of the specimen. The polarizing films, i.e., the polarizer and the analyzer, are the most important components of the optical system. The polarizer is used to produce the polarized laser light, and the analyzer is utilized to filter the polarized light (which is rotated after interacting with the magnetic field). The charge-coupled device camera is adopted to record the polarized light which contains the magnetic field information (especially the magnetic field induced by the crack). In our experiment, the specimens include two ferromagnetic thin plates. The ferromagnetic thin plate with the “Z” shape crack is utilized for the training process, and the other specimen with a scratch crack (1 mm width and 0.2 mm depth) is used for the testing process. The specimens are shown in Fig. 9.

Fig. 9
figure 9

The specimens used in the experiment

Full size image

Data preprocessing

As a challenge in deep learning, a large amount of data is required for training satisfactory deep neural networks. Unfortunately, it is difficult to produce a large number of electromagnetic images for crack detection due to the high cost of the electromagnetic NDT technique. In this paper, data preprocessing is employed to produce effective training data.

The procedure of data preprocessing is given as below:

  1. (1)

    Initialization: Select the reference images which contains crack, fake crack and background areas. Then, normalize the selected images (with the size of M rows and N columns).

  2. (2)

    Vectorization: Convert each normalized image into a separate one column vector.

  3. (3)

    Training image selection: Randomly select \(m\cdot n\) (\(m< M, n < N\)) elements from an arbitrary vector obtained in (2). Then, convert the \(m\cdot n\) elements to a m by n matrix as a training image.

  4. (4)

    Repeat (3) till a number of training images are obtained.

The process of the training image selection is shown in Fig. 10.

Fig. 10
figure 10

The process of producing training images for GANs

Full size image

The purpose of the training image selection is to choose effective electromagnetic images that include the crack area, the non-crack area and the “fake crack” area. Moreover, the size of the training images is reduced by comparing with that of the electromagnetic images, which improves the efficiency of the training process. Notice that the selected electromagnetic images need to be binarized so as to obtain the ground truth information. The preprocessed images are also called as the real images in this paper for training the GANs. Particularly, the electromagnetic images are also preprocessed according to experiment experience and details can be seen in [7].

To sum up, a pair of training images of the GAN consist of one reference image and the corresponding real image. Then, the training images are cropped into images with a smaller size comparing with that of the raw images. As shown in Fig. 11, an MOI image (which is also known as a reference image) is obtained using the electromagnetic NDT technique. The MOI image is then binarized to obtain the real image, where the pixel value of the crack area is 1 and the pixel value of other area is 0.

Fig. 11
figure 11

The method of sampling training data

Full size image

In the testing process, the testing images are preprocessed into the same size as the training images. To be specific, the testing images are divided into sub-images to feed the generator. The output images of the generator are then spliced together so as to obtain the reconstructed images. The strategy of the testing process is depicted in Fig. 12.

Fig. 12
figure 12

The testing process

Full size image

The pre-training technique is widely used in deep learning to enhance the generalization ability of the deep neural networks [15]. To improve the feature learning and generalization abilities of the GAN, an auto-encoder (AE) is employed to pre-train the generator. In addition, the pre-training process contributes to a faster convergence for the GAN training process than the standard training process.

Results and analysis

The magnetic optical images are preprocessed by extracting the circle area that is only covered by the laser. The training data of the discriminator is displayed in Fig. 13.

Fig. 13
figure 13

The training data of the discriminator G: generator of GAN; D: discriminator of GAN

Full size image

The size of the raw magnetic optical image is \(1761\times 1833\). In this paper, 60,000 training images are generated with the size of \(7\times 7\) based on the aforementioned preprocessing method. The labels of the pixels in the training images are obtained according to [7]. Each training image and its corresponding real image form a pair of training data. In the improved GAN, the “ReLU” function is chosen as the activation function of the hidden layers, and the “Sigmoid” function is utilized as the activation function of the output layers. The batch size is set to be 100. The hyperparameters \(\lambda _1\) and \(\lambda _2\) are set up to be 0.8 and 0.2 according to experiment experience.

The testing data is the magnetic optical image of the specimen with the scratch crack. The testing image is displayed in Fig. 14. In Fig. 14, the sub-figures (a), (b), and (c) represent the magnetic optical image, the normalized image, and the image processed using the “Sigmoid” function with (b) as the input, respectively. The sub-figures (d), (e), and (f) denote the pixel value distributions of (a), (b), and (c), respectively. It should be pointed out that the crack area also include noise, which is called as the “fuzzy area” as shown in (d). In the “fuzzy area”, there are a large number of points with high pixel values even when the image is processed using different image processing methods and their results are shown in (e) and (f). As such, it is difficult to choose a suitable threshold to segment the magnetic optical images for crack detection. Furthermore, the tail of pixel distribution in images (d), (e), and (f) is high and sharp, which leads to a big value as the threshold. Therefore, the performance of crack detection using such a big value as a threshold is poor.

Fig. 14
figure 14

a Magnetic optical image; b normalized a; c processed b with the “Sigmoid” function; d pixel distribution of a; e pixel distribution of b; f pixel distribution of c

Full size image

To eliminate the noise and identify the crack area accurately, the improved GAN-based method is developed to reconstruct the magnetic optical images. The designed loss function enhances the contrast ratio of the magnetic optical images in the training process. In Fig. 15, the reconstructed images are displayed. In Fig. 15a, b are output images of the generator trained using the loss function (2), and image (c) represents the output image of the generator trained using the designed loss function (3). In Fig. 15d–f are the binary images (using the threshold of 0.6) of (a), (b), and (c), respectively.

It can be seen in Fig. 15 that (a) and (b) are generated using two GANs (with different hidden layer numbers) trained by using the same loss function (2). Figure 15c is generated using the proposed GAN with the designed loss function. It can be seen that noise exists in both the reconstructed images and the binary images, which affects the crack detection. In (c), the noise is largely reduced using the proposed improved GAN. The contrast ratio of the reconstructed image (c) is high, which indicates that the proposed GAN-based method performs better than the traditional GAN-based methods.

In Fig. 16, the pixel distribution of reconstructed images (a), (b), and (c) in Fig. 15 are displayed. Comparing the curves of “result 1” and “result 2” in Fig. 16 with that of (d), (e), and (f) in Fig. 14, the pixel distribution becomes smooth, especially in the tail of the curves. In addition, the changing rates of the curves of “result 1” and “result 2” are big in the middle area, and small in the end area, which indicates that the pixels are easy to be classified.

Fig. 15
figure 15

a Result 1 using loss function (2); b result 2 using loss function (2); c result using loss function (3); d the binarization of image a using the threshold 0.6; e the binarization of image b using the threshold 0.6; f the binarization of image c using the threshold 0.6

Full size image
Fig. 16
figure 16

The pixel distributions of reconstructed image

Full size image

As shown in the curves “result 1” and “result 2”, the number of pixels in the uncertain area is still huge so that the dynamic range of the threshold-selection is big which may lead to the low crack identification accuracy. Using the loss function (3) to train the GAN, the distribution of the pixel value is sharp in the uncertain area and smooth in the end. In this case, there are very few pixels in the uncertain area which makes it easier to find an appropriate threshold, as shown in Fig. 16. To summarize, the GAN trained using the proposed loss function contributes to a better image segmentation than that of the original loss function.

To comprehensively evaluate the performance of the improved GAN-based crack detection method, a single pixel point is utilized as a training image. The corresponding experiment results are shown in Fig. 17.

Fig. 17
figure 17

The result using \(1\times 1\) sampled image

Full size image

It can be seen in Fig. 17 that the reconstructed image (a) and its binary image (b) show that the noise is reduced and the performance of the image segmentation is satisfactory. Figure 17c presents that the pixels are classified appropriately because most of the pixel value of the points approach to 0 (as shown by the black area) or 1 (as shown by the white area of the image). The distribution curve of image (a) is almost vertical in the uncertain area by comparing with that of binary image (b), which indicates that the performance of the reconstructing process is satisfactory.

According to previous research on the magnetic optical images [36, 37], it is simple to extract the crack area depending on the reconstructed image as shown in Fig. 18. In particular, the shape/contour of the crack is identified, and the location of the crack in the specimen is detected, which contributes to the fault diagnosis and maintenance of equipment.

Fig. 18
figure 18

Crack extraction using the reconstructed image

Full size image

Conclusion

In electromagnetic NDT, crack detection depends on the features of the imaging mechanism instead of the geometric features (such as the shape and length of the crack). In this paper, an improved GAN-based crack detection method has been proposed for electromagnetic NDT by analyzing the magnetic optical images. Data preprocessing has been employed to improve the generalization ability of the improved GAN, which overcomes the problem of lacking training images. Two designed penalty terms have been introduced in the loss function to enhance the contrast ratio of the reconstructed images and reduce the noises in the MOI images. Experiment results have shown that the improved GAN outperforms the standard GAN for reconstructing the magnetic optical images, and the developed GAN-based crack detection method has demonstrated satisfactory performance in detecting the cracks in magnetic optical images. In the future, we aim to (1) employ our proposed GAN-based crack detection for other NDT methods such as thermal imagery detection, ultrasound detection and X-ray imaging detection; (2) deploy evolutionary computation methods to optimize the parameter selection in the GAN-based crack detection system [24,25,26,27]; (3) employ signal processing and state estimation methods to investigate the stability of the NDT process [17, 19,20,21, 28, 34, 35, 42, 46, 54]; and (4) design an advanced control strategy to improve the detection accuracy and detection rate of the GAN-based crack detection system [14, 18, 33, 47, 50, 52, 55, 56].