1 Introduction

Temperature is one of the basic parameters defining the technical state of an object. In numerous applications, temperature is used as a primary measure for assessing the condition of a test object. Significant temperature deviations from an expected baseline value indicate abnormalities in the tested system. The abnormal temperature may indicate the disease of a patient (medicine), machine failure (mechanical engineering), faulty process parameters (chemical industry), short circuits (electronics and electric power systems), and many more [1,2,3]. Frequently, the temperature is monitored using sensors located only in critical locations like engines, bearings, controllers, and others. This approach can be very effective and economical as it requires only a limited number of sensors. However, a significant amount of a priori knowledge about the monitored system is necessary to apply it correctly. In many other applications, however, a single-point measurement cannot provide sufficient diagnostic information. Applications like thermal inspection of buildings or nondestructive testing (NDT) of materials need to rely on full-field rather than point information. This can be achieved with the use of infrared (IR) cameras which detect the infrared band of the electromagnetic radiation emitted by an object. Recorded radiation is then translated to the objects’ temperature through the appropriate calibration curves. Modern IR cameras allow for precise temperature measurements with thermal sensitivities down to several mK and high spatial resolutions up to over a million pixels (e.g., ~ 1.3 M for 1280 × 1024 commercially available detectors). The frame rates of modern IR cameras vary from less than one frame per second (< 1 Hz) up to over a thousand frames per second (> 1 kHz) at full resolution. However, the most significant practical advantage of infrared thermography (IRT) is its fully non-contact nature, as the recorded electromagnetic radiation propagates without any supporting medium. This can be important for all applications where direct access to the object is limited, or a sterile environment is desirable, e.g., manufacturing of medical and electronic products or food processing. Infrared thermography (IRT) does not require attaching sensors and guiding cables, which simplifies the test setup and shortens the inspection time. Therefore, infrared thermography is increasingly used in many engineering applications, among other fields.

2 Laser Spot Thermography (LST)

Infrared thermography (IRT) is one of the nondestructive testing (NDT) approaches. There are several different IRT techniques, which can be classified into passive and active testing modalities [2, 3]. Active approaches, predominant in NDT applications, require energy delivery to a measured object from an external source. This group includes pulsed thermography [4,5,6,7], pulsed phase thermography [7], lock-in thermography [8,9,10], eddy current thermography [11], and vibrothermography [12,13,14,15,16,17]. One of the emerging IRT techniques, which has a large diagnostic potential, is laser spot thermography (LST) [18, 19]. The technique utilizes a coherent laser light source to thermally excite the test object while an IR camera registers the resultant temperature field evolution. Identification of defects is based on the interpretation of the dynamic changes in an object’s thermal signature due to laser stimulation [20,21,22]. Figure 1 shows a schematic view of laser spot thermography. The measurement system comprises an infrared camera (1), a pulsed laser source (2), and laser optics (3) for delivering the laser light to the surface of a test object (4). The measurement is orchestrated from a software framework implemented on a personal computer (5) through a hardware controller (6), allowing for precise synchronization between the laser source and the IR camera. The measurement procedure includes sequential excitation of the consecutive points on the inspected structure (Point_1 to Point_n in Fig. 1) and collecting thermograms of temperature evolution in these locations. To reach the consecutive measurement points, either the laser optics and the IR camera can be translated relative to the test sample, or a test sample can be moved in space through a positioning system. Typically a single inspection involves several to thousands of measurement points. Each thermographic sequence can be evaluated individually or collectively after merging all sequences.

Fig. 1
figure 1

Laser spot thermography (LST)—test setup and measurement procedure [16]

Full size image

Laser spot thermography has several advantages over the other existing thermographic techniques. Firstly, the heat is induced only locally by the laser beam and propagates omnidirectionally from the source. This allows for detecting defects of different orientations, including those perpendiculars to the surface (e.g., cracks), which are difficult to detect by other IRT modalities such as pulsed thermography. Secondly, the laser source parameters can be precisely controlled, with accuracy down to single mW for pulse power and ms for pulse duration, offering benefits not achievable by traditional light sources such as halogen or flash lamps. In addition, the laser source has negligible thermal inertia allowing complex spatio-temporal shaping of excitation pulses. Thanks to that, both the heating and the cooling phases can be precisely distinguished, which simplifies the postprocessing stage. The shape of the laser spot can be changed from a point to a line in order to speed up the measurement for larger scanning areas [21, 23]. Scanning can also be performed with a continuous movement of the laser spot over the sample. This technique, called flying laser spot thermography, is faster than LST and has the potential for further development [24]. The drawbacks of LST include more complex and expensive hardware necessary to perform testing as well as huge amounts of data generated during a test. The datasets produced in LST are much higher than for other TNDT modalities, which poses challenges for the transfer, storage, and interpretation of data for damage detection.

This work aims to streamline the LST testing process by proposing a signal processing scheme based on data parameterization and machine learning. The added value is the automated detection of IR sequences deviating from the baseline, which allows for detecting structural defects while reducing the number of datasets needing manual assessment by a human expert.

3 Problem Definition

The laser spot thermography involves recording multiple thermographic sequences, resulting in large amounts of data that have to be processed cumulatively to evaluate the diagnostic information. This is a demanding task that needs significant computational resources to process the measurement data and expert diagnostic knowledge to interpret the results. Therefore, in this paper, we intend to support this process with the use of data parameterization and machine learning (ML). ML techniques are able to learn and adapt without following explicit instructions by using algorithms and statistical models to analyze and draw inferences from patterns in data. Successful applications of machine learning have been demonstrated in many fields, including image and speech recognition, medical diagnostics, financial services, and others [25]. Therefore, it is not surprising that machine learning techniques also gain interest in the context of thermographic nondestructive testing (TNDT) methods [25,26,27].

A machine learning model has to be trained on sample datasets (training data) to reveal patterns and dependencies in the data. A well-trained model can then be used on new datasets to make predictions. As in most data processing techniques, the effectiveness of machine learning is ultimately limited by the quality of the provided data. Usually, raw data holds too much unimportant information, which should be excluded or transformed. It is therefore important to perform careful feature selection and feed the model with only the data relevant to the analyzed phenomena. Unfortunately, in laser spot thermography, the use of machine learning techniques is significantly limited. The limitations come from the specificity of the test procedure, which involves acquiring a large number of individual measurements carried out point by point until the whole area of interest is covered. All collected sequences are then merged and used for damage detection. The procedure is shown schematically in Fig. 2. The example illustrates a test procedure for a small test sample with only five excitation points (P1-P5). Thermographic sequences for all excitation locations had to be collected, stored, and processed in order to identify the crack present in the component. An individual sequence was 1 s long with a spatial resolution of 640 × 512 pixels and a 20 Hz frame rate, which resulted in approximately 144 MB of raw data. Therefore the input dataset merged from five measurement points was approximately 0.7 GB. This example illustrates the problem of large dataset sizes for this approach. The same procedure for larger areas of interest could easily result in several or more gigabytes of raw data that would need to be processed for damage detection.

Fig. 2
figure 2

Laser spot thermography—sequence merging and damage detection [28]

Full size image

The effectiveness of machine learning-based thermography is heavily influenced by the size and the information content of the raw input data, as in other ML applications. Therefore, some form of data reduction is highly desirable. One possibility to reduce the size and dimensionality of the input data (preserving the physics of heat transfer) is the Thermographic Signal Reconstruction (TSR) method [29]. This method, patented almost two decades ago, was intended for pulse thermography. The method consists of fitting a polynomial to the temperature curve obtained for every pixel in the input sequence. For a log–log curve, the polynomial has the following form:

$$ log_{10} \left( {\Delta T} \right) = a_{0} + a_{1} log_{10} \left( t \right) + a_{2} \left[ {log_{10} \left( t \right)]^{2} + ... + a_{n} } \right[log_{10} \left( t \right)]^{2} $$
(1)

where \(\Delta T\) represents the temperature increase as a function of time \(t\) (thermogram) for each pixel \(\left( {i,j} \right)\), and the fitting allows reducing an IR sequence to a set of polynomial coefficients. As a result, each pixel is described by only a small set of parameters. The number of these parameters depends on the degree of the polynomial in Eq. (1).

In the TSR, the polynomials are fitted independently for every pixel of an IR sequence. This assumption is valid for pulsed thermography, where one-dimensional (normal to the surface) heat transfer can be assumed. However, the three-dimensional heat transfer should be assumed for the LST due to the localized heat source.

4 Proposed Methodology

The proposed LST data processing scheme is shown in Fig. 3 and represents a typical ML workflow consisting of the development phase, where the ML model is created, and the operation mode, where the ML model is utilized. The development phase consists of the following steps: Data collection (1), Feature selection (2), Training dataset (3), then iteratively: Model building (4) Model training (5) and Model evaluation (6). The Development stage ends when a built model passes the evaluation stage. The operation stage consists of Data collection (1), Feature extraction (7), ML Model (8), Classification (9) and finally Assessment (9).

Fig. 3
figure 3

Machine learning workflow

Full size image

In the development phase, data processing starts with feature selection in step 2. At this stage, the collected raw data is transformed into a format, which can be efficiently interpreted with a machine learning model. It can include converting data (e.g., from categorical to numerical types), ignoring the missing values, filling the missing values, outliers detection, noise reduction, etc.

Thermographic sequences captured by an IR camera are three-dimensional datasets consisting of multiple time-related image frames. Machine learning techniques applied directly to such datasets would have to face all of the image recognition challenges. Hence, successful practical applications of ML-supported IRT are very limited. Therefore, we propose an approach where a parametrized three-dimensional infrared sequence forms an input to ML. It is important to point out that parametrization is physics-based, rather than machine-learning-based, in order to preserve the information about the underlying physical phenomena. The proposed concept is shown schematically in Fig. 4.

Fig. 4
figure 4

Data reduction of an individual IR sequence: physics-based parametrization of the experimental data [30]

Full size image

As indicated previously, the measurement procedure in laser spot thermography involves multiple individual measurements captured on a predefined grid of points on the structure. The scanning is obtained by physically moving the test sample with respect to the laser head and the IR camera. Each individual measurement is then compressed to a set of physics-based parameters for each IR sequence, much better fitted for the subsequent damage identification and classification. The individual measurements are then stitched together to obtain a 2D scan of the test sample. The positioning system provides spatial coordinates that relate each acquired IR sequence with the actual geometry of the test structure needed for stitching.

The parametrized 2D scan is then fed into the ML model for classification and assessment. The data processing workflow is shown in Fig. 5. An additional benefit of the physics-based parametrization is that the final dataset has a much smaller data footprint in the order of several megabytes rather than gigabytes or more for the raw dataset. The reduced dataset can be easily transferred to the cloud, where the ML model can be run in the high-performance computing (HPC) environment more efficiently. This can be a more economically justified solution for users who do not want to maintain their own IT/R&D infrastructure.

Fig. 5
figure 5

Physics-based parametrization of individual IR sequences (left) is followed by stitching of the 2D scan, which is fed into the ML model for classification and assessment (right)

Full size image

Both supervised and unsupervised classification can be utilized for the LST data. The supervised classification can detect defects, given that they were represented in the training dataset. The unsupervised classification, on the other hand, can be used as a novelty detector. In both cases, the goal is to automatically detect the IR sequence deviating from the baseline and reduce the number of cases needing manual assessment by a human expert. In this work, we have used k-means clustering that aims to partition n observations into k clusters in which each observation belongs to exactly one cluster defined by centroid. The number of clusters k has to be chosen before the algorithm starts [31].

5 Experimental Validation

The proposed concept of LST data processing was validated experimentally on a test system shown schematically in Fig. 6. The system has a modular structure, consisting of the main module with user interface (1), the control module synchronizing excitation and acquisition (2), the laser source (3), the positioning system (4), the IR camera (5), the data storage (6), the data reduction module (7) and the ML model and postprocessing module (8).

Fig. 6
figure 6

The main components of the modular LST test system

Full size image

The main module directly or indirectly controls all of the other system modules and provides the graphical user interface (user panel). This module in the presented system is based on the LabVIEW (National Instruments) and the ResearchIR (FLIR Systems) software working on Windows 10 OS. The main module also controls a positioner module via USB. The control module is based on Compact DAQ hardware (National Instruments), which provides digital and analog signals. The module provides synchronization between the laser controller and the IR camera. The positioning module consists of two motorized linear stages (Thorlabs) assembled in a 2D Cartesian setup allowing for scanning of flat samples. The laser module consists of a 100 W power water-cooled CW laser source (Limo, Thermotec) and optical fiber delivery of the laser beam to the optical head used for beam shaping. A 10 mm laser spot diameter is used in the tests. The position of the laser optics is fixed with respect to the IR camera; hence laser spot always remains in the same position in the IR image. The IR module consists of a photon detector IR camera (FLIR) and a frame grabber (Silicon Software). The resolution of the IR camera is 640 × 512 pixels. The parametrization and postprocessing modules are implemented in the MATLAB environment and can be executed offline when the scanning procedure is complete. Data storage is done in the cloud. The photograph of the test rig is shown in Fig. 7.

Fig. 7
figure 7

Photograph of the experimental test rig. The main components seen in the image are the laser source (1), the IR camera (2), the controller used for measurement synchronization (3), and the 2D positioning system (4)

Full size image

The test sample under investigation was a laminated carbon fiber reinforced polymer (CFRP) plate made of 985-GF-3070PW prepreg material. The dimensions of the sample were 450 × 500 × 2.4 mm. The ply stacking sequence was [0°/90°/0°/90°/0°]s, with each ply having a thickness of approximately 0.24 mm. The sample contained a set of artificial defects in the form of Teflon inserts that were introduced into the sample during the manufacturing process. The inserts of different diameters were placed at two different depths in the sample, mimicking the delamination type defects. Figures 8 and 9 show the schematic view of the tested sample and its real picture as mounted on the test rig, respectively.

Fig. 8
figure 8

Tested sample—drafting

Full size image
Fig. 9
figure 9

Tested sample—photograph

Full size image

The travel range of the motorized stages offered a maximum scan area of 200 × 160 mm. A grid of 40 × 30 measurement points with a uniform 5 mm spacing was defined within this allowable area to collect the measurements. The scan area was, therefore, equal to 195 × 145 mm, as shown in Fig. 10. The scan area covers four defects of diameters 30 mm and 20 mm placed between layers 1 and 2 (left column) and layers 3 and 4 (right column). The measurements were taken from the first ply side of the laminate, and the IR camera was positioned perpendicular to the surface in order to reduce the perspective distortion.

Fig. 10
figure 10

Tested sample with a highlighted scan area of size 200 × 160 mm

Full size image

The measurements for all excitation locations were performed using the same parameters. The IR camera frame rate was set to 20 Hz. The laser pulse duration was 800 ms and delivered the power of 97 J per pulse. For each measurement point, a total of 244 frames were recorded, which resulted in 12.2 s sequence duration. The IR camera field of view was equal to 30 × 24 mm, and the laser spot of 10 mm diameter was positioned in the center of the field of view. The full scan consisted of 1280 thermographic sequences with a total size of 199 388 MB of raw data.

The speed of the sequential scanning process in LST is hindered by a low thermal diffusivity (thermal conductivity divided by density and specific heat capacity) of CFRP, causing slow cooling down of the optically excited areas. If a complete cooldown between the consecutive measurement locations cannot be provided, increasing background temperature has to be expected, as shown in Fig. 11. This can affect data postprocessing and complicate the diagnostic procedure.

Fig. 11
figure 11

Sequential scanning procedure—non-optimal manner

Full size image

To overcome this problem, we have applied a modified scanning scenario, as shown in Fig. 12. The measurements are taken every 5 points on the grid, rather than at every grid point sequentially, in order to avoid thermal influences from the nearest neighbors. In each measurement loop, the points are then shifted by one to cover the whole grid in an automated process. This way, each measurement point is thermally independent, and the scanning procedure is more time-efficient.

Fig. 12
figure 12

Modified scanning procedure (same grid size as in Fig. 11)

Full size image

6 Results

Figure 13 shows an exemplary thermographic image acquired on the test sample with the highlighted region of interest (ROI) taken for further analyses. The standard background subtraction method was performed for each sequence. Figure 14 shows the temperature evolution curve obtained as an average of the pixels in the ROI. Three phases can be identified in the time–temperature plot: (1) heating (red background), (2) rapid cooling (bright green background), and (3) slow cooling (light green background). The heating phase coincides with the energy delivery by the laser pulse. The rapid cooling phase starts at the end of laser exciation when energy is dissipated into the volume of the material and to the surroundings. The heat transfer, according to Fourier’s law, is exponential and slows down with time. It is known that the rapid cooling phase contains the most valuable information on the damage detection process. Thus only the data from this phase were chosen for further processing. In our experiments, the rapid cooling phase was defined between the 20th and 100th frame of the acquired sequence, which corresponds to 4 s in time.

Fig. 13
figure 13

Exemplary frame with the highlighted region of interest (ROI) of the size of 5 × 5 mm

Full size image
Fig. 14
figure 14

Temperature—time curve with distinguished three phases: heating (red), rapid cooling (bright green) and slow cooling (light green) (Color figure online)

Full size image

The complete 2D scan of the test sample consists of 1280 individual IR sequences. For each of them, the same processing procedure was applied starting with the selection of pixels in the ROI, as shown in Fig. 13. Then, for each frame in the IR sequence, the average of pixel values in the ROI was calculated to obtain the representative time–temperature curve. The output data consists of 1280 time–temperature curves, with spatial coordinates assigned to them from the XY positioning system (corresponds to ‘2D scan’). Next, the curve fitting algorithm was applied to each temperature curve, according to the exponential form in Eq. (2), in order to obtain a set of four parameters (a, b, c, d).

$$ f\left( x \right) = a \cdot e^{b \cdot x} + c \cdot e^{d \cdot x} $$
(2)

The use of Eq. (2) preserves the physical nature of temperature decay and is a good representation of laser spot thermography testing. The output dataset at this stage consists of 1280 sets of parameters a, b, c, d, and two-dimensional spatial coordinates. The raw dataset of 199 388 megabytes was therefore reduced to only several kilobytes. Figure 15 shows the result of curve fitting, according to Eq. (2), performed for one of the acquired time–temperature curves. The upper plot in Fig. 15 shows the measured data (blue dots) and the fitted curve (red line), while the lower plot shows the fitting residuals (blue dots) and the zero line (red line), which together show the fitting error. In order to put higher weight on the more informative initial phase of cooling than on the later phases, a varying number of time steps was applied. For frames in the range 21–30 all-time steps were included, for frames in the range 31–50 every second time step was included, and for frames 51–100 every fourth frame was included. The same could have also been achieved using a weighted regression model.

Fig. 15
figure 15

Curve fitting for exemplary measurement point. Top: actual data (blue dots) and fitted curve (red line). Bottom: fitting residuals (blue dots) and zero line (red line) (Color figure online)

Full size image

In the next step, all measurement locations were manually labeled using three different labels: (1) ‘healthy’ for the measurements taken over the undamaged sections of the plate, (2) ‘damaged’ for the measurements taken over the delaminated areas, and (3) ‘partially damaged’ for the measurements taken on the border between the former two. It could be done considering the known layout of the test sample. For simplicity, all data labeled as partially damaged was excluded from further analyses.

Independently of the manual labeling, an unsupervised classification was performed on the measured data. The k-means algorithm was used to classify the measurement points into three groups. The 0–2 labels (A defect, B defect, Healthy) were assigned to every measurement point due to the classification. In the end, those labels were compared with the manually assigned labels described in the previous paragraph. Figure 16 shows the confusion matrix calculated for performed classification. It has to be mentioned that the used classifier was non-deterministic; thus, slight differences in the final results had to be expected. The confusion matrix shows that the healthy regions were correctly classified in 99% with 1% false positives. The effectiveness of detecting defects (class A and B cumulatively) was 94.5%, with 5.5% of false negatives. Figure 17 shows the spatial distribution of labels obtained from the classification. The blue and black crosses indicate successfully detected defects of classes A and B, respectively. The red crosses indicate non-detected defects, while the red circles indicate a healthy area wrongly classified as damaged. The green circles indicate the correctly classified healthy area of the sample. As can be seen, the obtained map corresponds well with the layout of the test sample shown in Fig. 10.

Fig. 16
figure 16

Confusion matrix for unsupervised classification based on a k-mean algorithm

Full size image
Fig. 17
figure 17

Classification results—map of labels

Full size image

Figures 18, 19, 20, 21, 22 and 23 show six projections of the four-dimensional parameter space containing all measurement points with the assigned classification labels. The labels correspond with the labels used in Fig. 17. Four separate groups of measurement points can be easily distinguished. The A and B class defects are separable from points corresponding to the measurements performed for a healthy area. The class A defect corresponds to the delaminations at the 1/2 ply interface, and the class B defect corresponds to the 3/4 ply interface. Several outliers are also present in the data.

Fig. 18
figure 18

Parameter space: ‘b’ to ‘a’

Full size image
Fig. 19
figure 19

Parameter space: ‘c’ to ‘a’

Full size image
Fig. 20
figure 20

Parameter space: ‘d’ to ‘a’

Full size image
Fig. 21
figure 21

Parameter space: ‘c’ to ‘b’

Full size image
Fig. 22
figure 22

Parameter space: ‘d’ to ‘b’

Full size image
Fig. 23
figure 23

Parameter space: ‘d’ to ‘c’

Full size image

7 Summary and Conclusions

In this paper, we demonstrated that it is feasible to detect multiple damages in a laminated composite plate with the use of laser spot thermography (LST) and the proposed signal processing and inference algorithm. The data processing workflow encompasses the reduction of raw thermal sequences through physics-based parametrization and classification. The regression model is used to parametrize the measurement data similarly to the thermographic signal reconstruction (TSR). The approach was, however, modified and adapted for the specificity of the laser spot thermography technique. The second stage of the proposed procedure involves data classification using the k-means classification algorithm. The feasibility study was performed on a dataset acquired with the use of the in-house developed LST test rig. The test rig allows for automated testing of flat samples with the use of a two-axial linear positioning system and a dedicated data management system. The proposed signal processing and inference framework, in its current form, can also be applied to the input data from the classical TNDT modalities such as pulse thermography.

The effectiveness of the proposed approach was demonstrated on a test sample made of a carbon fiber-reinforced composite laminate with multiple simulated delaminations of different sizes and depths. The obtained results show that the healthy regions were correctly classified in 99% of cases, with 1% false positives. The overall effectiveness of detecting defects was 94.5%, with 5.5% false negatives. Performed research shows that the proposed data processing scheme is effective and has the potential for practical applications of LST for damage detection.

Further work is necessary to investigate other classification algorithms on a greater number of classes, develop numerical models capable of delivering training data for initial testing as well as to optimize the scanning strategies to reduce the measurement time and the amount of data. Also, currently, the spatial coordinates are not used by the machine learning algorithm. They are used by the stitching procedure to create 2D scans, such as the one shown in Fig. 17, which can be easily interpreted by humans. The authors see the potential in using spatial coordinates as additional parameters to the machine learning algorithm in order to improve the diagnostic process. For example, the threshold for the minimum area, which could be classified as damaged, could be introduced, and the classification of the uncertain points could be improved by considering their proximity to the damaged/healthy neighbors etc. Further work is planned in this area to tackle these issues.