Introduction

Cold spray processing is employed in various industrial sectors with wide ranging applications possible for coating, repair and additive manufacturing. Developed in the early 1980s, the advancement of the technology has been rapid as evident in the increasing number of publications being produced per year (Ref 1). In addition to this general knowledge base, the development of cold spray for industrial applications also requires significant effort to identify, in practice, the manner in which application-specific considerations can affect processing parameters and ultimately deposit properties. In this respect, the use of process–microstructure–property relationships for cold spray can significantly reduce application development cost and time compared to legacy trial and error strategies.

Investigations have been reported relating microstructural features, such as grain size and precipitates, to the deposit properties (Ref 2, 3). Several studies have also looked into the degree of particle deformation via changes in particle morphology, such as particle flattening, and established relationships with bonding mechanisms (Ref 4, 5) and deposit strengths (Ref 6). However, due to the heterogeneous microstructure of a cold spray deposit, with (prior) particle boundaries outlining consolidated splats (deformed particles) in the as-spray condition, the use of automated analysis methods for deformed particle morphology measurement is challenging. Consequently, investigations into deformed particles have been restricted in practice to single splat measurements (Ref 5) or limited splat counts in selected samples (Ref 4). More broadly, a solution to time-intensive quantification of general microstructure features observed in cold spray deposits would be a valuable tool for accelerating the development of microstructure–property relationships.

Digital image analysis techniques have long been applied in general microstructure features extraction (Ref 7). However, the ability to generate relevant image descriptors of microstructure at scale with precision is the key to extract the microstructure–properties relationship of materials. In recent years, with more advanced image analysis techniques powered by neural network-based approaches, namely deep learning, the feature extraction of many challenging microstructure images becomes possible. In fact, since the development of AlexNet in 2012 (Ref 8), deep learning methods, notably convolutional neural networks (CNN), constantly outperform conventional image analysis with human engineered filters (Ref 9). CNN can provide not only the classification of the images according to characteristics of the content (Ref 10), but also the semantic pixelwise segmentation (Ref 11). The capacity of CNN in combining non-linear functions and automatically extracting features skips the manual image preprocessing step of tedious selection/engineering of functions to highlight the region of interest. CNN have been widely used in the domain of medical image analysis (Ref 12,13,14,15), and its implementation in the material science field to investigate structure–property relationship is starting to thrive as well (Ref 16,17,18). Azimi et al. (Ref 19) employed CNN followed by a max-voting scheme to automatically segment different phases in steel microstructures and increase classification accuracy. Herriott et al. (Ref 20) applied CNN to voxel-based microstructural feature data to fit simulated mechanical properties for metal additive manufacturing applications.

In fact, deep learning methods including artificial neural network (ANN) and CNN have been applied to various aspects of cold spray and thermal spray such as deposition modeling and process control (Ref 21). An ANN model was applied to predict the critical velocity for cold spray deposition based on mechanical and thermal parameters (Ref 22). Another ANN approach was used to predict single-track profiles at different spray angles in cold spray additive manufacturing (Ref 23). With respect to process monitoring and parameter prediction, Zhu et al. (Ref 24) fine-tuned a pre-trained CNN model, which predicts plasma spray parameters using in-flight yttria-stabilized zirconia particle characteristics as inputs, and applied it to a similar task with NiCrAlY particles. In addition, Malamousi et al. (Ref 21) applied CNN to predict the powder feed rate in the high velocity oxygen fuel (HVOF) jet from time series audio files. It was also demonstrated in the article that a CNN-pre-trained model could perform image classification task such as assigning different microstructures of sprayed samples to different categories with regard to their porosity and oxidation presence. With respect to using CNN for microstructure segmentation in the thermal spray domain, Shan et al. (Ref 25) applied a deep CNN to perform accurate automatic segmentation of cracks, equiaxed pores, unmelted regions and defects in air-plasma-sprayed thermal barrier coatings. However, limited research has been conducted using CNN for semantic segmentation of cold spray microstructures in order to obtain quantitative measurements of microstructural features such as particle morphology.

In this work, we demonstrate that the utility of quantitative data developed from CNN for feature extraction of cold spray microstructures. Specifically, the power of CNN is harnessed to automatically segment the deformed particles, which is hardly accessible at scale with traditional image processing techniques. Cold spray deposits were evaluated with metallography, and the particle morphology was quantified and correlated to mechanical properties.

Experimental Methods

The feedstock material was a 6061 aluminum alloy powder (Valimet Inc., USA) with a d50 of ~ 34 µm. Powder particles were generally spherical with some elongated morphologies, as shown in Fig. 1. Substrates were in the form of 76 × 76 × 6 mm plates of 6061-T6 alloy prepared by grit blasting with alumina. However, as the substrate was not part of the investigation, results do not include the deposit-substrate interface.

Fig. 1
figure 1

Cross-section of powder particles

Full size image

The cold spray of ~ 3-5-mm thick deposits for characterization was performed using a Kinetiks 4000 (CGT GmbH, Germany) high pressure cold spray system with nitrogen as the process gas. In order to provide a wide variation in deposit microstructures for the training of the CNN model, the cold spray process parameters were selected according to different deposit fracture strength values from a proprietary cold spray material database. Fracture strength data obtained from 10 samples are shown in Fig. 2(a) as normalized values, with sample A, B and C highlighted for further discussion. These strength data were obtained from tensile testing on an Instron test frame (Illinois Tool Works Inc., USA) using flat, dogbone specimens with a gage section of 25 × 6 × 1 mm and crosshead speed of 0.05 mm/s. All tensile specimens were extracted by EDM with the tensile direction normal to the deposition direction, which is similar to other cold spray investigations in the literature (Ref 26, 27). It should be noted that the orientation will affect both the strength data and particle morphology observed, as reported for Al6061 by Hutasoit et al. (Ref 28).

Fig. 2
figure 2

(a) Information about samples normalized fracture strength and (b) spray and imaging directions

Full size image

Samples for optical microscopy were prepared using typical metallographic preparation methods for sectioning, cold mounting, grinding and polishing. Specimens were etched with Keller’s reagent (95 mL H2O + 2.5 mL HNO3 + 1.5 mL HCl + 1 mL HF). Optical images were acquired using an Olympus BX50 (Evident Corp., Japan) with a 200X magnification. These images were used for image analysis to measure porosity and particle morphology. The direction of the imaging with respect to the spray direction is shown in Fig. 2(b). Samples for scanning electron microscopy (SEM) were prepared using typical metallographic preparation methods for sectioning, hot mounting, grinding and polishing. The hot mounting step was performed at 180 °C for 20 min with conductive resin. Electron back scatter diffraction (EBSD) mapping was performed for sample A, B and C. EBSD analysis was performed using an SU-70 SEM (Hitachi, Japan) at 20-kV acceleration tension and with 70 degrees tilting angle of samples. A 0.2-µm step size was used during the map acquisition using a Symmetry S3 CMOS detector (Oxford Instruments, UK). The post processing software was AztecCrystal (Oxford Instruments).

The CNN model used in this work was trained with 20 original images from 10 different specimens with data augmentation. As shown in Fig. 3, the model architecture is a U-Net (Ref 11, 29). In this paper, the “U-Net model” and the “U-Net” both refer to the “CNN model” shown in the figure. The original image is divided into 2 × 2 smaller input images (one is highlighted in red in Fig. 3) with top and bottom edges mirrored to fit the square shape of an input layer. Image augmentation was performed on the images and corresponding annotated masks with rotations, distortions and flipping (Ref 11) to reach in total 400 image-mask pairs for training. Human annotation was performed on all the particle boundaries in the original 20 images, which serve as ground truth as well as the training data. In brief, the order of operations is original image → mirroring → flipping, rotation and distortion → model training.

Fig. 3
figure 3

U-Net model architecture from original optical image to binary particle boundary map. Schematic generated with VisualKeras

Full size image

This U-Net architecture is similar to the model reported by Ronneberger et al. (Ref 11). The model contains an encoder – the contracting path and a decoder – the expansive path. The contracting path includes the input layer, convolutional layers (Conv2D) for feature extraction and max-pooling layers (MaxPooling2D) for downsampling. In the expansive path for the decoder, the “Conv2DTranspose” layers play a role of upsampling while the concatenate layer connects them to the convolutional layers from the encoder (arrows shown in Fig. 3). These connections, termed “skip connections” in the literature (Ref 11, 30), allow the network to pass from the encoder to the decoder the spatial information of the features lost during downsampling (Ref 31).

Post-processing of the binary image output from the model was performed using image analysis software Clemex Vision PE 8.0 (Clemex Technologies, Canada) with reconstruction to finish segmenting the particles. The particle count for each sample was in the range of ~ 500-600 particles. The particle analysis was carried out with the same software, and the results are presented in the form of morphological descriptors, such as compactness, built-in to the software. The compactness, C, of a particle is defined as:

$$C = \frac{4\pi A}{{P^{2} }}$$

where A and P are the area and perimeter, respectively. The compactness value is close to 1 when the particle is more roundish and lower when the particle is more concave or deformed. In addition to the common descriptors available in the software, the post-processed binary image outputs can also be used to determine flattening ratios, with suitable assumptions for the diameters of undeformed particles. The flattening ratio, F, of a particle, was defined as (Ref 5):

$$F = \frac{D - h}{D}$$
$$D = \sqrt[3]{{w^{2} h}}$$

where D denotes the original diameter of particle, and w and h are the width and height, respectively, of deformed particle after spray.

Results and Discussion

Microstructure

Typical optical images from the cross section of samples with the lowest and highest fracture strengths are shown in Fig. 4. The boundaries of deformed particles are readily observed, with primary alpha and eutectic phases from the initial powder feedstock still present within the particle interiors. Qualitatively, the micrograph in Fig. 4(a) features more porosity and particles that are less deformed than those in Fig. 4(b), which is consistent with the relative strengths of the associated samples. With respect to features specific to image segmentation, dark lines appear both at the edge of the particles and within the particles. The dark regions between some particles in Fig. 4(a) are porosities, which are not as present in Fig. 4(b). The lines in the particles are grain and phase boundaries, inclusions or defects that are revealed by the etchant, and it is difficult to suppress the contrast of dark lines inside particles while bringing out the contrast of the particle boundaries. The coexistence of these contrasts is the main challenge for the contrast-sensitive convolutional image segmentation methods, often used in ImageJ (Ref 32), to perform particle separation (Ref 9).

Fig. 4
figure 4

Typical optical micrographs for (a) sample with lowest fracture strength (sample A) and (b) sample with highest fracture strength (sample C)

Full size image

EBSD analysis results in the form of inverse pole figure (IPF) maps on samples A, B and C are shown in Fig. 5. The maps reveal the presence of small grains decorating the boundaries of deformed particles. This is consistent with various studies reporting dynamic recrystallization at particle boundaries in cold spray deposits (Ref 33,34,35), with higher strain rate resulting from greater impact during cold spray usually indicative of a higher chance of recrystallization (Ref 2). In this work, the EBSD samples were hot mounted at a temperature of 180 °C which may have triggered static restoration as well. Similar to the case with the optical micrographs, clear differences can be observed between sample A and C, with sample A clearly displaying a fewer quantity of fine grains at deformed particle boundaries. The map for sample B is similar qualitatively to sample C, with the area of recrystallized grains in samples A, B and C measured at 1.4%, 2.0% and 2.5%. For this quantification of the deposit microstructures, all grains with a maximum orientation spread less than two degrees were defined as recrystallized.

Fig. 5
figure 5

EBSD inverse pole figure map for (a) sample A, (b) sample B and (c) sample C showing recrystallized grains at particle boundaries. The spray direction is horizontal (x-axis)

Full size image

U-Net Model

Figure 6(a) shows a representative set of original images and corresponding predictions of particle boundaries obtained from Otsu’s thresholding filter (a traditional image processing method), the U-Net prediction and the human annotations as ground truth. It should be noted that the top and bottom of these images have mirroring to fit the square input of the model. The performance of the U-Net model can be visually assessed by comparing each column. The Otsu’s thresholding fails to distinguish the internal contrast of the eutectic phases within particles from the particle boundaries. On the contrary, the U-Net model can in general predict the particle boundaries without revealing this internal contrast, although some boundaries remain unclosed compared to the human annotations, as indicated with red arrows in Fig. 6(b).

Fig. 6
figure 6

(a) Comparison of traditional image processing using Otsu’s method, U-Net prediction of the particle boundaries and ground truth by human annotation of the particle boundaries and (b) overlayed mask of U-Net prediction and of ground truth on original images for visualization

Full size image

With such a level of performance in the U-Net prediction, the masks of predicted particle boundaries can be post-processed to be closed and connected by functions such as “watershed” and “reconstruct” using traditional image processing software to ultimately enable the identification and analysis of the particles. In order to evaluate the performance of the U-Net model, particle size and morphology measurements obtained from U-Net-predicted masks of particle boundaries were compared to respective results from human annotated images. Figure 7 shows scatter plots of the average particle area (on the left) and the average particle flattening ratio (on the right). In this type of visual sanity check, the line y = x indicates exact agreement between U-Net prediction (x-axis) and ground truth by human annotation (y-axis). Due to a number of systematic errors (discussed below), Fig. 7 shows measured values from the U-Net-predicted masks that have been corrected using constants fitted for each measure as follows:

$$A_{{{\text{Unet}} - {\text{corrected}}}} = A_{{{\text{Unet}}}} + c_{1}$$
$$F_{{{\text{Unet}} - {\text{corrected}}}} = F_{{{\text{Unet}}}} + c_{2}$$

where AU-Net and FU-Net are the average area and flattening ratio, respectively, measured directly from the U-Net-predicted masks, and AU-Net-corrected and FU-Net-corrected are the corrected values of average predicted area and flattening ratio, respectively, after offset by experimental constants. The values of the constants c1 and c2 were − 20 µm2 and 0.05, respectively.

Fig. 7
figure 7

Measurements from corrected U-Net prediction compared to ground truth by human annotation for (a) particle area and (b) particle flattening ratio. The dashed lines represent y = x

Full size image

In general, the data are linearly correlated; however, there exist some systematic error and outliers. First, the predicted areas tend to be larger than the annotated ones. This could be due to the fact that some boundaries of the particles are not fully closed after the U-Net prediction and post-processing. Second, the predicted flattening values also tend to be smaller than the annotated ones. This could result from actual boundaries being unrecognized by the U-Net model, thus rendering a particle appearing rounder when the ground truth is two deformed and elongated particles. Moreover, unclosed boundaries (Fig. 6b) may also contribute to an underestimation of the flattening ratio because the final branch growing during post-processing may close the particles with a short end.

Given the resemblance of the U-Net prediction images and the annotated images (Fig. 6) along with the linear correlation of the particle analysis results between U-Net model and annotation (Fig. 7), the U-Net model was then exploited to automatically process 240 new images from 10 samples in order for the morphologies of the particles to be analyzed with a larger statistical dataset. These are the same samples that were tested for fracture strength and from which training images for the U-Net model were acquired.

Microstructure–Property Correlation

Figure 8(a) and (b) shows the average measurements of compactness and flattening ratio obtained from U-Net-processed micrographs against fracture strength values, respectively. Particles with equivalent diameters smaller than 10 µm were excluded from the dataset in order to reduce the artifacts induced by unclosed particles in U-Net-processed micrographs mentioned in the previous section. The particle size distribution was also considered to be a cause of the relatively large standard deviation (error bars) for the average values for the morphology descriptors. In general, the results show that deposits with microstructures featuring lower compactness and higher flattening ratio are correlated with higher fracture strengths, albeit with some deviation. As previously noted, a lower compactness is obtained when a particle is more deformed. Consequently, both compactness and flattening ratio values are consistent with the general notion that higher particle deformation is correlated to better particle bonding.

Fig. 8
figure 8

Correlation between fracture strength and morphology data extracted from U-Net prediction for (a) compactness and (b) flattening ratio. (c) Correlation between fracture strength and porosity

Full size image

Higher flattening ratios have been shown to be correlated with higher fracture strength for copper deposits (Ref 36) and higher adhesion strength for titanium splats (Ref 37). Calculated flattening ratios from finite element modeling (Ref 6) have also been shown to be a good indicator of cohesive strength in cold spray copper deposits, as both flattening ratio and strength were related to the ratio of particle velocity and critical velocity for deposition. The particle and critical velocities are dependent on process parameters and material properties. Particle temperatures in the gas jet during cold spray have been calculated using CFD simulation to be in the range of ~ 50-330 °C for aluminum (Ref 38). In general, cold spray particle temperatures have not been measured experimentally; however, an investigation with a high-speed IR camera measured particle temperatures for titanium particles to be less than 50 °C (Ref 39). In cold spray, metallurgical bonding at a particle interface is dependent on the formation of clean metal contact surfaces that is achieved via an adiabatic shear instability mechanism (Ref 40). Consequently, the oxide layer on the surface of the feedstock powder particles can significantly affect the quality of particle bonding (Ref 2). Yin et al. (Ref 41), using finite element analysis, detailed how the pre-existing oxide film on Al6061 powder particles can be broken up and ejected from the contact interface as the particle deforms and flattens during impact. However, cracked oxides that remain in the deposit act as a barrier to particle bonding and lead to defects that can be detrimental to deposit properties. Kang et al. (Ref 42) characterized aluminum splats using TEM to reveal the aluminum oxide layer at the interface and corresponding pore defects. No nitride scale was reported in their study (Ref 42). For copper deposits, Li et al. (Ref 36) reported that enhanced particle deformation led to higher strength deposits due to a more dispersed oxide scale. In this study, the characterization of oxygen content in the deposit was not considered within the scope of work.

For the data presented in Fig. 8(a) and (b), the effects of individual cold spray parameters were not isolated in the analysis in order to determine whether the deformed particle morphology in and of itself was a sufficient predictor of the deposit strength. The capability to evaluate the properties of a deposit without knowledge of its processing history was considered in this initial study to provide the most flexible use case. In general, however, a number of spray parameter combinations can produce similar deformed particle morphologies, which may have complicated the observed trend. For example, different spray parameter combinations may result in a variation of the relative contribution to particle flattening from an initial impact versus peening effects. This could potentially give rise to different levels of particle bonding and associated deposit strength. Figure 8(b) shows that the separation of the spray parameters combinations that used large standoff distances (>70 mm) reduces the spread in the strength-flattening ratio relationship. Although not evaluated in this study, the use of larger standoff distances can be expected to have significant effects on the particle velocity, particle temperature and substrate surface temperature that, in turn, would affect the degree of particle deformation and particle morphology. Increasing the dataset will be needed to more fully explore the robustness of these relationships. In addition, the sensitivity of the measurement to particle counts and the statistics to reach a stable measurement needs to be considered.

The particle morphology measurements were consistent with the results obtained from porosity and EBSD measurements. Although the sample quantity processed was not sufficient to explore potential quantitative relationships between these metrics, higher flattening ratios and lower compactness values generally corresponded to lower porosity and higher recrystallized areas. For the development of microstructure–property relationships, the use of particle morphology measurement features practical benefits in comparison with the other metrics. The porosity is a commonly used indicator of deposit quality; however, Fig. 8(c) shows that the sample porosity in this study was relatively low (typically < 1%) and measured values at these levels tended to be very sensitive to the metallographic preparation, especially the etching step. Although the data generally indicate that increased porosity results in poorer mechanical properties, the presence of noise complicates the interpretation at lower levels of porosity. Li et al. (Ref 36) reported that the evaluation of cold spray copper deposits only by porosity may not be sufficient as significant differences in the degree of particle deformation, tensile strength and electrical conductivity were obtained with increasing particle impact velocity even though porosity levels were less than 0.6% in the samples tested. With the CNN approach demonstrated, the particle morphology measurement is nearly as straightforward as porosity measurement and may provide a more consistent measure when comparing dense deposits. The results shown in Fig. 8 do suggest that caution is warranted when a single morphological factor such as flattening ratio is used for mechanical property prediction with a limited dataset over wide test ranges. However, the benefits of optical microscopy compared to EBSD in terms of automation, productivity, ease of preparation, etc. are clearly significant.

Further analysis to refine particle morphology–property correlations with consideration of spray parameters is expected to render these types of relationships clearer. In addition, the U-Net-predicted micrographs can also be post-processed for other shape measures in order to help compare the effectiveness of deformed particle features for predicting strength or other deposit properties of interest. Finally, although not discussed in this work, it is clear that process parameter–microstructure relationships based on deformed particle metrics can also be investigated. With microstructure–properties relationships in hand, the development of such parameter–microstructure relationships would accelerate the definition of process windows and/or optimization of parameters by limiting the amount of (mechanical) testing required.

Application to Other Materials and Alloys

In general, the effectiveness of particle identification from the CNN model is dependent on the amount of training data available and a robust model that can accommodate greater variations in the image features. Although this case study was focused on only one aluminum alloy, the powder microstructure of primary alpha with a secondary eutectic phase is common for other plasma atomized aluminum alloys. Consequently, an application of the developed capability for cold spray of various aluminum alloys is relatively straightforward with the amount of additional training data required, if any, dependent on the similarity of the microstructures. It is expected that the CNN model can be applied as-is to other 6xxx and 7xxx series aluminum alloys. By extension, the overall approach can be easily envisioned for the cold spray of other powder materials, with suitable modification for the different powder microstructures. For example, powders featuring martensitic microstructures such as titanium-based alloys would require a completely new dataset for model training. Future work will investigate the general applicability of the CNN model to other material alloy systems.

Conclusion

This work demonstrates that a CNN model such as U-Net is capable of properly identifying the particles in optical microscopy images of a cold spray deposit of aluminum alloy. As this is challenging with a traditional image processing technique or extremely time consuming with manual annotation, the model development offers a new capability to study process–microstructure–property relationships based on deformed particle features. For a demonstration case, the U-Net model was employed to successfully capture subtle differences in the flattening ratio of particles in microstructures across different specimens for correlation with deposit strength.