In-process monitoring and prediction of droplet quality in droplet-on-demand liquid metal jetting additive manufacturing using machine learning

Abstract

In droplet-on-demand liquid metal jetting (DoD-LMJ) additive manufacturing, complex physical interactions govern the droplet characteristics, such as size, velocity, and shape. These droplet characteristics, in turn, determine the functional quality of the printed parts. Hence, to ensure repeatable and reliable part quality it is necessary to monitor and control the droplet characteristics. Existing approaches for in-situ monitoring of droplet behavior in DoD-LMJ rely on high-speed imaging sensors. The resulting high volume of droplet images acquired is computationally demanding to analyze and hinders real-time control of the process. To overcome this challenge, the objective of this work is to use time series data acquired from an in-process millimeter-wave sensor for predicting the size, velocity, and shape characteristics of droplets in DoD-LMJ process. As opposed to high-speed imaging, this sensor produces data-efficient time series signatures that allows rapid, real-time process monitoring. We devise machine learning models that use the millimeter-wave sensor data to predict the droplet characteristics. Specifically, we developed multilayer perceptron-based non-linear autoregressive models to predict the size and velocity of droplets. Likewise, a supervised machine learning model was trained to classify the droplet shape using the frequency spectrum information contained in the millimeter-wave sensor signatures. High-speed imaging data served as ground truth for model training and validation. These models captured the droplet characteristics with a statistical fidelity exceeding 90%, and vastly outperformed conventional statistical modeling approaches. Thus, this work achieves a practically viable sensing approach for real-time quality monitoring of the DoD-LMJ process, in lieu of the existing data-intensive image-based techniques.

Introduction

Motivation and objective

Droplet-on-demand liquid metal jetting (DoD-LMJ) is an additive manufacturing technique in which molten metal is ejected from a sub-millimeter nozzle using a momentum pulse. The relative motion of the nozzle and substrate creates a three-dimensional part. The momentum pulses required to create and control the rate and size of ejected droplets can be achieved piezoelectrically (Luo et al., 2016a; Wang et al., 2017, 2018), magneto-hydrodynamically (Simonelli et al., 2019; Sukhotskiy et al., 2017, 2021), electro-hydrodynamically (Han & Dong, 2017a, b), pneumatically (Beck et al., 2020; Chang et al., 2020; Gerdes et al., 2018), electromagnetically (Luo et al., 2016b), or with a laser (Stein et al., 2018).

In the DoD-LMJ process, studied in this work, the momentum pulse is generated pneumatically. A schematic representation of this system is shown in Fig. 1a. Metal is melted inside a furnace that is housed in a crucible. Metal droplets (in this work Sn) are deposited (jetted) on a substrate in a low-oxygen (argon) environment. The droplet jetting is achieved pneumatically by controlling the valves that regulate the pressure and flow of argon gas. The valves, and thereby the jetting frequency, are controlled by a digital signal. Shown in Fig. 1b are images of jetted tin (Sn) droplets demonstrating the variation in droplet size and shape produced during this process, which in turn adversely affects the functional integrity of the part (Idell et al., 2019; Wang et al., 2016).

Fig. 1

a Schematic representation of the pneumatically controlled droplet-on-demand liquid metal jetting additive manufacturing process studied in this work. b High-speed imaging of the jetted droplets depicts variation in their size and velocity, resulting in poor functional integrity

The DoD-LMJ process offers significant advantages in terms of build time, reduced material cost, and sustainability, compared to powder-based metal additive manufacturing processes, such as laser powder bed fusion and directed energy deposition (Idell et al., 2019). A variety of new applications, such as high-performance electronic packaging, are also envisioned with DoD-LMJ (Idell et al., 2019; Lee et al., 2008b). Despite the potential benefits of DoD-LMJ, the lack of process consistency has constrained the creation of functional parts at scale. As exemplified in Fig. 1b, large variations in the droplet characteristics—size, velocity, and shape of the droplets—contributes to process inconsistency and undesirable part quality (Idell et al., 2019; Wang et al., 2016). Therefore, a critical challenge in DoD-LMJ is to monitor and control the droplet characteristics within a tight window during the printing process.

The characteristics of an ejected droplet are influenced by complex physical interactions amongst various process factors including nozzle diameter, driving voltage, orifice pressure, atmosphere composition, temperature, and liquid metal material properties (viscosity, density, and surface tension) (Beck et al., 2020; Wang et al., 2018). As an example, the different shapes of droplets observed in the DoD-LMJ experiments studied in this work shown in Fig. 1b occur largely due to varying oxygen concentration in the jetting environment.

As will become evident from the forthcoming summary of the literature presented in Sect. “Prior work, challenges, and novelty”, existing approaches for in-situ monitoring of droplet behavior in DoD-LMJ rely on high-speed imaging sensors. The resulting large volume of droplet images acquired is computationally demanding to analyze and hinders real-time control of the process.

To overcome the foregoing challenge, the objective of this work is to use data acquired from an in-process millimeter-wave sensor for predicting the size, velocity, and shape characteristics of droplets in DoD-LMJ process. As opposed to high-speed imaging, this sensor produces data-efficient time series signatures that allows rapid, real-time process monitoring (Chang et al., 2020). Additionally, we devise computationally tractable machine learning models that use the millimeter-wave sensor data to predict the droplet characteristics (size, velocity, and shape). This work thus takes the critical first-step towards real-time monitoring and control of the DoD-LMJ process.

Prior work, challenges, and novelty

Thus far, the predominant method for monitoring the droplet characteristics in DoD-LMJ systems is high-speed videography at a frame rate exceeding 1 kHz. In a recent work, Beck et al. (2020) demonstrated the satellite-free droplet ejection in a pneumatically-controlled liquid metal jetting process. The stability of the process was validated with the help of high-speed video camera analysis which revealed that droplets were ejected in near pill-like shapes devoid of any tail formation. Further, the high-speed camera analysis was used to extract parameters, viz., Weber number and Ohnesorge number, for the volume-of-fluid model. Similarly, Luo et al. (2012) developed a 2D axisymmetric droplet generation model for the pneumatic DoD metal jetting system. The model was validated by analyzing images collected from a high-speed video camera. The authors observed that the size of the ejected droplet is proportional to the nozzle diameter which aligns with the observations made in the current work.

Lee et al. (2008b) demonstrated the capability of monitoring droplet dynamics in a DoD solder jetting system using a high-speed video camera. The authors characterized the effect of various process parameters (e.g., chamber pressure, operating frequency) on droplet diameter and velocity with the aid of high-speed videography. In a similar jetting system for solder, Wang et al. (2016) used a high-speed video camera to characterize the droplet shape upon impact at varying levels of velocity, and distance between substrate and nozzle. It was observed that both the parameters have a significant effect on the droplet shape upon impact. recent work by Wang et al. (2018) demonstrates the use of a closed-loop framework in which the driving voltage of an alternating magnetic field was tuned using optical imaging data to monitor the droplet characteristics, such as volume, velocity, presence of satellite and ligament characteristics.

Similar machine-learning and high-speed videography-based approach for in-situ monitoring of the LMJ process was proposed by Wang et al. (2018). The proposed framework utilizes image processing of video camera data in conjunction with a neural network to predict the droplet behavior. The authors extracted numerous features from the high-speed images, such as, number of satellites, area and volumes of droplets, and aspect ratio of the ligaments. These features were subsequently used as inputs to a neural network to predict the voltage which is in turn correlated to the jetting stability. To the best of knowledge, these are the only works that utilize machine learning-based in-situ monitoring in LMJ. Contrarily, machine learning has been extensively used for process monitoring in other additive manufacturing processes (Jin et al., 2020; Meng et al., 2020; Wang et al., 2020). A few works are summarized herewith.

From a broader vista, machine learning has been used extensively for process monitoring and quality assurance in additive manufacturing (Meng et al., 2020; Qin et al., 2022; Wang et al., 2020). Recently, Larsen and Hooper (2022) developed a deep semi-supervised Variational Autoencoder machine learning model that used high-speed imaging data for anomaly detection in laser powder bed fusion (LPBF) additive manufacturing. The proposed approach was able to differentiate between optimal and undesirable processing conditions with an accuracy in terms of the area under the curve (receiver operating curve) ~ 0.99. Yuan et al. (2018) also proposed a machine learning-based in-situ monitoring framework for LPBF wherein high-speed video camera data was analyzed using a deep Convolutional Neural Network (CNN). The authors predicted the mean and standard deviation of single-track widths with an accuracy of 0.93 and 0.77 (R²), respectively.

In a similar vein, Lee et al. (2020) examined the performance of multiple deep learning models to detect part quality in two-photon lithography additive manufacturing. By varying various process parameters, three distinct quality regimes of part quality, viz., cured, uncured, and damaged, were obtained. These part quality regimes were predicted by analyzing in-situ optical images with a CNN and a Long-short Term Memory (LSTM) Recurrent Neural Network (RNN). The authors were able to achieve a maximum accuracy of 95% while classifying the part quality regimes.

Gaikwad et al. (2020) proposed a Sequential Artificial Neural Network for predicting the single-track width and continuity using high-speed video camera data and single-wavelength pyrometer in laser powder bed fusion. The authors demonstrated that their approach performed at par to highly complex CNN and LSTM-RNN models and required nearly 1/10th of the computation time. Similarly, Imani et al. (2018) demonstrated the ability of an ANN to predict laser powder bed fusion process parameter regimes using layer wise optical images.

Kumar and Maji (2020) in their recent work employed a Genetic Algorithm-based (GA) framework for extracting the optimal process parameters of wire-arc additive manufacturing process. Similarly, Vaissier et al. (2019) demonstrated the efficacy of using GA for optimizing support structure generation in additively manufactured parts. Xia et al. (2021) used a Particle Swam Optimization (PSO) algorithm in conjunction with an Adaptive Network-based Inference System (ANFIS) machine learning model to predict surface roughness of wire-arc additively manufactured parts. The authors used in-situ measurements acquired from a laser line scanning system to quantify the surface roughness of the parts. Subsequently, the ANFIS model was used to predict the surface roughness and its hyperparameters were optimized using the PSO algorithm.

While deep learning models, such as CNN and LSTM-RNN, have been successfully applied for process monitoring in additive manufacturing, a vital requirement for their practical deployment is the availability of large datasets (Chartrand et al., 2017; Sanaat et al., 2022). Insufficient data for training these models often leads to overfitting thereby hampering the performance of these models. In other words, while suitable for research and development systems, the exorbitant data volume (storage), and computational burden of processing continuous, high-resolution droplet images presents a major bottleneck to commercially relevant systems.

Additionally, processing data, especially in form of 2D images is computationally intensive, as several images have to be stored and analyzed, which imposes a latency and thus hinders practical deployment of such image-based approaches. Furthermore, majority of complex machine learning models for image-based data analysis require Graphical Processing Unit-based (GPU) computation which further makes these data-intensive approaches cost prohibitive. Lastly, although the computational efficacy of simple (shallow) ANNs has been reported in previous works these models do not have the ability to capture the temporal dependencies in a time series data, noting that the droplets in DoD-LMJ are temporally correlated (Abiodun et al., 2018). In other words, data points inputted to an ANN are independent of each other.

In this work we demonstrate the use of a millimeter-wave sensor that significantly reduces data storage and processing compared to high-speed video-based monitoring in DoD-LMJ. This sensor monitors the changes in magnitude and phase of the return loss (a ratio of input to output signal) due to near-field electromagnetic scattering by the jetted droplet stream (Chang et al., 2021). As opposed to data intensive image-based monitoring, the signal is a 1D waveform. The focus of this work is to develop and apply machine learning models to characterize the ejected droplets in terms of their size, velocity and shape using data from the millimeter-wave sensor. As we will demonstrate in forthcoming sections, the sensing and data analysis approach presented in this work enables the replacement of the cumbersome high-speed video data streams with tractable millimeter-wave time series sensor data that is easy to acquire, store, and analyze. Pertinently, the temporal information in the millimeter-wave sensor time series data, which it is essential to capture the evolution of droplet size and velocity, is encapsulated within a novel multilayer-perceptron-based nonlinear autoregressive model.

The rest of the paper is organized as follow. In Sect. “Methods”, we describe the methods encompassing the experiments, in-process millimeter wave sensing setup, and machine learning approaches devised to leverage the data. Section “Droplet shape classification” reports the results, and provides insights into the statistical fidelity, as well as transferability of the machine learning model when used for different scenarios. Conclusions and avenues for future work are provided in Sect. “Conclusions and future work”.

Methods

The DoD-LMJ machine setup and in-situ monitoring sensor are described in Sect. “In-process sensing setup”, and the design of experiments and representative data collected from the in-situ monitoring sensors are summarized in Sect. “Experiments”. The acquired data and machine learning approach to predict the droplet characteristics are described in Sects. “In-process sensor data” and “Machine learning-based droplet characteristic prediction”, respectively.

In-process sensing setup

The pneumatic DoD-LMJ system used in this work is depicted in Fig. 2 (Beck et al., 2020; Chang et al., 2020). Droplets were ejected by pressurizing the crucible containing melted 99.95% pure tin (Sn). The temperature-controlled crucible was set to 340 °C to melt the Sn feedstock, and the nozzle temperature was set above the melting point (232 °C). Pressure in the crucible was generated with argon gas controlled by fast switching, high flow valves. The actuation of these valves was governed by a digital waveform generator.

Fig. 2

source aids the HSVC’s image contrast

Schematic representation of the DoD-LMJ setup with two in-process sensors, viz. millimeter-wave (MW) sensor and high-speed video camera (HSVC). The MW sensor is connected to an open-ended waveguide. An illumination

The printhead can accommodate varying nozzle sizes and was housed in a transparent enclosure that was sealed with an aluminum covering to create a slight overpressure of argon purge gas to ensure oxygen did not enter the chamber. A purge line was provided to regulate the flow of argon gas in the chamber. The location of droplet deposition was controlled by moving the nozzle in the Z-direction and the substrate in the X–Y direction.

Shown in Fig. 2, the DoD-LMJ system was equipped with two different sensors, viz. high-speed video camera (HSVC) and millimeter-wave (MW) sensor. Additionally, a pressure transducer monitored the pressure in the crucible. The HSVC was placed higher than the MW sensor with respect to the substrate. An illumination lamp was placed in the field-of-view of the HSVC (and behind the nozzle) to enhance the contrast of the droplets in the HSVC frames. The HSVC data was acquired at a sampling rate of 2.5 kHz.

The MW sensor is composed of an open-ended waveguide connected to a vector network analyzer, which generates a continuous-wave signal at 40 GHz. As droplets pass in front of the aperture of the open-ended waveguide, they induce variations in the reflected signal (electromagnetic scattering) as a function of time, which is detected by monitoring the return loss on the vector network analyzer. Because of the continuous operation of this MW sensor, dynamics of individual droplets can be extracted and studied in-situ. Furthermore, the MW sensor operates at a sampling rate of 25.6 kHz, thus providing the high time-scale resolution necessary for capturing the droplet dynamics. As it will be explained shortly in Sect. “Experiments”, the MW sensor provides both magnitude and phase information. While a similar setup was demonstrated in Ref. (Chang et al., 2020), the current setup has the advantage of being less invasive.

Experiments

Droplets are characterized based on the following three aspects: (i) droplet size, (ii) droplet velocity, and (iii) droplet shape. Two experiments, labeled Experiment 1 and Experiment 2 summarized in Table 1, were conducted. Also provided in Table 1 are the representative droplets imaged by the HSVC during the two experiments. The aim of Experiment 1 was to influence the size and velocity of the droplet by changing the nozzle diameter and pressure setpoint in the crucible. Likewise, in Experiment 2, the shape of the droplet was varied by changing the percent oxygen in the argon container (via changing the purge gas flow rate) and the pressure setpoint in the crucible. The rationale and procedure adopted for each experiment is described herewith.

Table 1 Process parameters under which the two sets of experiments were performed along with representative high-speed video camera data

Experiment 1: Variation in droplet size and velocity

The focus of Experiment 1 was to form droplets of various sizes and velocities in a controlled manner by modulating two key process parameters, namely, nozzle diameter and pressure setpoint. The droplet velocity was varied by regulating the pressure setpoint of the pneumatic DoD-LMJ setup. Increasing the pressure results in a higher amount of momentum delivered to the droplet, thereby increasing its velocity post ejection (Idell et al., 2019). To encapsulate this phenomenon, two different pressures (Experiment 1.A = 3 psig, Experiment 1.B = 2 psig) were chosen at a constant nozzle ID = 400 µm to compare two different velocities for approximately the same droplet size.

Based on prior experiments reported in the literature, it was observed that the size of droplets is influenced by the internal diameter (ID) of the nozzle (Idell et al., 2019). Accordingly, it was observed that the size of droplets ejected from the ID = 100 µm (Experiment 1.C) nozzle were relatively smaller than the droplets ejected from the ID = 400 µm (Experiment 1.A) nozzle. Hence, in Experiment 1.C a higher pressure of 6 psig for the 100 µm nozzle was used because a larger amount of force is needed to break the surface tension of a smaller nozzle.

Lastly, to ensure that surface oxidation did not adversely alter the shape of droplets (not desired in Experiment 1), the build chamber was purged with Argon gas (at least 5 SLPM). Experiment 1.C also serves a vital purpose; it used for testing the transferability of the machine learning models. In these so-called transferability tests the models trained with data in Experiment 1.A and 1.B, are applied directly to data from Experiment 1.C.

Experiment 2: Variation in droplet shape

The aim of Experiment 2 was to quantify the fidelity of the MW sensor to detect variation in droplet shape. Accordingly, the amount of percent oxygen in the build chamber was varied by controlling the Argon gas flow to obtain a variety of droplet shapes. The surface oxidation of the Sn droplets has a significant influence on their ejected shape (Beck et al., 2020; Li et al., 2014; Pasandideh-Fard et al., 1998). While droplet cooling may have a similar shape-modulating effect as oxidation, it was prevented in our experiments by heating the nozzle tip to above the melting point of Sn (232 °C).

Furthermore, the pressure in Experiment 2.B and 2.C was varied as the amount of oxide on a droplet affects the force required to break the surface tension at the nozzle (Song et al., 2020). Lastly, for this experiment, ID = 200 µm nozzle was used to obtain intermediate-sized droplets to facilitate detection at a desirable signal-to-noise ratio and to negate dripping issues that are observed in nozzles with larger ID (Song et al., 2020).

In-process sensor data

Experiment 1: Data acquired for droplet size and velocity characterization

In Experiment 1, the pressure was varied at two levels, viz., 3 psig (Experiment 1.A) and 2 psig (Experiment 1.B) resulting in droplets with differing size and velocity (Table 1). Other processing parameters, such as percent oxygen, purge gas flow rate, and inner nozzle diameter were held constant (Table 1). Figure 3 exemplifies the representative sensor data acquired during Experiment 1.

Fig. 3

Representative a high-speed video camera data acquired from Experiment 1.A (Table 1). b and c Corresponding millimeter-wave sensor data magnitude and phase, respectively. Data from four droplet streams is shown. The overall magnitude of the millimeter-wave sensor signal is higher in droplet streams 1 and 4 in comparison to droplet streams 2 and 3. The signal amplitude is proportional to the size of the droplets

From Fig. 3 it is evident that the size of droplets between streams can vary substantially despite maintaining fixed processing conditions. A representative example of this phenomena is highlighted in Fig. 3a which shows the HSVC data of four droplet streams. Particularly, comparing droplet stream 3 and 4, the droplets from the latter (stream 4) are observed to be significantly larger than that in the former (stream 3). Shown in Fig. 3b and c are the corresponding magnitude and phase of the MW sensor, respectively. Accordingly, the amplitude of the MW sensor magnitude and phase are larger for stream 4 in comparison to stream 3 in proportion to the droplet size. We made similar observations from the MW sensor data collected from the experiment done under Experiment 1.B for which the representative data is shown in Fig. 4. The MW sensor magnitude (Fig. 4b) and phase (Fig. 4c) increases in amplitude in proportion to the size of the droplets in a stream as shown in the HSVC frames (Fig. 4a).

Fig. 4

Representative a high-speed video camera data acquired from Experiment 1.B (Table 1). b and c Corresponding millimeter-wave sensor data magnitude and phase, respectively. Data from three droplet streams is shown

This visual analysis suggests that the MW sensor signatures encapsulate the information concerning the size of droplets. However, the challenge is to precisely quantify the size and velocity characteristics of droplets as a function of the MW sensor signatures. In this work, this challenge is addressed via machine learning-based time series analysis (Medsker & Jain, 2001; Rao et al., 2014).

Further experimental tests were performed to affirm the transferability of the machine learning models. These model transferability experiments, detailed under Experiment 1.C in Table 1, have drastically different processing conditions compared to Experiment 1.A and 1.B. Along with the process parameters given in Table 1, the following process parameters were changed for Experiment 1.C: the valve frequency was increased to 80 Hz from 5 Hz in Experiments 1.A and 1.B, and the valve duty cycle was increased from 1.5% in Experiments 1.A and 1.B to 32% for Experiment 1.C. On account of this drastic change in the process conditions, the droplet characteristics of this experiment vary significantly in comparison to Experiment 1.A and 1.B.

Experiment 2: Data acquired for droplet shape characterization

In Experiment 2, five different combinations of the process parameters were used to obtain droplets of varying shapes (Table 1). Figure 5 shows the representative data acquired during Experiment 2. From the HSVC frames in Fig. 5a, it is apparent that distinct droplet shapes result from altering the processing conditions (pressure setpoint of the crucible and oxygen concentration). The corresponding time series wave profiles of the MW sensor (magnitude) are shown in Fig. 5b. The response (amplitude and shape) of the MW sensor waveform is visually correlated to the shape of the physical droplet. In this work, a supervised machine learning model is used to classify the droplet shape as a function of the spectral characteristics of the MW sensor waveforms.

Fig. 5

a High-speed video camera (HSVC) data and b millimeter-wave (MW) sensor and from Experiment 2 (Table 1). The wave profile of the magnitude of the MW sensor data varies significantly with varying processing conditions. This variation is also captured in the HSVC data where the shape of the 2D profiles of the droplets vary with the processing conditions

The pressure used to eject the droplets in Experiment 2.B is significantly lower (2 psig) in comparison to Experiment 2.D. This difference in pressure leads to a difference in the velocity of droplets after leaving the nozzle. In other words, the droplets ejected at higher pressure have higher velocity post ejection. This difference in velocity contributes towards the difference in the waveforms of the droplets from the two experiments. Furthermore, the apparent difference in the droplet shapes leads to a difference in their corresponding waveforms. As evident from Fig. 5, the droplets from Experiment 2.B lead to peanut-shaped droplets, whereas the droplets from Experiment 2.D result in paramecia-shaped (single-celled organism) droplets.

Machine learning-based droplet characteristic prediction

In this section, we present two machine learning approaches used for droplet characteristic prediction in terms of their size and velocity (Sect. “Predicting droplet size and velocity”), and shape (Sect. “Droplet size prediction”) respectively. The machine learning-based approach of prediction of droplet characteristics using MW sensor signatures is depicted in Fig. 6.

Fig. 6

Schematic representation of the approach adopted to predict droplet characteristics using MW sensor data

Predicting droplet size and velocity

(a) Obtaining ground truth data The HSVC data (images) serve as ground truth for droplet size and velocity and are used for training the machine learning models. The aim is to represent each HSVC image as a discrete data point in a time series. The image processing steps are shown in Fig. 7. First as shown in Fig. 7a, each frame was cropped to a size of 943 × 582 pixels to focus on only the region-of-interest around the nozzle. Next, the 3-dimensional RGB images were converted into grayscale (Fig. 7b) and binarized, i.e., converted to black and white (Fig. 7c). Lastly, each droplet in the image was isolated and given a label (Fig. 7d) using object detection which was done using the 8-connectivity approach (Rosenfeld, 1970). This procedure is detailed in “Appendix A1”. After object detection, the size and velocity of the droplets was quantified using the following procedure.

Fig. 7

Basic image pre-processing steps applied to all a as-received high-speed video camera images. RGB images are first converted to b grayscale images, followed by c binarization and d object detection

The size of the droplets was estimated from their pixels in the HSVC image (Fig. 7d). The size of a droplet is defined as the number of enclosed pixels. Mathematically, this can be defined as; ${\alpha }_{i}^{j}= \sum_{k=1}^{M}{I}_{k}$, where $i$ is the HSVC image number, $j$ is the droplet label in the image, $M$ is the number of pixels enclosed by the droplet, and ${I}_{k}=1$ is the pixel value (for binarized images). A visual representation of this is shown in Fig. 14a in “Appendix A2”.

The velocity of a droplet is estimated as the time taken by the droplet to travel a certain distance from the tip of the nozzle. This distance is approximated as the Euclidean distance from the centroid of the droplet to the nozzle in a given HSVC image. The time is obtained using the camera frame rate and the number of elapsed frames between separation of the droplet from the nozzle until a certain distance is reached. This distance is monitored over a sequence of frames thereby resulting in a distance-based time series. We then compute the first-order approximation of this time series using the second-order finite difference method. The mathematical formulation of this approach is given in “Appendix A2”. The MW sensor data was down-sampled and truncated to register it with the HSVC data as the two sensors have significantly different sampling rates. A detailed explanation of this is given in “Appendix A3”.

(b) Multilayer perceptron-based non-linear autoregressive model

The aim is to predict the size and velocity of droplets as a function of the MW sensor time series data. The size and velocity ground truth information are estimated from the HSVC data as described in Sect. “Predicting droplet size and velocity”a). A multilayer perceptron-based non-linear autoregression with exogeneous inputs (MLP-NARX) model was developed for predicting the droplet size and velocity from the time series signatures captured from the MW sensor data. The rationale for using this MLP-NARX approach is as follows. The sensor data streams are time series that contain temporal dependencies, i.e., data obtained at time ($t+1$) is autocorrelated data with time ($t$) as observed in Figs. 3 and 4. Conventional machine learning models, such as the feed-forward artificial neural network, cannot account for these temporal dependencies (Abiodun et al., 2018). To explain further, machine learning models typically treat data points to be independent from each other. However, the droplet characteristics evolves from the time it is jetted, hence its characteristics are temporally correlated.

The MLP-NARX model mathematically represented in Eq. (1) captures the temporal aspects of the data (Medsker & Jain, 2001).

$$\begin{aligned} y\left(t\right)&=f\left(y\left(t-1\right),\dots y\left(t-p\right), {x}_{m}\left(t-1\right),\dots {x}_{m}\left(t-a\right),\right. \\ & \quad \left.\, {x}_{n}\left(t-1\right),\dots {x}_{n}\left(t-b\right)+ e(t-1)\right)\end{aligned}$$

(1)

where $y\left(t\right)$ is the value of the time series to be predicted at the current time step $t$; ${x}_{m}\left(t-1\right)$ and ${x}_{n}\left(t-1\right)$ are the values of the exogenous inputs at the previous time step. In this work, $y\left(t\right)$ is the droplet size (or velocity) at the current time step ($t$), ${x}_{m}\left(t-1\right)$ is the amplitude of the magnitude component of the MW sensor signal at the previous time step $\left(t-1\right)$, and ${x}_{n}\left(t-1\right)$ is the corresponding phase component. The term $p$ is the autoregression order (Fig. 9), i.e., the number of time steps preceding the current time step in the time series being modeled, viz. droplet size (or velocity). Similarly, $a$ and $b$ are the exogenous orders [the number of time steps preceding the current time step (Fig. 9)] of the magnitude and phase components of the MW sensor signal. The terms $p$, $a$ and $b$ are tunable parameters of the model. The term $e\left(t-1\right)$ is the error associated with model at the previous time step. The machine learning task is to approximate the function $f(\cdot )$ to minimize $e\left(t\right)$.

Two different models are trained based on the MLP-NARX structure, one each for droplet size and velocity. Although the two droplet characteristics were derived from the same physical process, they have different relationships with the MW sensor data, and hence require different function approximations. Upon extensive hyperparameter tuning, we found that the values of NARX hyperparameters given in Table 2 yielded the best results. In other words, a naïve-grid search method was adopted on a heuristically selected range of hyperparameter values. Further, we a use MLP with a single hidden layer (Fig. 9) with 20 and 15 neurons for the droplet size and velocity prediction models respectively, and a rectified linear unit activation function. The cost function used to compute the prediction loss during training of the MLP was mean squared error.

Table 2 Optimum values of the hyperparameters used in the MLP-NARX models

(c) MLP-NARX model training and testing

The MLP-NARX models were trained and validated on the data acquired from Experiment 1.A and were tested on the data from Experiment 1.B (Table 1) as depicted in Fig. 8. Further, as mentioned in Sect. “Experiment 1: Data acquired for droplet size and velocity characterization”, the MLP-NARX models were tested for transferability on data acquired from Experiment 1.C (Fig. 8) in which the process parameters were significantly changed in comparison to Experiment 1.A. The training process refers to approximating the function $f(.)$ in Eq. (1) using the MLP. Thus, we performed training and testing of the MLP-NARX model on data from independent Experiments (1.A and 1.B, respectively), which attests to its generalizability and transferability to different data. During training, the MLP-NARX model is in the so-called open-loop mode shown in Fig. 9a, i.e., the output (droplet size or velocity) is obtained from the HSVC data and used as ground truth. The mathematical formulation of the MLP-NARX model in open-loop is identical to Eq. (1).

Fig. 8

Schematic representation of the data used for training, validating, testing and transferability testing the two MLP-NARX models used for droplet size and velocity prediction. The models were trained and validated on data acquired from Experiment 1.A and were tested on data acquired from Experiment 1.B. Subsequently, the transferability tests on the two models were performed on data acquired from Experiment 1.C

Fig. 9

Block network representation of an MLP-NARX model. a Open-loop mode which is used for the training phase. b Closed-loop mode that is used for validation and testing, wherein the predicted values of the time series are used as inputs. Here, h is the number of hidden layers and N is the number of neurons in it. Further, a and p are the autoregression orders of the MW sensor magnitude and HSVC sensor signatures, respectively. Note, the second exogeneous input, i.e., phase of MW sensor is not shown for simplicity

During the testing phase, the MLP-NARX model is switched to closed-loop mode (Fig. 9b). Here the predicted output (not the ground truth) is recursively fed into the model as shown in Eq. (2).

$$\begin{aligned}y{^{\prime}}\left(t+1\right)&=f\left(y{^{\prime}}\left(t\right),\dots y{^{\prime}}\left(t-p\right), {x}_{m}\left(t\right), \dots {x}_{m}\left(t-a\right),\right. \\ & \quad \left. {x}_{n}\left(t\right), \dots {x}_{n}\left(t-b\right)\right)\end{aligned}$$

(2)

where $y{^{\prime}}\left(t\right)$ is the droplet size or velocity predicted by the model at the previous time step. ${x}_{m}$ and ${x}_{n}$ are the magnitude and phase (exogenous inputs) from the MW sensor data, respectively.

To ensure model generalizability, during the training phase, the data from the Experiment 1.A is divided into model approximation and validation sets. To elaborate, 80% of the data is dedicated for model approximation and 20% for validation of the model. This division of data was done sequentially to maintain the temporal information in the MW sensor and HSVC time series. In other words, the first 80% of the data points in the time series (MW sensor and HSVC) are used for model approximation and the remaining time series is used for validation. Subsequently, the approximated and validated models are used to predict the droplet size or velocity from the data acquired from Experiment 1.B.

Droplet shape classification

For droplet shape analysis, we used the data acquired from Experiment 2 (Table 1). As shown in Fig. 5 (Sect. “Experiment 2: Data acquired for droplet shape characterization”), each processing condition in Experiment 2 results in droplets with distinct shapes. Hence, we characterize the shape of droplets in terms of one of the five distinct shape classes (labels). Although the DoD-LMJ process can result in numerous types of droplet shapes, this work focuses on the five shapes that could be generated in a controlled manner. A machine learning model was trained to classify these five labels using the MW sensor data as an input.

To elaborate, from Fig. 5 we observe that the magnitude and shape of the MW sensor data varies with the droplet shape. Hence, a Fourier transform was used to characterize the MW sensor signal in terms of magnitude and shape. The Fourier transform decomposes the stationary signal into its characteristic frequencies. Additionally, the strength of a characteristic frequency is denoted by its frequency coefficient. In this work, droplet-wise MW sensor signals were decomposed in terms of the six most pertinent frequency coefficients. Subsequently, these frequency coefficients were used as inputs to a machine learning model, i.e., support vector machine, to perform droplet shape classification.

The motivation to perform this analysis in the frequency domain over the time domain was the ability of the Fourier transform to efficiently resolve the signal and noise from the time series data. Furthermore, this signal can be decomposed into sinusoidal components at different frequencies whose magnitude can be used as input features in simple machine learning frameworks rather than extracting statistical features and using these within complex time domain-based neural networks, such as, recurrent neural network. In other words, the Fourier coefficients facilitate rapid characterization of the signal shape that are time independent. Lastly, the implementation of Fourier transform using the Fast Fourier transform algorithm significantly reduces computational cost in comparison to complex neural networks (Nussbaumer, 1981).

(a) Feature extraction from millimeter-wave sensor data

The first step was to resolve the MW sensor magnitude corresponding to a droplet. For this, the local minima in the MW sensor magnitude were located as it relates to the instance when a droplet is exactly in front of the MW-sensor waveguide, as shown in Fig. 10a (inset). Prior to this step, the MW sensor magnitude were de-trended with least squares regression method (Fig. 10a) (Brockwell & Davis, 2016). Subsequently, the local minima were extracted by first finding the data points that were smaller than data points preceding and succeeding them (Fig. 10b). Next, a hard threshold was used to pinpoint the instance when a droplet is exactly in front of the MW sensor waveguide (Fig. 10c). For further analysis, we selected 20 data points before and after each local minimum and designated that segment of the sensor signal to represent a particular droplet (Fig. 10d).

Fig. 10

Representation of the approach used to extract MW sensor signature for a single droplet. a Sample MW sensor data that was detrended using least squares regression (Brockwell & Davis, 2016). b The data points highlighted in black are the identified local minima in the MW sensor signal. c The datapoints highlighted correspond to the temporal location when a droplet is exactly in front (center of droplet path) of the open-ended waveguide as shown in the inset. These datapoints were obtained by thresholding the MW sensor signal with a heuristically determined limit. d An example of the MW sensor signal belonging to a single droplet

After isolating the MW sensor signatures corresponding to individual droplets, Fourier analysis was performed to decompose the MW signal into its characteristic (spectral) frequencies. The Fourier spectral frequencies were used as features in machine learning models which were trained to classify the droplet shape. To avoid overfitting, the six most relevant frequency coefficients were selected to reduce the machine learning model complexity. The maximum relevance and minimum redundancy (MRMR) approach was used for identifying these frequencies (Radovic et al., 2017). The mathematical formulation of MRMR is summarized in “Appendix C”. The following six Fourier frequencies were determined to be the most ideal in capturing the droplet shapes: 0 Hz (signal mean), 62.5 Hz, 125 Hz, 187.5 Hz, 250 Hz, 562.5 Hz.

(b) Machine learning models used for droplet classification

The droplets were classified into the five different shapes shown in Fig. 5 using spectral features of the MW sensor data. Baseline classification was first done using conventional statistical models, such as, ridge regression-based (regularized least squares) classifier and nonlinear multinomial logistic regression classifier. These statistical classification models are based on ordinary least squares minimization, and do not use active learning phase (Dobson & Barnett, 2018; Rifkin & Lippert, 2007). The inadequate performance of these models motivated the use of slightly more complicated yet interpretable support vector machine classifier. A total of 345 droplets were used for the machine learning models out of which 80% (276 droplets) were used for training and the rest (69) for testing. The F1-score was used as our performance metric as it considers both the Type I (false alarm) error and the Type II (failing to detect) error (Montazeri et al., 2019).

Results and discussion

In this section, the droplet size and velocity predicted using the MLP-NARX model are described in Sect. “Droplet size and velocity prediction using MLP-NARX models”, and the droplet shape predicted based on the spectral features are described in Sect. “Droplet shape classification”.

Droplet size and velocity prediction using MLP-NARX models

Droplet size prediction

Figure 11 shows the results from Experiment 1. In Fig. 11 (a1), the droplet size predictions for Experiment 1.A using the MLP-NARX model with the MW sensor data are overlaid on those observed from the HSVC (ground truth). Figure 11 (b1) and (c1) are the regression and comparative discrete probability distribution plots, respectively, both of which indicate a fit exceeding ${R}^{2}$ = 0.95 (regression). The tight fit of predicted data points around the perfect fit line (red) in Fig. 11 (b1), signifies a low sum of squared error and lack of bias in the predictions. Figure 11 (c1) affirms that the model does not overestimate nor does it underestimate the droplet size, i.e., the predictions are not biased towards one droplet size. This result is further affirmed by the residual analysis of the MLP-NARX model as elucidated in Fig. 16a in “Appendix B1”.

Fig. 11

Droplet size prediction on data from Experiment 1.A and 1.B using MLP-NARX model. a1 and a2 The predicted values of the size of droplets are overlaid with the actual values from Experiment 1.A and 1.B, respectively. b1 and b2 Regression plot showing the droplet size prediction on data from Experiment 1.A and 1.B, respectively. The predicted values mostly lie along the linear line thereby suggesting that the MLP-NARX model yields low prediction error. c1 and c2 Histograms of the actual and predicted values of droplet size from Experiment 1.A and 1.B, respectively

This MLP-NARX model was then re-tested as-is (without changing any model parameters) on the data acquired from Experiment 1.B, wherein the crucible pressure setpoint was changed to 2 psig from 3 psig. The corresponding results presented in Fig. 11 (a2), (b2) and (c2) demonstrate the validity of the model for an independent dataset. The regression ${R}^{2}$ reduces to 0.85 due to few outlier misclassifications of large magnitude. For example, in Fig. 11 (b2) approximately 20 data points (out of 1000 data points) are predicted to have size of 0 mm² when the observed size was larger. This level of misclassification is reasonable as the model is not trained on data from Experiment 1.B. This result shows that the model performance does not degrade drastically when used on a different data set. In other words, the model maintains an appreciable degree of generalizability (transferability) to different processing conditions in DoD-LMJ. Residual analysis of the MLP-NARX model is detailed in “Appendix B1”. The residual results affirm that the model has no significant prediction bias, and devoid of overfitting.

Droplet velocity prediction

Continuing with the analysis, we applied an MLP-NARX model to predict droplet velocity from Experiment 1. We follow the same approach discussed for droplet size prediction in Sect. “Droplet size prediction”. The MLP-NARX model was trained and validated on data acquired from Experiment 1.A and tested on data from Experiment 1.B. The results for Experiment 1.A presented in Fig. 12 (a1), (b1) and (c1) show that the model predicts the droplet velocity with ${R}^{2}$ ~ 0.93. Remarkably, when the same model (without any further training) is transferred to data from Experiment 1.B, the prediction fidelity remained around ${R}^{2}$ ~ 0.90. As was the case droplet size, the MLP-NARX does not show prediction bias for droplet velocity. These results are further affirmed by the residual analyses of the MLP-NARX model in “Appendix B1”.

Fig. 12

Droplet velocity prediction on data from Experiment 1.A and 1.B using MLP-NARX model. a1 and a2 The predicted and actual values of the droplet velocities overlaid from Experiment 1.A and 1.B, respectively. b1 and b2 Regression plot showing the droplet velocity prediction on data from Experiment 1.A and 1.B, respectively. The predicted values mostly lie along the linear line thereby suggesting that the MLP-NARX model yields low prediction error. c1 and c2 Histograms of the actual and predicted values of droplet velocity from Experiment 1.A and 1.B, respectively are overlayed

MLP-NARX model robustness and transferability

We performed further experimental tests to affirm the transferability of MLP-NARX models with data acquired from Experiment 1.C (Sect. “Experiment 1: Data acquired for droplet size and velocity characterization”). Figure 13 (a1) and (a2) reports the results of droplet size prediction for this new experiment. From Fig. 13 (a1) it is apparent that the model captures the general trend of the actual droplet size, with a small bias. Further, the prediction accuracy of the model reduced to ${R}^{2}$ = 0.65 [Fig. 13 (a2)] from ${R}^{2}$ = 0.98 (Experiment 1.A) and ${R}^{2}$ = 0.85 (Experiment 1.B). This can be attributed to the large change in inner nozzle diameter to 100 µm from 400 µm (Experiment 1).

Fig. 13

Transferability test results of MLP-NARX models. Droplet size (a1 and a2) and droplet velocity (b1 and b2) prediction results using the MLP-NARX model. The models were trained on data from Experiment 1.A and applied to an experiment with drastically different process conditions (e.g., nozzle size was changed to 100 µm from 400 µm). The models can predict the droplet size with a prediction accuracy of R² = 0.65, and droplet velocity with R² = 0.81

Previous studies have shown that the inner nozzle diameter has a significant effect on the droplet size (Amirzadeh et al., 2013; Castrejón-Pita et al., 2008; Cheng et al., 2005; Lee et al., 2008a; Poozesh et al., 2016; Sohn & Yang, 2005; Zhong et al., 2017). In contrast, the MLP-NARX model for droplet velocity prediction yielded an ${R}^{2}$ ~ 0.81. There is a good agreement between the predicted and actual droplet velocity profiles as shown in Fig. 13 (b1). Additionally, the model predicts the droplet velocity with a satisfactory statistical fidelity of ${R}^{2}$ = 0.81. This result is validated by the regression plot shown in Fig. 13 (b2).

The droplet velocity predictions are sustained better than droplet size correlation between the two experiments, because the nozzle size has a more significant on droplet size than droplet velocity. The performance of the MLP-NARX can be enhanced by performing hyperparameter tuning of the models before deploying them on a dataset from an experiment with vastly different process conditions. However, the model architecture remains the same.

Droplet shape classification

We classified the droplet shapes into five types using data obtained from the Experiment 2 (Table 1). The labels for the classification task were extracted from the HSVC data, and the spectral features were obtained from the MW sensor data as explained in Sect. “Droplet size prediction”. The spectral features were used as inputs to a support vector machine (SVM) to classify the shape of droplets into five different types (Fig. 5). The use of machine learning was motivated by the sub-par performance of two statistical models, namely, ridge regression-based (regularized least squares) classifier and nonlinear multinomial logistic regression classifier compared to the support vector machine model. Table 3 shows that the support vector machine model classifier (machine learning model) outperforms the two conventional statistical models.

Table 3 Comparison of the statistical fidelity of different classification models used to classify droplet shape in terms of the F1-score

To reiterate, we evaluated the performance of all the models by randomly sampling the training and testing datasets 10 times. The confusion matrix depicting the prediction fidelity of the support vector machine model in one of the ten iterations is shown in Table 4. Out of the 69 droplets used for testing, only one was misclassified. Furthermore, Table 5 elucidates the precision, recall and F1-score values calculated from the confusion matrix given in Table 4. Lastly, the learning-curves-based analysis suggests that the support vector machine model optimally estimates the underlying function without over-or-underfitting, and the model does not require more data to improve its performance. A detailed explanation of this analysis is given in “Appendix B2”. These results indicates that the MW sensor can detect abrupt changes in droplet shape that occur due to process drifts.

Table 4 Confusion matrix of droplet shape classification using support vector machine. This confusion matrix is of one of the ten iterations performed to test the fidelity of the model

Table 5 Precision, recall and F1-score values of the support vector machine model calculated from the confusion matrix given in Table 4

Conclusions and future work

This work develops and applies machine learning algorithms on data obtained from a novel millimeter-wave (MW) sensor for in-process characterization of the size, velocity, and shape of droplets in the droplet-on-demand liquid metal jetting (DoD-LMJ) additive manufacturing process. The machine learning models studied in this work predicted the droplet characteristics with an accuracy exceeding 90% using MW sensor data as inputs. These results demonstrate the applicability of the MW sensor for real-time monitoring of droplet characteristics, providing an avenue for the replacement of data-intensive image-based monitoring with tractable time series signatures, and thus paving the way for closed-loop process control.

Specific conclusions are as follows:

• 1. To predict the droplet size and velocity, we devised a multilayer perceptron-based non-linear autoregression (MLP-NARX) model. The model encapsulates the temporal dynamics of the process and can predict the droplet size and velocity with a statistical fidelity (measured in terms of the regression ${R}^{2}$) of 95% and 93%, respectively.

• 2. Multiple transferability tests were performed by applying the MLP-NARX models to independent experimental datasets conducted with different process parameters. In the first transferability test, only one process parameter was changed (setpoint pressure of the crucible) and the MLP-NARX models performed satisfactorily yielding regression ${R}^{2}$ values of 85% and 90% for droplet size and velocity prediction, respectively. In the second transferability test, a wide range of process parameters were changed, and the MLP-NARX models yielded ${R}^{2}$ values of 65% and 81% for droplet size and velocity prediction, respectively.

• 3. To classify the shape of droplets, we use the characteristic (spectral) frequencies of the MW sensor signature obtained from Fourier analysis. Using six identified characteristic frequencies within a support vector machine model, the droplet shapes were predicted with statistical fidelity (F1-score) exceeding 95%. Furthermore, this classification accuracy was observed to be significantly higher than conventional statistical models.

The overarching goal of this work was to minimize defects of DoD-LMJ parts. In pursuit of this goal, in our future work, we will endeavor to understand the effect of processing parameters on not only the droplet characteristics, but also the part quality, in terms of its geometric integrity, microstructure and functional behavior. While this work was focused on monitoring the droplet characteristics prior to deposition, our next step will be to predict flaw formation and build failures in DoD-LMJ as a function of the droplet characteristics. This future work will involve building actual parts, followed by ex-situ characterization, such as X-ray computed tomography and scanning electron microscopy of parts to study the effect of varying process conditions on their geometric integrity and microstructure. Subsequently, various time series analysis models will be explored to establish the vital link between part quality and in-process MW sensor signatures.

Appendices

Appendices

Appendix A: Preprocessing of the in-process sensor data

Appendix A1: Droplet labeling strategy used for the HSVC data

The object detection on the droplets was performed using the 8-connectivity approach (Rosenfeld, 1970). This technique looks for connectivity in a pixel depending on its neighboring pixels, i.e., pixels connecting a given pixel along its four edges and four corners. Once the droplets were detected, each of them was assigned a label as shown in Fig. 7d. The droplet furthest away from the nozzle was labeled the first object, followed by the next closest droplet to the nozzle was the second object, and so forth (Fig. 7d). Droplets that were attached to the nozzle (‘Object 3’ in Fig. 7d) were excluded from this analysis as studying their characteristics is not critical to this study. To elaborate further, the droplets attached to nozzle have a minimal (close to no) effect on the MW sensor readings, hence they were ignored.

Appendix A2: Approach adopted to compute droplet velocity from HSVC data

Below given is the mathematical formulation of the approach followed to compute the velocity of droplets.

$${\delta }_{i}^{j}\left({n}_{c},{d}_{c}\right)= \sqrt{{({n}_{c}^{y}- {n}_{c}^{x})}^{2}+{({d}_{c}^{y}- {d}_{c}^{x})}^{2}}$$

(3)

where, ${\delta }_{i}^{j}\left({n}_{c},{d}_{c}\right)$ is the Euclidean distance between points ${n}_{c}$ and$,{d}_{c}$; $,{n}_{c}$ is a point at the tip of the nozzle, and ${d}_{c}$ is the centroid of droplet $j$ in frame $i$ (Fig. 14b). It must be noted that we track the distance travelled by droplets and hence the velocity of droplets in each region-of-interest (ROI) as shown in (Fig. 14b). We studied the velocity of droplets in the ROI that is towards the end of the HSVC frame to monitor the terminal velocity of the droplets.

Second order forward finite difference:

$${\delta }_{i}^{j{^{\prime}}}= \frac{(-3{\delta }_{i}^{j}+4{\delta }_{i}^{j+1}- {\delta }_{i}^{j+2})}{2}$$

(4)

Second order backward finite difference:

$${\delta }_{i}^{j{^{\prime}}}= \frac{(3{\delta }_{i}^{j}-4{\delta }_{i}^{j-1}+{\delta }_{i}^{j-2})}{2}$$

(5)

Second order centered finite difference:

$${\delta }_{i}^{j{^{\prime}}}= \frac{({\delta }_{i}^{j+1}- {\delta }_{i}^{j-1})}{2}$$

(6)

Fig. 14

Schematic representation of the droplet characteristics extracted from the high-speed video camera. a Size of droplets is given in terms of area of the droplets, b velocity of droplets is computed by tracking the distance travelled by a droplet with a given region of interest (highlighted). ${n}_{c}$ and ${d}_{c}$ are the centroids of the nozzle and droplet, respectively. ${\delta }_{i}^{j}\left({n}_{c},{d}_{c}\right)$ and ${\delta }_{i+1}^{j}\left({n}_{c},{d}_{c}\right)$ are the Euclidean distance between nozzle and droplet centroids in HSVC frame $i$ and $i+1$, respectively

Appendix A3: Down-sampling procedure for the millimeter-wave sensor data

The MW sensor outputs time series data at high temporal resolution as shown in Sect. “Experiment 1: Data acquired for droplet size and velocity characterization”. Given its high sampling rate in comparison to the high-speed video camera (HSVC), the MW sensor data was down-sampled to register with the HSVC data. To do so, we split the MW sensor data into non-overlapping windows with 8 data points per window [Fig. 15 (a2)]. A segment of the down-sampled data is shown in Fig. 15 (a3). Subsequently, the mean of each window was computed and was considered as a representative data point. Lastly, the down-sampled data was then truncated to match the temporal length of the HSVC data at 4.362 s, as shown in Fig. 15 (a1).

Figure Fig. 15b shows a sample registration between the two data streams (MW sensor and HSVC data). It is apparent that there is a lag in the MW sensor data in comparison to HSVC data. This lag manifests from the difference in the spatial location of the two sensors. As shown in Fig. 2, the MW sensor is placed below (closer to the substrate) the HSVC; as a result, the MW sensor observes the falling droplets fractionally later than the HSVC. This delay in observing the droplets by the MW sensor leads to the lag seen in Fig. 15b.

Fig. 15

a1 Truncation of the MW sensor data up to 4.3672 s to match the length of the HSVC data. a2 non-overlapping window-based down-sampling approach used on the MW sensor data, and a3 sample down-sampled data. b Overlaid normalized readings of the MW sensor and HSVC data. A slight lag between the two data streams is evident which is due to the location of the two sensors relative to each other

Appendix B: Machine learning model diagnoses

The machine learning models proposed in this work were developed on a machine with the following specifications: Intel i7-7700HQ CPU @ 2.80 GHz and 16 GB RAM. Open-source Python libraries, such as, Scikit-learn and PyNeurGen were used build the algorithms for the machine learning models.

Appendix B1: Residual analysis of MLP-NARX models

Residual analysis was performed to evaluate the performance of the two MLP-NARX models that were trained to predict droplet size and droplet velocity. To reiterate, the MLP-NARX models were trained and validate on data acquired from Experiment 1.A and were tested on data acquired from Experiment 1.B.

Fig. 16

Residual analysis of MLP-NARX model used for predicting droplet size. a Residual plot of data acquired from Experiment 1.A, and b Residual plot of data acquired from Experiment 1.B

Figure 16a and b show the residual plots of the validation and testing of MLP-NARX model used for predicting droplet size, respectively. These residual plots suggest that the residuals (errors) are random and normally distributed with the center (mean) zero. This indicates that MLP-NARX model does not overfit to the training data and is generalizable to unseen datasets. Similarly, residual plots shown in Fig. 17a and b suggest that the MLP-NARX model used to predict droplet velocity does not overfit the training data.

Fig. 17

Residual analysis of MLP-NARX model used for predicting droplet velocity. a Residual plot of data acquired from Experiment 1.A, and b Residual plot of data acquired from Experiment 1.B

Appendix B2: Fit analysis of the SVM classifier

A SVM model was trained to classify the droplet shape using spectral features extracted from the MW sensor. The fit (over-or-underfit) analysis of this model was performed using the learning curves. Learning curves are a widely used diagnostic tool in machine learning algorithms that learn from a training dataset incrementally. To elaborate as shown in Fig. 18, the SVM model was trained and validated on 60 data points in the first iteration. Subsequently, 1 data point was added to the dataset in the next round of training and validation. The training and validation fidelity (F1-score) is recorded for all the iteration and plotted. Figure 18 suggests that the performance of the SVM model (in terms of the validation score) increases as the training data size is incrementally increased. But the model’s performance stops improving when approximately 380 data points are used for training. This suggests that the size of training dataset (386) used in this work is optimal for the droplet shape classification. The saturation of the model performance towards the end also suggests that the model has accurately estimated the underlying function and is not over-or-underfitting.

Fig. 18

Learning curves of the support vector machine (SVM) used to classify droplet shapes. The SVM is able to satisfactorily estimate the underlying function, i.e., it does not over-or-underfit the training data

Appendix C: Feature selection using minimum redundancy maximum relevance (MRMR)

The MRMR approach works on the concept of mutual information (MI), in which features (frequency coefficients in this work) are ranked in order of highest to lowest relevance to the response variable, and lowest to highest amount of redundancy that they add to the feature set. Redundancy and relevance of a frequency coefficient are given below in Eq. (7).

$${W}_{{F}_{n}}= \frac{1}{\left|S\right|}\sum_{F \in S}I({F}_{n}, F)$$

(7)

$${V}_{{F}_{n}}=I({F}_{n},y)$$

where, ${W}_{{F}_{n}}$ and ${V}_{{F}_{n}}$ are the redundancy and relevance of a frequency coefficient ${F}_{n}$, respectively. $S$ is a subset of features from the entire feature set, and $F$ is any other feature other than ${F}_{n}.$ Next, $y$ is the response variable which in this work is a categorical variable that represents the five different shapes of the droplets. Lastly, $I$ is the mutual information which is mathematically expressed in Eq. (8).

$$\begin{aligned}I\left(A,B\right)&= {\sum\limits}_{i,j}P\left(A={a}_{i},B={b}_{i}\right)\\ & \quad \times \mathrm{log}\left(\frac{P\left(A={a}_{i},B={b}_{i}\right)}{P\left(A={a}_{i})P(B={b}_{i}\right)}\right)\end{aligned}$$

(8)

where $A$ and $B$ are two random variables. $P\left(A,B\right)$ is the joint distribution of the two random variables, and $P\left(A\right)$, $P\left(B\right)$ are their marginal distributions. Next, each frequency coefficient was ranked using the Mutual Information Quotient (MIQ) which is given in Eq. (9).

$${MIQ}_{{F}_{n}}=\frac{{V}_{{F}_{n}}}{{W}_{{F}_{n}}}$$

(9)