Introduction

As an emerging material process, additive manufacturing (AM) continues to increase in industry adoption due to its versatility to realize novel geometries, provide highly tailorable object performance, and reduce costs by virtue of digital dependencies and automation (Ref 5, 7, 72). These realizations are not only highly dependent on precision within the manufacturing process parameters, but also require tight control of the AM feedstock material as well (Ref 28, 45). It has been well-documented that low-quality AM feedstock can reduce predictability within the fabrication process (Ref 76, 81) and account for a large contribution to the inertial error that engineers account for within the design-for-manufacturing process (Ref 3, 12, 72). This has the knock-on effect of frequently burdening AM designs with relatively large factors of safety and redundancies (Ref 26, 63, 80), numerous iterations (Ref 21, 27, 79), and proof testing (Ref 13, 29) to accommodate the lack of engineering authority (Ref 18, 50, 57). Consistently generating and testing material coupons are critical to characterizing (Ref 12, 22) and maintaining (Ref 42, 65) the AM process; however, this itself is both cost and time inefficient due to the relatively slow and laborious nature of fabricating materials and testing them on other platforms (Ref 15, 43). Sensing platforms have been integrated into AM as a method to control the processing parameters in situ, such as by Ref 2, 74 and 77; however, these methods provide difficulty with maintaining the tractability of errors within the material process (Ref 20). Reference 16 and 40 specifically note the difficulty of correlating AM directly to material qualities and reference the need for more research in this area.

This paper presents a method that assists AM characterization by leveraging automation and machine learning to quickly generate and test large sets of data for automated testing and characterization of additive manufacturing (ATCAM). The development of the method required the ability to generate and sense large amounts of empirical AM data. To attain this, the concept of a dynamic coupon and in situ actuator (Ref 48) are demonstrated, which allow AM material to be automatically fabricated and ejected sequentially for a high-throughput FDM process. Ejected coupons impact a load cell to generate a data point. Logging and repeating the process automatically allows for the rapid accumulation of a database, enabling many automated applications. This includes systematically identifying a feedstock’s quality control, standardizing the calibration of a machine’s processing parameters, understanding the sensitivities of an AM material under various conditions, recognizing a machine that is drifting from its intended material process, determining if a material is from a specific vendor or origin, embedding and detecting counterfeiting protocols, potentially adapting a design for its environment through repeated iteration and exploration of design space, among many others.

To test ATCAM’s utility, three polylactic acid (PLA) feedstocks at differing price-points and effective quality were utilized to generate, deploy, and capture data from 3000 dynamic coupons in batches of 1,000 coupons per feedstock, conducted with 100 coupons per in situ actuator. The machine utilized was an open-source and widely available Prusa MK3S. Coupon impact force was logged through a load cell connected to an analog-to-digital converter attached to a microcontroller. The data were analyzed, correlated, and compared through three machine learning algorithms (deep learning, support vector machine, and boost gradient) supported by cloud computing with the DisplayR platform. At the time of writing, the entirety of this AM material testing setup costs less than USD $1000. This made the test widely adoptable, portable, low risk, and highly cost-effective. Section 2 provides the methods utilized for creating ATCAM, with section 3 presenting the data generated by the system and the machine learning correlations. Section 4 provides an analysis of the results, potential sources of error within the data, and discusses ATCAM’s main findings. The paper concludes in section 5 by over-viewing the novel processes ATCAM introduces to AM testing and characterization within the limitations of the current configuration and provides potential next steps to improve the system.

ATCAM’s Framework, Method for Implementation, and Testing Procedure

The concept of ATCAM was developed to integrate AM fabrication and testing into a single consolidated process. Figure 1 provides an overview of the system’s components, configuration, and process flow to automate the translation of AM feedstock into characterized data. In the following section, we present the system’s framework and components in detail by their subsystem, describe how the platform functions, discuss the data output and provide the testing procedure used to analyze ATCAM’s performance.

Fig. 1
figure 1

Rendering of ATCAM setup, including process framework. (A) is the feedstock material, (B) is the 3D printer, (C) is the in situ actuator after deploying the dynamic coupon (visualized in (1 - 3) chronologically in the top left), (D) is a dynamic coupon under deployment following the black trajectory to impact the load cell, (E) is the load cell unit for physical load sensing of the impact of the dynamic coupon, (F) is a Wheatstone bridge equipped with a 24-bit analog-to-digital converter, (G) is an Arduino Uno to read and log the Wheatstone bridge data, and (H) is a Raspberry Pi 3B to analyze and correlate the data

Full size image
Fig. 2
figure 2

Technical drawing of the dynamic coupon. All dimensions given are in millimeters. Note A is a cavity in the base to reduce the surface area in contact with the build plate and ISA force required for deployment. Features B, C, and D describe a channel embedded within the dynamic coupon to reduce the material consumption and printing time per coupon. Feature B is the interface between the base and the channel in Feature C. Feature D is a cut that removes the need for the support structure which would be required if the channel was enclosed within the coupon. Feature E is the base on the bottom of the coupon which creates a tailored surface area and aids object deployment. Feature F demonstrates the base’s external geometry, notably an incorporated radius to reduce the overhang angle from the base to the rest of the coupon to increase manufacturability

Full size image

AM Platform

The additive manufacturing process utilized to develop ATCAM was FDM. This decision was primarily to increase the applicability of the platform, as FDM is a widely accessible (Ref 34, 66, 70) and relatively mature AM process (Ref 23, 36, 55). Additional consideration for selecting FDM was the wide variety of vendors available to source feedstock materials as demonstrated by Ref 69, the relative low cost of the platform (Ref 15), and the flexible nature of both the machine language (G-code) (Ref 8) and its firmware (Ref 36, 57, 66). Due to the open-market nature of FDM, a wide variety of materials are available for the process (Ref 33, 49, 64). The quality control of these materials typically varies significantly with price point and origin (Ref 28), making characterization of FDM materials a highly relevant challenge (Ref 12, 54, 65). Of the FDM machines available on the market, the Prusa MK3S (Ref 59) was selected due to the robustness of the machine (Ref 67), portability, cost (Ref 10), and ease of reconfiguration (Ref 47).

Dynamic Coupons and In Situ Actuation

Developing a method to automate a high throughput characterization of coupons, in particular, was not trivial; it proved to be the most difficult task in creating ATCAM. This was primarily due to the lack of reference, as a literature review concluded that there is no practical integrated solution to fully automate the creation and testing of AM coupons. The association of mechanical properties with AM processing parameters has been documented in reviews by Ref 62 and 46. Works by Ref 45 and 12 provide the critical need for consistent AM coupon testing to control mechanical properties and process quality, with (Ref 31) providing a novel strategy to use ultrasound for AM control. Reference 68 discusses a method to integrate printed circuitry into structures for AM part health monitoring and crack detection; however, this method of testing was not demonstrated as directly applicable for determining the quality of AM materials or processes and is highly object-specific. These works demonstrate that although constant testing is crucial for increased industry adoption of AM, there currently exists a testing throughput limitation for AM characterization due to lack of automation.

The premise of automation by in situ actuation has been previously demonstrated through programmed interactions between AM platforms and printed parts, such as by Ref 4 and 39. In both examples, the modifications to the printers are not purely software-based, as physical modifications are made to machines to provide new capabilities. Our method allows for the retrofits to be printed and manipulated entirely in situ through programmed revisions. We wanted to keep to remove the need for physical modifications that were ex situ to reduce the inertial error which can accumulate into the system. This is especially important for deducing the origination of an error source. Additionally, extensive physical modifications limit the adoption and portability of the system. Reference 6 provided insights into functional AM leaf springs, including associations regarding their deflections and energy storage. The springs in these results were not automated, nor were they deflected on the AM platform themselves, but they were referenced as a validated method for printed energy storage.

We developed dynamic coupons to address the technological gap between these different fields, allowing a large quantity of AM objects to be printed and tested entirely in situ. The process works by printing an in situ actuator (ISA) onto the build surface (Ref 48). This allows for the printer to print and pull back the ISA on the build platform to eject objects and clear the surface for further parts. ATCAM leverages this by printing a 25.4mm (1-inch) sphere onto the build surface, utilizing the tool head and ISA to transfer energy to the coupons through high-force impact. This ejects and transports them to a load cell which detects the force of the coupon’s impact. The cleared build surface is then used to print a subsequent dynamic coupon, and the ejection process is repeated. This process is repeated 100 times per ISA before manual removal, which was the experimentally determined non-destructive lifespan (Ref 48). One cycle of this process is visualized in the top-left (1-3) of Fig. 1.

The dynamic coupon was designed as a sphere for compatibility with the ISA to achieve reliable deployment within the 2N of force generated. A base was implemented on the bottom of the coupon to provide a tailored surface for deployment. This base also increased the coupon’s manufacturability, as the surface area of the sphere’s base was too small for the printer’s configuration. The base’s edges were integrated onto the dynamic coupon through a radius to decrease the overhang angle and increase manufacturability. An infill density of 20% was used in a rectangular configuration to increase the platform’s throughput and minimize thermal issues occurring from dense raster patterns. This resulted in the coupon having a designed mass of 4 grams. A diagram of the dynamic coupon is provided in Fig. 2.

Physical Sensing and Data Logging

An aluminum 1kgf-rated single-ended shear loading cell was utilized to collect dynamic coupon impact force through contact. The load cell was calibrated against the manufacturer datasheet, providing measurement linearity of +/-0.5%. The process flow for data logging is shown in Fig. 3. After a coupon is ejected, it impacts the load cell. This is registered as strain; the change in resistance is fed into a Wheatstone bridge and is digitized by a 24-bit analog-to-digital converter. This digital data are then read by an Arduino Uno microcontroller. The Arduino saves the value to a text file, formatting the coupon’s data as a force within a row of comma-separated variables including the timestamp and the number within the test cycle. After logging these values, the microcontroller then waits for the next data point. Figure 3 demonstrates that a minimum force threshold is imposed onto the load cell’s reading to reduce the likelihood of erroneous data occurring from ATCAM’s environment; a minimum of 1N force was required to log a data point as a dynamic coupon.

Fig. 3
figure 3

Process flow diagram of ATCAM’s physical sensing capability. The data file generated from this process contains the load cell readings and time stamps of each impact, enabling force readouts. Temporal and force resolution is dependent on hardware

Full size image
Fig. 4
figure 4

Histogram demonstrating ATCAM’s frequency distribution over 1000 dynamic coupons within each feedstock. The average force and standard deviation of force ATCAM generated from each feedstock are provided

Full size image

As seen in Fig. 1, the load cell is relatively small at 80.0mm x 12.7mm x 12.7mm (3.15x0.50”x0.50”), so a solution was required to ensure that the dynamic coupons would reliably and repeatably be registered. The surface area sensed by the load cell was increased by affixing an acrylic plate. This provided a highly reliable target for the dynamic coupons. The modified load cell was placed within the average deployment trajectory, shown in Fig. 1.

Machine Learning and Cloud Computing to Analyze Dynamic Coupon Data

To analyze the information from the deployments at the end of a testing cycle, the dynamic coupon data logged by the Arduino Uno is captured by a Raspberry Pi 3B due to its networking and processing capabilities. Data are converted from comma-spaced variables accumulated by the text document into columns of data sets associating dynamic coupon forces and uniquely identifiable information by their deployment number from their testing cycle. This information is formatted in association with each feedstock OEM, vendor name (where applicable), and batch number (e.g., feedstock material, vendor 1, batch 1). The consolidation of these identifiers serves as ATCAM’s database, which allows for dynamic coupons to be correlated with feedstock information.

Analysis of ATCAM’s database relies on machine learning (ML) algorithms to systematically associate intrinsic material information with extrinsic parameters. This process allows for larger sets of data with altered parameters to be paired to differences within the intrinsic data (Ref 53, 56). These correlations generally benefit from analysis through statistical regression (Ref 58, 61) and are consequently adequate methods for analyzing (Ref 60) and correlating otherwise complex multi-variable properties from lower-dimensional values. An example of this form of analysis is the printed electronics characterization demonstrated by Ref 52. They compared multiple ML algorithms such as decision trees, support vector machine, and logistical regression for appropriateness to their low dimensional AM and electronics data. Reference 25’s overview of the status of ML for AM specifically notes the emerging need for automated data handling and libraries for in situ processes, such as the method by Ref 17. Additionally, Goh references ML’s utility for AM material characterization (Ref 25). ML’s usage in AM extends into toolpathing considerations within the work of Ref 14), where optimized orientations for fabricating scaffolding are considered. Similarly, (Ref 19) investigates the applicability of ML in the loop for optimized AM of medicines. These papers discuss the ML link between AM characteristics and the final object’s performance; however, both processes have a relatively small data library to pull from compared to ATCAM and consequently do not fully leverage the capabilities of linear ML models.

Fig. 5
figure 5

Scatter plot demonstrating ATCAM’s fatigue distribution of dynamic coupons within each feedstock. The data are presented with a trendline and error bars correlating to each feedstock’s average force over the index

Full size image
Fig. 6
figure 6

The prediction-accuracy tables are presented comparing various machine learning results correlating feedstock 2’s automated dynamic coupon data to its database. The tables present a heat map to visualize the frequency in which a predicted measurement was observed and correlated successfully within the data. The accuracy is provided below each table to demonstrate the precision of the respected method for correlating 100 dynamic coupons to its raw feedstock allowing for automated characterization

Full size image

ATCAM Testing Procedure

An initial test of ATCAM is provided to gauge the automated platform’s AM characterization performance and identify potential error sources for improvement. Three 1.75mm PLA feedstocks from different manufacturers were tested by the system in 10 batches of 100 dynamic coupons per feedstock. PLA dictated a standard 215 °C, 0.4 mm extruder and 60 °C build surface temperature. The toolpath was configured for 0.2 mm layer height with 20% rectangular infill due to the thermal considerations previously noted. The resulting settings provided an average volumetric flow rate of \(5mm^3/s\) at 2 mm/s feedstock extrusion speed. This created a dynamic coupon that required an average of 4.5g of material and took approximately 30 minutes to fabricate. The initial cycle with the ISA generated a toolpath which required 82.5 min for the first test cycle and a subsequent cycling frequency of about 2 dynamic coupons tested per hour, which made the testing regime require a little over two days of automated fabrication and testing per 100 coupons generated. The three feedstocks were selected by their price points, USD$60/kg, USD$30/kg, and USD$20/kg, identified as Feedstock 1, 2, and 3. To gain a basic understanding of the generic quality control parameters taken by the feedstock OEM’s, the feedstocks were each measured for filament diameter consistency at 30 points taken at random along the spool during the AM process with a Mitutoyo 500-464 dial caliper of 5-micron tolerance. Weight was measured per each 1kg quoted spool of filament with an OHAUS V31XH2 bench scale, which had 0.1-gram readability. These two parameters were utilized due to their importance in the FDM process; filament diameter consistency creates variability of mass flow rate in the extrusion process (Ref 11, 24). The considerations of these filament issues are consolidated and outlined in Table 1. The weight was measured and taken for the average filament diameter to understand the material density to provide an indicator of the material’s moisture content. This parameter is one of the leading sources of print variance in FDM (Ref 37), as water vapor cavities are formed in the FDM tool head, causing sputtering, the variability of mass flow rate, and inconsistent print results (Ref 41, 44, 51). To provide an additional metric for material performance, the surface roughness of three dynamic coupons fabricated from each feedstock was measured with a digital profilometer, the Sunnen SP-211. As per (Ref 73)’s review of surface texture metrology, conducting this profile topography with a contact stylus provides a measurement that is both widely accepted and utilized in both industry and academia. The testing scheme followed along the top of the coupon and recorded the roughness every 5 mm of the height for a total of five data points per test. Each test was repeated 10 times, generating 150 data points per feedstock. These results are provided in Table 2. Basic mechanical properties of the three feedstock were derived by conducting a three-point bending test to the ISO 178:2019 standard for the determination of flexural properties of thermoplastics. Three 80mm x 10mm x 4mm coupons for each feedstock were fabricated to the standard in a planar orientation, with the longest axis printed parallel to the build-plate. The printing parameters were taken as the same as the dynamic coupons. The bending test was conducted on a Chatillon TCM 201, fitted with a 2.22KN Interface 1110AF-500 strain gauge. The loading rate was set to .3175 millimeters per minute. The fixturing was fitted with 6.35mm diameter pins, with the base set to 25.44 mm apart to follow a conventional 16:1 ratio for the length of the tooling to the specimen’s thickness, respectively. The tests’ results were taken from a customized lab-view interface to gather physical load, time, and displacement. These measurements were used to calculate properties for each feedstock, with the results consolidated in Table 2.

Table 1 Various filament issues and their potential correlations to the dynamic coupon data spread
Full size table
Table 2 Measured feedstock properties
Full size table

Force measurements from the 3000 dynamic coupons were collected through the deployment and sensing process described above and processed through three different machine learning algorithms for comparative purposes. Deep Learning, Support Vector regression, and Gradient Boost regression were selected to provide a diverse assessment of the data. Due to the computationally intensive nature of Deep Learning, we decided to process the data through cloud computing. The Raspberry Pi leveraged the DisplayR cloud platform through an internet connection due to its simplicity, accessibility, and visualization capabilities. This process allowed for the intensive ML computations to be performed in DisplayR’s servers. Deep Learning was processed under multiple parameters until the accuracy proved consistent and independent of training errors from the data set. This converged to 1,000 epochs with predictors normalized and 1 hidden layer to prevent over-fitting. To gauge ATCAM’s ability to generate correlating data between dynamic coupons and feedstock, 100 randomly selected data points were obtained and removed from each feedstock’s 1,000 coupon database to be processed through each algorithm and observe whether they could be correlated back toward the data. Success would demonstrate ATCAM’s ability to generate uniquely identifying coupons that could be automatically characterized and originated back to their feedstock source, paving the path for further correlations. The output of these tests was visualized by prediction-accuracy tables comparing the predicted force values 100 data points would theoretically have from the previous 900 data points to the 100 observed values. An additional ML test was conducted using simple linear regression to determine if 100 data points from a random feedstock could be compared to 2,700 data points (100 removed from each to provide the same library for each feedstock) and utilized to identify the parent feedstock. Diagnostic plots were created to analyze the constant variance of the input data and gauge the ML data’s sensitivity to external variables. Histograms demonstrating frequency distribution of force by the number of occurrences were also generated over the 1,000 coupons per feedstock, with a complimenting scatter plot of the cumulative dynamic coupon data to provide an additional perspective of correlating information.

Results

The results of ATCAM’s testing configuration are provided in this section. The flexural properties and qualitative metrics of coupons from each feedstock are provided in Table 2. The histogram depicting the frequency distribution of the dynamic coupon force measurements is visualized in Fig. 4. To present the dynamic coupon readings over the index of each testing cycle, Fig. 5 demonstrates results by feedstock. Trendlines and 5% error bars are provided of each feedstock’s average force reading per respective index point to display the resulted grouping.

Fig. 7
figure 7

Diagnostics of the machine learning regression model characterizing AM feedstock. The data present the correlation between Feedstock 2’s developed material library and 100 dynamic coupons. The top figure distributes the studentized residuals over the coupon cycle index, while the middle figure demonstrates the hat-values, and the bottom visualizes the sensitivity associated with Cook’s distance

Full size image
Fig. 8
figure 8

Sensitivity of the machine learning model characterizing AM feedstock. The data present the correlation between Feedstock 2’s developed material library and 100 dynamic coupons. Subfigure A provides Cook’s distance with respect to the leverage each of the 100 data points have on correlating to the feedstock material. Subfigure B demonstrates a quantile–quantile plot of the data which indicates whether there are independent errors, or whether the errors are normally distributed and have constant variance within the relationship between the x- and y-axis. Subfigure C is plotted with the standardized residuals being compared to the data’s leverage over the model prediction, while subfigure D compares the residuals against fitted values to analyze potential external correlations or bias introduced into the model

Full size image

A comparison of the three algorithms on Feedstock 2 is presented in Fig. 6. This comparison is presented for Feedstock 2 as a concise indicator for the accuracy and applicability of ATCAM for material characterization due to this AM material providing the preliminary average between cost and quality control as per the frequency distribution demonstrated histogram in Fig. 4.

The diagnostic plots in Fig. 7 and 8 of the simple regression model are provided. These are used for determining whether a randomized collection of dynamic coupons could be correlated back to their parent feedstock. It gauges whether the data’s dimensionality is correct or if there existed external influences (such as temperature differences for dynamic coupon numbers), which could alter the data. The sensitivity of these plots correlates to the linearity of the trend lines. Polynomial relationships indicate the model’s sensitivity to lack of variability within the data, a condition that reduces the accuracy of the results.

Discussion

The results from ATCAM’s testing configuration provide highly relevant insight into the novel capabilities associated with relatively large amounts of AM coupons. Although the coupons only generated one data point at a time, cumulative data indicated the inherent quality of the feedstock. Figure 4 provides a high-level perspective of a feedstock’s apparent manufacturing tolerances, as the spread of the distribution was lower compared to a material’s price point. The standard deviation of force distribution of Feedstock 1, 2, and 3 was 0.17N, 0.19N, 0.27N, respectively. The average deviation of each feedstock from its mean was 0.14N, 0.15N, and 0.22N, respectively. The various filament quality-related issues that can cause variability with the impact force are outlined in Table 1, and section 2.5. When paired to these generic empirical filament quality indicators, Feedstock 1 provided an average filament tolerance of +/- 14 micron, Feedstock 2 had a tolerance of +/- 18 micron, and Feedstock 3 was +/- 41 micron. When filament mass was compared, the average density tolerance of Feedstock 1 was consequently +/-\(0.023g/cm^3\), Feedstock 2 had +/-\(0.036g/cm^3\), with Feedstock 3 at \(0.062g/cm^3\). Table 2 provides additional metrics for analyzing the quality of the feedstock. Feedstock 1 demonstrated the highest flexural strength, modulus of elasticity, filament diameter tolerance, density tolerance, and smallest surface roughness. These results demonstrate a highly controlled manufacturing process as the deviations of the material’s characteristics were the smallest. The, respectively, high flexural properties indicate that the printer had minimal underextrusion and moisture content. Feedstock 2 provided results that demonstrate lower tolerance than Feedstock 1, but higher than Feedstock 3. The flexural strength, modulus, surface roughness, diameter tolerance, and density tolerance were consistently impressive given the material’s pricepoint. Feedstock 3 demonstrated the lowest mechanical properties, and the least control over the density and feedstock diameter. This material also demonstrated the highest surface roughness, agreeing with the loose diameter tolerance and indicating geometric errors within the manufacturing process. As the flexural strength and modulus of elasticity of FDM PLA under the described configurations for Feedstocks 1 and 2 were within the range of the expected values (Ref 1, 75), but lower than those from Feedstock 1, it is considered that the lower mechanical properties and tolerancing of Feedstock 2 and Feedstock 3 are primarily an indicator of their comparative quality.

The average force generated by Feedstock 1 was comparatively lower than Feedstock 2 and 3. We speculate that this is due to the reduced dampening of the ISA generated by Feedstock 2 and 3, increasing the force generated upon its release. Another causation that would increase the force of lower quality feedstock is the reduced adhesion to the build plate from lack of an initial uniform layer. The underextrusion and resulting cavities would effectively reduce the dynamic coupon’s surface area in contact with the build plate, subsequently reducing the force required for deployment. The deviation of ATCAM’s detected forces followed the trend observed regarding the quality of the feedstock materials, implying that the average mechanical properties cannot be independently referenced as a direct indicator of the average expected force through this experiment alone. However, it is noted that the deviations of these forces are correlated to the measured flexural properties, with higher quality feedstock generating smaller force deviations sensed through ATCAM, and better mechanical properties through testing. The agreement of the measured quality controls of the feedstock fabrication with ATCAM’s independent data demonstrates an initial utility for automatically indicating the grade of a feedstock.

These results agree with those displayed for the testing cycle in Fig. 5. Feedstock 1’s data grouping was the tightest with respect to both an \(R^2\) trendline correlation of its average data per index point of 0.961, with Feedstock 2 providing an \(r^2\) of 0.943 and Feedstock 3 at 0.910. Visually, the data point and error bar grouping cumulatively indicate a higher quality process for Feedstocks 1 and 2 than 3. Interestingly, Fig. 5 also clearly demonstrates that ATCAM can register material fatigue, identifying differences for effective performance degradation that occurs throughout ISA use. This result indicates an additional utility of ATCAM provided by gauging the life cycle of a test setup, catching when a machine is falling out of calibration, or at a minimum providing an extra dimension of data. The observed decay in Fig. 5 is relatively consistent between multiple ISA operating of the same feedstock, meaning that there are likely no significant variations within the batches of feedstock. The Prusa MK3 printing platform is configured to re-home its axes between the testing of each coupon and to fully self-calibrate between actuators. This allows for it to impart very consistent actuation, making it unlikely to be the source of the drift. The drift in the impact force of the ATCAM system over the testing-life of a particular actuator is predominantly due to the drifting of the experimental setup due to the mechanical wear fatiguing the ISA’s spring. As the actuator is taken to full deflection and released, there will be residual plastic deformation that causes a reduction in overall elasticity and maximum energy the spring section can store. This slight decrease over each of the actuations causes the decaying drift of the impact force. We consider that because ATCAM can detect the drifting of its own experimental setup, characterizing and filtering the drifting factor from the test results would allow for increased sensitivity and potential insights into the drifting of the AM process itself. Future work for ATCAM would leverage this filtering to vary both printer processing parameters and materials intentionally and gauge the effectiveness of potential correlations. The figure also shows a low-fidelity measurement of the inertial manufacturing errors introduced by each feedstock undergoing the same material process under the same parameters.

Figure 6 gauges the relevancy of three machine learning algorithms to correlate 100 dynamic coupons from Feedstock 2 to its database, scaling their accuracy in the bottom right when the predicted data paired with the observed data. The diagonal heat-map in Gradient Boost regression displays a 71% accuracy at predicting the dynamic coupon to a deviation of tolerance of force over 100 data points, with Support Vector dialing at 56%, and Deep Learning measuring at 35%. These results concur with Deep Learning’s general requirements for larger data sets to be effective correlating tools at describing phenomena. Consequently, the Deep Learning results do not provide initial confidence in their relevance for correlating the dynamic coupon data in its current configuration. The Support Vector data provided better results than Deep Learning but tended to overshoot the results. Utilizing Gradient Boost regression exhibits promising preliminary results to serve as ATCAM’s primary correlation method.

In observing the sensitivity of the machine learning results to potential external factors driving the variability in the outcome, Fig. 7 provides valuable insights. The studentized residuals of the dynamic coupon results demonstrate the desirable constant variance required by regression over the coupons’ index (Ref 32, 35). For example, coupon 9’s residual lies at a magnitude of -2N, while 12’s residual lies at a magnitude of +2N. This magnitude of outlying error is displayed at random intervals throughout the index and is demonstrated by coupons 51, 71, and 95 all having nearly equal adjacency to opposite residuals. There does not exist a trend of increasing magnitude in the hat-values or Cook’s distance over the span of the index, providing confidence over numerous perspectives of the validity of the machine learning results for a constant variance (Ref 9, 71).

Due to the intrinsic reliance of regression on the quality of the data, Fig. 8 provides an additional diagnostic analysis of the output and is required to identify any potential dependence or patterns within the predictions and observations (Ref 30, 78). Comparing the Cook’s distance from the leverage each data point provides on the model displays potential issues individual results can imply (Ref 38, 71). Subfigure A displays that the grouping of the data is near the trend, and visualizes that the model is not overly sensitive to any specific data-point altering the characterization. Subfigure B’s quantile–quantile plot comparing the standardized residuals to the theoretical quantiles designates desirable linearity and packing of individual data points to the trendline (Ref 78). Subfigure C and D compare standardized and regular residual to leverage and fitted values, respectively. The data from both subfigures indicate a primarily flat trendline that seems to overshoot when standardized and undershoot when not. The random spread of the residuals and the seeming inverse correlation over the two subfigures is a good sign that the values are not overly influenced by external variables, systematically bias, or indicate signs of heteroscedasticity, in which the variance of the data is time-dependent (Ref 56). Consequently, the results in Fig. 7 and 8 cumulatively indicate that there is minimal unexplained phenomena with high leverage within the data and regression serves as a statistically appropriate method to analyze this data and draw conclusions from its correlations.

It is crucial to note that ATCAM’s results are a virtue of not only the error associated with the manufacturing process and quality control of AM feedstock but significant limitations from the test setup and configuration as well. The accuracy of the generated data has multiple points of introduced ambiguity, including but not limited to the inherent control of the FDM process on a desktop machine, the in situ actuator’s tolerance, the cleanliness of the build-surface potentially altering the friction imposed on a deployed dynamic coupon, the dynamic coupon’s susceptibility to the variability of the contact from the in situ actuator, the dampening and induced moment from the physical load cell extension board, algorithmic computational errors associating the AM process to coupon performance results, and even variability of an unenclosed platform such as changes in temperature, humidity, and UV from sunlight over the number of days of the testing cycle of 3,000 dynamic coupons. Given these limitations, the quality of the results associated with ATCAM’s automated testing of the three feedstocks is still statistically distinguishable, with data able to be traced and associated with its parent feedstock with the assistance of machine learning and sheer quantities of coupons to train and reference from.

Conclusion

Given the discussed limitations, the results automatically generated, tested, and characterized by ATCAM’s dynamic coupons provided statistically relevant information that can identify and correlate data output by the AM process. The results presented are confined to automatically implying the quality of FDM feedstock, demonstrating drifting of the AM process and visualizing the life cycle of a part retrofitted onto the build-surface, fingerprinting the material process systematically through a single dimension of data, and demonstrating the initial applicability of Gradient Boost regression as an ML correlation tool for large quantities of AM data with low dimensionality. The significance of these initial results demonstrates promise for ATCAM’s continued testing and development as a low-cost test-bed for characterizing the AM process. The output from ATCAM’s current experimental configuration is not inherently compatible with current testing industry standards, as the results do not imply a comparable value or metric that is currently used in industry. The results do, however, provide an additional novel metric that consolidate manufacturability as an empirical metric which is implied over the characterized grouping of thousands of coupons. Additionally, the ability for the ML algorithms to identify a strong correlation of a set of dynamic coupons to their feedstock source imply that new uses for cybersecurity and traceability are capable of being generated by ATCAM. In its current FDM-specific configuration, ATCAM’s future tests can be conceptualized as maintaining a consistent feedstock material and intentionally altering process-parameters to identify sensitivities of the material process potentially identifiable through large quantities of controlled coupons. This consideration requires the characterization and filtering of the experimental drift referenced in the discussions to increase the sensitivity of the testing configuration to controlled variables. Another test that is of interest to the authors is to vary the feedstock’s material (such as printing ABS, Nylon, etc.) and observing the quality of the generated data. Additionally, leveraging the portable in situ methods and low-cost sensing platform can allow the current ATCAM testing procedure to be automatically conducted in intervals of production printing to catch if a machine’s calibration is potentially drifting from its optimal parameters. Although this would limit the build-volume to the space unoccupied by the in situ actuator, this result would be an extremely low-cost method for ensuring an FDM machine is performing to specification. Conversely, by controlling the calibration and material process, the dynamics introduced by the retrofitted in situ actuator may be exploited for testing large quantities of iterated geometries to potentially allow for automated design space exploration by AM. Improvements to increase the precision of ATCAM are to add additional dimensions of systematic testing to complement the physical sensors, retrofit an enclosure for the printer and the sensing setup, develop a robust automated method to remove an ISA at the end of its life cycle, develop a larger database for improvement in regression performance, and conceptualize the ATCAM process for non-FDM platforms.