Abstract
The sensitivity of additive manufacturing (AM) to the variability of feedstock quality, machine calibration, and accuracy drives the need for frequent characterization of fabricated objects for a robust material process. The constant testing is fiscally and logistically intensive, often requiring coupons that are manufactured and tested in independent facilities. As a step toward integrating testing and characterization into the AM process while reducing cost, we propose the automated testing and characterization of AM (ATCAM). ATCAM is configured for fused deposition modeling (FDM) and introduces the concept of dynamic coupons to generate large quantities of basic AM samples. An in situ actuator is printed on the build surface to deploy coupons through impact, which is sensed by a load cell system utilizing machine learning (ML) to correlate AM data. We test ATCAM’s ability to distinguish the quality of three PLA feedstock at differing price points by generating and comparing 3000 dynamic coupons in 10 repetitions of 100 coupon cycles per material. ATCAM correlated the quality of each feedstock and visualized fatigue of in situ actuators over each testing cycle. Three ML algorithms were then compared, with Gradient Boost regression demonstrating a 71% correlation of dynamic coupons to their parent feedstock and provided confidence for the quality of AM data ATCAM generates.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
As an emerging material process, additive manufacturing (AM) continues to increase in industry adoption due to its versatility to realize novel geometries, provide highly tailorable object performance, and reduce costs by virtue of digital dependencies and automation (Ref 5, 7, 72). These realizations are not only highly dependent on precision within the manufacturing process parameters, but also require tight control of the AM feedstock material as well (Ref 28, 45). It has been well-documented that low-quality AM feedstock can reduce predictability within the fabrication process (Ref 76, 81) and account for a large contribution to the inertial error that engineers account for within the design-for-manufacturing process (Ref 3, 12, 72). This has the knock-on effect of frequently burdening AM designs with relatively large factors of safety and redundancies (Ref 26, 63, 80), numerous iterations (Ref 21, 27, 79), and proof testing (Ref 13, 29) to accommodate the lack of engineering authority (Ref 18, 50, 57). Consistently generating and testing material coupons are critical to characterizing (Ref 12, 22) and maintaining (Ref 42, 65) the AM process; however, this itself is both cost and time inefficient due to the relatively slow and laborious nature of fabricating materials and testing them on other platforms (Ref 15, 43). Sensing platforms have been integrated into AM as a method to control the processing parameters in situ, such as by Ref 2, 74 and 77; however, these methods provide difficulty with maintaining the tractability of errors within the material process (Ref 20). Reference 16 and 40 specifically note the difficulty of correlating AM directly to material qualities and reference the need for more research in this area.
This paper presents a method that assists AM characterization by leveraging automation and machine learning to quickly generate and test large sets of data for automated testing and characterization of additive manufacturing (ATCAM). The development of the method required the ability to generate and sense large amounts of empirical AM data. To attain this, the concept of a dynamic coupon and in situ actuator (Ref 48) are demonstrated, which allow AM material to be automatically fabricated and ejected sequentially for a high-throughput FDM process. Ejected coupons impact a load cell to generate a data point. Logging and repeating the process automatically allows for the rapid accumulation of a database, enabling many automated applications. This includes systematically identifying a feedstock’s quality control, standardizing the calibration of a machine’s processing parameters, understanding the sensitivities of an AM material under various conditions, recognizing a machine that is drifting from its intended material process, determining if a material is from a specific vendor or origin, embedding and detecting counterfeiting protocols, potentially adapting a design for its environment through repeated iteration and exploration of design space, among many others.
To test ATCAM’s utility, three polylactic acid (PLA) feedstocks at differing price-points and effective quality were utilized to generate, deploy, and capture data from 3000 dynamic coupons in batches of 1,000 coupons per feedstock, conducted with 100 coupons per in situ actuator. The machine utilized was an open-source and widely available Prusa MK3S. Coupon impact force was logged through a load cell connected to an analog-to-digital converter attached to a microcontroller. The data were analyzed, correlated, and compared through three machine learning algorithms (deep learning, support vector machine, and boost gradient) supported by cloud computing with the DisplayR platform. At the time of writing, the entirety of this AM material testing setup costs less than USD $1000. This made the test widely adoptable, portable, low risk, and highly cost-effective. Section 2 provides the methods utilized for creating ATCAM, with section 3 presenting the data generated by the system and the machine learning correlations. Section 4 provides an analysis of the results, potential sources of error within the data, and discusses ATCAM’s main findings. The paper concludes in section 5 by over-viewing the novel processes ATCAM introduces to AM testing and characterization within the limitations of the current configuration and provides potential next steps to improve the system.
ATCAM’s Framework, Method for Implementation, and Testing Procedure
The concept of ATCAM was developed to integrate AM fabrication and testing into a single consolidated process. Figure 1 provides an overview of the system’s components, configuration, and process flow to automate the translation of AM feedstock into characterized data. In the following section, we present the system’s framework and components in detail by their subsystem, describe how the platform functions, discuss the data output and provide the testing procedure used to analyze ATCAM’s performance.
AM Platform
The additive manufacturing process utilized to develop ATCAM was FDM. This decision was primarily to increase the applicability of the platform, as FDM is a widely accessible (Ref 34, 66, 70) and relatively mature AM process (Ref 23, 36, 55). Additional consideration for selecting FDM was the wide variety of vendors available to source feedstock materials as demonstrated by Ref 69, the relative low cost of the platform (Ref 15), and the flexible nature of both the machine language (G-code) (Ref 8) and its firmware (Ref 36, 57, 66). Due to the open-market nature of FDM, a wide variety of materials are available for the process (Ref 33, 49, 64). The quality control of these materials typically varies significantly with price point and origin (Ref 28), making characterization of FDM materials a highly relevant challenge (Ref 12, 54, 65). Of the FDM machines available on the market, the Prusa MK3S (Ref 59) was selected due to the robustness of the machine (Ref 67), portability, cost (Ref 10), and ease of reconfiguration (Ref 47).
Dynamic Coupons and In Situ Actuation
Developing a method to automate a high throughput characterization of coupons, in particular, was not trivial; it proved to be the most difficult task in creating ATCAM. This was primarily due to the lack of reference, as a literature review concluded that there is no practical integrated solution to fully automate the creation and testing of AM coupons. The association of mechanical properties with AM processing parameters has been documented in reviews by Ref 62 and 46. Works by Ref 45 and 12 provide the critical need for consistent AM coupon testing to control mechanical properties and process quality, with (Ref 31) providing a novel strategy to use ultrasound for AM control. Reference 68 discusses a method to integrate printed circuitry into structures for AM part health monitoring and crack detection; however, this method of testing was not demonstrated as directly applicable for determining the quality of AM materials or processes and is highly object-specific. These works demonstrate that although constant testing is crucial for increased industry adoption of AM, there currently exists a testing throughput limitation for AM characterization due to lack of automation.
The premise of automation by in situ actuation has been previously demonstrated through programmed interactions between AM platforms and printed parts, such as by Ref 4 and 39. In both examples, the modifications to the printers are not purely software-based, as physical modifications are made to machines to provide new capabilities. Our method allows for the retrofits to be printed and manipulated entirely in situ through programmed revisions. We wanted to keep to remove the need for physical modifications that were ex situ to reduce the inertial error which can accumulate into the system. This is especially important for deducing the origination of an error source. Additionally, extensive physical modifications limit the adoption and portability of the system. Reference 6 provided insights into functional AM leaf springs, including associations regarding their deflections and energy storage. The springs in these results were not automated, nor were they deflected on the AM platform themselves, but they were referenced as a validated method for printed energy storage.
We developed dynamic coupons to address the technological gap between these different fields, allowing a large quantity of AM objects to be printed and tested entirely in situ. The process works by printing an in situ actuator (ISA) onto the build surface (Ref 48). This allows for the printer to print and pull back the ISA on the build platform to eject objects and clear the surface for further parts. ATCAM leverages this by printing a 25.4mm (1-inch) sphere onto the build surface, utilizing the tool head and ISA to transfer energy to the coupons through high-force impact. This ejects and transports them to a load cell which detects the force of the coupon’s impact. The cleared build surface is then used to print a subsequent dynamic coupon, and the ejection process is repeated. This process is repeated 100 times per ISA before manual removal, which was the experimentally determined non-destructive lifespan (Ref 48). One cycle of this process is visualized in the top-left (1-3) of Fig. 1.
The dynamic coupon was designed as a sphere for compatibility with the ISA to achieve reliable deployment within the 2N of force generated. A base was implemented on the bottom of the coupon to provide a tailored surface for deployment. This base also increased the coupon’s manufacturability, as the surface area of the sphere’s base was too small for the printer’s configuration. The base’s edges were integrated onto the dynamic coupon through a radius to decrease the overhang angle and increase manufacturability. An infill density of 20% was used in a rectangular configuration to increase the platform’s throughput and minimize thermal issues occurring from dense raster patterns. This resulted in the coupon having a designed mass of 4 grams. A diagram of the dynamic coupon is provided in Fig. 2.
Physical Sensing and Data Logging
An aluminum 1kgf-rated single-ended shear loading cell was utilized to collect dynamic coupon impact force through contact. The load cell was calibrated against the manufacturer datasheet, providing measurement linearity of +/-0.5%. The process flow for data logging is shown in Fig. 3. After a coupon is ejected, it impacts the load cell. This is registered as strain; the change in resistance is fed into a Wheatstone bridge and is digitized by a 24-bit analog-to-digital converter. This digital data are then read by an Arduino Uno microcontroller. The Arduino saves the value to a text file, formatting the coupon’s data as a force within a row of comma-separated variables including the timestamp and the number within the test cycle. After logging these values, the microcontroller then waits for the next data point. Figure 3 demonstrates that a minimum force threshold is imposed onto the load cell’s reading to reduce the likelihood of erroneous data occurring from ATCAM’s environment; a minimum of 1N force was required to log a data point as a dynamic coupon.
As seen in Fig. 1, the load cell is relatively small at 80.0mm x 12.7mm x 12.7mm (3.15x0.50”x0.50”), so a solution was required to ensure that the dynamic coupons would reliably and repeatably be registered. The surface area sensed by the load cell was increased by affixing an acrylic plate. This provided a highly reliable target for the dynamic coupons. The modified load cell was placed within the average deployment trajectory, shown in Fig. 1.
Machine Learning and Cloud Computing to Analyze Dynamic Coupon Data
To analyze the information from the deployments at the end of a testing cycle, the dynamic coupon data logged by the Arduino Uno is captured by a Raspberry Pi 3B due to its networking and processing capabilities. Data are converted from comma-spaced variables accumulated by the text document into columns of data sets associating dynamic coupon forces and uniquely identifiable information by their deployment number from their testing cycle. This information is formatted in association with each feedstock OEM, vendor name (where applicable), and batch number (e.g., feedstock material, vendor 1, batch 1). The consolidation of these identifiers serves as ATCAM’s database, which allows for dynamic coupons to be correlated with feedstock information.
Analysis of ATCAM’s database relies on machine learning (ML) algorithms to systematically associate intrinsic material information with extrinsic parameters. This process allows for larger sets of data with altered parameters to be paired to differences within the intrinsic data (Ref 53, 56). These correlations generally benefit from analysis through statistical regression (Ref 58, 61) and are consequently adequate methods for analyzing (Ref 60) and correlating otherwise complex multi-variable properties from lower-dimensional values. An example of this form of analysis is the printed electronics characterization demonstrated by Ref 52. They compared multiple ML algorithms such as decision trees, support vector machine, and logistical regression for appropriateness to their low dimensional AM and electronics data. Reference 25’s overview of the status of ML for AM specifically notes the emerging need for automated data handling and libraries for in situ processes, such as the method by Ref 17. Additionally, Goh references ML’s utility for AM material characterization (Ref 25). ML’s usage in AM extends into toolpathing considerations within the work of Ref 14), where optimized orientations for fabricating scaffolding are considered. Similarly, (Ref 19) investigates the applicability of ML in the loop for optimized AM of medicines. These papers discuss the ML link between AM characteristics and the final object’s performance; however, both processes have a relatively small data library to pull from compared to ATCAM and consequently do not fully leverage the capabilities of linear ML models.
ATCAM Testing Procedure
An initial test of ATCAM is provided to gauge the automated platform’s AM characterization performance and identify potential error sources for improvement. Three 1.75mm PLA feedstocks from different manufacturers were tested by the system in 10 batches of 100 dynamic coupons per feedstock. PLA dictated a standard 215 °C, 0.4 mm extruder and 60 °C build surface temperature. The toolpath was configured for 0.2 mm layer height with 20% rectangular infill due to the thermal considerations previously noted. The resulting settings provided an average volumetric flow rate of \(5mm^3/s\) at 2 mm/s feedstock extrusion speed. This created a dynamic coupon that required an average of 4.5g of material and took approximately 30 minutes to fabricate. The initial cycle with the ISA generated a toolpath which required 82.5 min for the first test cycle and a subsequent cycling frequency of about 2 dynamic coupons tested per hour, which made the testing regime require a little over two days of automated fabrication and testing per 100 coupons generated. The three feedstocks were selected by their price points, USD$60/kg, USD$30/kg, and USD$20/kg, identified as Feedstock 1, 2, and 3. To gain a basic understanding of the generic quality control parameters taken by the feedstock OEM’s, the feedstocks were each measured for filament diameter consistency at 30 points taken at random along the spool during the AM process with a Mitutoyo 500-464 dial caliper of 5-micron tolerance. Weight was measured per each 1kg quoted spool of filament with an OHAUS V31XH2 bench scale, which had 0.1-gram readability. These two parameters were utilized due to their importance in the FDM process; filament diameter consistency creates variability of mass flow rate in the extrusion process (Ref 11, 24). The considerations of these filament issues are consolidated and outlined in Table 1. The weight was measured and taken for the average filament diameter to understand the material density to provide an indicator of the material’s moisture content. This parameter is one of the leading sources of print variance in FDM (Ref 37), as water vapor cavities are formed in the FDM tool head, causing sputtering, the variability of mass flow rate, and inconsistent print results (Ref 41, 44, 51). To provide an additional metric for material performance, the surface roughness of three dynamic coupons fabricated from each feedstock was measured with a digital profilometer, the Sunnen SP-211. As per (Ref 73)’s review of surface texture metrology, conducting this profile topography with a contact stylus provides a measurement that is both widely accepted and utilized in both industry and academia. The testing scheme followed along the top of the coupon and recorded the roughness every 5 mm of the height for a total of five data points per test. Each test was repeated 10 times, generating 150 data points per feedstock. These results are provided in Table 2. Basic mechanical properties of the three feedstock were derived by conducting a three-point bending test to the ISO 178:2019 standard for the determination of flexural properties of thermoplastics. Three 80mm x 10mm x 4mm coupons for each feedstock were fabricated to the standard in a planar orientation, with the longest axis printed parallel to the build-plate. The printing parameters were taken as the same as the dynamic coupons. The bending test was conducted on a Chatillon TCM 201, fitted with a 2.22KN Interface 1110AF-500 strain gauge. The loading rate was set to .3175 millimeters per minute. The fixturing was fitted with 6.35mm diameter pins, with the base set to 25.44 mm apart to follow a conventional 16:1 ratio for the length of the tooling to the specimen’s thickness, respectively. The tests’ results were taken from a customized lab-view interface to gather physical load, time, and displacement. These measurements were used to calculate properties for each feedstock, with the results consolidated in Table 2.
Force measurements from the 3000 dynamic coupons were collected through the deployment and sensing process described above and processed through three different machine learning algorithms for comparative purposes. Deep Learning, Support Vector regression, and Gradient Boost regression were selected to provide a diverse assessment of the data. Due to the computationally intensive nature of Deep Learning, we decided to process the data through cloud computing. The Raspberry Pi leveraged the DisplayR cloud platform through an internet connection due to its simplicity, accessibility, and visualization capabilities. This process allowed for the intensive ML computations to be performed in DisplayR’s servers. Deep Learning was processed under multiple parameters until the accuracy proved consistent and independent of training errors from the data set. This converged to 1,000 epochs with predictors normalized and 1 hidden layer to prevent over-fitting. To gauge ATCAM’s ability to generate correlating data between dynamic coupons and feedstock, 100 randomly selected data points were obtained and removed from each feedstock’s 1,000 coupon database to be processed through each algorithm and observe whether they could be correlated back toward the data. Success would demonstrate ATCAM’s ability to generate uniquely identifying coupons that could be automatically characterized and originated back to their feedstock source, paving the path for further correlations. The output of these tests was visualized by prediction-accuracy tables comparing the predicted force values 100 data points would theoretically have from the previous 900 data points to the 100 observed values. An additional ML test was conducted using simple linear regression to determine if 100 data points from a random feedstock could be compared to 2,700 data points (100 removed from each to provide the same library for each feedstock) and utilized to identify the parent feedstock. Diagnostic plots were created to analyze the constant variance of the input data and gauge the ML data’s sensitivity to external variables. Histograms demonstrating frequency distribution of force by the number of occurrences were also generated over the 1,000 coupons per feedstock, with a complimenting scatter plot of the cumulative dynamic coupon data to provide an additional perspective of correlating information.
Results
The results of ATCAM’s testing configuration are provided in this section. The flexural properties and qualitative metrics of coupons from each feedstock are provided in Table 2. The histogram depicting the frequency distribution of the dynamic coupon force measurements is visualized in Fig. 4. To present the dynamic coupon readings over the index of each testing cycle, Fig. 5 demonstrates results by feedstock. Trendlines and 5% error bars are provided of each feedstock’s average force reading per respective index point to display the resulted grouping.
A comparison of the three algorithms on Feedstock 2 is presented in Fig. 6. This comparison is presented for Feedstock 2 as a concise indicator for the accuracy and applicability of ATCAM for material characterization due to this AM material providing the preliminary average between cost and quality control as per the frequency distribution demonstrated histogram in Fig. 4.
The diagnostic plots in Fig. 7 and 8 of the simple regression model are provided. These are used for determining whether a randomized collection of dynamic coupons could be correlated back to their parent feedstock. It gauges whether the data’s dimensionality is correct or if there existed external influences (such as temperature differences for dynamic coupon numbers), which could alter the data. The sensitivity of these plots correlates to the linearity of the trend lines. Polynomial relationships indicate the model’s sensitivity to lack of variability within the data, a condition that reduces the accuracy of the results.
Discussion
The results from ATCAM’s testing configuration provide highly relevant insight into the novel capabilities associated with relatively large amounts of AM coupons. Although the coupons only generated one data point at a time, cumulative data indicated the inherent quality of the feedstock. Figure 4 provides a high-level perspective of a feedstock’s apparent manufacturing tolerances, as the spread of the distribution was lower compared to a material’s price point. The standard deviation of force distribution of Feedstock 1, 2, and 3 was 0.17N, 0.19N, 0.27N, respectively. The average deviation of each feedstock from its mean was 0.14N, 0.15N, and 0.22N, respectively. The various filament quality-related issues that can cause variability with the impact force are outlined in Table 1, and section 2.5. When paired to these generic empirical filament quality indicators, Feedstock 1 provided an average filament tolerance of +/- 14 micron, Feedstock 2 had a tolerance of +/- 18 micron, and Feedstock 3 was +/- 41 micron. When filament mass was compared, the average density tolerance of Feedstock 1 was consequently +/-\(0.023g/cm^3\), Feedstock 2 had +/-\(0.036g/cm^3\), with Feedstock 3 at \(0.062g/cm^3\). Table 2 provides additional metrics for analyzing the quality of the feedstock. Feedstock 1 demonstrated the highest flexural strength, modulus of elasticity, filament diameter tolerance, density tolerance, and smallest surface roughness. These results demonstrate a highly controlled manufacturing process as the deviations of the material’s characteristics were the smallest. The, respectively, high flexural properties indicate that the printer had minimal underextrusion and moisture content. Feedstock 2 provided results that demonstrate lower tolerance than Feedstock 1, but higher than Feedstock 3. The flexural strength, modulus, surface roughness, diameter tolerance, and density tolerance were consistently impressive given the material’s pricepoint. Feedstock 3 demonstrated the lowest mechanical properties, and the least control over the density and feedstock diameter. This material also demonstrated the highest surface roughness, agreeing with the loose diameter tolerance and indicating geometric errors within the manufacturing process. As the flexural strength and modulus of elasticity of FDM PLA under the described configurations for Feedstocks 1 and 2 were within the range of the expected values (Ref 1, 75), but lower than those from Feedstock 1, it is considered that the lower mechanical properties and tolerancing of Feedstock 2 and Feedstock 3 are primarily an indicator of their comparative quality.
The average force generated by Feedstock 1 was comparatively lower than Feedstock 2 and 3. We speculate that this is due to the reduced dampening of the ISA generated by Feedstock 2 and 3, increasing the force generated upon its release. Another causation that would increase the force of lower quality feedstock is the reduced adhesion to the build plate from lack of an initial uniform layer. The underextrusion and resulting cavities would effectively reduce the dynamic coupon’s surface area in contact with the build plate, subsequently reducing the force required for deployment. The deviation of ATCAM’s detected forces followed the trend observed regarding the quality of the feedstock materials, implying that the average mechanical properties cannot be independently referenced as a direct indicator of the average expected force through this experiment alone. However, it is noted that the deviations of these forces are correlated to the measured flexural properties, with higher quality feedstock generating smaller force deviations sensed through ATCAM, and better mechanical properties through testing. The agreement of the measured quality controls of the feedstock fabrication with ATCAM’s independent data demonstrates an initial utility for automatically indicating the grade of a feedstock.
These results agree with those displayed for the testing cycle in Fig. 5. Feedstock 1’s data grouping was the tightest with respect to both an \(R^2\) trendline correlation of its average data per index point of 0.961, with Feedstock 2 providing an \(r^2\) of 0.943 and Feedstock 3 at 0.910. Visually, the data point and error bar grouping cumulatively indicate a higher quality process for Feedstocks 1 and 2 than 3. Interestingly, Fig. 5 also clearly demonstrates that ATCAM can register material fatigue, identifying differences for effective performance degradation that occurs throughout ISA use. This result indicates an additional utility of ATCAM provided by gauging the life cycle of a test setup, catching when a machine is falling out of calibration, or at a minimum providing an extra dimension of data. The observed decay in Fig. 5 is relatively consistent between multiple ISA operating of the same feedstock, meaning that there are likely no significant variations within the batches of feedstock. The Prusa MK3 printing platform is configured to re-home its axes between the testing of each coupon and to fully self-calibrate between actuators. This allows for it to impart very consistent actuation, making it unlikely to be the source of the drift. The drift in the impact force of the ATCAM system over the testing-life of a particular actuator is predominantly due to the drifting of the experimental setup due to the mechanical wear fatiguing the ISA’s spring. As the actuator is taken to full deflection and released, there will be residual plastic deformation that causes a reduction in overall elasticity and maximum energy the spring section can store. This slight decrease over each of the actuations causes the decaying drift of the impact force. We consider that because ATCAM can detect the drifting of its own experimental setup, characterizing and filtering the drifting factor from the test results would allow for increased sensitivity and potential insights into the drifting of the AM process itself. Future work for ATCAM would leverage this filtering to vary both printer processing parameters and materials intentionally and gauge the effectiveness of potential correlations. The figure also shows a low-fidelity measurement of the inertial manufacturing errors introduced by each feedstock undergoing the same material process under the same parameters.
Figure 6 gauges the relevancy of three machine learning algorithms to correlate 100 dynamic coupons from Feedstock 2 to its database, scaling their accuracy in the bottom right when the predicted data paired with the observed data. The diagonal heat-map in Gradient Boost regression displays a 71% accuracy at predicting the dynamic coupon to a deviation of tolerance of force over 100 data points, with Support Vector dialing at 56%, and Deep Learning measuring at 35%. These results concur with Deep Learning’s general requirements for larger data sets to be effective correlating tools at describing phenomena. Consequently, the Deep Learning results do not provide initial confidence in their relevance for correlating the dynamic coupon data in its current configuration. The Support Vector data provided better results than Deep Learning but tended to overshoot the results. Utilizing Gradient Boost regression exhibits promising preliminary results to serve as ATCAM’s primary correlation method.
In observing the sensitivity of the machine learning results to potential external factors driving the variability in the outcome, Fig. 7 provides valuable insights. The studentized residuals of the dynamic coupon results demonstrate the desirable constant variance required by regression over the coupons’ index (Ref 32, 35). For example, coupon 9’s residual lies at a magnitude of -2N, while 12’s residual lies at a magnitude of +2N. This magnitude of outlying error is displayed at random intervals throughout the index and is demonstrated by coupons 51, 71, and 95 all having nearly equal adjacency to opposite residuals. There does not exist a trend of increasing magnitude in the hat-values or Cook’s distance over the span of the index, providing confidence over numerous perspectives of the validity of the machine learning results for a constant variance (Ref 9, 71).
Due to the intrinsic reliance of regression on the quality of the data, Fig. 8 provides an additional diagnostic analysis of the output and is required to identify any potential dependence or patterns within the predictions and observations (Ref 30, 78). Comparing the Cook’s distance from the leverage each data point provides on the model displays potential issues individual results can imply (Ref 38, 71). Subfigure A displays that the grouping of the data is near the trend, and visualizes that the model is not overly sensitive to any specific data-point altering the characterization. Subfigure B’s quantile–quantile plot comparing the standardized residuals to the theoretical quantiles designates desirable linearity and packing of individual data points to the trendline (Ref 78). Subfigure C and D compare standardized and regular residual to leverage and fitted values, respectively. The data from both subfigures indicate a primarily flat trendline that seems to overshoot when standardized and undershoot when not. The random spread of the residuals and the seeming inverse correlation over the two subfigures is a good sign that the values are not overly influenced by external variables, systematically bias, or indicate signs of heteroscedasticity, in which the variance of the data is time-dependent (Ref 56). Consequently, the results in Fig. 7 and 8 cumulatively indicate that there is minimal unexplained phenomena with high leverage within the data and regression serves as a statistically appropriate method to analyze this data and draw conclusions from its correlations.
It is crucial to note that ATCAM’s results are a virtue of not only the error associated with the manufacturing process and quality control of AM feedstock but significant limitations from the test setup and configuration as well. The accuracy of the generated data has multiple points of introduced ambiguity, including but not limited to the inherent control of the FDM process on a desktop machine, the in situ actuator’s tolerance, the cleanliness of the build-surface potentially altering the friction imposed on a deployed dynamic coupon, the dynamic coupon’s susceptibility to the variability of the contact from the in situ actuator, the dampening and induced moment from the physical load cell extension board, algorithmic computational errors associating the AM process to coupon performance results, and even variability of an unenclosed platform such as changes in temperature, humidity, and UV from sunlight over the number of days of the testing cycle of 3,000 dynamic coupons. Given these limitations, the quality of the results associated with ATCAM’s automated testing of the three feedstocks is still statistically distinguishable, with data able to be traced and associated with its parent feedstock with the assistance of machine learning and sheer quantities of coupons to train and reference from.
Conclusion
Given the discussed limitations, the results automatically generated, tested, and characterized by ATCAM’s dynamic coupons provided statistically relevant information that can identify and correlate data output by the AM process. The results presented are confined to automatically implying the quality of FDM feedstock, demonstrating drifting of the AM process and visualizing the life cycle of a part retrofitted onto the build-surface, fingerprinting the material process systematically through a single dimension of data, and demonstrating the initial applicability of Gradient Boost regression as an ML correlation tool for large quantities of AM data with low dimensionality. The significance of these initial results demonstrates promise for ATCAM’s continued testing and development as a low-cost test-bed for characterizing the AM process. The output from ATCAM’s current experimental configuration is not inherently compatible with current testing industry standards, as the results do not imply a comparable value or metric that is currently used in industry. The results do, however, provide an additional novel metric that consolidate manufacturability as an empirical metric which is implied over the characterized grouping of thousands of coupons. Additionally, the ability for the ML algorithms to identify a strong correlation of a set of dynamic coupons to their feedstock source imply that new uses for cybersecurity and traceability are capable of being generated by ATCAM. In its current FDM-specific configuration, ATCAM’s future tests can be conceptualized as maintaining a consistent feedstock material and intentionally altering process-parameters to identify sensitivities of the material process potentially identifiable through large quantities of controlled coupons. This consideration requires the characterization and filtering of the experimental drift referenced in the discussions to increase the sensitivity of the testing configuration to controlled variables. Another test that is of interest to the authors is to vary the feedstock’s material (such as printing ABS, Nylon, etc.) and observing the quality of the generated data. Additionally, leveraging the portable in situ methods and low-cost sensing platform can allow the current ATCAM testing procedure to be automatically conducted in intervals of production printing to catch if a machine’s calibration is potentially drifting from its optimal parameters. Although this would limit the build-volume to the space unoccupied by the in situ actuator, this result would be an extremely low-cost method for ensuring an FDM machine is performing to specification. Conversely, by controlling the calibration and material process, the dynamics introduced by the retrofitted in situ actuator may be exploited for testing large quantities of iterated geometries to potentially allow for automated design space exploration by AM. Improvements to increase the precision of ATCAM are to add additional dimensions of systematic testing to complement the physical sensors, retrofit an enclosure for the printer and the sensing setup, develop a robust automated method to remove an ISA at the end of its life cycle, develop a larger database for improvement in regression performance, and conceptualize the ATCAM process for non-FDM platforms.
References
M. Ahmed, M.R. Islam, J. Vanhoose, L. Hewavitharana, A. Stanich, and M. Hossain, Comparisons Of Bending Stiffness of 3d Printed Samples Of Different Materials, in ASME 2016 International Mechanical Engineering Congress and Exposition, American Society of Mechanical Engineers Digital Collection (2016)
D.A. Anderegg, H.A. Bryant, D.C. Ruffin, S.M. Skrip Jr., J.J. Fallon, E.L. Gilmer and M.J. Bortner, In-Situ Monitoring of Polymer Flow Temperature and Pressure in Extrusion Based Additive Manufacturing. Addit. Manuf. 26, 76–83 (2019)
A. Arias-Montano, C.A.C. Coello and E. Mezura-Montes, Multi-objective Airfoil Shape Optimization Using a Multiple-Surrogate Approach, in 2012 IEEE Congress on Evolutionary Computation, IEEE, pp 1–8 (2012)
R.V. Aroca, C.E. Ventura, I. De Mello and T.F. Pazelli, Sequential Additive Manufacturing: Automatic Manipulation of 3d Printed Parts. Rapid Prototyp. J. (2017)
S.S. Babu, L. Love, R. Dehoff, W. Peter, T.R. Watkins and S. Pannala, Additive Manufacturing of Materials: Opportunities and Challenges. MRS Bull. 40(12), 1154–1161 (2015)
D. Besnea, C. Rizescu, D. Rizescu, D. Comeaga, R. Ciobanu and E. Moraru, Study of deflection behavior of 3d printed leaf springs, in IOP Conference Series: Materials Science and Engineering, IOP Publishing, vol. 444, p. 042008 (2018)
D.L. Bourell, D.W. Rosen and M.C. Leu, The Roadmap for Additive Manufacturing and its Impact. 3D Print. Addit. Manuf. 1(1), 6–9 (2014)
A. Bowyer, RepRap. University of Bath, Open Source 3D Printer Hardware and Software Standards (2020)
R.L. Brown, J. Durbin and J.M. Evans, Techniques for Testing the Constancy of Regression Relationships Over Time. J. R. Statist. Soc. Ser. B Methodol. 37(2), 149–163 (1975)
J. Brus, R. Barvíř, Coping with Integrating Low-Cost 3d Printing and Surface Models: A Case Study on Prusa i3, in Surface Models for Geosciences, Springer, pp. 45–59 (2015)
C. Cardona, A. Curdes and A. Isaacs, Effects of Filament Diameter Tolerances in Fused Filament Fabrication. IU J. Undergrad. Res. 2(1), 44–47 (2016)
C.K. Chua, C.H. Wong and W.Y. Yeong, Standards, Quality Control and Measurement Sciences in 3D Printing and Additive Manufacturing (Academic Press, Cambridge, 2017)
G. Coccia, G. Di Nicola and M. Sotte, Design, Manufacture and Test of a Prototype for a Parabolic Trough Collector for Industrial Process Heat. Renew. Energy 74, 727–736 (2015)
A. Conev, E.E. Litsa, M.R. Perez, M. Diba, A.G. Mikos and L.E. Kavraki, Machine Learning-guided Three-dimensional Printing of Tissue Engineering Scaffolds. Tissue Eng. Part A (2020)
G. Costabile, M. Fera, F. Fruggiero, A. Lambiase and D. Pham, Cost Models of Additive Manufacturing: A Literature Review. Int. J. Ind. Eng. Comput. 8(2), 263–283 (2017)
M. Cotteleer and J. Joyce, 3d Opportunity: Additive Manufacturing Paths to Performance, Innovation and Growth. Deloitte Rev. 14, 5–19 (2014)
U. Delli and S. Chang, Automated Process Monitoring in 3d Printing Using Supervised Machine Learning. Procedia Manuf. 26, 865–870 (2018)
F. Demoly, N. Troussier, B. Eynard, H. Falgarone, B. Fricero and S. Gomes, Proactive Assembly Oriented Design Approach Based on the Deployment of Functional Requirements. J. Comput. Inf. Sci. Eng. 11(1)(2011)
M. Elbadawi, B.M. Castro, F.K. Gavins, J.J. Ong, S. Gaisford, G. Pérez, A.W. Basit, P. Cabalar and A. Goyanes, M3diseen: A Novel Machine Learning Approach for Predicting the 3d Printability of Medicines. Int. J. Pharmaceut. 590, 119837 (2020)
S.K. Everton, M. Hirsch, P. Stravroulakis, R.K. Leach and A.T. Clare, Review of In-situ Process Monitoring and In-situ Metrology for Metal Additive Manufacturing. Mater. Des. 95, 431–445 (2016)
D. Floreano and R.J. Wood, Science, Technology and the Future of Small Autonomous Drones. Nature 521(7553), 460–466 (2015)
A.M. Forster and A. M. Forster, Materials Testing Standards for Additive Manufacturing of Polymer Materials: State of the Art and Standards Applicability. N/A (2015)
L.M. Galantucci, M.G. Guerra, M. Dassisti and F. Lavecchia (2019) Additive manufacturing: New trends in the 4th industrial revolution, in International Conference on the Industry 4.0 model for Advanced Manufacturing, Springer, pp. 153–169
E.L. Gilmer, D. Miller, C.A. Chatham, C. Zawaski, J. J. Fallon, A. Pekkanen, T.E Long, C.B. Williams and M.J. Bortner, Model analysis of feedstock behavior in fused filament fabrication: Enabling rapid materials screening. Polymer 152:51 – 61, sI: Advanced Polymers for 3D Printing/Additive Manufacturing (2018)
G. Goh, S. Sing and W. Yeong, A Review on Machine Learning in 3d Printing: Applications, Potential and Challenges. Artificial Intelligence Review pp. 1–32 (2020)
J.R. Hadley and D.J. McCarthy, Producibility and confidence indices during defense acquisition, in NDIA Ground Vehicle Systems Engineering and Technology Symposium (2011)
I. Harvey, E.D. Paolo, R. Wood, M. Quinn and E. Tuci, Evolutionary Robotics: A New Scientific Tool for Studying Cognition. Artif. Life 11(1–2), 79–98 (2005)
A.A. Hassen and M.M. Kirka, Additive Manufacturing: The Rise of a Technology and the Need for Quality Control and Inspection Techniques. Mater. Eval. 76(4), 438–453 (2018)
E. Hilton, J. Linsey and J. Goodman, Understanding the prototyping strategies of experienced designers, in 2015 IEEE Frontiers in Education Conference (FIE), IEEE, pp 1–8 (2015)
R.R. Hocking, Methods and Applications of Linear Models: Regression and the Analysis of Variance (Wiley, New York, 2013)
F. Honarvar and A. Varvani-Farahani, A Review of Ultrasonic Testing Applications in Additive Manufacturing: Defect Evaluation, Material Characterization and Process Control. Ultrasonics, p. 106227 (2020)
D.W. Hosmer, T. Hosmer, S. Le Cessie and S. Lemeshow, A Comparison of Goodness-of-fit Tests for the Logistic Regression Model. Statist. Med. 16(9), 965–980 (1997)
E.J. Hurst, 3d Printing in Healthcare: Emerging Applications. J. Hosp. Librarianship 16(3), 255–267 (2016)
A.O. Inoma, O.O. Ibhadode and A.A. Ibhadode, The Perception and Deployment of Pd printing in the Nigerian Educational Sector for Science and Engineering Programs. Sci. Afr. p. e00641 (2020)
C.M. Jarque and A.K. Bera, A Test for Normality of Observations and Regression Residuals. International Statistical Review/Revue Internationale de Statistique, pp. 163–172 (1987)
R. Jones, P. Haufe, E. Sells, P. Iravani, V. Olliver, C. Palmer and A. Bowyer, Reprap - The Replicating Rapid Prototyper. Robotica 29(1), 177–191 (2011). https://doi.org/10.1017/S026357471000069X
P. Kakanuru and K. Pochiraju, Moisture Ingress and Degradation of Additively Manufactured Pla, Abs and Pla/sic Composite Parts. Addit. Manuf. 36, 101529 (2020)
K.S. Kannan and K. Manoj, Outlier Detection in Multivariate Data. Appl. Math. Sci. 9(47), 2317–2324 (2015)
S. Katakura and K. Watanabe, Printmotion: Actuating Printed Objects Using Actuators Equipped in a 3d Printer, in The 31st Annual ACM Symposium on User Interface Software and Technology Adjunct Proceedings, pp. 137–139 (2018)
S.K. Kennedy, A.M. Dalley and G.J. Kotyk, Additive Manufacturing: Assessing Metal Powder Quality Through Characterizing Feedstock and Contaminants. J. Mater. Eng. Perform. 28(2), 728–740 (2019)
E. Kim, Y.J. Shin and S.H. Ahn, The Effects of Moisture and Temperature on the Mechanical Properties of Additive Manufacturing Components: Fused Deposition Modeling. Rapid Prototyp. J. 22(6), 887–894 (2016). https://doi.org/10.1108/rpj-08-2015-0095
H. Kim, Y. Lin and T.L.B. Tseng, A Review on Quality Control in Additive Manufacturing. Rapid Prototyp. J. (2018)
L. Koester, H. Taheri, L.J. Bond, D. Barnard and J. Gray, Additive Manufacturing Metrology: State of the Art and Needs Assessment, in AIP conference Proceedings, AIP Publishing LLC, vol. 1706, p. 130001 (2016)
M. Leite, J. Fernandes, A.M. Deus, L. Reis and M.F. Vaz, Study of the Influence of 3d Printing Parameters on the Mechanical Properties of pla. https://doi.org/10.25341/D4988C (2018)
Q.Y. Lu and C.H. Wong, Additive Manufacturing Process Monitoring and Control by Non-destructive Testing Techniques: Challenges and In-process Monitoring. Virtual Phys. Prototyp. 13(2), 39–48 (2018)
C. Mandache, Overview of Non-destructive Evaluation Techniques for Metal-Based Additive Manufacturing. Mater. Sci. Technol. 35(9), 1007–1015 (2019)
I. Mardiyansah and M. Yamin, Re-design prusa i3 3d printer using software solidworks 2016. Int. J. Des. Manuf. Technol. (IJDMT) 10, 1–11 (2019)
A.A. Mazhari, A. Zhang, R. Ticknor, S. Swei, E. Hyde and M. Teodorescu, Printing and programming of in-situ actuators, in 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), IEEE, pp. 445–450 (2020)
V. Mazzanti, L. Malagutti and F. Mollica, Fdm 3d Printing Of Polymers Containing Natural Fillers: A Review of Their Mechanical Properties. Polymers 11(7), 1094 (2019)
D. Miller, I. Fitzner, S. Fuller and S. Revzen, Focused modularity: Rapid iteration of design and fabrication of a meter-scale hexapedal robot, in ASSISTIVE ROBOTICS: Proceedings of the 18th International Conference on CLAWAR 2015, World Scientific, pp 430–438 (2016)
P.S. Minh and T.T. Tuyet, Influence of 3d-Printing Parameters on Flexural Strength of Pla Plastic Products. Int. J. Mech. Eng. 7(3), 1–4 (2020)
M.H. Mubarik, D.D. Weller, N. Bleier, M. Tomei, J. Aghassi-Hagmann, M.B. Tahoori and R. Kumar, Printed machine learning classifiers, in 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), IEEE, pp. 73–87 (2020)
R.H. Myers and R.H. Myers, Classical and Modern Regression with Applications, vol. 2 (Duxbury press, Belmont, CA, 1990)
A. Nasirov, Multiscale modeling of fused filament fabricated specimens. Ph.D thesis, Tennessee Technological University (2019)
A. Nazir, A. Azhar, U. Nazir, Y.F. Liu, W.S. Qureshi, J.E. Chen and E. Alanazi, The Rise of 3d Printing Entangled With Smart Computer Aided Design During Covid-19 Era. J. Manuf. Syst. (2020)
M.A. Poole and P.N. O’Farrell, The Assumptions of the Linear Regression Model. Transactions of the Institute of British Geographers, pp. 145–158 (1971)
D. Popescu and C. Amza, Additive Manufacturing Automation for Industry 4.0. Res & Sci Today 13:50 (2017)
K.J. Preacher, P.J. Curran and D.J. Bauer, Computational Tools for Probing Interactions in Multiple Linear Regression, Multilevel Modeling and Latent Curve Analysis. J. Edu. Behav. Statist. 31(4), 437–448 (2006). https://doi.org/10.3102/10769986031004437
J. Prusa, Prusa i3 Mk3 3D printer. Prusa Research a.s., https://www.prusa3d.com/, commercial 3D printer (2020)
X. Qi, G. Chen, Y. Li, X. Cheng and C. Li, Applying Neural-Network-based Machine Learning to Additive Manufacturing: Current Applications, Challenges and Future Perspectives. Engineering 5(4), 721–729 (2019)
K. Regulski, D. Wilk-Kołodziejczyk, T. Szymczak, G. Gumienny, Z. Pirowski, K. Jaśkowiec and S. Kluska-Nawarecka, Data Mining Methods for Prediction of Multi-component Al-Si Alloy Properties Based on Cooling Curves. J. Mater. Eng. Perf. 28(12), 7431–7444 (2019)
A. Rodríguez-Panes, J. Claver and A.M. Camacho, The Influence of Manufacturing Parameters on the Mechanical Behaviour Of Pla and Abs Pieces Manufactured By Fdm: A Comparative Analysis. Materials 11(8), 1333 (2018)
R.H. Sanatgar, C. Campagne and V. Nierstrasz, Investigation of the Adhesion Properties of Direct 3d Printing of Polymers and Nanocomposites on Textiles: Effect of Fdm Printing Process Parameters. Appl. Surf. Sci. 403, 551–563 (2017)
N. Sa’ude, M. Ibrahim, M. Ibrahim, M. Wahab, R. Haq, O. Marwah and R. Khirotdin, Additive manufacturing of copper-abs filament by fused deposition modeling (fdm). J. Mech. Eng. pp 23–32 (2018)
M. Seifi, A. Salem, J. Beuth, O. Harrysson and J.J. Lewandowski, Overview of Materials Qualification Needs for Metal Additive Manufacturing. JOM 68(3), 747–764 (2016)
E. Sells, S. Bailard, Z. Smith, A. Bowyer and V. Olliver, Reprap: the replicating rapid prototyper: maximizing customizability by breeding the means of production, in Handbook of Research in Mass Customization and Personalization: (In 2 Volumes), World Scientific, pp. 568–580 (2010)
J. Sierra, D.S. Villa, A.M. Velasquez and W. Villaneda, Relation Between Mechanical Properties and 3d Printer Configurations Parameters Using Pla at Open-source Prusa i3. Int. J. Int. Eng. 12(8), 97–108 (2020)
T. Smith, J.A. Failla, J.M. Lindahl, P. Kim, A.A. Hassen, C.E. Duty, P. Joshi, C. Stevens and V. Kunc, Structural health monitoring of 3d printed structures. Tech. rep., Oak Ridge National Lab.(ORNL), Oak Ridge, TN (United States) (2018)
J.W. Stansbury and M.J. Idacavage, 3d Printing With Polymers: Challenges Among Expanding Options and Opportunities. Dental Mater. 32(1), 54–64 (2016)
H. Steenhuis and L. Pretorius, Additive Manufacturing or 3d Printing and Its Adoption, in International Association for Management of Technology 2015 Conference Proceedings (University of Pretoria, Pretoria, South Africa, 2015), pp. 2468–2479
J.P. Stevens, Outliers and Influential Data Points in Regression Analysis. Psychol. Bull. 95(2), 334 (1984)
S.A. Tofail, E.P. Koumoulos, A. Bandyopadhyay, S. Bose, L. O’ Donoghue and C. Charitidis, Additive Manufacturing: Scientific and Technological Challenges, Market Uptake and Opportunities. Mater. Today 21(1), 22–37 (2018)
A. Townsend, N. Senin, L. Blunt, R. Leach and J. Taylor, Surface Texture Metrology for Metal Additive Manufacturing: A Review. Precis. Eng. 46, 34–47 (2016)
J. Trapp, A.M. Rubenchik, G. Guss and M.J. Matthews, In situ Absorptivity Measurements of Metallic Powders During Laser Powder-bed Fusion Additive Manufacturing. Appl. Mater. Today 9, 341–349 (2017)
J.A. Travieso-Rodriguez, R. Jerez-Mesa, J. Llumà, O. Traver-Ramos, G. Gomez-Gras and J.J. Roa Rovira, Mechanical Properties of 3d-Printing Polylactic Acid Parts Subjected to Bending Stress and Fatigue Testing. Materials 12(23), 3859 (2019)
P.A. Vargas, E.A. Di Paolo, I. Harvey and P. Husbands, The Horizons of Evolutionary Robotics. MIT press (2014)
T. Wang, T.H. Kwok and C. Zhou, In-Situ Droplet Inspection and Control System For Liquid Metal Jet 3d Printing Process. Proc. Manuf. 10, 968–981 (2017)
M.N. Williams, C.A.G. Grajales and D. Kurkiewicz, Assumptions of Multiple Regression: Correcting Two Misconceptions. Pract. Assess. Res. Eval. 18(1), 11 (2013)
K.V. Wong and A. Hernandez, A Review of Additive Manufacturing. ISRN Mechanical Engineering 2012 (2012)
R.J. Wood, B. Finio, M. Karpelson, K. Ma, N.O. Pérez-Arancibia, P.S. Sreetharan, H. Tanaka and J.P. Whitney, Progress on Pico Air Vehicles. Int. J. Robot. Res. 31(11), 1292–1302 (2012)
G.Z. Yang, J. Bellingham, P.E. Dupont, P. Fischer, L. Floridi, R. Full, N. Jacobstein, V. Kumar, M. McNutt and R. Merrifield et al., The Grand Challenges of Science Robotics. Sci. Robot. (2018). (3(14):eaar7650)
Acknowledgments
The authors thank the NASA Ames Advanced Manufacturing Division, the Office of the Chief Scientist, and the Office of the Chief Technologist for their support in developing this work.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This invited article is part of a special topical focus in the Journal of Materials Engineering and Performance on Additive Manufacturing. The issue was organized by Dr. William Frazier, Pilgrim Consulting, LLC; Mr. Rick Russell, NASA; Dr. Yan Lu, NIST; Dr. Brandon D. Ribic, America Makes; and Caroline Vail, NSWC Carderock
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Mazhari, A.A., Ticknor, R., Swei, S. et al. Automated Testing and Characterization of Additive Manufacturing (ATCAM). J. of Materi Eng and Perform 30, 6862–6873 (2021). https://doi.org/10.1007/s11665-021-06042-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11665-021-06042-2