Introduction

Materials development is slow and expensive, taking decades from inception to fielding. For materials in coupled extremes, the situation is even more demanding, as the desired property combinations such as strength and high-temperature oxidation resistance can have complex interactions. Here, we discuss the role of AI and autonomous experimentation (AE) in the design, synthesis, and characterization of materials for extreme and coupled environments. Although a focus on AE has become prominent in much of chemical and materials synthesis and discovery space, a specific focus on AE for extreme environments has seen a budding interest. Because this is a nascent area, our intent is to bring readers’ attention to issues impacting AI for materials science for coupled extremes. While we focus primarily on structural materials, many of the issues carry over to functional materials at coupled extremes. Over a decade ago, the Materials Genome Initiative first implored the world of science and engineering to achieve an accelerated pace of discovery and insertion into applications by combining experiment, theory, and computation using high-throughput methodologies.1 Since that time, the ability to optimize experiments for automation and high-throughput computation and analysis has successfully been achieved, yet broad autonomous research processes remain limited. Some approaches, for example, required intervention of a human in a biased decision process. Gómez-Bombarelli et al.2 was able to leverage computational libraries and then mine this space to develop a decision based on a human vote, leading to the synthesis and characterization of devices. Although autonomous research, such as the Air Force-based Autonomous Research Systems (ARES) program, has recently experienced some initial success, many of these studies have used computation, database or library building, machine learning to accelerate discovery and synthesis in monotonic, uncoupled campaigns.3 More recently, on-the-fly approaches have gained momentum, and discovery has not only been accelerated but has also occurred more autonomously, including work by Kusne et al.4 Still, the ability to decide new pathways during discovery, on-the-fly, remains largely unseen.

Critically, we explore the potential advantages and strategies that will further enable AE for a given experimental parameter space; in particular, we explore the complexities of coupled extremes and the particular challenge of simultaneously occurring phenomena with often disparate spatial and temporal scales. The cost of experiments is described in terms of the speed and throughput of automated experiments in contrast with the human resources needed to execute manual experiments. Although low-fidelity experimentation is relatively inexpensive, we discuss the role of AE and AI/ML strategies in high-fidelity “expensive” experimentation, along with modeling, simulation, and characterization. The combination of these strategies is deemed necessary to further materials understanding of materials under extreme environments in the AE loop, and we note the dependence of in situ and inline characterization to enable AE. Finally, a future outlook on intelligent or decision-informed AE, the mark of autonomy rather than automation, is discussed.

Materials selection and design

Selection and tailoring of materials for coupled extreme environments present unique challenges. Over the years, many different materials selection tools and databases have been developed with the democratization of open data material sets. At a macro level, materials selection tools such as Ashby plots5 can help initially screen for material classes that maximize a relevant performance index. For material classes that pass this initial screening, more accurate handbook data (classically, ASM Handbook6), data sheets provided by material suppliers, and online databases7,8,9 can help further whittle down the appropriate materials selection for the intended application. The ability to aggregate the collection of these data for materials selection has thus become a critical requirement for more rapid and comprehensive materials selection.

For coupled extremes however, the list of static materials properties/charts (e.g., strength-temperature versus temperature) has limited transferability to properties (e.g., strength, creep, fracture toughness, oxidation/corrosion resistance, and other degradation mechanisms) that may be dynamically changing due to exposure to extreme environments. Thus, there is a need to be able to characterize (via modeling and experiments) the physical mechanisms that drive these materials transients for extreme environment applications. Recent examples exploring novel alloy spaces while searching for a desired outcome in an extreme environment still follow traditional paths based on Ashby plots: Work by Nascimento et al. reveals both the promise and limitations of such approaches.10 These approaches work well for target functionality, be it magnetism, corrosion resistance, or strength with structure. In these cases, target properties in chemistry or structure are often benchmarked by a single functional property. This approach has worked well for theoretical searches of structural or elastic properties.11

However, assessing more than two projected outcomes, as is the case with coupled extremes, materials selection requires a more complex approach. Recent work12 in high-throughput density functional theory exploration has provided a pathway for multiple outcomes and a basis for multiscale exploration and higher fidelity theoretical or experimental approaches. Perhaps most notably, these high-throughput methods are rooted in machine learning models that allow for more complexity in property assessment,13 in some cases multiple environments14 and how these deep learning methods15 provide a translation to high-throughput experimentation, discussed in the next section.

High-throughput and autonomous experimentation

Materials development and deployment can take decades.16 Conventional component processing of advanced materials in a production facility can go through a series of processes (e.g., metal powder processing and consolidation, casting, billet formation, forging, heat treatment, machining, and final surface treatment). A major opportunity for expediting the materials development process is in the early stages of materials discovery, development, and property optimization. To do so requires a means to synthesize and characterize these exploratory materials rapidly, which is the goal of high-throughput experimentation (HTE). The materials by design (MbD) approach has shown significant promise across the field. It recognizes that efficient materials design requires that we thread a disciplined focus on specific materials performance goals throughout the processing, modeling, and testing of a particular material application. There are a number of MbD success stories across the broad materials community, but fewer in the domain of materials in extreme conditions. The Army Research Laboratory funded a 10-year program on Materials in Extreme Dynamic Environments, in which a 15–20% weight decrease in lightweight armor materials was realized through tightly knitted collaborations across processing, characterization, testing, and modeling efforts. General Electric applied principles of MbD to develop, validate, and commercialize an entirely new superalloy composition for use in gas-fired turbines for electricity generation.

The MbD approach that provides material improvements in years is a significant improvement over traditional Edisonian trial and error approaches that required decades to make such advances. AI-guided autonomous and automated high-throughput systems promise further orders of magnitude acceleration over MbD, by speeding up the individual design loops and leveraging rapid decision-making offered by AI tools.17 Nikolaev et al.18 demonstrated the first autonomous research system for materials development in their work controlling the growth rates of carbon nanotubes. Kusne et al.4 applied Bayesian active learning to phase diagram determination for metallic materials by creating continuous libraries of ternary alloys using cosputtering. The phase diagram of these libraries was mapped by autonomous selecting the compositions at which x-ray diffraction determined the phase boundaries and crystal structures.

During these materials exploration processes, there is a critical need to synthesize specimens with a wide range of composition and microstructure space in order to meet the expectation of capturing the potential materials landscape. Because microstructure drives important properties for materials at extremes (e.g., high cycle fatigue), it is important that microstructure be captured in samples that will be used to test those properties. However, some materials behavior is insensitive to microstructure (e.g., equilibrium phase behavior), and thus samples that may be easier to make and test may be used despite not having the target microstructure. In such cases, one potential strategy would be to screen for microstructure-insensitive (relying primarily on alloy composition as shown by Kusne et al.4) versus structure-sensitive (relying on both composition and microstructure19) properties. For structure-sensitive properties, further consideration should be devoted to properties that are driven by mean structure (e.g., yield strength) versus those that are driven by microstructural extremes (e.g., ductility and fatigue life) and an appropriate material element size to adequately capture the behavior of each property.20

Additive manufacturing (AM) is a potentially powerful technology for exploring compositional21,22 and/or processing parameter space to establish robust linkages between them. Using AM, one can rapidly make samples or test coupons with the target composition, shape, and size. However, the current means of AM building often involves significant human intervention in otherwise automated processes. For an AM development campaign to satisfy the requirements of typical standardized tests23,24 needed for component design, the whole process from build design planning to AM part synthesis, to testing and data processing can take weeks or months. And this process only comprises a single iteration of the design-build-test loop, and so is currently too slow for accelerated R&D goals. Another challenge for AM is that it is prone to generating material defects that can have a particularly significant impact on material performance in extreme conditions. Finally, despite its advantages, there does not exist a reliable transfer function ensuring an alloy that can be synthesized via AM can be made by traditional solidification processes or vice versa. As an example, Lass et al. demonstrated that standard heat treatments of AM-grown Inconel 625 resulted in the precipitation of negative side products, and new homogenization treatments were required to recover the desired microstructure.25 Despite these challenges, although AM can still contribute significantly to accelerated materials development, specimen preparation and manipulation should not be neglected when planning for an automated materials development workflow.

Characterization

There is a need for more rapid characterization methods and feedback mechanisms for materials properties in extreme environments. Important for AE workflows, the characterization method should be in situ, operando, or inline for best integration.17 We note that the characterization method for these extreme environments need not be as close to the final application space as possible, provided there is a correlation or transfer function that can translate between the characterization measurement and the material property of interest. For example, high-throughput nanoindentation26 extracts mechanical properties directly from the indentation load–displacement curve at both room and elevated temperatures very quickly.27 However, the materials properties obtained from this technique may not necessarily be indicative of the performance of the material in extreme environments, such as at high temperatures in which oxidation is likely to become much more prevalent. This highlights the need to consider coupling of extremes, in this case how extremely high temperature influences oxidation, which in turn influences the high-rate/high-pressure mechanical behavior of materials.

There are a wide variety of high-throughput characterization techniques available to evaluate phase stability, thermomechanical properties, corrosion, and oxidation. In situ x-ray diffraction can be performed under controlled environments up to moderately high temperatures, as has been demonstrated by Kramer and Ott.28 This type of approach can be used to study phase stability and provide information about oxidation of bulk and thin-film alloy samples for HTE or AE. More commonly, ex situ Raman spectroscopy is used, provided the oxide phase is alpha-alumina or otherwise Raman-active. Performing Raman measurements at relevant temperatures and pressures is possible as demonstrated by Maslar et al. for supercritical steam (25 MPa and 500°C).29 An analogous cell design was subsequently used to measure hydrogenation of HTE metal thin films at pressures up to 100 MPa.30 Aqueous corrosion can be measured through immersion in corrosive media or through electrochemical testing; both parallel and sequential screening techniques have been demonstrated for the latter. Recent ex situ studies on AM samples have looked at molten salt corrosion of cantor-based high-entropy alloys using ICP-MS of dissolved salt pills.31 Ambient mechanical characterization of thin-film, diffusion couple, and AM HTE samples is routinely performed ex situ via nanoindentation, which can provide measures of hardness, Young’s modulus, and wear resistance.

An important lesson for developing AE experiments can come from how HTE experiments learned to balance experimental pacing (throughput) and the fidelity of the measurement. It is useful to consider the acquisition time versus spatial resolution of each potential measurement technique as depicted in Figure 1. Often, HTE workflows are tiered such that a lower fidelity parallel or rapid serial primary screening is first used to interrogate a large phase space rapidly. Subsequent experiments reduce the overall throughput, say from hundreds to tens to ones, while simultaneously increasing the fidelity between steps. One point of emphasis is that the time to characterize (and gain knowledge from) a material should be balanced with the time required to synthesize and process the material. Long count times, extensive life-cycle analysis, or detailed synchrotron experiments are required for material validation but are best used in the secondary or tertiary screening step, unless synthesis and processing are time-consuming. Additionally, conscious decisions need to be made during the design of the HTE workflow whether to be risk averse (minimize false positives) or loss averse (minimize false negatives). This is only more complicated by the need to weigh the value of measuring different types of materials properties relevant to multiple and potentially coupled extremes.

Figure 1
figure 1

Bubble plot showing potential acquisition time and resolution ranges of various methods used to characterize fundamental material structure and properties. EBSD-single, Electron backscattered diffraction-single tile; EBSD-tile, EBSD-multiple tiles; EBSD-w/SS = EBSD-with Serial Sectioning; EDS-SDD, Energy Dispersive Spectroscopy-Silicon Drift Detector; GDOES, Glow-Discharge Optical Emission Spectroscopy; n-Indent, Nano-Indentation; OM, Optical Microscope; OM-w/SS, Optical Microscope-with Serial Sectioning; Opt Contrast, Optical Contrast; PLS, Photostimulated Luminescence Spectroscopy; SAWS, Surface Acoustic Wave Spectroscopy; SEM-single, Scanning Electron Microscopy-Single Tile; SEM-tile, SEM-Multiple Tiles; SEM-w/SS, SEM-with Serial Sectioning; TDTR, Time-Domain Thermo-Reflectance; X-ray PS, X-ray Photoelectron Spectroscopy; XRD-Lab, XRD-Lab Standard Wide Angle X-ray scattering; XRD-synch, XRD-synchrotron Standard Wide Angle X-ray Scattering; XRF, X-ray Fluorescence; μ-Indent, Micro-Indentation; 3DXRD (HEDM), 3DXRD-synchrotron (HEDM).

Full size image

For high-throughput experiments, scientists should carefully consider the gauge volume size of the characterization probe relative to the material’s dominant microstructural feature (e.g., grain, phase, etc.) size. This sampling ratio of gauge volume versus microstructure feature can be adjusted to explore a wide spectrum of mesoscale/heterogeneous to continuum response behavior. For example, XRD, which measures the material volume being irradiated by the x-ray beam, can be used for mesoscale (grain-level)32,33 or macroscale (powder/polycrystalline)34 studies. Likewise, the indenter size used for (nano-, micro-, and macro-) indentation studies35 will dictate the grain sampling size (subgrain, 10s of grains, 100s of grains, respectively). For imaging, adjusting the field-of-view to sample a large number of microstructural features or rastering the surface with faster measurements (e.g., EBSD tiling, nanoindentation grids,36 etc.) can be used to help circumvent the grain sampling size effect. In either approach, it is desirable to use a sampling size sufficiently large enough to capture a representative average of the stochastic behavior of the microstructure. In addition, appropriate sampling sizes may vary depending on the physical mechanism or material property characterized.20 As a result, although individual measurements for a particular measurement technique could be quite fast, the amount of time to obtain a physically useful (actionable) sampling size may make a particular measurement technique less amenable to high-throughput experiments.

Motivated by the potential to test the processing/structure/property landscape with minimum amount of material possible, miniaturization of specimens for subscale tensile testing,37 microtensile,38 microbending,39 and micropillar compression40 experiments will play an important role in material property exploration. Although this miniaturization certainly provides representative grain structures for mesoscale material modeling development and provides the ability to study fundamental grain-boundary deformation mechanisms, there are well-known size effects for these microscale experiments that can introduce errors in interpretation. The mechanical response is quite sensitive to loading-boundary conditions and misalignments. These smaller specimens also become more sensitive to near surface microstructure conditions (e.g., as-built AM) and consequences of sample preparation (surface roughness, machining, grinding, polishing, and residual stress imposition) as the surface area-to-volume ratio increases. Moreover, these microscale experiments often have dimensions on the same order as microstructure features (grains, phases, etc.). The sampling of a limited number of grain/phase boundaries, the limited grain-to-grain kinematic constraint, and the abundance of free surface-to-bulk ratio result in material response that is quite heterogeneous and may not be fully representative of bulk material response. Furthermore, sufficient statistical sampling size depends on the property of interest. For example, mean-driven materials properties (e.g., aggregate modulus) require a smaller sampling size relative to materials properties driven by extremal microstructure features (e.g., fatigue).

There is typically a tradeoff between gathering high-fidelity, information-rich data versus the ability to gather statistically significant quantities of data in a reasonable amount of time. Optical microscopy, profilometry, and EDS can provide large quantities of information about the material surface, but these tools lack detailed information about the materials three-dimensional microstructure. EBSD and XRD provide more information about crystallographic orientations and texture, but they are demanding to implement in a high-throughput workflow. All of these two-dimensional tools have the severe limitation that they provide information only at the surface of the material, which may or may not be representative of the bulk material. XRD and XRF have some ability to probe below the surface, providing “2.5D” information, which hold promise for high-throughput experimentation, but three-dimensional data remain the gold standard for more realistic representations of the material. Techniques such as serial-sectioned EBSD provide valuable and detailed data, but the destructive nature of these techniques makes it difficult to use these data for mapping material microstructure to performance. Synchrotron beamlines provide a powerful tool for characterization via lab diffraction contrast tomography (Lab-DCT), 3DXRD, high-energy diffraction microscopy (HEDM), and computed tomography (CT); however, the limited access to such facilities inhibit our ability to generate sufficient quantities of the necessary data from these facilities. There have been recent advances in laboratory-scale capabilities in CT and HEDM, but these suffer from poorer resolution data than that afforded by the beamlines (Figure 2).

Figure 2
figure 2

Spatiotemporal resolution ranges for core-loss spectroscopy methods. Direct detection electron energy-loss spectroscopy (EELS) offers the highest spatial and temporal resolution, at the cost of signal-to-noise ratio. EELS, Electron Energy-Loss Spectroscopy; STEM-EELS, Scanning Transmission Electron Microscope-Electron Energy-Loss Spectroscopy; XAFS, X-ray Absorption Fine Structure.

Full size image

Complexity of experimental sampling scales with the complexity of the fingerprint on which a high-throughput or autonomous experiment relies. Recent work in spectroscopy has provided a pathway for multimodal data to be utilized in autonomous frameworks.4,41,42,43,44,45,46

Modeling and simulation

Modeling and simulation serve to ground experimental observations in chemical and physical understanding, and thus are core to the development of AI tools, but these efforts are also critical to accelerated materials design and development. The models help to determine which chemistries are likely to have certain stable phases or desired fundamental properties. They help to link characterization and testing results at one scale to material performance at the application scale. They are also capable of incorporating multiple loading and boundary conditions that can reflect the effect of coupled extreme conditions. Finally, they provide a means to draw key physics-based conclusions from experimental data. All this said, there are clear challenges presented by modeling and simulation that must be understood before they can be a solid foundation for AI. A primary question is the balance of phenomenological and physics-based models. Physics-based models promise to incorporate behavior that is consistent with the materials performance in extreme environments, but such physics-based models are likely far too slow for high-throughput decision-making. Models that take hours to run cannot be an effective guide for high-throughput experiments that run in seconds. There is therefore a clear need for faster-running surrogate models, such as those developed from ML, that can effectively represent these physics-based models.

A major challenge to surrogate models is developing appropriate tools for rigorous verification and validation. Uncertainty quantification tools are needed to identify when the analyses stray too far from grounded data. If the data become too drastically extrapolated, the resulting data-driven surrogate is likely to be incorrect. Furthermore, the relative lack of interpretability of ML models makes it difficult to assess what important physical insights could be lost in using these approaches. In a high-throughput AI-driven setting, care must be taken to ensure that grounding to experiments is performed regularly so that the material iterations do not start down an unreasonable design path. Finally, any AI-driven decisions must come with a sense of the probability of both Type I errors (proceeding with a material that is not acceptable) and Type II errors (rejecting a material that is acceptable).

As previously stated, there currently do not exist examples of performing true in situ (or ex situ) coupled extremes experiments within the context of HTE or even non-HTE active learning settings. Challenges associated with performing true in situ coupled extreme measurements in either context are largely to the difficulty and cost in obtaining nondestructive measurements simultaneously. For instance, measuring oxidation mechanisms during high-temperature corrosion under mechanical loads requires a reliable noncontact nondestructive method of interrogation. Optical spectroscopy, such as Raman or luminescence spectroscopy, can be obtained (as in the work of Maslar et al.);29 however, not all oxides will have clearly defined Raman-active vibrational modes. Conversely the design of sample geometries and optical path lengths for diffraction measurements (a more general technique for phase analysis) require transmission measurements using high-energy x-rays typical of synchrotrons, as well as requiring small, nonrepresentative sample geometries.

Unfortunately, given the need to understand the dynamic processes in coupled extremes, oxide growth, failure, etc., ex situ measurements will not be sufficient to provide a comprehensive view of the underlying mechanisms. As such, careful consideration should be given to identifying minimally invasive direct and indirect tools for monitoring material behavior. In this respect, the successes (and failures) of the HTE community can serve as guidance for the coupled extremes community as it builds capability. As an example, different regimes of oxidation behavior, including oxide growth, roughening, spallation, etc. (although not equivalent to phase formation and tracking), could readily be observed optically and models trained to label the different regimes. Acoustic response of a material (as has been practiced by metalworkers for centuries) could be used to discern phase transformations, grain growth, and potential imminent failure.

Although HTE has provided a foundation for an increase in the number of systems both tested and discovered, challenges remain when precise mechanisms, intermediate or metastable states, or other complex structures or phenomena are targeted. AI/ML has provided a pathway for us to integrate this level of complexity into both discovery and analysis of materials synthesis and behavior at extremes.

Machine learning methods for design and development of materials

Machine learning focuses on the design and analysis of computer algorithms that improve their performance on a task with experience obtained from data.47 The integration of machine learning with materials research includes use of existing machine learning methods, libraries, and code to analyze experimental data,48,49,50,51,52 material simulations,53,54 and leverage prior knowledge from literature51,55 and experts.56 Additionally, material applications have led to the creation of a new subarea of “physics informed machine learning,” which is focused on enhancing extrapolation capabilities of machine learning using knowledge of physical phenomena to enable good performance even in domains where no data are available. The latter includes use of physics knowledge in the form of in-variances such as translation, rotation, scale, and permutation in-variance often present in material systems,57 differential equations or other relations governing materials properties,58 as well as physical constraints on material systems.59 Many applications of machine learning to materials science previously mentioned have focused on classical supervised (classification, regression) learning of models such as linear regression, logistic regression, support vector machines, decision trees, random forests, neural networks, and their deep extensions, which require significant amounts of labeled data. Other machine learning methods, such as unsupervised (clustering, distribution estimation, and dimensionality reduction) learning, which can work with unlabeled data, semi-supervised learning, graphical models, and causal inference, are also finding applicability in materials science,60,61,62,63,64 given their potential to enable interpretable predictions and discovery of unknown physics.

Given that data are not as readily available in the materials science domain (cost of experiments, run-time of simulation codes), in this article, we focus on machine learning methods that can be used to guide data collection. Specifically, this includes methods such as active learning, Bayesian optimization, bandit optimization, and reinforcement learning, where machine learning models are used in a closed loop to iteratively make decisions about what experiments to perform, what simulations to run, what queries to ask an expert, etc. Active learning and Bayesian optimization are increasingly being adopted for experimental design in materials science. Prior work in the past few years3,4,59,65,66,67,68,69,70 has demonstrated the promise of active learning and Bayesian optimization for efficient experimental design and AE by yielding significant savings over conventional methods such as factorial design. At the same time, more complex sequential decision-making methods such as bandit optimization and reinforcement learning have not been explored much by the materials science community, barring a few exceptions.71

Unlike classical supervised learning, machine learning methods for sequential decision-making iteratively refine a model by focusing on regions where the model is most uncertain (explore only) or regions that provide best tradeoff between uncertainty and maximal/minimal value of labeling function (explore-exploit tradeoff). The sampling is determined by an acquisition function that includes a specific notion of uncertainty of prediction and potentially a tradeoff with the predicted function value. These concepts will be explained further as we introduce the different sequential decision-making methods next.

In active learning, the goal is to learn the labeling function everywhere by sampling in regions with most uncertainty, and error is measured by probability of misclassification or mean square error in regression. This leads to explore only setting with common acquisition functions being the variance or standard deviation of prediction, disagreement regions between different ensemble learners, expected model change, expected error reduction, etc.,72 and often amounts to more sampling near the decision boundary in classification and in regions of less smoothness for regression as shown in Figure 3. Although active learning mostly refers to supervised setting, there is also work on active unsupervised learning such as learning clusters, matrices, and graphs using selective sampling of data points, features, edges, similarities, etc.73,74,75

Figure 3
figure 3

Sequential decision-making methods—active learning that focuses on exploration by sampling uncertain regions, Bayesian and bandit optimization as well as reinforcement learning (RL), which trade off exploration of uncertain regions with exploitation of optimum function value. Bayesian and bandit optimization focus on finding the optimal value of the function at each step or cumulatively over the entire steps, respectively, while RL considers cumulative value of the labeling function, where the latter can change depending on the state of the system (blue, orange, or green).

Full size image

Bayesian optimization falls under the umbrella of more general “stochastic optimization using machine learning” that involves machine learning methods to approximate the unknown function being optimized, where the latter is only accessible via samples. Instead of learning the function everywhere, here the goal is to optimize (find the maximum or minimum of) the function by trading off exploration of uncertain regions with exploitation of promising regions that are more likely to have optimal (maximum/minimum) function value (Figure 3). Bayesian optimization76 uses a probabilistic model for the function being optimized, most popular being the Gaussian process regression model. Bayesian models naturally come with a notion of uncertainty due to probabilistic modeling such as variance in Gaussian process models. Common acquisition functions include upper confidence bounds, probability of improvement, expected improvement, entropy, etc.

Bandit optimization also focuses on the goal of optimizing the labeling function; however, it requires controlling the value of the sampled function all through the optimization and not just finding the optimum or optimal function value at the end.77 This requirement is encoded in the error metric, which is cumulative over all steps for bandit optimization, thus exploring for all but the last step and yielding a good estimate of optimum at the last step does not suffice. This is relevant as materials scientists often want each experiment/simulation to be meaningful (e.g., yield a material with improving property). Often the same acquisition functions used in Bayesian optimization work to control the cumulative function value; however, the role and need for exploitation is much more pronounced when using bandit methods.

Reinforcement learning considers the setting where the same decision (which experiment/simulation to run) could result in different function values depending on the state of the system, and each decision could affect the state (which is not directly controllable) leading to a different labeling function.78 For example, applying different external fields could change the phase and hence resulting properties of the material. The goal is to optimize the cumulative (over all steps) value of the function while accounting for changed states and resulting labeling function at each step. This requires learning (explicitly or implicitly) the state dynamics and hence optimization is carried out over a longer time duration to find the best policy that generates an optimal sequence of decisions. As a byproduct, reinforcement learning is also able to handle settings where the function values (often called rewards) could be delayed. Methods for reinforcement learning include value-based learning, Qlearning, temporal difference learning, Monte Carlo, and policy gradient,78 as well as acquisition methods such as upper confidence bound or Thompson sampling, given connections with bandit optimization (the latter is essentially a constant state version of reinforcement learning).79,80

The methods for sequential decision-making using machine learning can also be extended to integrate and guide collection of data from multiple modalities, including multi-fidelity experiments and simulations,81,82,83,84 combining experiments and simulations,85 and combining experiments, simulations with expert queries.56 These methods are also amenable to achieving Pareto optimal solutions under multiple objectives.86,87

The machine learning methods for sequential decision-making use various notions of uncertainty (as previously mentioned) to guide sampling; however, these notions focus on uncertainty arising from noise or randomness in the label generation process, also known as aleatoric uncertainty. In many settings, there are many other epistemic sources of uncertainty such as model misspecification, biased sampling, data distribution or labeling function shift, etc.88 Although many of these issues are addressed in classical works, they focus on simple settings—for example, model misspecification and model selection are handled for simple models that have limited approximation capabilities and good solutions for complex models such as Gaussian processes and deep neural networks are mostly lacking,89 even for simple models these solutions are only known for supervised settings, while model selection for bandit optimization and reinforcement learning are just starting to be investigated,90,91,92 similarly biased and missing data solutions are known for missing-under-random settings but robustness to challenges of modern real data such as distribution/label shifts have only gained momentum recently.93,94,95,96 Additionally, uncertainty measures that scale well to high-dimensional settings and associated sequential decision-making methods are still in their infancy.97,98,99

Thus, while sequential decision-making methods using machine learning are increasingly being used in materials science, and have significant potential to accelerate the pace of material discovery, leveraging this potential to enable efficient exploration of high-dimensional material design space requires further work. In particular, tight-knit integration of materials and machine learning researchers to jointly explore the challenges and opportunities at this intersection holds great promise.

Toward autonomy

As discussed previously, spectroscopy toolsets have provided a platform for ML development due to their complexity.100,101,102,103 Developments have been focused on electron microscopy data, in part due to the availablility of large amounts of data as well as high data rates.104 For example, novel detector designs enable manufacturing processes to be viewed “in operando” at high acquisition speeds (>4000 frames per second, or fps). With data acquisition rates on the order of 2–3 TB/min, TEM gathers more data than ever before.41,105 The availability of rapidly acquired, detailed hyperspectral data sets has enabled the tracking and discovery of intermediate states, providing a hint toward grand challenges in materials at extremes, such as harnessing metastable phenomena at regimes far from equilibrium. These advances are critical, as auto-encoders and related frameworks have become increasingly important as a foundation for autonomous science. A major factor in the shift from automation to autonomy is the ability to make decisions and alter processes on the fly, based on those decisions. For the majority of materials phenomena, this is not achievable due to the speed at which the phenomena (phase transition, defect formation, etc.) occur.

An increased interest in edge computing and hardware specific ML106 for fast processes has made its way from communities such as high energy physics to materials science, allowing for rapid decisions, large data sets, and thus complex phenomena (as in the case of coupled extremes) to be tackled.

Summary and outlook

Although it is currently challenging to develop and assess materials rapidly and accurately, advances have been made that show the way forward for AI-guided high-throughput methods and AE that provide a foundation for advancing accelerated materials development for coupled extremes. Significant investment in rapid synthesis and in situ/inline characterization methods for high throughput and AE is needed. For materials in coupled extremes, the parameter space for composition, synthesis, and even characterization is highly complex, and not amenable to traditional reductionist approaches.107 The point of high throughput and AE is a rapid flow of complex results directly followed by the supply of new conditions for processing of subsequent materials iterations, made even more complicated by the multiple objectives inherent in coupled extremes. This complex space drives a need for rapid decision-making capabilities enabled by AI/ML methods. In particular, ML methods that exploit known or hypothesized physical and chemical behavior are expected to be powerful tools to accelerate the development of materials in coupled extremes (Figure 4).

Figure 4
figure 4

Spatiotemporal and complexity regimes for various phenomena. Autonomous platforms for a given process must account for latency to a decision and modality of the operando data collection.

Full size image

Autonomous platforms provide an opportunity to integrate functionality into materials evolution processes and, thus, we can now reimagine the way in which we develop, synthesize, and manufacture materials. In a world in which automation is pervasive, we must move from robotic tasks to machines that are capable of high-level decision-making. An overarching question when selecting, characterizing, or testing materials at extremes is, “do we need to choose between high throughput and precision approaches?” The answer is no, and thus the future lies in a smarter, more robust “high throughput” through which we maximize efficiency but maintain ability to pinpoint at low latency. Codesign of AI/ML, computing, and control methodologies will enable precise, low-latency decisions “on the fly”108 and are undoubtedly needed when spanning variable length and time scales for coupled extreme phenomena.