Biased decision making in materials science: Where does it originate and can it be avoided?

Science is often considered the domain of explicit truths about the world. These truths are learned and proven through observation and experiment. The scientific method and science as a whole have been extremely successful in helping humans obtain a clearer understanding of the world around them. Despite the seemingly token objectivity of science, philosophers have long warned of the limitations of individual scientists. In the search of absolute knowledge of the world, scientists are unintentionally and often unknowingly filtering and constructing a viewpoint that is influenced by human faculties. Awareness of this concept is known as epistemic humility.¹ Individual bias, both explicit and implicit, is one of the major factors that can influence a scientist’s understanding of the world and, in turn, their recording of it. In this way, the perspective of the scientist can shape everything from the interpretation of data to the kind of data that are collected, ultimately influencing the presentation of science. While independent perspectives have sometimes paved the way for important paradigm shifts that have changed humankind’s conception of the world, individual biases can also lead the scientific community down the wrong path if left unchecked.

To effectively collect, analyze, and present knowledge about the world, scientists and scientific organizations must make decisions throughout the scientific process. These decisions can range from large, easy to identify decisions about which projects to pursue to smaller, more implicit decisions, such as which observations to record during an experiment. This article will distinguish between larger decisions about the process and smaller decisions that direct the process. Oftentimes, the latter type of decisions receives less attention since these decisions are frequent, everyday tasks that would seem to have a small impact on the overall outcome of the project when compared to larger, more pronounced decisions. This article will examine different techniques used by individual scientists to make everyday decisions that direct the process of research in the field of materials science.

The field of materials science serves as a particularly interesting focal point for a study of how bias informs decisions made by scientists. First, the discipline is fairly research heavy, as it focuses on connecting the structure and properties of a material that result from processing its performance through characterization. Thus, there are plenty of examples throughout all stages of materials development where scientists must make decisions that affect the direction of the research process. Additionally, there are various paradigms regarding the research process within the discipline. A more recent paradigm shift involves the growing area of materials informatics,² a research strategy that uses computational techniques to increase the speed and efficiency of materials development. Elucidating the role of bias in different research paradigms can help researchers understand the situations in which each model, standard, or general way of thinking is most useful. As a whole, the field of materials science has been highly successful in achieving an advanced understanding of matter and its ubiquitous applications. At its best, the field operates as a macrocosm that catches its own mistakes through peer review and other continuous group efforts. However, even if these systems work well, it is possible for individual biases, limited foresight, and an occasional yield to external pressures to have damaging impacts on an individual’s career and, in the worst cases, taint the public’s trust in science.

Scientists are susceptible to some degree of epistemic humility throughout their research. This article will first broadly address how bias arises by explaining the interplay between heuristics (e.g., rules of thumb, mental short cuts),³ models (e.g., decision trees, combinatorial methods), and decision making in science. Next, several examples of heuristics and models used throughout the process of materials research are provided along with an analysis of the strengths and weaknesses of each in regard to attempting to mitigate bias for the purpose of making rational decisions. This article will also attempt to demonstrate that researchers in the field of materials science make decisions by using both natural heuristics and developed models, and as neither are inherently free of bias, the researcher must consciously find ways to combat individual bias.

The interplay between heuristics, models, and decision making in science

Individual scientists have to make many decisions throughout the scientific process. These decisions may be generally informed by theory or otherwise based on objectivity, but especially in more specific aspects of the research process, there is room for loosely informed decision making. The latter area is often referred to as “researcher degrees of freedom.”⁴ Decisions made in experimental design can be founded in known best practices, models, and theory, but unconscious decisions that occur during the execution of the experiment, such as when to stop collecting data, may be more arbitrary and based on heuristics.

Heuristics are perhaps the most common tool used by humans to make everyday decisions. Simply put, heuristics are “efficient rules or procedures for converting complex problems into simpler ones.”⁵ Heuristics will be considered part of implicit decision-making processes, while constructed models will fall into the category of explicit processes for decision-making. Heuristics can range from casual rules of thumb to more structured implicit processes that allow for efficient, short-term decision making. Tversky and Kahneman⁶ famously identified three types of heuristics used to make judgments in conditions of uncertainty, where scientists often find themselves operating: representativeness, availability, and adjustment. The representativeness heuristic states that if one thing resembles another, humans tend to assume that they are connected. The availability heuristic claims that humans are more likely to conclude the solution that comes to mind most easily. The adjustment heuristic claims that the starting point heavily influences the conclusion, because humans are bad at properly adjusting their thinking to come to the correct end result.⁶ Scientists may use heuristic techniques such as these throughout everyday decision-making processes. It is difficult to enumerate all of the heuristics used by humans, and as they come naturally and oftentimes subconsciously, they can be hard to identify. Likewise, the biases that come from relying on heuristics can often go undetected.

Historically, heuristics have been viewed as problematic for the field of science because of their susceptibility to human bias,⁵ especially on the level of the individual, where heuristics are most often used. If science is viewed as the attempt to produce objective knowledge to inform our understanding of the world, it seems there should not be room for assumptions or decisions based on subjectivity. On the other hand, philosophers and psychologists often point out the necessity and efficiency of heuristics. While they are prone to cognitive limitations, they nonetheless appear to be a necessary agent of regular decision making, as they aid in quick, cost-free assessment.⁵ Oftentimes, heuristics may be good enough to inform decent decisions. Regardless of whether heuristics are preferable for scientific decision making, they are natural even for scientists and can affect the scientific process in countless ways.

Though it appears that heuristics are both natural and necessary for individuals to perform efficient scientific research, they can be especially problematic for science. While intentional deception, such as overstatement of claims, is a clear example of bias in science, the idea that scientists are accidentally deceiving in science because of biased heuristic use is perhaps a bigger problem.⁷ Generally speaking, decisions are founded in the values and beliefs of the decision maker.⁸ These values and beliefs are likely influenced by societal or personal factors, which in turn influence the decision of the individual.⁹ As such, quick decision making relies on the experiences, general feelings, and developed habits or associations of the researcher.⁸ Though rational thinking may appear to be the hallmark of good decisions in science, emotion and intuition are often not only convenient but can perhaps be beneficial for reasonable decision making. Tried and true rational decision-making techniques such as lengthy pros and cons lists or cost-benefit analyses are not only time-consuming but may also present no clear choice. Emotions and intuitive thinking prevent us from remaining indifferent, enabling a decision even when the rational pros and cons seem to balance. However, a reliance on intuition invites personal bias. Thus, the efficiency that comes from relying on heuristics may come at the cost of systematic errors resulting from implicit bias.⁶ This biased judgment can affect everything from laboratory studies to data analysis. Without some kind of methodology or technique to check the objectivity of judgment, there is room for bias throughout the scientific process.

Relying on intuition can be problematic, so instead, scientists may prefer to rely on established methodology to come to a decision. Without some kind of universal methodology to guide decision making, decisions will rely strictly on the individual and may, in turn, be subject to a plethora of biases. In fact, it may even seem that heuristics go against the very intentions and goals of science, as science is deeply tied to rational thinking.⁷ Instead, scientists can choose to create and utilize models to reintroduce surefire logic into the decision-making process. The goal of a model is to represent a system as accurately as possible outside of the presence of the real system. Today, mathematical and computer-based models are praised for their apparent objectivity. However, models are constructed by humans and thus are subject to perpetuating the same biases as humans, though perhaps in more subtle (and potentially dangerous) ways. Nonetheless, models can be effective at informing decisions.

While models offer the opportunity to somewhat remove the human from the analysis process and introduce an external logical analysis system, misguided use or overuse of models can be a problem. Models are often not critically considered or continually developed, and they can be intrinsically biased. An analysis of a course in the Department of Materials Science at Stanford University entitled “Introduction to Heuristics of Invention and Discovery” found that students are typically presented with models and encouraged to treat them as fact instead of critically evaluating or continuing to develop them.¹⁰ Conversely, most models are constructed imperfectly and are subject to change over time. The authors of this analysis suggested that the way students used models indicated limited emphasis on abstract thinking in engineering curricula.¹⁰ However, this ability to abstractly and critically consider previously established models and methodology may be key to identifying biases or other weaknesses perpetuated by the model. After all, models are created by humans and thus may be based upon biases that could generate inaccurate models.⁵ In an attempt to create an ultimate exterior logic source, humans may inevitably build biased logic patterns and then mistakenly assume the models are perfect. Unlike the issue of basing decisions purely on heuristics, this decision-support tool may conceal the bias and thus make scientists over-trusting of the rationality of the model.

Up until this point, the focus has mainly been on bias at the level of the individual without any consideration of outside forces that may impact an individual’s decision-making agency. However, to paint a more complete picture of decision making in science, it is necessary to consider the external forces that impact an individual’s decision-making process. For example, data collection is often costly and time-intensive, and these factors may add pressure to make tradeoffs and quick decisions.⁷ Sometimes, pressures such as funding and publication timelines inhibit a scientist from proceeding in the most conscientious way possible. It is also true that not all decisions are left up to individual scientists; especially in the junior career stages, decisions about what projects to work on and how work is completed are made by external funding agencies or other organizations. On the other hand, individuals in positions of power occasionally have the opportunity to make top-down decisions. This type of decision is often problematic, though uncommon. Regardless of one’s level of individual power in the scientific community, any decisions made consistently enough can impact research culture, contributing to an eventual group bias if left unchecked. Bias on a group level is perhaps more dangerous than bias on an individual level. Once an entire group becomes susceptible to the same bias, it becomes extremely difficult for individuals of the group to recognize and speak up against it, a phenomenon referred to as “groupthink.”¹¹

At this point, decision making in science appears to be rather bleak: heuristics are biased due to the reliance on the individual, but models are also biased, as they may be based on assumptions made using heuristics and not questioned or revised over time as new information becomes available. How can the well-meaning scientist make informed, unbiased decisions about science? Hey suggested the use of second-order, or “meta-” heuristics to guide the mindful use of heuristic methods.⁵ This process would add an extra layer of consideration, ideally identifying and intentionally eliminating some of the repeated biases in heuristic use.

Additionally, becoming aware of personal bias and the role it may play in decision making is an important step toward working against biased judgment. Solomon pointed out that just because heuristics have biases and produce error does not mean humans are incapable of acting rationally.⁹ Scientific work is often carried out in groups, offering the opportunity for scientists to catch and correct errors that may arise because of individual bias.⁹ Individual bias is typically as obvious as it is problematic and the scientific community has built in checkpoints that serve to catch mistakes on the level of the individual. Throughout the academic research process, there are multiple opportunities for both peer and editorial review before publication. In industry, internal and external review panels exist to prevent or mitigate pitfalls such as bias. While these processes are largely successful at mitigating the impact of individual bias on published science, it is still possible for the review process to fail in the face of especially hidden biases or for group biases to prevail. The argument for diverse collaboration is especially strong here, as a broad range of perspectives and life experiences may be the most surefire way to catch errors created by intrinsic biases.

Case studies from the field of materials science

The field of materials science offers an interesting range of case studies detailing the opportunity for bias in scientific research. The field is heavily dependent on the research of materials throughout their life cycle, from ideation to production and testing to analysis and risk assessment. More specifically, this process often involves the identification of new materials, testing of materials and their key properties, presentation of data and information regarding the properties of materials, and analysis of the risks and benefits of material options. In this section, four case studies that relate to each of these areas will be presented. The general premise of each case study is discussed, along with the benefits and potential errors during their utilization to demonstrate the extent of bias that can arise in materials research.

Combinatorial materials science: A model for materials selection

Conventional materials discovery processes can be described as sequential methods that involve diving deeply into a chosen material and analyzing its properties to discover if it will fit a given purpose. Oftentimes, this process relies heavily on the knowledge and intuition of the scientists studying the material and may take a significant amount of time. However, a newer focus by researchers in the area of materials informatics has demonstrated the potential utility of involving computing techniques in the materials discovery process. Combinatorial materials science is one such process involving high-throughput experimentation and the evaluation of fitness functions to identify promising materials algorithmically by creating endless combinations of elements. This process may speed up the materials discovery process, giving users a competitive advantage when trying to get a new material to the market. Combinatorial materials science has been specifically applied in the search for new inorganic luminescent materials, also known as phosphors, which are used in energy-saving white light-emitting diodes.¹² Thus far, researchers have identified scientifically promising phosphors such as Sr₂CeO₄ using combinatorial methods, but these materials are not considered practical candidates. In fact, the materials were never further adopted by industry.¹² Though it is unclear exactly why Sr₂CeO₄, for example, was not pursued by industry, it is not uncommon for scientifically feasible materials to never make it to commercial development. Oftentimes, the cost of producing the material on a larger commercial scale may be unreasonable either because the process of scaling-up the fabrication is difficult and time-consuming, or the material itself is comparably more expensive than commercial competitors without a significant benefit. Additionally, the scientifically promising properties emphasized by academics may capture unique rather than practical aspects of the material, and important considerations such as operation lifetime may be overlooked. Despite the potential efficiency boost, choosing combinatorial methods over sequential methods can have risks, such as encouraging costly pursuit of materials that are not practically feasible.

Regardless of why phosphors identified through combinatorial methods never made it to the commercial world, this case study highlights an important critique of models. Some have criticized combinatorial methods as being simply “a methodology for methodology’s sake.”¹² To recognize the utility of models, it is necessary to consider how the model should be used. While combinatorial materials science, in particular, may not always present the most practical choice as the best option, it can limit the search space when looking for a specific new material. After all, models cannot replicate the intuition that researchers provide when performing a holistic analysis of a material, especially with respect to the material’s practicality. When models are used in ways contrary to their intended application, biased conclusions may be drawn. For example, misuse of this model may encourage application of Tversky and Kahneman’s availability heuristic during interpretation of the model’s outputs.⁶ The user may select the first material output by the model, as this is the easiest, fastest way to choose a material to pursue. If that material does not prove fit for market, the user may suspect a flaw in the model. However, in biasing the selections displayed first, the user is making the mistake by allowing the model to act as a decision maker instead of a tool to inform decisions, as it is intended.

As seen in this case study, assuming the combinatorial materials science model is a method for identifying the best overall material can lead to setbacks when industry rejects the model’s selection. The utility of the model can often reflect the goals of its creators. Generally speaking, a model developed solely by academics may be more focused on identifying scientific possibilities, while a model made in industry may be more focused on incorporating a variety of practical aspects, such as manufacturability, cost, and reliability. If combinatorial materials science is to achieve Park et al.’s original goal of producing phosphors that are fit for application in industry,¹² future work on their model may include finding ways to further incorporate the industry’s perspective into the computational model.

Scanning probe microscopy: The problem of small sample size in materials testing

Researchers in the field of psychology have found that scientists may be prone to making judgments about the success or failure of entire experiments based on a limited amount of preliminary results. Furthermore, they appear more likely to continue experiments that seem, based on the first few results, to succeed. Researchers also point out that time and money can have an effect on the tradeoffs that scientists are willing make during the research process. For example, a scientist with limited time or money may feel more pressured to maximize efficiency, which may lead to an increased emphasis on initial results.⁷ This natural heuristic process leads to biased interpretations of results and could negatively control decisions made during the execution of an experiment.

To understand how these pressures may be at play in the field of materials science, a specific materials assessment method is examined. Scanning probe microscopy (SPM) is a materials characterization technique used to image surface properties at the atomic scale. One of the most popular types of SPM is atomic force microscopy (AFM), in which a small probe is moved over the surface of a sample in a raster pattern. A single scan (Figure 1) typically covers an area on the order of microns for a sample whose size is on the order of centimeters; this can take several minutes to an hour to produce, depending on the desired resolution. As such, this technique is known to sample relatively small areas and take longer than other techniques. It is generally considered unreasonable to image the entire surface of the sample using this technique, thus, assumptions are made about the sample based on the results of several AFM scans taken at different areas over the sample.

Figure 1

An AFM image of a 2 µm × 2 µm area imaged by the author. The peaks indicate a height difference of ~10 nm and were likely caused by dust on the sample.

SPM may be inherently prone to errors resulting from the use of Tversky and Kahneman’s representativeness heuristic⁶ by assuming small sample sizes are representative of the whole. Each scan is small and costly, which may persuade scientists to limit the number of scans. However, as the probe used to scan the surface is very sensitive to things such as dust, initial results may measure the overall properties inaccurately. Some scanning probe microscopists recommend rastering the probe over the same area multiple times to clear the dust before considering a scan representative of the sample properties, making the process of obtaining a representative scan even more time-consuming and costly. After taking these factors into consideration, it is easier to see how an observed similarity between the first few scanned areas of a sample may cause a scientist to assume that this similarity is strong enough to indicate representativeness and support termination of scan collection. Additionally, some scanning probe microscopists wishing to scan as many areas as possible may choose to use smaller scan area sizes to reduce time and cost. However, this choice introduces new problems: One area may be small enough to fit within the size of a scratch or other abnormal feature of the sample, but the scientist may not be able to identify this region as part of a scratch because there is no different area within the scan to compare. At any rate, extrapolating results from a small sample size to formulate a general conclusion about the material imaged may lead a scientist to draw an incorrect conclusion about the fabrication or processing of the material.

Despite SPM’s susceptibility to bias, most scanning probe microscopists are aware of the problems that come with the technique’s limited sampling. Practically speaking, small sample size is often an issue of sample availability, not of an individual scientist’s poor judgment. When having a small sample size is unavoidable, bias can be mitigated by acknowledging the limitations of collected data and performing analyses to confirm that results are statistically significant. Small sample size can create significant bias when a scientist neglects to discuss potential error and ignores opportunities for statistical analysis. Another way for scanning probe microscopists to limit the likelihood of committing the small sample error is for them to be aware of the factors that influence decisions regarding the number of scans to be taken on each sample. Going into the experiment with a set number of scans of specified areas instead of trying to make decisions about whether to continue scanning during the duration of the experiment can be a way to prevent this intrinsic bias from affecting decision making during the experimental process. Despite scanning probe microscopists’ familiarity with small sample size bias, they must remain diligent throughout the experimental process, data analysis, and discussion to avoid mistakes motivated by attempting to save time and money when faced with external funding pressures or publication deadlines.

Data visualization heuristics

A paper published by Parish and Edmondson¹³ provides six suggested rules for improving the visibility of data. This paper provides examples of how materials scientists can apply heuristics to make decisions regarding data presentation for publishing. The six rules are to programmatically (as opposed to manually) generate figures, represent multivariate data in more creative ways than univariate charts, represent data more broadly than simply showing the mean and standard deviation, intentionally select colors that coordinate well with the data type, use small multiples, and not rely primarily on a software’s automatically generated data visuals to ideally represent data.¹³ Each of these rules can help minimize the likelihood that a reader erroneously interprets published data as a result of Tversky and Kahneman’s adjustment heuristic.⁶ In accordance with the adjustment heuristic, since a reader’s first interaction with data may be the visual presentation of it in a paper, the way in which it is presented can heavily influence conclusions. Parish and Edmondson emphasize the importance of being intentional in creating data visuals so that the key point comes across to the reader during their initial experience with the data, which is not always possible when using program defaults to generate figures. That being said, many of Parish and Edmondson’s heuristic rules are founded in intuition. This article provides an interesting example of meta-analysis of heuristics, as the authors not only identified the heuristic tools at work when attempting to present data but also analyzed them to identify best practices.

Parish and Edmondson attempt to elucidate why researchers make particular decisions. For example, they specifically examine the heuristic regarding the color choice for a specific data set. They suggest that mapped data should be displayed using colors that have some relation to a dimension of the data.¹³ The data visualized in Figure 1 are presented as a sequential color map, meaning the range of colors follows a smooth variation from one extreme to another, in this case from dark to light. Sequential color maps are especially useful when there are understood reference points or limits.¹³ Topographical AFM data are easily visualized using sequential color mapping because light peaks and dark troughs are a clear deviation from the flat reference point at zero displayed in a medium-toned color. However, without knowing the named heuristic or what a sequential color map is, a researcher may arrive at the decision to use the same color map to display the data. For instance, in the case of an AFM topographical map, it may make intuitive sense that the brighter colors represent peaks, and the darker colors represent troughs, as this aligns with our understanding of how light interacts with natural topography. Does it matter why the researcher chose to use a sequential color map? If the decision can be retroactively supported by reason, then a case can be made that intuition (which is often criticized for diverging from rationality) is more in line with rationality than previously considered, illustrating a specific case where heuristics can be used to make reasonably good decisions. That being said, it is important to recognize the reasoning behind intuitive decisions to identify best practice trends. By carefully considering how data are presented, it is possible to foresee and combat any perceptual failures that the reader could experience.

Decision trees: One of many models for nanomaterial risk assessment

A final example of a decision-making tool used in materials science are decision trees,¹⁴ which are relatively common models used to outline the choices that should be made based on certain premises. The utility of the decision tree is in its repeatability: Once a decision tree is developed for a certain circumstance, it can be used to make decisions in a multitude of similarly structured cases. In this way, it is possible to know that the cases are being analyzed in a parallel fashion, which appears to eliminate some of the space for individual bias in the decision-making process. However, decision trees are often constructed based upon an incomplete body of information, which could lead to systematic bias. There is always some risk (e.g., monetary, safety) of generating faulty conclusions from a poorly constructed decision tree, depending on the context of the decision.

One area where decision trees have been used is in the risk assessment of nanomaterials. Since the onset of their rapid development, some nanomaterials have been found to pose a variety of risks to humans and the environment. Research on nanomaterial development oftentimes outpaces the research into investigating potential negative effects of their use. As a result, various risk-assessment methods have been proposed to assist with faster, more proactive decision making regarding the safety of specific nanomaterials. Hansen et al. developed an assessment known as NanoRiskCat.¹⁵ The decision tree, seen in Figure 2, was used to determine the level of risk. This methodology is intended to help a researcher determine the relative (high, medium, or low) human hazard potential of an emerging nanomaterial.

Figure 2

A decision tree used in the NanoRiskCat methodology to designate an indicator of high (red), moderate (yellow), low (green), or unknown (gray) risk.¹⁶ Reprinted with permission from Reference.16 © 2017 Springer Nature

Incorrect use of decision trees can introduce bias in the same way as the combinatorial materials science model by inviting Tversky and Kahneman’s availability heuristic during human interpretation and use of the model’s outputs.⁶ Hansen et al. made it clear that the decision tree used in the NanoRiskCat methodology is intended primarily for knowledge communication, not to make a decision regarding future use of the material. It should clearly express the estimated magnitude of the potential hazard and exposure and provide stakeholders with easy-to-interpret information.¹⁵ Because of the possibility of systematic bias in models, it is more dangerous to have a decision tree that makes a decision about the future of a material. Instead, the decision made by the NanoRiskCat decision tree is about knowledge communication. There is still room for error in this model; if not assessed carefully, it can over- or under-represent certain risks. As is the case with all models, it should be revisited periodically and continuously developed by a group of scientists that can offer different perspectives.

Overcoming bias

Despite the ongoing success of the field of materials science, there are avenues for individual bias to percolate through the entire materials development process. However, materials science research includes many rigorous peer-review processes, as collaboration is key to minimizing bias. Incorporating different perspectives into the development of models as well as day-to-day decisions in the research process can provide the opportunity for bias-checking. Additionally, formulating experimental plans, ideally in a group setting, can prevent researchers from making biased decisions, such as premature termination of sampling, throughout the experimental process. If intuitive judgments are tested logically and at an abstract level, they can be developed into standing rules that guide future work. Decisions should be made by presenting data in an understandable way to various stakeholders and researchers and discussing the best course of action, not through automatic application of rules and models. Most importantly, the first step to catching bias in the research process is to be aware of its existence.