Scientific eminence and scientific hierarchy: bibliometric prediction of fellowship in the Australian Academy of Science

Abstract

Research metrics are known to predict many markers of scientific eminence, but fellowship in learned academies has not been examined in this context. The present research used Scopus-based citation indices, including a composite index developed by Ioannidis et al., (PLoS Biol 14:e1002501, 2016, https://doi.org/10.1371/journal.pbio.1002501) that improves cross-field comparison, to predict fellowship in the Australian Academy of Sciences (AAS). Based on ideas of a hierarchy of the sciences, the study also examined whether researchers from natural science fields were advantaged in achieving AAS fellowship relative to researchers from fields toward the social science end of the hierarchy. In a comprehensive sample of top global researchers, the composite index and its components all strongly differentiated Australian researchers who were elected as AAS fellows from those who were not. As predicted, when composite index scores were statistically controlled, researchers in physical and mathematical sciences were more likely to achieve fellow status than biological scientists, who were much more likely to achieve it than psychological, cognitive, and social scientists. Researchers in basic science fields also had an election advantage over those in more applied and technological fields. These findings suggest that recognition by learned academies may be predicted by citation indices, but may also be influenced by the perceived hardness, prestige, and purity of research fields.

It is now very well-established that bibliometric measures, such as citation indices, are associated with many aspects of individual scientists’ achievement, impact, and eminence. Metrics such as the h-index (Hirsch, 2005), for example, have been found to predict academic promotions (e.g., Jensen et al., 2009), academic salaries (e.g., Miller et al., 2022), success in obtaining research funding (e.g., Salaykar et al., 2017), and winning prestigious prizes (e.g., Kosmulski, 2020). The capacity of metrics to capture scientific distinction has driven a thriving field of research on the development, validation, and enhancement of bibliometric indices.

Two major challenges this research faces are the proliferation of bibliometric indices (Bornmann et al., 2011) and the difficulty of making fair comparisons of researchers from different scientific fields. The diversity of indices, each assessing different aspects of scientific productivity and impact, makes it challenging to develop an economical and comprehensive picture of a researcher’s impact or distinction. Similarly, substantial variability between fields in publication and citation practices—for instance, disciplinary differences in number of outputs and co-authors (Glänzel & Thijs, 2004; Harzing et al., 2014)—have made it difficult to develop commensurable indices that allow unbiased cross-field comparisons of scientific achievement.

One promising approach to addressing these two challenges has been developed by Ioannidis and colleagues (Ioannidis et al., 2016, 2019, 2020). Using Scopus, these researchers developed an annually updated database of 100,000 top researchers, classified into 22 broad fields and 176 Science-Metrix subfields, based on whole-of-career citation impact. The database includes several citation metrics that collectively capture not only raw productivity and impact but also evidence of research leadership and fractional contribution within research teams. Importantly, Ioannidis and colleagues developed a composite index (c) that sums standardized values of six (log-transformed) metrics: (1) total citations, (2) Hirsch’s h-index (Hirsch, 2005), (3) co-authorship-adjusted Schreiber h_m-index (Schreiber, 2008), (4) number of citations to papers as single author, (5) number of citations to papers as single or first author, and (6) number of citations to papers as single, first, or last author. Their database contains the 100,000 researchers who score highest on this index among the more than 6 million researchers whose Scopus profiles include at least five publications. The top researcher sample therefore represents approximately the top 1.5% of global researchers according to career-long holistic citation impact.

The composite index has two major strengths. First, it appears to be a more valid predictor of scientific eminence than its individual components. For example, Ioannidis et al. (2016) found that the 47 Nobel laureates from 2011 to 2015 were more likely to excel on c (n = 31) than on total citations (n = 15), h (n = 18), or h_m (n = 26). Follow-up work by Kosmulski (2020) on 97 Nobelists from 2010 to 2019 found that c out-performed citation count, an h-index variant, Web of Science ‘Highly Cited’ status, and number of ‘hot’ or highly cited papers. Second, in addition to being more comprehensive and predictively powerful than single component indices, the composite c-index has the advantage of being relatively commensurable across major scientific fields, unlike h or total citation count. Because c effectively adjusts for cross-field differences in co-authorship patterns, it removes the advantage fields typified by large team publication have on those indices, and because it is based on log-transformed components it reduces cross-field differences due to positively skewed metrics such as raw citation counts. Normative data presented by Ioannidis et al. (2019) show that the standard deviation of 90^th percentile total citations across 22 research fields is 70.1% of the cross-field mean, whereas the corresponding standard deviation for c is 10.4% of its mean. To illustrate, a biomedical researcher at the 90^th percentile for total citations has 255% more citations than an engineering researcher at that percentile, but their c index would only be only 15% higher.

One benefit of a predictively valid and commensurable holistic citation index is that it would enable fair comparisons of the achievement or eminence of researchers across research fields. Such an index could provide a level playing field for evaluation and a tool for detecting evaluative bias. Bias might be suspected if equally impactful researchers from different fields, as assessed by the index, systematically receive differential recognition or reward. The possibility of such biases is raised by the idea of a ‘hierarchy of sciences.’ This concept, advanced by Auguste Comte (1875), proposes a hierarchy running from mathematics, through astronomy, physics, chemistry, and biology to sociology. Moving up the hierarchy from mathematics to sociology, Comte argued, the phenomena of interest become increasingly complex and decreasingly abstract and general. Historians and sociologists of science have also argued that the hierarchy is associated with declining perceived “hardness,” marked by decreasing levels of scientific consensus, decreasing rigor and certitude, and slower explanatory progress (Fanelli & Glänzel, 2013). The hierarchy of the sciences is also associated with a hierarchy of prestige or status (Becher, 1994).

In recent years substantial evidence for the hierarchy of sciences has emerged. On multiple factors, studies consistently find evidence of a gradient with physical and mathematical sciences at one end, the social sciences at the other, and the biological sciences intermediate. Behavioral or cognitive sciences invariably fall between social and biological sciences. This ordering is evident across a wide variety of bibliometric and other indices, from the prominence of graphs in scholarly work (Smith et al., 2000), to the level of consensus in peer evaluations of research and the ratio of theories to laws (Simonton, 2004, 2015), to a diverse collection of bibliometric indices (Fanelli & Glänzel, 2013). For example, Simonton demonstrated a robust ordering of physics, chemistry, biology, psychology, and sociology across a diverse set of indices. Studies have shown that this ordering is also associated with the likelihood that a field’s articles report positive results (Fanelli, 2010), and with the degree to which fields share a common scientific vocabulary (Benjafield, 2020). Relations among fields appear to be much better captured by a “gradualist” ordering than by a dichotomy of natural science versus social science. Comparable orderings can also be observed among subfields within particular sciences. Smith et al. (2000), for example, found that graph prominence varied widely across psychology’s subdisciplines, resembling biology (but below chemistry and physics) in the neuroscientific and animal behavior fields, and resembling the social sciences (economics and sociology) in the counselling, educational, and clinical areas. Similarly, Fanelli and Glänzel (2013) showed that among the biological sciences, bio-molecular disciplines had a more natural-scientific bibliometric signature than zoology, botany, or ecology.

In view of the substantial evidence for a Comtean hierarchy of sciences and the association of greater prestige with the natural or physical scientific end of the spectrum, it is plausible that the evaluation of research might be influenced by its field’s position on the hierarchy. If this were the case, otherwise equivalent research works or researchers might be evaluated differentially based on their relative positions, with a bias toward more natural-scientific fields. This possibility has yet to be formally evaluated in research on the hierarchy of the sciences.

If that hierarchy plays a role in the recognition of scientific eminence, this role should be most evident in contexts in which scientists from a broad range of fields are evaluated and compared, whether implicitly or explicitly. One such context is election to fellowship in learned academies. These academies, usually national in scope, recognize distinguished achievement in broad fields of research. Fellowship is bestowed following a thorough process of nomination and evaluation of candidates, often with strict limits on the number of new fellows to be admitted each year. Academies such as the Royal Society of London and the National Academy of Sciences (NAS), harness the work and prestige of their fellows to promote their field of expertise to government and the wider public. Because fellowship of learned academies is a formalized sign of distinction, esteem, and impact in a field of research, it is a promising focus for research into the predictors of scientific eminence. As yet, there has been little published research on bibliometric prediction of fellowship, although Hirsh (2008) demonstrated that NAS fellows tended to have very high h-index scores.

The Australian Academy of Sciences (AAS) is a not-for-profit learned academy that was founded in 1954. According to its website (www.science.org.au), “Fellows of the Australian Academy of Science are among the Nation’s most distinguished scientists, elected by their peers for ground-breaking research and contributions that have had clear impact.” Election is an annual process in which nominated candidates are evaluated by 13 sectional committees representing different scientific fields (including interdisciplinary), based on scientific achievement (currently weighted 60–85%), national and international profile (10–35%) and leadership, mentorship, and promotion of science (5–30%). Their recommended candidates are then reviewed by the AAS Council who select a final list that goes to a ballot of current fellows, up to 20 new fellows being elected annually. Between its inception and 2022, 895 fellows had been elected and as of March 2023 the living fellowship, not including foreign corresponding membered, numbered 581. The academy’s mission statement lays out its primary aims.

The Australian Academy of Science provides independent, authoritative and influential scientific advice, promotes international scientific engagement, builds public awareness and understanding of science, and champions, celebrates and supports excellence in Australian science.

The AAS sits alongside four other national academies: the Australian Academy of Technological Sciences and Engineering (ATSE), the Australian Academy of Health and Medical Sciences (AHMS), the Academy of the Social Sciences in Australia (ASSA), and the Australian Academy of the Humanities (AAH). The first three of these academies represent groups of scientific fields, but the AAS has a broad, umbrella remit and is not in field-based competition with them. Indeed, many AAS fellows also hold fellowship in the other scientific academies (118 in ATSE, 70 in AHMS, 8 in ASSA as of April 2023). Humanities researchers would normally not be elected fellows. In view of the academy’s broad remit to encompass science as a whole, AAS’s fellowship is a suitable eminence marker for use in bibliometric studies of scientific achievement.

The present study investigated whether bibliometric indices predict fellowship of the AAS and examined two key research questions. First, we asked whether the c-index would predict AAS fellowship. Second, we asked whether likelihood of having been elected to fellowship is associated with scientific field in a manner consistent with the hierarchy of sciences. Specifically, we evaluated whether, holding scientific accomplishment constant (indexed with cross-field commensurability by c), scientists working in fields at the social end of the hierarchy—such as the psychological, behavioral, cognitive, or social sciences—would have a lower likelihood of achieving AAS fellowship than scientists in fields higher in the hierarchy (e.g., mathematics and physical sciences).

Method.

Sample of Researchers

The latest version of the science-wide author database of standardized citation indicators developed by Ioannidis and colleagues (e.g., Ioannidis et al., 2016) was used to estimate career impact. It uses Scopus author profiles from a September 1, 2022, snapshot, to select the top 100,000 scientists ranked by c, plus any scientists outside that set who are in the top 2% in their sub-field. The database contained 194,983 scientists from 20 fields defined by the Science-Metrix classification (see Ioannidis et al., 2020, for details on this journal-based classification). We created a full name variable from the existing variables (surname and forename). A dataset of all AAS fellows (inclusive of 2022 fellows) was also extracted in early 2023 from a listing of all fellows awarded on the Academy website. From the 949 listed fellows, we excluded those who were deceased or corresponding international fellows, leaving 581 ordinary and specially elected fellows. We created a full name variable from the existing variables (surname and first name) and changed the primary country of affiliation values to iso codes.

The final dataset was created by linking the citation metrics from the Scopus database to the AAS dataset by fellows’ full names. The linked dataset yielded more than 50 cases where two or more researchers’ records were matched in the citation database, so a manual search was conducted to disambiguate these cases. Each case was checked to see whether the information in the AAS profile matched the Scopus data, country iso-code, institutional affiliation, and scientific field. For a few researchers who had more than one Scopus record, the record with the longer publication period was selected. Name-matched cases with a mismatch between AAS and Scopus profiles on country, institutional affiliation, and/or scientific field were resolved by searching the fellow’s last name in the Scopus database (e.g., often the author was recorded by their preferred or middle name instead of their first, or with an initial as their first name). In some cases, this resolution process determined that the researcher was not included in the top researcher database. To reduce the probability of retaining false matches in the combined database, the manual case resolution was undertaken independently by both authors. Once it was concluded, 85% (492) of the 581 eligible (living, non-corresponding) AAS fellows had records in the top researcher database, and 88% (432) of these researchers were listed there as being affiliated with an Australian institution. For each researcher, the final data set included their national affiliation, institutional affiliation, multiple citation indices, their broad research field, and AAS fellowship status.

Measures

Research fields were classified in two ways. First, we employed the top researcher database’s classification of researchers into 20 Science-Metrix (SM) fields, which assigns a single primary field to each researcher based on a detailed and comprehensive classification of journals. Second, to construct a less differentiated classification, we combined these fields onto the six broad fields recognized by the National Academy of Sciences (2023) as follows: 1. Applied Biological, Agricultural, and Environmental Sciences (SM: “Agriculture, Fisheries & Forestry” and “Earth & Environmental Sciences”); 2. Behavioral and Social Sciences (SM: “Economics & Business,” “Psychology & Cognitive Sciences,” and “Social Sciences”); 3. Biological Sciences (SM: “Biology”); 4. Biomedical Sciences (SM: “Biomedical Research,” “Clinical Medicine,” and “Public Health & Health Services”); 5. Engineering and Applied Sciences (SM: “Built Environment & Design,” “Enabling & Strategic Technologies,” “Engineering,” and “Information & Communication Technologies”); and 6. Physical and Mathematical Sciences (SM: “Chemistry,” “Mathematics & Statistics,” and “Physics & Astronomy”). Four additional Science-Metrix fields from the humanities (“Historical Studies,” “Philosophy & Theology,” “Visual & Performing Arts,” “Communication & Textual Studies”) were excluded from this classification as they fall outside the purview of a science-based academy, their scholarly output is not well captured by Scopus, and few researchers from these fields were represented in the top researcher database (n = 2,305; 1.2%).

The present study employed citation metrics reported in the 2022 update of the Scopus database (Ioannidis et al., 2016, 2019, 2020), which covers citations accrued from 1996 to the end of 2021. The database includes alternative versions of each metric that either include or exclude self-citations, and we consistently employed the latter. Of primary interest was the c-index (without self-citations), which sums standardized values of its six component log-transformed citation metrics (i.e., log(index + 1)/log(index^max + 1)).

Results

Table 1 presents descriptive data on the number of researchers and their mean c-index score for all researchers in the Scopus top researcher database (N = 194,989), for all researchers with a listed Australian affiliation in the database (n = 6,629; 3.4%), and for AAS fellows who had a listed Australian affiliation. The latter restriction reduced the set of identified fellows from 492 to 432 and was made to enable direct comparisons among Australia-based fellows and non-fellows, although there was no significant difference on c between AAS fellows with (3.86) and without (3.87) Australian affiliations, t(71.1) = 0.13, p = 0.89. AAS fellows made up 6.5% of Australian-affiliated researchers in the top researcher database, or 7.4% of the Australian-affiliated researchers in Science-Metrix fields with at least one AAS fellow. Table 1 indicates that AAS fellows consistently exceeded the mean c-index score for all Australian researchers. Based on a reported fellowship of 581, the median AAS fellow has a c score of 3.75, implying a ranking at the 99.4th percentile of researchers in Ioannidis and colleagues’ initial set of more than six million. The presence of 492 AAS fellows in the database of 194,989 top researchers indicates that 88% of fellows fall in the top 2.9% of researchers globally based on c, with 415 fellows (71.4%) in the top 100,000 (1.5%).

Table 1 Number of researchers and mean composite index (c) score by Science-Metrix field for all researchers, Australian-affiliated researchers, and Australian-affiliated AAS fellows in the Scopus top scientist database

Table 2 presents comparable data to Table 1 with scientific fields reorganised into the more economical NAS field classification. The table also reports data for an adjusted version of c (c*), which improves the index’s cross-field equivalence. Table 1 shows modest differences in mean c-scores in the top researcher database by field, with ‘Psychology & cognitive sciences’ highest (3.76) and (among scientific fields) ‘Engineering’ lowest (3.25). To remove these remaining cross-field discrepancies, c* scores were computed for each researcher by adding the discrepancy between their field’s mean c-score and 3.76 to their personal c-score (e.g., adding 0.51 to every engineer). Table 2 confirms that c* does not differ between broad scientific fields, that Australian researchers in the top scientist database do not differ meaningfully by field on it, but that in every field Australia-based AAS fellows again substantially exceed the mean c* score for all Australia-based researchers.

Table 2 Number of researchers and mean composite index (c) or adjusted composite index (c*) score by National Academy of Science field for all researchers, Australian-affiliated researchers, and Australian-affiliated AAS fellows in the Scopus top scientist database

To test whether there are differences in scores on c and its components between AAS fellows and other top scientists, we compared Australian-based fellows (n = 432) and non-fellows working in fields with at least one AAS fellow (n = 5030). Table 3 reports Welch’s t-tests and shows that AAS fellows strongly exceed non-fellows on every index, with citations to single-authored papers the weakest differentiator. Cohen’s d values indicated that these effects were typically very large. To replicate the c* index finding using a different method, we carried out a logistic regression analysis testing for a linear relationship between c* and the log(odds) of fellowship and obtained the same relationship (z = 22.51, p < 10^–16, R²_McFadden = 0.13). The distribution of c* for AAS fellows and non-fellows is presented in Fig. 1 and the predicted likelihood of AAS fellowship as a function of the index in Fig. 2, which reveals a steep rise in probability of election at higher levels of the index. Evidently, AAS fellows tend to excel on a holistic citation metric even relative to the population of leading Australian and global researchers.

Table 3 Differences between Australian-based AAS fellows and non-fellows on the c-index and its components (note: component means represent transformed values)

Fig. 1

Distribution of Australian-based AAS fellows and non-fellows in fields with at least one fellow on the c* index (means represented by dashed lines). Note Dark grey distribution = AAS fellows. Light grey distribution = non-fellows

Fig. 2

Predicted probability of AAS fellowship as a function of c* index (black dots = AAS fellows, grey circles = non-fellows)

Turning to the question of whether there are field-based differences in likelihood of AAS fellowship, holding constant scientific eminence or impact, and whether any such differences are compatible with a ‘hierarchy of sciences’ explanation, data from Tables 1 and 2 offer some preliminary evidence. Figure 3 presents the proportion of Australian-based researchers in the Scopus database who work in each Science-Metrix field – including only fields with at least one AAS fellow – who are AAS fellows. Fields with clear positions in the science hierarchy are represented (in order) by black bars and other fields (in alphabetical order) by grey bars.

Fig. 3

Proportion of top Australian scientists in Science-Metrix research fields who are AAS fellows

Although Table 1 reveals no systematic tendency for Australian-based researchers to deviate from the global mean c-score in some fields more than others, there are massive differences between fields in the likelihood that top researchers are recognized as AAS fellows. More than 30% of top researchers in mathematics and statistics are fellows, compared to less than 2% of top researchers working in psychological and cognitive sciences, or in agricultural science, fisheries, and forestry. Consistent with the hierarchy of science hypothesis, the probability of being an AAS fellow dropped monotonically from mathematics, to physics and astronomy, to chemistry, to biology, to psychological and cognitive sciences. Relatively low fellowship probabilities were also evident among the more applied and technological fields relative to their adjacent basic science fields (e.g., clinical medicine versus biology, earth and environmental sciences versus physics). This disparity suggests a basic or pure versus applied hierarchy separate from the Comtean hierarchy of basic sciences.

Figure 4 shows the proportion of AAS fellows for the more parsimonious National Academy of Sciences field classification, using data from Table 2. As before, there is clear evidence for a hierarchy of science effect, with the probability of fellowship markedly higher for top researchers in physical & mathematical sciences (17.9%) than those in biological sciences (13.5%), and especially those in behavioral & social sciences (0.4%). We note that this last value may be an under-estimate, because some of the social science fields included in the category sit in the humanities: all AAS fellows in the behavioral & social science grouping were from the psychological and cognitive sciences. Applied physical and biological sciences and biomedical science have fellowship probabilities that are low relative to the basic physical and biological sciences.

Fig. 4

Proportion of top Australian scientists in National Academy of Sciences research fields who are AAS fellows (Applied BAE sciences = applied biological, agricultural & environmental sciences)

To test whether probability of AAS fellowship varies by scientific field after controlling for career scientific impact (assessed as c*), we conducted logistic regression analyses that predicted fellowship from c* and included dummy variables representing scientific fields using the Science-Metrix and National Academy of Science classifications. Field classifications that included the physical sciences were selected as the baseline field (i.e., the one with no dummy) relative to which higher or lower fellowship probability was tested. Findings of the respective analyses are presented in Tables 4 and 5.

Table 4 Summary of logistic regression analysis predicting AAS fellowship among top Australian scientists in fields with at least one fellow, using the c* index and Science-Metrix research field

Table 5 Summary of logistic regression analysis predicting AAS fellowship among top Australian scientists using the composite citation metric and National Academy of Sciences research field

Using the Science-Metrix research field classification, Table 4 indicates that although c* is the most powerful predictor of likelihood of AAS fellowship, there are also many significant differences between fields when the index is statistically controlled. Consistent with the hierarchy of sciences hypothesis, the likelihood of fellowship for researchers in mathematics & statistics was marginally (albeit non-significantly) higher than for researchers in physics & astronomy. The likelihood was marginally lower for chemistry, significantly lower for biology (46% lower odds; OR = 0.54, 95% CI [0.37, 0.80]), and much lower for psychological and cognitive sciences (95% lower odds; OR = 0.05 (95% CI [0.008, 0.18]). Fellowship likelihood was also substantially reduced for researchers in more applied fields (agriculture, fisheries & forestry; clinical medicine; enabling & strategic technologies; engineering) and modestly or nonsignificantly lower for earth and environmental scientists and biomedical researchers. Table 5 tells a similar story for the coarser National Academy of Science research fields. The c*-index was again the most powerful predictor of fellowship status, but all five fields were associated with a significantly lower likelihood of fellowship relative to the physical and mathematical sciences. This lowered likelihood was weakest for biological sciences (39% lower odds; OR = 0.61, 95% CI [0.43, 0.85]), consistent with its adjacency to physical sciences in the hierarchy of sciences, and it was greatest for behavioral & social sciences (94% lower odds; OR: 0.06, 95% CI [0.009, 0.21]) and engineering & applied sciences (92% lower odds; OR: 0.08, 95% CI [0.05, 0.12]).

Figure 5 illustrates the differential association between c* and AAS fellowship for the NAS fields based on modelling for each field separately. It shows that Australian scientists of equivalent global distinction—that is, equal scores on an index of holistic citation impact that is distributed very similarly across different fields—have different likelihoods of recognition as AAS fellows. From highest likelihood to lowest, the ranking is: (1) physical and mathematical scientists (inclusive of chemistry researchers), (2) biologists, (3) applied biological, agricultural & environmental scientists, (4) biomedical scientists, (5) engineers & applied scientists, and (6) behavioral and social scientists. For a scientist whose c* score is 4.09, the mean for AAS fellows in the database of top scientists, the probability of fellowship is 30% for physical and mathematical scientists, 22% for biological scientists, 11% for applied biological and biomedical researchers, 3% for engineers and applied scientists, and 2% for behavioral and social scientists.

Fig. 5

Predicted probability of AAS fellowship as a function of the c* index, separately for National Academy of Sciences research fields

Discussion

The present research offered support for both of its main predictions. First, being elected to fellowship in the AAS was associated with very high levels of accomplishment on standard citation metrics and on the composite c-index. The median score of Australian-affiliated fellows on this index placed them above the 99.4th percentile of global researchers, and most fellows were represented in the database of top researchers. AAS fellows excelled even relative to this elite cohort, scoring substantially higher than non-fellows. The likelihood of election to fellowship is nonlinearly related to c-index score, rising steeply at very high levels of the index. By implication, AAS fellows are a distinguished group of scientists, and c captures their distinction with impressive predictive validity.

Nevertheless, our findings make clear that citation metrics fall far short of fully accounting for who has been elected to AAS fellowship. The c-index explained 13% of the variance in fellowship status in the cohort of top Australian researchers who work in fields in which fellows have been elected. Restriction of range in the predictor ensures that this is an under-estimate of the index’s predictive power among all researchers, but factors other than holistic citation impact must play a large role in election to fellowship. An Australian researcher in an AAS-relevant field with a c-index score at the mean for current AAS fellows has a less than one in five chance of being a fellow. This finding suggests that although very high citation impact is typical of fellows, it is at best a quasi-necessary but clearly insufficient condition for fellowship.

The additional attributes that enable bibliometrically outstanding scientists to become fellows in learned academies such as the AAS is a matter for speculation. Scientific reputations may be boosted by single breakthrough findings that are not well represented by career-long citation summaries. They can be enhanced by service as a leader in public or professional organizations or by visibility as a science communicator. More problematically, reputations can be burnished by institutional prestige, and nomination and election processes can be distorted by informal scientific networks and by gender and other biases. A fuller understanding of the determinants of election to learned academies requires attention to factors beyond bibliometrics, although our findings suggest that these metrics are a key element as valid signals of scientific accomplishment and impact. Few fellows had not achieved exceptional holistic citation impact in their field.

Our second prediction, that researchers’ likelihood of being elected as AAS fellows would differ according to their field’s location on the hierarchy of sciences, holding holistic citation impact constant, was also supported. A much higher proportion of top Australian researchers in some scientific fields have been elected to fellowship than in others. Whether research field was classified into fine-grained Science-Metrix categories or into broader NAS groupings, researchers in fields at the social science end of the hierarchy were substantially less likely to be AAS fellows than their equally distinguished peers at the physical and mathematical science end—as indicated by a holistic index that removes mean differences between fields—with biological scientists intermediate. Our findings demonstrate that relative to physicists and mathematical scientists, researchers in all other Science-Metrix fields except chemistry and biomedicine, and in all other NAS field groupings, were significantly disadvantaged in relation to AAS fellowship status. This disadvantage was largest for researchers in engineering and applied sciences and those in behavioral and social sciences. Among basic science fields the magnitude of the disadvantage tracked the field’s rank distance from physics and mathematics: least for chemistry, more for biology, most for behavioral and social science. As a corollary of some fields being less likely to be recognized by AAS fellowship, fellows working in these fields tended to have higher c-scores than those in advantaged fields.

Our findings imply that the composition of the AAS fellowship would look very different if the probability of fellowship conditional on exceptional citation impact was equalized across scientific fields. The proportion of fellows from physics, mathematics, chemistry, and biology would shrink substantially, and the proportion of applied, clinical, behavioral, and social scientists would increase. However, we do not argue that this outcome would be necessarily just or desirable, nor that the unequal likelihood of fellowship as a function of research field is necessarily an indication of bias in the election of AAS fellows. For a start, the current composition of the fellowship is a legacy of decades of election decisions, stretching back to a time when the national and international landscape of science was markedly different. Current imbalances, if they are such, cannot be ascribed to current biases in the selection of fellows, though such biases may persist.

More fundamentally, it could be argued that bestowing honors on researchers from one field over researchers of equal accomplishment and distinction from another, ceteris paribus, may not constitute selection bias. Is it necessarily a bias to recognize a 99th percentile scientist from field X over a 99^th percentile scientist from field Y? It might be argued that other things are not equal because some scientific fields are (a) more valuable or (b) more difficult than others, or that (c) the evaluation of career-long scientific accomplishment afforded by citation indices is an invalid baseline for establishing equivalence across fields. Alternatively, a defender of the AAS status quo might accept that election outcomes have been biased against some fields, but (d) that this is justified by other goals of the organization.

In our view, none of these defences are persuasive. The suggestion that some fields are intrinsically more worthy, challenging, or competitive is not self-evident and may be an expression of the existing prestige hierarchy in science rather than of empirically defensible claims. We see no evidence that rising to the top of some fields is easier than others, or that physical or mathematical knowledge is intrinsically or on average more valuable than biological, biomedical, technological, psychological, or societal knowledge. Similarly, there is no evidence that c is a less valid measure of research distinction across the hierarchy of sciences: it was associated with probability of AAS fellowship in each NAS field in our study and has been found to be associated with winning a Nobel Prize across multiple fields better than alternative metrics (Ioannidis et al., 2016; Kosmulski, 2020). On the other hand, achieving distinction in some fields may be associated with greater opportunities to have a cultural or practical impact that might legitimately count towards fellowship of a science academy. Research breakthroughs in some fields may be more likely to generate major new technologies, products, or treatments with major societal impact, and that impact might reasonably boost overall career impact or contribution. Similarly, research accomplishment in some fields may be a better springboard to senior scientific leadership positions that generate impact and reputation than accomplishment in others. If research and researchers in the physical and mathematical sciences have these advantages, then their greater likelihood of being AAS fellows relative to equally distinguished researchers from other fields may be at least partly explainable.

The alternative defence of apparent inequities in election of fellows across scientific fields is the argument that favouring some fields over others is appropriate and justified. It could be argued, for example, that the AAS is primarily dedicated to the advancement of natural science and should therefore prioritise natural science fields in its fellowship. A difficulty of this view is that the academy represents itself as a voice for science as a whole, and regularly elects fellows from outside the natural sciences. A related argument is that, because the AAS operates in an ecology of learned academies that includes bodies dedicated to clinical and health, technological and engineering, and social sciences, it is legitimate for it to leave the recognition of researchers from outside the core natural sciences to other academies. A counterargument is that the high rate of overlap between the fellowships of the AAS and the health- and technology-focused academies, noted in the Introduction, reveals no general reluctance for AAS to recognize scientists working in fields represented by other academies. The more plausible interpretation, given the lack of fellowship overlap with the social sciences academy and the presence of only three representatives of the behavioral and social sciences in the AAS fellowship, is that there is either a specific reluctance to include these sciences within the AAS, or an election process that over the years has had an exclusionary effect.

Even if it were conceded that apparent field-based inequities in election to fellowship of learned academies are not, in fact, inequities when other factors are considered, or that they represent justifiable bias, it could still be argued that the magnitude of the inequities is excessive. Unequal recognition of otherwise equally distinguished scientists might be accepted in principle while querying their size. For researchers in behavioral and social sciences to have 94% lower odds of AAS fellowship than their peers in physical and mathematical sciences, holding c* constant, suggests that the magnitude of the potential inequity is large. If learned academies wish to reduce these large apparent inequities, they might be advised to consult bibliometric indices such as c and comprehensive databases such as the Scopus database. These tools might help to level the playing field by identifying impactful scientists who would otherwise be overlooked in the election process.

Although the focus of the present study was on the hierarchy of the sciences, its findings suggest that differences in the representation of scientific fields in the AAS fellowship are not fully explained by this hierarchy. The hierarchy primarily represents differences among basic research fields, but our analyses revealed that applied research fields are disadvantaged in entry to the AAS fellowship in similar ways to social science fields. The odds reduction for researchers in engineering and applied sciences relative to mathematical and physical scientists was 92%, for example. Previous research (e.g., Biglan, 1973) indicates that a pure-applied dimension captures differences among scientific fields independently of the natural-social dimension. Future research might examine how these dimensions influence differential reward and recognition of researchers from different fields.

The present research addresses a single learned academy in one country. Its methodology might be applied to other academies—and other forms of academic recognition and reward—to evaluate whether comparable effects occur, such as recognition disadvantages for ‘softer’ or more applied fields. It might be profitable to conduct cross-national comparisons in this context. For example, whereas the AAS fellowship contains two researchers (0.5%) in the “psychology & cognitive sciences” field, the NAS fellowship directory lists 87 researchers (3.0%) with primary membership in its “psychological and cognitive sciences” section, and formally recognizes “behavioral and social science” as one of its six component discipline classes. Over (1981) criticized the NAS for its hierarchy of science-resonant favouring of experimental psychologists over their equally accomplished clinical, developmental, and social psychologist peers. He wrote that this factor, alongside the domination of NAS by scientists trained at elite institutions, “suggest that in the past, selection to the academy may have been governed by criteria other than or additional to outstanding research achievement” (p. 745). We encourage bibliometric researchers to pursue these factors and suggest that the hierarchy of sciences may be a fruitful concept in that pursuit.