New fractional classifications of papers based on two generations of references and on the ASJC Scopus scheme

This paper presents and evaluates a set of methods to classify individual Scopus publications using their references back to the second generation, where each publication can be assigned fractionally into up to five ASJC (All Science Journal Classifications) categories, excluding the Multidisciplinary area and the miscellaneous categories. Based on proposals by Glanzel et al. (1999a, 1999b, 2021), some additional parameters are established that allow different results to be obtained depending on how category membership is weighted or how the acceptance thresholds for multiple assignments are established. Various classifications are obtained, and then compared with each other, with the original ASJC Scopus journal classification, and with the AAC (Authors Assignation Collection) classification of a previous study (Alvarez-Llorente et al., 2023) in which the papers corresponding authors assign them the most appropriate categories. Classifications in which a high threshold is set for allowing assignments to multiple categories, combined with the use of first- and second-generation references and averaging over the number of references, provide the most promising results, improving over other reference-based reclassification proposals in terms of granularity, and over the Scopus classification itself in such aspects as the homogeneity of the publications assigned to a category. They also show greater coincidence with the AAC classification.


Introduction
The assignment of papers to scientific disciplines is very important in scientometrics, not only for the disaggregation of scientific output by categories but also for the normalization of citation indicators (Althouse et al., 2009;Lancho-Barrantes et al., 2010), and in interdisciplinary studies, inter alia.Some time ago now, Glänzel et al. (1999aGlänzel et al. ( , 1999b) ) proposed the use of papers' references in assigning them to categories, the proposal being for papers published in journals classified within the WoS's Multidisciplinary category.More recently, Glänzel et al. (2021) set out an improvement on that method with the use of a parametric model taking multiple generations of references into account.Janssens et al. (2008Janssens et al. ( , 2009) ) and Boyack et al. (2013) propose hybrid systems supported by text mining in addition to citation links.One can also find studies such as that of Glenisson et al. (2005) who combine citations and text mining, although in this case applying the latter to the titles of the references instead of exploring the network they map out.A contrary perspective had been taken in Janssens et al. (2006) who proposed classifications based on full text mining, but concluded that a combination with traditional citation-based bibliometric methods may well be appropriate.
Neural network algorithms, which since their inception have shown a great capacity to detect the key characteristics in document organization systems (Guerrero-Bote et al., 2002), have also been used as support for classification systems based on texts of titles and/or abstracts, whether the learning is supervised (Eykens et al., 2019) or unsupervised (Kandimalla et al., 2021).As affirmed in Kandimalla et al. (2021), most unsupervised systems base their self-learning on citation networks.Milojević (2020) uses the sum of the categories of the cited journals to assign a single category to all WoS items (those that have some classifiable reference), and add an iterative pass to increase the number of classified elements.The validations/evaluations of the classifications obtained are quite rudimentary, involving a comparison with the initial classification and a manual evaluation of about two hundred items.In some cases it was concluded that, although the classification obtained differs from the original, both are correct.Waltman et al. (2020) argue that any classification has legitimacy, depending on its intended use, although sometimes it is necessary to establish absolute measures of precision.To this end, they propose that, when comparing two different classifications, a third should be used which is based on a method as different as possible from that of the other two.Šubelj et al. (2016) analyse multiple automatic classification systems, and value as desirable characteristics the uniformity of the sizes of the thematic areas, that there are no areas which are too small, and other details of automatic classifications such as computing time or stability of the results.
Although with application to a very specific area, Zhang et al. (2022) compare various classifications applied to papers from the multidisciplinary journal Nature, including one based on the authors' labeling of topics.They find major discrepancies between the automated reference-or content-based classifications and those based on human labeling.They also stress the importance of the assignments to multiple categories.
The WoS assigns categories to scientific journals, with in total about 250 categories.More than one category may be assigned to a given journal and therefore to all of its papers, and there is a Multidisciplinary category.The average number of categories assigned per journal is 1.6 (Wang & Waltman, 2016).Some scientometric studies have grouped these scientific categories into other broader areas, thus forming a hierarchical classification.
In Scopus, an average of 2.1 categories are assigned to each journal (Wang & Waltman, 2016), and the average per paper across the journals rises to 2.50, far more than that of the WoS.There has been a notable increase in recent years (see Fig. 1), indicative of growing interest in classification systems with multiple categories.

Fig 1 Average number of categories per paper in Scopus by year
In this work, we shall be studying the application of a method to fractionally assign nonmultidisciplinary ASJC categories to all the citable papers published in Scopus in 2020, based on the category assignment system that uses the multi-generation parametric method proposed by Glänzel et al. (2021).We shall obtain different classifications by varying some configuration parameters, and analyse the characteristics of each to select those that provide the most homogeneous categories.This selection we shall then compare with the following other two classifications: a) The "fractional assignment" of the classification of Scopus journals in 2020, in which the Multidisciplinary area and the miscellaneous categories have been eliminated and relocated (Álvarez-Llorente et al., 2023).
b) The "AAC" (Author's Assignation Collection) classification formed by the set of assignments made by the papers' corresponding authors as research guarantors (Moya-Anegón et al., 2013), as compiled by Álvarez-Llorente et al. (2023).In this classification, the papers' own authors fractionally establish the assignments they consider most accurate using the Scopus category scheme (again excluding the Multidisciplinary area and the miscellaneous categories) which covers a subset of the Scopus publications of 2020.
The main purpose of limiting the data sample to the citable papers published in Scopus in 2020 is to allow these comparisons, but it also has the advantage of facilitating the calculations.We consider also that this is a data set that not only quite faithfully represents current citation trends but is also already reasonably well established.
In this comparison we wish to respond to the following research questions: a. Which methods coincide most closely with the AAC? b.Which methods obtain categories with papers that are most homogeneous in terms of citation habits?c.What kinds of papers cannot be classified using these methods, and how many of them are there?d.What granularity is obtained with these methods?e.How large are the categories obtained?f.How are papers classified which are published in journals assigned to the Multidisciplinary category?g.Are papers published in miscellaneous categories classified into categories of the same scientific area?h.Are classifications obtained that have characteristics more desirable for Scientometrics than those based on ASJC journal assignments?
In the following Method and Data section, we shall explain in detail the proposed classification algorithm with all its parametric variations as well as the initial treatment of the sample data, and identify the works that cannot be classified.
In the Results section, we shall respond to the rest of the research questions by analyzing the classifications obtained with the different parameter combinations.Firstly, the number, size, and granularity of the categories obtained will be examined, as well as the distribution of references among the different categories.This will provide us with a first approximation to the combinations of parameters that provide the scientometrically most desirable results.In a second step, we shall compare the different classifications with the AAC given by the corresponding authors.This will complement the information obtained to allow us to assess whether the proposed classifications seem appropriate and with which parameter combinations they are most accurate so as thus to select those best for further in depth analysis.Then with just these selected classifications we shall analyse how the publications are distributed among the ASJC's thematic areas and how they classify the publications that initially belonged to the Multidisciplinary categories and the miscellaneous areas.The result of all these analyses will finally allow us to determine the most interesting combination of parameters.

Method and Data
Scopus (Hane, 2004;Pickering, 2004) is a more recent but broader database than WoS (Gómez-Crisóstomo, 2011).It classifies journals using the All Science Journal Classification (ASJC) (Elsevier, 2023).This is a hierarchical classification scheme comprising 311 specific scientific areas or categories grouped into 26 scientific areas, plus a Multidisciplinary scientific area.Furthermore, each of the 26 scientific areas includes a "miscellaneous" category, which if eliminated would leave 285 categories grouped into those 26 scientific areas.A "fractional assignment" is defined (Álvarez-Llorente et al., 2023) in which each area's miscellaneous category assignments are distributed among the rest of that area's categories with a weight divided by the number of categories, and the Multidisciplinary area assignments are distributed among all the categories with a weight of 1/285.
In a proposal to use references to assign categories to papers, Glänzel et al. (1999aGlänzel et al. ( , 1999b)), only took into account the references from journals classified into just one or two categories, and Milojević's (2020) proposal initially only took into account references from journals assigned to a single category.Glänzel et al. (2021), while not citing so directly, do indicate that they adopt a full-counting method for the assignment classification system.We shall also add fractional assignments (which henceforth we shall call weighted-counting as against full-counting), classifying each assignment by the weight given to the category in the journal, so that if a journal is assigned equally to 3 categories then 1/3 of the weight will be added to the count of each category instead of 1, without a Multidisciplinary area or miscellaneous categories.
For our study, we shall be considering the three weight-allocation models proposed by Glänzel et al. (2021) for normalizing the share of research fields, with the weight of the references being based on the generation number in accordance with three schemes -M1 in which only the first generation references are considered, M2 in which only the second generation references are considered, and M3 in which the first generation references are weighted by 0.618 and the second generation references by 0.382.
Some first generation references may be over-represented with many second generation references, and others under-represented with few second generation references, therefore altering the proportions.For this reason, we introduced a second level of normalization for the second generation by dividing them by the number of them that come from each of the first generation.In this way, the weights of all the second generation references that come from any one of the first generation will sum to unity.Nonetheless, we shall evaluate the impact of this averaging procedure by presenting the results of both applying and not applying it (of course, only in cases where second generation references are used, i.e., in the M2 and M3 schemes).
Milojević (2020) places particular interest in achieving assignments exclusively to a single category.Any tie between categories is broken by opting for the paper's original category derived from the journal.In contrast, Glänzel et al. (2021) use a procedure to assign up to three categories, as long as the second has at least a threshold of two-thirds of the normalized share of the first category and the third has at least two-thirds of the second.We shall be using this system but including up to 5 categories and, instead of always using two-thirds, we shall be testing three different thresholds -1/2, 2/3, and 4/5.
In sum, we shall be generating a large set of reference-based classifications using the three schemes (M1, M2, M3), with each scheme being combined using the full-counting and the weighted-counting methods to find the share of references, and in turn combined with averaged and non-averaged counting (except for M1 in which second generation references are not counted).The three thresholds will be applied for each of these combinations.
Note that, although the starting point is the fractional ASJC classification (by journals), the proposed classification in whichever of its variants is at the article level since each publication will be classified into one of the ASJC categories (other than Multidisciplinary or miscellaneous) based solely on the references it contains, not on the journal to which it belongs.
None of the reference counting based methods discussed above (Glänzel et al., 1999a and1999b;Milojević, 2020;Glänzel et al., 2021) allude to any minimum number of active references (i.e., references to which categories can be assigned) for a paper's assignment to be taken as valid.It can therefore be understood that a single active reference is sufficient for a classification to be made.
Table 1 gives the percentages of the year 2020 papers indexed in Scopus with a low number of active references from the first generation, from the second generation, and the sum of the first and second generations.Note that a work may have a low number of active references in the first generation (even none) but a high number in the second, for example, if it cites books (that are not assignable to categories) in which assignable articles are referenced.And just the opposite can also be the case.We believe that an assignment cannot be considered effective if it is not based on at least three active references, so as to be capable of providing a minimum of significance.This means that we shall only be making assignments in 89.61% of the papers with first generation references, 92.53% with second generation references, and 93.10% with both the first and second generation combined.Thus, in all the classifications that we generate, the documents not assigned due to lack of references maintain their initial assignment by journal.
Table 2 presents the analysis by category, and prior to the fractioning of the Multidisciplinary category, of those documents not reassigned due to having fewer than three active references.The Weight columns indicate the weight accumulated by each category (recall that each paper is distributed proportionally among the assigned categories, so that the total of all the categories is equivalent to the number of works).The percentage columns indicate the percentage of papers assigned to the category that are discarded, and the NI columns show the average of the Normalized Impact (normalized citation, item-oriented field-normalized citation score average) (Lundberg, 2007;Waltman et al., 2011) of the said publications.The summary row, which gives the aggregated data of all categories, shows that the highest discard percentage corresponds to considering only the first generation, and the lowest to including both the first and the second generations.One also sees that they are works of low impact.
For the percentages by area, the highest are in Arts and Humanities with 55.35% of the output, followed by Social Sciences with 23.55%, and Nursing with 14.61%.However, these values fall to 38.69%, 15.52%, and 11.83%, respectively, when the two generations are combined.

Size and uniformity of the categories
Table 3 lists the results for the classifications generated by, let us recall, the combination of the three citation generation schemes (M1, first generation only; M2, second generation only; M3, first and second generations) plus the fractional ASJC classification as a fourth scheme, with the counts from full-counting and weighted-counting averaged (only M2 and M3) and not averaged (identified as AFC, AWC, FC, and WC), and with the three reference thresholds (1/2, 2/3, and 4/5).The following values are presented for each of these resulting classifications: • Categories: total number of categories with some assignment.
• Max.Cat.: maximum accumulated weight (that of the category with the greatest accumulated weight).• Min.Cat.: minimum accumulated weight (that of the category with the smallest accumulated weight).• C.V.: coefficient of variation of the weights of the different categories.
• Granularity: granularity, as defined in Waltman et al. (2020) and Milojević (2020), i.e., the total number of works divided by the sum of the squares of the works (weights) of each category.Thus, the more balanced the categories are, the greater the granularity.As was to be expected, there exists a clear correlation between the granularity and the coefficient of variation.It was calculated to be -0.9923.
As indicated by the data in Table 3, all the classifications obtained have lower granularity than those based on the ASJC, i.e., since the number of categories is similar, the categories are less equal in size.Increasing the threshold also increases the coefficient of variation and reduces the granularity.With regard to the counting methods, full-counting leads to a lower coefficient of variation and greater granularity than weighted-counting.One can also see that, among those which use the second generation, the averaged methods have a greater coefficient of variation and lower granularity, although all the granularity values lie within a quite narrow range of the same order of magnitude of 10 -5 .This is comparatively significantly greater than the value of 1.5 × 10 -6 reported by Milojević (2020) for the WoS reclassification system and that of the WoS itself which is 2.3 × 10 -6 according to that same author.
The classification is also very important for the normalization of the impact, since this is not comparable between disciplines.Lancho-Barrantes et al. ( 2010) showed that the disciplines' impact indicators correlate strongly with the average number of references to the documents in the database.One understands the practice of calculating the normalization of the impacts by dividing by the average number of citations in the discipline and year to be well-founded because one assumes that each discipline's documents are more homogeneous among themselves in terms of the citations they receive, and that their citation habits are closely related (Lancho- Barrantes et al., 2010).
Given this foundation, any new classification that we propose should not lead us to categories that are less homogeneous.To evaluate our classifications in this sense, we calculated both the coefficient of variation of the number of references per paper for each category, but weighted by the degree of membership of the paper to the category, and the average of all categories.It would be undesirable if the new classifications were to have a greater average because this would mean that the new categories include documents that are less homogeneous than the original ones.
The main indicators of normalized impact per paper take the complete citation into account, which means that the reference distribution which will correlate most closely is that of those directed to the database (Scopus in this case).Nevertheless, in Table 4 we present not only those directed to Scopus but also the coefficients of variation of the number of total references.We also include the averages of the coefficients of variation of the number of references, both total and those directed to Scopus, but limited to the 2 and 3 preceding years.One observes in the table that the coefficients of variation of the weighted-counting classifications are lower than those of the full-counting classifications.Also, increasing the threshold leads to lower coefficients of variation.The lowest values correspond to the classifications that use the second generation of references.The effect of the averaging in the second generation is barely perceptible, but in most cases the coefficient of variation tends to increase.Coincidence with the AAC Table 5 contrasts the classifications generated here with the AAC generated in our previous work (Álvarez-Llorente et al., 2023).The "%Coinc." column presents the percentages of coincidence between the weights assigned by each classification and those assigned by the authors themselves in the AAC, as was defined in the aforementioned work.
"1st AAC cat.rank" indicates the average rank of the assignment in which, in accordance with each classification generated, the winning categories of each AAC paper appear according to their authors."1st AAC cat.w/o C" is a count of how many of those winning categories according to the authors do not appear among the 5 assigned in the classification generated.I.e., for each AAC paper we take its winning categories according to the AAC itself, and then we look to see whether those categories are among the 5 possible assigned in the classification obtained.If they are, we note the rank (1 to 5) for each in order to calculate the mean in "1st AAC cat.rank".
Otherwise, we add it to the "without classification" count ("1st AAC cat.w/o C").In the two rightmost columns we record just the opposite.We take the winning categories of the classification generated and calculate the average rank in the AAC classification (column "1st class.cat.rank"), or we count it as not among the 5 possible ("1st class.cat.w/o C").A swift overview of the table shows that 0.5 is the threshold with the lowest coincidence percentages, and that the values for the other two are quite similar.Taking only the second generation of references into account yields the poorest results, and weighted-counting gives better results than full-counting.With regard to those using the second generation of references, once again there are no major differences between the averaged and the non-averaged results.
What is most striking is that those based on full-counting are below the ASJC, while those based on weighted-counting are above it.
For the "1st AAC cat.rank" and "1st AAC cat.w/o C" columns, one observes that the results are clearly related to the use of the different thresholds -higher thresholds tend to assign fewer categories, thus increasing the count of unassigned winning categories but at the same time lowering the rank of those which are assigned.In the case of the two rightmost columns ("1st class.cat.rank" and "1st class.cat.w/o C"), although there are very few variations, it is intuitive to interpret that the higher the threshold the fewer the categories assigned, and therefore the lower the average rank they are assigned and the fewer of them that remain to be assigned.It is striking that ASJC classifications with a threshold obtain a lower "1st AAC cat.rank", and that they do not have an excessive number of winners in the unassigned AAC ("1st AAC cat.w/o C").
From the opposite perspective however (the "1st class.cat.rank" and "1st class.cat.w/o C" columns), the ranks are significantly higher, and the number of unclassified assignments rises notably.
Table 6 presents the data on how the different methods classify the documents included in the AAC, and how the AAC assignment itself does sothe total number of assigned categories, average number of categories assigned per work, and the percentage of works with 1, 2, 3, 4, and 5 or more assignments (actually, the no-threshold ASJC scheme is the only one that can present more than 5 assignments).
As explained in Álvarez-Llorente et al. ( 2023), the average number of categories assigned by ASJC is very high because it repartitions the Multidisciplinary category assignments into low-weight assignments to all categories, and the miscellaneous categories into all of those in the corresponding area.Setting a threshold reduces it by half, although it is still very high.
The AAC has an average of 1.94 categories assigned to each paper.There are only 5 methods which have an average of fewer than 2 categories per paper, and they all have the threshold 0.8 and use weighted-counting.The AAC grants five categories to only 4.68% of the documents, an especially low percentage.The aforementioned 5 methods also assign fewer than 5%, although more than 64% are assigned to a single category compared with 44.85% for the AAC.For all of the above, we consider that the most promising methods are those that use the 0.8 threshold and weighted-counting since, although they have somewhat lower granularity, they have the greatest coincidence with the AAC (only very slightly surpassed by those with the 2/3 threshold) and the resulting classifications give rise to classes more homogeneous in number of references and to fewer categories assigned to each paper, characteristics that are highly desirable in scientometrics.For this reason, we shall focus on them as methods selected for the following analyses.

Distribution by category and subject area
In these analyses, we include the AAC classification in the comparisons.Hence, for the other classification methods, we shall necessarily limit their application to the publications included in the said classification (as in Tables 6 and 7).This is not a problem since, as Álvarez-Llorente et al. ( 2023) demonstrated, the sample of publications included in the AAC classification "represents with great fidelity the thematic variety by areas and categories, as well as by countries of affiliation of their authors" of Scopus publications in 2020.
Table 7 presents the correlations of the resulting sizes of each category (sum of weights) when classifying the AAC documents according to the AAC itself, the 5 selected classification systems, and the fractional AJSC.All are fairly strong correlations.The 5 selected systems have high correlations with each other, slightly lower with the fractional ASJC, and somewhat lower with the AAC, demonstrating that these classifications maintain high coherence with the original ASJC classification.To delve deeper into analysing the resulting categories in the different classifications, Fig. 2 shows, for each of the classifications, the number of categories that accumulate different weight percentage bands.For example, the band "0.2" indicates how many categories in each classification accumulated a weight percentage in the interval [0.1%, 0.2%).The conclusion that can be drawn from the figure is that the selected classifications show similar distributions among themselves but different from the ASJC and the AAC, being notably greater the number of categories that accumulate less than 0.1%, and also greater although not so markedly the number that accumulate more than 1%.There is a somewhat more even distribution among the intermediate categories.As it is difficult to compare the thematic distribution in the 285 ASJC categories, Table 8 lists the accumulation of weights for each classification, but grouped into the ASJC areas.Fig. 3 illustrates that data graphically.It can be seen that the selected methods once again have a very similar behaviour to each other, and in general also quite similar to that of the ASJC classification.The greatest differences are found in Agricultural and Biological Sciences and in Medicine where there is greater accumulation observed in the selected methods.The opposite is the case for Arts and Humanities, Chemical Engineering, Computer Science, Environmental Science, Materials Science, and Veterinary.

Relocation of multidisciplinary and miscellaneous publications
Possibly one of the most delicate aspects of any attempt at reclassification is how to treat multidisciplinary publications.In what follows therefore we shall analyse how the proposed systems behave with AAC publications that initially belonged to the ASJC Multidisciplinary area (1000) and the miscellaneous categories existing in each of the other areas.
In Table 9 and Fig. 4, one sees how the different strategies classify by area the AAC articles of journals assigned exclusively to the Multidisciplinary category.One can see that some effects commented on for the previous table are repeated, especially the strong increase for the selected weight classifications in Agricultural and Biological Sciences and in Medicine to the detriment of others such as Environmental Science.Nevertheless, in general, despite these striking variations, one sees at a glance that the weights in all strategies conserve a certain correlation, i.e., the distribution of these papers by area is fairly proportional to their size.Indeed, all the columns in Table 9 have a strong correlation with those in Table 8 (greater than 0.8, except for that of AAC which is 0.76).9 data, where the greatest differences between the AAC and the selected methods can be appreciated In a similar way to how Table 7 presents the correlations between the accumulated weights in the ASJC categories and the different classifications of the AAC publications (Table 6), Table 10 presents those same correlations, but restricted to AAC publications from exclusively the Multidisciplinary category.There is confirmation of major correlations between the proposed classifications, as well as a certain distance between all of them and the AAC.Finally, in Table 11 we analyse the behaviour in the different classifications of the AAC papers that initially belonged exclusively to the each area's miscellaneous category, indicating the number of papers that belonged to that category and the percentage that stay in that same area for each of the classification methods.In this case, no clear pattern is observed -there are large variations from one area to another, and smaller variations between the different classifications.
Most notable is the almost always greater percentage of papers that remain in the same area for the AAC.

Conclusions
We have presented a set of methods for classifying papers based on their references in which each publication can be assigned up to 5 weighted categories of the ASJC scheme without the Multidisciplinary area or the miscellaneous categories.The classification methods combine 3 different thresholds for limiting multiple assignments, 4 weighting methods for computing category membership, and 3 schemes of reference generations.To validate them, their scientometric characteristics were compared with the original classification and with the AAC classification done by the papers' authors themselves.
In these classifications, we decided to rule out the reclassification of between about 7% and 11% (depending on the generations used) of publications that did not have a minimum of 3 classifiable references since we considered that in these cases the references provide no significance better than the paper's initial journal-based classification.The allocation is not homogeneous over all thematic areas -that of Arts and Humanities accumulates between 39% and 55%, Social Sciences 16% and 24%, and Nursing 12% and 15%, while the rest accumulate an average of less than 5%.In any case, these are low-impact works, with an average NI of 0.36.
Most of the methods yielded categories that were slightly more homogeneous in citation habits than those of the ASJC, and that had a granularity comparable (slightly lower) with that of the ASJC and an order of magnitude greater than that reported by Milojević (2020) in what was also a reference-based classification but based on the WOS scheme.
Some of the classifications also exceed the ASJC assignment in coincidence with the AAC.What stand out are those that use weighted-counting to compute categories, two generations of references, and parametrized with a threshold of 0.67.Since, however, the results differ little from those with a threshold of 0.8, and with 0.8 the resulting number of categories per paper is closer to what the authors assign in the AAC, this latter threshold was finally selected for the deeper analysis.
There was a strong correlation between the sizes of these classifications and the distribution of weight by subject area with respect to the original journal-based ASJC classification, and a little less with respect to the AAC carried out by the papers' authors.
Analysing how the publications that originally belonged to the Multidisciplinary area are allocated to the different categories showed them to be distributed proportionally to the size of the areas, with the correlation being very high, but much lower with respect to the AAC.
Regarding the reassignment of the publications originally belonging to the miscellaneous categories of each main area, the different methods also behaved very similarly, although they all showed a significantly lower tendency to maintain the assignments within the same principal area than was the case in the AAC classification.
For all these reasons, the classifications parametrized with a 0.8 threshold combined with weighted-counting for the computation of categories demonstrated more desirable characteristics for scientometrics than those derived from the original ASJC journal classification.Among them, we would highlight the one that uses two generations of references and averaged computation (M3-AWC-0.8),which in our opinion, although it is not the best in all areas, shows the most balanced results on average.
However, not much agreement is achieved with the authors' assignments in the AAC.This shows that there is no excessive coherence between the references the authors use in each paper and the categories they themselves assign to it in the AAC.This is in line with the conclusions of Zhang et al. (2022) on categorization based on human labeling, and with Milojević (2020) who indicated that both the original journal-based classification and a reference-based classification are correct.
The application of this classification method has been analyzed for the whole of science, but it can also be applied to specific domains.This would require previously clearly delimiting the scope of the domains as well as having a database with sufficient density of references in those domains.
Lastly, although the study has been constrained to publications registered in Scopus in 2020 with the principal objective of being able to compare its results with the AAC classification of the corresponding authors of our previous work (Álvarez-Llorente et al., 2023), we believe that there would be no obstacle to being able to apply the classification method to any other temporal range or any other database of publications classified by journal.

Fig 2
Fig 2 Number of categories by percentage associated by the 5 classification systems with a 0.8 threshold, the AAC, and the fractional ASJC.Production with 2 or less active references is associated by means of the fractional ASJC

Table 1
Percentage of 2020 production indexed in Scopus with more than a given number of active references With more than: % 1st Ref Gen. % 2nd Ref Gen. % 1st + 2nd Ref Gen.

Table 2
Scientific production with at most two active references

Table 3
Structure of the classification obtained with various models taking into account only those papers with more than two active references in both the first and the second generations.Included for each scheme, counting method, and threshold are the number of non-empty categories, the size (cumulative weight) of the largest and the smallest category, the coefficient of variation of the size, and the granularity

Table 4
Average of the coefficients of variation of the number of references per paper in each category for each model, indicator, and threshold: ACVS (Average coefficient of variation of the number of references to the Scopus Source Set); ACVS3 (Average coefficient of variation of the number of references to the Scopus Source Set in the previous 3 years); ACVS2 (Average coefficient of variation of the number of references to the Scopus Source Set in the previous 2 years); ACV (Average coefficient of variation of the number of references); ACV3 (Average coefficient of variation of the number of references in the previous 3 years); ACV2 (Average coefficient of variation of the number of references in the previous 2 years)

Table 5
Comparison of classifications with the AAC.Shown are the percentage of coincidence, the average rank that the winning categories in the AAC obtain in the new classifications, the number of winning categories in the AAC that have no mention in the classification, the average rank that the winning categories in each classification have in the AAC, and the number of winning categories of the new classification that are uncategorized in the AAC

Table 6
Comparison of the assigned categories with the AAC collection

Table 7
Correlations of the sum of weights in the ASJC categories between the 5 classification systems with a 0.8 threshold, the AAC, and the fractional ASJC, limited to the documents included in the AAC

Table 9
Distribution of papers published in journals assigned exclusively to the 1000 Multidisciplinary category into the 26 ASJC areas by the 5 classification systems with a threshold of 0.8 and the AAC Bar chart representation of the Table

Table 10
Correlations between the distributions of papers published in journals assigned exclusively to the

Table 11
Percentage of AAC papers published in journals exclusively assigned to the miscellaneous categories that are assigned to categories belonging to the same ASJC area by the 5 classification systems with a 0.8 threshold and the AAC itself