On the relationship between citations of publication output and Hirsch index h of authors: conceptualization of tapered Hirsch index hT, circular citation area radius R and citation acceleration a
- First Online:
- Cite this article as:
- Sangwal, K. Scientometrics (2012) 93: 987. doi:10.1007/s11192-012-0805-7
The nature of the empirical proportionality constant A in the relation L = Ah2 between total number of citations L of the publication output of an author and his/her Hirsch index h is analyzed using data of the publication output and citations for six scientists elected to the membership of the Royal Society in 2006 and 199 professors working in different institutions in Poland. The main problem with the h index of different authors calculated by using the above relation is that it underestimates the ranking of scientists publishing papers receiving very high citations and results in high values of A. It was found that the value of the Hirsch constant A for different scientists is associated with the discreteness of h and is related to the tapered Hirsch index hT by A1/2 ≈ 1.21hT. To overcome the drawback of a wide range of A associated with the discreteness of h for different authors, a simple index, the radius R of circular citation area, defined as R = (L/π)1/2 ≈ h, is suggested. This circular citation area radius R is easy to calculate and improves the ranking of scientists publishing high-impact papers. Finally, after introducing the concept of citation acceleration a = L/t2 = π(R/t)2 (t is publication duration of a scientist), some general features of citations of publication output of Polish professors are described in terms of their citability. Analysis of the data of Polish professors in terms of citation acceleration a shows that: (1) the citability of the papers of a majority of physics and chemistry professors is much higher than that of technical sciences professors, and (2) increasing fraction of conference papers as well as non-English papers and engagement in administrative functions of professors result in decreasing citability of their overall publication output.
KeywordsCitations LCitation acceleration aCircular citation area radius RFerrers diagramsHirsch constant A = L/h2Hirsch index hPublication outputTapered h index (hT)
For over two decades there has been an increasing interest in the evaluation of the scientific research output of scientists in terms of numerical indexes quantifying it unequivocally. During the last 6 years, the introduction of h index by Hirsch (2005) has provided enormous impetus in finding tools to quantify the research output of individual scientists, university faculties and research institutions. The Hirsch index h is defined as the highest number of papers of an author that received h or more citations. The main drawback of the h index of an author is that it does not count citations received by h papers with citations ln > h and (n − h) papers having citations ln < h. Here ln denotes the number of citations of an individual paper of rank n such that the total number of citations L is the sum of citations coming from individual papers. In order to improve and modify the h index several contributions have been devoted (for example, see: Alonso et al. 2009; Burrell 2009; Egghe 2010; Franceschini and Maisano 2010a, b; Glänzel and Schubert 2010; Jin et al. 2007; Kosmulski 2006; Navon 2009). Among the different h variants, the tapered h index (hT), proposed by Anderson et al. (2008), is one which takes into account the total number of citations.
The aim of the present paper is fourfold: (1) to analyze the nature of the Hirsch constant A of the publication output of different professors working in different institutions in Poland and their Hirsch index h, (2) to propose a simple non-integer index, the radius R of circular citation area, defined as R = (L/π)1/2 ≈ h, which is easy to calculate and improves the ranking of scientists publishing high-impact papers, (3) to describe some general features of citations of publication output of Polish scientists in terms of their citability from the concept of citation acceleration a = L/t2 = π(R/t)2 (t is publication duration of a scientist), and (4) to discuss the influence of conference papers, non-English papers and administrative functions of professors on the citability of their overall publication output.
Selection of authors and their citation data for analysis
Polish Institutions considered in analysis
University of Warsaw (Uniwersytet Warszawski)
University of Wrocław (Uniwersytet Wrocławski)
University of Silesia, Katowice (Uniwersytet Śląski, Katowice)
Maria Curie-Skłodowska Univerisity (Uniwersytet Marii Curie-Skłodowska, Lublin)
University of Opole (Uniwersytet Opolski)
University of Kazimierz Wielki, Bydgoszczy (Uniwersytet im. Kazimierza Wielkiego, Bydgoszcz)
Warsaw University of Technology (Politechnika Warszawska)
Ph, Ch, T
Łódź University of Technology (Politechnika Łódzka)
Gdańsk University of Technology (Politechnika Gdańska)
Ph, Ch, T
Military University of Technology, Warsaw (WAT im. Jarosława Dąbrowskiego, Warsaw)
Ph, Ch, T
Czestochowa University of Technology (Politechnika Częstochowska)
Ph, Ch, T
UTP of J.J. Sniadeckis, Bydgoszczy (UTP im. J. i J. Śniadeckich w Bydgoszczy)
Ph, Ch, T
Koszalin University of Technology (Politechnika Koszalińska)
Ph, Ch, T
Technical-Humanities Academy, Bielsko-Biała (Akademia Techniczno-Humanistyczna, Bielsko-Biała)
Ph, Ch, T
Institute of Low Temperature and Structural Research PASc (Instytut Niskich Temperatur I Badań Strukturalnych PAN), Wrocław
Institute of Physical Chemistry PASc (Instytut Chemii Fizycznej PAN), Warsaw
Institute of Metallurgy and Materials Science PASc (Instytut Metalurgii i Inżynierii Materiałowej PAN), Cracow (Kraków)
The basic bibliometric data, including the Hirsch index h and the tapered Hirsch index hT, for the Royal Society scientists are taken from Anderson et al. (2008). The data for Polish professors were collected by the present author and cover the period up to 2010. These basic data include the total number of papers N, research career length t, the number of citations L with self-citations and the Hirsch index h, but in the case of Polish professors the number of papers published in conferences/meetings and the number of papers published in limited-access journals such as those published in Polish, Russian or other regional languages are also given. The data for Lublin University of Technology (LUT) and non-Lublin University of Technology (non-LUT) professors, respectively, were collected in December 2010 whereas those for professors working in six traditional universities, eight technical universities and three research institutes of Polish Academy of Sciences (PASc), respectively, were collected during 10–15 March 2012 from Thomson Reuters’ ISI Web of Knowledge (Web of Science).
Anderson et al. (2008) chose the six Royal Society scientists from a consideration of their eminence in their scientific fields, whereas the present author selected the LUT and non-LUT professors arbitrarily taking into consideration solely their scientific, administrative or organizational activities. Among the non-LUT professors, four of them work in PASc, and two in Warsaw University of Technology (WUT). Three of the PASc professors also have university affiliations. Two of the nine professors selected from LUT have served before, and one is serving now, as prorectors for scientific affairs, three of them are deans of different faculties, while six have not been involved much in administrative work. One of the non-LUT professors has served before as a rector.
In contrast to the above arbitrarily chosen LUT and non-LUT professors, the selection of other Polish professors was made from the staff members of faculties/Departments of Physics, Chemistry and different technical sciences chosen from lists of ranking of traditional and technical universities for the year 2010, prepared jointly by Polish educational monthly “Perspektywy (Perspectives)” and daily “Rzeczpospolita (Republic)”. The universities were selected using the criterion: y = 3x − 2, where x is an integer ≥ 1. In the case of traditional universities, typically humanities universities were omitted from the selection. The three PASc institutes were selected according to their scientific fields (i.e., physics, chemistry, and metallurgy and materials engineering) from the list of PASc institutes prepared by the monthly “Forum Akademickie (Academic Forum)”.
Depending on whether the list of the members of the staff, available from the home page of a university or PASc institute, was prepared alphabetically or according to their academic positions, the professors, designated as y, were selected in two ways; (1) from the lists of the members of the staff of faculties/departments of selected universities and PASc institutes, using the criterion: y = 4x − 3 (where the integer x ≥ 1), and (2) in the case of small faculties, institutes and departments, their deans, directors and heads. In the former case, when the bibliometric data of a drawn professor could not be found in the Thomson Reuters’ database, his/her next neighbor was selected. In the case of UW, the selection of professors was mainly restricted to experimental solid state physics and physical chemistry. The publication output of professors active in agrophysics, biophysics and astrophysics was not taken into consideration due to their narrow domains. No special attention was made to trace the past or present administrative functions of all professors.
Tapered index hT and its relationship with Hirsch index h
Ferrers diagrams and the tapered hT index
As mentioned before, the main drawback of the h index is that it neglects all citations which fall outside the Durfee square (i.e., the h-core). These neglected citations are: (1) (ln > h) citations, lying on the right of the Durfee square, received by (n < h) papers and (2) (ln < h) citations, lying below the Durfee square, received by (n > h) papers. The citations falling in the two areas are also called areas of “big hits” and “sleeping beauties” and are designated here as BH and SB areas, respectively. Anderson et al. (2008) proposed a new version of the h index, called the tapered h index (hT) that accounts for citations lying outside the h-core. These authors also suggested a scoring mechanism to calculate hT on equitable basis to that of h, thereby allowing direct comparison of the two measures of publication output.
Franceschini and Maisano (2010b) pointed out that the main drawbacks of the hT index are: (1) its arbitrariness of the scoring system as the basis of its construction, which leads to its illusory discriminating power with respect to h, and (2) hT contains no information on the shape of the corresponding Ferrers diagram. These authors proposed citation triad based on subdivision of the total number of citations L into three contributions: (1) H citations in the Durfee square, (2) BH citations to the right of the Durfee square (area BH) and (3) SB citations below the Durfee square (area SB). However, in view of arbitrariness in the scoring system for the citations of areas BH and SB, the discriminatory capability of these additional measures BH and SB in the citation triad can also be questioned. Consequently, as in the original version of the hT index of Anderson et al. (2008), in this paper we shall follow the equitable system of scoring for citations in both BH and SB areas.
Figure 1 illustrates examples of the Ferrers diagrams of the citations of four fictitious scientists A, B, C and D, who published five papers each which earned them a total of 19 citations. Therefore, as far as the numbers of citations and papers of the four scientists are concerned, they are equally good. However, the citations of their papers are different, and, in the order of decreasing citations of the papers, are: 5, 5, 4, 4, 1 for scientist A; 7, 5, 4, 2, 1 for scientist B; 10, 4, 3, 1, 1 for scientist C; and 10, 4, 2, 2, 1 for scientist D. It may be seen that scientist A has h = 4, B and C have h = 3, whereas D has h = 2. Obviously, if h index is taken as a measure of the impact of papers of the above scientists, scientist A published the most influential papers, D the least influential ones, but B and C published papers which had similar impact and this impact was in between the impact of the papers of A and D. This is due to the fact that the citations which fall outside the Durfee square (i.e., the h-core) do not contribute to the h index.
It may be noted from Fig. 1 that increase in h − 1 index to h requires addition of (2h − 1) citations. For example, an increase in h index from one to two requires three citations whereas that of h from two to three requires five citations. Thus, the contribution of each citation in increasing h − 1 to h is 1/(2h − 1). Using this concept, Anderson et al. (2008) calculated the contributions of all citations in area BH (coming from papers with ln > h for papers n < h) and area SB (originating from papers with ln < h for papers n > h) to the hT index. Obviously, hT = h when there are no citations to the right of the Durfee square and below the Durfee square. However, hT > h when there are citations in these areas.
Relationship between h and hT and the value of constant A
From Fig. 1 one finds that the value of hT is 4.33, 4.215, 3.997 and 3.940 whereas that of h is 4, 3, 3 and 2 for scientists A, B, C and D, respectively. The corresponding values of the ratio hT/h are 1.083, 1.405, 1.332 and 1.970 such that the average hT/h = 1.448 ± 0.375, whereas those of the ratio L/h2 = A are 1.188, 4.359, 3.539 and 4.75 for these scientists (average L/h2 = 3.459 ± 1.596). Obviously, both hT/h and A have different values for different scientists and decrease in the sequence: A, C, B and D. We consider some more examples below. First we examine the data on hT, h and L reported by Anderson et al. (2008) and then those of LUT and non-LUT professors, respectively. The values of hT in the latter case were calculated by the author following the procedure described above.
It was found that the values of the ratio hT/h for scientists of the Royal Society lies between 1.656 and 1.948. In view of relatively small differences in the value of hT/h for different authors, the values of h and hT of the chosen scientists are related such that hT/h = 1.719 ± 0.115. In contrast to this, the ratio hT/h lies between 1.32 and 2.06 for LUT professors and between 1.59 and 1.99 for non-LUT professors, whereas the corresponding mean values of h/hT are 1.698 ± 0.241 and 1.796 ± 0.152, respectively.
From the Ferrers diagrams presented in Fig. 1 it may be concluded that hT/h ratio increases with increasing A. Since determination of hT is based on the total number of citations L, a similar trend in hT/h and A is indeed expected. However, in order to compare the value of the ratio hT/h for an author with his/her total number of citations L, one should take into account the ratio L1/2/h = A1/2 instead of A. Then, as discussed below, when various papers of an author receive citations “appropriately”, one anticipates the ratio hT/h comparable with that of A1/2 (i.e., hT/h = A1/2).
Best-fit values of constants of relation (2)
1.4459 ± 0.1534
1.9713 ± 0.0547
0.5318 ± 0.0692
0.9990 ± 0.0245
0.1506 ± 0.0772
0.9856 ± 0.0275
1.2247 ± 0.1234
2.0081 ± 0.0502
0.0400 ± 0.0617
1.0041 ± 0.0251
It should be recalled here that addition of more citations from area BH or SB outside the Durfee square to the h index increases the value of hT. In other words, the tapered Hirsch index hT does not distinguish between contributions from citations lying in areas BH and SB outside the Durfee square. This implies that the value of the hT index with respect to the value of the original Hirsch index h is enhanced by a factor related to the constant A1/2. However, in contrast to the h index which defines the largest filled-in square of points in the Ferrers diagram, both hT index and constant A1/2 provide no information on the shape of the corresponding Ferrers diagram (cf. Franceschini and Maisano 2010b). Consequently, the situation A1/2 > hT/h results when contributions to the hT index of an author come from citations lying in area BH or SB outside the Durfee square.
The examples considered above corroborate the observation in the Introduction that the value of the Hirsch constant A spans over a wide range. However, the situations with A = 1 and A > 1 occur when there are no citations and when there are citations, respectively, outside the Durfee square. The different values of A are due to different arrangements of citations outside the Durfee square.
Concept of circular citation area and its radius R
Relation (1) between the total number of citations L of the papers of a scientist and his/her Hirsch index h implies that the citations form a close-packed arrangement of points or squares in two-dimensional citation space, as represented by Ferrers diagrams. The Durfee square or the h-core represents the area of the largest filled-in square of side h. The tapered index hT is also based on the construction of Ferrers diagrams, in which contributions of citations outside the Durfee square to the total area are counted. However, if the contributions of citations outside the Durfee square are arranged as contributions around it to form a new square of sides d = L1/2, d = A1/2h. Then h, hT and d, all the three, are manifestation of two-dimensional square space, but the latter two measures include contributions of citations outside the Durfee square.
The approach of quantifying the scientific output of a scientist in terms of h index has been well recognized since it was proposed by Hirsch (2005). However, the main difficulty associated with it is that it is confined to the Durfee square in two-dimensional square space. Consequently, there are situations when an author has a low h index despite having published during his/her publication career a few outstanding paper(s) which received very high citations. In the case of Royal Society scientists (see Anderson et al. 2008), a typical example is ADB, who published 55 papers only which received L = 40,094 citations, but has a lower h = 28 in contrast to h = 31 of MREP, who published 89 papers which received L = 2,356 citations. In this specific example, the value of Hirsch constant A for ADB is more than nine times higher than that of MREP. Obviously, in this case, ADB has been penalized by the h factor despite the fact that his papers received 17 times more citations than MREP’s papers.
The above highly different values of A for citations of the publication output of different authors are a consequence of the definition of h. For example, in the case of the four virtual scientists of Fig. 1 with L = 19, the Hirsch constant A changes by a factor of 3.9 from 1.188 for scientist A to 4.359 for scientist B, but by a factor of 3.17 from 1.118 for scientist A to 3.539 for scientist C. Similarly, for L = 500 citations for a moderately active scientist with h = 12, A = 3.472, but different arrangements of citations of his/her papers in the Ferrers diagrams resulting in h = 11 and 13, L = 500 citations would give A = 4.132 and 2.959, respectively.
Note that both hT and R include all of the citations comprising citation areas H, BH and SB of an author, but hT is also not an effective measure of the impact of his/her scientific output. Among the Royal Society authors for example (see Anderson et al. 2008), the hT index of ADB (hT = 68.18) is lower than that of RJJ (hT = 72.03). This ranking is the same as that predicted by the h indexes of these two authors, although the papers of ADB received citations four times higher than those of RJJ. However, when R is taken as a measure of the impact, ADB is the unquestionable leader in the ranking of Royal Society scientists. In this sense, R is superior to hT. Moreover, R can be calculated immediately without constructing Ferrers diagram for the publication output of an author to calculate his/her hT index.
We note that the same values of circular citation radius R and Hirsch index h for an author are expected when (A/π)1/2 = 1 (i.e., A = π = 3.142). This situation occurs in the case of citation data for a majority of scientists considered above. All deviations from the equality R = h result when (A/π)1/2 ≠ 1. When A1/2 > 1.772, R > h, whereas when A1/2 < 1.772, R < h. The citation data of real scientists and several fictitious scientists considered above can be explained in this way.
Some general features of citations of publication output of Polish scientists
Concepts of citation acceleration a and R-rate
It is well known that the cumulative citations L(t) and the Hirsch index h(t) of an author increase with his/her publication career t, the number N of published papers and their citability. Analysis of citation data of different authors using Hirsch relation (1), written in the form of Eq. (2), does not provide any useful information on the effects of the above factors. In order to account for the effects of such factors as publication time t, publication rate ΔN and scientific field, here we introduce and use concepts of citation acceleration a and circular citation radius rate (i.e., R-rate). For this purpose we follow the ideas of stochastic model of Burrell (2007a, b).
It should be mentioned here that, recently, the present author (Sangwal 2012) used the concept of citation acceleration a to compare the publication output of different authors and interpret the meaning of an age-independent index proposed previously by Burrell (2007c), Kosmulski (2009) and Abt (2011) on the basis of average value of h index over time t. For the purpose of comparison, the author used the a1/2 parameter which is essentially related to the circular citation radius R.
From the above figures it may be seen that the citability of the papers, as determined by the slope ΔL/2 of the plots of L/t2 against ΔN for various authors differs enormously with the upper limit of about 1.5 for physics and chemistry professors (see Figs. 6c and 7a), and about 0.5 for technical sciences professors (see Fig. 7c). However, the citability of a majority of the physics and chemistry professors lies between 0.2 and 0.5 (Figs. 6a–c, 7a, b and 8), whereas that of technical sciences professors is below 0.25 (Fig. 7c). These observations are consistent with the range of relatively high h values usually lying between 5 and 30 for physics and chemistry professors and relatively low h values lying frequently between 1 and 20 for technical sciences professors. Among the technical sciences professors, professors working in universities with low rankings in the hierarchy and specializing in electrical and high-power engineering have very low citability ΔL/2 and publication rate ΔN (see Fig. 7c).
The values of Hirsch index h and circular citation area radius R of professors working in traditional and technical universities are comparable with those of professors working in research institutes of PASc. This feature is evident from the plots of Figs. 6, 7, 8, which reveal that physics and chemistry professors working in universities have comparable publication rate ΔN and citability ΔL/2 with those of their colleagues working in PASc research institutes, where they do not have teaching load. This observation is in conflict with the general belief that teaching load reduces the publication output. One of the possible explanations of this apparently contradictory observation lies in the fact that students contribute to research activities of professors in the universities whereas professors working in research institutes are devoid of such a contribution.
Physics and chemistry professors have a much higher h and R than their counterparts specialized in technical sciences. This inference also follows from the plots of Figs. 6, 7, 8, which show that the former professors have, in general, a higher publication rate ΔN and citability ΔL/2 than those of the latter professors. It was also found that, among the physics and chemistry professors, professors active in the subfields of nuclear physics, crystal chemistry or structural crystallography, and solution, environmental or food chemistry have relatively higher h, R and citability ΔL/2 than their colleagues, whereas, among the technical sciences professors, professors active in the subfields of electrical energy and high power engineering have very low h, R and citability ΔL/2 than their other colleagues. These observations are associated with the specifics of the trends of citations in these disciplines and subdisciplines.
Influence of some other factors on citation acceleration a
Participation in conferences devoted to specific research areas is a part of the research activities of practically all scientists whereas, apart from involvement in research activities, bearing the administrative responsibility for some time by elected or appointed professors/scientists as rectors, deans and directors is a part of the functioning of departments, faculties and universities. In the former case, papers based on presentations in the conferences are usually published in special issues of journals and are included as the research output of an author. There are also researchers who, for different reasons, publish in non-English language journals which are not accessible to the common reader. Many of such limited-access journals are included in the databases. For example, among the Polish journals, Przegląd Elektrotechniczy (Electrical Engineering Review), Polimery (Polymers) and Rynek Energii (Energy Market) are typical examples of such journals which are included in the database available from Thomson Reuters’ ISI Web of knowledge (Web of Science) used in the present study. Therefore, it is of general interest to analyze the influence of conference papers, non-English papers and administrative functions of professors on the citability of their overall publication output. The concept of citation acceleration a = L/t2 (and R-rate equal to (a/π)1/2; cf. Eq. (7)) is easy to understand and interpret scientometrically.
The role of above parameters in the publication output of the authors of different specializations can be analyzed from the plots of the citation acceleration a = L/t2 as functions of fraction fN of conference papers (defined as the number Nconf of papers published in special issues of journals divided by the total number N of papers), fraction fNE of papers in limited-access journals (defined as the number NNE of papers published in limited-access journals divided by the total number N of papers), and publication rate ΔN of scientists involved in administrative work. These dependencies are presented below.
Increasing fraction of papers published by different authors in proceedings of conferences published as special issues of journals and in limited-access journals results in a lower citability of the papers (see Figs. 9 and 10).
Professors who have/had been engaged in administrative functions such as rectors, prorectors and deans, usually have h and R and relatively low values of publication rate ΔN and citability ΔL/2. This observation may be attributed to the fact that these professors devote a part of their time meant for research work to administration.
Summary and conclusions
The nature of the Hirsch constant A in the classical relation L = Ah2 between total number of citations L of the publication output of an author and his/her Hirsch index h, advanced by Hirsch (2005), is analyzed using data of the publication output and citations for six scientists elected to the membership of the Royal Society in 2006 and 199 selected physics, chemistry and technical sciences professors working in selected traditional universities, technical universities and PASc research institutes in Poland. It was found that different values of the Hirsch constant A for different scientists are associated with the discreteness of h and are related to the tapered Hirsch index hT by A1/2 ≈ 1.21hT. In order to overcome the drawback of a wide range of A associated with the discreteness of h for different authors, a simple index, the radius R of circular citation area, defined as R = (L/π)1/2 is proposed. It turns out that R ≈ h.
It is well known that the cumulative citations L(t) and the Hirsch index h(t) of an author increases with his/her publication career t, the number N of published papers and their citability. In order to account for the effects of such factors as publication time t, publication rate ΔN and scientific field, following the ideas of stochastic model of Burrell (2007a, b), the concept of citation acceleration a = L/t2, related to the circular citation radius R by Eq. (7), is introduced and used to analyze the citation data of Polish professors. It was found that the citability, given by ΔL/2, lies between 0.2 and 0.5 and the publication rate ΔN < 4 for a majority of physics and chemistry professors (Figs. 6a–c, 7a, b and 8), whereas the citability ΔL/2 < 0.25 and ΔN < 2 for technical sciences professors (Fig. 7c). These observations are consistent with the range of relatively high values of h and circular citation radius R usually lying between 5 and 30 for physics and chemistry professors (Figs. 2, 3, 4a, b and 5) and relatively low values of h and circular citation radius R lying frequently between 1 and 20 for technical sciences professors (Figs. 3 and 4c). Among the technical sciences professors, professors working in the universities with low rankings in the hierarchy and specializing in electrical and high-power engineering have very low citability ΔL/2 and publication rate ΔN (see Fig. 7c. However, physics and chemistry professors working in universities have comparable publication rate ΔN and citability ΔL/2 with those of their colleagues working in PASc research institutes (see Figs. 6a–c, 7a, b and 8).
Analysis of the influence of conference papers, non-English papers and administrative functions of professors on the citability of their overall publication output revealed that increasing fraction of papers published by different authors in limited-access journals and in proceedings of conferences published as special issues of journals results in a decreasing citability of the papers (see Figs. 9 and 10). Similarly, professors who have/had been engaged in administrative functions such as rectors, prorectors and deans, usually have low h and R and relatively low values of publication rate ΔN and citability ΔL/2.
The author is indebted to the anonymous referees for their critical comments and constructive suggestions. In fact, the manuscript would never have taken the present form without the advice of one of the referees. He also expresses his gratitude to Dr. Kazimierz Wójcik for his technical assistance and to Dr. Jarosław Borc for his assistance with the calculations of the values of tapered index.
This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.