Citations: The Rules They Didn’t Teach You

Citations, so many citations, so what is right and what is wrong? As an editor-in-chief, as a reviewer, and as an author of my own journal articles I have personally dealt with this question on multiple levels. At times, the emergence of PubMed has resulted in a citation explosion and a resulting quagmire as many authors are just not selective about referencing papers and tend to be a bit over zealous in citing more and more papers many of which are not on the topic. In such cases, I often question whether these authors have read anything more than just the titles of these papers that they cite left and right to support their own work. Last August, Dr. William J. Pearce authored an intriguing opinion piece in The Scientist entitled “Citations: Too Many, or Not Enough” (The Scientist 24(8):29). In this piece he waxed somewhat nostalgic regarding the use of Index Medicus, bringing back fond memories of how we found citations in the “old days”. Of course, of late, I have been referred to as a dinosaur. Although I am not “Singin’ the Dinosaur Blues” (a reference to a Steven Fromholz song sung by Jerry Jeff Walker), I do think it is indeed an important point to understand what are the best practices for citing the literature. Dr. Pearce also brought forth additional points about the current use of citations in the literature such as the number of citations in a manuscript, the quality of citations used, the appropriateness of these citations, and the use of reviews as a substitute for the primary literature. These are all extremely important issues that are a concern for many in the field and are addressed herein.

How do you find a citation? How do you know if a citation is appropriate? Prior to the invention of PubMed and Google as a means to search the scientific literature and back when people recognized such names as Index Medicus or Current Contents, we would start with several papers on the topic of interest. After thoroughly reading and understanding the content of these papers, the references would lead us to the next round of papers from the literature that were deemed to be pertinent. Scampering off to the library and pulling these 20–30 new papers to read we determined if these papers could be used as citations in the burgeoning manuscript that we were composing. Someone skilled in the art of citation seeking would then have another round of papers to pull as these 20–30 papers would lead to new potentially citable papers in the literature.

There are several important themes in this process that may be absent today. First, papers were read completely because that was the only way to really determine if there were important references contained in the paper being read that could be potentially useful. Second, there was a need to really understand and comprehend what was being discussed in the paper being read to make an informed decision whether it was indeed worth citing in your manuscript. Hence, decisions were made based upon a lot of time devoted to reading and understanding the literature. What concerns me is the emerging practice of authors having not really read the papers that they cite, but an even greater concern is whether authors have read anything more than the title of a paper that they are citing. Hence, these authors have not really made any decision on citing a particular paper based upon a true comprehension of that paper, but rather are citing that paper based upon a rather superficial understanding of it based solely upon the title or abstract.

In this type of authors’ defense, it is a really quick process to get the citation. But should it be? In the past, finding citations took time, but it also meant reading papers and being more judicious in selecting papers on the topic to cite, reducing the superfluous citations that are common today. As a graduate student coming of age at that time, the citations of another laboratory’s work in the literature or the repeated reference to a particular paper would bolster my confidence that this paper was truly worth reading and that it might serve as a solid citation in my own manuscript. This confidence was a key point in teaching yourself to place a value on the quality of the work presented in potential citations and to pick only those that met an ever evolving selectivity.

Does this mean that Google and PubMed should not be used to find citations? Absolutely not as these are modern and useful tools for searching through the ever growing literature. However, there is something to be said about having to go the library to see the newest copy of Lipids, Biochemistry, Journal of Neurochemistry, or Journal of Lipid Research to see what the newest papers are in each issue. This used to be accomplished by a weekly trip to the library to actually look at something called a printed copy of these journals. Hence, there was an effort to actually look at the literature, not merely use a computer search to find a particular topic. In the past, this was how we often found important papers that otherwise might have been overlooked. Today of course we have the table of contents for our favorite journals sent to us by e-mail, which enables us to look through the various papers published in an issue and select ones to print off or to save as a PDF to our computer. The question then is whether or not these printed or saved papers are ever really read and fully digested by the reader.

This brings me to the ever increasing citation of reviews as if they were primary literature. In the past, reviews were written by absolute leaders in the field who where imparting to the readers years of knowledge in the field and putting the current status of the field into perspective. Hence, there was a sense of validation to the papers that were cited in the review as you knew that the author was indeed one of the world’s leading experts on that topic. Is that still the case today? Over the past few years, I often find reviews written by people who I don’t recognize as a leader in the field and may not recognize them at all. This causes a major credibility gap, but more importantly, it suggests that less well-known individuals recognize that reviews, good or bad, will get cited by individuals who are unwilling to bother with reading the primary literature.

Let me give you a personal example as I am currently composing a review on fatty acid binding proteins (FABP) in the brain. I have used PubMed to look for potential missing key papers, which was more or less not overly productive. However, reading the literature, which when combined with the PubMed search, led to about 35 novel papers. After reading these papers, I generated another list of about 15 or more papers to acquire and then read prior to initiating writing of that review. While I know this literature quite well, I was amazed by the number of papers I didn’t have in my collection and these papers I came to via reading the literature and looking for unique citations in these papers that would be useful and pertinent to the review. However, this is a bit of a tangent, nonetheless an important one about composing a review, yet it includes the ideas of reading the papers being cited as well as using someone else’s citations to find previously unknown literature to you on the topic of interest. This newly acquired literature was located despite doing a thorough PubMed search, which of course depended upon me picking the correct key words as well as the authors picking the correct key words. While I figure that the authors might have chosen correctly, my confidence in myself on this point is less than overwhelming.

However, does a review on, let’s say FABP in the brain, suffice for a reference for the role of FABP3 in brain fatty acid uptake? No. A review might be useful to cite as a global vision on the role of FABP in brain function, yet the actual paper(s) in which the observation regarding fatty acid uptake was illuminated should be cited. This is where I find many authors repeatedly making this mistake, assuming that citing a review is sufficient, but it is absolutely not sufficient nor is such a citation correct.

Why is this so critical? Well for two overlapping reasons. First, it fails to give credit to the individuals who make a particular discovery and report this in the literature. The author of the review may not have made that observation, so why give them credit? So many times I see a repeated citation of a review by multiple authors over several years, yet when looking at the actual review I find it has little to do with what it is credited to report. Hence, it is especially important not to start an inappropriate base for others to continue to make an incorrect citation, but not giving credit where credit is due does exactly that. Second, citing a review does not permit an individual to grow the number of references to their work nor to the journal in which it was originally printed. This can impact many aspects of career development such as promotion and salary, but for a journal it impacts how well a journal is perceived by the greater science community. Thus, in the end, I guess the “Golden Rule” again comes into play, cite others’ work properly as you hope others will cite your work properly.

Yet amazingly, editors love to publish reviews because this type of publication is often highly cited and that will drive up a journal’s impact factor. Whether such a citation is right or wrong is based upon the willingness of the individuals doing the citing actually to read the review and the primary literature cited therein. Well-written reviews offer an incredible resource to a novice in the field giving a historical background and a current perspective by a leader in the field. Well-written reviews offer readers an up-to-date literature review and a great starting-off point to begin collecting the most pertinent literature that they can begin to read. Undoubtedly, this will give them an additional insight into particular discoveries as well as offer a new source for potential pertinent references not contained in the review. Hence, reviews are crucial for every field in science, but the use of reviews as the key citation for this or that discovery is not proper as such citations should go back to the original literature.

While Dr. Pearce highlighted the increase in the number of citations per paper using the American Journal of Physiology as an example, he noted that the average number of citations per paper grew from 29 in 1989 to 37 in 1999 and then to 42 in 2009 [1], I really don’t see much of a problem with this trend. As the literature expands, it is my expectation that authors will need to cite more and more papers from the literature to be complete. Adding an additional 13 papers cited on average over a 20 year period is not overwhelming nor is the growth of 5 papers over the last 10 years. I would rather see an author include 10–15 additional references in a manuscript that are all deemed important rather than pick and choose which work is cited or even worse use one or two reviews to fill this void. So, in the end I think it is important to cite papers that bring into perspective for the reader the important background required to introduce the topic properly, the methods used in the study, and the required citations to have a proper discussion of how the author's work fits into the field. This of course includes citations that both support the author’s conclusions as well as those that might dispute the author’s conclusions.

This editorial condenses down to my rather simple rules for citations. These rules may not be all-comprehensive, but I have done the best I can to be inclusive. Here they are:

• 1. Read and comprehend all of the literature that you are citing in your manuscript

• 2. Cite the primary literature and the actual papers to which a particular discovery is attributed; if multiple citations need to be made, do so

• 3. Cite the literature that agrees as well as that which disagrees with your point-of-view; be fair to multiple points-of-view

• 4. Be as inclusive as possible, but know where to draw the line, citing the most pertinent literature and the original papers

• 5. Use reviews judiciously in your manuscript and use them properly

• 6. Do NOT cite reviews in lieu of citing the original literature

• 7. Remember that PubMed goes back to 1966, but many worthwhile discoveries were made prior to that time and it is important to be historically accurate in your citations

• 8. When citing your own work, do so to support your point-of-view or to put the current work into the proper context for the reader. Do not attempt to grow your own citation base solely on the dreaded “self-citation”

Following these simple rules will certainly help limit the use of improper, superfluous citations because you have actually read the literature. An additional advantage is that by reading the literature, you will be able to find those elusive, yet important papers on your topic of interest. This adds importance and perhaps makes an Introduction and Discussion much more relevant and inclusive. By avoiding merely citing your own work and including work of others as well as those papers with a different point-of-view, you are in the end being a good citizen of science and working to recognize the contribution of your colleagues in the field. In the end, following these simple rules avoids many of the issues brought forth in this editorial and in the article by Dr. Pearce. So if you have been a citation abuser, try this 8-step program.