Skip to main content
Log in

Bidirectional Citation Searching to Completion: An Exploration of Literature Searching Methods

  • Practical Application
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

Literature reviews underpin the majority of research projects in the health sciences, and yet relatively little analysis has been published as to the most appropriate method to identify relevant literature, outside of specialist information journals. The method of applying keyword search queries to bibliographic databases using Boolean logic dominates literature reviews due to its easy application to the major online databases. However, it is recognised increasingly as being problematic where the research question cannot be clearly defined or requires an element of exploration, due to its reliance on author’s use of titling and keywords and is unable to identify topics other than those defined in the search query. This paper discusses the relative merits of a systematic citation searching approach as both an alternative and a concurrent method to keyword searching. A method of citation searching, both forwards and backwards, which is iterated to form a closed loop solution, is discussed. An illustrative example is presented of both methods, applying them to the topic of the UK National Institute for Health and Care Excellence (NICE) cost-effectiveness threshold. The case study finds the citation searching approach dominates the traditional keyword searching approach, finding 76 papers of relevance, including all 15 found by the alternative approach. Conceptually, and in the example presented, it is demonstrated that the proposed method can represent a dominant strategy to the more traditional approach in some situations, highlighting that, wherever possible, it is preferential to employ multiple methods of searching. However, it is clear that a better understanding is required as to how we can most efficiently search the ever-growing sea of literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Box 1
Fig. 2

Similar content being viewed by others

References

  1. NICE, Guide to the methods of technology appraisal 2013. London: NICE, 2013.

  2. Weinstein MC, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices-Modeling Studies. Value Health. 2003;6(1):9–17.

    Article  PubMed  Google Scholar 

  3. Schlosser RW, et al. Use of information-seeking strategies for developing systematic reviews and engaging in evidence-based practice: the application of traditional and comprehensive Pearl Growing: a review. Int J Lang Commun Disord. 2006;41(5):567–82.

    Article  PubMed  Google Scholar 

  4. Garfield E. Citation indexes for science. Science. 1955;122:108–11.

    Article  CAS  PubMed  Google Scholar 

  5. Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005;331(7524):1064–5.

    Article  PubMed Central  PubMed  Google Scholar 

  6. Higgins J, Green S. Cochrane handbook for systematic reviews for interventions. The Cochrane Collaboration, 2008.

  7. Golder S, Loke Y, Zorzela L. Comparison of search strategies in systematic reviews of adverse effects to other systematic reviews. Health Information Libraries Journal, 2014. Forthcoming.

  8. Golder S, Loke YK, Zorzela L. Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011. J Clin Epidemiol. 2013;66(3):253–60.

    Article  PubMed  Google Scholar 

  9. Kulkarni AV, et al. Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. JAMA. 2009;302(10):1092–6.

    Article  CAS  PubMed  Google Scholar 

  10. Bakkalbasi N, et al. Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomed Digit Libr. 2006;3:7.

    Article  PubMed Central  PubMed  Google Scholar 

  11. Robinson KA, et al. Citation networks of related trials are often disconnected: implications for bidirectional citation searches. J Clin Epidemiol. 2014;67(7):793–9.

    Article  PubMed  Google Scholar 

  12. Claxton K, et al. Methods for the estimation of the NICE cost effectiveness threshold. CHE Research Paper, 2013. 81.

  13. Papaioannou D, et al. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Info Libr J. 2010;27(2):114–22.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The authors would like to acknowledge the contributions of Su Golder, Prof. Karl Claxton and Prof. Mark Sculpher, all of whom helped with the conceptualisation and editing of this research.

Research was conducted as part of funding received from the Medical Research Council/National Institute for Health Research through its Methodology Research Programme (Award G0901498/1), both researchers having received funding from the sponsor for the duration of the project.

Author contribution

Sebastian Hinde conducted the literature searches and was first author of the paper. Eldon Spackman provided supervision during the literature searches, defining the aims of the reviews, as well as helping to draft the paper.

Conflict of interest

Neither author has any conflicts of interest to declare.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sebastian Hinde.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hinde, S., Spackman, E. Bidirectional Citation Searching to Completion: An Exploration of Literature Searching Methods. PharmacoEconomics 33, 5–11 (2015). https://doi.org/10.1007/s40273-014-0205-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-014-0205-3

Keywords

Navigation