Abstract
Literature reviews underpin the majority of research projects in the health sciences, and yet relatively little analysis has been published as to the most appropriate method to identify relevant literature, outside of specialist information journals. The method of applying keyword search queries to bibliographic databases using Boolean logic dominates literature reviews due to its easy application to the major online databases. However, it is recognised increasingly as being problematic where the research question cannot be clearly defined or requires an element of exploration, due to its reliance on author’s use of titling and keywords and is unable to identify topics other than those defined in the search query. This paper discusses the relative merits of a systematic citation searching approach as both an alternative and a concurrent method to keyword searching. A method of citation searching, both forwards and backwards, which is iterated to form a closed loop solution, is discussed. An illustrative example is presented of both methods, applying them to the topic of the UK National Institute for Health and Care Excellence (NICE) cost-effectiveness threshold. The case study finds the citation searching approach dominates the traditional keyword searching approach, finding 76 papers of relevance, including all 15 found by the alternative approach. Conceptually, and in the example presented, it is demonstrated that the proposed method can represent a dominant strategy to the more traditional approach in some situations, highlighting that, wherever possible, it is preferential to employ multiple methods of searching. However, it is clear that a better understanding is required as to how we can most efficiently search the ever-growing sea of literature.
Similar content being viewed by others
References
NICE, Guide to the methods of technology appraisal 2013. London: NICE, 2013.
Weinstein MC, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices-Modeling Studies. Value Health. 2003;6(1):9–17.
Schlosser RW, et al. Use of information-seeking strategies for developing systematic reviews and engaging in evidence-based practice: the application of traditional and comprehensive Pearl Growing: a review. Int J Lang Commun Disord. 2006;41(5):567–82.
Garfield E. Citation indexes for science. Science. 1955;122:108–11.
Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005;331(7524):1064–5.
Higgins J, Green S. Cochrane handbook for systematic reviews for interventions. The Cochrane Collaboration, 2008.
Golder S, Loke Y, Zorzela L. Comparison of search strategies in systematic reviews of adverse effects to other systematic reviews. Health Information Libraries Journal, 2014. Forthcoming.
Golder S, Loke YK, Zorzela L. Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011. J Clin Epidemiol. 2013;66(3):253–60.
Kulkarni AV, et al. Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. JAMA. 2009;302(10):1092–6.
Bakkalbasi N, et al. Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomed Digit Libr. 2006;3:7.
Robinson KA, et al. Citation networks of related trials are often disconnected: implications for bidirectional citation searches. J Clin Epidemiol. 2014;67(7):793–9.
Claxton K, et al. Methods for the estimation of the NICE cost effectiveness threshold. CHE Research Paper, 2013. 81.
Papaioannou D, et al. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Info Libr J. 2010;27(2):114–22.
Acknowledgments
The authors would like to acknowledge the contributions of Su Golder, Prof. Karl Claxton and Prof. Mark Sculpher, all of whom helped with the conceptualisation and editing of this research.
Research was conducted as part of funding received from the Medical Research Council/National Institute for Health Research through its Methodology Research Programme (Award G0901498/1), both researchers having received funding from the sponsor for the duration of the project.
Author contribution
Sebastian Hinde conducted the literature searches and was first author of the paper. Eldon Spackman provided supervision during the literature searches, defining the aims of the reviews, as well as helping to draft the paper.
Conflict of interest
Neither author has any conflicts of interest to declare.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hinde, S., Spackman, E. Bidirectional Citation Searching to Completion: An Exploration of Literature Searching Methods. PharmacoEconomics 33, 5–11 (2015). https://doi.org/10.1007/s40273-014-0205-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40273-014-0205-3