Skip to main content
SpringerLink
Log in

The ABC of systematic literature review: the basic methodological guidance for beginners

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

There is a need for more methodological-based articles on systematic literature review (SLR) for non-health researchers to address issues related to the lack of methodological references in SLR and less suitability of existing methodological guidance. With that, this study presented a beginner's guide to basic methodological guides and key points to perform SLR, especially for those from non-health related background. For that, a total of 75 articles that passed the minimum quality were retrieved using systematic searching strategies. Seven main points of SLR were discussed, namely (1) the development and validation of the review protocol/publication standard/reporting standard/guidelines, (2) the formulation of research questions, (3) systematic searching strategies, (4) quality appraisal, (5) data extraction, (6) data synthesis, and (7) data demonstration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Athukorala, K., Głowacka, D., Jacucci, G., Oulasvirta, A., Vreeken, J.: Is exploratory search different?: A comparison of information search behavior for exploratory and lookup tasks. J. Assoc. Inf. Sci. Technol. 67(11), 2635–2651 (2016). https://doi.org/10.1002/asi.23617

    Article  Google Scholar 

  • Barnett-Page, E., Thomas, J.: Methods for the synthesis of qualitative research: a critical review. BMC Med. Res. Methodol. (2009). https://doi.org/10.1186/1471-2288-9-59

    Article  Google Scholar 

  • Bates, J., Best, P., McQuilkin, J., Taylor, B.: Will web search engines replace bibliographic databases in the systematic identification of research? J. Acad. Librariansh. 43(1), 8–17 (2017)

    Google Scholar 

  • Berrang-Ford, L., Pearce, T., Ford, J.D.: Systematic review approaches for climate change adaptation research. Reg. Environ. Change 15(5), 755–769 (2015). https://doi.org/10.1007/s10113-014-0708-7

    Article  Google Scholar 

  • Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006). https://doi.org/10.1191/1478088706qp063oa

    Article  Google Scholar 

  • Burgers, C., Brugman, B.C., Boeynaems, A.: Systematic literature reviews: four applications for interdisciplinary research. J. Pragmat. 145, 102–109 (2019)

    Google Scholar 

  • Brunton, G., Oliver, S., Oliver, K., Lorenc, T.: A Synthesis of Research Addressing Children’s, Young People’s and Parents’ Views of Walking and Cycling for Transport. EPPI-Centre, Social. Science Research Unit, Institute of Education, University of London, London (2006)

    Google Scholar 

  • Cañón, M., Buitrago-Gómez, Q.: The research question in clinical practice: a guideline for its formulation. Rev Colomb Psiquiatr. 47(3), 193–200 (2018). https://doi.org/10.1016/j.rcp.2016.06.004

    Article  Google Scholar 

  • Centre for Reviews and Dissemination.: Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care. University of York, York (2006)

    Google Scholar 

  • Charrois, T.L.: Systematic reviews: What do you get to know to get started? Can. J. Hosp. Pharm. 68(2), 144–148 (2015)

    Google Scholar 

  • Cooke, A., Smith, D., Booth, A.: Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual. Health Res. 22(10), 1435–1443 (2012)

    Google Scholar 

  • Cooper, C., Booth, A., Campbell, J., Britten, N., Garside, R.: Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med. Res. Methodol. 18, 85 (2018)

    Google Scholar 

  • Creswell, J.: Qualitative Inquiry & Research Design: Choosing Among Five Approaches, 3rd edn. Sage Publications, Inc., Thousand Oaks, CA (2013)

    Google Scholar 

  • del Amo, I.F., Erkoyuncu, J.A., Roy, R., Palmarini, R., Onoufriou, D.: A systematic review of augmented reality content-related techniques for knowledge transfer in maintenance applications. Comput. Ind. 103, 47–71 (2018)

    Google Scholar 

  • Delaney, A., Tamás, P.A.: Searching for evidence or approval? A commentary on database search in systematic reviews and alternative information retrieval methodologies. Res. Synth. Method 9(1), 124–131 (2018)

    Google Scholar 

  • Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., Sutton, A.: Synthesising qualitative and quantitative evidence: a review of possible methods. J. Health Serv. Res. Policy 10(1), 45–53 (2005)

    Google Scholar 

  • Doody, O., Bailey, M.E.: Setting a research question, aim and objective. Nurse Res. 23(4), 19–23 (2016)

    Google Scholar 

  • Durach, C.F., Kembro, J., Wieland, A.: A new paradigm for systematic literature reviews in supply chain management. J. Supply Chain Manag. 53(4), 67–85 (2017)

    Google Scholar 

  • Fagan, J.C.: An evidence-based review of academic web search engines, 2014–2016: implications for Librarians’ Practice and Research Agenda. Inf. Technol. Libr. 36(2), 7–47 (2017)

    Google Scholar 

  • Flemming, K., Booth, A., Garside, R., Tunc¸alp, O., Noyes J.: Qualitative evidence synthesis for complex interventions and guideline development: clarification of the purpose, designs and relevant methods. BMJ Global Health (2018)

  • Gomersall, J.S., Jadotte, Y.T., Xue, Y., Lockwood, S., Riddle, D., Preda, A.: Conducting systematic reviews of economic evaluations. Int. J. Evid.-Based Healthc. 13(3), 170–178 (2015). https://doi.org/10.1097/XEB.0000000000000063

    Article  Google Scholar 

  • Green, B.N., Johnson, C.D., Adams, A.: Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J. Chiropr. Med. 5(3), 101–117 (2006). https://doi.org/10.1016/S0899-3467(07)60142-6

    Article  Google Scholar 

  • Greyson, D., Rafferty, E., Slater, L., MacDonald, N., Bettinger, J.A., Dubé, È., MacDonald, S.E.: Systematic review searches must be systematic, comprehensive, and transparent: a critique of Perman et al. BMC Public Health 19(1), 1–6 (2019). https://doi.org/10.1186/s12889-018-6275-y

    Article  Google Scholar 

  • Gusenbauer, M.: Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases. Sciencetometrics 118(1), 177–214 (2019)

    Google Scholar 

  • Gusenbauer M, Haddaway NR (2020) Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res. Synth. Methods. 11(2):181–217. https://doi.org/10.1002/jrsm.1378

    Article  Google Scholar 

  • Haddaway, N.R., Collins, A.M., Coughlin, D., Kirk, S.: The role of google scholar in evidence reviews and its applicability to grey literature searching. PLoS ONE 10(9), e0138237 (2015). https://doi.org/10.1371/journal.pone.0138237

    Article  Google Scholar 

  • Haddaway, N.R., Macura, B., Whaley, P., Pulin, A.S.: ROSES Reporting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environ. Evid 7, 7 (2018). https://doi.org/10.1186/s13750-018-0121-7

    Article  Google Scholar 

  • Halevi, G., Moed, H., Bar-Illan, J.: Suitability of Google Scholar as a source of scientific information and as a source of data for scientific evaluation. Revi Lit 11(3), 823–834 (2017)

    Google Scholar 

  • Hannes, K.: Critical appraisal of qualitative research. In: Noyes, J., Hannes, K., Harden, A., Harris, J., Lewin, S., Lockwood, C. (eds.) Supplementary Guidance for Inclusion of Qualitative Research in Cochrane Systematic Reviews of Interventions. Cochrane Collaboration Qualitative Methods Group, London (2011)

    Google Scholar 

  • Higgins, J.P.T., Altman, D.G., Gotzsche, P.C., Juni, P., Moher, D., Oxman, A.D., Savovic, J., Schulz, K.F., Weeks, L., Sterne, J.A.C.: The cochrane collaboration's tool for assessing risk of bias in randomised trials. BMJ 343(7829), 1–9 (2011). https://doi.org/10.1136/bmj.d5928

    Article  Google Scholar 

  • Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., Welch, V.A. (eds.): Cochrane Handbook for Systematic Reviews of Interventions, 2nd edn. Wiley, Chichester (UK) (2019)

    Google Scholar 

  • Hong, Q.N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M-C., Vedel, I.: Mixed Methods Appraisal Tool (MMAT), version 2018. Registration of Copyright (#1148552), Canadian Intellectual Property Office, Industry Canada (2018)

  • Housyar, M., Sotudeh, H.: A reflection on the applicability of Google Scholar as a tool for comprehensive retrieval in bibliometric research and systematic reviews. Int. J. Inf. Sci. Manag. 16(2), 1–17 (2018)

    Google Scholar 

  • Hopia, H., Latvala, E., Liimatainen, L.: Reviewing the methodology of an integrative review. Scand. J. Caring Sci. 30(4), 662–669 (2016)

    Google Scholar 

  • Johnson, B.T., Hennessy, E.A.: Systematic reviews and meta-analyses in the health sciences: best practice methods for research syntheses. Soc. Sci. Med. 233, 237–251 (2019)

    Google Scholar 

  • Kastner, M., Straus, S., Goldsmith, C.H.: Estimating the horizon of articles to decide when to stop searching in systematic reviews: an example using a systematic review of RCTs evaluating osteoporosis clinical decision support tools. AMIA Annu. Symp. Proc. Arch. 2007, 389–393 (2007)

    Google Scholar 

  • Kitchenham, B.A., Charters, S.M.: Guidelines for performing systematic literature reviews in software engineering. EBSE Technical Report (2007)

  • Kraus, S., Breier, M., Dasí-Rodríguez, S.: The art of crafting a systematic literature review in entrepreneurship research. Int. Entrep. Manag. J. 16(3), 1023–1042 (2020). https://doi.org/10.1007/s11365-020-00635-4

    Article  Google Scholar 

  • Kushwah, S., Dhir, A., Sagar, M., Gupta, B.: Determinants of organic food consumption. A systematic literature review on motives and barriers. Appetite 143, 104402 (2019)

    Google Scholar 

  • Levy, Y., Ellis, T.J.: A systems approach to conduct an effective literature review in supports of information system research. Inf. Sci. J. 9, 181–212 (2006)

    Google Scholar 

  • Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gøtzsche, P.C., Ioannidis, J.P., Clarke, M., Devereaux, P.J., Kleijnen, J., Moher, D.: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6(7), e1000100 (2009)

    Google Scholar 

  • Linares-Espinós, E., Hernández, V., Domínguez-Escrig, J.L., Fernández-Pello, S., Hevia, V., Mayor, J., Padilla-Fernández, B., Ribal, M.J.: Methodology of systematic review. Actas urologicas españolas 42(8), 499–506 (2018)

    Google Scholar 

  • Lockwood, C., Munn, Z., Porritt, K.: Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int. J. Evid. Based Healthc. 13(3), 179–187 (2015). https://doi.org/10.1097/XEB.0000000000000062

    Article  Google Scholar 

  • Long, A.F., Godfrey, M.: An evaluation tool to assess the quality of qualitative research studies. Int. J. Soc. Res. Methodol. 7(2), 181–196 (2004)

    Google Scholar 

  • Mallet, R., Hagen-Zanker, J., Slater, R., Duvendack, M.: The benefits and challenges of using systematic reviews in international development research. J. Dev. Eff. 4, 445–455 (2012)

    Google Scholar 

  • Mantzoukas, S.: Facilitating research students in formulating qualitative research questions. Nurse Educ. Today 28(3), 371–377 (2008)

    Google Scholar 

  • Mays, N., Pope, C., Popay, J.: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J. Health Serv. Res. Policy 10(1), 6–20 (2005)

    Google Scholar 

  • Mengist, W., Soromessa, T., Legese, G.: Method for conducting systematic literature review and meta-analysis for environmental science research. MethodsX 7, 100777 (2020). https://doi.org/10.1016/j.mex.2019.100777

    Article  Google Scholar 

  • Methley, A.M., Campbell, S., Chew-Graham, C., McNally, R., Cheraghi-Sohi, S.: PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv. Res. (2014). https://doi.org/10.1186/s12913-014-0579-0

    Article  Google Scholar 

  • Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., PRISMA Group: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 21;6(7), e1000097 (2009)

    Google Scholar 

  • Morton SC, Murad MH, O’Connor E, Lee CS, Booth M, Vandermeer BW, Snowden JM, D’Anci KE, Fu R, Gartlehner G, Wang Z, Steele DW (2018) Quantitative synthesis—an update. methods guide for comparative effectiveness reviews. (Prepared by the Scientific Resource Center under Contract No. 290-2012-0004-C). AHRQ Publication No. 18-EHC007- EF. Rockville, MD: Agency for Healthcare Research and Quality. Posted final reports are located on the Effective Health Care Program search page. https://doi.org/10.23970/AHRQEPCMETHGUIDE3

  • Noble, H., Mitchell, M.: What is grounded theory? Evid. Based Nurs. 19(2), 34–35 (2016). https://doi.org/10.1136/eb-2016-102306

    Article  Google Scholar 

  • Okoli, C.: A guide to conducting a standalone systematic literature review. Commun. Assoc. Inf. Syst. 37, 879–910 (2015)

    Google Scholar 

  • Okoli, C., Schabram, K.: A guide to conducting a systematic literature review of information systems research. Philos. Methodol. Econ. eJ. (2010). https://doi.org/10.2139/ssrn.1954824

  • Onwuegbuzie, A.J., Leech, N.L.: Sampling designs in qualitative research: making the sampling process more public. Qual. Rep. 12(2), 238–254 (2007)

    Google Scholar 

  • Pace, R., Pluye, P., Bartlett, G., Macaulay, A.C., Salsberg, J., Jagosh, J., Seller, R.: Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int. J. Nurs Stud. 49(1), 47–53 (2012). https://doi.org/10.1016/j.ijnurstu.2011.07.002

    Article  Google Scholar 

  • Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T., Mulrow, C.D., Shamseer, L., Moher, D.: Mapping of reporting guidance for systematic reviews and meta-analyses generated a comprehensive item bank for future reporting guidelines. J. Clin. Epidemiol. 118, 60–68 (2020)

    Google Scholar 

  • Paterson, B.L., Thorne, S.E., Canam, C., Jillings, C.: Meta-Study of Qualitative Health Research. A Practical Guide to Meta-Analysis and Meta-Synthesis. Sage Publications, Thousand Oaks (2001)

    Google Scholar 

  • Palaskar, J.N.: Framing the research question using PICO strategy. J. Dent. Allied Sci. 6(2), 55 (2017)

    Google Scholar 

  • Patino, C.M., Ferreira, J.C.: Inclusion and exclusion criteria in research studies: definitions and why they matter. J. Bras. Pneumol. 44(2), 84 (2018)

    Google Scholar 

  • Peters, M.D., Godfrey, C.M., Khalil, H., McInerney, P., Parker, D., Soares, C.B.: Guidance for conducting systematic scoping reviews. Int. J. Evid.-Based Healthc. 13(3), 141–146 (2015). https://doi.org/10.1097/XEB.0000000000000050

    Article  Google Scholar 

  • Petticrew, M., Roberts, H.: Systematic Reviews in the Social Sciences: A Practical Guide. Blackwell Publishing Ltd, Oxford (2006)

    Google Scholar 

  • Prieto and Rumbo-Prieto: The systematic review: plurality of approaches and methodologies. Enferm. Clín. (English Edition) 28(6), 387–393 (2018)

    Google Scholar 

  • Reim, W., Parida, V., Örtqvist, D.: Product-Service Systems (PSS) business models and tactics – a systematic literature review. J. Clean. Prod. 97, 61–75 (2015). https://doi.org/10.1016/j.jclepro.2014.07.003

    Article  Google Scholar 

  • Robinson, P., Lowe, J.: Literature reviews vs systematic reviews. Aust. N. Z. J. Public Health 39(2), 103 (2015)

    Google Scholar 

  • Rousseau, D. M., Manning, J., Denyer, D.: Evidence in management and organizational science: assembling the field’s full weight of scientific knowledge through syntheses. In: AIM Research Working Paper Series: Advanced Institute of Management Research (2008)

  • Sandelowski, M., Barroso, J., Voils, C.I.: Using qualitative metasummary to synthesize qualitative and quantitative descriptive findings. Res. Nuurs. Health 30(1), 99–111 (2007). https://doi.org/10.1002/nur.20176

    Article  Google Scholar 

  • Sandelowski, M., Voils, C.I., Barroso, J.: Defining and designing mixed research synthesis studies. Res. Schools: Nat. Ref. J. Spons. Mid-South Educ. Res. Assoc. Univ. Alabama 13(1), 29 (2006)

    Google Scholar 

  • Schardt, C., Adams, M.B., Owens, T., Keitz, S., Fontelo, P.: Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med. Inf. Decis. Mak. 7, 16 (2007). https://doi.org/10.1186/1472-6947-7-16

    Article  Google Scholar 

  • Seehra, J., Pandis, N., Koletsi, D.: Use of quality assessment tools in systematic reviews was varied and inconsistent. J. Clin. Epidemiol. 69, 179–184 (2016)

    Google Scholar 

  • Shaffril, H.A.M., Krauss, S.E., Samsuddin, S.F.: A systematic review on Asian’s farmers’ adaptation practices towards climate change. Sci. Total Environ. 644, 683–695 (2018)

    Google Scholar 

  • Shaffril, H.A.M., Abu Samah, A., Samsuddin, S.F., Ali, Z.: Mirror-mirror on the wall, what climate change adaptation strategies are practiced by the Asian’s fishermen of all? J. Clean. Prod. 232, 104–117 (2019)

    Google Scholar 

  • Shorten, A., Shorten, B.: What is meta-analysis. Evid. Based Nurs. 16(1), 3–4 (2013)

    Google Scholar 

  • Siering, U., Eikermann, M., Hausner, E., Hoffmann-Eßer, W., Neugebauer, E.A.: Appraisal tools for clinical practice guidelines: a systematic review. PLoS ONE 8(12), e82915 (2013)

    Google Scholar 

  • Soares, C.B., Hoga, L., Sangaleti, C., Yonekura, T., Peduzzi, M., Silva, D.: Integrative review in nursing research and EBP: a type of systematic review? Int. J.Evid.-Based Healthcare 11(3), 246–247 (2013)

    Google Scholar 

  • Thomas, J., Noel-Storrb, A., Marshall, I., Wallace, B., McDonald, S., Mavergames, C., Glasziou, P., Shemilta, I., Synnote, A., Turnere, T., Elliott, J.: Living systematic reviews: 2. Combining human and machine effort. J. Clin. Epidemiol. 91, 31–37 (2017)

    Google Scholar 

  • Thomas, J., Kneale, D., McKenzie, J.E., Brennan, S.E., Bhaumik, S.: Chapter 2: Determining the scope of the review and the questions it will address. In: Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., Welch, V.A. (eds.) Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (2020). Cochrane (2020). https://www.training.cochrane.org/handbook

  • Wanden-Berghe, C., Sanz-Valero, J.: Systematic reviews in nutrition: standardized methodology. Br. J. Nutr. 107, S3–S7 (2012)

    Google Scholar 

  • Whittemore, R., Knafl, K.: The integrative review: updated methodology. J. Adv. Nurs. 52(5), 546–553 (2005)

    Google Scholar 

  • Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., Pawson, R.: RAMESES publication standards: realist syntheses. BMC Med. 11, 21 (2013). https://doi.org/10.1186/1741-7015-11-21

    Article  Google Scholar 

  • Xiao, Y., Watson, M.: Guidance on conducting a systematic literature review. J. Plan. Educ. 39(1), 93–112 (2019)

    Google Scholar 

  • Younger, P.: Using Google Scholar to conduct a literature search. Nurs. Stand. 24(45), 40–46 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Social Science Studies, Universiti Putra Malaysia, Putra Infoport, 43400, Serdang, Selangor, Malaysia

    Hayrol Azril Mohamed Shaffril & Asnarulkhadi Abu Samah

  2. Faculty of Computer Science and Information Technology, University of Malaya, 50603, Kuala Lumpur, Malaysia

    Samsul Farid Samsuddin

  3. Faculty of Human Ecology, Universiti Putra Malaysia, 43400, Serdang, Selangor, Malaysia

    Asnarulkhadi Abu Samah

Authors
  1. Hayrol Azril Mohamed Shaffril
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samsul Farid Samsuddin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Asnarulkhadi Abu Samah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hayrol Azril Mohamed Shaffril.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mohamed Shaffril, H.A., Samsuddin, S.F. & Abu Samah, A. The ABC of systematic literature review: the basic methodological guidance for beginners. Qual Quant 55, 1319–1346 (2021). https://doi.org/10.1007/s11135-020-01059-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-020-01059-6

Keywords

Search

Navigation