Patterns and Pitfalls of Short-cuts Used in Environmental Management Rapid Reviews

Abstract

Environmental managers and policy-makers need reliable evidence to make effective decisions. Systematic reviews are one way to provide this information but are time-consuming and may not meet the needs of decision-makers when faced with rapidly changing management requirements or transient policy-windows. Rapid reviews are one type of knowledge synthesis that follow simplified or truncated methods compared to systematic reviews. Rapid reviews on environmentally-relevant topics are growing in prevalence, but it is unclear if rapid reviews use similar short-cuts or follow available guidelines. In this methodological review, we assess 26 rapid reviews published between 2002 and 2023. Numerous rapid review short-cuts and approaches were identified, with few consistencies among studies. Short-cuts were present in all stages of the review process, with some of the most common short-cuts including not developing an a priori review protocol, not including stakeholder involvement, or not conducting critical appraisal of study validity. Poor quality in reporting of methods was observed. Fewer than half of assessed rapid reviews reported using available guidelines when developing their methods. Future rapid reviews should aim for improved reporting and adherence to published guidelines to help increase the useability and evidence-user confidence. This will also enable readers to understand where short-cuts were made and their potential consequences for the conclusions of the review.

Appendix 1

Searches:

• Web of Science Core Collection (WoS) - May 10, 2023; no date or language restrictions. Searched using advanced search for terms in the title, abstract, and keywords (Topic search).

  • Update search ran September 27, 2023 to capture new articles since the original search.

• Scopus – May 24, 2023; no date or language restrictions. Searched using advanced search for terms in title, abstract and keywords (TITLE-ABS-KEY)

  • Updated search run on September 27, 2023 to capture new articles published since the original search.

• Google Scholar Search 1–2 – September 27, 2023; no date or language restrictions. Did not include citations or patents. Searched for words occurring “anywhere in the article”. Sorted by relevance and selected first 50 articles only.

• Google Scholar Search 3–5 – September 30, 2023; no date or language restrictions. Did not include citations or patents. Searched for words occurring “anywhere in the article”. Sorted by relevance and selected first 50 articles only.

Final search string (WoS):

TS = ((“Rapid review$“ OR “Accelerated review$“ OR “Abbreviated review$“ OR “Rapid evidence synthesis” OR “Accelerated systematic review$“ OR “Brief review$“ OR “Evidence summary” OR “Evidence summaries” OR “Expedited systematic review$“ OR “Quick review$“ OR “Rapid evidence assessment$“ OR “Rapid evidence review$“ OR “Rapid evidence-based literature review$“ OR “Rapid interim review$“ OR “Rapid literature review$“ OR “Rapid narrative review$“ OR “Rapid nonsystematic review$“ OR “Rapid structure review$“ OR “Rapid structure literature review$“ OR “Rapid systematic review$“ OR “Systematic rapid evidence review$“ OR “Ultrafast review$“) AND (“Environmental management” OR “Ecosystem management” OR Conserv* OR Ecolog* OR Environment* OR Biolog* OR “Climate change” OR Agricultur* OR Forestry OR Fisher* OR “Natural resource*“ OR Biodivers* OR “Ecosystem service$“ OR “Sustainable energy” OR Soil* OR Aquatic OR “Water quality” OR “Wastewater”) NOT (“COVID-19”))

Final search string (Scopus):

TITLE-ABS-KEY (“Rapid review?” OR “Accelerated review?” OR “Abbreviated review?” OR “Rapid evidence synthesis” OR “Accelerated systematic review?” OR “Brief review?” OR “Evidence summary” OR “Evidence summaries” OR “Expedited systematic review?” OR “Quick review?” OR “Rapid evidence assessment?” OR “Rapid evidence review?” OR “Rapid evidence-based literature review?” OR “Rapid interim review?” OR “Rapid literature review?” OR “Rapid narrative review?” OR “Rapid nonsystematic review?” OR “Rapid structure review?” OR “Rapid structure literature review?” OR “Rapid systematic review?” OR “Systematic rapid evidence review?” OR “Ultrafast review?”) AND TITLE-ABS-KEY (“Environmental management” OR “Ecosystem management” OR Conserv* OR Ecolog* OR Environment* OR Biolog* OR “Climate change” OR Agricultur* OR Forestry OR Fisher* OR “Natural resource*“ OR Biodivers* OR “Ecosystem service?” OR “Sustainable energy” OR Soil* OR Aquatic OR “Water quality” OR “Wastewater”) AND NOT TITLE-ABS-KEY (“COVID-19”)

Final search string (Google Scholar 1):

“rapid review” “Environmental management” OR “Ecosystem management” OR Conservation OR Ecology OR Environment OR Biology OR “Climate Change” OR Agriculture OR Forestry OR Fisheries OR “Natural resource” OR Biodiversity OR “Ecosystem Service” OR “Sustainable energy” OR Soil OR Aquatic OR “Water quality” OR “wastewater” -“Covid 19”

Final search string (Google Scholar 2):

“Rapid evidence” “Environmental management” OR “Ecosystem management” OR Conservation OR Ecology OR Environment OR Biology OR “Climate Change” OR Agriculture OR Forestry OR Fisheries OR “Natural resource” OR Biodiversity OR “Ecosystem Service” OR “Sustainable energy” OR Soil OR Aquatic OR “Water quality” OR “wastewater” -“COVID 19”

Final search string (Google Scholar 3):

“Rapid * review” “Environmental management” OR “Ecosystem management” OR Conservation OR Ecology OR Environment OR Biology OR “Climate Change” OR Agriculture OR Forestry OR Fisheries OR “Natural resource” OR Biodiversity OR “Ecosystem Service” OR “Sustainable energy” OR Soil OR Aquatic OR “Water quality” OR “wastewater” -“COVID 19”

Final search string (Google Scholar 4):

“Brief review” “Environmental management” OR “Ecosystem management” OR Conservation OR Ecology OR Environment OR Biology OR “Climate Change” OR Agriculture OR Forestry OR Fisheries OR “Natural resource” OR Biodiversity OR “Ecosystem Service” OR “Sustainable energy” OR Soil OR Aquatic OR “Water quality” OR wastewater -“COVID 19”

Final search string (Google Scholar 5):

“Quick review” “Environmental management” OR “Ecosystem management” OR Conservation OR Ecology OR Environment OR Biology OR “Climate Change” OR Agriculture OR Forestry OR Fisheries OR “Natural resource” OR Biodiversity OR “Ecosystem Service” OR “Sustainable energy” OR Soil OR Aquatic OR “Water quality” OR “Wastewater” -“COVID 19”

Benchmark articles (all found in WoS; Scopus = Rowland et al. 2021, Pullin 2023 and Miller et al. 2018):

• Rowland et al. 2021

• Forbes et al (2021)

• Naz and Chowdhury (2022)

• Pullin (2023)

• Miller et al (2018)