Skip to main content

Peer review during demanding times: maintain rigorous standards

Dear editor,

The current COVID-19 pandemic has cascaded into the realm of scientific publishing. As highlighted in the letter by Xianqiang Yu in the current issue of Scientometrics, there has been an alarming number of retracted articles (Yu 2020). In addition, articles published in preprint archives such as medRxiv have received broad and disproportionate media attention prior to their peer review (Bagdasarian et al. 2020). Also, the loudest voices in the COVID-19 debate, both original as well as opining articles, have been from mainly European and North-American researchers (Benjamens et al. 2020). Rapid dissemination of sometimes premature research findings, and the lack of inclusion of global voices may both contribute to public distrust in scientific integrity and findings. This requires improvement of scientific communication and structural improvement of the peer review process, when considering cutting edge research papers on emerging topics such as COVID-19.

Multiple reports were accepted for publication, although the data did insufficiently support the conclusions drawn (Type I error) (Teixeira da Silva et al. 2020) Prominent examples are the retracted publications on hydroxychloroquine in the New England Journal of Medicine and the Lancet (Mehra et al. 2020a, b). Additionally, opinion articles have been misquoted as original research, leading to misconception with potentially profound consequences. For instance, authors speculating on worsening of COVID-19 symptoms by ibuprofen (due to an induced increase of angiotensin-converting enzyme 2 receptor expression) ultimately led to a publication in the bulletin of the French ministry Health and to an unfunded advice by the World Health Organization to halt its use (Santé publique France n.d.; Teixeira da Silva et al. 2020). Unequivocally, retractions attract less media attention, when compared to the initial publications.

For publication of research on timely, emerging topics such as COVID-19, multiple potential solutions for improved peer review and public understanding of findings have been suggested. Publishers should clearly state the article type, i.e., expert opinion paper, original research paper, or rapid review paper. Preprint servers, such as medRxiv, should always clearly state the non-peer reviewed status of published research manuscripts, and additionally update manuscripts after formal peer review with clear revisions. Authors should also refrain from speculation, or clearly state lacking foundational evidence—as shown by the miscommunication on ibuprofen. Also, training in media communication and press releases can increase effectiveness and efficiency in dissemination of findings (Bagdasarian et al. 2020).

Apart from rapid retractions, the peer review process might require further reform for appropriateness in crisis situations such as the current COVID-19 pandemic. Potential steps to improve the review process of rapid, emerging research manuscripts have been proposed by Teixeira da Silva et al. These authors propose a six-step process for peer-review, including fast pre-screening by the editorial board, minimum of three independent peer review experts, and review within three weeks with a four week deadline for revisions (Teixeira da Silva et al. 2020). In 2015, Pagan and Torgler proposed the four-Rs for a structured review process, referring to reproduction, replication, robustness, and revelation. Especially revelation, pointing towards the need for accountability and transparency, can be considered significant to ensure quality when rapid peer-review is required (Pagan and Torgler 2015). Frijters and Torgler suggested implementation of an open-market peer-review system with clear quality indicators for reviewed manuscripts and review reports in 2019. Their two-sided, marketlike system consists of quality indicators for peer review, addressing both authors and peer reviewers. Key to successful implementation is installment of proper community incentives for participation (Frijters and Torgler 2019).

Uses in the scientific community evolve careful and deliberate—like the empirical method itself. Preparing for a crisis event such as the current COVID-19 pandemic is paradoxical and seems unnecessary because it inherently comes as a surprise and in an unpredictable form. Yet, an increased awareness on peer review in the context of emerging research on pressing topics is needed. Appropriate, high-quality peer review standards should be similar in all situations, even when society calls for rapid dissemination of new medical research findings. Scientific integrity and methodology should never be at stake—whatever the circumstances.

References

  1. Bagdasarian, N., Cross, G. B., & Fisher, D. (2020, December). Rapid publications risk the integrity of science in the era of COVID-19. BMC Medicine. https://doi.org/10.1186/s12916-020-01650-6

  2. Benjamens, S., de Meijer, V. E., Pol, R. A., & Haring, M. P. D. (2020, October). Are all voices heard in the COVID-19 debate? Scientometrics. https://doi.org/10.1007/s11192-020-03730-z

  3. Frijters, P., & Torgler, B. (2019). Improving the peer review process: A proposed market system. Scientometrics, 119(2), 1285–1288. https://doi.org/10.1007/s11192-019-03076-1

    Article  Google Scholar 

  4. Mehra, M. R., Desai, S. S., Kuy, S., Henry, T. D., & Patel, A. N. (2020a). Cardiovascular disease, drug therapy, and mortality in covid-19. New England Journal of Medicine, 382(25), e102. https://doi.org/10.1056/nejmoa2007621

    Article  Google Scholar 

  5. Mehra, M. R., Desai, S. S., Ruschitzka, F., & Patel, A. N. (2020). RETRACTED:Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. The Lancet. https://doi.org/10.1016/S0140-6736(20)31180-6

  6. Pagan, A., & Torgler, B. (2015). Use “4Rs” criteria to assess papers. Nature, 522(7554), 34–34. https://doi.org/10.1038/522034c

    Article  Google Scholar 

  7. Santé publique France. (n.d.). Coronavirus (COVID-19).

  8. Teixeira da Silva, J. A., Bornemann-Cimenti, H., & Tsigaris, P. (2020). Optimizing peer review to minimize the risk of retracting COVID-19-related literature. Medicine, Health Care, and Philosophy. https://doi.org/10.1007/s11019-020-09990-z

    Article  Google Scholar 

  9. Yu, X. (2020). What role should medical journals play during COVID-19? Scientometrics.

Download references

Funding

The authors have no funding to be reported.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Stan Benjamens.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Benjamens, S., Pol, R.A., de Meijer, V.E. et al. Peer review during demanding times: maintain rigorous standards. Scientometrics 126, 6115–6117 (2021). https://doi.org/10.1007/s11192-021-04007-9

Download citation