Avoid common mistakes on your manuscript.
Arshad et al. [1] assessed whether the use of a sensitive method for signal generation and then a specific method for signal evaluation in real-world studies improves the performance of vaccine safety surveillance. They did so by comparing type I and type II errors of three approaches against vaccine exposures with negative control and simulated positive control outcomes on the same data. The approaches are the historical comparator method alone, the self-controlled case series (SCCS) method alone, and the two methods combined serially. The authors state that serially combining epidemiological designs does not improve overall signal detection in vaccine safety surveillance.
We support their conclusion for vaccine monitoring systems to consider the relative gains that could come from using one design alone and postulate what a proper surveillance methodology would be. For example, the SCCS method is not suitable for outcomes that have insidious onset and long latency. In prospective surveillance, SCCS analysis will be delayed as it requires additional data accrual following the postvaccination risk window.
We also would like to provide additional context to explain why a serial approach can still be beneficial. The serial design can be useful in reducing systematic error, but the authors used it in a way that differs from how this is typically applied in vaccine safety surveillance. Rather than simply retesting signals flagged in the initial hypothesis generation by using a different analytic design on the same database, the second stage of signal evaluation (‘hypothesis testing’) in vaccine safety studies typically involves chart review validation as a gold standard [2]. We understand such validation is resource intensive and very time consuming, but given the importance of vaccines to public health, most researchers have deemed it worthwhile to do so to minimize the risk of both type I and type II errors. We suggest the following actions to refine signals and make hypothesis testing timelier: check data quality; examine diagnostic code frequencies and check reliability against laboratory and/or diagnostic imaging datasets; control properly for confounding; and replicate studies using different source data [2].
References
Arshad F, Schuemie MJ, Bu F, Minty EP, Alshammari TM, Lai LYH, et al. Serially combining epidemiological designs does not improve overall signal detection in vaccine safety surveillance. Drug Saf. 2023;46:797–807.
Yih WK, Kulldorff M, Fireman BH, Shui IM, Lewis EM, Klein NP, et al. Active surveillance for adverse events: the experience of the Vaccine Safety Datalink Project. Pediatrics. 2011;117:S54-64.
Acknowledgements
The authors thank the Safety Platform for Emergency vACcines (SPEAC) Executive Board for fruitful discussions on this topic.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
This work was supported by the Coalition for Epidemic Preparedness Innovations (CEPI) under a service order entitled Safety Platform for Emergency vACcines (SPEAC) Project with the Brighton Collaboration, a program of the Task Force for Global Health, Decatur, Georgia.
Conflict of interest
All the authors have no conflicts of interest to declare.
Availability of data and materials
Not applicable.
Ethics approval
Not applicable.
Code availability
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Author contributions
Conceptualization: WTH, RTC. Writing—original draft: CC. Writing—review & editing: WTH, RTC. All authors read and approved the final version.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.
About this article
Cite this article
Huang, WT., Chen, R.T. & Cassard, C. Comment on “Serially Combining Epidemiological Designs Does Not Improve Overall Signal Detection in Vaccine Safety Surveillance”. Drug Saf 47, 401–402 (2024). https://doi.org/10.1007/s40264-024-01410-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40264-024-01410-y