Abstract
Introduction
The method of time-to-onset (TTO) has been proposed to overcome the drawbacks of traditional disproportionality analyses (DPAs), and it has been used for detecting safety signals of vaccines and some non-vaccine products in spontaneous reporting systems (SRSs). However, there is no consensus on its superiority over DPAs. Further, it is still not clear whether this novel approach can be generalized to the entire national SRS database.
Objective
The purpose of this study was to generalize the TTO method to the Chinese SRS and to identify suitable parameters for its optimal performance.
Methods
Reports submitted to the national SRS of China in 2014 were used as the data source for analysis. We evaluated the performance of TTO by using product labels as proxies for the gold standard. A series of values of significance level and time windows were explored to identify the most suitable parameters for TTO based on Youden’s index, a statistic that summarizes the performance of a diagnostic test. Additionally, we compared TTO with traditional DPAs and explored the characteristics of signals detected by these methods.
Results
Compared with DPAs, TTO had a lower sensitivity, but higher specificity and positive predictive value. At a significance level of 0.2 and no restrictions on time windows, TTO had the highest Youden’s index. The kappa coefficients between TTO and DPAs were rather low, indicating poor agreement between the two methods. More than 30% of the true signals detected by TTO were not identified by DPAs. Furthermore, TTO needed more number of reports to be able to detect signals.
Conclusions
TTO can detect signals missed by traditional DPAs and could be an important complementary tool to the currently used DPAs in the SRS of China. We recommend a significance level of 0.2 and no restrictions on time windows for TTO.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
China Food and Drug Administration. 2015 annual report for national adverse drug reaction monitoring released. 2016 [cited 2016 14/07]; Available from: http://www.sda.gov.cn/WS03/CL0757/159101.html.
Almenoff JS, Pattishall EN, Gibbs TG, DuMouchel W, Evans SJ, Yuen N. Novel statistical tools for monitoring the safety of marketed drugs. Clin Pharmacol Therapeutics. 2007;82(2):157–66.
Bate A, Evans SJ. Quantitative signal detection using spontaneous ADR reporting. Pharmacoepidemiol Drug Saf. 2009;18(6):427–36.
Hou Y, Li X, Wu G, Ye X. National ADR monitoring system in China. Drug Saf. 2016;39(11):1043–51.
Evans SJ, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf. 2001;10(6):483–6.
Bate A, Lindquist M, Edwards IR, Olsson S, Orre R, Lansner A, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol. 1998;54(4):315–21.
Niklas Norén G, Bate A, Orre R, Ralph Edwards I. Extending the methods used to screen the WHO drug safety database towards analysis of complex associations and improved accuracy for rare events. Stat Med. 2006;25(21):3740–57.
Van Holle L, Zeinoun Z, Bauchau V, Verstraeten T. Using time-to-onset for detecting safety signals in spontaneous reports of adverse events following immunization: a proof of concept study. Pharmacoepidemiol Drug Saf. 2012;21(6):603–10.
van Holle L, Bauchau V. Signal detection on spontaneous reports of adverse events following immunisation: a comparison of the performance of a disproportionality-based algorithm and a time-to-onset-based algorithm. Pharmacoepidemiol Drug Saf. 2014;23(2):178–85.
Scholl JH, van Puijenbroek EP. The value of time-to-onset in statistical signal detection of adverse drug reactions: a comparison with disproportionality analysis in spontaneous reports from the Netherlands. Pharmacoepidemiol Drug Saf. 2016;25(12):1361–7.
Zhang L, Yan J, Liu X, Ye Z, Yang X, Meyboom R, et al. Pharmacovigilance practice and risk control of traditional Chinese medicine drugs in China: current status and future perspective. Acta Psychiatr Scand. 2015;83(4):262–6.
Guo XJ, Ye XF, Wang XX, Wang J, Shi WT, Gao QB, et al. Reporting patterns of adverse drug reactions over recent years in China: analysis from publications. Expert Opin Drug Saf. 2015;14(2):191–8.
Hou Y, Ye X, Wu G, Cheng G, Du X, He J. A comparison of disproportionality analysis methods in national adverse drug reaction databases of China. Expert Opin Drug Saf. 2014;13(7):853–7.
Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3(1):32–5.
Schisterman EF, Bondell H. Optimal cut-point and its corresponding Youden index to discriminate individuals using pooled blood samples. Epidemiology. 2005;16(1):73–81.
Altman DG. Practical statistics for medical research: London: Chapman and Hall; 1991.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
This study was granted by the National Nature Science Foundation of China (81373105), the Fourth Round of Three-year Action Plan on Public Health Discipline and Talent Program: Evidence-based Public Health and Health Economics (15GWZK0901), Foundation for Young Scholars of Second Military Medical University (2014QN09), and Innovative programme for PhD in Second Military Medical University.
Conflict of interest
Tianyi Zhang, Xiaofei Ye, Xiaojing Guo, Guizhi Wu, Yongfang Hou, Jinfang Xu, Wentao Shi, Tiantian Zhu, Yuan Zhang, Xinji Zhang, Jiaqi Song and Jia He have no conflicts of interest that are directly relevant to the content of this study.
Additional information
Tianyi Zhang, Xiaofei Ye, Xiaojing Guo, Guizhi Wu and Yongfang Hou contributed equally and are co-first authors of this article.
Rights and permissions
About this article
Cite this article
Zhang, T., Ye, X., Guo, X. et al. Signal Detection Based on Time to Onset Algorithm in Spontaneous Reporting System of China. Drug Saf 40, 343–350 (2017). https://doi.org/10.1007/s40264-016-0503-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40264-016-0503-0