Innovative Digital Tools and Surveillance Systems for the Timely Detection of Adverse Events at the Point of Care: A Proof-of-Concept Study
Introduction and Objective
Regulatory authorities often receive poorly structured safety reports requiring considerable effort to investigate potential adverse events post hoc. Automated question-and-answer systems may help to improve the overall quality of safety information transmitted to pharmacovigilance agencies. This paper explores the use of the VACC-Tool (ViVI Automated Case Classification Tool) 2.0, a mobile application enabling physicians to classify clinical cases according to 14 pre-defined case definitions for neuroinflammatory adverse events (NIAE) and in full compliance with data standards issued by the Clinical Data Interchange Standards Consortium.
The validation of the VACC-Tool 2.0 (beta-version) was conducted in the context of a unique quality management program for children with suspected NIAE in collaboration with the Robert Koch Institute in Berlin, Germany. The VACC-Tool was used for instant case classification and for longitudinal follow-up throughout the course of hospitalization. Results were compared to International Classification of Diseases , Tenth Revision (ICD-10) codes assigned in the emergency department (ED).
From 07/2013 to 10/2014, a total of 34,368 patients were seen in the ED, and 5243 patients were hospitalized; 243 of these were admitted for suspected NIAE (mean age: 8.5 years), thus participating in the quality management program. Using the VACC-Tool in the ED, 209 cases were classified successfully, 69 % of which had been missed or miscoded in the ED reports. Longitudinal follow-up with the VACC-Tool identified additional NIAE.
Mobile applications are taking data standards to the point of care, enabling clinicians to ascertain potential adverse events in the ED setting and during inpatient follow-up. Compliance with Clinical Data Interchange Standards Consortium (CDISC) data standards facilitates data interoperability according to regulatory requirements.
KeywordsAseptic Meningitis Progressive Multifocal Leukoencephalopathy Convulsive Seizure Vaccine Adverse Event Reporting System Case Classification
Compliance with Ethical Standards
No sources of funding were used to assist in the preparation of this study. Virus diagnostics for the quality management program were provided in-kind by the Robert Koch Institute.
Conflicts of interest
Christian Hoppe, Patrick Obermeier, Susann Muehlhans, Maren Alchikh, Lea Seeber, Franziska Tief, Katharina Karsch, Xi Chen, Sindy Boettcher, Sabine Diedrich, Tim Conrad, Bron Kisler, and Barbara Rath have no conflicts of interest that are directly relevant to the content of this study.
The quality management program was approved by the Charité Institutional Review Board (EA2/161/11). Informed consent procedures were waived by the Institutional Review Board for the purpose of quality improvement and infection control.
- 3.Miller ER, Haber P, Hibbs B, Broder K. Surveillance for adverse events following immunization using the Vaccine Adverse Event Reporting System (VAERS). 2014 1st April 2014. http://www.cdc.gov/vaccines/pubs/surv-manual/chpt21-surv-adverse-events.html#f25. Accessed 1 Jan 2016.
- 4.Uppsala Monitoring Centre. To improve worldwide patient safety. 2015. http://www.who-umc.org/. Accessed 2 Jan 2016.
- 12.Bailey C, Peddie D, Wickham ME, Badke K, Small SS, Doyle-Waters MM, et al. Adverse drug event reporting systems:a systematic review. Br J Clin Pharmacol. 2016 [Epub ahead of print].Google Scholar
- 13.Clinical Data Interchange Standards Consortium. Analysis Data Model (ADaM): data structure for adverse event analysis. 2012. http://www.cdisc.org/system/files/all/standard_category/application/pdf/adam_ae_final_v1.pdf. Accessed 5 Jun 2016.
- 14.Beresniak A, Schmidt A, Proeve J, Bolanos E, Patel N, Ammour N, et al. Cost-benefit assessment of using electronic health records data for clinical research versus current practices: contribution of the electronic health records for clinical research (EHR4CR) European project. Contemp Clin Trials. 2016;46:85–91.CrossRefPubMedGoogle Scholar
- 25.World Health Organization. Causality assessment of an adverse event following immunization (AEFI). 2013. http://www.who.int/vaccine_safety/publications/aefi_manual.pdf?ua=1. Accessed 28 Apr 2016.
- 27.Sejvar JJ, Kohl KS, Bilynsky R, Blumberg D, Cvetkovich T, Galama J, et al. Encephalitis, myelitis, and acute disseminated encephalomyelitis (ADEM): case definitions and guidelines for collection, analysis, and presentation of immunization safety data. Vaccine. 2007;25(31):5771–92.CrossRefPubMedGoogle Scholar
- 32.Clinical Data Interchange Standards Consortium. Clinical Data Acquisition Standards Harmonization (CDASH). 2016.; http://www.cdisc.org/cdash. Accessed 9 Mar 2016.
- 36.The Brighton Collaboration. AEFI case definition document. https://brightoncollaboration.org/public/what-we-do/setting-standards/case-definitions/process/main/02/link/Case_Definition_Format_Template.pdf. Accessed 3 Jan 2016.
- 37.Clinical Data Interchange Standards Consortium. Study Data Tabulation Model (SDTM). http://www.cdisc.org/sdtm. Accessed 10 Jan 2016.
- 38.Clinical Data Interchange Standards Consortium. Clinical Data Acquisition Standards Harmonization (CDASH). http://www.cdisc.org/cdash. Accessed 10 Jan 2016.
- 39.Clinical Data Interchange Standards Consortium. Biomedical Research Integrated Domain Group (BRIDG). http://www.cdisc.org/bridg. Accessed 10 Jan 2016.
- 40.U.S. Food and Drug Administration. Providing regulatory submissions in electronic format: standardized study data. December 2014. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM292334.pdf. Accessed 9 Mar 2016.
- 41.U.S. Food and Drug Administration. Study data standards resources. October 2015. http://www.fda.gov/forindustry/datastandards/studydatastandards/default.htm. Accessed 9 Mar 2016.
- 43.U.S. Food and Drug Administration. Statistical guidance on reporting results from studies evaluating diagnostic tests. 2007. http://www.fda.gov/RegulatoryInformation/Guidances/ucm071148.htm. Accessed 10 Jan 2016.
- 44.Donner A, Rotundi MA. Sample size requirements for interval estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters. Int J Biostat. 2010;6(1):Article 31.Google Scholar
- 45.Hall MA. Correlation-based feature subset selection for machine learning. Hamilton: University of Waikato; 1998.Google Scholar
- 46.Kullback S. Letter to the eitor: the Kullback-Leibler distance. Am Stat. 1987;41(4):340–1.Google Scholar
- 48.Miller E, Haber P, Hibbs B, Broder K. Surveillance for adverse events following immunization using the Vaccine Adverse Event Reporting System (VAERS). 2014. http://www.cdc.gov/vaccines/pubs/surv-manual/chpt21-surv-adverse-events.html#f5. Accessed 17 June 2016.
- 55.World Health Organization. International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10)-WHO Version for 2016. http://apps.who.int/classifications/icd10/browse/2016/en#/G51. Accessed 25 Feb 2016.
- 60.Rath B, Gidudu JF, Anyoti H, Bollweg B, Caubel P, Chen YH, et al. Facial nerve palsy including Bell’s palsy: case definitions and guidelines for collection, analysis, and presentation of immunisation safety data. Vaccine. 2016 [Epub ahead of print].Google Scholar
- 64.Twilt M. Precision medicine: the new era in medicine. 2016. http://dx.doi.org/10.1016/j.ebiom.2016.02.009. Accessed 17 June 2016.