Application of Information Retrieval Approaches to Case Classification in the Vaccine Adverse Event Reporting System
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
Automating the classification of adverse event reports is an important step to improve the efficiency of vaccine safety surveillance. Previously we showed it was possible to classify reports using features extracted from the text of the reports.
The aim of this study was to use the information encoded in the Medical Dictionary for Regulatory Activities (MedDRA®) in the US Vaccine Adverse Event Reporting System (VAERS) to support and evaluate two classification approaches: a multiple information retrieval strategy and a rule-based approach. To evaluate the performance of these approaches, we selected the conditions of anaphylaxis and Guillain–Barré syndrome (GBS).
We used MedDRA® Preferred Terms stored in the VAERS, and two standardized medical terminologies: the Brighton Collaboration (BC) case definitions and Standardized MedDRA® Queries (SMQ) to classify two sets of reports for GBS and anaphylaxis. Two approaches were used: (i) the rule-based instruments that are available by the two terminologies (the Automatic Brighton Classification [ABC] tool and the SMQ algorithms); and (ii) the vector space model.
We found that the rule-based instruments, particularly the SMQ algorithms, achieved a high degree of specificity; however, there was a cost in terms of sensitivity in all but the narrow GBS SMQ algorithm that outperformed the remaining approaches (sensitivity in the testing set was equal to 99.06 % for this algorithm vs. 93.40 % for the vector space model). In the case of anaphylaxis, the vector space model achieved higher sensitivity compared with the best values of both the ABC tool and the SMQ algorithms in the testing set (86.44 % vs. 64.11 % and 52.54 %, respectively).
Our results showed the superiority of the vector space model over the existing rule-based approaches irrespective of the standardized medical knowledge represented by either the SMQ or the BC case definition. The vector space model might make automation of case definitions for spontaneous report review more efficient than current rule-based approaches, allowing more time for critical assessment and decision making by pharmacovigilance experts.
- Varricchio, F, Iskander, J, Destefano, F, Ball, R, Pless, R, Braun, MM (2004) Understanding vaccine safety information from the vaccine adverse event reporting system. Pediatr Infect Dis J 23: pp. 287-294 CrossRef
- Manning, CD, Raghavan, P, Schutze, H (2008) Introduction to information retrieval. Cambridge University Press, Cambridge CrossRef
- Manning, CD, Schutze, H (1999) Foundations of statistical natural language processing. MIT Press, Cambridge
- Brown, EG, Wood, L, Wood, S (1999) The medical dictionary for regulatory activities (MedDRA). Drug Saf 20: pp. 109-117 CrossRef
- Bonhoeffer, J, Kohl, K, Chen, R, Duclos, P, Heijbel, H, Heininger, U (2002) The Brighton Collaboration: addressing the need for standardized case definitions of adverse events following immunization (AEFI). Vaccine 21: pp. 298-302 CrossRef
- Humphreys, BL, Lindberg, DAB, Schoolman, HM, Barnett, GO (1998) The unified medical language system. J Am Med Inform Assoc 5: pp. 1-11 CrossRef
- Liu, H, Hu, ZZ, Zhang, J, Wu, C (2006) BioThesaurus: a web-based thesaurus of protein and gene names. Bioinformatics 22: pp. 103-105 CrossRef
- Thompson, P, McNaught, J, Montemagni, S, Calzolari, N, Gratta, R, Lee, V (2011) The BioLexicon: a large-scale terminological resource for biomedical text mining. BMC Bioinformatics 12: pp. 397 CrossRef
- Botsis, T, Nguyen, MD, Woo, EJ, Markatou, M, Ball, R (2011) Text mining for the Vaccine Adverse Event Reporting System: medical text classification using informative feature selection. J Am Med Inform Assoc. 18: pp. 631-638 CrossRef
- Pedersen, T, Pakhomov, SV, Patwardhan, S, Chute, CG (2007) Measures of semantic similarity and relatedness in the biomedical domain. J Biomed Inform 40: pp. 288-299 CrossRef
- Lin D. An information-theoretic definition of similarity. In: Proceedings of 15th international conference on machine learning. San Francisco: Morgan Kaufmann Publishers Inc.; 1998. p. 296–304.
- Cao, H, Melton, GB, Markatou, M, Hripcsak, G (2008) Use abstracted patient-specific features to assist an information-theoretic measurement to assess similarity between medical cases. J Biomed Inform 41: pp. 882-888 CrossRef
- Markatou, M, Kuruppumullage-Don, P, Hu, J, Wang, F, Sun, J, Sorrentino, R (2012) Case-based reasoning in comparative effectiveness research. IBM J Res Dev 56: pp. 5 CrossRef
- Botsis, T, Buttolph, T, Nguyen, MD, Winiecki, S, Woo, EJ, Ball, R (2012) Vaccine Adverse Event Text Mining (VaeTM) system for extracting features from vaccine safety reports. J Am Med Inform Assoc. 19: pp. 1011-1018 CrossRef
- Brown, EG (2004) Using MedDRA: implications for risk management. Drug Saf 27: pp. 591-602 CrossRef
- Mozzicato, P (2007) Standardised MedDRA queries: their role in signal detection. Drug Saf 30: pp. 617-619 CrossRef
- Ruggeberg, JU, Gold, MS, Bayas, JM, Blum, MD, Bonhoeffer, J, Friedlander, S (2007) Anaphylaxis: case definition and guidelines for data collection, analysis, and presentation of immunization safety data. Vaccine 25: pp. 5675-5684 CrossRef
- Sejvar, JJ, Kohl, KS, Gidudu, J, Amato, A, Bakshi, N, Baxter, R (2011) Guillain–Barré syndrome and Fisher syndrome: case definitions and guidelines for collection, analysis, and presentation of immunization safety data. Vaccine 29: pp. 599-612 CrossRef
- MedDRA Maintenance and Support Services Organization. Introductory guide for standardised MedDRA queries (SMQs) Version 14.1. Chantily: MedDRA; 2011.
- Medical Dictionary for Regulatory Activities: Maintenance and Support Services Organization. http://www.medramsso.com. Accessed 4 Apr 2013.
- Food and Drug Administration. Guidance for industry: good pharmacovigilance practices and pharmacoepidemiologic assessment. US Department of Health and Human Services. http://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM126834.pdf. Accessed 4 Apr 2013.
- Application of Information Retrieval Approaches to Case Classification in the Vaccine Adverse Event Reporting System
Volume 36, Issue 7 , pp 573-582
- Cover Date
- Print ISSN
- Online ISSN
- Springer International Publishing AG
- Additional Links
- Industry Sectors
- Author Affiliations
- 1. Office of Biostatistics and Epidemiology, Center for Biologics Evaluation and Research (CBER), US FDA, Woodmont Office Complex 1, Rm 306N, 1401 Rockville Pike, Rockville, MD, 20852, USA
- 2. Department of Computer Science, University of Tromsø, Tromsø, Norway