Drug Safety

, Volume 29, Issue 5, pp 375–384 | Cite as

Data Mining in the US using the Vaccine Adverse Event Reporting System

  • John Iskander
  • Vitali Pool
  • Weigong Zhou
  • Roseanne English-Bullard
Leading Article


The US Vaccine Adverse Event Reporting System (VAERS), which is charged with vigilance for detecting vaccine-related safety issues, faces an increasingly complex immunisation environment. Since 1990, steady increases in vaccine licensing and distribution have resulted in increasing numbers of reports to VAERS. Prominent features of current reports include more routine vaccine co-administration and frequent reports of new postvaccination clinical syndromes. Data-mining methods, based on disproportionality analyses, are one strategy being pursued by VAERS researchers to increase the utility of its complex database. The types of analyses used include proportional reporting ratios, association rule discovery, and various ‘historic limits’ methods that compare observed versus expected event counts. The use of such strategies in VAERS has been primarily supplemental and retrospective. Signals for inactivated influenza, typhoid and tetanus toxoid-containing vaccines have been successfully identified. Concerns flagged through data mining should always be subject to clinical case review as a first evaluation step. Persistent issues should be subject to formal hypothesis testing in large linked databases or other controlled-study settings.

Automated data-mining techniques for prospective use are currently undergoing development and evaluation within VAERS. Their use (as one signal-detection tool among many) by trained medical evaluators who are aware of system limitations is one legitimate approach to improving the ability of VAERS to generate vaccine-safety hypotheses. Such approaches are needed as more new vaccines continue to be licensed.


  1. 1.
    Wise R, Kiminyo K, Salive M. Hair loss after routine immunizations. JAMA 1997 Oct 8; 278(14): 1176–8PubMedCrossRefGoogle Scholar
  2. 2.
    Woo E, Ball R, Bostrom A, et al. Vaccine risk perception among reporters of autism after vaccination: vaccine adverse event reporting system 1990-2001. Am J Public Health 2004 Jun; 94(6): 990–5PubMedCrossRefGoogle Scholar
  3. 3.
    Varricchio F, Iskander J, DeStefano F, et al. Understanding vaccine safety: information from the Vaccine Adverse Event Reporting System (VAERS). Pediatr Infect Dis J 2004; 23: 1–8Google Scholar
  4. 4.
    Zanardi LR, Haber P, Mootrey GT, et al. Intussusception among recipients of rotavirus vaccine: reports to the Vaccine Adverse Event Reporting System. Pediatrics 2001; 107(6): E97 [online]. Available from URL: http://www.pediatrics.org/cgi/content/full/107/6/e97 [Accessed 2006 April 3]PubMedCrossRefGoogle Scholar
  5. 5.
    Wise RP, Salive ME, Braun MM, et al. Postlicensure safety surveillance for varicella vaccine. JAMA 2000; 284: 1271–9PubMedCrossRefGoogle Scholar
  6. 6.
    Braun MM, Mootrey GT, Salive ME, et al. Infant immunization with acellular pertussis vaccine in the US: assessment of the first two years’ data from the Vaccine Adverse Event Reporting System. Pediatrics 2000; 106: E51 [online]. Available from URL: http://pediatrics.aappublications.org/cgi/reprint/106/4/e51.pdf [Accessed 2006 April 3]PubMedCrossRefGoogle Scholar
  7. 7.
    Pool V, Braun M, Kelso J, et al. Prevalence of anti-gelatin IgE antibodies in people with anaphylaxis after measles-mumps rubella vaccine in the United States. Pediatrics 2002 Dec; 110(6): e71PubMedCrossRefGoogle Scholar
  8. 8.
    Barwick-zEidex R for the Yellow Fever Vaccine Safety Working Group. History of thymoma and yellow fever vaccination. Lancet 2004 Sep 11; 364 (9438): 936Google Scholar
  9. 9.
    Centers for Disease Control and Prevention. Withdrawal of rotavirus vaccine recommendation. MMWR 1999; 48: 1007Google Scholar
  10. 10.
    Centers for Disease Control and Prevention. Thimerosal in vaccines: a joint statement of the American Academy of Pediatrics and the Public Health Service. MMWR 1999; 48: 563–5Google Scholar
  11. 11.
    Wakefield AJ, Murch SH, Anthony A, et al. Ileal-lymphoidnodular hyperplasis, non-specific colitis, and pervasive developmental disorder in children. Lancet 1998 Feb 28; 351(9103): 637–41PubMedCrossRefGoogle Scholar
  12. 12.
    Vaccine adverse event reporting system [online]. Available from URL: https://secure.vaers.org/VaersDataEntryintro.htm [Accessed 2005 Apr 21]Google Scholar
  13. 13.
    Centers for Disease Control and Prevention. Smallpox vaccine adverse events monitoring and response system for the first stage of the smallpox vaccination program. MMWR 2002; 52: 88–9Google Scholar
  14. 14.
    Centers for Disease Control and Prevention. Notice to readers: public comment sought on CDC’s vaccine safety. MMWR 2004; 53 (31): 720Google Scholar
  15. 15.
    Centers for Disease Control and Prevention. Adverse events following immunization. Surveillance Report No. 4, 1979–1990. Atlanta (GA): Centers for Disease Control and Prevention, 1995 JunGoogle Scholar
  16. 16.
    Immunization Action Coalition [online]. Available from URL: http://www.immunize.org [Accessed 2005 Apr 21]Google Scholar
  17. 17.
    National Immunization Program [online]. Available from URL: www.cdc.gov/nip/publications/images/schedule1989s.jpg [Accessed 2006 Apr 13]Google Scholar
  18. 18.
    Data on file, Centers for Disease Control and Prevention, 2004Google Scholar
  19. 19.
    US Code of Federal Regulations, 21CFR600.80. Postmarketing reporting of adverse experiences [online]. Available from URL: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?.FR = 600.80 [Accessed 2005 Apr 21]Google Scholar
  20. 20.
    Coding Symbols for Thesaurus of Adverse Reaction Terms (COSTART). 5th ed. Rockville (MD): US Food and Drug Administration, Center for Drug Evaluation and Research, 1995Google Scholar
  21. 21.
    Chen R, Pool V, Takahashi H, et al. Combination vaccines: postlicensure safety evaluation. Clin Infect Dis 2001; 33Suppl. 4: S327–33PubMedCrossRefGoogle Scholar
  22. 22.
    Finney DJ. The design and logic of a monitor of drug use. J Chronic Dis 1965 Jan; 18: 77–98PubMedCrossRefGoogle Scholar
  23. 23.
    Evans SJW, Waller PC, Davis S. Use of proportional reporting ratios for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 2001; 10: 483–6PubMedCrossRefGoogle Scholar
  24. 24.
    Meyboom RH, Egberts AC, Edwards IR, et al. Principles of signal detection in pharmacovigilance. Drug Saf 1997; 16: 355–65PubMedCrossRefGoogle Scholar
  25. 25.
    Virtanen M, Peltola H, Paunio M, et al. Day-to-day reactogenicity and the healthy vaccinee effect of measles-mumps-rubella vaccination. Pediatrics 2000 Nov; 106(5): E62PubMedCrossRefGoogle Scholar
  26. 26.
    Centers for Disease Control and Prevention. General recommendations on immunization: recommendations of the Advisory Committee on Immunization Practices (ACIP) and the American Academy of Family Physicians (AAF). MMWR 2002, 51 (No. RR-2): 19–21Google Scholar
  27. 27.
    Offit PA. The power of “box a”. Expert Rev Vaccines 2003 Feb; 2(1): 1–3PubMedCrossRefGoogle Scholar
  28. 28.
    Zhou W, Pool V, Iskander J, et al. Surveillance for safety after immunization: vaccine adverse event reporting system (VAERS): United States, 1991-2001. In: Centers for Disease Control and Prevention DC surveillance summaries (January 24). MMWR 2003, 52 (NoSS-1)Google Scholar
  29. 29.
    Rosenthal S, Chen R, Hadler S. The safety of acellular pertussis vaccine vs whole-cell pertussis vaccine. A postmarketing assessment. Arch Pediatr Adolesc Med 1996; 150(5): 457–60PubMedCrossRefGoogle Scholar
  30. 30.
    Ball R. Methods for ensuring vaccine safety. Expert Rev Vaccines 2002; 1: 161–8PubMedCrossRefGoogle Scholar
  31. 31.
    Haber P, Iskander J, English-Bullard R, et al. Use of proportional reporting rate ratio in monitoring vaccine adverse event reports, US [abstract]. Pharmacoepidemiol Drug Saf 2002; 11: S229CrossRefGoogle Scholar
  32. 32.
    Khromova A, Pool V, Chen R. Oculo-Respiratory Syndrome (ORS) following influenza vaccine: United States, 1990-2002: new or previously unrecognized? [poster]. Epidemic Intelligence Service Conference; 2003 Apr; Atlanta (GA)Google Scholar
  33. 33.
    Dumouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am Stat 1999; 53: 177–90Google Scholar
  34. 34.
    Bate A, Lindquist M, Orre R. Data-mining analyses of pharmacovigilance signals in relation to relevant comparison drugs. Eur J Clin Pharmacol 2002; 58: 483–90PubMedCrossRefGoogle Scholar
  35. 35.
    Rolka H, Bracy D, Russell C, et al. Using simulation to assess for the sensitivity and specificity of a signal detection tool for multidimensional public health surveillance data. Stat Med 2005; 24: 551–62PubMedCrossRefGoogle Scholar
  36. 36.
    Pool V, Chen RT. Association rule discovery as a signal generation tool for the vaccine adverse event reporting system. Pharmacoepidemiol Drug Saf 2003; 11: S57Google Scholar
  37. 37.
    Evans S, Lemon SJ, Deters CA, et al. Automated detection of hereditary syndromes using data mining. Comput Biomed Res 1997 Oct; 30(5): 337–48PubMedCrossRefGoogle Scholar
  38. 38.
    Evans S, Lemon SJ, Deters C. Using data mining to characterize DNA mutations by patient clinical features. Proc AMIA Annu Fall Symp. 1997, 7Google Scholar
  39. 39.
    van Puijenbroek EP, Bate A, Leufkens HG, et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol Drug Saf 2002 Jan-Feb; 11(1): 3–10PubMedCrossRefGoogle Scholar
  40. 40.
    Szarfman A, Machado SG, O’Neill RT. Use of screening algorithms and computer systems to efficiently signal higher than expected combinations of drugs and events in the US FDA’s spontaneous reports database. Drug Saf 2002; 25(6): 381–92PubMedCrossRefGoogle Scholar
  41. 41.
    Teutsch SM, Churchill RE. Principles and practice of public health surveillance. New York: Oxford Univerisity Press, 1994Google Scholar
  42. 42.
    Lawson BM, Fitzhugh EC, Hall SP, et al. Multifaceted syndromic surveillance in a public health department using the early aberration reporting system. J Public Health Manag Pract 2005 Jul-Aug; 11(4): 274–81PubMedGoogle Scholar
  43. 43.
    Sawyer J. Use of temporal cluster analysis for vaccine safety signal detection. Centers for Disease Control and Prevention/ATSDR Statistical Symposium; 2005 Feb; Washington, DCGoogle Scholar
  44. 44.
    Haber P, Zhou W, English-Bullard R, et al. Positive re-challenge reports following hepatitis B vaccination in adults, Vaccine Adverse Event Reporting System (VAERS), USA, 1995-2003. Pharmacoepidemiol Drug Saf 2004; 13: S279–80Google Scholar
  45. 45.
    World Health Organization [online]. Available from URL: http://www.who-umc.org/graphics/8321.pdf [Accessed 2006 Apr 13]Google Scholar
  46. 46.
    Centers for Disease Control and Prevention. Simultaneous administration of varicella vaccine and other recommended childhood vaccines: United States, 1995–1999. MMWR 2001; 50: 1058–61Google Scholar
  47. 47.
    Begier E, Burwen D, Haber P, et al. Postmarketing safety surveillance for typhoid fever vaccines from the Vaccine Adverse Event Reporting System, July 1990 through June 2002. Clin Infect Dis 2004 Mar 15; 38(6): 771–9PubMedCrossRefGoogle Scholar
  48. 48.
    McMahon A, Iskander J, Haber P, et al. Adverse events alter inactivated influenza vaccination among children less than 2 years of age: analysis of reports from the vaccine adverse event reporting system, 1990-2003. Pediatrics 2005 Feb; 115(2): 453–60PubMedCrossRefGoogle Scholar
  49. 49.
    Centers for Disease Control and Prevention. Notice to readers: inadvertent intradermal administration of tetanus toxoid-containing vaccines instead of tuberculosis skin tests. MMWR 2004; 53: 663–4Google Scholar
  50. 50.
    Graham D, Dan B, Bertagnoll P, et al. Cutaneous inflammation caused by inadvertent intradermal administration of DTP instead of PPD. Am J Public Health 1981; 71: 1040–3PubMedCrossRefGoogle Scholar
  51. 51.
    Zhou W, Pool V, DeStefano F, et al. A potential signal of Bell’s palsy after parenteral inactivated influenza vaccines: reports to the Vaccine Adverse Event Reporting System (VAERS): United States, 1991-2001. Pharmacoepidemiol Drug Saf 2004 Aug; 13(8): 505–10PubMedCrossRefGoogle Scholar
  52. 52.
    Zhou W. A population-based study of Bell’s Palsy after parenteral inactivated influenza vaccines. Epidemic Intelligence Service Conference; 2003 Apr; Atlanta (GA)Google Scholar
  53. 53.
    Data on file, Centers for Disease Control and Prevention, 2005Google Scholar
  54. 54.
    Niu MT, Erwin DE, Braun MM. Data mining in the US Vaccine Adverse Event Reporting System (VAERS): early detection of intussusception and other events after rotavirus vaccination. Vaccine 2001; 19: 4627–34PubMedCrossRefGoogle Scholar
  55. 55.
    Banks D, Woo EJ, Burwen D, et al. Comparison of 4 data mining methods in the US Vaccine Adverse Event Reporting System (VAERS). Pharmacoepidemiol Drug Saf 2003; 12: S138Google Scholar
  56. 56.
    Pool V, Pless R, English-Bullard R. Use of proportionate reporting ratios in postmarketing surveillance for vaccine adverse events: comparison of two methodological approaches. Pharmacoepidemiol Drug Saf 2002; 11: S243–4Google Scholar
  57. 57.
    McMahon B, Pugh TF. Epidemiology principles and methods. Boston: Little, Brown and Company, 1970Google Scholar
  58. 58.
    Moore N, Thiessard F, Begaud B. The history of disproportionality measures in spontaneous reporting of adverse drug reactions. Pharmacoepidemiol Drug Saf 2005; 24: 285–6CrossRefGoogle Scholar
  59. 59.
    Evans S. The mountain peaks and pitfalls of being spontaneous. International Society of Pharmacovigilance; 2004 Oct; DublinGoogle Scholar
  60. 60.
    Lasky T, Terracciano GJ, Magder L, et al. The Guillain-Barre syndrome and the 1992-1993 and 1993-1994 influenza vaccines. N Engl J Med 1998; 339: 1797–802PubMedCrossRefGoogle Scholar
  61. 61.
    Niu MT, Rhodes P, Salive M, et al. Comparative safety data of two recombinant hepatitis B vaccines in children: data from the Vaccine Adverse Event Reporting System (VAERS) and Vaccine Safety Datalink (VSD). J Clin Epidemiol 1998; 51: 503–10PubMedCrossRefGoogle Scholar
  62. 62.
    Centers for Disease Control and Prevention. Suspension of rotavirus vaccine after reports of intussusception: United States, 1999. MMWR 2004; 53: 786–9Google Scholar
  63. 63.
    Von Kries R, Toschke AM, Strassburger K, et al. Sudden and unexpected deaths after the administration of hexavalent vaccines: is there a signal? Eur J Pediatr 2005 Feb; 164(2): 61–9CrossRefGoogle Scholar
  64. 64.
    US Food and Drug Administration. Guidance for industry: good pharmacovigilance practices and pharmacoepidemiologic assessment [online]. Available from URL: www.fda.gov/cder/guidance/6359OCC.htm [Accessed 2005 Apr 21]Google Scholar
  65. 65.
    Bonhoeffer J, Kohl K, Chen R, et al. The Brighton Collaboration-enhancing vaccine safety. Vaccine 2004 May 7; 22(15-16): 2046PubMedCrossRefGoogle Scholar
  66. 66.
    Stratton K, Almario DA, Wizeman TM, McCormick MC, editors. Immunization Safety Review Committee, Immunization safety review: vaccinations and sudden unexpected death in infancy. Washington, DC: Institute of Medicine, 2003Google Scholar
  67. 67.
    Lindquist M. Data quality management in pharmacovigilance. Drug Saf 2004; 27(12): 857–70PubMedCrossRefGoogle Scholar
  68. 68.
    Haber P, Rashidee A, Zhou W, et al. Web-based reporting: 10 months experience in the vaccine adverse event reporting system (VAERS), USA [abstract]. Pharmacoepidemiol Drug Saf 2003; 12: S092Google Scholar
  69. 69.
    Fritzell B. Detection of adverse events: what are the current sensitivity limits during clinical development. Vaccine 2002; 20: S47–8CrossRefGoogle Scholar
  70. 70.
    Chen RT, Lane JM. Myocarditis: the unexpected return of smallpox vaccine adverse events. Lancet 2003; 362(9393): 1345–6PubMedCrossRefGoogle Scholar
  71. 71.
    Shapiro S. Clinical judgment, common sense and adverse reaction reporting. Pharmacoepidemiol Drug Saf 2004 Aug; 13(8): 511–3PubMedCrossRefGoogle Scholar
  72. 72.
    Zhou W, Pool V, DeStefano F, et al. Reply to the editorial. Pharmacoepidemiol Drug Saf 2004 Aug; 13(8): 515–7CrossRefGoogle Scholar
  73. 73.
    Wilson AM, Thabane L, Holbrook A. Application of data mining techniques in pharmacovigilance. Br J Clin Pharmacol 2004 Feb; 57(2): 127–34PubMedCrossRefGoogle Scholar
  74. 74.
    Trontell A. Expecting the unexpected: drug safety, pharmacovigilance, and the prepared mind. N Engl J Med 2004 Sep 30; 351(14): 1385–7PubMedCrossRefGoogle Scholar
  75. 75.
    Institute of Medicine. Stratton K, Gable A, Shetty P, McCormick M, editors. Immunization safety review: measlesmumps-rubella vaccine and autism. Washington, DC: National Academy Press, 2001Google Scholar
  76. 76.
    Centers for Disease Control and Prevention. National smallpox vaccine in pregnancy registry. MMWR 2003; 52: 256Google Scholar
  77. 77.
    Wise RP, Iskander J, Pratt RD, et al. Poslicensure safety surveillance for 7-valent pneumococcal conjugate vaccine. JAMA 2004 Oct 13; 292(14): 1702–10PubMedCrossRefGoogle Scholar
  78. 78.
    Verstraeten T, Davis RL, Destefano F, et al. Safety of thimerosal-containing vaccines: a two-phased study of computerized health maintenance organization databases. Pediatrics 2003 Nov; 112(5): 1039–48PubMedGoogle Scholar
  79. 79.
    DeStefano F, Bhasin TK, Thompson WW, et al. Age at first measles-mumps-rubella vaccination in children with autism and school-matched control subjects: a population-based study in metropolitan Atlanta. Pediatrics 2004 Feb; 113(2): 259–66PubMedCrossRefGoogle Scholar
  80. 80.
    Halsey NA, Hyman SL, Conference Writing Panel. Measlesmumps-rubella vaccine and autistic spectrum disorder: report from the New Challenges in Childhood Immunizations Conference convened in Oak Brook, Illinois; 2000 June 12-13. Pediatrics 2001 May; 107(5): E84PubMedCrossRefGoogle Scholar
  81. 81.
    Madsen KM, Vestergaard M. MMR vaccination and autism: what is the evidence for a causal association? Drug Saf 2004; 27(12): 831–40PubMedCrossRefGoogle Scholar
  82. 82.
    Evans SJ. Pharmacovigilance: a science or fielding emergencies? Stat Med 2000 Dec 15; 19(23): 3199–209PubMedCrossRefGoogle Scholar
  83. 83.
    Davis RL, Kolczak M, Lewis E, et al. Active surveillance of vaccine safety: a system to detect early signs of adverse events. Epidemiology 2005 May; 16(3): 336–41PubMedCrossRefGoogle Scholar
  84. 84.
    Centers for Disease Control and Prevention. Rotavirus vaccine for the prevention of rotavirus gastroenteritis among children: recommendations of the Advisory Committee on Immunization Practices. MMWR 1999 March 19, 48 (No. RR–2)Google Scholar
  85. 85.
    American Academy of Pediatrics Committee on Infectious Diseases. Recommendations for the use of live attenuated varicella vaccine. Pediatrics 1995; 95: 791–6Google Scholar
  86. 86.
    Chen RT, Glasser JW, Rhodes PH, et al. The vaccine safety datalink project: a new tool for improving vaccine safety monitoring in the United States. Pediatrics 1997; 99: 765–73PubMedCrossRefGoogle Scholar

Copyright information

© Adis Data Information BV 2006

Authors and Affiliations

  • John Iskander
    • 1
  • Vitali Pool
    • 1
  • Weigong Zhou
    • 2
  • Roseanne English-Bullard
    • 3
  1. 1.Office of Immunization Safety, Office of the Chief Science Officer, Centers for Disease Control and PreventionAtlantaUSA
  2. 2.Division of Viral and Rickettsial Diseases, Influenza BranchNational Center for Infectious Diseases, Centers for Disease Control and PreventionAtlantaUSA
  3. 3.National Center for Public Health Informatics, Centers for Disease Control and PreventionAtlantaUSA

Personalised recommendations