Skip to main content
SpringerLink
Log in

Leveraging Case Narratives to Enhance Patient Age Ascertainment from Adverse Event Reports

  • Original Research Article
  • Published:
Pharmaceutical Medicine Aims and scope Submit manuscript

Abstract

Introduction

Missing age presents a significant challenge when evaluating individual case safety reports (ICSRs) in the FDA Adverse Event Reporting System (FAERS). When age is missing in an ICSR’s structured field, it may be in the report’s free-text narrative.

Objectives

This study aimed to evaluate the performance and assess the potential impact of a rule-based natural language processing (NLP) tool that utilizes a text string search to identify patients’ numerical age from unstructured narratives.

Methods

Using FAERS ICSRs from 2002 to 2018, we evaluated the annual proportion of ICSRs with age missing in the structured field before and after NLP application. Reviewers manually identified patients’ age from ICSR narratives (gold standard) from a random sample of 1500 ICSRs. The gold standard was compared to the NLP-identified age.

Results

During the study period, the percentage of ICSRs missing age in the structured field increased from 21.9 to 43.8%. The NLP tool performed well among the random sample: sensitivity 98.5%, specificity 92.9%, positive predictive value (PPV) 94.9%, and F-measure 96.7%. It also performed well for the subset of ICSRs missing age in the structured field; when applied to these cases, NLP identified age for an additional one million ICSRs (10% of the total number of ICSRs from 2002 to 2018) and decreased the percentage of ICSRs missing age to 27% overall.

Conclusions

NLP has potential utility to extract patients’ age from ICSR narratives. Use of this tool would enhance pharmacovigilance and research using FAERS data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. US Food and Drug Administration. Questions and answers on FDA’s Adverse Event Reporting System (FAERS). https://www.fda.gov/drugs/surveillance/questions-and-answers-fdas-adverse-event-reporting-system-faers. Accessed 25 Nov 2019

  2. De Salas CSRL. Pharmacovigilance in pediatric population. In: Pharmacovigilance. IntechOpen; 2019.

    Google Scholar 

  3. Hoon D, Taylor MT, Kapadia P, Gerhard T, Strom BL, Horton DB. Trends in off-label drug use in ambulatory settings: 2006–2015. Pediatrics. 2019;144(4): e20190896.

    Article  Google Scholar 

  4. Moore TJ, Furberg CD, Mattison DR, Cohen MR. Completeness of serious adverse drug event reports received by the US Food and Drug Administration in 2014. Pharmacoepidemiol Drug Saf. 2016;25(6):713–8.

    Article  Google Scholar 

  5. Misu T, Kortepeter CM, Munoz MA, Wu E, Dal Pan GJ. An evaluation of “Drug Ineffective” postmarketing reports in drug safety surveillance. Drugs Real World Outcomes. 2018;5(2):91–9.

    Article  Google Scholar 

  6. Marwitz K, Jones SC, Kortepeter CM, Dal Pan GJ, Munoz MA. An evaluation of postmarketing reports with an outcome of death in the US FDA adverse event reporting system. Drug Saf. 2020;43(5):457–65.

    Article  Google Scholar 

  7. Duggirala, HJ, Tonning JM, Smith E, Bright RA, Baker JD, Ball R, et al. Data mining at FDA. http://www.fda.gov/downloads/ScienceResearch/DataMiningatFDA/UCM443675.pdf. Accessed 28 Nov 2019

  8. Johann-Liang R, Wyeth J, Chen M, Cope JU. Pediatric drug surveillance and the Food and Drug Administration’s adverse event reporting system: an overview of reports, 2003–2007. Pharmacoepidemiol Drug Saf. 2009;18(1):24–7.

    Article  Google Scholar 

  9. US Food and Drug Administration. Best Pharmaceuticals for Children Act and Pediatric Research Equity Act. https://www.fda.gov/science-research/pediatrics/best-pharmaceuticals-children-act-and-pediatric-research-equity-act. Accessed 3 May 2020

  10. Kreimeyer K, Foster M, Pandey A, Arya N, Halford G, Jones SF, et al. Natural language processing systems for capturing and standardizing unstructured clinical information: a systematic review. J Biomed Inform. 2017;73:14–29.

    Article  Google Scholar 

  11. Wong A, Plasek JM, Montecalvo SP, Zhou L. Natural language processing and its implications for the future of medication safety: a narrative review of recent advances and challenges. Pharmacotherapy. 2018;38(8):822–41.

    Article  Google Scholar 

  12. Wunnava S, Qin X, Kakar T, Socrates V, Wallace A, Rundensteiner E. Towards transforming FDA adverse event narratives into actionable structured data for improved pharmacovigilance. In: 2017 Proceedings of the symposium on applied computing, pp 777–82. https://doi.org/10.1145/3019612.3022875

  13. Electronic Code of Federal Regulations. Postmarketing reporting of adverse drug experiences. 21 CFR §314.80. https://www.ecfr.gov/cgi-bin/text-idx?SID=394813c1cb14662de9713c5f7b783043&mc=true&tpl=/ecfrbrowse/Title21/21cfr314_main_02.tpl. Accessed 14 May 2020

  14. US Food and Drug Administration. MedWatch: The FDA Safety Information and Adverse Event Reporting Program. Last updated: May 12, 2020. https://www.fda.gov/safety/medwatch-fda-safety-information-and-adverse-event-reporting-program. Accessed 14 May 2020

  15. International Council for Harmonisation (ICH). Efficacy guidelines. https://www.ich.org/page/efficacy-guidelines. Accessed 30 Apr 2020

  16. US Department of Health and Human Services. E2BM data elements for transmission of individual case safety reports. 2002. https://www.fda.gov/media/71208/download. Accessed 30 Apr 2020

  17. Hripcsak G, Rothschild AS. Agreement, the f-measure, and reliability in information retrieval. J Am Med Inform Assoc. 2005;12(3):296–8.

    Article  Google Scholar 

  18. US Food and Drug Administration. Pediatric safety. Last updated: March 22, 2018. https://www.fda.gov/science-research/pediatrics/pediatric-safety. Accessed 14 May 2020

  19. US Food and Drug Administration. Guidance for Industry: E2B(R3) Electronic Transmission of Individual Case Safety Reports (ICSRs) implementation guide—data elements and message specification. February 2014. https://www.fda.gov/media/81904/download. Accessed 30 Apr 2020

  20. Harinstein L, Kalra D, Kortepeter CM, Munoz MA, Wu E, Dal Pan GJ. Evaluation of postmarketing reports from industry-sponsored programs in drug safety surveillance. Drug Saf. 2019;42:649–55.

    Article  CAS  Google Scholar 

  21. Jokinen J, Bertin D, Donzanti B, Hormbrey J, Simmons V, Li H, et al. Industry assessment of the contribution of patient support programs, market research programs, and social media to patient safety. Ther Innov Regul Sci. 2019;53(6):736–45.

    Article  Google Scholar 

  22. Bergvall T, Noren GN, Lindquist M. vigiGrade: a tool to identify well-documented individual case reports and highlight systematic data quality issues. Drug Saf. 2014;37(1):65–77.

    Article  Google Scholar 

  23. Schmider J, Kumar K, LaForest C, Swankoski B, Naim K, Caubel PM. Innovation in pharmacovigilance: use of artificial intelligence in adverse event case processing. Clin Pharmacol Ther. 2019;105(4):954–61.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the Regulatory Science Staff within FDA’s Office of Surveillance and Epidemiology, who worked with students and staff at the Worcester Polytechnic Institute to develop the NLP algorithm. This project was supported in part by an appointment to the ORISE Research Participation Program at the CDER administered by the Oak Ridge Institute for Science and Education through an agreement between the U.S. Department of Energy and the U.S. FDA. Phuong Pham conducted this research while she was an ORISE fellow in the Office of Surveillance and Epidemiology, Center of Drug Evaluation and Research, FDA.

Author information

Authors and Affiliations

  1. Department of Pharmaceutical Outcomes and Policy, College of Pharmacy, University of Florida, Gainesville, FL, USA

    Phuong Pham & Monica A. Muñoz

  2. Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, MD, 20993, USA

    Phuong Pham, Carmen Cheng, Eileen Wu, Ivone Kim, Cindy M. Kortepeter & Monica A. Muñoz

  3. Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, 10903 New Hampshire Avenue, Silver Spring, MD, 20993, USA

    Rongmei Zhang & Yong Ma

Authors
  1. Phuong Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carmen Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eileen Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ivone Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rongmei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yong Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Cindy M. Kortepeter
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Monica A. Muñoz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carmen Cheng.

Ethics declarations

Funding

No sources of funding were used to assist in the preparation of this study.

Conflict of interest

Phuong Pham, Carmen Cheng, Eileen Wu, Ivone Kim, Rongmei Zhang, Yong Ma, Cindy M. Kortepeter, and Monica A. Muñoz have no conflicts of interest.

Availability of data and material

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Code availability

Not applicable.

Authors' contributions

All authors contributed to the study conception, design, and data collection. Data analysis was performed by Phuong Pham, Carmen Cheng, and Monica Munoz. The first draft of the manuscript was written by Phuong Pham and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Ethics approval

This study was granted an exemption for review by the U.S. Food and Drug Administration Institutional Review Board.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

This article reflects the views of the authors and should not necessarily be construed to represent FDA’s views or policies.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 107 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pham, P., Cheng, C., Wu, E. et al. Leveraging Case Narratives to Enhance Patient Age Ascertainment from Adverse Event Reports. Pharm Med 35, 307–316 (2021). https://doi.org/10.1007/s40290-021-00398-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40290-021-00398-5

Search

Navigation