Skip to main content
SpringerLink
Log in

A Comparative Assessment of Observational Medical Outcomes Partnership and Mini-Sentinel Common Data Models and Analytics: Implications for Active Drug Safety Surveillance

  • Original Research Article
  • Published:
Drug Safety Aims and scope Submit manuscript

An Erratum to this article was published on 02 July 2015

Abstract

Introduction

An often key component to coordinating surveillance activities across distributed networks is the design and implementation of a common data model (CDM). The purpose of this study was to evaluate two drug safety surveillance CDMs from an ecosystem perspective to better understand how differences in CDMs and analytic tools affect usability and interpretation of results.

Methods

Humana claims data from 2007 to 2012 were mapped to Observational Medical Outcomes Partnership (OMOP) and Mini-Sentinel CDMs. Data were described and compared at the patient level by source code and mapped concepts. Study cohort construction and effect estimates were also compared using two different analytical methods—one based on a new user design implementing a high-dimensional propensity score (HDPS) algorithm and the other based on univariate self-controlled case series (SCCS) design—across six established positive drug-outcome pairs to learn how differences in CDMs and analytics influence steps in the database analytic process and results.

Results

Claims data for approximately 7.7 million Humana health plan members were transformed into the two CDMs. Three health outcome cohorts and two drug cohorts showed differences in cohort size and constituency between Mini-Sentinel and OMOP CDMs, which was a result of multiple factors. Overall, the implementation of the HDPS procedure on Mini-Sentinel CDM detected more known positive associations than that on OMOP CDM. The SCCS method results were comparable on both CDMs. Differences in the implementation of the HDPS procedure between the two CDMs were identified; analytic model and risk period specification had a significant impact on the performance of the HDPS procedure on OMOP CDM.

Conclusions

Differences were observed between OMOP and Mini-Sentinel CDMs. The analysis of both CDMs at the data model level indicated that such conceptual differences had only a slight but not significant impact on identifying known safety associations. Our results show that differences at the ecosystem level of analyses across the CDMs can lead to strikingly different risk estimations, but this can be primarily attributed to the choices of analytic approach and their implementation in the community-developed analytic tools. The opportunities of using CDMs are clear, but our study shows the need for judicious comparison of analyses across the CDMs. Our work emphasizes the need for ongoing efforts to ensure sustainable transparent platforms to maintain and develop CDMs and associated tools for effective safety surveillance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. MedDRA® terminology is the international medical terminology developed under the auspices of the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). MedDRA® trademark is owned by the International Federation of Pharmaceutical Manufacturers and Associations (IFPMA) on behalf of ICH.

References

  1. Behrman RE, Benner JS, Brown JS, McClellan M, Woodcock J, Platt R. Developing the Sentinel system—a national resource for evidence development. N Engl J Med. 2011;364(6):498–9. doi:10.1056/NEJMp1014427.

    Article  CAS  PubMed  Google Scholar 

  2. Suissa S, Garbe E. Primer: administrative health databases in observational studies of drug effects–advantages and disadvantages. Nat Clin Pract Rheumatol. 2007;3(12):725–32. doi:10.1038/ncprheum0652.

    Article  CAS  PubMed  Google Scholar 

  3. Brown JS, Holmes JH, Shah K, Hall K, Lazarus R, Platt R. Distributed health data networks: a practical and preferred approach to multi-institutional evaluations of comparative effectiveness, safety, and quality of care. Med Care. 2010;48(6 Suppl):S45–51. doi:10.1097/MLR.0b013e3181d9919f.

    Article  PubMed  Google Scholar 

  4. Stang PE, Ryan PB, Racoosin JA, Overhage JM, Hartzema AG, Reich C, et al. Advancing the science for active surveillance: rationale and design for the Observational Medical Outcomes Partnership. Ann Intern Med. 2010;153(9):600–6.

    Article  PubMed  Google Scholar 

  5. Curtis LH, Weiner MG, Boudreau DM, Cooper WO, Daniel GW, Nair VP, et al. Design considerations, architecture, and use of the Mini-Sentinel distributed data system. Pharmacoepidemiol Drug Saf. 2012;21(S1):23–31.

    Article  PubMed  Google Scholar 

  6. McGraw D, Rosati K, Evans B. A policy framework for public health uses of electronic health data. Pharmacoepidemiol Drug Saf. 2012;21:18–22. doi:10.1002/pds.2319.

    Article  PubMed  Google Scholar 

  7. Observational Medical Outcomes Partnership. OMOP project website. http://omop.fnih.org. Accessed 15 May 2015.

  8. Ryan PB, Madigan D, Stang PE, Overhage JM, Racoosin JA, Hartzema AG. Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership. Stat Med. 2012;31(30):4401–15. doi:10.1002/sim.5620.

    Article  PubMed  Google Scholar 

  9. FDA Mini-Sentinel Program. Mini-Sentinel project website. http://www.mini-sentinel.org/. Accessed 15 May 2015.

  10. Exploring and Understanding Adverse Drug Reactions (EU-ADR) project. EU-ADR project website. http://euadr-project.org/. Accessed 15 May 2015.

  11. Toh S, Avorn J, D’Agostino RB, Gurwitz JH, Psaty BM, Rothman KJ, et al. Re-using Mini-Sentinel data following rapid assessments of potential safety signals via modular analytic programs. Pharmacoepidemiol Drug Saf. 2013;22(10):1036–45.

    PubMed  Google Scholar 

  12. Overhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Inform Assoc. 2012;19(1):54–60. doi:10.1136/amiajnl-2011-000376.

    Article  PubMed Central  PubMed  Google Scholar 

  13. Zhou X, Murugesan S, Bhullar H, Liu Q, Cai B, Wentworth C, et al. An evaluation of the THIN database in the OMOP common data model for active drug safety surveillance. Drug Saf. 2013;36(2):119–34.

    Article  CAS  PubMed  Google Scholar 

  14. Coloma P, Avillach P, Salvo F, Schuemie M, Ferrajolo C, Pariente A, et al. A reference standard for evaluation of methods for drug safety signal detection using electronic healthcare record databases. Drug Saf. 2013;36(1):13–23. doi:10.1007/s40264-012-0002-x.

    Article  CAS  PubMed  Google Scholar 

  15. Ryan PB, Schuemie MJ, Welebob E, Duke J, Valentine S, Hartzema AG. Defining a reference set to support methodological research in drug safety. Drug Saf. 2013;36(1):33–47.

    Article  Google Scholar 

  16. Schneider G, Kachroo S, Jones N, Crean S, Rotella P, Avetisyan R, et al. A systematic review of validated methods for identifying anaphylaxis, including anaphylactic shock and angioneurotic edema, using administrative and claims data. Pharmacoepidemiol Drug Saf. 2012;21(S1):240–7.

    Article  PubMed  Google Scholar 

  17. Lo Re V, Haynes K, Goldberg D, Forde KA, Carbonari DM, Leidl KB, et al. Validity of diagnostic codes to identify cases of severe acute liver injury in the US Food and Drug Administration’s Mini-Sentinel Distributed Database. Pharmacoepidemiol Drug Saf. 2013;22(8):861–72.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Noren GN, Hopstadius J, Bate A, Edwards IR. Safety surveillance of longitudinal databases: methodological considerations. Pharmacoepidemiol Drug Saf. 2011;20(7):714–7. doi:10.1002/pds.2151.

    Article  PubMed  Google Scholar 

  19. Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512–22.

    Article  PubMed Central  PubMed  Google Scholar 

  20. FDA Mini-Sentinel Program. Prospective Routine Observational Monitoring Program Tools (PROMPT) users guide. http://www.mini-sentinel.org/methods/methods_development/details.aspx?ID=1045. Accessed 10 Jan 2014.

  21. Whitaker HJ, Hocine MN, Farrington CP. The methodology of self-controlled case series studies. Stat Methods Med Res. 2009;18(1):7–26.

    Article  PubMed  Google Scholar 

  22. Madigan D. Implementing study designs in the OMOP environment. OMOP Symposium Day 1, Workshop 5 Nov. 2013. http://omop.org/sites/default/files/OMOP%20Symposium%20Day%201%20Workshop%205Nov2013%20Madigan%20study%20design.pdf. Accessed 11 May 2015.

  23. Gagne JJ, Rassen JA, Walker AM, Glynn RJ, Schneeweiss S. Active safety monitoring of new medical products using electronic healthcare data: selecting alerting rules. Epidemiology. 2012;23(2):238.

    Article  PubMed Central  PubMed  Google Scholar 

  24. Reich CG, Ryan PB, Suchard MA. The impact of drug and outcome prevalence on the feasibility and performance of analytical methods for a risk identification and analysis system. Drug Saf. 2013;36(Suppl 1):S195–204.

    Article  PubMed  Google Scholar 

  25. Ryan PB, Schuemie MJ, Gruber S, Zorych I, Madigan D. Empirical performance of a new user cohort method: lessons for developing a risk identification and analysis system. Drug Saf. 2013;36(Suppl 1):S59–72. doi:10.1007/s40264-013-0099-6.

    Article  PubMed  Google Scholar 

  26. OMOP CDM Specifications version 4.0 and OMOP Standard Vocabulary Specification version 4.4. http://omop.fnih.org. Accessed 30 Oct 2013.

  27. Observational Medical Outcomes Partnership. OMOP methods library. http://archive-omop.fnih.org/MethodsLibrary. Accessed 10 Dec 2014.

Download references

Acknowledgments

This research was conceived, funded, and carried out collaboratively by Humana Inc., Pfizer Inc., and Comprehensive Health Insights, Inc. The research concept was approved by the Joint Research Governance Committee, consisting of Humana Inc. and Pfizer Inc. employees, and plans to publish results which were made known prior to commencing the study. We wish to thank Dr. James Harnett, Mr. Daniel Wiederkehr, and Dr. Robert Reynolds at Pfizer Inc. for their support and advice to this study. Prior to study initiation, the research protocol was reviewed and approved by an independent institutional review board (IRB) (Schulman Associates, study #13-03391).

Conflict of interest

Yihua Xu, Brandon Suehs, Keran Moll, and Margaret Pasquale are employees of Comprehensive Health Insights, a wholly owned subsidiary of Humana. Brandon Suehs is a stockholder of Humana. Vinit Nair is an employee of Comprehensive Health Insights, and serves as the primary investigator from Humana for both the Observational Medical Outcomes Partnership and the Mini-Sentinel program. Xiaofeng Zhou, Qing Liu, and Andrew Bate are employees and stockholders of Pfizer Inc. Abraham Hartzema, Michael Kahn, Brian Sauer, and Yola Moride received consulting fees and travel expenses in connection with providing input on the design of the study and interpretation of results.

Author information

Authors and Affiliations

  1. Comprehensive Health Insights, Humana Inc., 515 W. Market St., Louisville, KY, 40202, USA

    Yihua Xu, Brandon T. Suehs, Keran Moll, Margaret K. Pasquale & Vinit P. Nair

  2. Pfizer Inc., New York, NY, USA

    Xiaofeng Zhou, Qing Liu & Andrew Bate

  3. College of Pharmacy, University of Florida, Gainesville, FL, USA

    Abraham G. Hartzema

  4. Department of Pediatrics, University of Colorado, Denver Anschultz Medical Campus, Aurora, CO, USA

    Michael G. Kahn

  5. Faculty of Pharmacy, Université de Montreal, Montreal, QC, Canada

    Yola Moride

  6. Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, UT, USA

    Brian C. Sauer

Authors
  1. Yihua Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaofeng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brandon T. Suehs
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abraham G. Hartzema
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael G. Kahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yola Moride
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Brian C. Sauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Qing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Keran Moll
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Margaret K. Pasquale
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Vinit P. Nair
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Andrew Bate
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brandon T. Suehs.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 35 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, Y., Zhou, X., Suehs, B.T. et al. A Comparative Assessment of Observational Medical Outcomes Partnership and Mini-Sentinel Common Data Models and Analytics: Implications for Active Drug Safety Surveillance. Drug Saf 38, 749–765 (2015). https://doi.org/10.1007/s40264-015-0297-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40264-015-0297-5

Keywords

Search

Navigation