Abstract
Randomized controlled trials (RCTs) are the gold standard for assessing the efficacy of drugs but not necessarily so for drug safety where inadequate power to detect either multiple or rare adverse events is a major handicap. Furthermore, the conditions under which drugs are approved for market use are often different from the settings in actual use. Indeed, with their control mechanisms, trials are by design largely inadequate for the identification of potential safety signals, especially of the rare type, hence the value of post-marketing surveillance and risk management plan-based activities.
Today, clinical trials constitute only a part of the research that goes into assessing the safety of drugs. Observational studies, where the investigators merely collect data on treatments received by patients and their health status in routine clinical practice are increasing in uptake because they reflect the real-life utility of drugs, despite the absence of random treatment assignment. Although such studies generally provide less compelling evidence than RCTs, they can be far more useful to drug safety assessment activities than generally acknowledged.
An increasing number of post-authorization safety studies (PASS) within the European Medicines Agency’s jurisdiction are of the observational type —considered perhaps as more appropriate vehicles for exploring and documenting how products perform in the real world. A similar trend is emerging in the US following the FDA Amendments Act of 2007; since early 2010, an increasing number of post-approval commitments mandated by the FDA include observational studies. However, despite this pattern, not much is known about ongoing efforts to address many of the recognized inadequacies associated with existing methodologies and practices currently adopted in observational PASS. This current opinion presents an overview of some of the main challenges we face in prospective observational PASS, mainly from practical experience, and proposes certain steps for improvement.
Similar content being viewed by others
References
Vandenbroucke JP. What is the best evidence for determining harms of medical treatment? CMAJ 2006; 174(5): 645–6
FDA 2007. Strengthening drug safety: headlines, hoopla, and drug safety [online]. Available from URL: http://www.fda.gov/forconsumers/consumerupdates/ucm107769.htm#safety [Accessed 2012 Jul 20]
Papanikolaou PN, Christidi GD, Ioannidis JPA. Com parison of evidence on harms of medical interventions in randomized and nonrandomized studies. CMAJ 2006; 174(5): 635–41
Ioannidis JP, Lau J. Improving safety reporting from randomized trials. Drug Saf 2002; 25: 77–84
Loke YK, Derry S, Aronson JK. A comparison of three different sources of data in assessing the frequencies of adverse reactions to amiodarone. Br J Clin Pharmacol 2004; 57: 616–21
Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med 2000; 342: 1878–86
Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000; 342: 1887–92
Vandenbroucke JP. When are observational studies as credible as randomized trials? Lancet 2004; 363: 1728–31
Miettinen OS. The need for randomization in the study of intended effects. Stat Med 1983; 2: 267–71
Grootendorst DC, Jager KJ, Zoccali C, et al. Observa tional studies are complementary to randomized controlled trials. Nephron Clin Pract 2010; 114: c173–7
Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996; 312: 1215–8
Ray WA, Stein CM, Daugherty JR, et al. COX-2 selective non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease. Lancet 2002; 360: 1071–3
Zhao SZ, Burke TA, Whelton A, et al. Cost of heart failure among hypertensive users of nonspecific NSAIDs and COX-2-specific inhibitors. Am J Manag Care 2002 Oct; 8 (15 Suppl.): S414–27
Solomon DH, Schneeweiss S, Glynn RJ, et al. Relationship between selective cyclooxygenase-2 inhibitors and acute myocardial infarction in older adults. Circulation 2004 May 4; 109(17): 2068–73
Kimmel SE, Berlin JA, Reilly M, et al. Patients exposed to rofecoxib and celecoxib have different odds of non-fatal myocardial infarction. Ann Intern Med 2005 Feb 1; 142(3): 157–64
Graham DJ, Campen D, Hui R, et al. Risk of acute myocardial infarction and sudden cardiac death in patients treated with cyclo-oxygenase 2 selective and non-selective non-steroidal anti-inflammatory drugs: nested case-control study. Lancet 2005 Feb 5-11; 365(9458): 475–81
Johnsen SP, Larsson H, Tarone RE, et al. Risk of hospitalization for myocardial infarction among users of rofecoxib, celecoxib, and other NSAIDs: a population-based case-control study. Arch Intern Med 2005 May 9; 165(9): 978–84
Motsko SP, Rascati KL, Busti AJ, et al. Temporal relationship between use of NSAIDs, including selective COX-2 inhibitors, and cardiovascular risk. Drug Saf 2006; 29(7): 621–32
Gislason GH, Jacobsen S, Rasmussen JN, et al. Risk of death or reinfarction associated with the use of selective cyclooxygenase-2 inhibitors and nonselective non-steroidal antiinflammatory drugs after acute myocardial infarction. Circulation 2006 Jun 27; 113(25): 2906–13
Hernández-Díaz S, Varas-Lorenzo C, García Rodríguez LA. Non-steroidal antiinflammatory drugs and the risk of acute myocardial infarction. Basic Clin Pharmacol Toxicol 2006 Mar; 98(3): 266–74
Huang WF, Hsiao FY, Tsai YW, et al. Cardiovascular events associated with long-term use of celecoxib, rofecoxib and meloxicam in Taiwan: an observational study. Drug Saf 2006; 29(3): 261–72
Rahme E, Watson DJ, Kong SX, et al. Association between nonnaproxen NSAIDs, COX-2 inhibitors and hospitaliz ation for acute myocardial infarction among the elderly: a retrospective cohort study. Pharmacoepidemiol Drug Saf 2007 May; 16(5): 493–503
Cunnington M, Webb D, Qizilbash N, et al. Risk of ischaemic cardiovascular events from selective cyclooxy genase-2 inhibitors in osteoarthritis. Pharmacoepidemiol Drug Saf 2008 Jun; 17(6): 601–8
Mangano DT, Tudor IC, Dietzel C. The risk associated with aprotinin in cardiac surgery. N Engl J Med 2006; 354: 353–65
Karkouti K, Beattie WS, Dattilo KM, et al. A propensity score case-control comparison of aprotinin and tranexamic acid in high transfusion- risk cardiac surgery. Trans fusion 2006; 46: 327–38
Directive 2001/83/EC of the European parliament and of the council of 6 November 2001 on the community code relating to medicinal products for human use [online]. Available from URL: http://www.emea.europa.eu/docs/en_GB/document_library/Regulatory_and_procedural_guideline/2009/10/WC500004481.pdf [Accessed 2012 Jul 16]
Volume 9A of the rules governing medicinal products in the European Union: guidelines on pharmacovigilance for medicinal products for human use. September 2008 [on line]. Available from URL: http://ec.europa.eu/health/files/eudralex/vol-9/pdf/vol9a_09-2008_en.pdf [Accessed 2012 Jul 16]
Twachtman G. FDA’s post-market pharmacoepidemiologic study guidance is 3 years away. Pink Sheet 2008; 70(019): 21
US FDA. Guidance for Industry. Postmarketing studies and clinical trials: implementation of section 505(o)(3) of the Federal Food, Drug, and Cosmetic Act [online]. Available from URL: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM172001.pdf [Accessed 2012 Jul 16]
Farmer R, Williams T, Nightingale A. Pitfalls of pharmacoepidemiology [letter]. BMJ 2000; 321: 1352
Skegg DCG. Pitfalls of pharmacoepidemiology: oral contraceptive studies show a need for caution with databases. BMJ 2000; 321: 1171–2
Kaye JA, Vasilakis C, Jick SS, et al. Pitfalls of pharmacoepidemiology [letter]. BMJ 2000; 321: 1528–9
Farmer RDT, Williams TJ, Simpson EL, et al. Effect of 1995 pill scare on rates of venous thromboembolism among women taking combined oral contraceptives: analysis of General Practice Research Database. BMJ 2000; 321:477–9
Skegg DCG. Third generation oral contraceptives. BMJ 2000; 321: 190–1
Jick H, Jick SS, Gurewich V, et al. Risk of idiopathic cardiovascular death and nonfatal venous thromboem bolism in women using oral contraceptives with differing progestagen components. Lancet 1995; 346: 1589–93
Walker AM. Newer oral contraceptives and the risk of venous thromboembolism. Contraception 1998; 57: 169–81
Spitzer WO, Lewis MA, Heinemann LAJ, et al. Third generation oral contraceptives and risk of venous throm-boembolic disorders: an international case-control study. BMJ 1996; 312: 83–8
Lewis MA, Heinemann LAJ, MacRae KD, et al. The increased risk of venous thromboembolism and the use of third generation progestogens: role of bias in observa tional research. Contraception 1996; 54: 5–13
Farley TMM, Meirik O, Collins J. Cardiovascular disease and combined oral contraceptives: reviewing the evidence and balancing the risks. Hum Reprod Update 1999; 5:721–35
Suissa S, Blais L, Spitzer WO, et al. First-time use of newer oral contraceptives and the risk of venous thromboem bolism. Contraception 1997; 56: 141–6
Lewis MA, MacRae KD, Kuhl-Habich D, et al. The differential risk of oral contraceptives: the impact of full exposure history. Hum Reprod 1999; 14: 1493–9
Suissa S, Spitzer WO, Rainville B, et al. Recurrent use of newer oral contraceptives and the risk of venous throm boembolism. Hum Reprod 2000; 15: 817–21
Farmer RDT, Lawrenson RA, Todd J-C, et al. Oral contraceptives and venous thromboembolic disease: analyses of the UK General Practice Research Database and the MediPlus Database. Hum Reprod Update 1999; 5: 688–706
Vasilakis C, Jick SS, Jick H. The risk of venous thromboembolism in users of postcoital contraceptive pills. Contraception 1999; 59: 79–83
Vandenbroucke JP, Helmerhorst FM, Rosendaal FR. Competing interests and controversy about third generation oral contraceptives. BMJ 2000; 320: 381–2
Jick H, Kaye JA, Vasilakis-Scaramozza C, et al. Risk of venous thromboembolism among users of third generation oral contraceptives compared with users of oral contraceptives with levonorgestrel before and after 1995: cohort and case-control analysis. BMJ 2000; 321: 1190–5
Suissa S. Effectiveness of inhaled corticosteroids in chronic obstructive pulmonary disease: immortal time bias in observational studies. Am J Respir Crit Care Med 2003; 168(1): 49–53
Suissa S. Inhaled steroids and mortality in COPD: bias from unaccounted immortal time. Eur Respir J 2004; 23: 391–5
Kiri VA, Vestbo J, Pride NB, et al. Inhaled steroids and mortality in COPD: bias from unaccounted immortal time. Eur Respir J 2004; 24: 190–1
Kiri VA, Pride NB, Soriano JB, et al. Inhaled corticosteroids in chronic obstructive pulmonary disease: results from two observational designs free of immortal time bias. Am J Respir Crit Care Med 2005; 172: 460–4
Suissa S. Observational studies of inhaled corticosteroids in chronic obstructive pulmonary disease: misconstrued immortal time bias [letter]. Am J Respir Crit Care Med 2006; 173: 464
Kiri VA, Pride NB, Soriano JB, et al. Observational studies of inhaled corticosteroids in chronic obstructive pulmonary disease: misconstrued immortal time bias [reply]. Am J Respir Crit Care Med 2006; 173: 464a–465
Regulation (EU) No 1235/2010 of the European parliament and of the council, amending, as regards pharmacovigilance of medicinal products for human use, Regulation (EC) No 726/2004 laying down Community procedures for the au thorisation and supervision of medicinal products for human and veterinary use and establishing a European Medicines Agency, and Regulation (EC) No 1394/2007 on advanced therapy medicinal products [online]. Available from URL: http://ec.europa.eu/health/files/eudralex/vol-1/reg_2010_1235/reg_2010_1235_en.pdf [Accessed 2012 Jul 16]
Giezen TJ, Mantel-Teeuwisse AK, Straus SMJM, et al. Evaluation of post-authorization afety studies in the first cohort of EU risk management plans at time of regulatory approval. Drug Saf 2009; 32(12): 1175–87
EMA. Annex C: template for EU risk management plan (EU-RMP). EMEA/192632/2006 [online]. Available from URL: http://eudravigilance.ema.europa.eu/human/docs/19263206en.pdf [Acessed 2012 Jul 16]
Greenland A. The effect of misclassification in the presence of covariates. Am J Epidemiol 1980; 112(4): 564–9
Soriano JB, Maier WC, Visick G, et al. Validation of general practitioner-diagnosed COPD in the UK General Practice Research Database. Eur J Epidemiol 2001; 17: 1075–80
Rawson NS, Malcolm E. Validity of the recording of ischaemic heart disease and chronic obstructive pulmonary disease in the Saskatchewan health care data files. Stat Med 1995; 14: 2627–43
Lanes SF, de Luise C. Bias due to false-positive diagnoses in an automated health insurance claims database. Drug Saf 2006; 29(11): 1069–75
Bross I. Misclassification in 2×2 tables. Biometrics 1954; 10: 478–86
Bueil P, Dunn JE. The dilution effect of misclassification. Am J Public Health 1964; 54: 598–602
Copeland KT, Checkoway H, McMichael AJ, et al. Bias due to misclassification in the estimation of relative risk. Am J Epidemiol 1977; 105: 488–95
Keys A, Kihlberg JK. Effects of misclassification on estimated relative prevalence of a characteristic. Part 1: two populations infallibly distinguished. Part II: errors in two variables. Am J Public Health 1963; 53: 1656–65
Newell DJ. Errors in the interpretation of errors in epidemiology. Am J Public Health 1962; 52: 1925–8
Gullen WH, Bearmnn JE, Johnson EA. Effect of misclassification in epidemiologic studies. Public Health Rep 1968; 83: 914–8
Stampfer MJ. ITT for observational data: worst of both worlds? Epidemiology 1995; 6: 248–53
Miettenen OS, Caro JJ. Principles of nonexperimental assessment of excess risk, with special reference to adverse drug reactions. J Clin Epidemiol 1989; 42(4): 325–31
Guess HA. Behavior of the exposure odds ratio in a case-control study when the hazard function is not constant over time. J Clin Epidemiol 1989; 42(12): 1179–84
Kiri VA, McKenzie G. How real is intention-to-treat (ITT) analysis in non-interventional PASS? We can do better. Curr Drug Saf 2009; 4(2): 137–42
Kiri VA, Oyee J. Assessing the time-dependency nature of comorbidity influence in COPD [abstract]. Pharmacoepidemiol Drug Saf 2006; 15 Suppl. 2: S8
Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987; 40(5): 373–83
Kiri VA, Muellerova H, Visick G, et al. A novel metho dology for measuring the influence of comorbidity in disease outcome studies [abstract]. Pharmacoepidemiol Drug Saf 2005; 14 Suppl. 2: S135
Kiri VA, Muellerova H, Visick G, et al. Assessing the multivariate structure and influence of comorbidity in COPD [abstract]. Eur Respir J 2005; 26 Suppl. 49: A470s
Horwitz RI, Feinstein AR. The problem of protopathic bias in case-control studies. Am J Med 1980 Feb; 68: 255–8
Salas M, Hofman A, Stricker BH. Confounding by in dication: an example of variation in the use of epidemiologic terminology. Am J Epidemiol 1999; 149: 981–3
Clayton D, Hills M. Statistical methods in epidemiology. Oxford: Oxford University Press, 1993: 309
Rothman KJ, Greenland S, editors. Modern epidemiology. 2nd ed. Philadelphia (PA): Lippincott-Raven, 1998: 87
Strom BL, ed. Pharmacoepidemiology. 4th ed. West Sussex: John Wiley & Sons Ltd, 2005
Tamim H, Tahami Monfared AA, Le Lorier J. Application of lag-time into exposure definitions to control for protopathic bias. Pharmacoepidemiol Drug Saf 2007; 16(3): 250–8
Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for casual effects. Biometrika 1983; 70: 41–55
Rosenbaum PR, Rubin DB. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc 1994; 79(387): 516–24
D’Agostino Jr RB. Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat Med 1998; 17: 2265–81
Stürmer T, Schneeweiss S, Rothman KJ, et al. Performance of propensity score calibration: a simulation study. Am J Epidemiol 2007; 165(10): 1110–8
Stürmer T, Schneeweiss S, Rothman KJ, et al. Propensity score calibration and its alternatives. Am J Epidemiol 2007; 165(10): 1122–3
Greenland S. An introduction to instrumental variables for epidemiologists. Int J Epidemiol 2000; 29: 722–9
Stürmer T, Schneeweiss S, Avorn J, et al. Adjusting effect estimates for unmeasured confounding with validation data using propensity score calibration. Am J Epidemiol 2005; 162: 279–89
Arbogast PG, Ray WA. Use of disease risk scores in pharmacoepidemiologic studies. Stat Methods Med Res 2009; 18(1): 67–80
Heckman JJ, Vytlacil EJ. Local instrumental variables and latent variable models for identifying and bounding treatment effects. Proc Natl Acad Sci U S A 1999; 96: 4730–4
Crown WH, Obenchain RL, Englehart L, et al. The appli cation of sample selection models to outcomes research: the case of evaluating the effects of antidepressants ther apy on resource utilization. Stat Med 1998; 17: 1943–58
Shelton BJ, Gilbery GH, Lu Z, et al. Comparing longitudinal binary outcomes in an observational oral health study. Stat Med 2003; 22(12): 2057–7
Volume 9A of the rules governing medicinal products in the European Union: guidelines on pharmacovigilance for medicinal products for human use. March 2007 [online]. Available from URL: http://eudravigilance.ema.europa.eu/human/docs/vol9A_2007-04.pdf [Accessed 2012 Jul 16]
ISPE. Guidelines for good pharmacoepidemiology practices (GPP). Pharmacoepidemiol Drug Saf 2008; 17: 200–8
Gliklich RE, Dreyer NA, editors. Registries for evaluating patient outcomes: a user’s guide. (prepared by Outcome DEcIDE Center [Outcome Sciences, Inc. dba Outcome] under contract no. HHSA29020050035I TO1.) AHRQ Publication No. 07-EHC001-1. Rockville (MD): Agency for Healthcare Research and Quality, 2007 Apr
Dreyer NA, Schneeweiss S, McNeil BJ, et al. GRACE principles: recognizing high-quality observational studies of comparative effectiveness. Am J Man Care 2010; 16(6): 21–4
Berger ML, Mamdani M, Atkins D, et al. Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report — Part I. Value Health 2009: 1044-72 [online]. Available from URL: http://www.is-por.org/taskforces/documents/RDPartI.pdf [Accessed 2012 Jul 20]
Vandenbroucke JP, von Elm E, Altman DG. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Epidemiology 2007; 18(6): 805–35
EMA. ICH E2D: post approval safety data management. CPMP/ICH/3945/03. London, 20 November 2003 [online]. Available from URL: http://eudravigilance.ema.europa.eu/human/docs/ICH%20E2D.pdf [Accessed 2012 Jul 16]
European Federation of Pharmaceutical Industries and Associations. EFPIA code on the promotion of prescription only medicines to, and interactions with, healthcare professionals [online]. Available from URL: http://www.efpia.eu/code-conduct [Accessed 2012 Jul 20]
Council of Europe. Convention for the protection of human rights and dignity of the human being with regard to the application of biology and medicine: convention on human rights and biomedicine. Oviedo 4.IV. 1997 [online]. Available from URL: http://conventions.coe.int/Treaty/en/Treaties/Html/164.htm [Accessed 2012 Jul 20]
CIOMS/WHO. International ethical guidelines for epidemiological studies. Geneva: WHO, 2008 [online]. Avail able from URL: http://www.ufrgs.br/bioetica/cioms2008.pdf [Accessed 2012 Jul 20]
Tantsyura V, Grimes I, Mitchel J, et al. Risk-based source data verification: pros and cons. Drug Inf J 2010; 44: 745–56
Hughes MD, Williams PL. Challenges in using observa tional studies to evaluate adverse effects of treatment. N Engl J Med 2007; 356(17): 1705–7
Smeeth L, Douglas I, Hubbard R. Commentary: we still need observational studies of drugs — they just need to be better. Int J Epidemiol 2006; 35: 1310–1
Acknowledgements
I am grateful to Professor Gilbert MacKenzie and Dr Maurille Feudjo-Tepie for their encouragement and material support. I am also indebted to the editor and anonymous reviewers for their useful suggestions, which have helped to transform the original draft into a much improved piece of work — one that is humbly dedicated to the memory of my former colleague, Dr George Visick.
No sources of funding were used to assist in the preparation of this current opinion. Dr Kiri was an employee of GlaxoSmithKline Research and Development between November 1999 and April 2007, and Parexel International between May 2007 and June 2011.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kiri, V.A. A Pathway to Improved Prospective Observational Post-Authorization Safety Studies. Drug Saf 35, 711–724 (2012). https://doi.org/10.1007/BF03261968
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF03261968