Introduction

The basis for prescribing decisions is a medical assessment that the expected benefits of a drug exceed the risks for the specific patient, given their health condition. Therefore, the underlying evidence regarding benefits and harms, often summarised in systematic reviews and meta-analyses, is crucial in the process of prescribing. If the benefits are overestimated, patients will unjustifiably be put at risk. If, on the other hand, the risks are overstated, physicians may be guided to refrain from prescribing and the patients may not receive the benefits of treatment. Beneficial effects are generally well established through randomised controlled trials (RCTs), and meta-analyses of such studies are a standard procedure for evidence synthesis. Regarding safety aspects of drugs, not detected in RCTs, evidence often has to be based on non-RCTs using epidemiological methods, i.e. pharmacoepidemiological studies. When such studies are also pooled in meta-analyses, this may imply a considerably greater risk of bias, in particular because of selection bias and confounding by indication. Indeed, drug treatment is not random as physicians and, to a certain degree, patients make active and informed decisions.

As clinical pharmacologists educating health care personnel and contributing as experts in pharmacotherapeutic contexts [1,2,3], we occasionally encounter the belief that fall risk properties of specific cardiovascular drugs are a major concern. Two previous systematic reviews published in 1999 [4] and 2009 [5] may have contributed to this belief, cited in 481 and 638 publications, respectively (Scopus, August 2019). Indeed, there are general recommendations against prescribing cardiovascular drugs because of fall risk properties, for example in sets of indicators of prescribing quality for older patients [6, 7]. A third, and the most recent, systematic review with meta-analyses evaluating cardiovascular drugs as risk factors for falls was published in 2018 [8]. It concluded that loop diuretics are associated with a 36% increased risk of falls and beta-blockers with a 12% decreased risk.

Confounding by indication has recently been reported to be the most prevalent methodological problem in pharmacoepidemiological research [9]. Meta-analyses that led to conclusions regarding associations between cardiovascular drugs and falls provide an opportunity to systematically take a deeper look at methodological and reporting issues, including confounding by indication, in current comparative drug safety research that contributes to prescribing recommendations. We performed the present study with the aim to elucidate these issues and to provide informed suggestions to enhance future research within the field.

Methods

In this study, we reviewed all publications contributing data to statistically significant meta-analyses in the most recent of the systematic reviews investigating cardiovascular drugs as a risk factor for falls, that is, those concerning loop diuretics and beta-blockers [8]. We extracted study characteristics and assessed methodological as well as reporting issues with a focus on aspects related to confounding by indication. We also recorded the impact factor of the scientific journals during the year in question, retrieved from the Journal Citation Index.

One author (S.M.W.) extracted data from the studies, and the other author (M.H.) independently checked these. Data extraction included the study design (retrieved from the meta-analysis [8]), the data sources used, characteristics of the individuals analysed, recording of drug exposure and collection of outcome data as well as information on the methodology used regarding matching and adjustments. Further, the results for loop diuretics and beta-blockers were extracted.

To shed light on issues related to confounding by indication, we recorded if morbidities of compared groups were presented according to the design. If presented, we assessed whether the compared groups differed regarding morbidity. We also determined whether confounding by indication was highlighted in the abstract and in the discussion section and whether causality issues were considered. Finally, we recorded whether clinical advice was provided based on the results and whether there was a discussion on the benefit-risk balance of drug treatment.

The assessments were discussed by the authors in an iterative process where one author provided assessments and suggestions for classifications, and the other checked these and contributed with further information and suggested changes to the classifications. Disparities were resolved by discussion and consensus was reached.

A descriptive analysis of the data was performed. Based on the findings, we summarised suggestions to guide future research within the field.

Results

A total of nine studies were included in the review. In all, 60 individuals authored the publications which were each written by two to 17 authors. The impact factor of the journals in the year in which the studies were published ranged from 2.23 [10] to 6.35 [11].

In Table 1, characteristics of the studies are presented. The studies included 498 to 321,995 individuals, whose data were extracted either from registers (n = 3), from another study (n = 4) or were collected for the purpose of evaluating fall risk properties of drugs (n = 2). The patients came either from a general population sample (n = 2) or were nursing home residents (n = 2), inpatients (n = 1), older people with ability to walk (n = 3) or individuals who had experienced an ischaemic cerebrovascular event (n = 1). The mean age of the patients in the studies ranged between 70 and 84 years.

Table 1 Characteristics of reviewed studies
Full size table

Five studies had a cohort design while the remaining four were case-control studies (Table 2). In the cohort studies, characteristics of exposed vs unexposed individuals were either not reported (n = 3) or differed substantially regarding morbidity (n = 2). In one of the latter studies, age also differed between the comparison groups [15]. In all case-control studies, the controls were matched for age and gender. Cases and controls were matched for morbidity in only one of the case-control studies. Morbidity characteristics between fallers and non-fallers were either not reported (n = 2) or differed (n = 1), or were reported according to the matched-for diseases (n = 1).

Table 2 Methodological and reporting issues related to confounding by indication in studies contributing data to meta-analyses in a recent publication concluding that loop diuretics and beta-blockers are associated with falls
Full size table

In the cohort studies, drug exposure was determined only at baseline, and falls were recorded during a follow-up period of 6 months to 2 years. All studies then applied a cross-sectional analysis. In three of the case-control studies, drug use was estimated from prescribed/dispensed drugs 30–60 days before the index date; in the fourth, the prescribed drugs in the medical records on the day of falling were used for the estimation.

Five studies reported a statistically significant association between loop diuretics and/or beta-blockers and falls (Table 1). In two of these, morbidities in the compared groups were not considered in the analyses. Of the remaining studies, one study included the number of drugs as a covariate in the analyses, a factor that has been used as a proxy for morbidity [19], and two studies adjusted for a number of morbidities and medications used.

In four studies, the abstract conclusion highlighted the found fall risk properties of cardiovascular drugs. None of the conclusions in the abstracts considered causality issues or the possibility of confounding by indication. Confounding by indication was explicitly discussed in two out of nine studies and was vaguely described in another three. None of the concluding paragraphs of the discussions, however, mentioned confounding by indication.

The complexity of prescribing, taking into account the medical assessment of the benefit-risk balance for the specific patient according to the patient’s current health condition, was discussed in one of nine studies [14]. Five studies provided clinical advice for prescribers, three of which highlighted that the identified increased fall risk should be an essential consideration in decision-making.

Based on the findings of this review, seven explicit suggestions were summarised to guide future comparative drug safety research using pharmacoepidemiological methodology (Box 1).

Box 1 Suggestions for enhanced comparative drug safety research using non-randomised observational data, based on findings in this review

1. Make efforts to balance the comparison groups regarding relevant factors including morbidities and concurrent drug treatment

2. Report characteristics of compared groups according to design

3. When a cohort design is stated, ensure that there is a reasonable time relationship between the drug intake and the event; otherwise, state that a cross-sectional design is used

4. Discuss confounding by indication explicitly, including the potential influence of this issue on the reported results

5. Relate safety aspect to expected benefits of the treatment, to ensure that the benefit-risk balance is reflected in the text

6. Refrain from providing explicit advice to prescribers based on single study results, such advice should be based on the compiled evidence

7. Highlight in the abstract conclusion, as well as in the last paragraph of the discussion, that non-randomised groups are compared and that confounding by indication cannot be excluded

Discussion

In this review, we found that studies leading to the conclusion in a current meta-analysis of non-RCTs, that loop diuretics increase and beta-blockers decrease the risk of falls, had major methodological problems. For example, morbidity was not considered at all in two studies and only marginally in the rest. Further, characteristics of the compared groups were often inadequately described, severely limiting the potential for interpretation of the findings. In addition, drug exposure was often not adequately handled; the possibility of confounding by indication was discussed in only a few studies; and causality considerations were not mentioned in any abstract.

The implications of our findings are serious as meta-analyses have an important role in evidence-based decision-making. Indeed, the term meta-analysis is connected to a high level of evidence. However, this is true only if based on RCTs without serious biases. Meta-analyses on non-RCTs, on the other hand, may be controversial. If aggravating issues in non-RCTs are not visible in the pooled results, there may be room for misinterpretations and hasty conclusions. In fact, the study that most influenced the results in the significant meta-analyses of loop diuretics and beta-blockers, because of its large study size, did not consider morbidity in the analyses other than to give, in numerical categories, the number of unique pharmacological subgroups (Anatomic Therapeutic Chemical classification (ATC), level 3 [20]), dispensed over a 120-day period from pharmacies [14]. One may speculate, based on medical common sense, that the results of the meta-analyses could be explained by prescribing practices, with beta-blockers being prescribed to those in better health and loop diuretics to those in worse. We have not reviewed the studies contributing to previous meta-analyses on associations between cardiovascular drugs and falls [4, 5], but they were all based on observational data with risks of confounding by indication.

An important aspect when investigating the effects of drugs is that there needs to be a reasonable time relationship between drug intake and reaction. Although studies with a cohort design are acknowledged to allow conclusions on causality, the potential to learn about fall risk properties of drugs may diminish when, as in this review, exposure is estimated at baseline and information on falls is collected for up to 2 years after baseline. Under these circumstances, it may be more informative to state that a cross-sectional design has been applied, thereby minimising the risk of misinterpreting causality.

Interestingly, three studies advised physicians to consider the fall risk increasing properties of drugs before prescribing. Given that all drugs have benefits and harms and that our review suggests that fall risk may be overstated because of methodological problems, it ought to be emphasised that one-sided attention to harms could contribute to withholding valuable treatment from patients. Still, all but one publication lacked a discussion on the benefit-risk balance of drug treatment. Although safety issues may require a less solid base of evidence, according to the ethical principle not to harm others and also the better safe than sorry principle, providing prescribing advice based on biased evidence implies risks. If a patient on antihypertensive drugs is orthostatic or their blood pressure is too low, it would probably not be controversial to suggest that these drugs should be withdrawn or the dose should be adjusted because the benefit-risk balance would not be expected to be positive for that specific patient. However, when a drug group is said to cause falls without putting this within the context of clinical practice or science, readers may be confused and make hasty conclusions. Indeed, from a medical perspective, it is surprising that none of the studies provided information on blood pressure in the comparison groups.

In a time-pressured medical practice, reading of scientific articles may be limited to reading the abstract conclusion and the summarising paragraph at the end of the discussion. Therefore, our finding that these sections of the studies did not include important aspects for interpretation is somewhat disappointing. Although the limitations may be adequately discussed in some cases, the average reader may miss them. Similarly, readers of systematic reviews may miss cautions raised in the discussion if these are not also highlighted in the abstract conclusion. Indeed, the authors of the meta-analysis appraised in this review acknowledged in the discussion, but not in the conclusion of the abstract, that residual confounding by indication could not be ruled out as this was only partly addressed in the included studies [8]. To further facilitate interpretation, one may also consider requiring information on the type of studies pooled in the title, labelling the study as a non-RCT meta-analysis as opposed to an RCT meta-analysis.

To enhance the standards of comparative drug safety research, we provide suggestions regarding methodological and reporting issues (Box 1). Propensity score matching represents one technique to reduce the risk of confounding. Although one study used propensity scores as a covariate in a sensitivity analysis [15], none of the reviewed studies used this matching technique. According to findings of drug effectiveness studies [21], propensity score matching seems to have undergone limited divergence within the research on drug safety. Further, sensitivity analyses may be a valuable tool in pharmacoepidemiological research. One of the reviewed studies performed a sensitivity analysis in which antidepressants were used as a positive control [13]. As far as we are aware, however, the studies contributing to meta-analyses suggesting fall risk properties of antidepressants have not been reviewed. Therefore, it cannot be excluded that biases may have contributed to the significant findings on that drug group as well. Indeed, the association between antidepressants and fractures, a severe result of a fall, has recently been questioned [22].

Strengths and limitations

The most important strength of this review is that it highlights pertinent methodological and reporting problems within comparative drug safety research based on non-randomised observational data, providing illustrative examples that may contribute to misleading results in meta-analyses [23, 24]. In addition, explicit suggestions for enhanced performance of such research are provided. Another strength of the present study is that it starts off from a recent review of fall risk properties of cardiovascular drugs, thereby reflecting current practice. Furthermore, the assessments were performed by two clinical pharmacologists knowledgeable in pharmacoepidemiology and with experience in evidence-based decision-making, both at the patient and at the societal level.

In the review, we chose to focus on cardiovascular drugs with a statistically significant association with falls, according to the meta-analysis. This may be regarded as a limitation affecting the generalisability of the results. Indeed, it cannot be excluded that the reviewed studies represent the better research in the field as they were all published in established journals with acceptable impact factors. Further, they were assessed to be of high or intermediate quality in the original systematic review [8]. Nevertheless, this approach was sufficient to highlight common pitfalls in comparative drug safety research.

Conclusion

This review illustrates that methodological and reporting issues are prevalent in comparative drug safety studies using non-randomised observational data, especially regarding confounding by indication. Although seemingly clear and informative, meta-analyses of non-RCTs may bear a risk to conceal such shortcomings. Increased awareness of these issues among researchers is crucial to enhance research within the field and caution recommended when results are pooled in meta-analyses. Authors of pharmacoepidemiological studies are encouraged to explicitly discuss and highlight in the abstract methodological limitations such as confounding by indication when non-randomised data are used to investigate outcomes of drug treatment.