Introduction

The use of sports nutrition supplements, to improve performance or results in sports or physical fitness [1], is extensive [2] and has been linked with adverse events (AEs). Over the last decade, several case reports have been published linking sports nutrition supplements to severe AEs, for example involving the stimulant substances Ephedra, 1,3-dimethylamylamine (DMAA) and beta-methylphenylethylamine (BMPEA) [1, 3, 4]. Dietary supplements were estimated to contribute to 23,000 emergency department visits every year in the United States (US), a figure which may be an underestimation [5, 6]. Sports nutrition supplements in turn, have been estimated to constitute 13.8% of all dietary supplements [1].

Both in the US and the European Union (EU), the marketing company is responsible for safety issues related to dietary supplements including sports nutrition supplements [7,8,9]. In contrast to pharmaceutical drugs which are subjected to an elaborate approval process before market access, post-market surveillance is the main source of information regarding safety for supplements, primarily through spontaneous reporting of AEs and published case reports. For manufacturers in the US, reporting of AEs to the Food and Drug Administration (FDA) is mandatory only for those defined as serious [10]. In the EU, there are no harmonized post-market regulations regarding the reporting of AEs for dietary supplements; national legislation applies with no requirement of reporting to the European Food Safety Authority (EFSA) or any other authority [11].

Spontaneous reports, from health care, consumers and dietary supplement manufacturers, have limitations and often lack information required to assess causality [12,13,14]. It is not known to what extent such essential information is included in published case reports where a sports nutrition supplement has been associated with an AE. In addition, we have not found any studies reporting how causality is assessed and reported in published case reports; and to what extent systematic causality assessment methods are used. Such methods are essential for signal detection and evaluation in pharmacovigilance based on spontaneous reporting of AEs. Indeed, there are several systematic methods available, none being universally accepted [15]. Causality can also be assessment through ad hoc clinical investigation of alternative causes of an AE without using a systematic causality assessment method. The term clinical reasoning has previously been described for this type of causality assessment of drug-related AEs, and relates to the process of clinically evaluating the probability of causality, and the exclusion of alternative causes by the means of diagnostic tools [16]. The term clinical reasoning per se is defined according to Taber’s Cyclopedic Medical Dictionary as “The use of a patient’s history, physical signs, symptoms, laboratory data, and radiological images to arrive at a diagnosis and formulate plan for treatment” [17].

The aim of this meta-epidemiological study was to review published case reports regarding sports nutrition supplements and AEs, focusing on the presented data, causality assessment through clinical reasoning and the application of systematic causality assessment methods.

Materials and methods

To investigate the reporting patterns and the use of clinical reasoning and/or systematic causality assessment methods, we first conducted a systematic literature search in Cochrane, Embase and PubMed to find all published case reports, written in English and published between 1 January 2008 and 1 March 2019, where a sports nutrition supplement had been associated with an AE. The search was defined in a PICO (Patients, Intervention, Comparison, Outcome) statement. Patients (P) were individuals at any age; intervention (I) was sports nutrition supplements; comparison (C) was not applicable as this review focused on case reports; and Outcome (O) was any AE. Search strategies are provided in Online Resource 1.

Identified abstracts were screened by one author (R.Z.), and those that did not meet the PICO criteria were excluded. In case of uncertainties, the abstract was assessed by an additional author, followed by a consensus decision. If there were still uncertainties, the full-text article was retrieved and independently assessed by two authors, followed by a consensus discussion and a joint decision on inclusion/exclusion. For publications excluded after full-text reading, reason for exclusion was recorded. Data from the included studies were independently extracted by two authors, and potential discrepancies resolved in consensus. Data included age and sex of the patient, the sports nutrition supplement/s/ at issue, and the suspected AE/s/. We also recorded whether clinical reasoning was presented, i.e. an any investigation of alternative causes of the AE other than the sports nutrition supplement. Clinical reasoning was defined as ≥ 1 differential diagnosis mentioned and excluded by means of diagnostic measures and did not require any other specific reported factors. To further investigate the causality assessment in case reports with clinical reasoning, we recorded whether a systematic causality assessment method had been applied or not, defined as any specific method referred to by the case report authors.

To investigate the reporting of patient characteristics of relevance for causality assessment, we recorded previous health condition, as well as the use of medications, alcohol, tobacco smoking, illicit drugs and other supplements. We also recorded the reporting of AE and sports nutrition supplement characteristics of relevance for the causality assessment, including the time relationship between intake and reaction; the response to withdrawal and re-challenge; previous cases; a potential theoretic rationale; body fluid and supplement analysis and the presence of adulteration.

Statistics

Descriptive analyses were performed using SPSS (IBM) software version 26. We used the Chi Square test to compare the reporting of characteristics of importance for the causality assessment according to the presentation of clinical reasoning (yes or no). In the subgroup of case reports that presented clinical reasoning, the same comparisons were made according to the application of a systematic causality assessment method (yes or no). The significance level was set at an alpha of 0.05.

Results

After removal of duplicates, the literature search identified 277 unique publications, 72 of which fulfilled our PICO and were included in the review (Fig. 1, Online Resource 2). Exclusions after full-text reading are described in Online Resource 3. The included case reports concerned 97 patients (82 male, 13 female, 2 not specified), with a median age of 30 years (range: 14–60). Characteristics are described in Table 1.

Fig. 1
figure 1

Flowchart of the study selection

Full size image
Table 1 Characteristics of reported cases (n = 97)
Full size table

A total of 134 different sports nutrition supplements were suspected to have caused 37 different AEs. The supplements consisted of pre-workout supplements (PWO) (n = 35), unspecified blends (n = 27), proteins (n = 22), anabolic steroids (n = 21), creatine (n = 8), hormones (n = 7), amino acids (n = 6), herbals (n = 4), unspecified energy drinks (n = 2) or were not categorizable (n = 2). The supplements were analyzed in four cases and adulteration was demonstrated in two of these (BMPEA adulteration and arsenic contamination). The most frequently reported AEs were hepatotoxicity (n = 14), acne vulgaris (n = 10), rhabdomyolysis (n = 9), acute renal injury (n = 7), acute myocardial infarction (n = 6), cardiac arrest (n = 6), acute psychosis (n = 5), and cerebral hemorrhage (n = 5). The suspected substances for the most frequently reported adverse events are presented in Table 2, and fatal cases in Table 3.

Table 2 Suspected substances for the most frequently reported adverse events
Full size table

In 61 (63%) cases, clinical reasoning was presented. In the remaining 36 (37%) cases, no alternative cause of the AE/s/ other than the sports nutrition supplement was mentioned (n = 22), or an alternative cause was mentioned but without clinical reasoning (n = 14). In five of the six fatal cases, clinical reasoning was presented. When comparing cases with versus without clinical reasoning, it was more common among the former to present alcohol use, information regarding previous cases and body fluid analysis (Table 4). Analyses of body fluid was performed in five of the six fatal cases. In cases with clinical reasoning, there was also a trend towards more frequent reporting of a potential theoretic rationale and concurrent use of medications and/or other supplements.

Table 3 Fatal cases
Full size table

In 8 (13%) cases with clinical reasoning, a systematic causality assessment method was applied. The Naranjo Adverse Drug Reaction Probability scale [18] was used in 3 cases [19,20,21], the CIOMS/RUCAM (Council for International Organizations of Medical Sciences/Roussel Uclaf Causality Assessment Method) [22, 23] in 2 cases [24, 25], the WHO/UMC (World Health Organization-Uppsala Monitoring Center) causality assessment system [26] in 1 case [4], and the Teschke scale [27] in 1 case [28]. In 1 case [29], both the WHO/UMC causality assessment system and CIOMS/RUCAM were used. In none of the six fatal cases, a systematic causality assessment method was applied Among the case reports using clinical reasoning to support a relationship between the sports nutrition supplement and the AE, it was more common to report use of medication, alcohol consumption and illicit drug use when a systematic causality assessment method was applied, with a trend towards presenting previous health more often (Table 5). There was also a trend towards using systematic causality assessment methods more often for hepatopancreatic AEs compared to AEs from other organ systems (n = 4 [50%] versus n = 11 [21%], P = 0.073).

Table 4 Causality assessments factors, according to the use of clinical reasoning
Full size table
Table 5 Causality assessments factors, according to the use systematic causality assessment methods
Full size table

Discussion

Three out of ten published cases, where a sports nutrition supplement had been suspected to have caused an AE, lacked information regarding previous health of the affected individual. In addition, almost half of the cases lacked information regarding the use of regular prescription drugs. In almost two thirds of the cases, clinical reasoning was reported, that is, the negation of alternative causes by means of investigations. However, in about one fifth, an alternative cause of the AE was not mentioned at all. Our findings illustrate that case reports of events suspected to be related to sports nutrition supplements could be improved, to increase their value for surveillance and signal detection. Clinical reasoning could be presented, and systematic causality assessment methods could be applied, to a greater extent.

In cases including clinical reasoning, several factors of relevance for the causality assessment were reported to a greater extent compared with cases without such reasoning. It is important to note that our definition of clinical reasoning was entirely independent of the specific factors presented as relevant for causality assessment. Clinical reasoning captured a traditional approach to assessing causality when any substance is suspected to have caused a reaction or disease, whereas the factors of relevance for causality assessment were specific and pre-defined variables. Among the 61 cases with clinical reasoning, a systematic causality assessment method was applied in 8 cases. With the use of such a systematic method, several additional factors relevant for the causality assessment were reported to an increased extent. Further, most factors relevant for a causality assessment were numerically higher in cases where a systematic causality assessment had been applied, although not reaching statistical significance. Lack of power may be an issue in these cases, and a larger sample size would have been needed to elucidate these differences further. That said, our results indicate that the use of a systematic causality assessments may be superior to not using such a method, and may be superior to traditional ad hoc differential diagnostics.

The most common systematic causality assessment method in our review was the Naranjo score. The literature is scarce regarding the use of such methods to assess AEs suspected for dietary supplements. One study compared the agreement between different assessors when analyzing spontaneous reports from consumers using modified versions of the Naranjo score and an algorithm from the FDA, with more than substantial agreement, however adding little information about the validity of the causality assessments per se [30].

In our study, there was a trend towards an increased use of systematic causality assessment methods in case reports of hepatopancreatic AEs compared to AEs from other organ systems. This finding could be attributable to the fact that the CIOMS/RUCAM-score was initially developed to assess if liver damage was associated with certain pharmaceutical drugs [15], and the Naranjo score, although developed for all drug reactions, is widely used for drug-induced liver injury [18, 31,32,33].

We found a substantial variation in both the severity of AEs and the organ systems being affected. It is important to note that the summarized AEs presented in this study does not reflect the risks associated with sports nutrition supplements, but only the most reported AEs. We also found a wide variety of sports nutrition supplements being suspected to have caused an AE. For many AEs, more than one substance in a single supplement, or multiple supplements in combination, were suspected. Since the theoretic rationale supporting an association between a supplement and an AE is mainly applicable at the substance level, the specific substances may be more important than the specific supplement. Although our compilation of reported cases does not reflect the hazard of certain substances, it can serve as a basis for physicians when an AE is suspected, potentially contributing to relevant investigations in the specific patient.

An aspect that complicates causality assessments between a sports nutrition supplement and a suspected AE is the risk of adulteration, i.e. the “spiking” of products with unlabeled synthetic substances [4, 34]. In our study, adulteration was identified in two cases. However, the supplements were analyzed in only four cases, leaving 93 uninvestigated cases. Finally, it is important to point out that the reported fatal cases do not reflect the mortality associated with sports nutrition supplements, but merely describes fatal cases that have been reported, including the specific details.

Published case reports hold advantages compared with spontaneous reports. Although this review shows that there may be room for improvements regarding the reporting of issues of importance for causality assessments, detailed information is often available regarding why causality was suspected, investigations made, and how the AE was managed and treated. This may partly be attributed to the fact that the authors often are the treating physicians [12, 35, 36]. In spontaneous reports, on the other hand, available information, sometimes provided by health professionals and sometimes by the public, rarely suffices to determine more than a possible causal relationship [37], implying that the event could just as well have been caused by an emerging or worsening disease. Other strengths with published case reports are that these articles undergo peer review and are accessible through e.g. PubMed. They also often include reviews of previous cases which provides educational insights which can trigger others to report similar events [35].

A thorough investigation or argumentation of alternative causes other than the sports nutrition supplement is essential; confirmation bias and the inability to validate substances related to the AEs are often put forward as main limitations of case reports in general [12, 13]. However, case reports can generate hypotheses that can be further evaluated. In a study of 83 drug withdrawals due to published case reports of potentially fatal drug-related AEs, confirmatory studies had been conducted in 57 cases where evidence of an association was found in 52 [38]. Though generally ranked lowest on the evidentiary hierarchy, published case reports play a vital role in drug safety. Indeed, given the limited availability of evidence regarding effects of sports nutrition supplements, such reports may constitute the main source for signal detection.

In the present review, the value of systematic causality assessment methods as add-on to clinical reasoning was investigated. With the inverse approach, i.e., clinical reasoning as add-on to systematic causality assessment methods, some patient/substance characteristics appear that would perhaps not be captured if the latter methods are used alone. Indeed, smoking is not included in any of the systematic tools used in the studied case reports. Further, use of illicit drugs is included only in the Teschke scale [27]. Interestingly, none of the systematic causality assessment tools include an analysis of the contents of the product. Consequently, adulteration would not be captured if strictly following the protocol. As smoking, use of illicit drugs, and adulteration are important factors when assessing causality between a sports nutrition supplement and an event, inclusion of these aspects in standardized tools could be considered. The main strengths of this meta-epidemiological study are the comprehensive data collection; the systematic literature search and study selection; and the novel aspect of investigating causality assessment for AEs related to sports nutrition supplements. The small number of case reports where a systematic causality assessment had been made is, however, a limitation, with a risk of statistical type II errors. Consequently, non-significant differences between these groups have to be interpreted with caution.

In conclusion, a non-negligible proportion of published case reports, where sports nutrition supplements have been suspected to have caused AEs, leave out essential factors for the causality assessment. Clinical reasoning could be identified in most cases but a systematic causality assessment method was applied in a minority. Factors relevant for causality assessment were provided to a greater extent in cases with clinical reasoning. Among these, case reports applying a systematic causality assessment method were less likely to leave out additional such information. An increased use of systematic causality tools in future case reports may increase their value for the surveillance of these supplements.