Whether and to what extend supplement research makes a difference to consumers is a multi-faceted public health research question that this study only begins to assess. When supplement research results report harm, they may be more influential to consumers and their physicians than results reporting lack of efficacy. Conversely, when research shows supplements to be safe but not necessarily effective, there may be little or no influence in public consumption. The case of vitamin E suggests it is possible for negative evidence revealing adverse effects from clinical research to cause dramatic declines in the consumption of supplements. Between 2004 and 2005 sales dropped dramatically (33%) comparable to declines in hormone replacement therapy after publication of research by the Women’s Health Initiative.11
However, in the case of echinacea and saw palmetto, where the products were shown to be relatively safe but no better than placebo, declines in sales were more modest and consistent with secular sales trends. For glucosamine, publication of mixed outcome results was associated with no appreciable marginal change in product sales. And for St. John’s wort, reports regarding drug interactions predating publication of the large studies in 2001 and 2002, and smaller studies showing efficacy around the same time and subsequently 25,26, may have prevented those studies from having any additional marginal impact on sales trends.
For this analysis we assumed that press exposure was an important mediator by which supplement trials might influence public behavior. The prominent studies we report here varied little in their number of Lexus-Nexis citations associated with their publication. However, vitamin E had the largest number of citations (71) and the highest total of citations (178), which exceeded the total of citations for the other four supplements by 50–300 percent. How the media covers a study and how that coverage does or does not persist could not be captured using our simple methods.
Multiple other factors not measured in this study likely contribute to whether results from clinical research on supplements have an impact. It is important to acknowledge that the number and significance of prior positive studies may influence whether sales after a single publication decline. For instance, if a given supplement has no side-effects and multiple previous studies showing potential benefit, one might expect minimal declines in sales after one negative result. Multiple studies on the same supplement published within months of each other, as was the case in vitamin E, may have promoted a stronger association with sales changes not seen in single studies. Furthermore, consumers of different supplements may vary their response to the evidence based on the specific purpose of the supplement, the type of supplement, and the pharmacological alternatives. The purposes for which vitamin E are taken (prevention) differ from saw palmetto and St. John’s wort (chronic symptoms) or echinacea (acute symptoms). Furthermore, vitamin E may be more likely to be recommended by physicians than the other supplements, making them more concerned about advising their patients in light of the emerging safety data.
The type of supplement may make a difference too. As suggested by Nahin,27 consumers who use vitamin and mineral supplements had differences in their characteristics. They found that vitamin or mineral use was associated with female sex, white race, non-smoking, more years of schooling, difficulty walking, a history of osteoporosis, and reading health magazines, whereas non-vitamin, non-mineral dietary supplement use was primarily associated with residing in California and having difficulty with muscle strength. Accordingly, these different consumer groups may respond differently to new evidence. Finally, even if clear persuasive research were optimally reported and disseminated, consumers would still face persistent challenges in interpreting marketing messages they receive regarding supplements. For instance, in the wake of the glucosamine trial, manufacturers highlighted the secondary outcomes of the trial showing benefit, while excluding the results showing placebo equivalence. Consumers may be exposed to multiple sources of information including alternative medicine publications that may interpret the same data differently. Thus, achieving a public health impact for research on supplements may be particularly challenging in light of their current regulatory status and popular appeal.
Our approach has limitations. Temporal associations from sales trends cannot establish causation. At best our approach represents a quasi-experimental design in which the exposure was research publication. Such an approach threatens the internal validity of making accurate causal inferences related to the exposure.28 We did not directly measure whether media coverage actually reached the consumers of interest, nor did we measure public attitudes toward the research results. And it is difficult to say a priori how much individual studies ought to change supplement consumption. It may not be reasonable to expect every individual study to show demonstrable market impact. In fact, it often takes a long-term clinical research investment in order for the weight of the evidence to reach a compelling “tipping point”, as seems to be the case with vitamin E. Thus, achieving a significant public health impact in supplement research may require a longer, more sustained research time horizon than some had initially anticipated.