Background

Information on medicines is essential to help doctors ensure the safe and optimal use of medicines. Pharmaceutical advertisements in journal advertising are used by pharmaceutical companies to disseminate medicine information to doctors [1]. Medicines information includes product characteristics, marketing claims and references to support claims. Evidence shows that journal advertising often provides biased information [2–4]. It also appears that doctors generally underestimate the impact of pharmaceutical promotion on their prescribing practices [5–7].

In 2004, the World Health Organization reported that 89 (46%) countries regulate pharmaceutical promotion [8]. In most countries, the control of pharmaceutical promotion is overseen by pharmaceutical companies through self-regulation [9, 10]. Self-regulation is intended to complement government legislation on pharmaceutical promotion, where these are available. In some countries regulatory agencies directly regulate pharmaceutical promotion [10].

In Australia and Malaysia, promotion for prescription medicines is jointly regulated through government legislation and pharmaceutical companies' code of conducts [11, 12]. The codes state that all claims must be current, accurate, balanced and not misleading. Claims should be substantiated either by approved labelling or by scientific evidence [11, 12]. Despite these similarities, the regulation of pharmaceutical promotion in these two countries varies in terms of transparency, financial sanctions and monitoring of promotional materials [11, 12]. Unlike Malaysia, in Australia the information on all code breaches and sanctions imposed is publicly available on the pharmaceutical companies organisation's website. The level of financial sanctions in Australia is higher than in Malaysia [11, 12]. The Australian code requires proactive monitoring of a random sample of promotional materials of member companies on a regular and ongoing basis.

In contrast to the Australian code, the Malaysian code does not include any provision for monitoring promotional material [11, 12]. The Malaysian code relies on pharmaceutical companies to establish and maintain appropriate procedures to ensure full compliance with relevant codes and regulations and to review and monitor all of their promotional materials [11]. A unit of the Ministry of Health Malaysia, the Malaysian Advertisements Board (MAB) is authorised by law to control advertisements on medicines with medical and/or health claims [13]. To date the MAB set out standards and scrutinises publications from the print and electronic media only for non-prescription medicines to the public [13].

In the United States (US), the governmental agency, the Food and Drug Administration (FDA) [14–16], is legally mandated to regulate pharmaceutical promotion activities. The FDA requires medicines information provided by pharmaceutical companies in journal advertising to be accurate, balanced, capable of substantiation, not misleading and adhering with applicable laws and regulations. The FDA has responsibility for overseeing materials and activities that promote prescription drugs and for identifying potential violations. The FDA may regulate violations by issuing regulatory letters. The FDA may pursue enforcement action through the Department of Justice for companies that fail to undertake specific actions in response to the regulatory letters. Enforcement action may include stopping the dissemination of materials in breach of regulations and issuing corrections of previously distributed information [14–16].

Despite the control of pharmaceutical promotion in Australia and the US, prior studies in these countries have shown that pharmaceutical companies still provide poor quality marketing claims, references and information on risk results [17–20]. In 2002, Loke et al. [17] reviewed marketing claims in 174 advertisements in six Australian medical journals. Claims were classified as unambiguous, vague, emotive and presenting non-clinical outcomes such as information on half life of medicines and biochemical markers [17]. They found that only 28% of the claims made in the advertisements presented clinical outcomes in an unambiguous way [17].

References cited to substantiate marketing claims were assessed in several studies [21]. In 1994, a review of 127 distinct advertisements in four Australian medical journals found that 15% of the advertisements provided unacceptable references including unpublished company data (data on file), as well as non-English and not easily retrievable references [18]. Nearly a decade later another Australian study noted that 14% of the advertisements were unreferenced and 64% of clinical claims were not supported by randomised clinical trials or evidence from meta-analysis [17]. In the US, a review of 438 unique advertisements from the 1999 issues of 10 American medical journals found that 29% of advertisements contained no references to support marketing claims and 19% of references were data on file [19]. Another US study, reported that only 63% of 109 advertisements provided references to support the claims [20].

The results of clinical trials of medicines can be reported in different ways: absolute risk reduction (ARR), relative risk reduction (RRR) and number needed to treat (NNT) [22]. Several studies have shown that doctors' attitudes in choosing medicines for patients varies according to the presentation of the risk results [23, 24]. Doctors are more likely to recommend medicines when the presentation of benefits are presented as RRR [24]. The presentation of risk results in journal advertising has been studied in a few countries [17, 25–27]. However, only one study [17] has been conducted to examine how quantitative information on benefit and harm was presented in Australian journal advertising. It found that 7% of advertisements explicitly reporting quantitative outcomes provided information as RRR and none presented it as ARR or NNT. In the US, an analysis of 43 data presentations in 33 advertisements that contained quantitative research results published in four medical journals found none of the advertisements provided risk results as NNTs [27]. The availability of information on ARR and RRR was not reported in this study [27].

To our knowledge, no study has assessed the quality of claims, references and the presentation of risk results in medical journal advertising in Malaysia. The most recent studies in Australia and the US were published in 2002 and 2005 respectively. No comparative study has been conducted on the quality of claims, references and the presentation of risk results among these three countries.

This study aimed to compare types of claims, types of references and presentation of risk results in medical journal advertising in Australia, Malaysia and the United States. These countries represent two developed and one emerging country with different regulatory frameworks and resources to control promotional activities.

The major objectives were:

1- to classify types of claims made about benefits or harms outcomes as unambiguous, vague, emotive and non-clinical claims.

2- to compare the availability of references to support claims made about benefits or harms.

3- to examine if risk results were reported as ARR, RRR or NNT.

4 - to compare results for all outcomes across countries.

The minor objectives were:

1- to determine the proportion of claims supported by references.

2- to assess the types of references cited to support claims in journal advertising

3- to examine the research design and level of evidence provided by MEDLINE® references cited to support claims.

Methods

Selection of advertisements

We used a convenience sample of one medical journal from Australia, Malaysia and United States. Journals were selected because of their high circulation amongst general practitioners. We sampled advertisement-rich journals from Australia and the US. We also included one prescribing reference manual published in Malaysia as a preliminary survey showed that fewer advertisements were published in the only Malaysian medical journal available in three established medical school libraries in Malaysia than in Australian and US journals.

The journals selected covered primary care practitioners' publications:

- Australian Family Physician, which is the official journal of the Royal Australian College of General Practitioners (readership = 38,608 with about 28,000 of these being general practitioners) (Jonathon Tremain, personal communication 2009 Feb 02).

- American Family Physician, which is the official clinical journal of the American Academy of Family Physicians (readership = over 188,200) [28].

- MIMS, which is regarded as an official drug reference of the Malaysian Medical Association (MMA) (readership =7000, with about 4200 of these being general practitioners) (Eileen Khoo, personal communication 2009 Feb 03).

- Medical Journal of Malaysia (MJM), which is the only Malaysian medical journal that is subscribed by the three established medical schools in Malaysia, Universiti Sains Malaysia, Universiti Kebangsaan Malaysia and University of Malaya (readership = over 3500, no data are available on general practitioners' readership).

A consecutive sample of 85 unique advertisements for prescription medicines was chosen from the selected publications published between January 2004 to December 2006 from each country. A product advertisement different from other advertisements for the same product in terms of graphic presentation or written content was considered to be one unique advertisement.

We documented the claims made about benefits or harms (e.g. "Add X to a statin to achieve powerful cholesterol reduction"). A claim was defined as any marketing statement or a group of statements related to the same topic (e.g. "Durable control: Provides additive and durable glycaemia control" or "Vast clinical experience: # 1 prescribed statin in Malaysia, more than 48 million patient - years of experience, more than 400 ongoing and completed trials").

Information about the Pharmaceutical Benefits Scheme (PBS) listings appearing in Australian advertisements (e.g. "PBS Information: This product is listed on the PBS as a calcium channel blocker") was excluded because the information was authorised by the Australian government [29]. Any claim that appeared more than once in an advertisement was considered as a single claim. Medicines information provided in product information (Australia and Malaysia) or prescribing information (US) and a quote from product information was not considered as a marketing claim. Product information or prescribing information is a comprehensive document that contains information to ensure appropriate use of a medicine. Unlike marketing claims, it undergoes an extensive review process between the sponsoring companies and government regulatory bodies at the time of market approval [30, 31].

Claims made about benefits or harms were classified as: unambiguous clinical outcome, vague clinical outcome, emotive or immeasurable outcome and non-clinical outcome (Table 1). This classification has been previously used in two other studies of the quality of claims in medical journal advertising [17, 26].

Table 1 Definition of claims [17, 26].
Full size table

We noted whether the claims were supported by references. References cited from journal articles to support claims were retrieved and classified by study design and level of evidence according to a modification of the classification of the National Health and Medical Research Council (NHMRC) guideline of Australia (Table 2) [32].

Table 2 Level of evidence [44].
Full size table

For each claim providing quantitative information, we noted if the risk results were expressed as relative risk reduction (RRR), absolute risk reduction (ARR) or number needed to treat (NNT).

Inter-rater reliability

All data were extracted by one researcher. Three other researchers, a researcher from Australia and a pharmacist and a family medicine specialist from Malaysia independently selected the claims made about benefits or harms and determined the availability of risk results in a randomly selected sample of 30 advertisements from each country.

Kappa tests were undertaken using STATA version 10 to assess the consistency of ratings between observers.

Data Analysis

Data were analysed using SPSS database version 14.0. Chi-square analysis was used to assess differences between countries.

Results

A total of 255 distinct advertisements for 136 pharmaceutical products were included.

Kappa (κ) for inter-rater reliability for the classification of unambiguous, vague, emotive and non-clinical claims was calculated as 0.74 (substantial agreement) (z = 23.7, p < 0.001) [33]. Kappa (κ) for inter-rater reliability for availability of risk results between the researchers was calculated as 0.86 (almost perfect agreement) (z = 34.4, p < 0.05) [33] (Table 3 ).

Table 3 Inter-rater reliability test: Distribution of claims.
Full size table

Claims

There were 829 claims about benefits or harms including 165 in Australia (median 1, range 1 -7), 346 claims in Malaysia (median 4, range 1-11) and 318 in the US (median 3, range 1-9) (Table 4).

Table 4 Examples of marketing claims.
Full size table

The majority of claims were categorised as vague claims (Australia, 46%, Malaysia, 59%, US, 49%) (Table 5). Less than one-third of the claims were categorised as unambiguous claims (Australia 30%, Malaysia 17%, US, 23%) with significantly less unambiguous claims in the Malaysian advertisements compared to Australia and the US (χ2 = 29.4; df = 6, P < 0.001).

Table 5 Type of claims.
Full size table

References

We found that 19% -100% of claims were referenced (Table 6). Where reference, claims were substantiated by between one to 23 references with a median of 1 in Australia and 2 in Malaysia and the US. More references were provided in the Malaysian advertisements (n = 433) compared to the Australian (n = 244) and the US (n = 233) advertisements. Emotive and vague claims were less likely to be referenced by references than non-clinical and unambiguous claims. The most commonly cited references to support claims in all countries were journal articles [see Additional file 1]. Significantly less journal articles were cited to support claims in the US (41%) than in Australia (73%) and Malaysia (72%) (P < 0.001). Significantly more data on file were cited in the US (17%) than in Australia (2%) and Malaysia (4%) (χ2 = 174.4; df = 10, P < 0.001). All types of claims in all countries were usually supported by journal articles catalogued in MEDLINE® [see Additional file 1]. In all countries most references cited were substantiated by evidence obtained from at least one randomized controlled trial. Less than 7% of claims were supported by evidence obtained from a systematic review or meta-analysis.

Table 6 Claims supported by references.
Full size table

Twenty one (21%) advertisements reported risk results. Overall, most advertisements (86%) reported risk results exclusively as RRRs.

Discussion

This study found that the majority of claims made about benefits or harms in journal advertisements in Australia, Malaysia, and the United Sates (US) were considered vague claims. Vague claims are unlikely to help doctors to make informed decisions about the value of the medicines promoted. The paucity of evidence-based information raises concern about the educational value of advertisements.

Unambiguous claims provide clear and precise information of the promoted drugs (e.g "X cures more otitis externa patients than Y" and "Two to three times more patients maintained abstinence vs. placebo in long-and short-term studies, respectively"). These types of claims should be substantiated by scientific evidence as required by the pharmaceutical regulation in the three countries studied [11, 12, 14]. The failure of pharmaceutical companies to provide evidence to support some unambiguous claims in these three countries highlights the need for better control of pharmaceutical promotion with regards to the need for references to support claims.

Vague (e.g " X provides rapid sleep onset") and emotive claims (e.g "A powerful SSRI that's well tolerated") were less likely to be supported with any references. However, 45% to 76% of vague claims were substantiated by references. The use of references may make advertising more credible to readers [34, 35]. Several studies have shown that information from journal advertising is one of the main sources of information for newly marketed medicines [1, 36, 37, 39]. Clear guidelines on the use of references to support marketing claims should be developed.

In contrast with Australia and Malaysia, journal advertising in the US cited fewer journal articles and more data on file (17%). The use of data on file by pharmaceutical companies in the US is consistent with a previous finding [19] that this occurred in 19% of advertisements in 2005. While the use of data on file to substantiate pharmaceutical claims in pharmaceutical promotion is allowed by the FDA [14], data on file is unpublished and has not been independently reviewed. This raises concern about the quality of evidence supporting claims as the validity of data on file may be lower than that of published journal articles [40] and cannot be easily checked by health professionals.

We found that the majority of claims cited journal articles that were retrievable by MEDLINE®. MEDLINE® was chosen because it is widely used by health care professionals and available freely worldwide. However, non-clinical claims in Malaysia were less likely to be substantiated by retrievable journal articles. This is not due to Malaysian journals not being indexed in MEDLINE®. Of the 59 irretrievable journal articles cited in Malaysian journal advertising, only one reference was published in a Malaysian journal. Given the pharmaceutical companies in Malaysia tend to cite references that were not indexed in MEDLINE® or published in Malaysian journals, Malaysian health professionals might not be able to evaluate the cited references to justify the validity of claims. The Pharmaceutical Association of Malaysia (PhAMA) which administers a code of conduct as a guide for the pharmaceutical promotion in Malaysia [11] needs to strengthen its code by providing clearer guidelines on the availability of references to health professionals.

The majority of references in Australia, Malaysia and the US were supported by randomized controlled trial, systematic review or meta-analysis evidence. In Australia, there was an increase in the number of references supported by randomised controlled trials, systematic reviews or meta-analyses (84%) compared with an earlier study (45%) [17]. The use of these references was higher than reported previously in the UK (31%) [41], Finland (11%) [26] and Spain (67%) [42] in 1997, 2004, and 2003 respectively.

While only a few advertisements reported risk results, advertisements with quantitative information in Australia, Malaysia and the US presented risk results exclusively as RRR. This is consistent with the results of previous studies [17, 26]. The emphasis on relative rather than absolute statistics may overstate the benefit of medicines which may lead to irrational prescribing [23, 24]. The reliability of journal advertising would be improved if regulations and codes of conduct included specific requirements with regards to the presentation of quantitative information.

There are limitations to this study. The validity of our results may be limited by the small sample size and the choice of only three countries. We only surveyed advertisements published in three journals and one prescribing reference manual. Pharmaceutical companies and medical journals may have different advertising policies and target audiences. Analysis of the types of medicines by therapeutic groups advertised in these journals was found to differ between countries. Medications for cardiovascular (38/8, 41%) and respiratory diseases (15/85, 17%) were mostly advertised in Australia, cardiovascular (37/85, 44%) and infectious diseases (16/85, 19%) in Malaysia, and neurological (15/85, 17%), infectious (13/85,15%), psychiatry (12/85,14%) and cardiovascular diseases (11/85, 13%) in the US respectively. Our findings may not be generalisable to different journals or countries. However our findings are consistent with two previous studies that reviewed advertisements in four Australian primary care practitioners' publications [43] and 10 multi-disciplinary American medical journals [19]

We did not examine the accuracy of claims and did not assess whether the references adequately substantiated each claim made in the advertisements. We did not attempt to request data on file from pharmaceutical companies. This may represent a limitation as data on file might refer to a randomised controlled trial, systematic review or meta-analysis. The use of a single database to assess the retrievability of journal articles is also a potential limitation. MEDLINE® may not be equally representative of the scope of information that clinicians rely on in each of the countries studied.

Conclusion

Despite the differences in regulation of pharmaceutical promotion in Australia, Malaysia and the US, this study found that the majority of claims presented were vague. The evidence base to support claims appears to be improving with the majority of references cited providing randomized controlled trial, systematic review or meta-analysis evidence. However, the more frequent use of data that have not been published and independently reviewed in the US compared to Australia and Malaysia raises questions on the quality of references in the US. There are concerns that the use of references may make advertising more convincing to readers, even when supporting non-scientific evidence. The use of relative rather than absolute benefits may overemphasize the benefit of medicines which may leave doctors susceptible to misinterpreting information. Methods to further improve the quality of pharmaceutical promotion are still required.