Background

Stroke is a major cause of death and disability in both developed and developing countries worldwide. Thrombolysis with intravenous recombinant tissue-type plasminogen activator therapy remains the only proven effective pharmacological treatment for selected acute ischemic stroke patients within a relatively short therapeutic time window of 3 to 4.5 h after the onset of stroke symptoms [1]. Furthermore, the major risk of intravenous thrombolysis treatment also remains the symptomatic intracranial hemorrhage, which is a devastating complication with high mortality. What’s more, the enormous morbidities of ischemic stroke result from the interplay between the resulting neurological impairment, the emotional and social consequences of that impairment, and the high risk for recurrence [2]. Owing to the significant health risk of stroke and the limitations of currently available conventional therapies, unprecedented attention has been attached to complementary and alternative medicine (CAM) worldwide due to its potential efficacy on stroke.

Acupuncture is one of the most commonly used CAM therapies for stroke around the world. Up to now, at least 24 systematic reviews have been published, the available evidence suggests that acupuncture is effective for improving some aspects of poststroke neurological impairment and dysfunction, although there was insufficient evidence for stroke in preventing poststroke death [3]. Especially, electroacupuncture (EA) is an extension technique of acupuncture based on traditional acupuncture combined with modern electrotherapy [4]. There are many advantages of EA such as the readily quantifiable parameters for stimulation as frequency, intensity and duration, and the therapeutic benefit of EA is commonly identified to be equivalent to manual acupuncture [5]. In some situations, EA has been shown to be more effective than manual acupuncture, particularly when strong, continued stimulation was required, as when treating stroke [6]. Thus, EA is commonly used in current clinic and research. A systematic review from our team has indicated that the available evidence potentially supported the use of EA for acute ischemic stroke [4]. However, there is still a lack of enough evidence to recommend the routine use of EA for stroke.

Both systematic reviews of high-quality randomized controlled trials (RCTs) and RCT itself, especially those with double-blind placebo controls, are commonly regarded the highest level of evidence in judging the treatment efficacy and safety of interventions. The credibility of the evidence in support of a treatment approach depends on the quality of RCTs. However, a large body of evidence indicated that the quality of reporting of RCTs remains sub-optimal [7]. Researchers have accumulated and suggested that the RCTs which were of poor methodological quality tend to exaggerate the treatment effects and result in misleading in health care at all levels [8]. So far, two studies have already been conducted to evaluate the quality of reporting of RCTs on acupuncture for stroke. In 2006, one study [9] demonstrated that the quality of reporting of 74 RCTs on acupuncture for acute stroke was generally poor. In 2014, another study [10] indicated that the quality of reporting of only 15 RCTs on acupuncture for subacute and chronic stroke was improved but some central items were still insufficiently or inadequately reported in most of the studies. However, no study has yet been conducted to assess the quality of reporting RCTs on EA for stroke. Thus, this study aimed at evaluating the quality of reporting of RCTs on EA for stroke according to the consolidated standards of reporting trials (CONSORT) statement [11] and the standards for reporting interventions in clinical trials of acupuncture (STRICTA) statement [12].

Methods

Information sources and search

Eight English and Chinese databases were electronically searched from their inceptions to June 2014. They are Cochrane Controlled Trials Register, PubMed, EMBASE, AMED, China National Knowledge Infrastructure(CNKI), VIP Journals Database, Wanfang Database and Chinese Biomedical Database(CBM). The search terms were listed as follows: “electroacupuncture AND (stroke OR apoplexy OR cerebrovascular accident OR cerebrovascular attack OR cerebral infarction OR intracerebral hemorrhage OR cerebral vascular disease)”. Chinese databases were also searched using the above corresponding search terms in Chinese.

Eligibility criteria

All RCTs on EA as monotherapy or adjunct therapy for stroke compared with at least one control group as no treatment, sham/placebo EA or conventional treatment, regardless of publication status or language, were selected. The diagnostic criteria of stroke were clinically in accordance with the World Health Organization definition [13]. The diagnosis of stroke was confirmed by CT and/or MRI.

Exclusion criteria

Studies concerning EA therapy for paresthesia, post-strokedepression, bulbar paralysis and other non-functional dysfunction were excluded. Additional exclusion criteria were animal experiment, case report, review, single-arm study, retrospective study and historical control study, duplicated publications, and quasi-randomized trial. Crossover and cluster RCTs were excluded because of the employment of the CONSORT guidelines for parallel RCTs. Searches were limited to English and Chinese publications.

Data extraction

Two investigators underwent training in studying every item and multiple subitems listed in CONSORT2010 and STRICTA2010 to ensure the proper understanding of each standard. Each report was reviewed by two independent investigators. They extracted information according to CONSORT2010 and STRICTA2010 checklists. “1”or “0” was scored by the two authors independently to represent whether the RCT had reported the relevant item/subitem or not. “0” indicates no description of the corresponding item/subitem and “1” indicates that the author had mentioned the description of the item/subitem in the report. Investigators resolved discrepancies by consensus or consultations during the data-extraction process.

Data analyses

Microsoft Excel 2010 and R software (Version 3.1.1) were used for descriptive statistics analyses. The overall number of RCTs which corresponded with each item was counted. Subsequently results were represented as the percentage, and 95% confidence interval (CI) of each overall rate was calculated. We also classified the included studies into two groups according to which language they were published in Chinese or English. Proportions of reported items in two groups were compared using independent sample Student’s t-test. Statistical calculations were performed by using SPSS (version 17.0). Level of significance was set at P < 0.05.

Results

Study selection

A total of 2662 potentially relevant articles were identified. By reviewing titles and abstracts, 2168 papers were excluded for at least one of following reasons: (1) duplicate publication, (2) animal study, (3) not clinical trial, and (4) case report. After examining the remaining 494 literatures through reading the full text, we removed 424 papers. Of which, 61 were non-randomized controlled trials, 11 were not focusing on functional rehabilitation or with other indicators, 41 were about other diseases or using an ambiguous diagnostic criteria, 17 duplicate publications, and 294 studies with other reasons. Eventually, 70 eligible RCT studies [14–83] were selected for the final analysis (Fig. 1).

Fig. 1
figure 1

Flow diagram for the selection of articles for inclusion in the study

Full size image

Study characteristics

Seventy studies (one article was designed with 2 comparisons) involving 5468 stroke patients were identified. For the 5468 patients, there were 2420 male and 1739 female, and with the ages ranging from 24 to 89 years old. However, the gender and age of the remaining 1309 participants could not be obtained from the primary data. Sample sizes ranged from 6 to 160 participants. In 40 studies EA was used for cerebral infarction, while in the other 30 studies EA was used for both cerebral infarction and intracerebral hemorrhage (ICH). Seventeen studies were published in English, and the other 53 studies were published in Chinese. Six studies were online Master’s thesis and not formally published [17, 40, 64, 65, 81, 82]. For the control group, WCTs were used in 62 studies and sham EA plus WCTs in 8 studies. The duration of treatment varied from 10 days to 12 weeks. Six studies conducted follow-up assessment with duration from 6 weeks to 12 months. Four studies conducted sample size calculation [20, 23, 34, 63]. Twenty-one studies reported adverse effects. In 11 studies, the discripion of the professional acupuncturists who participated in the studies was very simple and without detailed background. Nine studies reported informed consent from patients [20, 23, 24, 27, 33, 34, 64, 65, 71]. Only 6 study [27, 50, 62, 64, 65, 74] reported ethical approval. Key data are summarized in Table 1.

Table 1 The characteristics of the included 70 studies
Full size table

Items reported according to CONSORT statement

The items reported from the 70 RCTs according to CONSORT statement are summarized in Table 2.

Table 2 The reporting number and percentage for each item of the CONSORT checklist of the included 70 studies
Full size table

Title and abstract

Twelve (18%) trials can be identified as random trials after reviewing the title (1a), among which 8 were in English. Fifty-four (77%) articles had abstracts that were comprised of objective, methods, results and conclusions (1b).

Introduction

Of the included studies, 90% provided the detailed description of backgrounds (2a). The proportion of studies with objectives (2b) was 93%.

Methods

Only 2 CONSORT items were described in all the included articles. One was the eligibility criterion for participants (4a) and the other was the interventions for each group with sufficient details to allow replication, including how and when they were actually administered (5). However, the proportion on the description of the patient’s allocation ratio was 58% (3a). None of the articles (0%) described the important changes after the beginning of the trial because of the recruitment (3b). Fifty-eight reports (83%) described the settings and locations where the data were collected (4b). The proportion on the description of definition of primary/secondary outcomes was 68% (6a). Four (6%) reports mentioned the method of how to determine the sample size (7a). Items on incomplete reporting were 1% (subitem 6b) and 10% (subitem 7b).

Randomization

Twenty studies (29%) mentioned the type of randomization as the simple random method (8b). However, the proportion of the description on sequence generation was 37% (8a), which used computer or random number table. Ten articles (14%) described the hidden mechanism by the use of opaque envelopes aiming to implement the allocation concealment (9). The detailed implementation was given in 7 articles (10%) (10). A total of 11 articles (16%) provided the description of blinding (11a), among which one was double blind (participants and evaluators) and the others were single-blind assessment. Sixty-eight studies (97%) provided the description of detailed statistical methods (12a), but no one provided methods for additional analyses (12b).

Result and discussion

Nine studies (13%) described the treatment progress of participants by a diagram (13a). Fifteen (21%) of these articles mentioned the number of the losses and exclusions after randomization with explanations (13b). Forty-four studies (63%) mentioned the periods of recruitment, but only 6 studies described the follow-up duration (14a). Two articles had reported a temporal interruption of the therapy because of the drop out of participants with personal reasons. Thirty-four reports (49%) offered the description of baseline data that included underlying disease or basic demographic or clinical characteristics, among which 23 studies (33%) represented the data in the form of a table (15). Fifty-seven studies (81%) described the statistics methods, including the use of intention-to-treat analysis (16). Almost all outcomes of the included reports were presented as the ratio of efficiency or means ± SD. Two papers (3%) applied 95% CI to describe the estimated value of the effect and its precision (17a). No study reported binary outcomes (17b). One study provided a kind of secondary analyses as “error type I” in statistics (18). In discussion section, 21 papers (30%) reported the occurrence of adverse events, such as acupuncture syncope, infection of puncture site and death (19). The proportions of papers reporting limitation (20), generalisability (21) and interpretation (22) were 14, 19, and 31%, respectively.

Other information: None of the papers reported the registration (23). Only 1 report (1%) gave the relevant electronic links for the obtainment of protocol (24). The proportion of paper with reporting of funding (25) was 420%.

Items reported according to STRICTA statement

The items reported from the 70 RCTs according to STRICTA statement are summarized in Table 3.

Table 3 The reporting number and percentage for each item of the STRICTA checklist of the included 70 studies
Full size table

Acupuncture rationale

Apart from several English articles, majority of the other included articles (91%) used the style of acupuncture from Traditional Chinese Medicine (1a). Eighty-six percent of the reports provided reasons for treatment based on historical context, literature sources, citing references where appropriate, and so on (1b). None of the studies had mentioned any alteration of the treatment after the beginning of the experiments (1c).

Needling details

Various intervention methods were used in EA treatment group, and were mainly as follows: EA plus conventional theory, EA plus acupoint injection, scalp EA plus acupoint injection, and EA plus internal carotid injection. All the 70 included reports provided the type of needle stimulation, including electrical acupuncture or electrical acupuncture combined with manual acupuncture (2e). Ninety seven percent of articles listed the names (or location if no standard name) of acupoints used at the uni/bilateral sides (2b); however, only 33% articles mentioned the number of needles, 33% (2a). Twenty nine percent of studies mentioned the depth of needle insertion (2c). The other STRICTA items on needling details were response elicited (de qi or muscle twitch response), 64% (2d), needle retention time, 55% (2f) and needle type, 66% (2 g).

Treatment regimen

All the reports mentioned the frequency and duration of treatment sessions (3b), whereas 69% articles provided the number of treatment sessions (3a).

Cointerventions

One item, details of other interventions, was mentioned in more than half of the reports, 86% (4a). Nine reports (13%) described some relevant information and explanations to patients, including informed consent (4b).

Practitioner background

Eleven articles (16%) provided vague and unspecific description on the background of acupuncturist which included expertise, duration of training and length of clinical experience (5). In the 11 articles, four mentioned that the acupuncturists were professionals, and the others mentioned the contents as expertise or duration of specific training.

Control intervention(s)

A total of 93% trials reported a precise description of the control or comparator (6b). Furthermore, 8 studies used sham EA as control with providing further details of items 1 to 3 in STRICTA. Ten percent of studies provided the quoted data to elucidate the rationality of contrasting and comparing other similar experiments (6a).

Comparison of reporting quality between Chinese and English studies

The total mean score in CONSORT items failed to achieve significant differences between English studies and Chinese studies (English vs. Chinese: 15.2 ± 4.3 vs. 12.3 ± 3.6, p = 0.05), Table 4. However, there is statistically significant improvement in three items published in English vs. in Chinese as follows: (1a) title (56% vs.6%, p = 0.01), (11a) blinding (44% vs. 7%, p = 0.014), (13b) losses and exclusions (56% vs. 11%, p = 0.004). As for the other items, they all showed no statistical significant differences, Table 4.

Table 4 Comparison of reporting quality between Chinese and English studies (CONSORT)
Full size table

There are no differences in proportions of items in STRICTA comparing studies in Chinese with that in English (Chinese vs. English: 10.3 ± 1.8 vs. 9.6 ± 2.1, p = 0.235), Table 3. Studies in Chinese have statistically siginificant improvement in the item (1b) reasoning for treatment provided (93% vs. 63%, p = 0.033) and (2d) response sought (72% vs. 44%, p = 0.035) compared with studies in English, whereas studies in English in the item (5) practitioner background (6% vs. 50%, 0.004) showed significant improvement compared with studies in Chinese, Table 3.

Discussion

A wealth of evidence indicated the very inadequate reporting of clinical researches. For example, information on the method of random sequence generation, primary outcome, sample size calculation, randomization stated in title, allocation concealment, and adequate blinding was reported in 34, 53, 45, 33, 25, and 18% of 616 reports indexed in PubMed in 2006, respectively [84]. Especially, in RCTs of traditional Chinese medicine that include herbal medicine, acupuncture and other no medication therapies, reporting of the key methods used for adequate randomization methods, adequate allocation concealment, adequate blinding, both adequate randomization methods and allocation concealment used, and all three used was only 12, 7, 19, 4, and 3% of 2580 reports, respectively [8]. Thus, several guidelines have been recommended to help incomplete and inaccurate reporting. The CONSORT statement [11] is an evidence-based, minimum set of recommendations for reporting randomized trials to alleviate the problems arising from inadequate reporting of RCTs. It offers a standard way for authors to prepare reports of trial findings, facilitating their complete and transparent reporting, and aiding their critical appraisal and interpretation. The 2010 version of STRICTA statement [12], an official extension to the CONSORT statement, is the standards for reporting interventions in clinical trials of acupuncture to facilitate transparency in published reports, enabling a better understanding and interpretation of results, aiding their critical appraisal, and providing detail that is necessary for replication.

In the present study, the quality of reporting of 70 RCTs on EA for stroke was generally moderate. The CONSORT scores achieved by the included studies ranged from 4.7 to 91.5% according to seven subdomains, and the STRICTA scores across six subdomains ranged from 16 to 84.5%. The central items in CONSORT of eligibility criterion, sample size calculation, primary outcome, method of randomization sequence generation, allocation concealment, implementation of randomization, description of blinding, and detailed statistical methods are reported in 100, 3, 68, 37, 14, 10, 16, and 97% of 70 reports, respectively. The reporting of detail items in STRICTA of acupuncture rationale is 1a (91%), 1b (86%) and 1c 0%; of needling details is 2a (33%), 2b (97%), 2c (29%), 2d (64%), 2e (100%), 2f (55%) and 2 g (66%); of treatment regimen is 3a (69%) and 3b (100%); of other components of treatment is 4a (86%) and 4b (13%); of practitioner background is item 5 (16%); of control intervention(s) is 6a (93%) and 6b (10%). Based on the results of present study, several key items need further improvement. First, a priori sample size calculation can reduce the risk of an underpowered (false-negative) result. However, in the present study sample size calculation was reported in only 3% of all the included trials. In fact, a survey of 215 studies published in 2005 and 2006 in six general medical journals with high impact factors revealed that only 34% of 73 studies adequately described sample size calculations [85]. If the trials were not conducted with pre-trial estimation of sample size, there will be a lack of statistical power to ensure appropriate estimation of the treatment effect [86]. Thus, we suggest that an effort should be made to increase transparency in sample size calculation. Second, successful randomisation reduces selection bias at trial entry, which depends on two hinge steps-adequate sequence generation and allocation concealment, and is the crucial component of high quality RCTs [87]. In the present study method of randomization sequence generation, allocation concealment, and implementation of randomization is reported in only 37, 14, and 10% of 70 RCTs, respectively. Inadequate or unclear allocation concealment can exaggerate clinical effects in 41 and 30%, respectively [88]. Thus, proper randomization should involve both random sequence generation and complete implementation of allocation concealment to minimize bias. Third, blinding is an essential method for preventing research outcomes from being influenced by either the placebo effect or the observer bias. Trials that were not double blinded yielded larger estimates of treatment effect than trials in which authors reported double blinding (odds ratios exaggerated, on average by 17%) [88]. In the present study, only 16% of 70 trials described blinding procedure. Thus, more attentions should be paid to this situation, especially in EA trials. Fourth, item 5 in STRICTA is practitioner background that required description of participating acupuncturists in qualification or professional affiliation, years in acupuncture practice, other relevant experience. However, practitioner background was reported only in 16% trials. Thus, practitioner qualifications should be completely reported, which could increase the certainty with regard to treatment quality and safe implementation of interventions.

Currently, the evidence from the study of manual and electrical needle stimulation in acupuncture researched by an executive board of the society for acupuncture research [5] demonstrated that fundamental gaps existed in the understanding of the mechanisms and relative effectiveness between manual and electrical acupuncture, and these two techniques are not interchangeable. In 2006, Zhang et al. [9] evaluated the reporting quality of 74 RCTs on acupuncture for acute ischemic stroke, indicating that the items in CONSORT of baseline demographic and clinical characteristics, method of random sequence generation, allocation concealment, blinding procedure, sample size calculation and intention-to-treat (ITT) analysis was 73, 35, 8, 11, 5, and 7% of 74 RCTs respectively; the items in STRICTA of the numbers of needles inserted, the needle type, the depths of insertion, the length of clinical experience, and the background of the acupuncture practitioners was 5, 47, 35, 1, and 8% of 74 reports, respectively. Compared with zhang’s study [9], the quality of reporting RCTs of EA for stroke in present study is better. In 2014, Zhuang et al. [10] analyzed the quality of reporting of only 15 RCTs on acupuncture for subacute and chronic stroke, indicating that poor reporting existed in terms of outcomes, sample size, outcomes and estimation, ancillary analyses, with positive rate less than 30% according to CONSORT statement. Meanwhile, based on STRICTA statement, item 4a: Details of other interventions and 4b: Setting and context of treatment, the positive rate was 20 and 33% respectively. The quality of reporting of RCTs on EA for stroke in present study is similar to the results of Zhuang’s study [10]. This result indicates some improvements in the quality of reporting of RCTs on both acupuncture and EA for stroke. One probable reason is that reporting of several important aspects of trial methods improved because the endorsement of the CONSORT Statement and STRICTA statement. Another possible reason is that Zhuang [10] studied only a small number of selected RCTs, thus the conclusions may not be scientifically sound and may be misleading. For present EA study, the third possible reason is that EA is more readily controlled, standardized and objectively measurable. Additionally, EA is mainly considered as a method to provide stronger treatment for nervous and mental diseases like stroke. Thus, the use of EA for stroke research can at least in part improve the standards of published RCTs and is favored in stroke trials.

From the comparison of the included studies published in Chinese and in English, we found the compliance with CONSORT statement is unsatisfactory. Thus, reporting of RCTs both in English and in Chinese should endorse the CONSORT items as complete as possible. In particular, studies published in Chinese need to improve the reporting of (1a) title, (11a) blinding, and (13b) losses and exclusions. For the STRICTA statement, the proportions of fulfilling the items (1b) reasoning for treatment and (2d) response sought in Chinese have statistically significant increase compared with those in English. The main reasons are as follows: (1) acupuncture has been practiced in China for over 2000 years [89] and Chinese journals lay emphasis on reasoning for treatment; (2) as one of the fundamental characteristics of acupuncture, deqi has been used as a prerequisite for clinical effects for a long time in China [90]. However, the proportion of reporting item (5) practitioner background achieved statistically significant improvement in English compared that in Chinese. The possible reason is that English journals pay more attention to endorsing the STRICTA statement [91]. Thus, both English and Chinese journals need to endorse reporting acupuncture RCTs based on the STRICTA checklist, especially item (5) practitioner background in Chinese and items (1b) reasoning for treatment and (2d) response sought in English, thereby actualizing an improvement in reporting quality of RCTs for acupuncture.

There are some limitations in this study. First, the searching languages are limited to only Chinese and English during sample selection. The reports which are published in other languages may be left out, and may harm the reliability of our results. Second, we only discussed the reporting quality of RCTs on EA in the present study, and compared with that of RCTs on acupuncture in the previous studies. The results may be potentially misleading, and the direct comparison between the reporting qualities of RCTs on manual acupuncture for stroke with that of RCTs on EA is needed in the future. Third, we carried out data extraction based on the published paper itself. This approach meant that we were unable to capture some primary trials with truly good quality in trial methodology but poor reporting in the final publication. Thus, when assessing trial quality of such studies, reviewing research protocols and contacting trialists for more information are needed.

Conclusions

Our study indicated that the overall quality of reporting of RCTs on EA for stroke according to CONSORT and STRICTA statement was moderate and the reporting quality needs further improvement. In particular, it must be emphasized that the poor quality reporting of crucial items which includes sample size calculation, sequence generation, allocation concealment, randomization implementation, blinding, and practitioner background should be adequately involved in RCTs on EA for stroke. More attention should be given to the reporting of RCTs on EA for stroke to ensure that all items in checklist of CONSORT and STRICTA are clearly delineated, especially the central items in the methodology. In addition, the use of EA for stroke research can possibly improve the standards of published RCTs when compared with manual acupuncture trials. However, this need further direct comparative studies.