Background

Vascular dementia is the second most common cause of dementia after that caused by Alzheimer’s disease [1–3]. It is thought to result from cognitive impairment caused by changes resulting from cerebrovascular disorders [4] and from various outcomes of ischemic and hemorrhagic encephalopathy that induce mental and physical disabilities [5].

According to a report from the World Health Organization (WHO), there were currently estimated to be 35.6 million people with dementia worldwide in 2012, which will double by 2030 and more than triple by 2050 [6]. Dementia is a challenge to patients, caregivers, and healthcare providers [7] and carries a heavy financial burden, with the annual cost of care per patient ranging from 17,000 to 55,200 US$ for serve dementia [8].

Scalp acupuncture therapy stimulates the lesion area on the scalp to produce therapeutic effects. Although this treatment modality has been performed for more than 1000 years, the treatment has advanced in recent years [9], and the WHO published the Standard International Acupuncture Nomenclature in 1991 [10]. The efficacy of scalp acupuncture for vascular dementia has been confirmed by numerous empirical clinical studies [11–13]. However, a more reliable scientific method should be developed to validate the effectiveness of scalp acupuncture for vascular dementia.

Some researchers have implemented randomized controlled trials (RCTs) as a means to validate the therapeutic effects and efficacy of acupuncture, and this study design is considered optimal to assess the effectiveness of an intervention [14]. However, despite the effective design of an RCT, inappropriate study methods can affect the reliability and validity of outcomes [15]. Thus, research methodology is a critical factor that determines overall study quality [16]. Hence, there is an urgent need to assess the quality of RCTs based on systematic quality control and assessment for all stages of clinical trials, from planning and execution to analysis [17].

The Consolidated Standards of Reporting Trials (CONSORT) guideline, which was developed in 2001 and updated in 2010, was intended to assist in the assessment and interpretation of parallel-group RCTs so that users could identify biased outcomes and ultimately improve RCT reporting [18]. The Standards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA) guideline was designed in 2001 and revised in 2010 to assist in the reporting of clinical studies on acupuncture [19]. The combination of these guidelines helps researchers to assess the completeness and transparency of RCTs [20].

The aim of this study was to evaluate the quality of reports on domestic and foreign RCTs in which scalp acupuncture was used for the treatment of vascular dementia, based on the CONSORT and STRICTA guidelines, in order to provide preliminary data from clinical trials on scalp acupuncture for vascular dementia.

Methods

Collection of the RCTs

We searched for all reports about RCTs in which scalp acupuncture was used for the treatment of vascular dementia that were published before the end of December 2015 in eight databases (PubMed, EMbase, Cochrane Library, China National Knowledge Infrastructure, National Institute of Informatics Scholarly and Academic Information Navigator, National Digital Science Library, Korean Traditional Knowledge Portal, and Korean Studies Information Service System). The search terms were “vascular dementia, dementia,” and “scalp acupuncture.”

Literature selection/exclusion criteria

Types of studies

RCTs in which a control group of patients with vascular dementia who received a placebo, sham treatment, or conventional treatment was compared with an experimental group treated with scalp acupuncture were included. Non-randomized, cross-over RCTs, case reports, and case-control studies were excluded.

Types of participants

The subjects of RCTs were patients who were diagnosed with vascular dementia based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, the U.S. National Institute of Neurological Disease and Stroke and the Association Internationale pour La Recherche et l'Enseignement en Neurosciences, or definitively via computed tomography or magnetic resonance imaging, regardless of age, sex, and other demographic factors.

Types of intervention groups

The types of scalp acupuncture interventions included were body electro-acupuncture and body acupuncture, accompanied by drug therapy using Western or Chinese medicine or physical therapy such as exercise and rehabilitation.

Assessment of the quality of the reports

This study assessed the quality of the reports based on the 25-item CONSORT 2010 guideline and the 6-item STRICTA 2010 guideline. Each item on the checklists detailed in these reports was answered with a “yes” or “no.” Two raters independently performed the assessments with reference to the explanation provided by the CONSORT guideline and STRICTA guideline. Their assessments were compared, and any discrepancies were determined by consensus with the third author [21, 22].

Rating of overall reporting quality

Each of 13 items included in the CONSORT guideline was scored to compute an overall quality score (OQS) (range 0–13) (Table 1) [23–25]. The discussion section items of the CONSORT guideline were excluded, due to the difficulty of objectively evaluating them [26–28]. Each of 17 items in the STRICTA guideline was also scored (range 0–17) (Table 2). For scoring of the quality of items, 1 point was given if the information for each item was stated in the study, and 0 points if the information was not stated or was unclear [19, 29].

Table 1 Rating of overall quality using items from the CONSORT guideline (n = 26)
Full size table
Table 2 Rating of overall quality using items from the STRICTA guideline (n = 26)
Full size table

Rating of key methodological items

In the CONSORT guideline, the key items pertaining to methodology (i.e., “randomization,” “allocation concealment,” “blinding,” “baseline characteristics,” and “intention-to-treat (ITT) analysis”) were assessed separately as they were related to potential factors of bias [30–32]. A combined key methodological index score (MIS) (range 0–5) of these five key methodological items was measured (Table 3). For each of the studies, 1 point was given when the reports had information on each item, and 0 points were given if the information was not stated or was unclear.

Table 3 Quality of key methodological items (n = 26)
Full size table

Data extraction and analysis

Two raters independently performed the assessments with reference to a discussion about the definitions of each item and the instructions provided in the CONSORT and STRICTA guidelines. Their assessments of the reports were compared, and any discrepancies were resolved by consensus with the third author.

Cohen’s к-statistic was quantified to assess the inter-rater agreement. We defined agreement of 0.20 as “poor,” that greater than 0.20 but less than 0.40 as “low,” that greater than 0.40 but less than 0.60 as “moderate,” that greater than 0.60 but less than 0.80 as “substantial,” that greater than 0.80 as “good,” and agreement of 1 as “perfect” [23]. Items upon which there was any disagreement were specifically examined to reach an agreement. Cohen’s к-statistic analysis was performed using SAS software, version 9.3 (SAS Institute, Inc., Cary, NC, USA).

In order to evaluate the overall quality of reported RCTs and relevant factors, OQS was used as a dependent variable modeled using linear regression. Only variables with p ≤ 0.10 on univariate analysis were included in the multivariate regression model to identify significant variables (p ≤ 0.05). To analyze the factors related to methodological quality, the MIS was used as an outcome variable in regression analysis. Linear and ordinal regression analysis was performed using SPSS software version 20.0 (SPSS, Chicago, IL, USA).

Results

The RCT selection process for this study is depicted in Fig. 1. After identification, screening, and determining the eligibility of the reports, a total of 26 relevant RCTs were included in the final analysis.

Fig. 1
figure 1

Flowchart of the article selection process. RCT randomized controlled trial

Full size image

Features of the included RCTs

A number of RCTs on scalp acupuncture for vascular dementia were reported in the last 10 years. Most of these studies were reported in the last 8 years, with one in 2006 (3.8%), one in 2007 (3.8%), four in 2008 (15.4%), three in 2009 (11.5%), one in 2010 (3.8%), two in 2011 (7.7%), five in 2012 (19.2%), seven in 2013 (26.9%), two in 2014 (7.7%), and none in 2015 (0%) (Fig. 2).

Fig. 2
figure 2

Number of publications

Full size image

Twenty-two reports were written in Chinese (84.6%) and four reports were in English (15.4%), but all of the 26 reports were published in China by Chinese researchers. With regard to the types of interventions in the control group, among the 26 RCTs evaluated, Western medicine was used in 20 studies (76.9%), body electro-acupuncture was used in three studies (11.5%), and body acupuncture was used in one study (3.8%), with a sample size ranging from 40 to 241 subjects. Twelve studies were conducted with external funding (46.2%).

Quality of the reports

Rating of overall quality

Table 1 shows the ratings of overall quality based on the CONSORT guideline. The median OQS of the 26 RCTs was 8 (minimum 5, maximum 11). The items that were relatively more neglected in the reports were “trial design,” “sample size,” “ancillary analyses,” and “harms,” showing less than 10% of a positive rate. The two raters reached a substantial (items 2, 6, 7, 12, 13, 14, 17, and 19), perfect agreement (items 1, 3, 4, 5, and 18) for all items with the exception of “outcomes and estimation” (Table 1).

Table 2 shows the ratings of overall quality based on the STRICTA guideline. The median OQS of the 26 RCTs was 12 (minimum 8, maximum 14). Items that were relatively neglected by RCTs were “extent to which treatment was varied (1c),” “number of needle insertions per subject per session (2a),” and “setting and context of treatment (4b),” with less than 10% of a positive rate. The two raters reached substantial (items 2c, 3a, 4a, 4b, 5, and 6a), good, or perfect agreement (items 1a, 1b, 1c, 2a, 2b, 2d, 2f, 2 g, 3b, and 6b) for all items with the exception of “needle stimulation (2e)” (Table 2).

Rating of key methodological items

The median MIS value for the five items (items 8, 9, 10, 11, 15, and 16) was 2 (minimum 0, maximum 5). Particularly, “allocation concealment and implementation,” “blinding,” and “intent-to-treat analysis” had less than 15% of a positive rate (Table 3). The two raters reached substantial (items 8 and 16), good (items 9, 10, and 15), or perfect agreement (item 11) for all items (Table 3).

Exploratory analysis: factors associated with better quality of the reports

In a single-factor analysis of variance, all the factors related to the CONSORT and STRICTA guidelines were found to have no mutual association. Ordinal regression analysis showed that the variables to enhance the quality of the methodology were not associated with the MIS (p > 0.05).

Discussion

In this study, a comprehensive and systematic search of domestic and foreign RCTs on scalp acupuncture for the treatment of vascular dementia was performed, and the quality of these reports was assessed based on the CONSORT and STRICTA guidelines. These two guidelines are useful tools that were developed to identify gaps in the reporting of RCTs and have been widely used to assess the quality of reports on acupuncture studies [20].

Based on the CONSORT guideline, the median OQS of the 26 RCTs was 8, which was greater than 50% of the total score, and that of the STRICTA items was 12, which was greater than 70% of the total score. However, the “trial design,” “sample size,” “ancillary analyses,” and “harms” items of the CONSORT guideline and the “extent to which treatment was varied (1c),” “number of needle insertions per subject per session (2a),” and “setting and context of treatment (4b)” items of the STRICTA guideline were inappropriately addressed or neglected in most of the reports. This finding is in line with the results of other quality assessment studies [20, 25, 29, 30].

It is important for researchers to clearly state the design of clinical trial studies (randomized, parallel, double-blind, placebo-controlled, sham-controlled, etc.), including the details of the randomization, as indicated in the “trial designs” item of the CONSORT guideline. Even in cases of a parallel randomization ratio such as 8:8 for two groups, it is important to clearly provide the allocation ratio to improve the quality of reporting. The importance of trial design is especially highlighted among non-universal clinical trials with large sample sizes or complex analyses [31].

Sample size calculation is required to maintain a balance between the statistical considerations and the differences in therapeutic effects for intervention groups. A clinically significant difference between the intervention group and the control group generates reliable outcomes if the sample size is sufficiently large [20]. However, it may be difficult to achieve the planned sample size because recruiting qualified subjects within a limited time period is challenging [33, 34]. RCTs with a small sample size are often prone to biases or are insufficient in measuring the therapeutic advantages [26]. Therefore, researchers should pay more attention to sample size measurements by consulting with professional clinical statisticians in order to identify highly reliable and significant differences between control and intervention groups.

With regard to the “ancillary analyses” item, performing multiple analyses of the same data may introduce biases that lead to exaggeration in the interpretation of the study results [35], and reporting of analytical outcomes that were not prespecified in the research protocol leads to selective reporting bias for subgroup analyses [36]. Therefore, authors should report the results of prespecified analyses for high reliability, and clearly state the reason for and purpose of conducting subgroup analyses.

A randomized trial is the optimal design to generate efficacy and safety data, but it is difficult to detect rarely occurring “harms” through such a design. Many RCTs have provided inappropriate reports of abnormal responses [37] or low-quality reports of abnormal responses [38]. Furthermore, very few studies mention severe abnormal responses or subjects who dropped out because of abnormal responses [39]. Information about the risk and the benefits of an intervention is required to help the participants of clinical trials make reasonable and balanced decisions. The occurrence and characteristics of abnormal responses affect the acceptance and usefulness of a particular intervention [20].

The “extent to which treatment was varied (1c)” item had a 3% of a positive rate in this study, which is similar to the results of a previous study [22]. Variations between treatments in each clinical trial should be minimized by using a standardized protocol, and researchers should describe the degree of personalized treatment to both the patient and practitioner [40]. The “number of needle insertions per subject per session (2a)” item had an 8% positive rate in this study. Researchers should strictly report the total number of needles used and as many other related details as possible [40]. The “setting and context of treatment (4b)” item is an additional important component to report [41]. As changes in medical practice or context of treatment of the patients affect the outcome of experiments [29], researchers must report information that is provided to the patients about the interventions for treatment and control groups [20].

The median MIS, which represents the quality of methodological reports of the CONSORT guideline, was markedly low at 2, and the majority of studies had insufficient or missing information on “allocation concealment and implementation,” “blinding,” and “intent-to-treat analysis.” These key methodological items play an important role in preventing selection bias, performance/detection bias, and attribution bias [42]. An inappropriate study design could lead to exaggeration of the clinical outcome [23, 24, 43, 44], which has been reported in a few other quality assessment studies [45, 46]. In reports of RCTs of Chinese medicine, the quality of the reporting of the methodology is poor, warranting the use of CONSORT guidelines. We recommend that clinical researchers should be professionally trained with regard to the design and reporting of RCTs, and that the studies should be stringently peer-reviewed to improve the overall quality of research articles for their submission and publication in international journals.

Although we have comprehensively and systematically assessed 26 RCTs, there are a few limitations in our study. First, it was difficult to search for studies published in languages other than English and Chinese, such as those published in Japanese or other Asian languages. Nevertheless, most of the studies on scalp acupuncture treatment are published in Chinese or English. Second, only articles published between 2006 and 2015 were included in this study, which may be associated with the increased incidence of geriatric diseases caused by an upsurge of the elderly population in the last 10 years.

Conclusions

This study confirmed the quality of reporting of RCTs on scalp acupuncture for the treatment of vascular dementia. The CONSORT and STRICTA guidelines should be more widely used to enhance the quality of the reporting of RCTs in the future. The present study findings suggest that more reliable research on scalp acupuncture for the treatment of vascular dementia, including larger sample sizes, is warranted in the future.