Background

Reporting guidelines promote transparent and rigorous reporting. In 1996, several experts from the United States, United Kingdom and Canada worked together and published the Consolidated Standards of Reporting Trials (CONSORT) statement in Journal of the American Medical Association (JAMA), which started the rapid development of reporting guidelines in medical research [1]. With the evolving methodology of randomized controlled trials (RCT) and evidence-based medicine (EBM), CONSORT has been updated in 2001 and 2010 [2, 3], CONSORT 2010 being the latest version. CONSORT has been endorsed and uptaken after its release by several medical journals [4], and reporting guidelines have improved the quality of both the reporting and the methodology [5, 6]. Because different fields of medicine and different types of research data differ in characteristics, several extensions have been developed. The STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) was officially published in 2001, and was updated along with CONSORT in 2010 [7, 8]. Authors of acupuncture trials and systematic reviews have expressed their belief that STRICTA substantially contributes to the reporting of acupuncture interventions [9] and has significantly improved the reporting quality [10].

High-quality systematic reviews (SRs) of acupuncture form the best evidence to inform guidelines and clinical practice. Accurate and complete reporting enables readers to determine the internal and external validity of the research result, and editors and reviewers to make comprehensive and objective judgement effectively. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement was developed for systematic reviews and meta-analyses (SR/MAs) [11]. Although PRISMA promotes the quality of the reporting and methodology of SR/MAs, in some specific areas, it cannot meet all the needs. The extensions PRISMA-Equity [12], PRISMA-Abstract [13], PRISMA-Protocol [14] and PRISMA-Network Meta-Analysis [15] were developed in 2012, 2013, 2015 and 2015, respectively. In 2012, the assessment of the published 476 acupuncture SRs/MAs with PRISMA and their included RCTs showed low quality in general. Information from those studies could not meet the needs of acupuncture practitioners, and therefore was not usable for implementation [16]. Furthermore, the results from a survey of the application status of the PRISMA statement indicated that it was rarely used in the field of acupuncture, possibly because of knowledge gaps among researchers. Another probable reason was lack of items tailored for the characteristics of acupuncture [17]. On the other hand, research in acupuncture develops rapidly with an increasing number of studies published every year [18]. Thus, it is necessary to develop an extension of PRISMA for acupuncture, and implement it along with PRISMA.

Our project aims to develop a reporting guideline for acupuncture SR/MAs. This project consists of three phases: the investigation about the requirements of evidence users for the information needs of reporting of acupuncture SR/MAs; three rounds of Delphi process; and a face-to-face meeting for reaching consensus. This paper described the first phrase, where we aimed to conduct a survey among the evidence users about their information needs, to directly capture their opinions about the current reporting of evidence and the further needs on information reported in acupuncture SR/MAs. Before this survey, we conducted a pilot study in Lanzhou on the questionnaire, and found that most of the questions and items were highly valued by the participants. We also investigated the reporting rates of the items in acupuncture SR/MAs, and found a range of 6.3 to 73.7 % [19]. In this survey, with comments from users and evidence from the literature review, we aimed to design a rigorous and valid questionnaire, and to conduct a survey among acupuncture clinicians, researchers, teachers, and postgraduates across the whole country. Based on this survey, we will draft the items for acupuncture SR/MAs reporting for the Delphi process in next period.

Methods

Design of the questionnaire

We designed a questionnaire containing three main parts: 1) demographic information of the respondent, including sex, age, occupation, education, and professional title; 2) the experience, awareness and knowledge on the clinical evidence of acupuncture, including the duration of career, working fields, and reading behavior, 3) the importance of the proposed items from the perspectives of evidence users, including background information about acupuncture, diagnostic criteria of Chinese medicine, types and details of acupuncture, outcome measures of the effects, experience of the operators, and the rationale of the follow-up time. Furthermore, respondents were invited to provide any item they thought important but were missing in our list. The initial items in part three were based on the analysis of existing reporting guidelines, such as STRICTA and PRISMA, and a systematic review of 476 SR/MAs on acupuncture [16] (see Additional file 1 for a copy of the questionnaire).

The questions were designed to be easy to complete, with several pre-defined options from which the respondents could simply choose their answer. An open-ended response marked “other” was added into some questions if necessary. Respondents were asked to score the reporting items for the necessity and feasibility of inclusion in SR/MAs of acupuncture using the Likert scale [20] ranging from zero (not necessary) to ten (essential). For the eight items need to be rated, each question included a space for “explanation” in the end for respondents who were willing to provide more details.

Method to conduct survey

We conducted a pilot survey by two trained investigators (XQW, DW) in Lanzhou to find out necessary adjustment and to evaluate the feasibility in an attempt to predict an appropriate sample size and improve the study design prior to performing the formal one. Then, the two trained investigators completed the formal survey between April and June, 2014 by distributing and collecting the questionnaires in person, which could prevent the low response rate of electronic questionnaire. Our investigators were responsible only for distributing the material, interpreting the background, and checking with the respondents to ensure the completeness of the questionnaires. For subjective questions, the respondents were asked to fill it themselves without our investigators’ interfering.

In the formal survey, we visited Traditional Chinese Medicine agencies, hospitals and colleges in five cities (Lanzhou, Chengdu, Shanghai, Nanjing and Beijing). With the inclusion criterion that respondents must have acupuncture clinical experience more than 1 year, we surveyed clinicians, researchers, postgraduates in their second year or higher, and doctors majoring in acupuncture. We surveyed at least 50 individuals in each city, with about 40 % of respondents being students. Acupuncture students, practitioners and researchers have a critical role in bridging evidence and practice, and their opinions are mainly based on practical work which can provide valuable experience on what is the most urgently needed information in acupuncture SR/MAs. Before we distributed the questionnaire, we identified a liaison person in every city, and collected information about the respondents in advance. Then, the investigator visited the respondents one by one. This investigation was approved by the Research Ethics Committee of the First Hospital of Lanzhou University, Lanzhou, China (approval number: LDYYL2013-0007). All the participants were required to sign informed consent. We marked student or non-student on the left top of each questionnaire to avoid errors.

Data analysis

Data were processed using Epidata 3.1 and analyzed with software Excel 2013. For the close-ended question with given options, we used frequencies and percentages to summarize the results. For evaluating the importance of the proposed items, we used mean value of the score, and analyzed the difference between postgraduates and non-postgraduates (including clinicians and researchers) with independent-Sample T-test with the SPSS Statistics 19 software. Additional items provided by respondents beyond this questionnaire were listed in a table. We abstracted all the questionnaire independently and grouped them according to the type of suggestion or opinion on reporting. All discrepancies were discussed and agreed on for final interpretation.

Results

A total of 269 questionnaires were collected, of which 251 (93 %) were used for analysis and 18 (7 %) were excluded because of missing data (Fig. 1). Of the included questionnaires, 52 (21 %) were from Lanzhou, 50 (20 %) from Chengdu, 48 (19 %) from Nanjing, 55 (22 %) from Shanghai, and 46 (18 %) from Beijing. The demographic and socio-economic characteristics of the respondents are shown in Table 1. The average age was 33 years (SD = 9.0, range 25–58).

Fig. 1
figure 1

Flow chart of the survey, including the source of the respondents and the process of identifying valid questionnaires

Full size image
Table 1 Characteristics of the respondents
Full size table

Table 2 showed the findings on the respondents’ experience and knowledge on clinical evidence of acupuncture. More than half of the respondents had work experience of less than 5 years. The main work of most respondents was clinical treatment with acupuncture. On the other hand, a notable percentage of the participants were involved in more than three types of work. Most of the respondents read one to five articles each month. According to the response, electronic databases, printed professional journal, academic conferences, search engines, and ancient literature were all commonly used to obtain information. Forty-three of the respondents chose at least three or more approaches. The most popular type of literature read by the respondents was RCT, and for the satisfaction of the completeness of information in acupuncture literature, only 2 % were very satisfied.

Table 2 Findings on respondents’ experience and knowledge on clinical evidence of acupuncture
Full size table

For the reporting items in our questionnaire, mean scores were all above five. Table 3 presents the respondents’ ranking of each item. The additional suggestions proposed by the respondents about what should be presented in acupuncture SR/MAs were shown in Table 4.

Table 3 Scores of the candidate items
Full size table
Table 4 Additional items proposed by respondents
Full size table

Discussion

Our survey and questions were designed for acupuncture evidence users with the hope of collecting reporting items for acupuncture SR/MAs, which was different from STRICA for reporting interventions in clinical trials of acupuncture. To ensure the representativeness, we discussed how to select the participants considering various aspects, including geographic location, gender, and work experience. We visited participants from five cities located in south, north, north-west and south-west China, and included about 40 % students. We surveyed students who were at least in the second year of their Master studies, and having completed an internship on acupuncture for at least one year, to avoid surveying participants whose knowledge of acupuncture is too limited to provide practical response. We recruited the participants from universities conducting research and teaching in TCM and their affiliated hospitals, research institutes for acupuncture, and TCM hospitals, to be able to fully reflect the need of scientific research among the various groups of evidence users.

Our findings were not optimistic in terms of the reading behavior of the respondents. Most of the respondents read one to five relevant articles per month, and surprisingly, we found also respondents who never read the literature. Yet most of the participants could give advice about the information they need in an article, based on their clinical experience of acupuncture. Forty-three percent of the respondents were not satisfied with the adequacy of information reported in SR/MAs, which corresponds to the poor reporting quality of acupuncture SR/MAs [16].

Acupuncture is used for various kinds of diseases [21] which differ from each other in terms of theoretical and evidence basis. The authors should clearly describe the theoretical knowledge on the use of acupuncture for this disease to promote the rapid understanding for readers. Acupuncture is characterized by a broad diversity of styles and approaches in both East Asia and Western countries [22], and the ways of conducting acupuncture and the theories behind these different styles also differ [23]. Therefore the appropriate background information on the style, and theoretical basis of acupuncture for the specific disease should be reported.

In general, there are two different sets of diagnostic criteria: syndrome differentiation used in Traditional Chinese Medicine, and diagnostic criteria of diseases used in Western medicine [24]. These criteria should be interpreted in completely different ways, and it is important to indicate which set of criteria, or both, should be used. There are diverse types of needles, and nine classical acupuncture needles are always mentioned, which include filiform needle, shear needle, round-pointed needle, spoon needle, lance needle, round-sharp needle, stiletto needle, long needle and big needle [25]. With different functions [26], it is necessary to state which type of needles the study can be applied to. At the same time, the details of the acupuncture intervention including numbers of needles, names of acupoints, depths of puncture, de-qi or not, and times for needle retention, will all influence the treatment effect. As for judgement on effect, there are many indicators used only in acupuncture, such as acupuncture-tailored symptomatic relief and adverse effects like bleeding. Between 2000 and 2009, 95 cases of serious adverse events were reported [27]. However, many such events have not been reported among adequately-trained acupuncturists, and they should therefore not be considered inherent to acupuncture, but instead being due to malpractice of acupuncturists [28]. Therefore, a qualified acupuncturist is needed to ensure the effectiveness and safety of the treatment. The necessary follow-up time differs greatly between different conditions, and the patients treated with acupuncture also need to be followed up for possible side effects.

Our results showed a high acceptability and recognition for most items among the respondents (Table 3). The mean scores of the third, fourth, fifth, sixth and eighth item were higher than eight, the first item was 7.2, while items two and seven were lower than six. But we did not exclude or include any items in this period. Instead, we would like to integrate these items with additional comments from the respondents in order to form the checklist of items for the following Delphi process. We also compared the mean scores between postgraduates and NPs (including methodologists, acupuncture clinicians, researchers and teachers), using the independent-sample T-test. The results of the T-test showed that the difference of the mean scores between postgraduates and NPs were non-significant (p value ≥0.05) for six of the eight items, and the p-value was close to 0.05 also for the two remaining ones where the difference was found significant (7.5 and 7.7). This indicated that the experienced acupuncture workers and the less experienced postgraduates tend to put similar attention on the most items, that is, both groups demanded more complete information in acupuncture SR/MAs. For details of acupuncture interventions and qualification of acupuncture clinicians in methods, postgraduates gave higher scores compared with non-postgraduates, which might be related with their experience.

For additional comments in Table 4, the top five mentioned items (mentioned by seven or more respondents) were carefully analyzed. The 7.9.1 and 7.9.2 were not tailored for acupuncture, and they have already been mentioned in PRISMA. As for the 7.9.3 and 7.9.5, they were similar or close to 7.1 and 7.4 respectively. And 7.9.4 will be integrated in the item checklist in the future period.

Our study has both strengths and limitations. We designed the questionnaire based on a pilot study and a literature review, combining the two key factors in evidence based medicine—— evidence and practitioners’ experience [29]. We designated two researchers to distribute and collect printed questionnaires and some explanatory materials without interfering the answer process, which helped us improve the response rate [30, 31]. We proposed eight items collected from current literature in our questionnaire, which probably does not cover all relevant issues for reporting SR/MAs of acupuncture. However, the open ended comments from the respondents gave us useful suggestions for further improvement. We limited our survey to five cities, but these cities represent in the different regions of China. We also considered the representative from the aspects of gender, geography, clinical experience, and experience in literature. The participants are representative of typical users of systematic reviews on acupuncture. Acupuncture is mainly practiced in specialized and general hospitals, where the health care workers are in general well educated and involved in research. In China, acupuncture is not commonly practiced in local health centers or private clinics, where the health care workers tend to be less well educated [32, 33]. In the next period, relevant experts will be identified and surveyed with an online questionnaire, and the results of this survey, including the comments and additional items suggested by the respondents, will be properly considered and integrated.

Conclusions

Most clinicians, researchers and students involved in acupuncture were not satisfied with the information provided in acupuncture SRs. This indicates the need for promoting the more complete and critical information reporting of acupuncture SR/MAs. Most of the items proposed in the questionnaire scored highly, and there were only small, mostly non-significant, differences between the opinions from postgraduates and more experienced NPs. The open-ended questions for respondents for collecting comments and additional items provided us with a lot information, which would promote our work in next step.