Background

Rates of antimicrobial resistance are growing worldwide, threatening public health and increasing morbidity, mortality and healthcare costs [1–3]. Research suggests that what underlies such high rates of antimicrobial resistance is the misuse of antibiotics in both human and veterinary practices [4–7].

Bearing in mind that antibiotic consumption in southern European countries is higher than that in the north of Europe [8], without obvious benefit to public health [9], it becomes fundamental to understand which factors underlie antibiotic misuse. Considering the pivotal role that physicians play in this process, an in-depth understanding of their attitudes to and knowledge of antibiotic prescribing and antimicrobial resistance is essential when it comes to developing effective interventions and improving antibiotic use [10, 11].

Two recent systematic reviews have highlighted physicians’ attitudes and knowledge as factors affecting physician antibiotic prescribing behaviour [12, 13]; those reviews were based on several qualitative and quantitative studies in different settings; however, any tool have been developed, validated and published which, based on these studies, allow researchers, health professionals and health authorities to measure these specific determinants of antibiotic prescribing.

Questionnaires are well-established tools for collecting data in health sciences and could be a very useful instruments for assessing physicians’ clinical-practice characteristics [14] and understanding their antibiotic-prescribing behaviour.

Several scales have been published for assessing antibiotic misuse and overuse [15]; these scales evaluate the relationship between patients’/parents’ [16] or physicians’ [17] characteristics and antibiotic misuse in the community. According to Alumran et al., however, such scales display weaknesses in development and assessment of validation, which hinder their use in different settings or countries [15]. Measurement of factors underlying physician misprescription of antibiotics should be made using reliable scales because these factors are extremely difficult to identify and assess.

Indeed, several parameters should be assessed to ensure the usefulness of the data collected, so that each question measures what it is intended to measure, all words and sentences are clearly understood, and all participants interpret the questions in the same way [18].

This study sought to develop and validate -in terms of face validity, content validity and reliability- an instrument designed to assess the attitudes and knowledge underlying physician antibiotic prescribing, in order to understand and improve antibiotic use in both hospital- and primary-care settings.

Methods

A questionnaire was developed with a dual purpose, namely: to assess physicians’ attitudes and knowledge vis-à-vis antibiotic prescribing, antibiotic use and antibiotic resistance, and the usefulness of different sources of knowledge used in clinical practice; and to collect socio-demographic and professional data on the physicians surveyed, including age, gender, medical specialisation, workplace and workflow. The final questionnaire took the form of a two-page long, self-administered document that was easy to complete, not time-consuming, and made up of the following five sections:

  • Section 1: instructions to complete the questionnaire;

  • Section 2 (“Antibiotics and Resistance”): 17 statements regarding attitudes and knowledge about antibiotic prescribing, antibiotic use and antimicrobial resistance;

  • Section 3 (“In the treatment of respiratory infections, how would you rate the usefulness of each of these sources of knowledge?”): 9 statements regarding the importance of various sources of knowledge, which can helps to understand the sources of knowledge underlying antibiotic misprescription;

  • Section 4: Socio-demographic and professional data (age, gender, medical specialisation, workplace and workflow);

  • Section 5: free to express ideas and views on antibiotics and resistance.

In sections 2 and 3, agreement with statements was measured using a horizontal, continuous, visual analogue scale (VAS), 8 cm long and unnumbered, scored from full disagreement to full agreement.

Figure 1 depicts the different steps of the study. The study was previously approved by the Northern Regional Health Administration (Permit No. 010484/ 2011), primary-care facility directors and Hospital Administration & Ethics Committee (Permit Date 10/2011). Authorisation from the Portuguese Data Protection Authority (Comissão Nacional de Proteção de Dados/ CNPD) was also obtained (Permit No. 2886/ 2013).

Fig. 1
figure 1

Flow diagram of questionnaire development and validation

Full size image

Step 1. Content and face validity of the questionnaire

Content validity was evaluated in two different stages, in accordance with published guidelines [19]:

  1. (i)

    The development stage, which included the build-up of the questionnaire and determination of the domain and concepts of the construct of interest. This stage consisted of a literature review [12, 13] and previous studies undertaken by a collaborative group in Spain [20]. It is important to refer that section 2 (17 statements about antibiotics and resistances) resulted from the literature review and concern the most important attitudes identified as influencing antibiotic prescribing. On the other hand, section 3 present the sources of knowledge identified in the literature review as being used by physicians nowadays.

  2. (ii)

    The judgment stage, in which the professional opinion of experts was evaluated. A convenience sample of two groups of physicians, one consisting of five primary-care physicians and the other of five specialists (2 internists, 2 paediatricians and 1 clinical pathologist) pre-tested the questionnaire and assessed the accuracy, clinical terminology, completeness and meaning of all the statements. An additional panel of two pharmacologists and two pharmacoepidemiologists also evaluated the content validity of the questionnaire.

Face validity, which includes an assessment of the grammar, syntax, organisation, appropriateness and logical sequence of the statements [15], was evaluated by two university professors, one an expert in clinical psychology and the other an expert in the Portuguese language.

Step 2. Reliability analysis

For an instrument to be valid, it must first be reliable [21]. Reliability refers to the consistency of a test or measurement [22]. To assess questionnaire reliability, two aspects were addressed, i.e., reproducibility and internal consistency.

In September 2013 we used the test-retest method to conduct a pilot test in a so-called NUTS II area (Nomenclatura das Unidades Territoriais para Fins Estatísticos/Nomenclature of Territorial Units for Statistics) of Portugal defined by the Northern Regional Health Administration (Administração Regional de Saúde do Norte, I.P./ARS-N), using a convenience sample of 61 primary-care and 50 hospital-care physicians. The questionnaire, accompanied by an explanatory letter outlining the study, was personally delivered to the head of each unit who then proceeded to distribute it to all participants who agreed to take part. In line with previous studies [23, 24], an interval of 2 to 4 weeks was allowed to elapse between the first and second administration of the questionnaire.

Statistical analysis

Agreement between answers was assessed by calculating the intraclass correlation coefficient (ICC), a measure of reproducibility of repeated measures on the same subject [25]. A one-way ICC was chosen because the effect of trials is not cross-checked against subjects (replication study; hence one-way) and the analysis is used to generalise from a trial that used a sample (thus random) [22].

Internal consistency was evaluated using Cronbach’s alpha [26], which is a very useful parameter to describe the extent to which all the items in a test measure the same concept or construct, and it is thus connected to the inter-relatedness of the items within the test [27]. When multiple traits underlie the items on a scale, the “tau equivalent model” assumption is not respected and Cronbach’s alpha underestimates internal consistency [27]. Since Section-2 statements evaluate different concepts (physicians’ attitudes and knowledge), assessing Cronbach’s alpha may be inappropriate according to the assumptions of this parameter. On the other hand, in view of the fact that Section 3 of the questionnaire exclusively evaluates one concept (the utility of different sources of knowledge), Cronbach’s alpha would seem to be a very useful parameter for assessing the internal consistency of this section.

Results

The response rates for the two groups of physicians studied were as follows: 64 % (n = 39) for the first questionnaire administered and 49 % (n = 30) for the retest among the 61 primary-care physicians, and 66 % (n = 33) and 60 % (n = 30) respectively among the 50 hospital-care physicians.

Based on data drawn from Section 4 of the questionnaire, Table 1 gives a breakdown of the socio-demographic and professional characteristics of the physicians included in the study. Notwithstanding the results reported above on the participants’ socio-demographic and professional characteristics, this study did not seek to evaluate the influence of any of these variables.

Table 1 Socio-demographic and professional characteristics of the study participants
Full size table

Step 1. Content and face validity of the questionnaire

The content-validity judgment stage consisted of the pre-test and professional appraisal of the questionnaire. It was assessed by two groups of physicians who made 9 changes (e.g., in Statement 6, “The use of antibiotics on animals is an important cause of the appearance of new resistance” was amended to read, “The use of antibiotics on animals is an important cause of the appearance of new resistance to pathogenic agents in humans”).

Face-validity assessment resulted in 19 changes to linguistic and interpretative terms in the questionnaire, e.g., Statement 2 was changed from “In a primary-care context, one should wait for the microbiology results to treat an infectious disease” to “In a primary-care context, one should wait for the microbiology results before treating an infectious disease.”

Step 2. Reliability analysis

Table 2 shows the ICC and Cronbach’s alpha results if any given item is deleted for both groups of physicians included in the study.

Table 2 Intraclass correlation coefficients (ICCs) for primary-care and hospital-care physicians
Full size table

The Cronbach alpha values yielded for Section 3 of the questionnaire were satisfactory (α > 0.70) [28] for both groups of physicians studied (αHospital-care physicians: 0.783; αPrimary-care physicians: 0.770).

Applying the definition proposed by Roster B. [25], ICC values smaller than 0.4 can be taken to indicate poor reproducibility, those ranging from 0.4 to 0.75 to indicate fair to good reproducibility, and those above 0.75 to indicate excellent reproducibility. On this basis, all statements displayed a minimum of fair to good reproducibility in both groups of physicians studied and in both sections of the questionnaire. Eight (Section 2) and four (Section 3) statements by primary-care physicians and ten (Section 2) and two (Section 3) statements by hospital-care physicians study yielded excellent ICC values.

When comparing the ICCs obtained for the two groups of physicians, considerable differences were found with respect to six statements in Section 2 (S4, S6, S9, S10, S14 and S16) and four statements in Section 3 (S1’, S2’, S3’ and S7’).

Discussion

Assessing knowledge and attitudes that guide antibiotic prescribing is an essential step when it comes to counteracting the problem of antimicrobial resistance. Several scales have been developed to measure the factors associated with antibiotic misuse world-wide [29–31] but most of them have not been fully validated [15]. This study reports on the development process and reliability evaluation of a questionnaire (Additional file 1) designed to assess physicians’ knowledge and attitudes concerning antibiotic use. The results yielded show that the questionnaire: (i) has content validity, face validity and is reliable in terms of internal consistency and reproducibility over time; and, (ii) could be applied to physicians working in both primary-care and hospital-care settings.

In terms of public health issues and related policies, this questionnaire for assessing physicians’ knowledge and attitudes offers several advantages over other published instruments, in that: (i) it allows for assessment, identification and description of attitudes and knowledge regarding antibiotic prescribing and resistance among hospital- and primary-care physicians simultaneously, and this in turn allows for comparison between the two groups of professionals; and, (ii) it uses a VAS which is a highly interesting scale for assessing small differences between physicians and between groups of physicians.

Indeed, considering the growing rates of antimicrobial resistance and decreasing effectiveness of antibiotics, antibiotic misuse is a major public health issue around the world, to the point where it is threatening a return to the pre-antibiotic era. Bibliographic reviews [12, 13] of physician antibiotic-prescribing behaviour have highlighted intrinsic factors, such as their socio-demographic characteristics, attitudes and diagnostic uncertainty, as being at the root of antibiotic misprescription. The literature also points the collegial advice and the logistics of microbiology test results as factors affecting antibiotic prescribing [32]. This questionnaire now enables factors -such as attitudes- related with antibiotic misuse to be identified.

Several interventions have been implemented around the world, targeting at improving antibiotic prescribing among primary-care [33] and hospital-care [34] professionals. Considering that these interventions should reflect the characteristics and barriers present in the setting where they are to be implemented, this questionnaire is an adequate instrument for assessing the factors affecting physician-prescribing behaviour, an essential pre-requisite for developing effective educational interventions and improving antibiotic use. When we aim to assess and improve attitudes influence on antibiotic prescribing, we can also use this questionnaire to evaluate the intervention effect (pre- vs post-assessment) on these determinants. Finally, use this questionnaire in different countries/ settings will also allows to compare the influence of this factors in different realities, which could help to understand different rates of antibiotic prescribing quality indicators in different countries.

Questionnaire development

Visual analogue scales are well-established instruments for collecting data in several different areas [35–37] and numerous studies have supported their usefulness, validity and reliability, when compared to Likert scales [38, 39]. While both scales capture similar information, a VAS assesses and enables the quantification of subjective phenomena, and might be able to discern subtle differences because of its greater range of possible scores [40, 41].

Face validity, as a form of validity that refers to a subjective assessment [42], and content validity, as a measure of the comprehensiveness and representativeness of a scale’s content [19], were assessed and guaranteed by the panels of experts, the literature review [12] and a qualitative study conducted by a collaborative group [20].

With regard to the methodology used in the pilot study (test-retest), the main problem lies in the potential for learning or recall effects that can affect the test [43]. The 2-week interval was fundamental to ensure that the time was [24]: (i) long enough to avoid carry-over effects due to memory; and, (ii) short enough to avoid changes in physicians’ knowledge or attitudes concerning antibiotic prescribing and antimicrobial resistance.

The high Cronbach’s alpha values obtained in Section 3 of the questionnaire (Primary-care physicians: 0.770; Hospital-care physicians: 0.783) revealed the internal consistency of the statements in this section, and we feel that this is linked to the objectivity of these statements, which exclusively evaluate the utility of different sources of knowledge.

A comparison of the results obtained for the two groups of physicians could be very useful for assessing the appropriateness of the questionnaire and its applicability in each setting. The usefulness of the questionnaire was not the same for both groups of physicians. We feel, however, that these differences were related to the specific characteristics of each group, e.g., the differences found in Statement 9 of section 2 (“In situations of doubt as to whether a disease might be of bacterial aetiology, it is preferable to prescribe an antibiotic.”) could well be related to the availability in a hospital setting of microbiological tests that are not to be found in primary-care facilities. As regards the differences seen in the Section 3 of the questionnaire, the fair ICCs obtained for statements about the information and courses provided by the pharmaceutical industry (S2 and S3), could be related to primary-care physicians’ greater exposure to sales representatives, something that is in line with the reported influence of pharmaceutical companies in primary-care physician training [44].

This study’s main limitation lay in the size and selection of the sample. Even so, 30 participants in each group of physicians would seem to be a reasonable number for a pilot study where the purpose is preliminary survey or scale development [45]. However, this sample size doesn’t allow to confirm the construct validity using exploratory factor analysis, which should be performed in future research with larger samples. While sample selection by a convenience method is the most widely-used process in validation studies, it nevertheless constituted our study’s main limitation [46]. TTo evaluate the construct validity of the questionnaire, a larger sample size is needed; however, to confirm the structure presented, future research should also evaluate this using exploratory factor analysis.

Conclusion

Bearing in mind that antibiotic resistance is one of the main public health problems world-wide, this questionnaire would seem to be a very useful tool for collecting data on physicians’ attitudes to and knowledge of antibiotic prescribing and resistance, which is one of the most important factors underlying this global concern. Hence, this questionnaire should be used to assess factors underlying physician-prescribing behaviour in hospital and primary care, along with other possible variables, such as socio-demographic or professional-practice variables (medical specialisation, workplace and workflow). We believe that the use of this questionnaire could prove of the utmost importance when it comes to identifying factors underlying antibiotic prescription and antimicrobial resistance in both hospital and primary care.

Questionnaire development would seem to be essential for improving the likelihood of success in achieving the main study objective, which, in this case, is to identify factors underlying antibiotic prescription and antimicrobial resistance. This study presents a very comprehensive questionnaire which could be used on both primary- and hospital-care physicians.