The results from the present study indicate that the quality of structure and language of responses to drug-related queries in Scandinavian DICs is by and large satisfactory to good. Still, there is room for further improvements. The analysis of the experts’ assessments has added to our understanding of elements affecting the quality of these responses. In particular, the majority of experts emphasised the importance of giving specific, evidence-based advice, and relatively small nuances in expressions between responses may have resulted in different interpretations in terms of whether advice was specific or not. The use of phrases like “be cautious” and “perform a risk-benefit assessment” (Table 3), in the lack of more specific information on how to handle a situation, may not be useful for the clinician. Most queries posed to Scandinavian DICs are patient-specific , and information given in the responses needs to be applied to the particular case  and be operational. Scandinavian DICs include a staff of pharmacists and residents and consultants in clinical pharmacology with education and training in literature search, interpretation of published studies and experience with provision of decision support. The desire for specific advice among our external experts might be interpreted as a vote of confidence in the DICs. It is important, however, that working methods are transparent so that the enquirer may assess whether a given advice is in accordance with the existing and presented literature.
There seem to be different opinions among the experts as to whether it is sufficient to use secondary/tertiary sources and formerly answered queries to produce responses, as opposed to primary articles; however, this issue needs to be further investigated. The use of appropriate and credible sources along with critical literature evaluation skills  may be more important than exactly what type of source is used. The great workload of some centres may limit their ability to scrutinise the primary literature, as this kind of searching is time consuming [21, 22]. Especially for frequently asked queries where the evidence is comprehensive, e.g. use of antidepressants during pregnancy and lactation, the use of secondary and tertiary sources may be preferable from the staff members’ point of view. However, to ensure the responses being up-to-date, it is important to search for recently published primary or review articles.
Both expert groups commented on the lack of explanation of medical terms and abbreviations. Several external experts specifically mentioned medical terms (e.g. weight-adjusted dosage) that they did not understand the meaning of, and therefore did not know how to handle. A potential problem sending written responses to enquirers without verbal communication is the lack of assurance that the information and/or advice are interpreted as intended. Readers are individuals , and texts are read in another context than they are written . Several experts criticised the lack of translation of study findings into clinically meaningful information. Rather than just cite sources passively, they recommended commenting and explaining data and results with regard to individual patient treatment and clinical practice. This suggests that it is important for DICs to maintain and develop this type of competence among the staff.
In addition to the explanation of medical and pharmacological terms and abbreviations, the plain language expert presented and assessed several factors that may contribute to better readability of the responses, all based upon plain language theory (Table 1) [10, 14]. The plain language method implies adjusting written information to the recipients’ need, structure the document clearly, use informative headings, apply active voice and explain difficult, but necessary words. The aim is to ensure that the most important informational content is identified and understood by the reader as intended by the author of the document . Interestingly, responses that did well in the medical experts’ analyses did not necessarily score high in the language assessment and vice versa. Especially, our medical experts did not seem to pay much attention to the criteria 2, and 4 to 6 (Table 1). The lowest scores in the assessment of language quality were given for features that may relate to the DICs scientific style in writing, e.g. the use of passive voice instead of active, which is a common style for writing scientific articles. In addition, writing a scientific text often encompasses the need to reduce the number of words, thereby compressing the language. Whether writing more plain language actually is necessary for enquirers and staff members to understand each other is unknown. Although not all criteria may be useful in relation to the DICs’ responses, the present study underlines the importance of precise formulations, e.g. in relation to giving advice. It may not be advisable to leave to the enquirer to understand and interpret what is more or less implicitly stated in a text.
Although the number of previous studies within this field is scarce, several of the issues pointed out in this article have been mentioned by the UK Medicines Information Centres in their “Guide to writing medicines Q & As”  and checklists for quality assurance of queries and answers  available online. Included in the checklist is that the information given should be concise and presented in a logical order, no unreferenced key statements should exist, the summary should reflect key points accurately and completely, no mistakes should be made in the referencing, and a primary literature search (including the use of Embase/Medline) should always be performed .
Strengths and limitations
No studies have previously published results from quality assessments of written DIC responses using qualitative data and assessment by a plain language expert. Although identical queries were posed to the centres, we did not expect seven identical responses to each of these. Each centre has its “style” and working methods (Table S1), and this clearly affected the responses. In addition, the choice of literature to refer to, interpretation of documentation and the exact structure and wording of the response are examples of working procedures that cannot be 100% similar between individual staff members. The lack of concordance between DIC responses in terms of advice illustrates the difficulties in streamlining the preparation of these responses. However, this lack of concordance was also very useful, as we thereby could identify a potentially important quality criterion in DICs’ responses; to give specific, clinically useful advice.
The method for collecting the data required that internal and external medical experts supplied written comments. As such, one might criticise the level of systematic data collection. Surely, experts contributed in a variable degree to the qualitative comments. Moreover, most comments represented the opinion of single experts and no attempt was made to reach consensus among the experts with this respect. Our external experts may not be representative for the average user of the Scandinavian DICs, as they were familiar with and perhaps had an inherent positive attitude to the DICs. In addition, the monetary incentive may have introduced a bias, as the external experts may have felt obligated to judge the responses better than they otherwise would. However, the assessment of responses was quite time consuming, and we feared we would not be able to recruit GPs for this task without any financial reward.
The wording of the only query in the assessment form specifically formulated for qualitative information may have increased the experts’ focus on advice and conclusions, and introduced a bias. The experts reviewed responses to prefabricated study queries rather than self-provided queries, and the comparison of several responses to the same query might have made them especially strict in their assessments. However, it may also have helped them realise what the essential factors in the “best responses” actually were, which was our intention.
We focused on written responses only, because all the included DICs produce written responses to most queries. However, some of the centres give oral responses by telephone instead and/or in addition. Quality-assuring verbal responses would require tape-recording of telephone calls, with an increasing possibility of violation of the blinding. In this study, we have no confirmation that unblinding took place, but some DICs and staff members did express suspiciousness towards some queries. This may have caused them to respond more thoroughly to these queries than they otherwise would have done.
We posed only six queries to the DICs during this study. Although these were typical for the centres included, mainly patient-specific, the responses may not be representative for all the responses given. Nevertheless, the importance of this study does not necessarily lie in the quality of each response, but in the possibility to compare different responses to the same query against each other.
Our plain language expert usually works with texts intended for lay people and had to do some adjustments when working with DIC responses. The responses are specific answers to unique, mostly patient-related queries, rather than general information. Therefore, the expert did not assess the necessity of an introduction, background information and referral to further information, as normally would have been done. The Norwegian plain language expert had experience in evaluation of both Swedish and Danish language. The principles of plain language are generally the same for all three languages, but being Norwegian, we cannot rule out that the expert may have been biased in the evaluation of texts in the three different languages. Due to the cost of the language evaluation, we limited the assessment to one expert only and the scores of the language criteria are based on qualitative assessments. Applying an assessment of the language in these responses have extended and improved our understanding of the importance of structure and language style in these responses, although we may not feel that all criteria, e.g. the use of active voice, were suitable for our context.