Impacts on practice

  • A validated set of indicators is useful to measure pharmaceutical care in community pharmacies.

  • A set of practice indicators is best compiled in cooperation with pharmacists and external stakeholders e.g. healthcare inspectorate, patient organizations and health insurers.

  • Measurement of indicators in pharmacy practice and feedback of indicator scores results in overall score improvement.

  • Further research is needed to better understand and facilitate quality improvement in individual pharmacies.

Introduction

Quality indicators in healthcare address measurable aspects of relevant systems, processes and outcomes. They provide insight into the performance of care providers and are used to stimulate continuous improvement of patient care [1]. Various quality indicator sets have been introduced throughout the western world [2–6]. Among others these sets are used to assess and improve the quality of medical practice [7, 8].

To assess pharmaceutical care as the pharmacist’s contribution to the care of individuals in order to optimize medicines use and improve health outcomes [9], in 2008 a national set of quality indicators has been established for community pharmacies in the Netherlands [10]. This was initiated by the Royal Dutch Pharmacists Association (KNMP) and the Netherlands Healthcare Inspectorate. The initial aims of the 2008 indicators were to gain insight into the quality of pharmaceutical services for supervision purposes, and to increase awareness among individual community pharmacists about their own performance. Since 2008, data on specific indicators have been collected from all community pharmacies in the Netherlands. All indicator scores were self-reported by pharmacists. To stimulate internal quality improvement, community pharmacists were annually given feedback reports with their own scores relative to those of the other pharmacists in the Netherlands.

As of 2010 other major stakeholders were involved in the quality measurement of community pharmacies. Beside community pharmacists and the Healthcare Inspectorate, this included representatives of patient and consumer organizations and health insurance companies. These parties were primarily interested in information that enabled patients to make a conscious choice of pharmacy based on performance. Thus beside internal quality assessment the set of quality indicators had to facilitate also external comparisons between pharmacies. Consequently under the supervision of a national steering body and with involvement of all major stakeholders an instrument was developed and used to validate the 2011 set for content validity, absence of selection or measurement bias, and statistical reliability [11]. This set of quality indicators was then thoroughly revised. Information for the resulting 2012 set was requested from all Dutch community pharmacies in 2013.

The comprehensive set of quality indicators for Dutch community pharmacies and it’s results may be useful for other countries and healthcare systems to increase awareness of individual community pharmacists about their own performance, improve supervision by the healthcare inspectorate and enable patients and health insurance companies to differentiate between pharmacies on the basis of performance.

Aim of the study

This study aimed to present a comprehensive quality indicator set for community pharmacies and to report the scores for these indicators as supplied by the majority of Dutch community pharmacies. In subanalyses the score development over 5 years was described for those indicators, that have been surveyed earlier, and the score consistency within one pharmacy over time was assessed.

Ethics approval

Data of pharmacists and patients were coded and anonymised prior to analysis. Use of observational data in descriptive retrospective studies in the Netherlands is not considered as an interventional trial according to Directive 2001/20/EC and to Dutch legislation, and therefore does not need to be submitted to a medical ethic committee for approval.

Methods

Study design

In this retrospective study information on the validated quality indicator set for the year 2012 was used. This information was requested in 2013 from all 1981 Dutch community pharmacies at that time [12]. Scores were calculated as a cross-sectional measurement of the quality of pharmaceutical care in the Netherlands.

For indicators that were measured as well in the prior 4 years and remained unchanged during the study period, in a longitudinal analysis the development of scores during the whole study period of 5 years was described and the score consistency within one pharmacy during time was assessed.

Setting for data collection

On average, each pharmacy served around 8000 clients in 2012 [12]. In the Netherlands clients mostly visit one particular community pharmacy [13]. All clients are registered in the pharmacy information system.

Community pharmacists in the Netherlands have a legal and professional responsibility to optimize the safe and effective use of medication, in cooperation with the prescribers. This implies that pharmacists can intervene if a prescribing does not follow the guidelines or does not seem suitable for an individual patient. According to their responsibility to ensure safe and effective medication use, pharmacists generally had high intrinsic motivation to explore their performance in providing pharmaceutical care.

Data have been collected using this national survey for 5 years, and annual participation is a condition for receiving a quality management certificate. The healthcare inspectorate specifically visits any non-responders. The process of data collection was widely announced in professional circles. Although reporting on indicators was voluntary, the relevance of providing data for both internal quality assessment and external accountability was emphasized in all communications [14–17].

Quality indicator survey tool

All Dutch community pharmacies were invited to complete the online quality indicator survey on 2012 between April and May 2013. This included 66 indicators within ten categories: ‘Quality management’, ‘Continuity of care’, ‘Communication with the patient’, ‘Clinical risk management’, ‘Compounding’, ‘Dispensing’, ‘Follow up of pharmacotherapy guidelines’, ‘Counseling’, ‘Logistics’, and ‘Training of pharmaceutical staff’. Information on each aspect was provided by the responsible pharmacist.

The quality indicators measured the presence of quality systems (‘structures’) and the performance of processes. As information on the clinical consequences of pharmaceutical care for individual patients was not available, outcomes of dispensings for patient groups were used as ‘outcome’ indicators (e.g. the concomitant use of interacting drugs or the absence of concomitant preventive co-medication).

The online survey contained automated controls, for example to prevent reporting of percentages above 100 % and to alert for missing scores. It also provided pharmacists with background information about specific quality indicators when necessary. Pharmacists could also obtain support from technical and professional helpdesks by e-mail or telephone.

Concurrent source of dispensing outcomes for quality indicator score validation

Information for numerical indicators based on the outcomes from dispensing to individual patients could also be measured by routinely-collected data from the Dutch Foundation of Pharmaceutical Statistics (SFK). The SFK collects drug dispensing data for nearly all community pharmacies [12]. Using validated algorithms, the indicators were calculated for each community pharmacy from the dispensing data delivered to the SFK, and median (50th percentile), 10th and 90th percentiles were available as quality indicator scores from that source. For answering the online survey, pharmacists could use the information from their own pharmacy information system or they could retrieve their indicator scores as pre calculated by the SFK from their dispensing data through a secure website of SFK.

Data analysis

Scores on the 2012 set were either expressed as categorical variables (yes/no) or as numerical variables (either a number or a proportion). Scores of categorically-measured quality indicators were given as the percentage of pharmacies reporting the presence of a quality aspect. For numerical quality indicators, the mean of the scores was reported, and the variance in answers was expressed as the 5th and 95th percentile.

For the longitudinal analyses, data from the pharmacies responding to the 2012 set were linked to their corresponding scores from the previous 4 years. Ten quality indicators (five on structures, one on process and four on dispensing outcomes) were surveyed in the same way over all five study years. For these quality indicators, the scores were also calculated for the preceding years.

The intra-class correlation coefficient (ICC) was calculated to compare the variance of scores for individual pharmacies over time (indicating changes over time for a particular pharmacy) to the overall variation in scores over the same period. The ICC is an indication of the correlation of scores for a quality indicator reported over time from the same pharmacy. The ICC could be calculated from logistic (for categorical indicators) and linear (for continuous indicators) mixed model analysis [18]. In this analyses, the repeatedly measured scores for an indicator were clustered within individual pharmacies. For continuous variables, the ICC can be calculated by dividing the variance of scores between community pharmacies by the total variance, where the total variance is defined as the sum of the variance between and within pharmacies [18]. For dichotomous variables, the ICC can be calculated with an additional equation [18]. The ICC can take values between 0 and 1. A high ICC (close to 1) means that the scores changed little for an individual pharmacy over time compared with the total variance in scores of all community pharmacies.

Descriptive statistics were performed with PASW statistics 20.0 (IBM Corp., Chicago, IL, USA). Stata SE-2009 (StataCorp LP Statistics/Data Analysis StataCorp, Texas, USA) was used to assess the ICC.

Results

Information was provided by 1739 of the 1981 Dutch community pharmacies (88 %). Table 1 gives the characteristics of the community pharmacies providing information for the quality indicator set on 2012. On average, each employed 1.3 pharmacists and 6.4 pharmacy technicians. Most pharmacies had a separate room for counseling and participated in night and weekend services. Two-thirds of them cooperated with other healthcare providers in a structured way. One-third also supplied nursing homes. One-third compounded medication within their pharmacy and <5 % also supplied other pharmacies with such preparations.

Table 1 Characteristics of Dutch community pharmacies
Full size table

Table 2 shows the 66 quality indicators across 10 categories, and their scores on the presence of quality structures, the degree to which pharmacies followed recommended procedures and outcomes from dispensing to individual patients. Within the set, 29 indicators measured the presence of structures, 24 focused on processes and 13 covered dispensing outcomes. Indicator scores showed high volumes for the presence of quality structures. For the presence of quality structures for instance, 84 % reported the presence of a validated quality certificate (QI 1.1), and 94 % responded that they had patients’ experiences within the past 3 years evaluated by a professional external party (QI 1.2). The presence of structures providing information about patients’ actual drug use (QI 2.1), contraindications (QI 4.3) and allergic reactions (QI 4.4) were reported by more than 95 % of those responding.

Table 2 Scores for the quality indicator set for community pharmacies in 2012
Full size table

Scores were lower for processes and dispensing outcomes, and the intervals between the 5th and 95th percentile were broad for indicators measured on a numerical scale. For instance, the process indicator ‘Percentage of patients older than 70 years with at least five different drug classes in chronic concomitant use, for whom the pharmacist contributed to the exchange of actual drug use information between the general practitioner and the hospital’ (QI 2.3) had a mean of 55 % with a range between the 5th and 95th percentile between 0 and 100 %. The outcome of dispensing indicator ‘Percentage of patients using opioids with concomitant laxatives’ (QI 7.3.1) had a mean of 54 % with a range between the 5th and 95th percentile from 35 to 75 %.

For longitudinal subanalyses on the development of scores during time and trends in score development of individual pharmacies, ten quality indicators were available (Table 3). Overall, the scores for these indicators improved from 2008 to 2012. As an example for the score development of the categorical indicator, ‘Participation in pharmacotherapy audit meetings on a regular basis and with specific agreements’ improved from 78 % in 2008 to 85 % in 2012 (QI 2.2.2). An example for a categorical indicator is the ‘Mean percentage of patients with a first dispensing of inhalation medication who had been offered information about its use’ (QI 3.1). This score increased from 58 % in 2008 to 70 % in 2012 Scores of those quality indicators, which were measured repeatedly, but not in all years, generally also improved (data not shown). The ICCs for the quality indicators had a range between 0.01 for ‘number of coumarin users with concomitant use of co-trimoxazole’ (QI 4.7) and 0.90 for the ‘presence of a valid quality management certificate’ (QI 1.1) (Table 2).

Table 3 Trends in quality indicator scores over 5 years of measurement
Full size table

Comparison of the self reported indicator scores to the scores measured by the SFK as an independent third party from routinely-collected dispensing data showed a high agreement between the two sources “Appendix” section.

Discussion

A comprehensive set of 66 quality indicators across 10 categories was developed by pharmacists and major stakeholders to continuously improve pharmaceutical care and to compare indicator scores between community pharmacies. Existing quality sets have been developed for specific diseases, such as urinary tract infections [19,] vulnerable patients such as children and older people, and specific settings such as nursing homes [20–26]. Other indicators on the quality of drug prescribing and dispensing mostly focused on drug use patterns [26–28] or described to what extent specific guidelines and recommendations have been followed [28–30].

The scores on this set of validated quality indicators supplied by 88 % of all community pharmacies provided insight into the pharmaceutical care performance of community pharmacies in the Netherlands. All indicators in the set assessed for 2012 and measured in the four preceding years showed a general improvement from 2008 to 2012. Small to moderate effects on professional practice have been previously described for audit and feedback [31]. A Cochrane review on the effects of audit and feedback reported a median change of 1.3 % in numeric outcomes, with an interquartile range of 1–29 % [32]. The changes in the indicator scores in our study were consistent with these results.

Structures to facilitate pharmaceutical care processes and dispensing outcomes for individual patients were present in at least 80 % of all Dutch community pharmacy. In this the presence of a valid quality certificate is the most meaningful, as this was acknowledged from a third, authorized party by national standards [33].

Scores for indicators for processes and dispensing outcomes were between 50 and 90 %. In principle, scores below 100 % suggest potential for improvement. Full compliance with treatment guidelines for all patients at risk is, however, unrealistic, and levels may also differ across indicators. For instance, the indicator ‘Percentage of patients using nitrates with concomitant antithrombotic medication’ (QI 7.2.1) had a score of 93 % in 2012, which had improved continuously from 76 % in 2008. Possibly this is the highest score to achieve in clinical practice for a patient population. The mean percentage of patients on opioids with concomitant use of laxatives (QI 7.3.1) remained between 45 and 56 % without a clear trend over time. The reasons for the deficient implementation of this recommendation should be studied in clinical practice.

Confidence intervals for the numerical indicators showed considerable variation and suggested differences in the performance of individual community pharmacies. Substantial changes in the scores of individual pharmacies were seen over time for users with the unfavorable combination of coumarin with co-trimoxazole (QI 4.7): in 2008, pharmacies had on average 18 coumarin users co-medicating with co-trimoxazole, but this had decreased dramatically by 2012, to an average of less than one per pharmacy. The low ICC of 0.01 shows that the absolute numbers reported by individual pharmacies for this interaction varied considerable over time, probably due to the huge decrease in numbers between 2008 and 2009 within all pharmacies. As the interaction of coumarin with co-trimoxazole should be avoided, this decrease means a substantial improvement in dispensing outcomes for patients [34]. This improvement might be due to increased awareness among community pharmacies resulting from the feedback reports on the first indicator assessment. It might also be due to extra alertness for this interaction by prescribers due to the new recommendations from literature [34].

The scores for the other quality indicators on the outcomes of dispensing processes (QI 7.11, QI 7.21, QI 7.3.1) measured percentages of patients co-medicated according to the guidelines. For these indicators, at least some consistency in the scores reported by individual pharmacies over time was suggested by ICCs between 0.46 and 0.54.The ICCs showed stable scores over time within individual pharmacies for the presence of systems such as a quality management certificate and the availability of protocols to provide information about contraindications (ICC for both aspects was 0.9).Further research should examine what factors might contribute to improvements in individual pharmacies and why some indicator scores improved more than others.

Strengths and limitations

A key strength of our study was that information was available from 88 % of all Dutch community pharmacies. Sub-analysis on the 12 % of non-responders in 2012 for their scores reported in previous years, however, showed lower scores on all quality indicators than the scores of the pharmacies included in our study. The scores presented here might therefore tend to overestimate the quality of care for community pharmacies in the Netherlands to some extent.

As all quality indicator scores were self-reported by community pharmacists, some of the indicators, especially on the presence of structures and processes, are too easy to claim without actually performing in the desired way. This attitude might be furthermore influenced by an increasing awareness of quality resulting from pay-for-performance policies of health insurance companies could have caused a bias in the scores towards the desired outcomes. Bias from inappropriate reporting of quality indicator scores was also assumed in the measurements of GP scores in the UK [35]. We therefore compared those scores that could be checked with scores calculated in a uniform way by a third party from routinely-collected dispensing data of SFK. These indicators on the outcomes of dispensing could thus not be claimed from self assessment by the pharmacists. The scores from the two sources were in agreement and there was no evidence of structural higher reported scores from self-assessment “Appendix” section. We therefore consider it unlikely that reporting bias influenced the scores presented here.

Conclusion

A set of quality indicators provided insight into the quality of pharmaceutical care at a national level for pharmacists, health care inspectorate, health insurance companies and patient organizations. Especially the presence of quality structures improved, and as of 2012 they were present in at least 80 % of the community pharmacies. Scores on care processes and on outcomes of dispensing varied between individual pharmacies and over time. These findings can be used in future research to understand the reasons for differences in quality improvement between individual pharmacies.