Background

Systematic reviews have demonstrated deficits in the quality and safety of medication use in primary care to an extent sufficient to constitute a public health threat. Three to four percent of all unplanned hospital admissions are due to preventable drug related morbidity (PDRM), with the majority attributed to high-risk prescribing and inconsistent monitoring [1–4]. Antiplatelets, diuretics, non-steroidal anti-inflammatory drugs (NSAIDs) and anticoagulants account for almost half of preventable drug-related admissions to hospital, with opioid analgesics, beta-blockers, drugs affecting the renin angiotensin system and anti-diabetic agents also frequently implicated [1]. In addition, safety alerts have been issued for drugs less commonly implicated in PDRM but associated with preventable deaths, such as prescribing and monitoring of methotrexate [5] and use of antipsychotics in older people with dementia [6]. These figures are likely to underestimate PDRM caused in primary care, since the negative consequences of under-use of effective guideline recommended drugs have not consistently been considered by the hospitalisation studies included in systematic reviews [1–4].

The 'Data-driven Quality Improvement in Primary care (DQIP)' research programme is designing and testing a complex intervention to improve the quality and safety of medication use in UK primary care. It is based on encouraging and facilitating primary care medical practices to systematically and continuously identify, correct or otherwise manage drug therapy risks that are potential pre-cursors to PDRM [7]. The DQIP approach requires explicit medication assessment criteria which can (1) be operationalised in existing UK electronic data sources in order to (2) identify patients at risk of common or severe PDRM in primary care.

A number of explicit medication assessment tools have been developed in recent years. The Beers criteria set [8] lists potentially inappropriate drugs in the elderly and can be relatively easily implemented in electronic data sets. However, a large proportion of listed items are not licensed or rarely used in the UK and many of the drug groups frequently associated with preventable harm are not considered. More recently published tools that also focus on the elderly, such as 'Assessing care of vulnerable elders' (ACOVE) [9], 'Screening Tool of Older Person's Prescriptions (STOPP)' and 'Screening Tool to Alert doctors to Right Treatment' (START) [10] have a broader scope, but many of the included criteria require manual record review and/or clinical judgement, which are barriers to routine or large scale applications. Other instruments that have been implemented in electronic records and target the primary care population at large [11–13] cover a limited spectrum of medication use issues, especially with respect to medication safety.

The study had two aims. First, we aimed to develop and classify by clinical importance a set of up-to-date medication assessment criteria that can be implemented in routine primary care clinical datasets to identify instances of (a) sub-optimally effective medication use for conditions commonly encountered in primary care and (b) high-risk use of drugs that have been shown to either commonly cause harm and/or cause severe harm in primary care. Second, we aimed to elicit the extent to which thematically-related medication assessment criteria, subsequently referred to as topics, are perceived to be priorities for quality improvement by professionals working in UK primary care.

Methods

Study design

The study was conducted in three stages. First, an extensive list of candidate medication assessment criteria was generated based on a structured literature review. Second, an expert panel participating in a modified RAND/UCLA (University College of Los Angeles) Appropriateness Method (RAM) study scored these items by clinical importance based on a summary of research evidence and their clinical judgement. Candidate criteria with high importance scores were translated into a final criteria set by removing redundancies (see below). Final criteria were characterised by the type of medication use targeted, informed by available taxonomies [13–15]. Third, thematically related final criteria were clustered into medication improvement topics and those derived from candidates with high importance scores were presented to a larger Delphi panel of clinicians working in UK primary care for prioritisation. The study was approved by the Tayside Committee on Medical Research Ethics A (reference no. 09/S1401/54).

Literature review

Prescribing is a ubiquitous feature of medical care which makes a systematic evaluation of the literature on prescribing quality or safety unfeasible in a single research project. We therefore focussed on medication use for conditions commonly encountered in primary care and drugs with clear evidence of significant benefit or harm. The literature review drew initially on UK national clinical guidelines, prescribing advice, and safety alerts, supplemented by European or other clinical guidelines and targeted primary literature review in selected areas as detailed below.

Candidate medication assessment criteria either described potentially beneficial medication use ('quality') or the use of potentially harmful treatments ('safety'). Candidate 'quality' criteria targeted common conditions where there are compelling indications for drug therapy based on UK and European guidelines [16–25]. Common conditions that may or may not require drug treatment for adequate management (depression, anxiety, dyspepsia, acute infections) or where we anticipated that undertreated patients would not be reliably identifiable from UK electronic data sets (chronic pain, chronic obstructive pulmonary disease, rheumatoid arthritis, thyroid disorders, epilepsy) were not considered. The following conditions were selected: primary and secondary prevention of vascular disease [26, 27], management of diabetes [28], heart failure [29], atrial fibrillation [30], asthma [31] and osteoporosis [32, 33].

In order to identify candidate safety criteria, the drug groups reported to be most frequently implicated in PDRM hospital admissions were identified from systematic reviews and large scale studies [1–4, 34]. For each drug or drug group identified, a more extensive literature search was conducted in order to identify patient and/or treatment related risk factors that make patients particularly vulnerable to drug-related toxicity by virtue of age, medical history, co-prescription, treatment duration and/or dose. Standard medicines information resources [35–39] and the primary research literature were considered in addition to selected previously published medication assessment instruments [8–10, 40]. Safety alerts in the British National Formulary [36], the UK National Prescribing Centre [38] and the Medicines and Healthcare products Regulatory Agency [39] were examined to identify prescribing that was less commonly reported to be implicated in drug-related hospital admissions but associated with severe harm. Candidate safety criteria targeting potentially harmful prescribing in vulnerable groups were identified drawing on the above literature sources (children and young adults, the elderly) as well as current clinical practice guidelines (heart failure [22]). Potentially important aspects of high-risk prescribing that relied on data items which are not consistently recorded in UK primary care electronic data sets (monitoring or achievement of international normalised ratio targets, monitoring of blood glucose in patients co-prescribed drugs known to enhance sensitivity to insulin or oral anti-diabetics, medication use in pregnancy/lactation) were excluded.

RAND/UCLA Appropriateness Method (RAM) study

The RAND/University of California Los Angeles (UCLA) appropriateness method is a rigorous way of combining research evidence with expert opinion [41], and has previously been applied to develop explicit criteria for the assessment of a range of health care procedures including medication use [42]. A panel of ten members was selected with clinical, public health or academic expertise in medication use in UK primary care. The panel was composed of four general medical practitioners (of whom two had National Health Service prescribing improvement roles) and six pharmacists (including two academics with a special interest in primary care, two working in medicines governance at health board level, and two working directly with general practices). All ten participants completed two rounds of scoring.

The questionnaire aimed to classify candidate medication assessment criteria derived from the literature as either 'necessary' or 'appropriate' care (table 1). 'Necessary' is a more stringent rating standard than 'appropriate', because it represents care that would be 'improper' not to be offered or avoided, whereas 'appropriate' is a more neutral balancing of net benefit or harm [43–45]. Following the RAM recommendations, ordinal scales of 1 to 9 were used for all ratings [43, 46].

Table 1 Definitions of rating categories used in the modified RAM study [55]
Full size table

All candidate quality and safety assessment criteria were scored for 'appropriateness'. Candidate criteria with a median rating of 4 to 6 ('uncertain') or disagreement (three or more ratings of 7 to 9 and three or more ratings of 1 to 3) on the appropriateness scale were rejected. Those items with median ratings of 7 to 9 were accepted as 'appropriate' and those with median ratings of 1 to 3 as 'inappropriate'.

Candidate quality assessment criteria were additionally scored on a 'necessary to do' scale, where items with a median rating of 7 to 9 (= clearly necessary to do) were accepted. Candidate safety assessment criteria were additionally scored on a 'necessary to avoid' scale, where items with a median rating of 1 to 3 (= clearly necessary to avoid) were accepted. Candidate criteria with median ratings of < 7 on the 'necessary to do' and > 3 on the necessary to avoid scale and those showing disagreement (defined as above) were rejected. The concept of 'necessary to avoid' was an extension to the original RAM method to differentiate between prescribing that is 'generally not worthwhile' from 'improper' in safety terms (see box 2).

The ten RAM panel members were emailed the first round questionnaire and a summary of the supporting evidence base. Panellists were asked to rate each item with reference to an 'average' patient consulting an 'average' primary care clinician in 2009 based on both the evidence summary and their clinical judgement [44]. Panellists subsequently met for a full day, where a summary of the first round ratings was fed back to panellists anonymously. This formed the basis for a moderated discussion of each item before the second round ratings were placed. All findings reported in this paper are based on second round ratings.

Delphi study

A random sample of general medical practitioners (GPs) and eligible pharmacists in Scotland and England was invited to participate by e-mail. In order to be eligible, pharmacists had to have experience of working in medicines governance, as a prescribing advisor or as a practice pharmacist. Twenty three (64%) GPs and 13 (36%) pharmacists agreed to participate.

The Delphi questionnaire listed the medication improvement topics to be scored together with a short summary of the scientific rationale for each topic. For each item, panellists were asked to state their level of agreement with the statement 'The described topic is a priority for collaborative quality improvement in primary care'. The term 'collaborative' was used in order to emphasise that the intended purpose of this study was to identify priority topics for quality improvement rather than measures for judging practitioners or practices as part of performance management.

As in the RAM study, all ratings used an ordinal scale of 1 to 9 (1 = strongly disagree and 9 = strongly agree). Panellists were instructed to rate topics in relation to primary care in general, rather than their own practice. The first round ratings were summarised and returned to participants by email for a second round of scoring. Topics with second round median ratings of 7 to 9 without disagreement (30% or more ratings of 1 to 3 and 30% or more ratings of 7 to 9) were accepted as 'priority', with median ratings of 8 or 9 defined as 'high priority'. All findings reported in this paper refer to second round ratings.

Results

Literature review and RAM study

The questionnaire listed 389 (100 quality and 289 safety) candidate assessment criteria. Upon completion of the second rating round, 318 (82%) candidates (93 quality and 225 safety) were accepted at the 'appropriate' and 275 (71%) items (73 quality and 202 safety) at the 'necessary' level. A number of candidate criteria were duplicates, in the sense that they were designed to determine thresholds beyond which care was judged appropriate and necessary. For example, 18 candidate quality assessment criteria related to glycated haemoglobin (HbA1c) levels beyond which treatment intensification was appropriate or necessary. Removing redundant candidate criteria yielded 52 quality and 124 safety assessment criteria to be included in the final set. Forty (77%) final quality assessment criteria and 107 (86%) final safety assessment criteria were derived from candidates accepted at the 'necessary' level. The results of the RAM study are summarised in tables 2 and 3 and the final list of quality and safety assessment criteria is presented in tables 4 and 5.

Table 2 Summary of final quality assessment criteria designed from candidates accepted in the modified RAM study as appropriate (App) or necessary to do (NecDo)
Full size table
Table 3 Summary of final safety assessment criteria designed from candidates accepted in the modified RAM study as inappropriate (InApp) or necessary to avoid (NecAv)
Full size table
Table 4 Quality assessment criteria generated from candidates that the RAM panel classified as 'appropriate but not necessary to do' (A) and 'appropriate and necessary to do' (N)
Full size table
Table 5 Safety assessment criteria generated from candidates that the RAM panel classified as 'inappropriate' (I) or 'necessary to avoid' (N)
Full size table

Table 2 shows the number of accepted quality assessment criteria categorised (1) by medical condition and (2) by four medication quality categories (MQ 1 to 4) referring to 'need (indication)', 'selection' or 'intensity' of drug treatment that were informed by available taxonomies [13–15]. The majority (87%) of the final 52 quality assessment criteria focus on the prevention (including diabetes mellitus) or management of vascular disease with lower proportions addressing asthma (8%) and osteoporosis (6%). Over half (52%) of final quality criteria target patients with unmet indications for drug therapy (MQ1) and 43% focus on treatment intensity (MQ3 and MQ4) for effective disease management with the remainder (8%) targeting selection of first line agents within a therapeutic class (MQ2).

Similarly, table 3 categorises the number of accepted safety assessment criteria generated (1) by high-risk drug or patient group targeted and (2) by eight medication safety categories (MS 1 to 8), referring to 'need (indication)', 'selection', treatment 'intensity', 'compliance' issues and 'monitoring'.

The majority of safety assessment criteria are drug-focussed (74%), either targeting drugs reported to be frequently implicated in PDRM hospital admissions (54% - section A) or others implicated in severe preventable harm (20% - section B). The remainder (26% - section C) target medication use in particularly vulnerable groups, namely the elderly (15%), patients with heart failure (8%) and children (4%). Over a third (36%) of final safety assessment criteria focus on potentially harmful use of NSAIDs, antiplatelets, anticoagulants and diuretics, the drug groups most frequently implicated in PDRM hospital admissions [1].

Over half (52%) of safety criteria target the selection of high-risk drugs (MS2 to 4), either for indications where safer (and equally effective) alternatives exist (MS2) or in patients particularly susceptible to adverse reactions because of age/co-morbidity (MS3) or co-prescription (MS4). A further 15 (12%) criteria target omissions of drugs indicated to mitigate the risk of adverse events from high-risk treatments (MS1), while twenty (16%) criteria target inconsistent laboratory monitoring (MS8). Two (2%) criteria focus on prescribing that may jeopardise patient compliance with methotrexate dosing schedules (MS7).

The majority of quality (81%) and safety (71%) assessment criteria are not restricted to the elderly (patients aged 65 years or older).

Delphi study

Grouping of thematically related assessment criteria that were derived from candidates accepted at the 'necessary' level yielded a total of 47 (18 quality and 29 safety) medication improvement topics to be rated by the Delphi panel. Thirty-six Delphi study participants completed a first round and 26 (73%) a second round questionnaire (table 6). Fifteen (83%) quality and 23 (79%) safety topics were accepted as 'priorities for quality improvement in primary care'. Eleven (7 quality and 4 safety) topics were classified as 'high priorities' and nine (3 quality and 6 safety) topics were rejected because of lower than stipulated median ratings (table 7). There were no differences between pharmacists and GPs with respect to the assignment of priority levels, and no panel disagreement on any topic (defined in terms of variation in scoring as detailed in the methods section above).

Table 6 Delphi study: Demographics of the 26 panellists, who completed both rounds of ratings
Full size table
Table 7 Delphi study priority ratings by the 26 panellists
Full size table

Discussion

This paper reports the development of a set of 176 explicit assessment criteria to identify patients at risk of PDRM from electronic data sources routinely held in UK primary care. The criteria set targets suboptimal selection, intensity or omissions of beneficial drug treatments (medication use quality) and high-risk use, inconsistent monitoring or patient instructions for drugs implicated in preventable harm (medication use safety) in primary care. All items are classified by clinical importance (appropriateness and necessity) as the output of an extended RAM process. Key professionals in UK primary care identified eleven clusters of thematically related medication assessment criteria (topics) as 'high priority' for quality improvement initiatives. The three highest rated topics related to methotrexate dosing instructions, high-risk prescribing of NSAIDs and antiplatelets and underuse of corticosteroids in asthma.

Development process of the DQIP criteria set

The RAM approach had advantages over the Delphi technique as an initial step in the criteria development process, because the face-to-face meeting ensured the necessary commitment of panellists to place ratings on an extensive and thematically broad list of candidate criteria that were grounded in the evidence base. The original RAM approach was extended in this study by introducing the concept of 'necessary to avoid', in order to distinguish between inappropriate ('not worthwhile') and 'improper' medication use in safety terms (see table 1). As for the distinction between 'appropriate' and 'necessary', panellists required examples to apply and reason the concepts, but the absence of paradoxical 'appropriateness' and 'necessity' ratings is consistent with a reliable rating process.

A limitation of consensus methods such the RAM is that ratings may depend on panel composition [42]. The chosen panel combined clinical, public health and academic expertise in primary care medication use in general, rather than specialist expertise in the management of each medical condition covered. It is possible that generalists underestimate the implications of suboptimal medication use because they do not individually see relatively rare PDRM events that have significant impact at population level. Conversely, specialists tend to overestimate the importance of practices that fall within their own specialty [47, 48]. However, since relatively few candidate criteria (22%) were rejected, it seems unlikely that including specialists would have substantially altered the results.

Scope and focus of the DQIP criteria set

Consistent with the intended use of the DQIP criteria set, our literature search targeted commonly encountered medical conditions and drug groups implicated in PDRM events in primary care rather than exclusively focussing on the elderly. As a consequence, only 27% of all developed criteria are restricted to patients over 65 years with the majority of generated assessment criteria covering aspects of medication use which are not or not exclusively relevant to the elderly [8–10, 49], such as primary prevention of vascular events, use of anti-diabetics in renal impairment [36] and treatments that are potentially harmful in children [36]. The fact that all topics identified as 'high priority' by the Delphi panel are age independent additionally underlines the relevance of not restricting a criteria set to be used in primary care to the elderly as is the case with many existing criteria sets [8–10, 49].

A limitation of the medication assessment criteria developed for this study is that several established and potentially important criteria were not considered because the study focused on those that could be applied routinely to existing UK electronic clinical data. For example, international normalised ratio (INR) results in the UK are often held in bespoke systems which hinder the implementation of meaningful measures for monitoring anticoagulant use [1–4]. Similarly, although a broad spectrum of medication use categories are covered, the criteria set is mainly focussed on the prescribing and monitoring stages of the medication use process with minimal coverage of patient education and compliance. In the future, the increasing sophistication of clinical information systems and the ability to link clinical datasets with laboratory systems and dispensing data would make an even broader set of assessment criteria feasible.

Although the DQIP criteria set has been developed for application in UK primary care, the drug groups reported to be implicated in PDRM events in primary care are similar internationally [50–52], and we would expect the areas focused on to be relevant in other countries and health care settings. Nevertheless, some local adaptation may be required in order to account for differences in drug licensing, available resources, and clinical guidelines.

Implications for quality improvement initiatives

The Delphi approach allowed stakeholders in primary care to prioritise the chosen medication use topics for improvement initiatives in UK primary care. The Delphi panel was deliberately chosen to include both day to day prescribers (GPs do almost all primary care prescribing, especially of the more complex kind being assessed in this study, but pharmacists prescribe for some patients and conditions) and those involved in prescribing governance and improvement (predominately pharmacists but including GPs with a more strategic role). A limitation is that our focus on professionals involved in primary care prescribing meant that we did not seek to include either specialist or patient/public perspectives in the Delphi panel. Since there is evidence that practitioners' perceptions of a targeted behaviour as meaningful is a pre-requisite to changing behaviour [53] we aimed to identify medication improvement topics which met this condition to inform the design of an intervention targeting primary care professionals.

It is important to note that even those topics that were not considered to be priorities (3 quality - and 6 safety topics) contain individual criteria that were agreed to be 'necessary' to do or avoid by the RAM panel. Examples are 'inadequate dose titration of ACEI, ARBs and beta blockers in chronic heart failure', and the 'using of warfarin without a compelling indication in atrial fibrillation with low risk of stroke'. These should therefore not be neglected. Lower priority ratings nevertheless indicate that changing and improving the corresponding medication use aspects may require targeted effort (or resources) in order to influence prescribing behaviour.

Conclusions

The DQIP medication assessment criteria set presented here has been developed using established consensus methods and complements existing medication assessment instruments by not being limited to the elderly and by targeting a wide spectrum of medication use practices implicated in common and/or severe PDRM events in primary care. As all previously published explicit medication assessment tools, the criteria set presented here does not, however, provide comprehensive coverage of all situations that put patients at risk of PDRM, reflecting the large scope and high complexity of medication use in primary care and the limitations of current UK clinical information systems. The best choice of criteria set will therefore depend on the main purpose to be addressed and will be guided by local priorities. Informed by the priority ratings of a panel of UK primary care professionals, we have selected a subset of the DQIP criteria to serve as outcome measures in a cluster randomised trial evaluating the effectiveness of a complex intervention to improve prescribing safety (Trial registration number NCT01425502).

The DQIP criteria were primarily developed to facilitate the identification of patients at risk of PDRM from routine electronic data sets for a targeted review of their medication. However, we anticipate that they could also serve a range of other purposes, for example by informing the design of clinical decision support systems, where the classification of criteria by 'appropriateness' and 'necessity' may guide the selection of alerts that should or should not be interruptive to clinicians' workflow. Performance feedback is a further potential application, but in order not to overwhelm practitioners, the developed criteria are likely to require further prioritisation and/or the design of meaningful composites, for example by aggregating items that address the same topic [54] or medication use category [13].

An inherent limitation of explicit assessment criteria is that they cannot fully account for clinical factors that may justify deviations from what is considered to be best practice in an 'average' patient. The extent to which patients identified to be at risk of PDRM are judged by practitioners to represent actual opportunities for improvement (concurrent validity) and the extent to which any improvements in prescribing or monitoring translate into improved patient outcomes (predictive validity) therefore deserve further study.