Background

Increasing numbers of older adults, primarily due to advanced age and frailty, are in need of nursing home (NH) care. At the same time, concerns about the quality of care provided to this vulnerable population persist. As a means of measuring and evaluating NH care, a standardized data collection and monitoring system, the Resident Assessment Instrument – Minimum Data Set 2.0 (RAI-MDS 2.0) was developed by the Centre for Medicare and Medicaid in the US. This system is now used in several countries including Canada. It allows for a valid, reliable, and standardized assessment of resident outcomes measured at the person level over time [1]. The use of standardized data, such as the RAI-MDS 2.0, makes it possible to define, compare, monitor, and report quality indicators (QIs) for clinical planning and decision making in NHs [2]. Although the RAI-MDS 3.0 is now used in the US, at present all jurisdictions in Canada use 2.0 with no immediate plans to change.

In our ongoing program of research, Translating Research in Elder Care (TREC), we focus on improving the quality and safety of care delivered to residents of NHs. Protocols for this program have been published elsewhere [3, 4]. Briefly, TREC closely follows a representative cohort of urban nursing homes in the Canadian Prairie Provinces, capturing RAI-MDS 2.0 data from those nursing homes from 2007 onward. As part of this research, the Safer Care for Older Persons [in residential] Environments (SCOPE) was developed with the goal of engaging frontline staff to become involved in the quality improvement process [5].

What is a quality indicator?

A QI is a computed measure based on a clinical outcome that is believed to be reflective of the quality of care. In other words, QIs are used as proxy or surrogate measures for quality of care. Outcomes can be undesirable, such as falls or pressure ulcers, or they may be desirable such as physical independence or improved continence. QIs were central in the original conceptualization of the RAI-MDS 2.0 assessment system. Public reporting of QIs has been done for many years in the US and is beginning to be used in Ontario long-term care [6]. Public reporting is thought to be a driver of improved quality either through consumer empowerment, or by 'naming and shaming’ [7]. But more importantly, QIs give individual facilities or operators a standardized and comparable measure by which to target and monitor quality improvement activities. When reported with transparency, poor performers can identify facilities with good performance, and seek to learn from them. Researchers can use QIs as a metric to shed light on the effects of ownership, funding, policy, care culture, and other factors.

Some QIs are strictly cross-sectional (e.g., use of indwelling catheters), while others use consecutive assessments to identify individual-level improvement or decline. Central to QI construction is the issue of risk adjustment, which arises from understood risk factors associated with poor outcomes, and these risk factors being unevenly distributed among facilities. Risk-adjusted QIs are designed to allow comparison of facility results with those of other facilities and to overall populations of interest. They take into account differences in the risk profiles of resident populations within individual facilities [2]. Methods for developing RAI-MDS 2.0 based QIs for use in NHs have been developed in the US [8] and have been applied in Canadian settings [9]. More recently 3rd generation risk adjustment techniques have been adopted [10, 11].

Practice sensitive QIs

In Canada, there are 35 validated QIs identified in the RAI-MDS 2.0 system; however, not all of them are equally sensitive to changes in practice, be it nursing, medical, allied or combined interventions. As our intent in the TREC program of research is to work with modifiable outcomes, we aimed to develop a set of what we term practice sensitive QIs. Similar to the SCOPE project [12], we intend to use the list of practice sensitive QIs and assess them for strength of evidence that would support developing or refining interventions in the NH population. This paper describes the process used to identify and develop this list of practice sensitive QIs.

Method

We began with the list of the 35 Canadian 3rd generation quality indicators for RAI-MDS 2.0 [10]. First we sought the opinions of two experts (Poss [13, 14] and Hirdes [15, 16]) familiar with the selection and construction of these indicators and they identified 10 as sensitive to nursing practice, two to physician practice, and one policy/legislation intervention (see Table 1 for the RAI-MDS 2.0 codes). Second using a modified Delphi technique [17], we then recruited informants based on their reputation as experts within the NH sector. The informant groups included practicing physicians (n = 4), nurses (n = 8), and decision/policy-makers (n = 4) all of whom were familiar with the RAI-MDS 2.0. More specifically, the physician group included two geriatric specialists and two family physicians with a specific interest in geriatric medicine, the nursing group included six nationally recognized nurse scholars with active research portfolios in the NH area and two practicing geriatric clinical nurse specialists, and the decision/policy makers were either NH Directors of Care or government level policy makers with a NH portfolio. We then submitted the 13-item list to informants (n = 16) via electronic mail asking them to anonymously and independently rank the items for (1) overall “practice sensitivity” and then, (2) to identify the domain to which the QI was most sensitive (nursing care, physician care, or policy maker).

Table 1 RAI-MDS codes and definitions of practice sensitive quality indicators
Full size table

Ethics

Ethics and operational approvals were obtained from Health Research Ethics Board of the University of Alberta and from the participating sites respectively.

Results

Results of the exercise are presented in Table 2. Overall, informants (n = 16) identified pressure ulcers as the most practice sensitive QI, followed by worsening pain, physical restraint use, the use of antipsychotic medications without a diagnosis of psychosis, and indwelling catheters. Additionally, the groups identified pressure ulcers, worsening pain, physical restraint use, declining behavioral symptoms, urinary tract infections, a decline in late loss activities of daily living (ADL) function (e.g., bed mobility, eating, toilet use), falls in the last 30 days, a decline in mood, and unexplained weight loss as most sensitive to nursing care. The use of antipsychotics – without a diagnosis of psychosis, indwelling catheters, delirium, and feeding tubes were deemed most sensitive to physician care. Lastly, none of the 13 QIs were deemed to be most sensitive to policy/decision makers. Use of antipsychotics, without a diagnosis of psychosis, followed closely by physical restraint use, and feeding tubes were the QIs identified as being the most sensitive to all domains of care (nursing, physician, and policy makers). Decline in mood and unexplained weight loss were acknowledged as the QIs least sensitive by any of the examined groups.

Table 2 Results of the modified Delphi exercise to identify the most practice sensitive quality indicators *
Full size table

When stratified by informant group, although the top five QIs stayed the same, the ranking of the 13 QIs differed by group (Table 3). The group of nurses ranked worsening pain and antipsychotic medications without a diagnosis of psychosis as the most practice sensitive QIs, physicians ranked pressure ulcers most sensitive, while policy makers saw indwelling catheters as the most practice sensitive of the QIs.

Table 3 Ranking and mean scores of most practice sensitive quality indicators as identified by informant group *
Full size table

Discussion

Improving the quality of care for NH residents has generated a considerable amount of attention in recent years. In response, QIs, based on available evidence and expert consensus, have been constructed and validated as reflections of both the process and outcome of care. In this paper, we described the process used for selecting and ranking the 13 practice sensitive QIs from an initial list of 35 indicators, all of which have been previously validated for use within the RAI-MDS 2.0. In addition, the QIs we have identified are congruent with those identified by the US Centre for Medicare and Medicaid Services [18] and the Health Quality Ontario [19]. These agencies utilize QIs primarily for public reporting purposes; therefore, the QIs identified have been judged to be both important and sufficiently valid (e.g. QIs included as parts of public reporting reflect the highest level of measurement quality). Table 4 provides a summary of the key indications of validity for the 13 QIs. While this work is based on RAI-MDS 2.0 data, we believe the process used to identify and rank the practice sensitive QIs as well as the actual indicators that we have identified will also be of interest to those using RAI- MDS 3.0.

Table 4 Evidence of validity for practice sensitive quality indicators
Full size table

Conclusion

While we have the ability to generate all 35 indicators we believe that focusing on these 13 practice sensitive QIs, not only provides a reduced and more manageable list of QIs for reporting purposes but also have the greatest potential for functional improvement and the slowing of the trajectory of decline that most NH residents experience. Using this information, combined with data related to the frequency of “events” and our ability to measure them sufficiently well enough to see change, we will generate a short list of 3–5 topical areas in which to focus future quality improvement interventions.