Background

Health system planners and policymakers are responsible for managing health system performance, and thus review quality indicators as well as assessments of the current illness burden in the population so as to help predict the future need for health care services. They need to use existing data to help them first measure health system performance, and second improve the quality of the health system, such as where to implement particular quality improvement activities. The Institute of Healthcare Improvement, an organization that is a leader of quality improvement and performance measurement at the health system level [1–4], has espoused the seminal concept that high performing health systems ought to pursue the Triple Aim: to 1) reduce health care costs; 2) improve population health; and 3) improve the care experience [5]. Within the U.K and U.S. especially, the Triple Aim has grown in popularity because new health care reforms, such as the NHS Outcomes Framework and the Affordable Care Act respectively, have broad goals of improving health system performance, such as improving quality, equity and access, but also include the three goals of the Triple Aim.

Within the past decade, many health system performance frameworks have been developed [6–23], comprised of various quality indicators, some of which relate to the Triple Aim goals. For example, quality indicators related to health system costs include health expenditures per capita [19], 30 day acute care readmissions [21], and preventable hospitalizations [22]. Population health has been assessed through indicators such as life expectancy, disability-adjusted life year [19], obesity rates in children, and the percentage of adults who smoke [22]. Care experience has been demonstrated by indicators such as the rate of adverse events, timely access to primary care, and wait times for surgery. On the one hand, the existing quality indicators have been useful to improve quality within particular populations because many of them are often disease-focused, setting-specific, or based on one episode of care. They have a well-defined denominator and thus a particular process of care or group of providers that can be held accountable to improve the rates. On the other hand, these measures are limited in other ways. They typically do not capture performance of the broader health care system, including care across different providers and settings throughout the continuum of care. As such current measures especially do not serve well patients with multiple comorbidities who represent the most expensive and medically-complex users of the system [24–26]. Thus it remains a challenge to measure the broader system performance, with respect to the Triple Aim.

Not only do planners want to measure Triple Aim and health system performance effectively, but want to do so in efficient and simple ways that can aid quality improvement. Health system planners and policymakers often use visual dashboards to integrate and compile important quality indicators and other key performance indicators into one place. An effective dashboard can help them to easily access and analyze important trends from the indicators, supporting timely decision-making and quality improvement. While other dashboards of health system performance exist, to our knowledge, no dashboards exist that measure the Triple Aim. Moreover, little research has explored how health system planners might use such as dashboard to aid quality improvement activities and help achieve the Triple Aim. The purpose of this study is to develop a visual dashboard, based on a case-mix adjustment method, and explore its applicability to measure and help improve health system performance, with respect to the Triple Aim.

Methods

We conducted a retrospective, population-based, cohort study of adults from Ontario, Canada using multiple administrative databases. To develop the dashboard generally, individuals were assigned a predicted illness burden score using a case-mix adjustment system from diagnoses and health utilization data occurring in fiscal year 2007–08, and then followed forward to assess the actual health care utilization and costs in the following year (fiscal year 2008–09).

Specifically, we identified a cohort of unique Ontarians with a valid Ontario Health Insurance Plan number, the universal provincial health insurance plan, who were alive on April 1, 2008 and deterministically linked with the other administrative databases [27–32]. We excluded children aged younger than 18 to simplify the analysis. Using the previous fiscal year’s administrative data (FY2007-08), we applied the Johns Hopkins Adjusted Clinical Group (ACG) Case-Mix System (version 7) in this cohort to categorize individuals into mutually exclusive levels of healthcare need, called ACGs. Then using the subsequent fiscal year’s administrative data (FY2008-09), we determined the actual health care utilization and costs for each individual during that year. In doing so, this methodology aims to demonstrate the value of the dashboard to predict future costs and utilization, based on previous utilization and illness burden.

Setting

In Ontario, Canada, because of the mainly single-payer nature of the health care financing system, the government collects a large amount of administrative health care data that can be used for research purposes. The provincial Ministry of Health and Long-Term Care provides funding for all Ontarians for hospital and physician services free at the point of care, with other services heavily subsidized. As such, Ontario’s Ministry of Health, in some ways, can be viewed as a large health insurer with approximately 13 M enrollees, and an annual budget of approximately $42B.

Data Sources

The Registered Persons Database was the source for valid health insurance number, age (at start of FY), and sex. The Discharge Abstract Database, maintained by the Canadian Institute of Health Information, contained data for all hospital in-patient admissions and length of stay. The National Ambulatory Care Reporting System contained data for all emergency department (ED) and hospital and clinic outpatient visits. Finally, the Ontario Health Insurance Plan Claims database contained information on physician claims and ancillary services (e.g. laboratory work, blood tests).

The ACG case-mix system

The Johns Hopkins ACG system is a diagnosis-based, case-mix method that uses administrative data to predict a population’s current and future healthcare need, healthcare use, and costs in one to two years. It captures the clinical complexity of multiple comorbidities without being disease-specific. ACGs have traditionally been used for risk-adjusted reimbursement and practitioner profiling in the US [33–37], Canada, and internationally [29, 38–44]. The method uses each person’s inpatient and outpatient diagnosis codes, age, and sex to assign individuals into 32 Ambulatory Diagnosis Groups (ADGs) — the building blocks of the ACGs. ADGs are unique in that they do not group by disease, but instead based on clinical similarities in need and expected resource use. Because people can have multiple chronic conditions, ADGs are not mutually exclusive. For example, ADG1 represents “time limited: minor conditions” (e.g. bell’s palsy or diaper rash), whereas ADG24 represents “injuries/adverse effects: major” (e.g. lower limb fractures or intracranial injury). Second, the methodology groups ADGs into 60 commonly occurring combinations, which are mutually exclusive ACG groups. Examples of ACG groups range from relatively small needs, such as “acute minor” (e.g. ear infection or cold), to the most expensive, “10+ ADG combinations, 4+ major ADGs” (e.g. multiple, unstable, chronic conditions with acute complications) [43].

The dashboard

At the individual-level, based on FY2007-08 administrative data, every adult was categorized into an ACG, and their subsequent year’s health care utilization and costs from FY2008-09 were determined. Our results were aggregated into sub-populations of ACG categories, and the all 60 ACGs collectively represent all adults in the province. Specifically, we analyzed and presented our data to create 3 figures, which we refer to as a health system “dashboard”. The dashboard’s 3 figures describe the:

  1. 1)

    Number of individuals in each ACG;

  2. 2)

    Total system costs for each ACG; and

  3. 3)

    Mean cost per person for each ACG.

We applied a costing methodology that has been validated previously that includes health utilization costs of only hospitalizations, ED visits, physician claims and ancillary services claims that were publicly financed by the Ontario government [45–47]. Thus, the dashboard represents the past illness burden, utilization, health system costs, as well as the predicted health care use of the entire population profile or of any particular ACG category. We created a dashboard for the overall province.

Besides ACG categories, health care need was also quantified using ACG utilization weights, calibrated to Ontario data [43]. A utilization weight for each ACG represents the multiplier of expected resource use relative to the population’s mean expenditures. By definition, the average ACG weight was 1.0 for the entire population. Thus, an ACG weight of 2.0 represents a two-fold increase in expected healthcare utilization in the subsequent year. Note the dashboard is sorted from lowest to highest ACG weight, which corresponds to lowest to highest predicted healthcare need. Furthermore, we sorted our ACG categories into resource utilization quintiles based on the ACG weights. Higher quintiles were associated with higher expected utilization levels. All individuals in a given quintile were expected to use approximately the same amount of care. Non-users were excluded from the quintiles and assigned a zero value. This study was reviewed by the ethics committee of the Sunnybrook Health Science Centre and deemed exempt research, as it uses de-identified secondary data analysis.

Results

Our provincial analysis of Ontario in FY2008-2009 identified 11.4 million adults. Nearly half of the cohort was female and 16.1% were aged 65 or older. (Table 1) Approximately 3.2 M (28%) did not use the healthcare system or had services with unclassified diagnoses (ACG = 5100); their average ACG weight was 0.17. Conversely, 45,383 adults (0.4%) were categorized into the highest average ACG weight of 12.61 (ACG = 5070). Figures 1A-1C comprise the provincial dashboard. Figure 1A displays the distribution of number of adults who are categorized in the various ACG categories. The second largest ACG category is adults older than 34 who are expected to have acute minor issue(s) (e.g. ear infections, sore throats, etc.), comprising 10.9% of the population (ACG = 4100). In the lower 4 resource utilization quintiles, the ACG weight is less than twice (2.00) the average adult.Figure 1B displays the total provincial health system costs in Canadian dollars for all adults within each of the ACG categories, totalling $15.8B of direct costs from hospital, fee-for-service physician and ancillary service billing, and ED services. Based on ACG weights, the sickest 1%, 5% and 15% of the adult population will use about 10%, 30% and 50% of total health system costs respectively. The largest 5 ACG categories with the most aggregate costs to the health system are: acute minor issues for adults >34 (ACG = 4100); 0–1 major ADG, age >34, and 6–9 other ADG combinations (ACG = 4910); 2 major ADG, age >34, and 6–9 other ADG combinations (ACG = 4920); 1 major ADG, age >44, and 4–5 other ADG combinations (ACG = 4420); and the 4+ major ADGs, age > 18, and 10+ other ADG combinations (ACG = 5070). Health system costs directly relate to both the number of adults within each group and the severity of the adults’ conditions. Hospital services comprise the largest percentage of cost within each ACG, followed by physician services. ED costs comprise a very small fraction of total costs within each ACG.Figure 1C displays the average cost per category (weight = 12.6) was $17,271. The average yearly cost per adult (ACG weight of 1) was $1,382. Similar to total system costs, hospital services comprise the largest percentage of costs.

Table 1 Cohort demographics and ACG assignment by ACG weight
Full size table
Figure 1
figure 1

Using the dashboard to help measure the Triple Aim.

Full size image

Discussion

Our study applied a case-mix and costing method to a large population-based sample to analyze the predicted health care needs, and then follow that population to calculate the actual utilization costs. This analysis was summarized as a health system dashboard of the sickest and highest cost users in the system, which acts as one measure of health system performance. Unsurprisingly, the sickest and most expensive adults had multiple major illnesses and multiple combinations of comorbidities. Similar to the US, a small percentage of our Ontarian study population (15%) use a disproportionate amount of the total health care system costs (48%).

The dashboard can help to partially measure the Triple Aims

We propose that the dashboard can measures 2 of the 3 dimensions of the Triple Aim, namely health system costs and population health. (Figure 2) First, the dashboard indicates lower system costs over time when the cost curve in Figure 2C shifts to the left (so average health costs/person are less) or when any ACG category in Figure 2B is shortened (so total system costs for an ACG category is reduced). Second, the dashboard indicates a healthier population over time when the healthcare utilization curve in Figure 2A shifts upwards (more people move upwards into ACG categories with less healthcare use) rather than downwards (more healthcare use). While past and predicted healthcare utilization is a proxy for population health, we recognize it does not truly capture all aspects of health, such as quality of life. Nonetheless, measurement of the Triple Aim, even partially, is important.

Figure 2
figure 2

Using the dashboard to help achieve the Triple Aim.

Full size image

The dashboard can help policymakers to achieve all of the Triple Aims

In addition to measurement, we also propose that the dashboard’s information can support health system planners to manage their health delivery systems more effectively. The dashboard represents a profile of the illness burden and healthcare needs of a population at a given time, and is thus a useful tool to predict the need for future healthcare services. By using this data, planners and providers can proactively intervene with appropriate policy, programs, and clinical and social interventions for the highest risk patients or the patients who are most likely to benefit so that ideally the same population is healthier the next time the dashboard is measured. Thus, the dashboard would need to be reported regularly (e.g. quarterly or biannually) in a continual feedback loop to indicate changes over time. Using the dashboard in this way would be supported by evidence that show the effectiveness health system measures depends on the extent that it reflects the goals of the health system, the quality of the data, and the incentives for providers to scrutinize and act on the data [48]. Below, we provide an example of how the dashboard could be used at various levels to support the achievement of the Triple Aim.

  •  Reduce costs: First, to achieve lower system costs and move the cost curve left, (See Figure 2C) policymakers and providers could use the dashboard to support cost-effective improvements in care (e.g. new technologies, diagnostic equipment, or surgical techniques), such as in ACGs with many individuals even if they are not the most complex cases. The dashboard might also identify medical provider specialties that ought to be high priority for better alignment with financial incentives that focus on patient outcomes or bundles of care, rather than number of visits, as in fee-for-service. Second, policymakers and providers can act to reduce costs within individual ACG categories. For highest-needs or highest-costs individuals, they could support interventions that provide patients with additional care support, intensive case management, or early referral to palliative care as appropriate, all of which can avoid unnecessary hospitalizations and ED visits. For medium- and low-risk individuals, they could support interventions that maintain or improve health, such as self-care, healthy nutrition, exercise, and wellness programs, which can reduce costs within an ACG category. (See Figure 2B)

  •  Improve population health: The dashboard can present data to providers at an individual or group practice level about their patient rosters to determine if their roster is getting healthier over time (See Figure 2A). Providers and policymakers could use the information to focus on particular health conditions, regions, or populations that could benefit from targeted interventions, and the size and scale required for such interventions. For the sickest population, they could refer them to interventions or supports that prevent the worsening of their condition (e.g. falls prevention), or help them manage complex conditions (e.g. intensive case management or nurse coaching). Just as importantly, for the medium and low-risk population, providers could encourage interventions that promote prevention and wellness, improve health, and reverse the disease progression, such as self management and exercise programs.

  •  Improve patient care experience: The dashboard does not measure care experience directly. However, the dashboard’s information potentially can help providers improve care experience and overall health. For instance, the dashboard could report to providers the high-risk, medically complex patients they are the most responsible physician for. The providers could then work proactively, rather than reactively, to intervene and develop multidisciplinary care plans for higher need individuals to prevent worsening of the condition(s) rather than wait for them to arrive in the hospital or their office with an issue. Thus the dashboard can support integration with the broader health system and health care team to improve the care experience for the patient.

The dashboard marries both an economic analysis with a case-mix adjustment analysis to help user predict future costs in relation to predicted illness burden and health care needs. When the dashboard is compared over multiple time periods, it can provide a dynamic tool to help measure population health and health system performance. A key strength of the dashboard is that it can be adapted to profile multiple populations: it could be presented at a macro level (e.g. NHS regional teams or state Medicaid insurance program) for high-level planning and resource allocation, but could also be useful at a micro level (e.g. small health region, community, or small group practice) where sample sizes are small enough for health care providers to assume responsibility for patients in an ACG. Moreover, the dashboard can be adapted to present data by age (e.g. children), by disease or condition (e.g. cancer or heart disease), by geography, income, etc. Thus disease specific organizations, or community programs that focus on particular ages or needs, can also use a similar methodology to target high risk patients, as well as track impact and improvement on the population’s health. The dashboard relies on case-mix methods, which have the advantage of typically relying on existing administrative data already available. Moreover, large administrative data can produce population-based profiles, connect across multiple sectors, and compare across regions to allow for a broader health system assessment. Finally, the dashboard is not constrained by the ACG case-mix method. Other case-mix methodologies could be applied in a similar way to population-based samples.

The dashboard could be useful to policymakers, health system planners, and providers for measuring health system performance, especially in countries with largely single payer health systems, such as in Canada, UK, or other European Union countries. Many federally and regionally funded programs could report this dashboard indicator publically and regularly in order to manage region-wide initiatives and to facilitate quality improvement. In the US, the dashboard would be especially useful in “closed” group model systems or capitated managed care organizations, where they provide comprehensive care team is within the same health delivery system, so care information such as medical records can be shared easily. It would also be useful in open group models, because the dashboard is not setting specific and can support providers to better coordinate and integrate care across settings and providers.

While the dashboard might be unique and useful, it needs further development in applicability and accountability. We did not explore the ACG categories by provider practice group, by region, by children versus adult, or by disease type, though have planned other subsequent research to do this. The costing methodology could be more comprehensive and inclusive, and does not include privately obtained services. Cost and utilization by ACG does not determine quality of care. Case-mix measures are dependent on observed morbidity and provider diagnostic coding, which may be biased by unobserved morbidity and coding behaviours [49]. Moreover, we have yet to explore the predictive nature of the dashboard for different outcomes from one year to the next. This paper showed a snapshot of one year, and did not explore the true dynamic nature of the dashboard over multiple time periods. Prior research has shown the ACGs alone are predictive of 40% of same year’s and 14% of next year’s total costs [42]. Other versions of the ACG system include prescription drugs when assigning categories, which might improve predictive ability, though this was not done in this study. Further research that combines ACG data with other administrative data, such as clinical, health behavior, drug data, or survey data about quality of care, may increase its predictive ability and should be explored.

The application of case-mix methodology to administrative, population based data is not novel. In the public domain, many research studies applied ACGs to various populations for the purposes of case-mix adjustment and reimbursement, practice variation, provider profiling, and the management of disease populations [33–37, 50]. However, these studies do not aim to indicate health system performance with respect to the Triple Aim. In the proprietary domain, many managed care organizations and health insurers in the US, responsible for healthcare delivery to large populations, have likely adapted various case-mix methods to predict future expenditures and to measure health system performance, at least in the populations they serve. However, to our knowledge, the results of such analyses, especially the costing data, have not been made publicly available due to proprietary nature of the data. We believe this dashboard is a unique contribution in that it applies a standard case-mix approach—commonly used for confidential reimbursement, insurance rate setting, and practitioner profiling—in a way that can be used by health system planners to measure health system performance and inform quality improvement activities, whether organizations choose to make the data public or not. To measure more dimensions of the Triple Aim and health system performance more broadly, the dashboard should be integrated and used in combination with other quality measures.

Conclusion

In summary, the health system performance dashboard, developed via case-mix methods, can help policymakers, planners, and providers better manage their health delivery systems because it measures two of three of the Triple Aim goals at a system level. Moreover, when fed back appropriately to providers and policymakers, the health system performance dashboard has the potential to support appropriate resource allocation and related quality improvement in the system and ultimately help to achieve the Triple Aim.