Keywords

Accountability and performance are vital pillars in healthcare organizations as both are key factors on the platform of global initiatives in healthcare and monitored through the management of performance measurements. In the context of rising healthcare expenditure, performance measurement (PM) is becoming increasingly integral to accountability. Healthcare accountability mechanisms have traditionally included business planning, annual reporting, and contracting. In recent years, a richer sense of accountability has emphasized the achievement of goals effectively and efficiently and has stimulated the growth of PM. PM has been described as “the use of statistical evidence to determine progress towards specific defined organizational objectives” (State of California 2003) [1]. The literature includes reports on performance measurement initiatives across the healthcare spectrum from primary through tertiary health care and public health and the voluntary sector, many driven by the backend as a reactive response to demands from governments, consumers, other payers, proponents of evidence-based practice, and accreditation organizations [2]. Substantial resources, by various organizations, have been invested in PM system development from policy level to front-line care delivery.

Performance measurement (PM) in its simplest form is the “measurement of performance”: the regular and continuous assessment of whether the current processes or practices in place are accomplishing the goals and objectives created, implemented, and monitored to sustain the organization, company, or program” [3]. Utilizing well defined measures will promote a culture of continuous growth and development, both clinically and operationally. Such necessity for PMs is the same for a stem cell transplant program – to ascertain and continuously monitor processes that function within the program from financial to operational to clinical aspects. PM is the process and performance measures are the drivers used to establish the level of success or need for improvement (Fig. 9.1).

Fig. 9.1
figure 1

Performance measures versus performance measurement

Full size image

SCT is a highly specialized and cost-driving service, operating at multiple levels of structures and processes, according to each program’s capacity and complexity of treated cases. Whether the focus of the program is to achieve excellence or to build a quality system or to comply with national or international accreditation, performance improvement and management can be challenging for the decision makers without objective customized performance measures for such complex medical care. The first part of this chapter will provide the reader with a comprehensive review of performance measurement definitions, development, and working framework. The second part will focus on performance measurement in SCT program.

Definitions for Healthcare Performance Measurement

Performance measurement

In general terms, it can be defined as the regular measurement of outcomes and results, which generate reliable data on the effectiveness and efficiency of various aspects of an organization that keep it afloat.

Input

Resources (human resources, employee time, funding) used to conduct activities and provide services.

Activity

Individual tasks funded by projects or programs.

Output

Products and services delivered. Output information does not tell you anything about the actual results achieved or the consequences of the products and services delivered. Output information is important to show the scope or size of what the inputs and activities produce.

Outcome

An outcome represents a specific result a program is intended to achieve. An outcome can also be defined as the specific objective of a specific program or service. An outcome is not what the program produced itself (the output), but the consequences of those products, services, or assistance. It is important to distinguish between end outcomes (objectives), on one hand, and intermediate outcomes (intermediate results), on the other.

End outcomes (objectives)

This is the highest-level objective toward which a program works. The end outcome is what the program is designed to ultimately achieve: which should be the most “ambitious” outcome or result program managers can materially affect or influence and for which they are willing to be held responsible.

Intermediate outcomes (intermediate results)

An intermediate outcome or intermediate result is a critical outcome or result that must occur in order to reach the higher-level, end outcome/objective. As the PM process advances, it is important to understand the necessity to obtain and act on the intermediate outcome or result before achieving the end outcome/objective.

Indicators

An indicator is an instrument that helps you measure change over time. It is important to remember that end outcomes and higher-level objectives require higher-level indicators. Intermediate outcomes/results require lower-level indicators. Indicators can be quantitative, or qualitative, or a hybrid of the two.

Performance indicator or key performance indicator (KPI)

A quantifiable measure used to evaluate the success of an organization, employee, etc., in meeting objectives for performance.

Measure Development Life Cycle

Figure 9.2 illustrates the five phases in measure development life cycle. Although the life cycle shows each phase as a discrete activity, the measure life cycle is dynamic. Some phases may overlap or take place concurrently or result in feedback with earlier phases [4].

Fig. 9.2
figure 2

Example for performance measure life cycle; https://www.cms.gov/Medicare/Quality

Full size image

Conceptualization

Develop measure concepts and then narrow down to specific measures. The developer conducts an environmental scan and requests input from a broad group of stakeholders, including patients.

Specification

Identify the population, the recommended practice, the expected outcome, and determine how it will be measured.

Testing

Assess the suitability of the quality measure’s technical specifications and acquire empirical evidence to help assess the strengths and weaknesses of a measure.

Implementation

Identify measures to submit for selection and rollout processes.

Use, continuing evaluation, and maintenance

Ensure that the measure continues to add value to quality reporting measurement programs and that its construction continues to be sound.

The Centers for Medicare & Medicaid Services (CMS) uses the following decision criteria throughout the measure development cycle to ensure that a measure meets the applicable standards before moving to the next phase:

Importance to measure and report

including analysis of opportunities for improvement such as reducing variability in comparison groups or disparities in healthcare related to race, ethnicity, age, or other classifications.

Scientific acceptability

including analysis of reliability, validity, and exclusion appropriateness.

Feasibility

including evaluation of reported costs or perceived burden, frequency of missing data, and description of data availability.

Usability

including planned analyses to demonstrate that the measure is meaningful and useful to the target.

Measure Development Process

Bringing It All Together

As previously stated, in this chapter, performance measurements for stem cell transplant programs operate, as they do, in general with all standard healthcare organizations, ensuring that all indicators and measures directly link to the organizations’ strategic objectives, mission, and vision statement – utilizing key performance indicators and the approach of Donabedian’s three domain quality framework of structure, process, and outcome measures [5]. Such is the case when conceptualizing the framework to construct, pilot, implement, and monitor stem cell transplant performance measures, ensuring that the critical key performance indicators that are essential to your operational strategic plan and also drive your measures.

Before one can create and implement measures or indicators, you should determine who your key stakeholders and process owners are as this is as important in determining your measures/indicators. These are those individuals who will work to bring to fruition your desired outcomes after you perform your intense education sessions with them and to provide the knowledge and training needed to meet the target set for your measures or indicators.

To begin with each measure or indicator is intentionally chosen through the process of conducting an RCA (Research, Compare, & Act), which begins with a rigorous internal research to determine specific programmatic needs, performing external literature reviews to determine best practice, and applying your finding in research to compare your current state with the desired state (Fig. 9.3).

Fig. 9.3
figure 3

SCT performance measurement RCA framework

Full size image

Once you have accomplished your research, compared your findings, determined what you need, your team is now ready to agree on the design of the selected qualitative and/or quantitative measure/indicator, determine your data source and collection frequency, create and validate a standardized data collection tool and perform a sample collection to test your tool. If the data tool collects what you designed and desired, perform a real data collection session . If not, go back to the drawing board, assess your tool, determine the cause, and make the revisions, as needed. You will not be able to move forward until your data collection tool is considered valid and confirmed as effective to collect the data it was designed to collect. Once you have collected your data, you are now you ready to Act: bring the outcome of your arduous work to action by performing an analysis on your data. Once your data has been analyzed and your targets set based on your current and desired state, you are now ready to develop your roll-out plan and schedule your action. First, disseminate your findings to your stakeholders and process owners to get them ready for your upcoming education and training sessions. A well-informed team is a well-equipped team and ready to assist you to shape the culture for success. It is now time to perform your educational sessions indicating your findings, beginning with the RCA, propose the plan to move forward and ensure to involve your team in continuous monitoring by setting champions in each area. If you have obtained your goal you will need to sustain the gain. If improvement is needed, use your well-informed team champion to perform continuous monitoring, education, and training until desired outcome is achieved.

As you now have set your data collection schedule, you will need to adhere to this to ensure continued success. Create dashboards or scorecards (Tables 9.3 and 9.4) to maintain intentional active continuous monitoring, allowing you to promptly act to any measure or indicator that may fall beyond the set target.

Performance Score

In most cases, at its basic level, a performance measure is a ratio. The denominator represents the number of eligible cases, less any exclusions or exceptions, and the numerator represents the number of instances the clinical action of interest was performed. It is helpful to note that the denominator is often derived from, and sometimes equal to, an initial population; this initial population is the broadest grouping (e.g., all patients age 14+ with transplanted specified diagnosis). The initial population can be reduced to a denominator (e.g., all initial population patients that underwent transplant) and then to a denominator with exclusions and exceptions removed. Figure 9.4 below visually depicts the anatomy of a performance measure from an arithmetic perspective.

Fig. 9.4
figure 4

Anatomy of a performance measure

Full size image

Goals for Measure Development in SCT Program

A critical aim of any stem cell transplant center or program is to ensure that such measures are directly aligned with the mission and vision statement of the healthcare organization in which it dwells. In addition, safeguarding the process within performance measurement affords one the ability to assess the current state, benchmark with competitors, ensure continuous assessment for improvement opportunities, ensure accrediting governing body’s continuous readiness, and monitor growth and development: internal assessment for an external cause corporately and publicly. Some challenges with PM in SCT can be the substantial variations in SCT center characteristics; centers have different care models to optimize the use of available resources, identifying external benchmarks and utility of comparative external benchmarks. However, the primary purpose of quality measurement is to identify opportunities to measure and thus improve patient care and other program-related outcomes. Table 9.1 summarizes some guiding principles for measure development.

Table 9.1 Criteria for performance measures selection [2]
Full size table

Quality measure development remains a focus for quality assurance and value staff to continually meet the evolving needs of its members, help its members meet national and international accreditation requirements, and to provide members with information that informs clinical decision–making.

SCT Program Performance Measure

There are several types of performance measures, but structural, process, and outcome performance measures are the most practical and objective that can be implemented in SCT across its three dimensions: clinical, collection, and processing. Table 9.2 outlines some of these performance measures, their definitions, and examples for their indicators in SCT program.

Table 9.2 Example of performance measures and indicators in SCT
Full size table
Table 9.3 Sample of annual operational dashboard
Full size table
Table 9.4 Sample of scorecard metrics
Full size table
Table 9.5 Sample of template for SCT quality KPI
Full size table

Structural measure

Structure of care is a feature of a healthcare organization or clinician related to the capacity to provide high-quality healthcare. Structure measures are supported by evidence that an association exists between the measure and one of the other quality domains.

Process measure

A process of care is a healthcare-related activity performed for, on behalf of, or by a patient. Process measures are supported by evidence that the clinical process, which is the focus of the measure, has led to improved outcomes. These measures are calculated using patients eligible for a service in the denominator and the patients who either do or do not receive the service in the numerator.

Outcome measure

An outcome of care is a health state of a patient resulting from healthcare. Outcome measures are supported by evidence that the measure has been used to detect the impact of one or more clinical interventions. Measures in this domain are attributable to antecedent healthcare and should include provisions for risk adjustment. The outcome of performance measurement processes should result in improved value. Value in the part of health care domains:

Improved patient care

Outcomes (lifestyle and survival)

Improved staff satisfaction

Processes and practices

Improved efficiency

Improved throughput, capacity, and quality of care and decreased cost

Improved competitive edge

Increased marketability and benchmarking

Improved resource utilization

Lowering cost (care and product services)

Performance Measurement and KPI

In its simplest form, a key performance indicator (KPI) is a type of performance measurement (PM); these are the critical (key) indicators of progress toward an intended result. KPIs provide a focus for strategic and operational improvement, create an analytical basis for decision-making, and help focus attention on what matters most.

Goals of an organization should set the focus of your quality, productivity, and financial metrics. The term “metric” means the same as a KPI to some, but to others, it means a collection of related measures that when placed together become a metric. Some have used metrics and KPIs for staffing needs, position justifications, quality assurance, revenue cycle, and strategic planning in various hematopoietic cellular therapy (HCT) programs. Various challenges discovered in implementing a PM within HCT programs are due to the substantial variation in the characteristics of HCT centers that have different care models to optimize the use of available resources, identifying external benchmarks, and the usefulness of comparative external benchmarks.

The Value of Dashboards for Metrics or KPI

Provide a user-friendly, visual summary of operational and clinical information.

  • Utilized to track data, improve operational and clinical performance, promote transparency, and improve accountability.

  • Manage a center’s performance by using key metrics, quality assurance, process improvement, clinical outcomes, financial growth strategies, and strategic planning outcomes.

KPIs can result in great limitations when used without setting in place operational effectiveness goals; however, goals should drive what you are measuring but measures should not drive goals. The question is: what are HCT centers or programs trying to assess and analyze? Here are some examples for initiative and strategic goals:

  • Evaluate resource utilization.

  • Reduce volume variability.

  • Use existing capacity more effectively.

  • Address staff stress and workflow inefficiencies.

  • Measure operational function that has an impact on quality.

  • Contrast staffing model with other departments or programs.

Dashboard Basics for Quality (Table 9.4)

  • Determine key objectives or areas for display.

  • Develop key performance indicators/measures for areas related to structure, process, and outcomes selecting best practice benchmarks externally and/or internally.

  • Develop your scorecard (Table 9.5), monitor/audit, and over time.

  • Use quality management principles to identify areas that require more attention whether exceeding or falling below your set benchmark.

  • Focus on high-risk high-volume indicators.

  • Use quality management principles of process mapping and step analysis to develop an improvement plan or document acceptable outcome.

Challenges HCT Programs Can Embrace

  • Developing quality management programs that use a variety of key performance indicators based on regulatory and accreditation requirements and program needs

  • Developing internal quality reporting data systems

  • Developing international collaboration to identify core measures for transplant programs to capitalize on the opportunity for external benchmarking

Examples of Quality KPI for Clinical SCT Program (Table 9.5)

  • Total transplant volumes by type and cell type

  • Other cellular therapy – DLI, HPC, volume, and outcomes

  • Length of stay by transplant type

  • Readmissions within 30 days of discharge

  • Bone marrow collections – volume, product cell counts, and recipient outcomes

  • Mortality rate at the +30 day, +100 day, and 1 year mark

  • Treatment related (non-relapse mortality)

  • Engraftment by type of HCT and sourced of stem cells, ANC and platelet count, median time to engraftment

  • Patient satisfaction (e.g. Press-Ganey-Transplant Survey), case management discharge survey, and donor follow-up reports

  • Critical event/quality reviews such as adverse events, intensive care admissions, and data audits

  • Transfusion medicine service indicators such as collection, processing, medication administration, infusion, mobilization, and positive microbial reports

  • Known complications – GVHD, ECP for GVHD, and infectious disease markers

  • Chemotherapy – verification procedure audits

  • Donor and recipient screening and informed consent

  • Support service reports and focus compliance audits

KPI Administrative Focus Areas

  • Staffing

  • Patient volume

  • Capacity planning

  • Strategic planning

  • Acuity versus procedure-based metrics

  • Length of stay

  • Mortality

  • Readmissions

  • Cost and resource utilization

  • Managed care/payer reimbursement

  • Analysis by type of transplant

  • Analyze by disease type

Conclusion

Performance measurement offers policy-makers in SCT program a major opportunity for improvement and accountability. Securing improved performance measurement often requires active leadership and it should aim to improve the quality of decisions made by all actors within the program. It is important to emphasize that the presentation of performance measurement data and how this influences its interpretation by patients, providers, and the public require more attention, as public reporting has many benefits but can lead to adverse outcomes; mechanisms should be put in place to monitor and counteract these adverse outcomes.