Abstract
Since their inception, small area studies intended to measure health system performance have been challenged by concerns that regional variation in health care may be primarily explained by differences in patient health risk. Controlling for regional population differences depends on appropriate risk adjustment, but the adequacy of the methods used in early analyses was contested. A novel response to these concerns was the development of end-of-life cohorts by Dartmouth Atlas investigators. These were used initially to control for differences in population health status in studies investigating relative efficiency across regions. Later, they became useful for studying hospital-level variation in chronic illness care, and for measuring utilization and patient experiences at the very end of life. Altogether, end-of-life cohorts have been invaluable for clarifying the contribution of health system and provider factors to health care variation and outcomes.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
By the early 1990s, geographic variation in health care delivery was a well-established field of academic inquiry and policy discussion [1, 2]. In the preceding two decades, researchers had observed large differences in utilization and spending for health services of all types—medical and surgical, inpatient and outpatient, primary and specialty care—in the United States and other countries, and across large and small areal units, including states, counties, cities, and market-based health service areas [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22]. The published literature had established unequivocally the occurrence of health care variation, although the magnitude and causes remained a matter of debate [23,24,25,26]. Was the variation explained by regional differences in the prevalence of illness and patient preferences, or by varying clinician practice styles? Did people living in areas where less medical care was delivered receive the care they needed and wanted, or was there evidence of under-service? Did people living in places with higher health care capacity, utilization, and spending live longer and have better quality of life? Or was there evidence of overservice?
Studies designed to explore these questions were often criticized on the grounds that the methods did not adequately adjust for differences across regional populations. Researchers employed several approaches to control for population differences, including analyzing relatively homogeneous [5, 27, 28] or demographically similar populations [18, 29], restricting studies to patients with the same illnesses [22, 30,31,32], and adjusting for case mix [10, 17, 33, 34]. As the scope of the research expanded from regional to national populations, the need for better risk adjustment grew. Some health systems and researchers argued that the risk adjustment methods used to date were inadequate, offering evidence that variation in medical care could be explained largely by patient need [20, 35,36,37,38,39].
The use of end-of-life cohorts by Dartmouth Atlas authors originated from the necessity of better risk adjustment; a secondary application was to measure care at the very end of life to understand the use of curative and palliative interventions. This paper will examine the development and use of this approach, first to address the risk-adjustment problem; then to investigate the consequences of variation in elderly patients with chronic illness; and finally, to identify care for patients for whom higher medical service intensity is generally viewed as futile or even harmful. We will discuss Dartmouth papers primarily, but we will also include studies that challenged the Dartmouth approach or extended its applications and findings.
Early Dartmouth end-of-life studies
Early variation studies from John Wennberg and his co-investigators were not primarily concerned with the specific issues of curative or palliative care in the elderly with advanced illness. However, several analyses used population mortality as a key measure to address the general question of whether residing in an area with higher-intensity health care was associated with better outcomes [5, 28]. Wennberg’s first small area analysis, published in Science in 1973, found no significant correlation between higher spending for hospital and physician services and age-adjusted mortality across 13 areas in the state of Vermont [5]. Subsequent research focused on Medicare beneficiaries living in the cities of Boston and New Haven and found that age, sex, and race-adjusted population mortality was not lower in Boston than in New Haven, despite the fact that beneficiaries in Boston had a 47 percent higher hospital discharge rate, 15 percent longer hospital stays, and 79 percent higher reimbursements per capita [19]. An earlier analysis of the two cities had attributed most of the higher utilization in Boston to the higher number of hospital beds per capita (4.5 per 1,000 residents, compared to 2.9 per 1,000 in New Haven) [18], foreshadowing a recurring theme of high capacity as a key driver of overuse.
The first Dartmouth Atlas of Health Care, published in 1996, expanded the small area variation studies of the 1970s and 1980s to a national scale [40]. Building upon the Boston-New Haven and other studies [41, 42], the report used population-based mortality rates as an indicator to demonstrate the association—or lack thereof—between either utilization or capacity and population health outcomes. Figure 1 shows the relationship (R2=0.36) between the number of hospital beds per 1,000 in a hospital referral region and, among other utilization measures, the likelihood that Medicare patients who died would do so in a hospital (Fig. 1, left). As with other measures, the capacity of the local health care system was associated with the propensity to hospitalize patients, including those who were at the end of their lives. The same report also showed that not only did a higher number of beds correlate with a higher rate of in-hospital death; it also correlated with slightly higher population-based mortality (R2=0.13), which is consistent with the earlier finding that having more beds in a region does not necessarily result in lower mortality rates (Fig. 1, right) [43]. The association is also consistent with the view that regions with higher health burdens use more hospital beds to care for sicker patients. This ambiguity motivated future efforts by Atlas investigators to develop measures of care intensity that were not confounded by population need.
The next Atlas editions, released in 1998 and 1999, extended the analysis to include other regional measures of care intensity during the last six months of life, including the propensity to hospitalize and admit patients to intensive care units; the number of days patients spent in hospitals and ICUs; reimbursements for inpatient care and ICU charges per patient; the number of physician visits per patient; and the number of different physicians involved in patients’ care [44, 45]. The 1999 Atlas also documented the strong association (R2=0.50) between spending for inpatient care during the last six months of life and average per capita Medicare reimbursements for all enrollees (Fig. 2), reporting that “how people are treated in the last six months of their lives is a good indicator of the overall intensity of medical intervention in the population” [45].
These early Atlas analyses were based on all Medicare patients who died, regardless of disease status or illness progression. Unlike earlier small area studies, which compared the care of patients in relatively localized regions with similar demographics, Medicare patients across the United States were not comparable, even though their outcomes were the same—they had all died. The mixture of chronic illness and age of death in the patient population are two examples of important differences across regions and hospitals; variation in the mix could affect both utilization and spending, as discussed below. Further control of patient factors was necessary for unbiased inference.
A subsequent study by Fisher et al published in the journal Health Services Research further explored the relationship between local hospital capacity, mortality, and the likelihood of dying in the hospital, this time controlling for disease burden in addition to sociodemographic characteristics [46]. The methods used to adjust for differences in population health included restricting to patient cohorts for whom hospital admission could be considered a legitimate proxy for disease incidence unaffected by area hospital bed supply—those hospitalized for hip fracture, heart attack, stroke, or surgical treatment of colon or lung cancer [18, 22, 33]—as well as differences in hospital day rates during the last six months of life among patients who died, under the assumption that these patients would be similarly ill in terms of their risk of death. Subsequent to the index admission, the researchers found no mortality benefit from increased area bed availability or propensity to hospitalize, and a greater likelihood that, among patients who died, death would occur in an inpatient setting. This study represented the first peer-reviewed analysis from Dartmouth investigators to use a decedent cohort to adjust for health status. The estimates of illness risk, however, were still ecologic measures of regional levels of hospitalization, education, income, poverty status, and disability. Although studies showed that these were valid proxies for underlying disease burden [47, 48], the chance for residual confounding remained.
These early studies did not find benefits to Medicare patients of greater resource capacity, higher spending, or more care during the last months of life. However, the studies did reveal positive correlations between capacity and the tendency to use the hospital for end-of-life care; between capacity and mortality; and between Medicare reimbursements for patients during the last six months of life and for beneficiaries overall. The use of end-of-life patients as a novel cohort with homogeneous outcomes confirmed these findings. The intensity of end-of-life care thus came to be understood as a measure that reflected utilization and spending that was unrelated to patient demand (i.e., need or preferences). Because all the patients in this group had an equal risk of mortality, the need for risk adjustment was reduced, though not eliminated entirely. Patients at the end of life are heterogenous across regions and hospitals in their age, sex, and mixture of chronic illness, requiring different clinical care. This challenge in risk adjustment would be addressed through a combination of study design and statistical modelling.
The evolution of Dartmouth end-of-life studies
In two papers published in the Annals of Internal Medicine in 2003 [49, 50], Fisher et al developed methods to use Medicare spending during the last six months of life as an exposure measure to examine whether patients living regions with higher intensity care received higher quality care or had better outcomes. First, each hospital referral region in the U.S. was assigned to a quintile of care intensity based on the End-of-Life Expenditure Index (EOL-EI), comprising Medicare spending on hospital and physician services per enrollee during their last six months of life. While exogenous to outcomes, it should be noted that the regional EOL-EI correlated highly with Medicare spending in the last two years of life. Second, cohorts of similarly ill patients—those hospitalized for hip fracture, colorectal cancer, or acute myocardial infarction—were assigned to their region of residence. Cohort characteristics were similar across the quintiles of EOL-EI. Medical care was then observed following their initial hospitalization. While patients living in regions with a high EOL-EI received 60 percent more care—largely explained by higher levels of inpatient-based and specialist-oriented care—they did not receive more recommended preventive services or appear to have greater access to care, [49] nor did they live longer, achieve better functional status, or express higher satisfaction with their care (Fig. 3) [50].
A subsequent study in Health Affairs applied the same cohorts and methodology to patients who were initially hospitalized and received most of their follow-up care at academic medical centers, finding similar results [52]; as was the case for hospital referral regions, high-intensity care at major teaching hospitals did not result in higher quality care or better survival. The authors concluded:
“Patients in the higher-intensity hospitals simply spend more time in the hospital and intensive care unit (ICU); have more frequent physician visits (especially in the inpatient setting); have more specialists involved in their care; and receive more imaging services, diagnostic testing, and minor (but not major) procedures. The similar results achieved with markedly different levels of resource inputs imply large differences in the longitudinal efficiency of chronic disease care across these hospitals.” [52]
In the same issue of Health Affairs, Wennberg et al demonstrated a method for measuring hospital-specific performance in managing chronically ill patients [53]. Building on research published earlier in the British Medical Journal [54], the researchers developed cohorts of patients with serious chronic illnesses associated with a high risk of in-hospital death [55]. Medicare patients who died with a diagnosis one of eleven chronic diseases were assigned to the hospital where they receive most of their care during the last two years of life. Utilization among the cohorts was further adjusted for the chronic illness case-mix; those who died without any evidence of chronic illness were excluded from the analysis. Both papers focused on seventy-seven hospitals that appeared on the 2001 U.S.News and World Report list of “America's best hospitals” for geriatric care, as well as care for cancer, heart, and lung disease [56].
The British Medical Journal paper documented twofold to sixfold variations in the use of medical care and resources to manage patients with chronic illness in the last six months of life among hospitals with reputations for high quality [54]. The Health Affairs paper demonstrated that the hospitals’ care patterns were consistent both for different types of patients and for earlier periods in patients’ lives. For example, utilization rates during the last six months of life were highly correlated among hospital-specific cohorts stratified by patients with cancer, congestive heart failure, and chronic obstructive pulmonary disease (Table 1); and Medicare spending per decedent during the last six months of life was closely related to per decedent spending during earlier phases of illness (Table 2) [53]. Another paper, focusing specifically on California regions and hospitals, extended the analysis to include a number of quality measures, derived from both claims data and technical quality measures posted on the Center for Medicare and Medicaid Services’ (CMS) Hospital Compare web site and summarized into quality scores by Jha et al, [57] as well as survey-based assessments of patient satisfaction with their hospital experiences [58]. Two utilization measures were portrayed as measures of care quality: the percent of patients admitted to intensive care units during the hospitalization in which they died, indicating aggressive terminal care; and the percent of patients seeing ten or more different physicians during the last six months of life, indicating the potential for poor communication and fragmented care [59]. The key findings included:
-
The volume of services (hospital days per patient) was a more important factor than the price per service (price per day) in determining overall Medicare reimbursements per decedent (R2=0.65 for volume vs. 0.39 for price);
-
The hospital most often used was a more important determinant of hospital day rates per patient than were age, race, gender, socioeconomic status, specific chronic illness, or severity of illness;
-
Neither quality nor patient satisfaction was associated with higher utilization or spending [60].
These studies advanced Dartmouth end-of-life research in several ways. First, they provided further evidence that end-of-life care intensity could be used as an independent proxy for the intensity of care overall at the regional and hospital level, not just care for dying patients. Second, they evolved the methods of risk adjustment by restricting the cohorts to patients who were similarly ill, minimizing the chances that unmeasured differences in illness levels explained the observed variations while not relying on potentially biased diagnostic coding practices. Third, they expanded the analyses from regions to specific hospitals—attributing patients to the hospitals where they actually received their care, not just to the regions in which they lived—allowing care intensity to be measured at the hospital level and focusing accountability for delivering that care on specific institutions. Finally, they introduced the strategy of evaluating the relative efficiency of medical centers and defining “best practice” benchmarks; hospitals that achieve equal or better outcomes while using fewer resources and spending less.
The 2006 edition of the Dartmouth Atlas of Health Care continued to explore these concepts, refining the methods of risk adjustment and broadening the scope to include data from not just selected regions or academic medical centers, but thousands of hospitals across the U.S. During the same period, additional peer-reviewed articles focused on specific topics in depth. A paper by Goodman et al published in Health Affairsand expanded upon in the 2006 Atlas examined physician workforce capacity—including the number of physicians per capita in a region, their work effort, and the relative proportions of primary care physicians and medical specialists—in end-of-life chronic illness cohorts, finding marked differences in hospital-level labor input (full-time equivalents per 1,000 beneficiaries) despite constant patient outcomes [51, 61]. Other studies continued to search for evidence that high-intensity end-of-life care was differentially preferred by patients in differing demographic or socioeconomic groups [62, 63], or that it was associated with higher quality [64].
A new exposure measure was introduced in the 2008 Atlas: the hospital care intensity (HCI) index, an age, sex, race, and illness-standardized score based on utilization—the number of days patients spent in the hospital and the number of physician visits they experienced as inpatients during the last two years of life [65]—rather than Medicare reimbursements, which require adjustments for regional costs of living, payments to hospitals with residency training programs, and payments to hospitals that serve a high percentage of low-income patients [66]. The HCI was used by Wennberg et al in a 2009 study in Health Affairs to compare the intensity of hospital care with measures of both technical quality and patient satisfaction, derived from the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey and published on the CMS website [67]. The results showed that patients living in regions with high-intensity care tended to rate the hospitals they used poorly, both overall and on specific measures such as whether their pain was controlled, whether the staff communicated well, and whether their rooms were clean or quiet. Lower overall ratings were also correlated with lower technical quality scores for the care of heart attack, pneumonia, and congestive heart failure patients [68]. Researchers outside of Dartmouth have also used the HCI as an exposure measure for regional and hospital care intensity [69,70,71,72,73,74].
Beyond risk adjustment: the experience of dying
Care at the end of life is not only important as a risk-adjusted proxy measure of care intensity; it is also of profound importance to the patient, family, and society. Unless death is the result of an unexpected catastrophic event, it is usually preceded by increased health care utilization intended to prolong life or improve its quality [75]. While assessing patient preferences is rarely straightforward, research has shown that patients do not generally prefer aggressive care in the face of serious chronic illness [62, 76,77,78], not only due to the challenges of disease progression and of medical treatments—such as pain, adverse events, and drug toxicity—but also the logistical burdens of increasingly intensive treatment such as frequent doctor visits and hospitalizations [79,80,81], along with higher copayments [82]. Though some patients may wish that everything possible be done to prolong their lives, most patients with a poor prognosis do not express a desire for life-sustaining treatment in the terminal phase of illness [83,84,85], nor is there evidence of a widespread preference to die in the hospital [86, 87].
Dartmouth investigators’ development of end-of-life cohorts for risk adjustment also generated descriptions of the variation in care near the very end of life that attracted widespread interest. The 2008 Atlas showed that, even among 93 prestigious hospitals included on the Council of Teaching Hospital’s list of integrated academic medical centers [88], the care delivered to patients with serious medical conditions varied remarkably. For example, among patients dying during the period from 2001 to 2005, the number of days spent in intensive care during the last six months of life varied more than eightfold, from 1.5 days per patient to 12.5, while the number of visits to medical specialists varied by a factor of nine, from 6.2 visits per patient to 55.8. There were also more than threefold variations in primary care visits per patient and in the percent of deaths that included an admission to intensive care [65]. In theory, these variations could be explained by patient preferences; however, in other studies using survey data, patient preferences have not been shown to be strongly associated with regional utilization and spending patterns [62, 89].
Later papers and reports concentrated on cohorts of cancer patients [90] and studied variations in the use of advanced life support interventions—such as endotracheal intubation, feeding tubes, and cardiopulmonary resuscitation—as well as in the use of palliative and hospice care for cancer patients with poor prognoses [91]. Little to no association was found between best practices in end-of-life cancer care and hospital characteristics such as membership in the National Comprehensive Cancer Network or designation as a comprehensive cancer center by the National Cancer Institute; in fact, uncomfortable treatments that are unlikely to prolong or enhance quality of life were more common in National Cancer Institute centers than in community hospitals [92]. A randomized clinical trial by Temel et al demonstrated that lung cancer patients receiving early palliative care integrated with standard oncologic care had better quality of life and longer survival—despite a lower likelihood of receiving aggressive end-of-life care—than those receiving standard oncologic care alone [93]. These papers extended the analysis to care that was not only potentially unnecessary, but arguably harmful.
Criticisms of Dartmouth methods
As the Dartmouth Atlas became increasingly recognized as an important source of information regarding potentially unnecessary care and opportunities for saving money in the Medicare program, the methods and conclusions came under further scrutiny. Of particular concern was the use of end-of-life cohorts rather than forward-looking cohorts. Some critics argued that even looking backward from death could not ensure that the patients were similarly ill [94, 95], while others asserted that the research ignored those who may have survived because they received more care [96, 97], despite the lack of an association between greater care intensity and average population mortality.
To address these criticisms, Dartmouth researchers developed hospital-specific, risk-adjusted prospective cohorts of acute myocardial infarction patients and followed them forward for one year from their inception event (i.e., their heart attack), measuring both expenditures and mortality and correlating these indicators with the “look-back” reimbursement rates for patients in their last two years of life [98]. Despite the differences in method, the “look-forward” and “look-back” expenditure rates were highly correlated for 144 of the largest hospitals in the United States (r=0.85) and for all 2,360 hospitals in the sample (r=0.73), reinforcing the conclusion that Medicare spending for chronically ill patients in the last two years of life is a reliable proxy for general spending intensity. In addition, there was a moderate positive correlation between both measures of spending and one-year mortality among heart attack patients, again revealing worse outcomes, on average, among patients hospitalized at institutions providing a higher level of care intensity. These results, the authors concluded, revealed something about high-spending hospitals in general:
“This does not mean that all high-cost hospitals are low-quality; some of the higher spending may be devoted to beneficial treatments, but some may well be devoted to unnecessary or cost-inefficient care, such as discretionary use of the hospital as a site of care and more frequent use of physician visits, specialist referrals, diagnostic tests, and minor procedures. And no amount of risk, price, or poverty adjustment—or argument about whether to look forward or back—will correct this problem.” [98]
A study by Cohen et al also examined the relationship between higher hospital spending and survival among acute myocardial infarction patients in Ontario, Canada [99]. While the results appeared to show that higher spending decreased the risk of death for heart attack patients, this effect was attenuated when the researchers accounted for “survivor-treatment selection bias”: the simple fact that, the longer a patient lives—and thus is available to receive treatment—the greater the potential for increased costs. This bias has the potential to overestimate the value of higher spending in terms of survival if not accounted for, and indeed, the study found that, at least for heart attack patients, “the protective effect of spending was overestimated by 62 percent… when the time-varying nature of inpatient cost was not accounted for.” [99] While it is possible that studying care intensity among retrospective cohorts of patients who died might not account for the benefit to those who survived, it is also important to recognize that higher spending might appear to benefit survivors only because they were observed for a longer period.
Recent studies
More recent research at Dartmouth and elsewhere has continued to use end-of-life measures to explore the influence of both patient factors and practice patterns on unwarranted variation in the treatment of seriously ill patients at risk of dying. In a 2018 prospective observational study, Kelley et al identified patients with a serious medical condition, calculated their one-year mortality risk, and followed them forward for one year. The results showed that, while poorer health status was—as one might expect—associated with high-intensity care in general, for patients estimated to have the highest mortality risk, residence in a region with high end-of-life spending overall was one of the strongest predictors of higher costs for these patients, irrespective of health status; excluding end-stage renal disease, health status was not significantly associated with costs in the highest-risk group. The authors stated: “in the setting of poor prognosis, nonmedical characteristics (i.e., factors likely related to discretionary treatment decisions and not medical need) have greater influences on treatment intensity.” [100]
Cutler et al linked results from patient and physician surveys to Medicare claims in a 2019 study to assess the influence of demand-side (patient) and supply-side (physician) factors in explaining variations in Medicare spending. Physicians were asked to respond to vignette-based questions about how they would manage elderly patients in various scenarios, [89] while a different survey assessed patient preferences [62]. The study concluded that, while patient demand is not a significant predictor of end-of-life spending variations, “35 percent of spending for end-of-life care and 12 percent of spending for heart attack patients (and for all enrollees) is associated with physician beliefs unsupported by clinical evidence.” [89] Both of these studies provide new approaches to measuring care intensity at the end of life and suggest that interventions on the supply side—for example, giving physicians better training to provide care consistent with high-risk patients’ values and goals [100], and better understanding how physicians’ beliefs and practice styles form and could be changed [89]—could be the focus of efforts to improve quality and reduce spending.
More recently, Zhang et al created a new measure of end-of-life care intensity—elevated end-of-life (EoL) spending, defined as the difference in Medicare spending between patients with the same predicted mortality risk who died versus those who survived—to identify potentially wasteful spending on patients that ultimately died [101]. The researchers compared this measure to Dartmouth Atlas end-of-life spending rates at the hospital referral region level. While positively correlated, the two spending estimates had important differences and captured different regional utilization patterns; nevertheless, high expenditures according to the new measure were not found to be associated with better quality, according to selected CMS Hospital Compare measures, or better outcomes for either survivors (receiving expensive but effective treatments) or decedents (receiving the type of end-of-life care they preferred, according to survey data). Again, elevated EoL spending was strongly related to physician practice patterns, particularly the use of the hospital as the site of care, suggesting that “treatment style preferences unrelated to the improvement of quality do drive elevated EoL spending.” [101] Developing prediction models to help physicians more accurately assess their patients’ prognosis and mortality risk in order to reduce wasteful spending on patients who ultimately die, the authors argued, could be another promising area of research.
The implications of end-of-life studies
Creating risk-adjusted measures of care intensity for seriously ill patients at the regional and hospital levels allowed Dartmouth investigators to focus on two key themes while at the same time reducing the potential for confounding. The first is that capacity and clinician practice styles drive utilization; one of the most important factors in determining whether chronically ill patients are more likely to receive high-intensity hospital care—or to die in a hospital—is the number of available beds per capita. This holds true for patients with different demographic characteristics, illnesses, and levels of disease severity. The second is that more care is not better; patients who receive higher-intensity care do not live longer, experience higher quality care, or express greater satisfaction with their care. Having more medical specialists involved in patient care often results in less effective care management, poor communication, and fragmented, disordered care. End-of-life studies do not show whether a particular intervention is futile, or an individual hospital delivers poor-quality care. Rather, by identifying aggressive, expensive end-of-life care as a measure of exposure to treatment that is often neither needed, wanted, nor effective, Dartmouth researchers created a proxy that could be used to identify regions and hospitals where there were opportunities for improvement.
Summary
The development of end-of-life cohorts by the Dartmouth Atlas Project provided risk-adjusted measures to show that Medicare patients with chronic illness receiving high-intensity care do not experience higher quality care, greater satisfaction, or better outcomes. The intensity of care delivered to patients at the end of life also closely correlates with overall utilization and spending at the regional level, demonstrating the effects of local factors such as hospital bed capacity, the composition of the physician workforce, and hospital and physician care processes. Regional and hospital-specific variation studies point out places where there are opportunities to improve care and save costs without negative impacts on patients.
References
Paul-Shaheen P, Clark JD, Williams D (1987) Small area analysis: a review and analysis of the North American literature. J Health Polit Policy Law 12(4):741–809
Andersen TF, Mooney G (1990) The Challenges of Medical Practice Variation. The Macmillan Press, Basingstoke, United Kingdom
Lewis CE (1969) Variations in the incidence of surgery. N Engl J Med. 281(16):880–4
Vayda E (1973) A comparison of surgical rates in Canada and in England and Wales. N Engl J Med. 289(23):1224–9
Wennberg JE, Gittelsohn A (1973) Small area variations in health care delivery: A population-based health information system can guide planning and regulatory decision-making. Science 182(4117):1102–08
Gornick M (1977) Medicare patients: geographic differences in hospital discharge rates and multiple stays. Soc Secur Bull 40(6):22–41
Deacon R, Lubitz J, Gurnick M, Newton M (1979) Analysis of variations in hospital use by Medicare patients in PSRO areas, 1974–1977. Health Care Financ Rev 1(1):79–107
Stockwell H, Vayda E (1979) Variations in surgery in Ontario. Med Care 17(4):390–6
Rothberg DL, Pinto J, Gertman PM (1980) Medicare utilization in the United States: PSRO and regional impacts. Health Serv Res 15(3):249–70
Roos NP, Roos LL (1981) High and low surgical rates: risk factors for area residents. Am J Public Health 71(6):591–600
Gornick M (1982) Trends and regional variations in hospital use under Medicare. Health Care Financ Rev 3(3):41–73
Lubitz J, Deacon R (1982) The rise in the incidence of hospitalizations for the aged, 1967 to 1979. Health Care Financ Rev 3(3):21–40
McPherson K, Wennberg JE, Hovind OB, Clifford P (1982) Small-area variations in the use of common surgical procedures: an international comparison of New England, England, and Norway. N Engl J Med 307(21):1310–4
Roos NP, Roos LL Jr (1982) Surgical rate variations: do they reflect the health or socioeconomic characteristics of the population? Med Care 20(9):945–58
Wennberg J, Gittelsohn A (1982) Variations in medical care among small areas. Sci Am 246(4):120–34
Chassin MR, Brook RH, Park RE et al (1986) Variations in the use of medical and surgical services by the Medicare population. N Engl J Med 314(5):285–90
Roos NP, Flowerdew G, Wajda A, Tate RB (1986) Variations in physicians’ hospitalization practices: a population-based study in Manitoba Canada. Am J Public Health 76(1):45–51
Wennberg JE, Freeman JL, Culp WJ (1987) Are hospital services rationed in New Haven or over-utilised in Boston? Lancet 1(8543):1185–9
Wennberg JE, Freeman JL, Shelton RM, Bubolz TA (1989) Hospital use and mortality among Medicare beneficiaries in Boston and New Haven. N Engl J Med 321(17):1168–73
Clark JD (1990) Variation in Michigan hospital use rates: do physician and hospital characteristics provide the explanation? Soc Sci Med 30(1):67–82
Latta VB, Keene RE (1990) Use and cost of short-stay hospital inpatient services under Medicare, 1988. Health Care Financ Rev 12(1):91–9
Fisher ES, Wennberg JE, Stukel TA, Sharp SM (1994) Hospital readmission rates for cohorts of Medicare beneficiaries in Boston and New Haven. N Engl J Med 331(15):989–95
Brook RH, Lohr K, Chassin M, Kosecoff J, Fink A, Solomon D (1984) Geographic variations in the use of services: do they have any clinical significance? Health Aff (Millwood) 3(2):63–73
Diehr P (1984) Small area statistics: large statistical problems. Am J Public Health 74(4):313–4
Diehr P, Cain K, Connell F, Volinn E (1990) What is too much variation? The null hypothesis in small-area analysis. Health Serv Res 24(6):741–71
Shwartz M, Ash AS, Anderson J, Iezzoni LI, Payne SM, Restuccia JD (1994) Small area variations in hospitalization rates: how much you see depends on how you look. Med Care 32(3):189–201
Wennberg JE, Gittelsohn A, Soule D (1975) Health care delivery in Maine II: conditions explaining hospital admission. J Maine Med Assoc. 66(10):255–61, 269
Wennberg JE, Fowler FJ Jr (1977) A test of consumer contribution to small area variations in health care delivery. J Maine Med Assoc 68(8):275–9
Wennberg JE (1984) Dealing with medical practice variations: a proposal for action. Health Aff (Millwood) 3(2):6–32
McPherson K, Strong PM, Jones L, Britton BJ (1985) Do cholecystectomy rates correlate with geographic variations in the prevalence of gallstones? J Epidemiol Community Health 39(2):179–82
Chassin MR, Kosecoff J, Park RE et al (1987) Does inappropriate use explain geographic variations in the use of health care services? A study of three procedures. JAMA 258(18):2533–7
Wennberg JE, Mulley AG Jr, Hanley D et al (1988) An assessment of prostatectomy for benign urinary tract obstruction. Geographic variations and the evaluation of medical care outcomes. JAMA 259(20):3027–30
Wennberg JE, McPherson K, Caper P (1984) Will payment based on diagnosis-related groups control hospital costs? N Engl J Med 311(5):295–300
Roos NP, Wennberg JE, McPherson K (1988) Using diagnosis-related groups for studying variations in hospital admissions. Health Care Financ Rev 9(4):53–62
Hershey JC, Luft HS, Gianaris JM (1975) Making sense out of utilization data. Med Care 13(10):838–54
Luft HS, Hershey JC, Morrell J (1976) Factors affecting the use of physician services in a rural community. Am J Public Health 66(9):865–71
Moore FD (1985) Small area variations studies: illuminating or misleading? Health Aff (Millwood) 4(1):96–101
McMahon LF, Jr., Wolfe RA, Griffith JR, Cuthbertson D. Socioeconomic influence on small area hospital utilization. Med Care. 1993;31(5 Suppl):Ys29-36.
Stano M (1993) Evaluating the policy role of the small area variations and physician practice style hypotheses. Health Policy 24(1):9–17
Bronner KK, Goodman DC (2022) The Dartmouth Atlas of Health Care – bringing health care analyses to health systems, policymakers, and the public. Res Health Serv Regions 1(1):6
Mor V, Hiris J (1983) Determinants of site of death among hospice cancer patients. J Health Soc Behav 24(4):375–85
Moinpour CM, Polissar L (1989) Factors affecting place of death of hospice and non-hospice cancer patients. Am J Public Health 79(11):1549–51
Wennberg JE, Cooper MM (1996) Hospital Capacity, Utilization, and Outcomes. Inc, The Dartmouth Atlas of Health Care. American Hospital Publishing, Chicago, p. 54–56
Wennberg JE, Cooper MM (1998) “The American Experience of Death”. The Dartmouth Atlas of Health Care 1998. American Hospital Publishing. Inc, Chicago, IL, p.82–97
Wennberg JE, Cooper MM (1999) “The Quality of Care in the Last Six Months of Life”. The Quality of Medical Care in the United States: A Report on the Medicare Program. The Dartmouth Atlas of Health Care 1999. American Hospital Publishing. Inc, Chicago, IL, p.176–199
Fisher ES, Wennberg JE, Stukel TA et al (2000) Associations among hospital capacity, utilization, and mortality of US Medicare beneficiaries, controlling for sociodemographic factors. Health Serv Res 34(6):1351–62
Gornick ME, Eggers PW, Reilly TW et al (1996) Effects of race and income on mortality and use of services among Medicare beneficiaries. N Engl J Med 335(11):791–9
Hofer TP, Wolfe RA, Tedeschi PJ, McMahon LF, Griffith JR (1998) Use of community versus individual socioeconomic data in predicting variation in hospital use. Health Serv Res 33(2 Pt 1):243–59
Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL (2003) The implications of regional variations in Medicare spending. Part 1: the content, quality, and accessibility of care. Ann Intern Med 138(4):273–87
Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL (2003) The implications of regional variations in Medicare spending. Part 2: health outcomes and satisfaction with care. Ann Intern Med 138(4):288–98
Wennberg JE, Fisher ES (2006) The Care of Patients with Severe Chronic Illness: A Report on the Medicare Program by the Dartmouth Atlas Project. The Dartmouth Atlas of Health Care 2006.The Dartmouth Institute for Health Policy & Clinical Practice, NH, Lebanon
Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ. Variations in the longitudinal efficiency of academic medical centers. Health Aff (Millwood). 2004;Suppl Variation:Var19-32.
Wennberg JE, Fisher ES, Stukel TA, Sharp SM. Use of Medicare claims data to monitor provider-specific performance among patients with severe chronic illness. Health Aff (Millwood). 2004;Suppl Variation:Var5-18.
Wennberg JE, Fisher ES, Stukel TA, Skinner JS, Sharp SM, Bronner KK (2004) Use of hospitals, physician visits, and hospice care during last six months of life among cohorts loyal to highly respected hospitals in the United States. BMJ 328(7440):607
Iezzoni LI, Heeren T, Foley SM, Daley J, Hughes J, Coffman GA (1994) Chronic conditions and risk of in-hospital death. Health Serv Res 29(4):435–60
America’s Best Hospitals. US News and World Report. 2001;131(3)
Jha AK, Li Z, Orav EJ, Epstein AM (2005) Care in U.S. hospitals–the Hospital Quality Alliance program. N Engl J Med 353(3):265–74
What Patients Think of California Hospitals: A Consumer Guide. California Institute for Health Systems Performance and California HealthCare Foundation; 2004.
Baicker K, Chandra A (2004) The productivity of physician specialization: evidence from the Medicare program. Am Econ Rev 94(2):357–61
Wennberg JE, Fisher ES, Baker L, Sharp SM, Bronner KK. Evaluating the efficiency of California providers in caring for patients with chronic illnesses. Health Aff (Millwood). 2005;Suppl Web Exclusives:W5-526-43.
Goodman DC, Stukel TA, Chang CH, Wennberg JE (2006) End-of-life care at academic medical centers: implications for future workforce requirements. Health Aff (Millwood) 25(2):521–31
Barnato AE, Herndon MB, Anthony DL et al (2007) Are regional variations in end-of-life care intensity explained by patient preferences? A study of the US Medicare population. Med Care 45(5):386–93
Barnato AE, Anthony DL, Skinner J, Gallagher PM, Fisher ES (2009) Racial and ethnic differences in preferences for end-of-life treatment. J Gen Intern Med 24(6):695–701
Yasaitis L, Fisher ES, Skinner JS, Chandra A (2009) Hospital quality and intensity of spending: is there an association? Health Aff (Millwood) 28(4):w566-72
Wennberg JE, Fisher ES, Goodman DC, Skinner JS (2008) Tracking the Care of Patients with Severe Chronic Illness: The Dartmouth Atlas of Health Care 2008. The Dartmouth Institute for Health Policy & Clinical Practice, Lebanon, NH
Acute Inpatient PPS. Centers for Medicare & Medicaid Services. Accessed 16 Aug 2023. https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS
Medicare Provider Data. Centers for Medicare & Medicaid Services. Accessed 16 Aug 2023. https://data.cms.gov/provider-data
Wennberg JE, Bronner K, Skinner JS, Fisher ES, Goodman DC (2009) Inpatient care intensity and patients’ ratings of their hospital experiences. Health Aff (Millwood) 28(1):103–12
Kelley AS, Ettner SL, Wenger NS, Sarkisian CA (2011) Determinants of death in the hospital among older adults. J Am Geriatr Soc 59(12):2321–5
Ryskina KL, Korenstein D, Weissman A, Masters P, Alguire P, Smith CD (2014) Development of a high-value care subscore on the internal medicine in-training examination. Ann Intern Med 161(10):733–9
Sheetz KH, Dimick JB, Ghaferi AA (2014) The association between hospital care intensity and surgical outcomes in medicare patients. JAMA Surg 149(12):1254–9
Ryskina KL, Halpern SD, Minyanou NS, Goold SD, Tilburt JC (2015) The role of training environment care intensity in US physician cost consciousness. Mayo Clin Proc 90(3):313–20
Ruhnke GW, Manning WG, Rubin DT, Meltzer DO (2017) The drivers of discretionary utilization: clinical history versus physician supply. Acad Med 92(5):703–708
Leep Hunderfund AN, Starr SR, Dyrbye LN et al (2019) Imprinting on clinical rotations: multisite survey of high- and low-value medical student behaviors and relationship with healthcare intensity. J Gen Intern Med 34(7):1131–1138
McCall N (1984) Utilization and costs of Medicare services by beneficiaries in their last year of life. Med Care 22(4):329–42
McQuellon RP, Muss HB, Hoffman SL, Russell G, Craven B, Yellen SB (1995) Patient preferences for treatment of metastatic breast cancer: a study of women with early-stage breast cancer. J Clin Oncol 13(4):858–68
Brundage MD, Davidson JR, Mackillop WJ (1997) Trading treatment toxicity for survival in locally advanced non-small cell lung cancer. J Clin Oncol 15(1):330–40
Silvestri G, Pritchard R, Welch HG (1998) Preferences for chemotherapy in patients with advanced non-small cell lung cancer: descriptive study based on scripted interviews. BMJ 317(7161):771–5
Jani B, Blane D, Browne S et al (2013) Identifying treatment burden as an important concept for end of life care in those with advanced heart failure. Curr Opin Support Palliat Care 7(1):3–7
Ahmed N, Ahmedzai SH, Harwood RH (2021) The geriatrician’s role in end-of-life care. Age Ageing 50(2):366–369
Siddiqui A, Ornstein KA, Ankuda CK (2021) Prevalence of treatment burden in the last three years of life. J Palliat Med 24(6):879–886
Kelley AS, McGarry K, Fahle S, Marshall SM, Du Q, Skinner JS (2010) Out-of-pocket health care expenditures at the end of life. Natl Bur Econ Res Bull Aging Health 2:3–4
Frankl D, Oye RK, Bellamy PE (1989) Attitudes of hospitalized patients toward life support: a survey of 200 medical inpatients. Am J Med 86(6):645–8
Emanuel LL, Barry MJ, Stoeckle JD, Ettelson LM, Emanuel EJ (1991) Advance directives for medical care–a case for greater use. N Engl J Med 324(13):889–95
Singer PA, Lowy FH (1992) Rationing, patient preferences, and cost of care at the end of life. Arch Intern Med 152(3):478–80
Groth-Juncker A, McCusker J (1983) Where do elderly patients prefer to die? Place of death and patient characteristics of 100 elderly patients under the care of a home health care team. J Am Geriatr Soc 31(8):457–61
Townsend J, Frank AO, Fermont D et al (1990) Terminal cancer care and patients’ preference for place of death: a prospective study. BMJ 301(6749):415–7
List provided by the Council of Teaching Hospitals and Health Systems (COTH). https://www.aamc.org/career-development/affinity-groups/coth
Cutler D, Skinner JS, Stern AD, Wennberg D (2019) Physician beliefs and patient preferences: a new look at regional variation in health care spending. Am Econ J Econ Policy 11(1):192–221
Berke EM, Smith T, Song Y, Halpern MT, Goodman DC (2009) Cancer care in the United States: identifying end-of-life cohorts. J Palliat Med 12(2):128–32
Goodman DC, Fisher ES, Chang CH, et al (2010) Quality of End-of-Life Cancer Care for Medicare Beneficiaries: Regional and Hospital-Specific Analyses. The Dartmouth Atlas of Health Care. The Dartmouth Institute for Health Policy & Clinical Practice, Lebanon
Morden NE, Chang CH, Jacobson JO et al (2012) End-of-life care for Medicare beneficiaries with cancer is highly intensive overall and varies widely. Health Aff (Millwood) 31(4):786–96
Temel JS, Greer JA, Muzikansky A et al (2010) Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med 363(8):733–42
Bach PB, Schrag D, Begg CB (2004) Resurrecting treatment histories of dead patients: a study design that should be laid to rest. JAMA 292(22):2765–70
Bach PB (2010) A map to bad policy–hospital efficiency measures in the Dartmouth Atlas. N Engl J Med 362(7):569–73 (discussion p 574)
Ong MK, Mangione CM, Romano PS et al (2009) Looking forward, looking back: assessing variations in hospital resource use and outcomes for elderly patients with heart failure. Circ Cardiovasc Qual Outcomes 2(6):548–57
McCaughey B. Treating seniors as ‘clunkers’. The New York Post. October 27, 2009. https://nypost.com/2009/10/27/treating-seniors-as-clunkers/
Skinner J, Staiger D, Fisher ES (2010) Looking back, moving forward. N Engl J Med 362(7):569–74 (discussion 574)
Cohen D, Manuel DG, Tugwell P, Sanmartin C, Ramsay T (2015) Does higher spending improve survival outcomes for myocardial infarction? Examining the cost-outcomes relationship using time-varying covariates. Health Serv Res 50(5):1589–605
Kelley AS, Bollens-Lund E, Covinsky KE, Skinner JS, Morrison RS (2018) Prospective identification of patients at risk for unwarranted variation in treatment. J Palliat Med 21(1):44–54
Zhang Y, Gupta A, Nicholson S, Li J (2023) Elevated end-of-life spending: a new measure of potentially wasteful health care spending at the end of life. Health Serv Res 58(1):186–194
Acknowledgments
The authors would like to thank Jonathan Skinner, PhD for his suggestions in the development of this manuscript.
Statements and declarations
David Goodman was a founding investigator of the Dartmouth Atlas and served over the next 25 years as investigator, author, and more recently, principal investigator. Kristen Bronner has been a member of the Dartmouth Atlas team since 1995. The authors did not receive support from any organization for the submitted work. No funding was received to assist with the preparation of this manuscript. David Goodman is an honorary member of the Research in Health Services & Regions Editorial Board.
Author information
Authors and Affiliations
Contributions
Kristen Bronner and David Goodman contributed to the study conception and design. The first draft of the manuscript was written by Kristen Bronner, and David Goodman commented on all versions of the manuscript. Both authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Bronner, K.K., Goodman, D.C. End-of-life cohorts from the Dartmouth Institute: risk adjustment across health care markets, the relative efficiency of chronic illness utilization, and patient experiences near the end of life. Res Health Serv Reg 3, 4 (2024). https://doi.org/10.1007/s43999-024-00039-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s43999-024-00039-9