People want to know the future. While too much information can be unpleasant, in general people like predictions. Just look at the daily newspaper: the horoscopes, the weather and sports pages, the financial section, and even the political columns are all devoted, at least in part, to making guesses about the future.

Why not? We would all like to know what is coming. But the natural human tendency to seek certainty produces a cognitive bias: people seem happier with a prediction in hand, even if that prediction is apt to be wrong. And that bias can cause problems.

This notion came to mind while reading a recent paper [4] that predicted that 3.48 million primary knee arthroplasties will be performed in the year 2030. While we will not know for a while whether the authors are correct, we do know the orthopaedic community took heed. According to my review of Thomson Reuters’ Web of Knowledge, as of October 2012, that paper had been cited 497 times—more than any other orthopaedic surgery manuscript in the last 5 years.

The popularity of this manuscript is understandable, as the number of knee arthroplasties anticipated in the coming decades is important information. As the authors noted, their analysis can “provide a quantitative basis for future policy decisions related to the numbers of orthopaedic surgeons needed” [4].

On the other hand, trying to project the number of joint arthroplasties in 2030 might be a fool’s errand. Sure, one can make projections, but how accurate will they be? The prevalence of osteoarthritis can be estimated, but only imprecisely; the perceived severity of disease and the impairment imposed are even harder to reckon. Any estimate of surgery based on disease prevalence will have to be adjusted for diagnostic sensitivity, access to health care, patient attitudes toward risk, new nonsurgical treatments that might be developed, and other factors. Even if we held a complete list of considerations (we don’t) and even if we could predict all of them with 95% accuracy (we can’t), the composite estimate, compounding the error of each individual metric, might reduce the final prediction to little more than statistical noise.

The real problem with mathematical modeling, however, is that it does not consider what Nassim Taleb [6] terms “black-swan events.” Taleb defines these as low-probability, seemingly random occurrences that exert potentially large effects on the outcome. Black-swan events can make hash out of the predicted demand for arthroplasty in 2030. For example, imagine the president of the United States in 2026 undergoes a routine TKA but dies on the table. It is easy to imagine the resulting adverse publicity depressing demand for arthroplasty by 10% for the following year. Of course, this example superimposes an extremely rare event (intraoperative death) on an unlikely event (that a US president will undergo a knee arthroplasty while in office), but there is probably an infinite set of events, well beyond the reach of conventional modeling techniques, that could upend any prediction about the demand for surgery. And while any single one of these events is improbable—by definition—the effect exerted by the entire set of events is to render the prediction model highly suspect if not worthless. Accordingly, to paraphrase Taleb, if the only choice we have is to use a defective prediction model, we might be better off with no prediction model at all.

In the end, the real question is not whether 3.48 million knee arthroplasties is our best guess (for it probably is), but whether that predication is good enough to serve as a basis for action today. I say it is not.

Maybe I am wise, or maybe just lazy, but I did not change professions after reading the 1998 manuscript [5] that predicted a large glut of orthopaedic surgeons by 2010. Similarly, even if you are not lazy, I believe the right action in response to reading the paper predicting 3.5 million knees in 2030 is inaction as well. Predicting the number of knee arthroplasties two decades hence makes about as much sense as predicting the April weather in Boston that far into the future. These kinds of guesses are more divination than science.

Commentary

Richard Iorio MD

Chief of Adult Reconstruction, William and Susan Jaffe Professor of Orthopaedic Surgery, Department of Orthopaedic Surgery, NYU Langone Medical Center, Hospital for Joint Diseases, New York, NY, USA

“Fixing?” the Future

In 2007, the forecast of 3.48 million annual primary TKA procedures in the United States by 2030 [4] caused a great deal of healthcare policy discussion. As Dr. Bernstein opined, this prediction was based on the best data available and may or may not prove to be accurate. However, much like the observer effect described in experimental physics, once the concept of a potential glut of expensive arthroplasty procedures was introduced, healthcare policy decision makers decided the potential increase in TKA procedures was excessive and since have attempted to moderate predicted growth.

Recent data have shown the growth rate of TKA intervention has been tempered by factors that were not considered in the 2007 forecast. The economic downturn, which began in 2008, caused the number of arthroplasty procedures performed to remain flat until recently. In particular, 40 to 60 year-old patients—those most affected by the economic crisis—had fewer joint arthroplasties than they were predicted to because of insurance and employment concerns. In the new millennium, the economy will increasingly affect healthcare decisions made by patients, physicians, and healthcare delivery systems.

The Centers for Medicare & Medicaid Services has decreased reimbursement for arthroplasties over the past 20 years. Despite this obvious attempt to influence intervention rates, surgeons continue to deliver high-quality arthroplasty services to seniors. This is a testament to the commitment of orthopaedic surgeons to their patients and an endorsement of the utility of arthroplasty as an unparalleled quality of life-improving intervention.

Unfortunately, decreasing reimbursements for arthroplasty surgery has reduced interest among trainees for adult reconstruction as a career choice. Dropping practice expense payments, Sustainable Growth Rate cuts, and reduced surgeon fees may also cause some surgeons to limit the number of Medicare patients seen in their practices.

The Patient Protection and Affordable Care Act (PPACA) will provide some form of insurance coverage to 30 to 60 million more Americans. Politics aside, one thing is clear: When insurance coverage is provided to populations, they will use it. This trend has been demonstrated in Massachusetts with the near-universal insurance coverage established by Governor Mitt Romney. Utilization has increased along with costs. Increases in demand for services from the newly insured and the growing senior population will affect national arthroplasty rates under the PPACA.

Forecasting under these conditions is difficult, and predictions can be wrong for many reasons. In the case of arthroplasty intervention rates, economic and political forces unimaginable in 2007 have caused policy makers, payers, and delivery systems to attempt to “fix” the future by decreasing patient access to arthroplasty surgeons through decreasing Medicare reimbursement. It remains to be seen if this “fix” is in the best interest of patients.

Commentary

Steven Kurtz PhD, Kevin Ong PhD

Exponent, Inc, Philadelphia, PA, USA

Edmund Lau MS

Exponent, Inc, Menlo Park, CA, USA

The Folly of Not Forecasting

“Plans are worthless, but planning is everything” [1]

Dwight D. Eisenhower

Planning is an essential part of human endeavor, but so is the urge to live in the moment and not worry about the future. We should not confuse either of these philosophical approaches with science, but at times philosophizing about one’s scientific profession can be illuminating, inspiring, and even controversial. So let us not debate about whether or not it is foolish to plan for the future. It is an undisputed fact that organizations such as the military, governments, and even societies of surgeons plan for the future. Take, for example, the latest US Census projections, which were generated up to 2050. Another example is the annual report of the trustees of the Federal Hospital Insurance Trust Fund, which projects the future solvency (or lack thereof) for Social Security and Medicare beyond 2080. Who would criticize the US Census Bureau or the trustees responsible for the financial stability of Medicare for the audacity of projecting 40 or even 70 years in the future?

Black swans notwithstanding, we should accept planning activities, such as projections, for what they are: instruments of and for public policy. The first projections of arthroplasty rates for the United States were undertaken by Frankowski and Watkins-Castillo from the American Academy of Orthopaedic Surgeons (AAOS) [2]. These projections, up to the year 2030, were published on the AAOS website in 2002 and circulated at orthopaedic meetings, which is how our research team was exposed to them. We studied their description of the input data and modeling assumptions, which were well described, and developed an alternative modeling approach, which we subsequently published in The Journal of Bone and Joint Surgery American Volume [4]. We chose 2030, not out of hubris, but so we could compare the projections from our model with those published by the AAOS’s statisticians.

Projections can be undertaken scientifically, with well-defined inputs and assumptions, and exercised to explore their sensitivity. But the only way to really know whether they are valid predictions is to compare the model, at a later date, with actual real world results. A comparison was made between published projections of the number of hip and knee arthroplasties in 2005 and 2010 [2] and actual national estimates available from the Nationwide Inpatient Sample [3] (Table 1). Considering both time points, the model projections are either consistent or conservatively less than the national estimates for those years.

Table 1 Comparison of the published projected incidence of primary and revision total joint arthroplasty [2] with actual Nationwide Inpatient Sample data [3]
Full size table

As Charles Dudley Warner, a good friend of Mark Twain, often remarked, “Everybody talks about the weather, but nobody does anything about it.” The same might be said about pundits and their criticisms of projections. But whoever argues about the supposed impossibility of a projection’s accuracy is really missing the point. And that point is not how accurate the projections are, or will be, but how to develop a toolbox for the orthopaedic community that will allow us take a more active role in public policy for the sake of the next generation of surgeons and their patients.