Predicting the future is easy. Predicting the future accurately, is hard.
Public policy deliberations about initiatives for improving healthcare delivery and financing are often handcuffed by an over-reliance on the accuracy of projections. This happens because estimates of costs, disease prevalence, utilization rates, etc. are embraced as descriptions of inevitable futures, rather than as well-executed analytical projections with inherent probability ranges. This metamorphosis from estimation to “factation” occurred though a predictable sequence:
- Quantitative analyses yield estimates;
- Estimates are published or presented;
- Summaries of estimates are extracted from tables and slides;
- These summaries – or “bottom lines” – are rhetorically converted by the media and others from “projections” and “estimates,” to what “will happen,” as in, “healthcare spending will be….” without any recognition of the underlying assumptions or likelihood of significant deviation.
And just like that, well constructed estimates built on assumptions of “what would happen assuming” scenarios are treated as inevitable and de facto futures. (Note – This scenario can also occur for private sector initiatives.)
“Everyone is entitled to his own opinion, but not to his own facts.”
US Senator Daniel Patrick Moynihan
Experience Can Lead to Clarity of Vision
Henry Aaron, the very experienced and respected Brookings Institution scholar, eloquently commented at an April 19th Atlantic Magazine Forum on the challenges of projecting healthcare spending – and particularly the expectations for savings from the ACA. [Click below for video of his Q&A on this topic – go to time code 36.00 minutes – or here for the Forum’s webpage and select Panel 1.]
Below are some of Henry Aaron’s comments:
“There is an enormous range of uncertainty. A very flat probability distribution that stretches a long way on both sides of the central estimate that CBO is making.”
The ACA “is first and foremost a declaration that the status quo is intolerable. And that we have to begin to try to do different things both to improve coverage and to change the delivery system… ”
And he concludes, “The key aspect of the bill is that we have started on a journey of change in our delivery system. A lot of the elements can’t be stopped. They have deeper roots than legislation. But don’t underestimate the importance of this legislation and the message that it sends.”
We Don’t Want to Repeat History – But Projections Are Dependent on Historical Precedents
The fewer precedents for a proposed change, the harder it is to accurately model the change’s effects. But failing to recognize this limitation of the process is to not learn from history. Below are several examples of how projections have deviated far from their original estimates or have changed as the underlying assumptions are updated:
- CBO projections for the 10 year savings from the ACA’s Independent Medicare Payment Advisory Board went from about $15 Billion in 2010, (shortly after the law’s enactment), to $2.8 billion in May 2011, to $3.1 billion in March 2012.
- The Balanced Budget Act of 1997 included provisions to increase geographic access to Medicare managed care plans – which the law also renamed Medicare+Choice (M+C). However, the opposite occurred. As noted in a 2002 research paper, “[T]he number of contracted M+COs has dropped, from a high of 456 in December 1998 to 253 in December 2001. Total enrollment in M+COs has dropped from 6.7 million in December 1998 to slightly over 6 million in December 2001.”¹ In addition, beneficiary cost sharing went up for those individuals who remained in M+C plans rather than return to “traditional” Medicare. [Note – the M+C program was significantly revised and renamed Medicare Advantage (MA) in the Medicare Prescription Drug, Improvement, and Modernization Act of 2003. These changes created significant growth in enrollment so that today about 25% of Medicare enrollees are in MA plans.]
- The overall costs and individual monthly premiums for the Medicare Part D prescription drug program have been much lower than projected. Competition from a greater number of plan choices may have kept the monthly premiums down, and it seems that lower enrollment numbers have kept the government costs below projections.
- Estimates for national healthcare spending created in 2007 were significantly higher than what occurred in 2008-2010 – in part because of the unforeseen economic downturn. (I’ll discuss this and other factors in greater detail in my next post.)
Bottom Line: Don’t be Fooled – Don’t be Foolish
Policy makers should strive to obtain the best possible analyses to guide, (but not dictate), their planning and actions. They should also scrutinize and question the assumptions underlying those analyses so when important factors change, they aren’t fooled into foolishly thinking that the previous projections are still valid.
“Half of what we know is wrong, we just don’t know which half.”
Day One Lecture to First Year Medical Students
1. Graham S. Updating Medicare Managed Care. Health Policy Newsletter 2002; 15(1): Article 13. Retrieved 5/19/2012 from http://jdc.jefferson.edu/hpn/vol15/iss1/13.
