Yogi Berra once famously said, “It is hard to make predictions, especially about the future.” We agree: There is too much attention paid to forecast results, and too little to the underlying assumptions. That can make for bad policy outcomes by biasing the political process in favor of initiatives that sound attractive, but which are only effective if all the assumptions in the forecast turn out to be true … which never, ever happens.

Yogi was brought to mind by three recent forecasting exercises. The first study —though you would not have seen this outside of the trade press — said that the Clean Power Plan (CPP) would:

  • Only reduce the electricity demand increase between 2012 and 2030 from 14.5% to 12%, and not from 8% to 1.6% as EPA has claimed;
  • Reduce GDP by a cumulative $535 billion (in 2015$) between 2020-2030 (annually averaging about a 0.3% loss of GDP)
  • Increase natural gas prices by an extra 14% in 2030 compared to the no-CPP case, rather than the 2% decrease EPA claimed for its favored CPP “mass” case.

This forecast—the 2016 Annual Energy Outlook Early Release (AEO)—is produced the Energy Information Agency (EIA), another arm of the Obama administration. Remarkably, the Administration successfully spun the 2016 AEO to the media as confirming the efficacy of the CPP despite the fact that the forecast contradicts most, if not all, of EPA’s economic analysis in the CPP’s Regulatory Impact Analysis. And everything would look even worse if EIA had not assumed that every state would adopt EPA’s cap-and-trade model and return all the revenue to their electricity consumers to offset rate rises. The chances of this happening in the real world of state politics is somewhere between zero and nil. Some argue that things might look better had EIA assumed anything beyond evolutionary technological change (something they are prevented by Congress from doing in the course of these forecasting exercises) but as EIA often points out, by definition you can’t predict when such changes will occur – or how fast they will be adopted. It seems to us that EIA’s approach is the appropriate basis for forecasts that shape a country’s overall energy policies.

The second forecast came in a presentation by Michael Liebreich, the Advisory Board Chairman of Bloomberg New Energy Finance (BNEF), at its recent gathering in New York. Liebreich had many interesting things to say, but we want to highlight BNEF’s forecasts for solar penetration — 99 million solar rooftops by 2020 — and electric vehicles — cost parity with conventional vehicles by 2022 and 50% of new vehicle sales globally by 2040. Liebreich’s basic argument is that well-informed people (like Bill Gates, initially) underestimated the growth of the Internet—partly because of Moore’s law—and they are likewise underestimating the growth potential of solar, wind and EVs. So their forecast is the antithesis of the EIA approach in that it assumes far more technological progress than is assumed by EIA.

Our point about the EIA and BNEF forecasts is simply that forecasts all depend entirely on their assumptions. Whose assumptions are more likely to prove correct? Who knows. The energy market is enormous, complex, and influenced by many factors. More complexity increases the modeling challenge for forecasters and requires even more assumptions, each of which have error bands that get larger the longer the time frame of the prediction. Vaclav Smil wrote about this in an excellent article, based on his own—and many others’—real world forecasting failures.

What is surprising is not that such forecasts are wrong, but that they are ever right.  Yet we all have to live with, and make decisions based on, imperfect forecasts and assumptions. Both the reason for climate action (climate models) and the measures of that action (economic, energy and emission forecasts) are controlled by such forecasts. Each of these seeks to predict circumstances orders of magnitude more complex those covered by EIA (U.S. electricity demand in 2030) and BNEF (demand for solar power and EVs in the next couple of decades). Think of the models attempting to predict world energy demand in 2080, or the economic impacts of (uncertain) temperature increases 300 years from now (that would be the Social Cost of Carbon).

Another forecast out this month, from the North American Electric Reliability Corporation (NERC), emphasizes the point. It sees U.S. electricity demand increasing by 0.61% per year under their base case, and 0.31% under their CPP case. Thus overall demand in 2030 is 10% higher than today absent the CPP, and 5% with it. They show that CPP’s lower level of demand requires that all of the “High Achievable” efficiency improvements identified by the Electric Power Research Institute (EPRI) in a well-regarded 2014 study are achieved. Meeting that demand reduction estimate requires the full use of the very limited scope the CPP draft model state plan rule allows for new fossil (natural gas) generating capacity. Getting this right is essential given the implications for costs, reserve margins, and capacity retirement decisions.

We think the evidence overwhelmingly suggests that a policy whose effectiveness depends on accurate forecasting is bound to fail. In particular, trying to forecast energy demand—let alone precisely how that demand will be met decades into the future—is a mug’s game, as Smil well demonstrates. Even if governments produce honest forecasts that aren’t shaped by vested interests or topicality bias (forecasting the future based on the fashionable technology, or crisis, du jour), they inevitably get it wrong. When they do, you end up with either very high or low prices that undermine the entire policy, or produce other unintended consequences, like power outages or emission leakage. The supposed precision of the forecast is one attraction of cap-and-trade to environmentalists, and exactly the premise (and the reasons for the failure) of the EU ETS system, and the entire basis for the CPP.

Hence, one of the reasons for our enthusiasm for revenue neutral carbon taxes. Any climate policy is prone to unexpected outcomes. The tax, in contrast to cap-and-trade regime, allows policy makers to periodically adapt to the changes and mistaken assumptions that will emerge. It allows investors to choose the lowest cost options without the unforeseen system effects that endanger both the economy and the survival of the policy. And it avoids (or at least minimizes) the political food fights that inevitably ensue from policy changes  in the “allowance trading” and “picking technology winners” approaches.

Or, as Yogi put it, “In theory there’s no difference between theory and practice. In practice, there is.”

Image by 12019 from Pixabay.