The Scandal of Prediction: Why Forecasts Fail

There's a quiet scandal running through every major organization on Earth. Billions of dollars flow each year to forecasters, analysts, consultants, and planning departments — all producing predictions that perform no better than a coin flip. And yet the industry grows.

This isn't a new problem. It isn't a failure of the latest technology or method. It is structural. And once you see it, you'll notice it everywhere: in your company's annual forecast, in the oil market consensus, in Silicon Valley's "10-year roadmaps," in the Federal Reserve's famous dot plot. The scandal of prediction is that the predictors know, at some level, that they are guessing — but organizations need numbers, so the guessing continues.

I learned this the hard way as someone who reads forecasts, relies on them, and occasionally has to trust them. The deeper I looked, the less I could escape a straightforward conclusion: treat any point forecast in an Extremistan domain as approximately worthless.

What the Research Actually Shows

The evidence is unambiguous, and it's been there for decades. Philip Tetlock, a political scientist, spent twenty years collecting 28,000 forecasts from 284 experts in politics, economics, and geopolitics. His conclusion: the experts performed barely better than chance. More damning — the more famous the expert, the worse their calibration. Bold predictions, the kind that get attention and build reputations, tend to miss.

Other researchers found similar patterns: - Security analysts' stock recommendations perform worse than a simple "buy the index" baseline - Economists' GDP forecasts miss by large and consistent margins - Corporate earnings projections disappoint almost systematically

The pattern extends beyond academia. When you track forecasts against outcomes, the hits are unremarkable — "we said growth would be 2%, it turned out to be 2.1%" — while the misses are spectacular. But by then, the forecasters have already moved on to the next prediction, the previous forecast is forgotten, and the organizational need for numbers continues.

This is the mechanism that keeps the scandal alive.

Why Forecasts Fail: The Corporate Annual Ritual

Every major corporation does this: in the fall, the finance team convenes. Revenue targets are debated. EBITDA margins are projected. Growth rates by segment are estimated. The numbers are refined, debated by committees, approved by the board, and communicated to shareholders with confidence. These numbers are produced with effort.

Then outcomes are measured. Actual results miss the forecast — usually systematically, almost always in the direction of being too optimistic, and in ways the original forecast did not flag as possible.

What happens next? Next year, the process runs again. Same methodology. Same confidence. Same systematic disappointment.

The scandal is not that the forecasts are wrong. Human prediction is hard in complex domains; some error is inevitable. The scandal is that the forecasting process, having been proven repeatedly to produce unreliable numbers, continues unchanged. Why? Because organizations need numbers for budgets, incentives, and planning. Those numbers must come from somewhere. Reliable or not, unreliable numbers are better than no numbers at all — from an organizational perspective.

The forecaster faces no consequences for being wrong. Compensation is paid regardless. The "I was almost right" defense is inexhaustible. And so the industry persists.

The Oil Price Consensus: $150 to $34 in Six Months

In July 2008, the consensus forecast of oil industry analysts was for oil to trade between $150 and $200 per barrel for the remainder of the year.

Six months later, oil was at $34.

That is not a small miss. That is not a 20% error or a 50% error. That is a 70–80% miss, across the entire forecast. No consensus forecast anticipated this move. Most missed by more than 300%. And notice: no confidence intervals, no error bars, no acknowledgment of uncertainty bounds.

What happened between July and December? The global financial crisis deepened and credit froze. Energy demand collapsed. Supply exceeded demand by a staggering amount. The consensus forecast had not even considered this as a scenario.

A decade later, the same analyst community, mostly the same people, same methodologies, same track record. Institutional investors still subscribe to the forecasts. Energy companies still build capital plans around them. The demonstrated uselessness of the output has had no effect on the demand for the output.

Silicon Valley's 10-Year Roadmaps: Dead on Arrival

Enterprise software companies routinely produce 10-year technology roadmaps. These are serious documents. They commit to specific architectures, timelines, and capabilities. A vendor's roadmap from 2015 promised certain features in 2020, certain capabilities in 2025.

Look at 2015's roadmap against 2025 reality. The match rate approaches zero.

Cloud computing exploded in ways nobody predicted the intensity of. Generative AI appeared suddenly and upended entire categories of software. Mobile dominance restructured how and where people worked. Security paradigms shifted. Every major inflection — every structural change that mattered — was either absent from or wrong in the 2015 roadmap.

Why do companies keep producing these documents? Because enterprise buyers want the reassurance. The roadmap is an artifact of commerce, not a genuine forecast. It projects confidence, not knowledge. But once the document exists, it acquires the weight of a commitment. People plan around it. People rely on it. People are disappointed when it does not materialize.

And the company produces a new roadmap ten years out, as if the previous one had any predictive value whatsoever.

The Federal Reserve's Dot Plot: Diverging at Every Turning Point

The U.S. Federal Reserve publishes a "dot plot" showing each governor's forecast of future interest rates. It is among the most closely watched documents in global finance. Hundreds of billions of dollars in positioning depend on interpreting it. Market participants treat the dots as information.

Over the past fifteen years, the dot plot's accuracy has been poor. At every major policy turning point, the rate paths in the dot plot have diverged materially from the realized paths. In 2019, the dots suggested rates would hold steady; then the pandemic hit and rates fell sharply. Before recent inflation, the dots suggested rates would remain low; then inflation surged and rates rose sharply.

The people issuing the dots are among the most informed about monetary policy anywhere on Earth. They have better information than any trader, any hedge fund manager, any economist in the private sector. They still cannot forecast their own policy decisions more than a few quarters out.

If the Fed cannot forecast the Fed, the market's reliance on the dot plot is a collective fiction that benefits commentators — who can spin whatever narrative around the dots they like — and harms everyone else who positions based on them.

The Core Problem: Numbers Without Uncertainty

Here is a sentence that should be tattooed on every forecaster's forehead: a forecast without a confidence interval is not information.

A statement like "GDP growth will be 2.3%" creates the illusion of precision. It implies that the forecaster has thought through this quantity to two decimal places. It implies that 2.2% or 2.4% would be surprising. It anchors the listener to a specific number.

The reality underneath that forecast is probably something like "we estimate growth somewhere between -1% and 5%, with a bunch of ways it could be much worse if financial conditions tighten." That is a radically different statement. It is less confident. It is messier. It is also closer to honest.

But a forecaster who produces that statement — who admits the massive uncertainty — will be overheard as wishy-washy, unprofessional, uncommitted. The forecaster who produces the precise-looking point estimate will sound authoritative, knowledgeable, professional. And so the precise estimates are what get communicated.

In domains where randomness is Extremistan — where outcomes can diverge from the average by orders of magnitude — this illusion of precision is actively dangerous. It leads to decisions built on false confidence. It leads to capital allocations designed around a single predicted path. It leads to surprise when the world deviates from the forecast.

Why the Industry Persists: The Organizational Need for Numbers

If forecasting does not work, why does the industry not collapse?

Because organizations operate on numbers. They budget based on numbers. They set incentives based on numbers. They communicate plans based on numbers. A publicly traded company cannot tell its shareholders "we have no idea what earnings will be, we're just going to wing it." Some forecast must be produced.

Given that a forecast must be produced, it might as well be confident and detailed. A forecast that sounds uncertain is, from a public relations perspective, worse than a forecast that sounds certain but turns out to be wrong. Certainty can be reframed post-hoc ("an unprecedented event occurred"), but uncertainty was a statement about the forecaster's incompetence.

And so the industry persists not because it produces value but because it produces what organizations demand: the appearance of foresight, wrapped in numbers, communicated with confidence.

The Practical Response: Treat Point Forecasts as Fiction

Given all of this, what should you do when you encounter a point forecast?

Treat it as approximately worthless — at least if it is a forecast for something in an Extremistan domain. Is it a stock price? Extremistan. Currency movements? Extremistan. Economic growth? Extremistan. Political outcomes? Extremistan. Company revenue five years out? Extremistan.

A forecast for something in Mediocristan — human height, calorie consumption, lifespan statistics for a large population — can have actual informational content. The distribution is stable, no single observation dominates, and the average means something.

But in Extremistan, the average hides the structure. The forecast hides the uncertainty. The point estimate hides the possibility of the Black Swan.

What to do instead:

Ask for the range and the confidence interval. If the forecaster says "2.3%," ask "what range contains the true value 90% of the time?" The answer you get will be far wider than 2.2–2.4%, and that wider range is closer to the truth.
Ask what has to be true for the forecast to be wrong. Specify in advance the conditions under which you would reject it. If no condition would falsify the forecast, it is unfalsifiable and therefore not knowledge.
Ask about tail scenarios. The forecast for the middle is probably worse than useless — it anchors you to a false center. But what are the ways things can go catastrophically wrong? What are the ways they could go dramatically better? The forecast's value is in exploring the tails, not in pinpointing the center.
Use scenario thinking instead. Don't try to predict the single most likely future. Map the range of plausible futures, the probability of each cluster, and the ways systems can break. Then design strategies that work across multiple scenarios rather than betting on one forecast.
Build in redundancy and flexibility. If you must make decisions in an uncertain world, make them in ways that don't require the forecast to be right. Hold optionality. Preserve the ability to change course. Design for robustness across scenarios rather than optimality in one scenario.

The Strategic Implication

The scandal of prediction has a strategic implication that is almost inverted from what most people believe. Most people think "if the forecasters are unreliable, I need better forecasters." The actual conclusion is "if the forecasters are unreliable, I should not be building strategies that depend on forecasts."

The financial world is full of firms that made exactly this error. They hired the best forecasters, paid the highest salaries, and built entire trading systems around point forecasts of future prices. When those forecasts failed — and they did, spectacularly — the systems collapsed.

The firms that survived were the ones that did not depend on being right about the future. They built positions that made money whether the forecast was correct or the opposite occurred. They held optionality. They did not assume they knew what would happen; they positioned to benefit from uncertainty.

This is the barbell strategy (linked below). This is via negativa — removing fragility rather than trying to predict and optimize. This is the practical response to a world where prediction fails: stop trying to be right about what will happen, and instead position yourself so that you thrive whatever happens.

Summary

The scandal of prediction is real, documented, and persistent. Forecasters perform at chance in Extremistan domains. The industry knows this at some level and continues anyway because organizations need numbers. The numbers that get produced are confident and precise, which makes them sound more reliable than they actually are, which makes the scandal invisible to most people.

The response is not to find better forecasters. It is to stop depending on forecasts, and instead to design strategies that work across multiple scenarios, that absorb errors rather than optimize around a single predicted path, and that remain flexible enough to change course when the world deviates from your assumptions — which it always does.