I want to show you a contrast that reveals everything about when expertise is real.

The Meteorologist's Five-Day Window

A meteorologist gives me a five-day weather forecast: "Tuesday will be 72 degrees, sunny, 10 mph winds."

I check on Tuesday. It's 71 degrees, mostly sunny, winds at 9 mph. The forecast was nearly perfect.

I trust this meteorologist. The forecast was good. It matches reality.

Now the same meteorologist gives me a five-year forecast: "2029 will average 72 degrees."

I should not trust this forecast at all. Five-year weather forecasts are notoriously inaccurate. The meteorologist's expertise does not extend to five-year horizons.

The Economist's Eternal Inaccuracy

An economist gives me a three-year GDP forecast: "Growth will average 2.3% annually."

I check three years later. Growth averaged 1.8%. The forecast was wrong.

I ask the economist why. The response is: "I was directionally correct—the economy did grow, just at a slower rate than expected."

That is not a useful forecast. It tells me almost nothing about what to plan for.

Now I give the economist a three-month forecast and see if they can beat a naive baseline: "GDP growth will be 95% of last quarter's rate."

Across many forecasts, the economist does not beat this naive baseline. In fact, the economist usually underperforms it.

Why This Difference Exists

The meteorologist's five-day forecast works because:

1. The system's sensitivity to initial conditions is bounded. Five days is short enough that small changes in starting conditions don't cascade into large differences. The weather you'll see Tuesday is strongly constrained by today's weather.

2. The rules are stable. The laws of thermodynamics, fluid dynamics, and solar radiation haven't changed. The rules governing weather are identical today and five days from now.

3. Feedback is immediate and clear. You know within a day whether the forecast was right.

The meteorologist's five-year forecast doesn't work because:

1. The system is chaotic. Five years is long enough that tiny uncertainties in initial conditions amplify into completely different weather patterns. (This is the butterfly effect.)

2. The rules stay the same, but you can't predict the consequences. You can't compute far enough into the chaotic system to know what those stable rules will produce.

The economist's forecasts don't work because:

1. The system is Extremistan. Outcomes are dominated by rare large events—recessions, wars, pandemics, technological disruptions. These events are not in any economist's training data if they're truly novel.

2. The rules change. Monetary policy regimes shift. Technological capabilities change. Political structures evolve. The economy of 2029 may operate under completely different rules than the economy of 2024.

3. Feedback is delayed and confounded. You can't cleanly test an economic forecast because the world changed between prediction and outcome in ways that affect what you're trying to measure.

The Oncologist vs. the Stockbroker

An oncologist with twenty years of experience makes a cancer diagnosis and prognosis. She has seen thousands of cases. She knows the staging systems. She understands the biology. She has access to clinical trial data.

When you follow her recommendations, you can eventually check whether she was right: did the patient respond to treatment? Did they survive?

Feedback is clear. Rules are stable (cancer biology doesn't change). Complexity is high enough to reward expertise.

A stockbroker with twenty years of experience makes an investment recommendation. He has traded many securities. He has strong conviction about company fundamentals. He has a track record of "beating the market."

But feedback is delayed or absent (the stock might rise for years then crash; correlation with manager skill is weak). Rules are unstable (market regimes change; technologies disrupt; regulatory changes shift). Selection effects hide poor stock pickers (the worst are no longer around, but the mediocre survived by luck).

When you follow his recommendations, you cannot easily tell whether he was right—because of confounding factors, selection effects, and the sheer difficulty of attributing outcome to the picker's skill versus luck.

The oncologist has environmental conditions that permit expertise. The stockbroker does not, despite similar career length and apparent competence.

The Decision Tool

Before trusting an expert in a domain you don't fully understand, ask:

1. How fast is feedback? Can I know within weeks or months if the expert was right, or is feedback delayed years or absent? If delayed, the domain cannot produce expertise.

2. Are the rules stable? Will the underlying principles governing this domain be the same in five years, or will they change? If they change, expertise in the training set doesn't transfer.

3. Is it Mediocristan or Extremistan? Are outcomes distributed normally around a mean (Mediocristan, where averages work), or are they dominated by rare large events (Extremistan, where averages hide the generator)? If Extremistan, expertise is harder to come by.

4. What is the baseline? Could I achieve similar results with a naive strategy? If yes, the expert's track record might be luck, not skill.

If the expert passes these tests, they might be trustworthy. If they fail, treat their forecasts as one input among many—but not as truth-claims from someone who has proven forecasting ability.

The Five-Day vs. The Five-Year

The key insight is that the same person can be trustworthy at one time horizon and completely untrustworthy at another.

The meteorologist is trustworthy at five days, completely unreliable at five years.

The oncologist is trustworthy about survival outcomes, potentially unreliable about causes (was it the treatment or the patient's genetics or their health habits?).

The economist is unreliable at any forecast horizon in my experience, but possibly trustworthy about mechanical relationships within a limited domain ("if the Fed raises rates 1%, what are the mechanical effects on credit?").

Before you listen to any expert, ask: at what domain size, time horizon, and within what constraints is this expertise actually valid?

The answer is usually narrower than the expert wants to admit.