Epistemic Arrogance: Definition and Calibration

Epistemic arrogance is the systematic overestimation of what we know and the systematic underestimation of uncertainty. We think we understand the world better than we actually do. Experts are especially prone to it—they're confident in their knowledge precisely because they've studied the domain.

I've learned this the hard way by watching confident experts blow up in their own fields. The more specialized they were, the more they were blindsided. Because they weren't accounting for how little they actually knew.

The Calibration Experiment

Taleb describes a classic experiment: experts in a field are asked to estimate a quantity within a range where they claim 98% confidence. When you check their actual estimates against reality, the true value falls within their "98% confidence intervals" only about 60–70% of the time.

This is the core insight: people are not well-calibrated about their own uncertainty. They think they're more precise than they actually are. A 98% confidence interval should contain the true value 98 times out of 100. It actually contains it 60–70 times.

And here's the practical solution Taleb proposes: double every interval you're tempted to give. If you think you can estimate something to ±10%, you probably can't—you can estimate it to ±20%. If your model says it's 95% accurate, it's probably 90%. If you claim 99% confidence, adjust down to 97%.

Why Experts Are Arrogant

The problem is deepest in specialized domains. A cardiologist knows more about hearts than a generalist. So they're confident in their knowledge. But that confidence is often misplaced because they're operating in Extremistan where Black Swans exist.

The more data points they have in their field—the more cases they've seen—the more confident they become. But if their field is Extremistan, those cases tell them almost nothing about what's coming. They've never seen the tail event. And tail events matter most.

The Danger in High-Stakes Domains

I'm especially concerned about epistemic arrogance in domains where mistakes are expensive: medicine, finance, policy, engineering. These are exactly the fields where experts are most likely to be overconfident about what they know.

A surgeon looks at your imaging and confidently recommends a procedure. How confident should you be in that recommendation? Maybe less than you think. Did they account for their own uncertainty? Did they study the cases where the surgery didn't help—or did those disappear into silence?

An economist predicts interest rates with confidence. But they're rarely as accurate as they sound. A risk manager models tail risk. But the model was built on data that didn't include the tail events that matter most.

Recalibrating Your Confidence

I've learned to build in buffers. I assume my knowledge is worse than I think it is. I double my estimates of uncertainty. When an expert claims 90% confidence, I assume it's actually 80%. When I claim something is nearly certain, I stay alert for evidence I'm missing.

The antidote to epistemic arrogance isn't to become agnostic—it's to be honest about the gap between what you think you know and what you actually know.

Go deeper:

For the full breakdown of how experts systematically overestimate their knowledge and how to calibrate uncertainty, read Epistemic Arrogance: The Overconfidence Crisis.