Calibration
The alignment between confidence in one's judgments and actual accuracy, reflecting how well subjective certainty matches objective correctness.
Also known as: Epistemic calibration, Confidence calibration, Probability calibration
Category: Decision Science
Tags: decision-making, rationality, psychology, forecasting, metacognition, epistemology
Explanation
Calibration refers to the correspondence between subjective confidence and objective accuracy. A well-calibrated person's confidence levels match their actual performance: when they say they're 80% sure, they're right about 80% of the time. Calibration is a key component of epistemic rationality and effective decision-making.
**Understanding calibration**:
Imagine making 100 predictions, each with an associated confidence level. For a well-calibrated person:
- Predictions made with 50% confidence should be correct about 50 times
- Predictions made with 90% confidence should be correct about 90 times
- Predictions made with 99% confidence should be correct about 99 times
Miscalibration occurs when confidence systematically deviates from accuracy:
**Overconfidence**: Believing you're more accurate than you are
- Saying you're 90% confident when you're only right 70% of the time
- This is the most common form of miscalibration
- Leads to insufficient hedging, inadequate preparation, and poor risk assessment
**Underconfidence**: Believing you're less accurate than you are
- Saying you're 60% confident when you're actually right 80% of the time
- Less common but can lead to excessive caution and missed opportunities
**Why calibration matters**:
- **Better decisions**: Accurate confidence allows appropriate risk-taking
- **Effective communication**: Others can rely on your stated certainties
- **Learning efficiency**: Knowing what you don't know focuses learning efforts
- **Credibility**: Well-calibrated people earn trust over time
- **Avoiding disasters**: Overconfidence contributes to catastrophic failures
**Calibration in practice**:
**Domain specificity**: People may be well-calibrated in familiar domains but poorly calibrated in unfamiliar ones. Experts are often better calibrated within their expertise.
**The hard-easy effect**: People tend to be overconfident on hard questions and underconfident on easy ones. Calibration is best for medium-difficulty items.
**Confidence intervals**: When estimating quantities, people typically make intervals too narrow. Asked for a 90% confidence interval, people often capture the true value only 50% of the time.
**Improving calibration**:
- **Track predictions**: Record predictions with confidence levels and check accuracy over time
- **Feedback**: Immediate, clear feedback enables calibration improvement
- **Consider alternatives**: Actively thinking about how you could be wrong reduces overconfidence
- **Reference classes**: Compare to base rates and similar past cases
- **Practice with calibration training**: Games and exercises that provide feedback on calibration
- **Probabilistic thinking**: Express beliefs as probabilities rather than certainties
- **Pre-mortems**: Imagine failure and explain why it happened
**Calibration and metacognition**:
Calibration is a metacognitive skill—it requires monitoring the relationship between your beliefs and reality. Well-calibrated metamemory (knowing what you know) is essential for effective learning and knowledge management. If you can't accurately assess what you know, you can't efficiently direct your learning efforts.
**Organizational implications**:
Poorly calibrated forecasts and risk assessments have contributed to financial crises, engineering disasters, and strategic failures. Organizations benefit from fostering calibration through prediction markets, tracking forecast accuracy, and rewarding honest uncertainty over false confidence.
Related Concepts
← Back to all concepts