Small Sample Fallacy
The error of drawing strong conclusions from insufficient data.
Also known as: Law of small numbers, Insufficient sample, Hasty generalization
Category: Concepts
Tags: statistics, cognitive-biases, thinking, probabilities, research
Explanation
The small sample fallacy (or law of small numbers) is the error of drawing confident conclusions from insufficient data - treating small samples as if they were representative of larger populations. The fallacy manifests as: overconfidence in patterns (seeing trends in noise), overgeneralizing from limited experience (one example becomes a rule), and underestimating variability (expecting small samples to look like large ones). Why it happens: we're pattern-seekers who find signal in noise, small samples often show extreme values (which are memorable), and we underestimate random variation. Examples: judging a restaurant by one meal, concluding a strategy works from few tries, or believing stereotypes based on limited encounters. Corrections include: demanding larger samples before concluding, expecting high variability in small datasets, asking 'how would I feel if the next few observations were opposite?', and distinguishing genuine patterns from statistical artifacts. The law of large numbers is real; the law of small numbers is a cognitive illusion. For knowledge workers, avoiding this fallacy means: not overinterpreting early results, maintaining appropriate uncertainty, and recognizing that anecdotes aren't data.
Related Concepts
← Back to all concepts