Thundering Herd
The thundering herd problem occurs when many waiting processes or clients are all woken at once by a single event and then contend simultaneously for the same resource, overwhelming it.
Also known as: Thundering herd problem
Category: Software Development
Tags: distributed-systems, concurrency, scalability, reliability, performance
Explanation
The thundering herd problem is a failure mode in concurrent and distributed systems where a large number of processes, threads, or clients are blocked waiting on the same event, and when that event finally occurs they are all released at the same instant. Instead of proceeding smoothly, they immediately compete for a single shared resource such as a lock, a connection pool, a database, or a newly available service, producing a sudden spike of contention that can overwhelm the resource they were waiting for.
The pattern arises because the waiters are synchronized by a common trigger. Classic examples include many operating-system threads blocked on a single lock or condition variable being woken together, thousands of clients reconnecting the moment a server or network partition heals, or every request that was queued behind a slow operation firing at once when it completes. In each case the fan-out of simultaneous activity is far larger than the resource was provisioned to absorb.
The consequences are a self-inflicted load spike: latency climbs, queues overflow, CPU and memory saturate, and the resource may fail or become so slow that it triggers timeouts. Those timeouts can in turn cause retries, feeding the herd and turning a transient event into a sustained outage. Because the excess load is generated by the system's own coordinated behavior, capacity that is adequate under steady state can still collapse under a herd.
Mitigations focus on desynchronizing and throttling the waiters. Waking only one or a few waiters at a time (for example a single-waiter wake instead of a broadcast wake), adding randomized jitter to wait and retry timers, applying rate limiting and admission control, using request coalescing so duplicate work is deduplicated, and spreading reconnection attempts with exponential backoff all reduce the simultaneity that defines the problem. Load balancing and graceful capacity ramp-up further smooth the recovery.
Related Concepts
← Back to all concepts