Threadpool Exhaustion in Node.js — The Silent Killer Behind Slow APIs

Many Node.js latency investigations start (and end) with the event loop.

Sometimes that’s correct.

But there’s a different failure mode that looks confusing in dashboards:

latency jumps from milliseconds to seconds
CPU doesn’t look pegged
event loop delay looks normal

Yet requests are clearly waiting.

In those incidents the bottleneck is often libuv’s threadpool—a small, shared pool that silently limits concurrency for a surprising set of “async” APIs.

The Problem

A typical production progression:

you ship a Node.js API
traffic increases
p95 latency spikes from ~20ms to 2s+

Host metrics remain deceptively calm.

What you’re experiencing is usually threadpool queueing: work is being submitted faster than the pool can execute it.

Node.js offloads several operations to a fixed-size threadpool. When that pool saturates, requests don’t fail—they wait.

Why It Happens

Node.js isn’t “async all the way down”.

Operations that can’t be handled directly by the OS’s non-blocking APIs are executed in libuv’s threadpool. Common categories include:

filesystem operations (fs.readFile, fs.writeFile)
crypto (bcrypt, pbkdf2, scrypt)
DNS (non-cached lookups)
compression (zlib)
some native addons

The critical detail

By default, the pool size is small:

UV_THREADPOOL_SIZE = 4

That means only 4 threadpool-backed operations can execute in parallel per Node.js process.

Everything else queues.

Internal Architecture

The event loop largely schedules and dispatches callbacks; the actual work is executed by threadpool workers:

flowchart LR
    ClientRequest --> EventLoop
    EventLoop -->|Delegates work| ThreadPool
    ThreadPool --> Worker1
    ThreadPool --> Worker2
    ThreadPool --> Worker3
    ThreadPool --> Worker4
    Worker1 --> CallbackQueue
    Worker2 --> CallbackQueue
    Worker3 --> CallbackQueue
    Worker4 --> CallbackQueue
    CallbackQueue --> EventLoop

Key insight:

The event loop is not doing the expensive work—it’s running callbacks after the work completes.

When the pool is saturated, completion callbacks come later, and your request latency increases even if the event loop itself isn’t blocked.

Example Scenario

Say your login path hashes passwords:

import bcrypt from "bcrypt";

app.post("/login", async (req, res) => {
    const hash = await bcrypt.hash(req.body.password, 10);
    res.send("OK");
});

Now assume:

100 concurrent requests
each bcrypt.hash takes ~100ms of threadpool time

With a 4-thread pool:

only 4 hashes run concurrently
the remaining 96 requests queue behind them

A rough back-of-the-envelope for service time is:

(100 / 4) * 100ms = 2500ms

You can observe multi-second tail latency without a dramatic change in “CPU %” at the host level.

Diagram

sequenceDiagram
    participant Client
    participant EventLoop
    participant ThreadPool

    Client->>EventLoop: Request (bcrypt)
    EventLoop->>ThreadPool: Submit task
    Note over ThreadPool: Only 4 threads available
    ThreadPool-->>EventLoop: Task complete
    EventLoop-->>Client: Response

    Note over ThreadPool: Remaining requests wait in queue

The Gotchas

1) `bcrypt` can dominate the pool

CPU-heavy crypto will occupy workers for relatively long durations. Under concurrency, queueing appears quickly and throughput collapses.

2) `fs` isn’t “free async”

await fs.promises.readFile("large-file.json");

This still consumes a threadpool worker. Under load (and especially with slow disks), it becomes a queue amplifier.

3) The threadpool is shared

These compete for the same pool:

bcrypt
fs
zlib
dns.lookup

A slowdown in one category can starve unrelated request paths.

4) Increasing `UV_THREADPOOL_SIZE` has tradeoffs

You can increase it:

UV_THREADPOOL_SIZE=32 node app.js

But more threads introduce real costs:

context switching overhead
CPU contention
memory pressure
diminishing returns

More threads is not automatically more throughput.

Internal Insight Most Engineers Miss

The threadpool is:

global per process
FIFO queued
unaware of request priority

Meaning:

A slow workload can starve unrelated requests.

For example, heavy file I/O can delay password hashing; compression can delay DNS lookups; and so on.

Practical Debugging Tips

1) Confirm it’s queueing (not event loop blocking)

Tools like clinic.js, 0x, and perf_hooks can help you distinguish:

event loop delay
time spent waiting on async completions

A common pattern is high tail latency with low event loop delay.

2) Reproduce under realistic concurrency

Synthetic tests that only hit “happy path” endpoints often miss threadpool saturation.

Include realistic mixes of:

crypto
file I/O
compression
DNS

How to Fix It (Properly)

Option 1: Offload CPU-heavy work

Move heavy tasks to worker threads or separate services:

import { Worker } from "worker_threads";

Option 2: Prefer streaming over full-buffer reads

Avoid:

await fs.readFile(...)

Prefer:

fs.createReadStream(...)

Option 3: Cache aggressively

Reduce repeated expensive operations where it’s safe to do so.

Option 4: Scale out before scaling threads

Running multiple Node.js processes and load balancing is often safer than blindly increasing the per-process threadpool.

Key Takeaways

Node.js has a hidden concurrency limit: the libuv threadpool.
The default is 4 threads per process.
Threadpool-backed “async” work can queue and inflate tail latency under concurrency.
The pool is shared across multiple categories of work.
Increasing the pool size can help, but it has costs and isn’t a silver bullet.
Robust fixes usually involve architectural separation and isolation of heavy workloads.

Final Thoughts

A common Node.js misconception is:

“Async means unlimited concurrency.”

It doesn’t.

Node.js can scale extremely well for I/O orchestration—until you hit the threadpool’s invisible ceiling.

When that happens, the service slows down quietly and progressively.

Next in the Series

Continue reading: Your Node.js API Is Slow — But CPU Looks Fine. Here’s the Real Bottleneck