Linux: Too many open files (EMFILE)

Hi, it's BlueByte. If a service starts throwing EMFILE under load, there are really two questions hiding in one error: is the limit too low, or is something leaking? Let's tell those apart first — because raising the limit fixes one and only postpones the other — then apply the fix where the process actually runs.

What EMFILE is telling you

A server or script starts failing under load:

Error: EMFILE: too many open files

The process tried to open a file, socket, or connection past the maximum number of file descriptors it is allowed. Every socket and pipe counts, so a busy network service hits this well before it runs out of actual files. The failure is per-process, and the fix has to be applied where that process runs — which is where people most often get it wrong.

Too low, or leaking? — the two real causes

Each process has a soft limit on open descriptors — often 1024 by default. Once it is reached, every new open(), socket(), or accept() fails with EMFILE. Two underlying situations need different responses:

• The limit is genuinely too low for the workload — a proxy or database handling thousands of concurrent connections needs far more than 1024.
• The process is leaking descriptors — opening connections or files it never closes, so the count only grows until it hits the ceiling.

Telling these apart is the whole game.

Watch the descriptor count to tell them apart

You do not have to guess which one it is — watch it. Check the current limit and how many the process holds right now:

ulimit -n
ls /proc/$(pgrep -f myservice)/fd | wc -l

Then watch that second number over a few minutes of steady traffic. If it climbs and never drops, you have a leak. If it plateaus near the limit under real load, the limit is simply too low. To see the effective limit on the running process itself — not your shell — read its limits file:

grep "open files" /proc/$(pgrep -f myservice)/limits

Raise the limit where the process runs

1. For a service, raise the limit where it runs. A systemd unit ignores your shell's ulimit, so set it in the unit:

# /etc/systemd/system/myservice.service
[Service]
LimitNOFILE=65536

sudo systemctl daemon-reload
sudo systemctl restart myservice

2. For login sessions, add limits in /etc/security/limits.conf, then open a new session:

*  soft  nofile  65536
*  hard  nofile  65536

3. If you found a leak, fix the code that opens without closing — the limit change only buys time.

A real case: a leak, not a low limit

A Node service starts throwing EMFILE after about an hour of traffic. ls /proc/<pid>/fd | wc -l reads 1024 and stays pinned there; watched over ten minutes on a previous run it climbed steadily and never fell. That is a leak signature, not a low limit — the app opens an HTTP client per request and never closes it. You fix the client to reuse a pooled connection, and the descriptor count now plateaus around 200 under the same load. You still raise LimitNOFILE to 65536 for headroom, but the leak fix is what actually stopped the failure.

Confirm the limit applies to the process, not your shell

This is the step people skip, so check it directly:

cat /proc/$(pgrep -f myservice)/limits | grep "open files"

The "Max open files" column should show the value you set. Then run the load that used to fail, confirm it no longer does, and that the descriptor count is stable.

Keep it from recurring

Set LimitNOFILE deliberately for network services as part of their unit file, match it to real concurrency plus headroom, and add a metric or alert on descriptor count so a slow leak is visible long before it takes the service down.

How this differs from ENFILE

EMFILE is per-process; ENFILE ("file table overflow") is the whole system running out — much rarer, and fixed with the kernel-wide fs.file-max sysctl. A Too many open files from a shell doing ulimit work is the same limit but reached interactively. When you hit EMFILE, watch the count before you raise anything — the graph tells you which problem you actually have.