BlueByte
driver failedFixed

docker: driver failed programming external connectivity (iptables)

By Haneul SeoUpdated August 31, 20264 min

Hi, it's BlueByte. If docker run -p suddenly refuses to publish a port and you changed the firewall recently, this one has a very specific cause — and a clean fix. Let's read the error, confirm what happened, rebuild what was lost, and make sure the next firewall reload does not undo it.

What the error is actually telling you

Publishing a port fails like this:

docker: Error response from daemon: driver failed programming external
connectivity on endpoint web: (iptables failed: iptables -t nat -A DOCKER ...
No chain/target/match by that name)

You might also see it as Failed to program NAT chain: ... No chain/target/match by that name, or on docker compose up simply as driver failed programming external connectivity. Read the last line: the daemon tried to add a NAT rule and the DOCKER chain it needs is missing. Nothing is corrupted — your image, compose file, and port number are fine. The problem is the host's firewall state underneath Docker.

Why a firewall reload wipes Docker's chain

When Docker starts a container with a published port, it creates a DOCKER chain and per-container rules in the nat table. Something deleted that chain while the daemon was still running. The usual suspects:

  • A firewall reload — firewall-cmd --reload, a ufw enable/reload, or netfilter-persistent — that rebuilt the ruleset from its own config, which does not include Docker's chain.
  • A manual iptables-restore without --noflush, which replaces the whole table and takes Docker's chain with it.
  • A hardening tool or cron job that flushes iptables on a schedule.

First, confirm the chain is actually gone

You do not have to guess. Check whether the chain exists:

sudo iptables -t nat -L DOCKER -n

No chain/target/match by that name confirms it was wiped. Then find what wiped it — look at the services that rewrite iptables and when they last ran:

systemctl status firewalld ufw netfilter-persistent
sudo journalctl -u firewalld --since "15 min ago"

If the breakage lines up with a reload or a reboot, that is your cause.

Restart the daemon to rebuild the chain

Restarting Docker recreates its chains and per-container rules. Nothing to memorize here — one command does it:

sudo systemctl restart docker
sudo iptables -t nat -L DOCKER -n   # the chain and your rule are back

Then keep the next reload from taking it out. On a firewalld host, run firewalld and Docker's integration together — Docker installs its rules through firewalld, so a reload re-applies them instead of dropping them:

sudo firewall-cmd --reload
sudo iptables -t nat -L DOCKER -n   # still present

If you script changes with iptables-restore, add --noflush so it appends rather than replacing the whole table.

A real case: a reload at 2pm

Say every docker run -p has failed since you tweaked a firewalld rule. iptables -t nat -L DOCKER -n returns "No chain/target/match by that name", and journalctl -u firewalld shows a reload two minutes before the failures started. That is the fingerprint — the reload flushed the chain. sudo systemctl restart docker rebuilds it, the same iptables command now lists the chain, and your container publishes. The next firewalld reload leaves it alone, because Docker registered through firewalld rather than around it.

Check a published port end to end

Confirm it really works, not just that the chain looks right:

docker run --rm -p 8080:80 nginx:alpine
curl -sSI http://localhost:8080 | head -1   # HTTP/1.1 200 OK

A 200 plus a visible rule in iptables -t nat -L DOCKER -n means the fix held.

Keep it from coming back

On boot, order the units so the firewall starts before Docker, or let firewalld own the integration so reloads never strip the chain. On a host you reload often, avoid a bare iptables -F or an unscoped iptables-restore against the nat table — scope any flush to your own chains so Docker's survive.

How this differs from "port is already allocated"

Bind for 0.0.0.0:8080 failed: port is already allocated is the opposite case — the chain is fine, but another container already holds the port. And setting "iptables": false in /etc/docker/daemon.json silences this error only by disabling published-port connectivity entirely, so outside traffic can no longer reach your containers. Leave iptables management on unless you manage every NAT rule by hand. Next time a port publish fails right after a firewall change, check the DOCKER chain first — it is almost always this.

Related questions

Does restarting the daemon stop my running containers?

It restarts containers that do not have a restart policy of always or unless-stopped. Check a container's policy with docker inspect before restarting in production.

The error comes back after every reboot. Why?

The firewall service is starting after Docker on boot and flushing the chains. Let firewalld own Docker's integration, or order the units so the firewall comes up before Docker.

Can I re-add just the DOCKER chain instead of restarting?

You can recreate rules by hand, but a daemon restart is the supported path: it rebuilds every chain consistently. Manual rule surgery drifts from what Docker expects and tends to break on the next container start.

Are host-network containers affected?

No. Containers on the host network do not use published ports, so they never touch the DOCKER chain. Only containers that publish ports on a bridge network hit this.

It started right after an OS upgrade. Related?

Likely. An upgrade can switch the iptables backend (legacy vs nf_tables). Make sure the host and Docker use the same backend, then restart Docker to rebuild its chains under it.

References

Haneul Seo

Infrastructure engineer · 10+ years running Linux fleets

More in this category

detected dubious ownershipFixed

Git: fatal: detected dubious ownership in repository

Since the CVE-2022-24765 fix in Git 2.35.2, Git refuses to read a repository whose working tree or .git directory is owned by a different user than the one running the command. It shows up in containers, CI jobs, sudo sessions and shared drives. Fix the ownership if the repo should be yours, or add the exact path to safe.directory in your global config — never in the repo's own config, which Git ignores for this.

Git
failed calling webhookFixed

Kubernetes: Internal error occurred: failed calling webhook

An admission webhook sits in front of your write, the API server could not get an answer out of it, and failurePolicy: Fail turned that silence into a rejection. The tail of the message is the whole diagnosis: context deadline exceeded means the call went nowhere, no endpoints available means nothing is running, and an x509 line means the API server does not trust the webhook's certificate. Each has a different fix, and none of them is your manifest.

Kubernetes
1205Fixed

MySQL: ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

A statement waited the full innodb_lock_wait_timeout for a row lock another transaction is still holding, and gave up. sys.innodb_lock_waits names the blocking session and hands you the KILL statement, and a blocking_query of NULL means the blocker is idle on an open transaction. The detail most retry loops get wrong: by default only the timed-out statement is rolled back, so your transaction is still open and still holds every lock it took earlier.

MySQL
MISCONFFixed

Redis: MISCONF Redis is configured to save RDB snapshots, but it's currently unable to persist to disk

Reads keep working and every write is rejected, because the last background save failed and stop-writes-on-bgsave-error defaults to yes. The log names the real cause — no space, a dir the redis user can't write, a read-only mount at rename time, or fork failing with Cannot allocate memory. Fix the cause, run one BGSAVE, and writes come back on their own with no restart: rdb_last_bgsave_status flips from err to ok. Setting stop-writes-on-bgsave-error no restores writes instantly but leaves the snapshot broken, so treat it as a deliberate trade, not the fix.

Redis
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memoryFixed

Node.js: FATAL ERROR: Reached heap limit — JavaScript heap out of memory (exit 134)

The V8 heap has its own ceiling, derived from system memory and the Node release, and it is often far below the RAM you have; when a build or server reaches it, V8 aborts with FATAL ERROR: Reached heap limit and exit code 134. Read the real limit with v8.getHeapStatistics().heap_size_limit, then raise it with --max-old-space-size (in MiB) or NODE_OPTIONS for a large workload, size it below the cgroup limit inside containers, and use --heapsnapshot-near-heap-limit to catch a leak in a long-running process. Exit 137 with no FATAL ERROR line is a container kill, not this.

Node.js
exec /docker-entrypoint.sh: exec format errorFixed

Docker: "exec format error" when the container starts — wrong-platform image, no emulator, or a script with no shebang

The container exits on its first instruction with exec format error — the kernel's ENOEXEC, meaning the file exists but cannot be executed here. In practice that is an image built on one CPU architecture (an Apple-silicon Mac produces linux/arm64) and run on another (an x86_64 server) with no QEMU handler registered in binfmt_misc, or an entrypoint script whose first line is not a shebang. uname -m, docker image inspect and ls /proc/sys/fs/binfmt_misc tell the causes apart; the fix is an explicit docker buildx build --platform (or a manifest list for both), QEMU registration or --platform when you mean to emulate, and a #!/bin/sh line for the script.

Docker