BlueByte
429 toomanyrequestsFixed

Docker Hub: toomanyrequests — You have reached your pull rate limit (429)

By Haneul SeoUpdated September 16, 20266 min

Hi, it's BlueByte. A deploy that has worked every night for months fails at 02:00 with toomanyrequests: You have reached your pull rate limit, and nothing in your code changed. Nothing is broken on your side either: Docker Hub counts pulls per account and per source IP, and something behind your address used up the shared budget. We'll walk through the message on the CLI, Kubernetes, and BuildKit, the headers that tell you which limit you hit and when it resets, the fix per cause, and how to confirm you are really under a bigger quota.

The message and where it shows up

Docker Hub answers the manifest request with HTTP 429 and this body, which the daemon relays with the registry's toomanyrequests code in front:

Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits

On Kubernetes the same text sits inside a pod event, and the pod cycles into ImagePullBackOff:

Warning  Failed   kubelet  Failed to pull image "nginx:1.27": ... 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit ...

A docker build fails on the FROM line with failed to resolve source metadata for docker.io/library/...: 429 Too Many Requests. Three surfaces, one cause: whoever asked for the manifest had no pulls left.

How the budget is counted, and why a whole office runs out together

The Docker Hub docs set the numbers: unauthenticated pulls are limited to 100 per IPv4 address (or IPv6 /64) and authenticated Personal accounts to 200, both "calculated on a 6-hour basis"; Pro, Team, and Business are unlimited. The unit that matters is the source address. A NAT gateway, an office egress, or a CI runner pool presents one IPv4 address to Docker Hub, so every anonymous pull from every machine behind it draws from the same 100. Your laptop being logged in changes nothing for the kubelet on a node or the runner in a pipeline — each pulls as whoever it is, and by default that is nobody.

Read the headers to see which limit you hit and when it resets

Don't guess; ask the registry. The ratelimitpreview/test image exists for this, and a HEAD request reads your counters without spending a pull:

TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token)
curl -sI -H "Authorization: Bearer $TOKEN" \
  https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest \
  | grep -iE '^(HTTP|ratelimit|docker-ratelimit)'
HTTP/2 200
docker-ratelimit-source: 203.0.113.10
ratelimit-limit: 100;w=3600
ratelimit-remaining: 100;w=3600

ratelimit-limit is the quota and w= the window in seconds; ratelimit-remaining is what's left; docker-ratelimit-source is the address being counted — if it isn't your machine's public IP, it's your NAT, and you are sharing. Run the check from the host that failed, because that is the address that matters. One honesty note: the docs show w=21600 (six hours), but when I ran this from a cloud host the header said w=3600, so read the window off your own headers rather than a remembered number. For the authenticated view, fetch the token with curl -s --user "$USER:$PAT" .... No ratelimit headers at all can mean a paid plan, where the limit doesn't apply.

Fix 1: log in where the pull actually happens

Use a personal access token, not your password, and pipe it in so it stays out of shell history:

echo "$DOCKER_PAT" | docker login --username your-user --password-stdin
Login Succeeded

The credentials land in $HOME/.docker/config.json for that user on that host. In CI, run the same command as a pipeline step with the token stored as a secret; on a build server, run it as the account that runs the builds. Then re-run the header check with --user and confirm the limit rose to 200 — or the headers disappeared, if the account is Pro or Team.

Fix 2: give the kubelet credentials with imagePullSecrets

Kubernetes nodes never see your laptop's login. Create a registry secret for Docker Hub's server name and reference it from the pod, or attach it to the namespace's service account so every pod inherits it:

kubectl create secret docker-registry regcred \
  --docker-server=https://index.docker.io/v1/ \
  --docker-username=your-user --docker-password="$DOCKER_PAT" \
  --docker-email=you@example.com
kubectl patch serviceaccount default \
  -p '{"imagePullSecrets": [{"name": "regcred"}]}'
spec:
  imagePullSecrets:
    - name: regcred
  containers:
    - name: web
      image: nginx:1.27
      imagePullPolicy: IfNotPresent

If you already ran docker login, kubectl create secret generic regcred --from-file=.dockerconfigjson=$HOME/.docker/config.json --type=kubernetes.io/dockerconfigjson reuses it. Delete the stuck pod once the secret exists; its replacement pulls authenticated.

Fix 3: stop asking Docker Hub the same question — a pull-through cache

When many hosts pull the same images, a mirror pulls once and serves the rest locally. The registry image runs as a proxy for Docker Hub:

# /etc/docker/registry/config.yml (on the mirror host)
proxy:
  remoteurl: https://registry-1.docker.io
  username: your-user
  password: your-pat

Then point every client's /etc/docker/daemon.json at it:

{ "registry-mirrors": ["https://mirror.internal:5000"] }

Restart the daemon and confirm it took: docker info lists the mirror under Registry Mirrors:. Two cautions from the docs: only Docker Hub can be mirrored this way, and if you give the mirror your credentials, every private image that account can reach becomes readable to anyone who can reach the mirror, so put authentication in front of it. On containerd nodes the equivalent lives in containerd's registry host configuration.

A real case: the 02:00 rollout that hit a NAT gateway's quota

A 12-node cluster rolled a new release each night. One night every pod sat in ImagePullBackOff, and kubectl describe pod showed the 429 above. From a node, the header check returned ratelimit-remaining: 0 and a docker-ratelimit-source equal to the VPC's NAT gateway — twelve kubelets plus a CI runner in the same subnet had been drawing anonymously from one address's 100. The fix was regcred on each namespace's default service account and imagePullPolicy: IfNotPresent on the workloads; the mirror came a week later. The re-run from the same node showed ratelimit-limit: 200, and the rollout pulled cleanly.

Verify, then keep the counter from climbing

Prove any fix from the host that failed, not from your laptop:

TOKEN=$(curl -s --user "$DOCKER_USER:$DOCKER_PAT" "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token)
curl -sI -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest | grep -i ratelimit

A 200 limit means your Personal login is in use; no headers means a paid plan. To stay under: pin image tags instead of latest and use imagePullPolicy: IfNotPresent so restarts reuse the node's cache, log in as a build step in every pipeline, put a mirror in front of clusters and runner pools, and watch ratelimit-remaining from a node on a schedule so you see the budget draining before the rollout does.

How this differs from ImagePullBackOff and the abuse limit

ImagePullBackOff is the state, not the cause — a wrong tag, a missing pull secret for a private repo, and this rate limit all end there, and only the event text tells them apart: manifest unknown versus unauthorized versus toomanyrequests. Docker Hub also has a separate abuse rate limit that answers 429 Too Many Requests to any request, web pages and APIs included, without the pull-limit body; it comes from a burst from one address, and the answer is to slow down, not to log in. Next time a pull dies at 429, run the header check from the failing host first — it gives you the address, the quota, and the clock in one line.

Related questions

I'm logged in on my laptop. Why is CI still rate limited?

Credentials live in $HOME/.docker/config.json for one user on one host, and the runner pulls as itself. Add docker login --password-stdin as a pipeline step with the access token stored as a secret, then run the header check from a runner to confirm the limit reads 200 or the headers vanish on a paid plan.

How long until the limit resets?

Read w= in the ratelimit-limit header — it is the window in seconds. The docs describe a 6-hour basis and show w=21600, but the header on your own host is the authority; when I ran the check from a cloud host it reported w=3600, so trust what the registry tells you rather than a remembered number.

Does a restart of a pod that already has the image count as a pull?

With imagePullPolicy: IfNotPresent and a pinned tag, the kubelet uses the image on the node and does not contact the registry. With Always, which is the default for the latest tag, every start goes back to Docker Hub, which is what drains a shared budget on a busy cluster.

Is a free Docker login enough, or do I need a paid plan?

A Personal login raises the budget to 200 pulls per window per account, and the docs list Pro, Team, and Business as unlimited. For a fleet that pulls the same images, a pull-through cache usually removes the problem before a plan change does, because the mirror pulls once for everyone.

I got 429 Too Many Requests without the pull rate limit text.

That is Docker Hub's separate abuse rate limit, which applies to every request including web pages and APIs when one address sends a burst. Back off and retry more slowly; logging in does not change it.

References

Haneul Seo

Infrastructure engineer · 10+ years running Linux fleets

More in this category

detected dubious ownershipFixed

Git: fatal: detected dubious ownership in repository

Since the CVE-2022-24765 fix in Git 2.35.2, Git refuses to read a repository whose working tree or .git directory is owned by a different user than the one running the command. It shows up in containers, CI jobs, sudo sessions and shared drives. Fix the ownership if the repo should be yours, or add the exact path to safe.directory in your global config — never in the repo's own config, which Git ignores for this.

Git
failed calling webhookFixed

Kubernetes: Internal error occurred: failed calling webhook

An admission webhook sits in front of your write, the API server could not get an answer out of it, and failurePolicy: Fail turned that silence into a rejection. The tail of the message is the whole diagnosis: context deadline exceeded means the call went nowhere, no endpoints available means nothing is running, and an x509 line means the API server does not trust the webhook's certificate. Each has a different fix, and none of them is your manifest.

Kubernetes
1205Fixed

MySQL: ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

A statement waited the full innodb_lock_wait_timeout for a row lock another transaction is still holding, and gave up. sys.innodb_lock_waits names the blocking session and hands you the KILL statement, and a blocking_query of NULL means the blocker is idle on an open transaction. The detail most retry loops get wrong: by default only the timed-out statement is rolled back, so your transaction is still open and still holds every lock it took earlier.

MySQL
MISCONFFixed

Redis: MISCONF Redis is configured to save RDB snapshots, but it's currently unable to persist to disk

Reads keep working and every write is rejected, because the last background save failed and stop-writes-on-bgsave-error defaults to yes. The log names the real cause — no space, a dir the redis user can't write, a read-only mount at rename time, or fork failing with Cannot allocate memory. Fix the cause, run one BGSAVE, and writes come back on their own with no restart: rdb_last_bgsave_status flips from err to ok. Setting stop-writes-on-bgsave-error no restores writes instantly but leaves the snapshot broken, so treat it as a deliberate trade, not the fix.

Redis
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memoryFixed

Node.js: FATAL ERROR: Reached heap limit — JavaScript heap out of memory (exit 134)

The V8 heap has its own ceiling, derived from system memory and the Node release, and it is often far below the RAM you have; when a build or server reaches it, V8 aborts with FATAL ERROR: Reached heap limit and exit code 134. Read the real limit with v8.getHeapStatistics().heap_size_limit, then raise it with --max-old-space-size (in MiB) or NODE_OPTIONS for a large workload, size it below the cgroup limit inside containers, and use --heapsnapshot-near-heap-limit to catch a leak in a long-running process. Exit 137 with no FATAL ERROR line is a container kill, not this.

Node.js
exec /docker-entrypoint.sh: exec format errorFixed

Docker: "exec format error" when the container starts — wrong-platform image, no emulator, or a script with no shebang

The container exits on its first instruction with exec format error — the kernel's ENOEXEC, meaning the file exists but cannot be executed here. In practice that is an image built on one CPU architecture (an Apple-silicon Mac produces linux/arm64) and run on another (an x86_64 server) with no QEMU handler registered in binfmt_misc, or an entrypoint script whose first line is not a shebang. uname -m, docker image inspect and ls /proc/sys/fs/binfmt_misc tell the causes apart; the fix is an explicit docker buildx build --platform (or a manifest list for both), QEMU registration or --platform when you mean to emulate, and a #!/bin/sh line for the script.

Docker