BlueByte
x509: certificate signed by unknown authorityFixed

kubectl: x509: certificate signed by unknown authority

By Haneul SeoUpdated September 8, 20264 min

Hi, it's BlueByte. You run a normal kubectl get pods and instead of your workloads you get Unable to connect to the server: x509: certificate signed by unknown authority. Nothing on the cluster is down — kubectl reached the API server, checked the TLS certificate it presented, and refused to trust it. We'll walk through the message variants, why the certificate authority in your kubeconfig stopped matching, how to see the certificate the server is really sending, how to fix it per cause, and how to keep it from coming back.

The x509 errors kubectl actually prints

Three related messages all come from the same TLS check. The most common one:

Unable to connect to the server: x509: certificate signed by unknown authority

kubectl connected, the API server sent its serving certificate, and kubectl couldn't trace that certificate back to the certificate authority (CA) recorded in your kubeconfig. Two close relatives show up too:

Unable to connect to the server: x509: certificate has expired or is not yet valid
Unable to connect to the server: x509: certificate is valid for 10.96.0.1, not 192.168.49.2

The first means the server's certificate is outside its validity window; the second means the certificate is genuine but was issued for a different name or IP than the server: address you're dialing. All three are kubectl declining to trust the endpoint — not the cluster being broken.

Why the CA stopped matching

A few concrete causes produce the "unknown authority" form:

  • The cluster was rebuilt or its certificates were regenerated — minikube delete && minikube start, a recreated kind cluster, or a re-provisioned managed cluster all mint a new CA, but your kubeconfig still carries the old certificate-authority-data.
  • You're pointed at the wrong cluster — a stale current-context, or a server: that resolves to a load balancer or proxy in front of a different API server.
  • A corporate TLS-inspecting proxy sits between you and the server, re-signing the connection with a CA your machine trusts for browsers but kubectl does not.
  • The kubeconfig has no CA at all for a self-signed server, so kubectl has nothing to verify against.

See which server and CA your kubeconfig is using

Don't guess which context is active — ask kubectl:

kubectl config view --minify
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://192.168.49.2:8443
  name: minikube
current-context: minikube

--minify collapses the output to just the current context, so you see the exact server kubectl is dialing and whether a CA is embedded. kubectl config get-contexts lists every context, so you can confirm you're on the one you meant rather than a leftover from another cluster.

Read the certificate the server is really presenting

See what the endpoint actually sends, straight from OpenSSL:

openssl s_client -connect 192.168.49.2:8443 -showcerts </dev/null 2>/dev/null \
  | openssl x509 -noout -issuer -subject -dates
issuer=CN=kubernetes
subject=CN=kube-apiserver
notBefore=... notAfter=...

If the issuer is your company's proxy CA instead of the cluster CA, you've found a MITM proxy in the path. If notAfter is in the past, the certificate expired. If the issuer looks right but kubectl still rejects it, the CA embedded in your kubeconfig is stale.

Fix it by cause: refresh the kubeconfig or embed the right CA

For a rebuilt or managed cluster, pull a fresh kubeconfig from the source of truth rather than hand-editing base64:

minikube update-context                                # minikube
aws eks update-kubeconfig --name mycluster             # EKS
gcloud container clusters get-credentials mycluster    # GKE

For a self-managed cluster, point the context at the correct CA file and embed it so it travels with the config:

kubectl config set-cluster mycluster \
  --server=https://192.168.49.2:8443 \
  --certificate-authority=/path/to/ca.crt \
  --embed-certs=true

For a corporate proxy, add the proxy's CA to the bundle kubectl reads, or route the API endpoint around inspection — don't disable verification globally. As a dev-only last resort you can skip the check, but understand it turns off exactly the protection the error provides:

kubectl config set-cluster mycluster --insecure-skip-tls-verify=true

A real case: a rebuilt minikube cluster

I ran minikube delete && minikube start to reset a broken local cluster, then kubectl get pods threw x509: certificate signed by unknown authority. kubectl config view --minify showed the same server: https://192.168.49.2:8443, so the address was fine — but the fresh cluster had a new CA the cached kubeconfig didn't know. minikube update-context rewrote the certificate-authority and client cert for the context, and the next kubectl get pods returned the node list. Under a minute, and no editing of base64 blobs by hand.

How it differs from Unauthorized and connection refused

x509: ... is about the server's identity — it fails during the TLS handshake, before you're ever authenticated. error: You must be logged in to the server (Unauthorized) is the opposite: the TLS check passed, but your credentials were rejected. The connection to the server ... was refused is earlier still — nothing answered on the API port, so there was no certificate to check. Reading which of the three you got tells you whether to look at the CA, at your credentials, or at whether the server is up at all.

Related questions

Can't I just add --insecure-skip-tls-verify and move on?

For a throwaway local cluster, sure. But it disables the check that proves you're talking to the real API server, so on anything shared or production it leaves you open to a machine-in-the-middle. Fix the CA instead.

Why does my browser or curl reach the server but kubectl doesn't?

Browsers trust your OS or corporate certificate store; kubectl only trusts the CA in your kubeconfig. A proxy CA your OS trusts is invisible to kubectl unless you add it to the kubeconfig's CA.

I updated the CA file but kubectl still fails.

Two common reasons: current-context points at a different cluster, or the config already has embedded certificate-authority-data (shown as DATA+OMITTED) that overrides your file. Re-run kubectl config set-cluster with --embed-certs=true and check kubectl config view --minify.

The message says certificate has expired, not unknown authority.

That's a different x509 variant — the server's serving certificate is out of its validity window. Renew it (for kubeadm clusters, kubeadm certs renew all), or refresh your kubeconfig if it's the client certificate that expired.

How do I stop this happening after every cluster rebuild?

Regenerate the kubeconfig from the provider each time you recreate a cluster (update-context / update-kubeconfig / get-credentials) instead of copying an old config between machines.

References

Haneul Seo

Infrastructure engineer · 10+ years running Linux fleets

More in this category

detected dubious ownershipFixed

Git: fatal: detected dubious ownership in repository

Since the CVE-2022-24765 fix in Git 2.35.2, Git refuses to read a repository whose working tree or .git directory is owned by a different user than the one running the command. It shows up in containers, CI jobs, sudo sessions and shared drives. Fix the ownership if the repo should be yours, or add the exact path to safe.directory in your global config — never in the repo's own config, which Git ignores for this.

Git
failed calling webhookFixed

Kubernetes: Internal error occurred: failed calling webhook

An admission webhook sits in front of your write, the API server could not get an answer out of it, and failurePolicy: Fail turned that silence into a rejection. The tail of the message is the whole diagnosis: context deadline exceeded means the call went nowhere, no endpoints available means nothing is running, and an x509 line means the API server does not trust the webhook's certificate. Each has a different fix, and none of them is your manifest.

Kubernetes
1205Fixed

MySQL: ERROR 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

A statement waited the full innodb_lock_wait_timeout for a row lock another transaction is still holding, and gave up. sys.innodb_lock_waits names the blocking session and hands you the KILL statement, and a blocking_query of NULL means the blocker is idle on an open transaction. The detail most retry loops get wrong: by default only the timed-out statement is rolled back, so your transaction is still open and still holds every lock it took earlier.

MySQL
MISCONFFixed

Redis: MISCONF Redis is configured to save RDB snapshots, but it's currently unable to persist to disk

Reads keep working and every write is rejected, because the last background save failed and stop-writes-on-bgsave-error defaults to yes. The log names the real cause — no space, a dir the redis user can't write, a read-only mount at rename time, or fork failing with Cannot allocate memory. Fix the cause, run one BGSAVE, and writes come back on their own with no restart: rdb_last_bgsave_status flips from err to ok. Setting stop-writes-on-bgsave-error no restores writes instantly but leaves the snapshot broken, so treat it as a deliberate trade, not the fix.

Redis
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memoryFixed

Node.js: FATAL ERROR: Reached heap limit — JavaScript heap out of memory (exit 134)

The V8 heap has its own ceiling, derived from system memory and the Node release, and it is often far below the RAM you have; when a build or server reaches it, V8 aborts with FATAL ERROR: Reached heap limit and exit code 134. Read the real limit with v8.getHeapStatistics().heap_size_limit, then raise it with --max-old-space-size (in MiB) or NODE_OPTIONS for a large workload, size it below the cgroup limit inside containers, and use --heapsnapshot-near-heap-limit to catch a leak in a long-running process. Exit 137 with no FATAL ERROR line is a container kill, not this.

Node.js
exec /docker-entrypoint.sh: exec format errorFixed

Docker: "exec format error" when the container starts — wrong-platform image, no emulator, or a script with no shebang

The container exits on its first instruction with exec format error — the kernel's ENOEXEC, meaning the file exists but cannot be executed here. In practice that is an image built on one CPU architecture (an Apple-silicon Mac produces linux/arm64) and run on another (an x86_64 server) with no QEMU handler registered in binfmt_misc, or an entrypoint script whose first line is not a shebang. uname -m, docker image inspect and ls /proc/sys/fs/binfmt_misc tell the causes apart; the fix is an explicit docker buildx build --platform (or a manifest list for both), QEMU registration or --platform when you mean to emulate, and a #!/bin/sh line for the script.

Docker