Kubernetes: Cert-Manager + Let's Encrypt

TL;DR

• Bottom line: cert-manager automates TLS certificate provisioning and renewal on Kubernetes using Let's Encrypt (ACME), supporting both HTTP-01 and DNS-01 challenges via Issuer/ClusterIssuer CRDs.
• Key tool/command: kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.17.2/cert-manager.yaml
• Watch out for: Using production Let's Encrypt before validating with staging -- you will hit rate limits (50 certs/domain/week) and get locked out.
• Works with: Kubernetes 1.29+ (cert-manager v1.18), 1.31+ (v1.19/1.20); NGINX Ingress, Traefik, HAProxy, AWS ALB, GKE Ingress.

Constraints

• cert-manager CRDs MUST be installed before creating any Issuer, ClusterIssuer, or Certificate resources -- otherwise resources are silently ignored [src1]
• Wildcard certificates (*.example.com) REQUIRE the DNS-01 challenge solver -- HTTP-01 cannot validate wildcard domains [src4]
• Let's Encrypt production rate limits: 50 certificates per registered domain per week -- always validate with the staging endpoint first [src1]
• The privateKeySecretRef Secret is your ACME account identity -- deleting it forces re-registration and may trigger rate limits [src1]
• DNS-01 solver credentials MUST be stored in Kubernetes Secrets, never hardcoded in the Issuer spec [src4]
• cert-manager v1.18+ defaults revisionHistoryLimit to 1 -- upgrading garbage-collects all stale CertificateRequests [src6]

Quick Reference

Decision Tree

START: Need TLS certificates on Kubernetes?
├── Single namespace only?
│   ├── YES → Use Issuer (namespace-scoped)
│   └── NO → Use ClusterIssuer (cluster-wide)
├── Need wildcard certificate (*.example.com)?
│   ├── YES → MUST use DNS-01 challenge
│   │   ├── DNS provider supported in-tree?
│   │   │   ├── YES → Configure dns01 solver with provider credentials
│   │   │   └── NO → Use webhook solver from community
│   │   └── Store provider API credentials in a Kubernetes Secret
│   └── NO ↓
├── Cluster publicly accessible on port 80?
│   ├── YES → Use HTTP-01 challenge (simplest setup)
│   └── NO → Use DNS-01 challenge (works behind firewalls/NAT)
├── Want automatic Certificate creation from Ingress?
│   ├── YES → Add cert-manager.io/cluster-issuer annotation to Ingress
│   └── NO → Create Certificate resource manually
└── DEFAULT → Start with staging endpoint, switch to production after validation

Step-by-Step Guide

1. Install cert-manager

Install cert-manager and its CRDs into the cluster. Helm is recommended for production; kubectl apply is simpler for quick setups. [src1]

# Option A: Helm (recommended for production)
helm repo add jetstack https://charts.jetstack.io --force-update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.17.2 \
  --set crds.enabled=true

# Option B: kubectl apply (quick setup)
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.17.2/cert-manager.yaml

Verify: kubectl get pods -n cert-manager -- all three Pods should be Running

2. Create a ClusterIssuer for Let's Encrypt staging

Always start with staging to avoid rate limits. Staging issues untrusted certificates but has much higher rate limits. [src1]

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email: [email protected]
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-staging-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: nginx

Verify: kubectl get clusterissuer letsencrypt-staging -o wide -- READY should be True

3. Create a ClusterIssuer for Let's Encrypt production

Switch to production once staging works. This endpoint issues browser-trusted certificates. [src1]

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    email: [email protected]
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: nginx

Verify: kubectl describe clusterissuer letsencrypt-prod -- Status should show The ACME account was registered with the ACME server

4. Request a certificate via Ingress annotation

The ingress-shim sub-component watches for annotated Ingress resources and automatically creates Certificate resources. [src3]

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - app.example.com
      secretName: app-example-com-tls
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-app-service
                port:
                  number: 80

Verify: kubectl get certificate -n default -- READY should become True within 1-2 minutes

5. Configure DNS-01 for wildcard certificates

Wildcard certificates require DNS-01 validation. Create a Secret with DNS provider API credentials, then configure the solver. [src4]

apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-token
  namespace: cert-manager
type: Opaque
stringData:
  api-token: "your-cloudflare-api-token-here"
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod-dns
spec:
  acme:
    email: [email protected]
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-dns-account-key
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token
              key: api-token
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: wildcard-example-com
  namespace: default
spec:
  secretName: wildcard-example-com-tls
  issuerRef:
    name: letsencrypt-prod-dns
    kind: ClusterIssuer
  dnsNames:
    - "*.example.com"
    - "example.com"

Verify: kubectl get challenges -- should show challenges progressing from pending to valid

Code Examples

YAML: Mixed HTTP-01 + DNS-01 solvers on one ClusterIssuer

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod-mixed
spec:
  acme:
    email: [email protected]
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-mixed-key
    solvers:
      - selector:
          dnsNames:
            - "*.example.com"
        dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token
              key: api-token
      - selector: {}
        http01:
          ingress:
            ingressClassName: nginx

YAML: CNAME delegation for DNS-01 (security best practice)

# Create CNAME: _acme-challenge.example.com -> _acme-challenge.acme.delegated.com
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod-delegated
spec:
  acme:
    email: [email protected]
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-delegated-key
    solvers:
      - dns01:
          cnameStrategy: Follow
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token
              key: api-token

Bash: Complete setup script

Full script: setup-cert-manager.sh (37 lines)

Anti-Patterns

Wrong: Using production Let's Encrypt from the start

# BAD -- hitting production endpoint during testing
# will exhaust rate limits (50 certs/domain/week)
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory

Correct: Start with staging, switch to production after validation

# GOOD -- use staging first
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory

Wrong: HTTP-01 for wildcard certificates

# BAD -- HTTP-01 CANNOT validate wildcard domains
dnsNames:
  - "*.example.com"     # Will never be issued via HTTP-01

Correct: Use DNS-01 solver for wildcard domains

# GOOD -- wildcard requires DNS-01
solvers:
  - dns01:
      cloudflare:
        apiTokenSecretRef:
          name: cloudflare-api-token
          key: api-token

Wrong: Missing tls.secretName in Ingress

# BAD -- annotation present but no tls section
# cert-manager's ingress-shim silently skips this
metadata:
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  rules:
    - host: app.example.com
  # Missing tls section entirely!

Correct: Include tls section with secretName and hosts

# GOOD -- tls section with secretName triggers cert-manager
spec:
  tls:
    - hosts:
        - app.example.com
      secretName: app-example-com-tls
  rules:
    - host: app.example.com

Common Pitfalls

• Certificate stuck in False READY state: Walk the resource chain: Certificate -> CertificateRequest -> Order -> Challenge. Fix: kubectl describe challenge <name> to see the specific error. [src2]
• HTTP-01 challenge returns 404: The solver Pod creates a temporary Ingress. Fix: verify ingressClassName matches your controller, or use acme.cert-manager.io/http01-edit-in-place: "true". [src2]
• DNS-01 TXT record not propagating: Internal DNS resolvers may not see the record. Fix: set --dns01-recursive-nameservers=8.8.8.8:53,1.1.1.1:53 and --dns01-recursive-nameservers-only. [src4]
• TLS handshake failure during HTTP-01: Ingress controller needs a cert to serve HTTPS. Fix: add cert-manager.io/issue-temporary-certificate: "true" annotation. [src2]
• ACME account secret conflict: Previous account Secret conflicts with new Issuer. Fix: kubectl delete secret <name> -n cert-manager and re-apply. [src2]
• Rate limit exceeded: Hit the 50 certs/domain/week limit. Fix: wait 7 days for reset; always use staging for testing. [src1]
• Challenge pods not created: cert-manager webhook or cainjector not running. Fix: kubectl get pods -n cert-manager and check logs. [src2]
• Ingress annotation ignored: Missing tls section or empty secretName. Fix: ensure every tls entry has a secretName and hosts. [src3]

Diagnostic Commands

# Check cert-manager pods are running
kubectl get pods -n cert-manager

# Check ClusterIssuer/Issuer status
kubectl get clusterissuer
kubectl describe clusterissuer letsencrypt-prod

# List all certificates and their READY status
kubectl get certificates --all-namespaces

# Follow issuance chain: Certificate -> CertificateRequest -> Order -> Challenge
kubectl describe certificate <name> -n <namespace>
kubectl get certificaterequest -n <namespace>
kubectl describe order <name> -n <namespace>
kubectl get challenges --all-namespaces
kubectl describe challenge <name> -n <namespace>

# Check TLS secret was created and inspect certificate
kubectl get secret <secretName> -n <namespace> -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -text

# View cert-manager controller logs
kubectl logs -n cert-manager deploy/cert-manager --tail=100

# Test DNS-01 TXT record propagation
dig _acme-challenge.example.com TXT +short

Version History & Compatibility

When to Use / When Not to Use

Important Caveats

• Let's Encrypt certificates are valid for 90 days. cert-manager renews them 30 days before expiry by default. If cert-manager goes down for more than 60 days, certificates will expire.
• HTTP-01 challenges require port 80 to be publicly accessible from the internet. If your cluster is behind a strict firewall or NAT, use DNS-01 instead.
• When using DNS-01, the API token needs write access to DNS records. Follow the principle of least privilege: create a scoped token for TXT records only.
• cert-manager v1.18+ garbage-collects old CertificateRequest resources on upgrade. Set revisionHistoryLimit explicitly if you need historical data.
• The ACME privateKeySecretRef secret is your account identity with Let's Encrypt. Back it up for cluster migrations.
• dnsmasq with --filterwin2k flag (common in OpenWRT) drops SOA records, breaking cert-manager DNS zone detection.