Installing cert-manager

6 Feb 2022 15:18 kubernetes cert-manager

Up to this point, I’ve been creating and installing certificates manually. Let’s see if cert-manager will make that easier.

Installation

$ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.7.1/cert-manager.yaml

$ kubectl --namespace cert-manager get all
NAME                                         READY   STATUS    RESTARTS   AGE
pod/cert-manager-6d8d6b5dbb-qfxr5            1/1     Running   0          7m4s
pod/cert-manager-webhook-85fb68c79b-gtj2z    1/1     Running   0          7m4s
pod/cert-manager-cainjector-d6cbc4d9-tw5pl   1/1     Running   0          7m4s

NAME                           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/cert-manager           ClusterIP   10.43.43.27     <none>        9402/TCP   7m5s
service/cert-manager-webhook   ClusterIP   10.43.181.148   <none>        443/TCP    7m5s

NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cert-manager              1/1     1            1           7m5s
deployment.apps/cert-manager-webhook      1/1     1            1           7m5s
deployment.apps/cert-manager-cainjector   1/1     1            1           7m5s

NAME                                               DESIRED   CURRENT   READY   AGE
replicaset.apps/cert-manager-6d8d6b5dbb            1         1         1       7m5s
replicaset.apps/cert-manager-webhook-85fb68c79b    1         1         1       7m5s
replicaset.apps/cert-manager-cainjector-d6cbc4d9   1         1         1       7m5s

Certificate Issuer

Before we can issue any certificates, we need to create an Issuer or ClusterIssuer resource. These represent a certificate authority. The former issues certificates for the namespace that it’s installed in. The latter can issue certificates for all namespaces.

Because my cluster isn’t multi-tenant, and everything uses the same root CA, I’m going to create a ClusterIssuer. Because I’m using my own root CA (rather than, say, Let’s Encrypt), I’ll use the CA type.

k3s-ca-cluster-issuer.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: k3s-ca-cluster-issuer
spec:
  ca:
    secretName: k3s-ca-key-pair

kubectl apply -f k3s-ca-cluster-issuer.yaml

If you take a look at the logs for the cert-manager pod, you’ll see that it’s detected the ClusterIssuer, but can’t find the secret (wrapped for readability):

E0206 15:29:47.347247       1 setup.go:48] cert-manager/clusterissuers/setup
    "msg"="error getting signing CA TLS certificate"
    "error"="secret \"k3s-ca-key-pair\" not found"
    "resource_kind"="ClusterIssuer"
    "resource_name"="k3s-ca-cluster-issuer"
    "resource_namespace"=""
    "resource_version"="v1"

By default, ClusterIssuer resources look for the named secret in the cert-manager namespace, so:

kubectl --namespace cert-manager \
    create secret tls k3s-ca-key-pair \
        --cert=k3s-ca.crt \
        --key=k3s-ca.key

Issuing a certificate

To request a certificate, create a Certificate resource. The documentation is exhaustive, but a minimal example would look like this:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-k3s-differentpla-net
  namespace: default
spec:
  secretName: example-k3s-differentpla-net-tls
  issuerRef:
    name: k3s-ca-cluster-issuer
    kind: ClusterIssuer
  dnsNames:
  - example.k3s.differentpla.net

If we apply the manifest:

$ kubectl apply -f example-k3s-differentpla-net-certificate.yaml
certificate.cert-manager.io/example-k3s-differentpla-net created

$ kubectl --namespace default get secret
NAME                                        TYPE                                  DATA   AGE
example-k3s-differentpla-net-tls            kubernetes.io/tls                     3      40s

$ kubectl --namespace default get secret example-k3s-differentpla-net-tls -o yaml
apiVersion: v1
data:
  ca.crt: LS0tLS1...
  tls.crt: LS0tLS1...
  tls.key: LS0tLS1...
kind: Secret
metadata:
  annotations:
    cert-manager.io/alt-names: example.k3s.differentpla.net
    cert-manager.io/certificate-name: example-k3s-differentpla-net
    cert-manager.io/common-name: ""
    cert-manager.io/ip-sans: ""
    cert-manager.io/issuer-group: ""
    cert-manager.io/issuer-kind: ClusterIssuer
    cert-manager.io/issuer-name: k3s-ca-cluster-issuer
    cert-manager.io/uri-sans: ""
  creationTimestamp: "2022-02-06T15:45:23Z"
  name: example-k3s-differentpla-net-tls
  namespace: default
  resourceVersion: "5999398"
  uid: 7f59a415-1b7c-40f3-9027-0453df5ee4e4
type: kubernetes.io/tls

And then, of course, we can reference that secret, either directly in an Ingress (of which more later), an IngressRoute, or in a mount or environment variable.

Inspecting the key/certificate

Let’s take a look at the actual certificate and key, to see what’s in them.

kubectl --namespace default get secret example-k3s-differentpla-net-tls \
    --template="{{index .data \"tls.key\" | base64decode}}" > example-k3s-differentpla-net.key
kubectl --namespace default get secret example-k3s-differentpla-net-tls \
    --template="{{index .data \"tls.crt\" | base64decode}}" > example-k3s-differentpla-net.crt

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ea:80:...
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: C = GB, L = London, O = differentpla.net, CN = differentpla.net k3s CA
        Validity
            Not Before: Feb  6 15:45:23 2022 GMT
            Not After : May  7 15:45:23 2022 GMT
        Subject:
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:d4:...
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier:
                keyid:C6:5D:...

            X509v3 Subject Alternative Name: critical
                DNS:example.k3s.differentpla.net
    Signature Algorithm: ecdsa-with-SHA256
         30:45:...

Comparing that with the the certificates created by my certs script, I can see a couple of issues:

It used an RSA key, rather than my preferred EC key.
The Extended Key Usage extension is missing.
There’s no subject. Using commonName is deprecated in favour of the subjectAlternativeName extension, but it just seems weird.

Some tweaks to the Certificate manifest can fix all of that:

...
  commonName: example.k3s.differentpla.net
  dnsNames:
  - example.k3s.differentpla.net
  privateKey:
    algorithm: ECDSA
  usages:
    - server auth
    - client auth

To recreate the certificate, delete the secret. cert-manager will recreate it. Alternatively, wait for it to expire (default 3 months).

Ingress

If you’re using Ingress resources, you don’t need to create Certificate resources yourself. cert-manager will automatically issue certificates for them, provided you’ve annotated the Ingress appropriately.

However, it doesn’t seem that you can change the private key algorithm by using annotations. See cert-manager#4769.

IngressRoute

Per the documentation:

When using the Traefik Kubernetes CRD Provider, unfortunately Cert-Manager cannot interface directly with the CRDs yet, but this is being worked on by our team.

The workaround seems to be to create a fake Ingress along with the real IngressRoute. I’ve not tried that yet.