Erlang cluster on Kubernetes: Submitting certificate requests to cert-manager

22 Dec 2022 10:01 kubernetes cert-manager

In the previous post we used OpenSSL to create a certificate signing request. In this post, we’ll submit it to cert-manager and get the certificate back.

Service account

We’re going to access the cert-manager API from inside the init container. To do that we’ll either need to change the permissions of the “default” service account, or we’ll need to create a new service account. I opted for the latter.

Since all containers in a pod run with the same service account, I named it erlclu:

apiVersion: v1
kind: ServiceAccount
  name: erlclu
  namespace: erlclu

We need a role that can request certificates. It needs create (to submit the request) and get (to retrieve the generated certificate):

kind: Role
  namespace: erlclu
  name: certificate-requester
- apiGroups: [""]
  resources: ["certificaterequests"]
  verbs: ["create", "get"]

…and we need a role binding:

kind: RoleBinding
  name: request-certificate
  namespace: erlclu
- kind: ServiceAccount
  name: erlclu
  kind: Role
  name: certificate-requester

Accessing the Kubernetes API from a container

I wrote something about this a while ago. We’ll need some preliminaries:

AUTH_TOKEN="$(cat /var/run/secrets/"
NAMESPACE="$(cat /var/run/secrets/"



Submitting the CSR

We need to base64-encode the CSR:

encoded_csr=$(base64 -w0 < "$CERTS_DIR/$CERT_FILENAME.csr")

…we need a name for the CertificateRequest; I’ll use the pod name:

request_name="$(hostname -s)"

Then we can actually make the request. We use a heredoc to build the JSON body. Piping heredocs is a little weird, but it looks like this:

cat << EOF | \
    curl -s -X POST \
        --header "Content-Type: application/json" \
        --header "Authorization: Bearer ${AUTH_TOKEN}" \
        --cacert "${CA_CERT_BUNDLE}" \
        --data-binary @- \
    "apiVersion": "",
    "kind": "CertificateRequest",
    "metadata": {
        "name": "$request_name",
        "namespace": "$NAMESPACE",
        "labels": {
            "app": "$APPLICATION_LABEL"
    "spec": {
        "request": "$encoded_csr",
        "issuerRef": {
            "kind": "$ISSUER_KIND",
            "name": "$ISSUER_NAME"

The APPLICATION_LABEL, ISSUER_KIND and ISSUER_NAME variables are set in the K8s deployment. We’ll see those below.

Getting the new certificate

When cert-manager signs the request, it will update the .status fields in the CertificateRequest object. Ideally, we’d actually poll the status. For now, however, a simple sleep 5s will do the job. I’ll fix that later.

sleep 5s

Once polling (cough) tells us that the certificate has been signed successfully, we can write the CA and certificate to files in the shared /certs volume, where the main container can find them later:

res=$(curl -s \
    --header "Accept: application/json" \
    --header "Authorization: Bearer ${AUTH_TOKEN}" \
    --cacert "${CA_CERT_BUNDLE}" \

# Write the cert and the CA to files.
echo "$res" | jq -r '' | base64 -d > "$CERTS_DIR/ca.crt"
echo "$res" | jq -r '.status.certificate' | base64 -d > "$CERTS_DIR/$CERT_FILENAME.crt"

Note that this required us to add apk add openssl curl jq to the docker container.

Updating the deployment

We need to specify the following:

  • The service account
  • Environment variables that the script is expecting.
  • The volume mount for the shared /certs volume.
  serviceAccountName: erlclu
    - name: erlclu-init
        - name: MY_POD_IP
              fieldPath: status.podIP
        - name: APPLICATION_LABEL
          value: erlclu
        - name: ISSUER_KIND
          value: ClusterIssuer
        - name: ISSUER_NAME
          value: k3s-ca-cluster-issuer
        - name: CERTS_DIR
          value: /certs
        - name: CERT_FILENAME
          value: tls-dist
        - name: tls-dist
          mountPath: /certs

Note that it’s using the ClusterIssuer that I created for * I’ll fix that later.

APPLICATION_LABEL is used to differentiate our CertificateRequest objects from the ones that cert-manager uses for generating TLS certficates. We’ll use it later to ensure we only clean up our requests.

Shared /certs volume

The certificates are shared between the init container and the main container, and nothing else. Because of this, I used a memory-backed emptyDir volume:

    - name: tls-dist
        medium: "Memory"
        sizeLimit: 32Ki

32Ki seems to be large enough. The certificate and CA are about 24KiB in size.

For the time-being, I left the existing certificates in /secrets. I’ll describe how I switched to using the new certificates and got rid of the old ones in the next post.