In this setup we will configure the prometheus community helm chart to scrap metrics from a SSL enabled etcd.

The etcd was installed through kupespray kubernetes cluster, but you should be able to adapt to your case.

Generating the SSL Client Keys

We will need to generate client certificate keys to connect to etcd.

Kubespray install the CA pem in /etc/ssl/etcd/ssl/ directory on each node

SSH into the first etcd node and generate a new ssl client certificate. The first node is the one with the file /etc/ssl/etcd/openssl.conf, you can find the file with the command find / -name openssl.conf

Create the client request:

openssl req -config /etc/ssl/etcd/openssl.conf -new -nodes   -keyout etcd-client-prometheus.key -out etcd-client-prometheus.csr -subj "/CN=etcd-client"

Signing the request

openssl ca -config openssl.cnf -extensions etcd_client -keyfile private/ca.key -cert certs/ca.crt -out certs/etcd-client.crt -infiles etcd-client.csr

Generate the certs

openssl x509 -req -in etcd-client-prometheus.csr -CAkey /etc/ssl/etcd/ssl/ca-key.pem -CA /etc/ssl/etcd/ssl/ca.pem -CAcreateserial -days 36500 -extensions ssl_client -out etcd-client-prometheus.crt -extfile /etc/ssl/etcd/openssl.conf

Test cert

curl --cert etcd-client-prometheus.crt --key etcd-client-prometheus.key https://10.8.0.2:2379/metrics

Change the ip for your server ip

Configuring Prometheus to user the certs

Create a secret to old the certs (must be in same namespace as prometheus pods)

kubectl create secret generic etcd-client-cert --from-file=./ca.pem --from-file=./etcd-client-prometheus.crt --from-file=./etcd-client-prometheus.key

Configure Prometheus community to read the secret and to use it in etcd monitoring:

## Component scraping etcd
##
kubeEtcd:
  enabled: true
  ## If your etcd is not deployed as a pod, specify IPs it can be found on
  ##
  endpoints:
    - 10.8.0.1
    - 10.8.0.2
    - 10.8.0.3

  ## Etcd service. If using kubeEtcd.endpoints only the port and targetPort are used
  ##
  service:
    enabled: true
    port: 2379
    targetPort: 2379
    # selector:
    #   component: etcd
  serviceMonitor:
    scheme: https
    insecureSkipVerify: false
    serverName: etcd.kube-system.svc # this is per kubespray default install
    caFile: /etc/prometheus/secrets/etcd-client-cert/ca.pem
    certFile: /etc/prometheus/secrets/etcd-client-cert/etcd-client-prometheus.crt
    keyFile: /etc/prometheus/secrets/etcd-client-cert/etcd-client-prometheus.key
prometheus:
  prometheusSpec:
    secrets: ["etcd-client-cert"]

Apply the new values, for example with the command:

helm upgrade -f values.yml prometheus prometheus-community/kube-prometheus-stack

Check your targets in prometheus and your etcd dashboard in grafana.