Grafana's two secrets, and where they live

grafana-auth carries admin-user and admin-password. It is a SealedSecret next to the chart that consumes it, and it is the only copy: Grafana mounts it, and terraform/grafana-config reads it back out of the cluster at apply time.

grafana-oidc carries client_id and client_secret for the authentik client. It is not sealed — terraform/authentik-config mints the client, writes the credentials to OpenBao at authentik/observability/grafana, and an ExternalSecret pulls them into the namespace. That is the right split: the admin password is chosen by an operator, while the OIDC secret is generated by the system that owns the client, so nobody has to see it.

The two live at different mount paths — /etc/secrets/auth and /etc/secrets/oidc — and grafana.ini reads each with $__file{}.

Sealing is offline — the pinned key pair lives in secrets/sealed-secrets.yaml, so no cluster access is needed to produce the ciphertext:

sops -d --extract '["tls_crt"]' secrets/sealed-secrets.yaml > /tmp/sealing.crt
kubeseal --cert /tmp/sealing.crt --format yaml < /tmp/grafana-auth.yaml \
  > flux/apps/observability/grafana/auth-sealed-secret.yaml

Write the plaintext input under /tmp, never in the working tree — nothing in .gitignore would catch it, and a rule broad enough to catch it would also shadow the sealed output.

Rotating the admin password is one resealed file. The tofu root picks the new value up on its next apply because it never stored the old one.