[k8saudit] Documentation clarification regarding audit webhook backend

[heitzflorian]

What to document

I would like to have your point of view, advices or recommendations regarding k8saudit events from it's origine (kube-apiserver) to reach falco k8saudit plugin endpoint: should we use an internal clusterIP service, or external HTTP API webhook backend in webhook-config.yaml file ?
Those clarifications could then be transcripted accordingly in documentation to be more explicit.

Context

After some extra-topic cross discussion with Brad Davidson (brandond) from SUSE Rancher, he do not recommend to host falco k8saudit webhook backend in the audited cluster, but rather outside of the cluster so it does not depend from cluster integrity.
As an alternative if log externalisation not possible, he advice to tailor falco daemonset to read logs from disk (a.k.a: on master nodes using volumes and volumeMounts). This point can be slightly challenged in terms of security as falco already run as a daemonset with privileged: yes which is quite high privileges yet, but this is not the point of the current clarification request.

Regarding the first recommendation to host falco k8saudit webhook backend outside of the audited cluster:

• PROS:
  • less fragile in case of cluster issue, stored elsewhere
  • no need to set the extra parameter dnsPolicy: ClusterFirstWithHostNet at kube-apiserver POD spec level which is defined with hostNetwork: true: external FQDN or IP references in the webhook-config.yaml file are natively fitted to external DNS resolution.
• CONS:
  • leads to complexification of falco deployment
    • 2 falco instantiation: 1 inside the audited clusters to raise SYSCALL source events, 1 outside the audited clusters to catch k8saudit logs. 2 may then forward to a single falcosidekick that will dispatch to N-outputs configured.

---

From the falco docs k8saudit plugins | Github:

Usage
Requirements
The Kubernetes cluster must have the audit logs enabled and configured to send the audit logs to the plugin. We provide the audit-policy.yaml, which is tailored for the k8saudit plugin. The audit-policy.yaml is of vital importance; it defines the rules about what events should be recorded and what data they should include. The rules shipped with the k8saudit plugins rely on those events. The webhook-config.yaml shows how to configure the webhook backend to send events to an external HTTP API.

---

From the webhook-config.yaml.in example file:

apiVersion: v1
kind: Config
clusters:
- name: falco
  cluster:
    server: http://$FALCO_SERVICE_CLUSTERIP:8765/k8s-audit
contexts:
- context:
    cluster: falco
    user: ""
  name: default-context
current-context: default-context
preferences: {}
users: []

Thanks in advance for your recommendations, advices or point of view.