bedrock

Cilium

Cilium ships with RKE2 and is configured through a HelmChartConfig that the bootstrap writes to the RKE2 manifests directory before the first server starts.

#Configuration

SettingValueWhy
kubeProxyReplacementtrueOne component handles all service traffic, so packet identities stay correct
k8sServiceHost / k8sServicePortlocalhost / 6443Without kube-proxy, Cilium needs a fixed API address at startup. Every node is a server, so the local API server is always there.
encryption.typewireguardPod traffic between nodes is encrypted
hubble.enabled, relay, uitrueFlow logs, a cluster-wide relay and the web interface

RKE2 itself runs with disable-kube-proxy: true and cni: cilium.

#Why kube-proxy is replaced

With kube-proxy and Cilium both handling services, traffic arriving through a node port was rewritten by kube-proxy before Cilium saw it. Cilium then classified it as coming from outside the cluster, so rules that allowed the node never matched. With Cilium alone on the path, a packet keeps its real identity from the first hop.

The localhost API address works because every node runs the API server. A cluster with worker-only nodes would need a different address there.

#Network policy

Policies are written as CiliumNetworkPolicy and CiliumClusterwideNetworkPolicy, not the plain Kubernetes NetworkPolicy. The Cilium types add what bedrock needs:

  • a cluster-wide policy that selects every pod at once
  • entities such as kube-apiserver, host and remote-node, so the API server and the nodes can be named without knowing their IP addresses
  • layer 7 rules that limit which HTTP requests a flow may carry

Each namespace has one file in tenant-workload/netpols/. The rules and the reasoning are in networking and every allowed flow is listed in the network flows reference.

#Entities used

EntityMeansUsed for
hostThe node the pod runs onKubelet health checks, scraping the node
remote-nodeAny other node in the clusterScraping other nodes, Longhorn traffic
kube-apiserverThe Kubernetes API serverOperators and controllers that watch the cluster
clusterAnything inside the clusterDNS queries to CoreDNS

#Hubble

Hubble records every flow Cilium handles, including the ones it drops. It is the first place to look when a connection fails after a policy change. See find a dropped connection.