bedrock

Disruption budgets

A PodDisruptionBudget tells Kubernetes how many pods of a workload must stay up during voluntary disruptions, such as kubectl drain before node maintenance. A drain waits rather than break a budget. Budgets do not protect against a node failing on its own.

Every budget in bedrock allows one pod down at a time. Each one is written as a minAvailable one below the workload’s replica count, so if the replica count changes, the budget must change with it.

BudgetNamespaceSelectsFile
kyverno-admission-controllerkyvernoKyverno’s admission controllertenant-workload/kyverno/admission/admission-pdb.yaml
garagegarageGarage, minAvailable: 2tenant-workload/storage/garage/garage.yaml
openbaoopenbaoOpenBao, minAvailable: 2tenant-workload/openbao/openbao.yaml
bedrock-deciderbedrock-failoverThe two deciders, minAvailable: 1tenant-workload/failover/decider.yaml
prometheusapplicationPrometheustenant-workload/observability/prometheus/prometheus.yaml
lokiapplicationLokitenant-workload/observability/loki/loki-pdb.yaml
<cluster>-pooler-rweach database namespaceThat cluster’s PgBouncer poolertenant-workload/database/pooler.yaml
natsnatsNATStenant-workload/event-buses/nats.yaml
valkeyvalkeyValkeytenant-workload/event-buses/valkey.yaml

#Workloads managed by an operator

Some workloads get their budgets from their operator instead of from the repository:

WorkloadBudget comes from
Database clustersCloudNativePG creates one for the primary and one for the replicas
Longhorn volumesLonghorn protects the last healthy replica of each volume during a drain
KafkaStrimzi

#Workloads with no budget

Tempo, Grafana, Alertmanager, OpenCost and Reloader run a single pod, so a budget would block every drain. They lose nothing important while they restart. The DaemonSets, Alloy, node-exporter and Falco, are not evicted by a drain.

#Check a budget

shell
kubectl get pdb -A

ALLOWED DISRUPTIONS must be at least 1 before a node is drained. If it is 0, a pod of that workload is already down, and draining would wait until it is back.