Ensuring high application availability is a key concern for anyone managing workloads on Kubernetes. Imagine Kubernetes as an orchestra, where each component of your application plays a vital role. Pod Disruption Budgets (PDBs) act like stage managers, ensuring the show goes on even when some components need a break. By defining a PDB, you set rules on how many pods can be offline at a given time, ensuring critical applications keep running even during node maintenance, cluster upgrades, or other routine operations. This blog dives deep into Pod Disruption Budgets, providing the knowledge you need to effectively manage disruptions and maintain application uptime.

What is a Pod Disruption Budget (PDB)?

A Kubernetes Pod Disruption Budget specifies a minimum level of availability for a given set of pods. Kubernetes uses PDBs to determine if pods can be evicted, preventing disruptions that would cause the number of running replicas to fall below the defined target size. If evicting pods would violate the PDB’s rules, Kubernetes halts the process to maintain the desired pod availability.

How to Define and Apply a PDB

Setting up a Kubernetes PDB typically involves:

1. Identify the application you want to protect with a PDB. This is typically an application specified by a built-in Kubernetes controller like Deployment, ReplicationController, ReplicaSet, or StatefulSet. Note the controller's .spec.selector so you can correctly apply Pod Disruption Budget parameters.
2. Assess how your application reacts to disruptions to determine the appropriate PDB configuration. For example, a stateless frontend might tolerate a 10% reduction in serving capacity, while a single-instance stateful application might require no downtime.
3. Create a PDB definition as a YAML file. This file will include the apiVersion, kind, metadata, and spec. The .spec section contains the selector, minAvailable, and/or maxUnavailable fields.

4. Apply the PDB in Kubernetes and use kubectl apply -f mypdb.yaml to create your Pod Disruption Budget in the cluster. 

Specifying a Pod Disruption Budget

A PDB has three key fields:

• .spec.selector: A label selector to specify the set of pods to which the Pod Disruption Budget applies. This field is required.
• .spec.minAvailable: Specifies the minimum number of pods from the selected set that must remain available after eviction. It can be an absolute number or a percentage. Kubernetes rounds up to the nearest integer when using percentages.
• .spec.maxUnavailable: Specifies the maximum number of pods from the selected set that can be unavailable after eviction. Available in Kubernetes 1.7 and higher, it can be an absolute number or a percentage.

You can specify only one of maxUnavailable or minAvailable in a single PDB. Using maxUnavailable is recommended as it automatically responds to changes in the number of replicas of the corresponding controller.

Practical Examples: Stateless vs. Stateful Workloads

Below are some typical k8s Pod Disruption Budget scenarios:

• Stateless frontends: Use a PDB with minAvailable: 90% to ensure no more than a 10% reduction in serving capacity.

• Single-instance Stateful Application: Set a PDB with maxUnavailable=0 to prevent termination without consultation.

• Multiple-instance Stateful application: Set maxUnavailable to 1 or minAvailable to quorum-size.

• PDB using minAvailable: The below example requires at least 2 pods to be available.

• PDB using maxUnavailable: The below example allows a maximum of 1 pod to be unavailable.

Verifying Your Pod Disruption Budget Status

After creating a Kubernetes PDB, it's important to verify its status using: 

kubectl get poddisruptionbudgets.

The output provides key information:

• MIN AVAILABLE: The minimum number of pods that should be available.
• MAX UNAVAILABLE: The maximum number of pods that can be unavailable.
• ALLOWED DISRUPTIONS: The number of disruptions that are currently allowed. A non-zero value indicates that the disruption controller has recognized the pods and updated the PDB status.

For more detailed information, use kubectl get poddisruptionbudgets <pdb-name> -o yaml to view the PDB's full status, including the current and desired health of the pods. A pod is considered healthy if it has a .status.conditions item with type="Ready" and status="True".

Common Pitfalls and Misconfigurations to Avoid

• UpgradeFailed errors due to eviction failures: Kubernetes cluster upgrades can fail due to eviction failures caused by Pod Disruption Budgets. This occurs when a pod is protected by a restrictive PDB policy, preventing it from being drained.
  
  • Solution: Adjust the PDB to enable pod draining by modifying the Min Available / Max unavailable parameter or increasing the number of Running pods / Replicas. Alternatively, back up, delete, and redeploy the PDB, or delete the pods that can't be drained. Learn more about resolving eviction failures in Kubernetes upgrades.
• Setting overly restrictive PDBs: Setting maxUnavailable to 0% or 0, or minAvailable to 100% or the number of replicas, requires zero voluntary evictions. This can prevent draining a Node running one of those Pods.
• Using PDBs with arbitrary workloads and selectors: When using PDBs with pods controlled by resources other than Deployment, ReplicaSet, StatefulSet, and ReplicationController, only .spec.minAvailable can be used with an integer value.
• Empty selector: The behavior for an empty selector differs between the policy/v1beta1 and policy/v1 APIs for PodDisruptionBudgets. For policy/v1beta1, an empty selector matches zero pods, while for policy/v1, an empty selector matches every pod in the namespace.
• Not truly guaranteeing availability: A disruption budget does not guarantee that the specified number/percentage of pods will always be up. Node failures can still bring the number of available pods below the specified size. The budget only protects against voluntary evictions.

You can define criteria for when unhealthy pods should be considered for eviction using .spec.unhealthyPodEvictionPolicy.

Detecting Misconfigured Pod Disruption Budgets in a Live Environment

Detecting misconfigured Kubernetes Pod Disruption Budgets in a live environment or during upgrades can get complicated quickly—often requiring specialized tooling and automated checks. If you prefer not to build and maintain all of that yourself, Chkk Risk Ledger can help you identify misconfigured PDBs in your running clusters, catching potential issues before they lead to bigger problems. And when it’s time to upgrade, Chkk Upgrade Copilot spots any PDB configurations that might block or complicate the process. By using these tools, you ensure that application teams fix PDBs at the right time and configure them correctly, maintaining the balance between availability and agility that Kubernetes requires.