Policy Management in Kubernetes is Changing

The Kubernetes API Server is one of the core components of the Kubernetes Control Plane. This component exposes the Kubernetes API and acts like a front end for the Control Plane. When a user or a process interacts with Kubernetes, the API Server handles those requests and also API Server validates and configures the Kubernetes API objects such as deployments or namespaces. But of course with the help of the Kubernetes Admission Controllers. So, what is an Admission Controller?

As explained in the official Kubernetes documentation, an Admissions Controller is a piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorised. There are two types of Admission Controllers, Mutating and Validating. Some controllers may be both but generally speaking, controllers are either mutating the requests and modify the objects or as the name suggests, validating the requests.

To understand it more clearly, let me explain how it works in general. Let’s say you have a YAML file with some Kubernetes object definitions such as a deployment or a pod and you want to apply that to your cluster. When you send that request to the API Server, it first checks if you have the necessary permissions to create that object. Then that request is forwarded to the Mutating Admission Controllers. Those controllers may change the definition of that object depending on your configurations in the cluster. After that, if the definition schema is valid, the request is then being forwarded to the Validating Admission Controllers. If everything is correct, the resource is going to be created and the details for the objects are sent to the etcd.

How Admission Controllers Work

Third party policy engine tools like Kyverno, OPA Gatekeeper and Datree use Mutating and Validating Admission Webhooks to manage policies. For example, Kyverno runs as a Dynamic Admission Controller inside the cluster. It receives Mutating and Validating webhook requests from the Kubernetes API Server and it applies matching policies to return results that enforce admission policies or reject requests. OPA Gatekeeper is similar as well. It’s a Validating and Mutating webhook that enforces policies executed by Open Policy Agent. But these tools have their own formats for defining these policies and there are of course differences between them. For example, OPA uses a language called Rego and it can be difficult for some people to write or even understand policies at first because it’s a different language and there is a learning curve.

But in Kubernetes 1.26, the first alpha release of Kubernetes Validating Admission Policies became available. With Kubernetes 1.28, it is now in beta stage. This feature is bringing standardisation for declarative policy management into the Kubernetes API. That means we are able to manage and define policies in a Kubernetes native way. Validating Admission Policies use Common Expression Language (CEL) to define policy rules and it allows us to have parameterised and scoped policies. Building, installing and managing third party webhooks can be quite complicated but this new feature will remove all that requirement for calling a remote webhook and it will allow us to manage policies inside the cluster as a built-in process within the API with the CEL expressions.

Does that mean it is the end of third party tools? I would say no personally and I believe tools like OPA Gatekeeper and Kyverno will adapt their methods to this new approach. But if they don’t, I think there might be a problem for these tools because most of the users of Kubernetes wouldn’t want to choose to manage a different tool for policy management if that’s already available natively in the Kubernetes API. That is my personal opinion of course.

Let’s see Kubernetes Validating Admission Policies in action!

Before jumping into the demo I should remind you that this feature is not enabled by default in the Kubernetes API and you need to enable the necessary ValidatingAdmissionPolicy feature gate. Depending on your Kubernetes version, you also need to enable admissionregistration.k8s.io/v1alpha1 or admissionregistration.k8s.io/v1beta1 in the API. In this demo, I’ll be using a Kubernetes cluster with the 1.27 version and I’ll use the ‘v1alpha1’ API version for the Validating Admission Policy resources.

For the policies, we need two resources. First, the policy that will define the actual rules for a resource and the validation actions and secondly, the binding resource that will bind the actual policy to for example a namespace.

I will use a simple example policy that will define a rule for deployment objects in the namespace that has a label defined to have a limit for the number of replicas. Let me first create a simple namespace resource.

apiVersion: v1
kind: Namespace
metadata:
  labels:
    environment: demo
  name: demo

Nothing special here of course and I will create this namespace using kubectl apply. Let’s define our policy:

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicy
metadata:
  name: "demo-policy"
spec:
  matchConstraints:
    resourceRules:
    - apiGroups:   ["apps"]
      apiVersions: ["v1"]
      operations:  ["CREATE", "UPDATE"]
      resources:   ["deployments"]
  validations:
    - expression: "object.spec.replicas <= 5"
      message: "You can not have more than 5 replicas in the demo namespace for deployments"

This policy will check the deployment resources and this will track the create and update actions. And these deployments needs to have 5 or less replicas defined. But I also need a Policy Binding resource:

apiVersion: admissionregistration.k8s.io/v1alpha1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: "demo-binding"
spec:
  policyName: "demo-policy"
  validationActions: [Deny]
  matchResources:
    namespaceSelector:
      matchLabels:
        environment: demo

As you can see from the binding resource, this policy is going to be enforced to the deployment resources in the namespaces that have a label “environment = demo”. Let’s create these resources with kubectl apply.

Now it’s time to test our new policy. I’m going to use a simple deployment definition:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-app
  labels:
    app: nginx
spec:
  replicas: 6
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx

I am not going define a Service or anything specific for this deployment object. Let’s apply this deployment:

kubectl apply -f deployment.yaml -n demo

I couldn’t create the deployment object as expected. The policy kicked in and here is the error message:

The deployments "demo-app" is invalid: : ValidatingAdmissionPolicy
'demo-policy' with binding 'demo-binding' denied request: 
You can not create more than 5 replicas in the demo namespace for deployments

That’s a success for the purposes of this demo of course. If I change the the spec.replicas to 5, the deployment will be created successfully.

As you can see from the example demonstration I’ve shown in this blog post, Validating Admission Policies makes it extremely easier to write, enforce and use policies in Kubernetes without the need of a third party tool. This is also really flexible. With the usage of CEL expressions, you can create well defined custom policies for multiple actions and multiple validation rules. You can of course add more validation rules to expand these policies or you can use different validation actions. I truly believe this will become the de facto approach for policy management in Kubernetes.

This feature is now in beta stage for Kubernetes 1.28, so you can try it out yourselves by enabling the feature gate but I believe when it’s enabled by default or when it’s moved to the stable, this feature will become really handy.