Intro to Azure Policy for Kubernetes

author_profile
Ori Abargil
Wednesday, Mar 27th, 2024

Amid rapid changes in the cloud native space and related technologies, Kubernetes has risen as the favored solution for deploying and managing applications at scale. Recognizing the extensive use of Kubernetes, providers are now focusing on developing security services for this technology.

Azure Policy is a service offered by Microsoft Azure that helps enforce organizational standards and compliance in Azure environments. Azure Policy allows users to create, assign, and manage policies to control Azure resources at scale.

This post provides a comprehensive introduction to Azure Policy for Azure Kubernetes Service (AKS) clusters, offering detailed insights into its functionality and implementation. There are 3 types of Azure policies:

  1. Built-in – policies that are maintained and created by Microsoft.
  2. Custom – policies that are created by customers based on their requirements.
  3. Static - Indicates a Regulatory Compliance policy definition with Microsoft ownership.

Azure Policy for AKS

Azure Policy for AKS extends Gatekeeper v3, an admission controller webhook for Open Policy Agent (OPA), to apply at-scale enforcements and safeguards to your cluster components in a centralized, consistent manner. Cluster components include pods, containers, and namespaces.

Azure Policy add-on for AKS allows customers to audit and enforce policies to their Kubernetes resources. For each AKS cluster Azure Policy add-on is enabled automatically. You can check if Azure Policy add-on is enabled in your cluster by running the following command:

az aks show --query addonProfiles.azurepolicy -g <rg> -n <cluster-name>

Azure Policy add-on architecture

The diagram below presents all components that Azure installs in your cluster when Azure Policy add-on is enabled

large_image

As we can see in the diagram, Azure installs 2 main components in the cluster when Azure Policy add-on is enabled. The first component is a regular installation of Gatekeeper which is installed in a dedicated namespace called gatekeeper-system, and the second is a Kubernetes resources which Azure installs in kube-system namespace.

In this post we will not get into the task and functionality of the installed pods by Azure Policy. We will focus on them in the next parts.

Gatekeeper Constraint and Constraint Template

ConstraintTemplate is a Kubernetes resource created by Gatekeeper. ConstraintTemplate defines how to validate Kubernetes objects in Gatekeeper's Kubernetes admission controller.
ConstraintTemplates contains two main elements:

  1. Rego code - the Rego policy is where you define the conditions that must be satisfied for a resource to be accepted or rejected during admission control.
  2. Schema - defines the expected structure of the parameters used in the Rego policies.

Constraint is a Kubernetes resource created by Gatekeeper, which is used to inform Gatekeeper that a ConstraintTemplate should be enforced, and what the enforcement actions are.

Note: A ConstraintTemplate can be used by several Constraints.

When Azure Policy add-on is deployed on the cluster, some ConstraintTemplates are created automatically. You can list them by running the following command:

kubectl get constrainttemplates

Each ConstraintTemplate is associated to a built-in policy definition that is provided by Azure. The following table shows the mapping between Azure policy definition names and their ConstraintTemplates.

Gatekeeper ConstraintTemplateAzure policy
k8sazurev2noprivilegeKubernetes cluster should not allow privileged containers
k8sazurev1blockdefaultKubernetes clusters should not use the default namespace
k8sazurev1ingresshttpsonlyKubernetes clusters should be accessible only over HTTPS
k8sazurev1serviceallowedportsKubernetes cluster services should listen only on allowed ports
k8sazurev2blockautomounttokenKubernetes clusters should disable automounting API credentials
k8sazurev2blockhostnamespaceKubernetes cluster containers should not share host process ID or host IPC namespace
k8sazurev2containerallowedimagesKubernetes cluster containers should only use allowed images
k8sazurev3allowedcapabilitiesKubernetes cluster containers should only use allowed capabilities
k8sazurev3allowedusersgroupsKubernetes cluster pods and containers should only run with approved user and group IDs
k8sazurev3containerlimitsKubernetes cluster containers CPU and memory resource limits should not exceed the specified limits
k8sazurev3disallowedcapabilitiesKubernetes clusters should not grant CAP_SYS_ADMIN security capabilities
k8sazurev3enforceapparmorKubernetes cluster containers should only use allowed AppArmor profiles
k8sazurev3hostfilesystemKubernetes cluster pod hostPath volumes should only use allowed host paths
k8sazurev3hostnetworkingportsKubernetes cluster pods should only use approved host network and port range
k8sazurev3noprivilegeescalationKubernetes clusters should not allow container privilege escalation1
k8sazurev3readonlyrootfilesystemKubernetes cluster containers should run with a read only root file system

Azure Policy definition for Kubernetes structure

With Azure Policy, you have the option to implement a custom policy to restrict and control specific configurations or actions as needed. The structure for Kubernetes policy definition is:

{
  "properties": {
    "displayName": "POLICY NAME",
    "policyType": "Custom",
    "mode": "Microsoft.Kubernetes.Data",
    "description": "POLICY DESCRIPTION",
    "metadata": {
      "version": "",
      "category": ""
    }
    "parameters": {LIST OF PARAMETERS}
    "policyRule": { 
      "if": {
        CONDITION | LOGICAL OPERATOR
      },
      "then": {
        "effect": "audit/deny/disabled",
        "details": {
          "templateInfo": {
            "sourceType": "PublicURL/Base64Encoded",
            "url/content": "URL/CONTENT"
          }
          "apiGroups": [""],
          "kinds": [K8S RESOURCES TO EVALUATE],
        }
      }
    }  
  }
}
  • displayName – the policy name.
  • policyType – one of the 3 types that we mentioned earlier.
  • mode - Microsoft.Kubernetes.Data for managing Kubernetes clusters and components such as pods, containers, and ingresses.
  • description – the policy description.
  • metadata - optional property stores information about the policy definition. Here you can find the valid properties for metadata.
  • parameters – simplify policy management, to define parameters in your policy rule each parameter should have the properties that listed here.
  • policyRule – consists of If and Then blocks.
    • The If contains one or more conditions that define when the policy is enforced.
    • The then consists of effect and the subproperties details (because we are using Microsoft.Kubernetes.Data mode).
      • effect: when using Microsoft.Kubernetes.Data Resource Provider mode the effect field is limited to audit, deny and disabled.
      • details: consists of templateInfo, apiGroups and kinds. templateInfo is defines the constraintTemplate that will deploy in the cluster by Azure.

Policy assignment

In a policy assignment we are connecting an Azure Policy definition with a Kubernetes cluster. During the policy assignment process we need set a Scope to the management group, subscription, or resource group of the K8s cluster. Since each cluster could have its own policies and restrictions, we recommend that you create each AKS cluster in a separate resource group. 

Another field that is set in the policy assignment is Policy enforcement that has 2 options:

  1. Enabled – The policy will be enforced on the cluster. Kubernetes admission requests with violations will be denied (when the policy definition effect is set to deny). When a resource violates a constraint, it means that the resource does not meet the conditions specified in the associated Rego policy, and therefore, it may be rejected or modified according to the policy enforcement.
  2. Disabled – The policy will not be enforced on the cluster. Kubernetes admission requests with violations will not be denied.

The policy enforcement field impacts on the final effect of the created constraint within the Kubernetes cluster. The table below shows the different setups of the policy enforcement and effect field, and the resulting final effect that is created in the Kubernetes constraint.

Policy enforcement/EffectAuditDeny
EnabledAuditDeny
DisabledAuditAudit

In the example below I assigned a policy definition to my cluster with policy enforcement disabled and set to “deny” in the effect parameter:

large_image


The constraint in the cluster deployed with dry-run (which equals to audit) enforcement action.

When I changed the policy enforcement to enable, the constraint created in the cluster with a deny enforcement action.

Policy evaluation

The add-on checks for changes in the policy assignments with Azure Policy service every 15 minutes. The changes in the assignments trigger a create, delete or update of constraint templates and constraints.

More details on this process will be covered in the next part.

Summary

Azure Policy, a Microsoft Azure service, plays a crucial role in enforcing organizational standards and ensuring compliance within Azure environments. In this blog, we covered the functionalities of Azure Policy for AKS, provided valuable information to help you consider the adoption of Azure Policy. We guided readers through the process of creating, assigning, and managing policies at scale and presented the structure of Azure Policy definitions.

Popup Image