Deploying Daemonset Defenders in GKE Autopilot Cluster

• Defender pods get stuck in Pending state as the nodes don't have capacity to support that workload.
• Often pod logs will reflect "GKE Policy Controller rejected the request because it violates one or more policies"

• Prisma Cloud Compute (Self-hosted) all versions
  • Defenders
  • Google Kubernetes Engine (GKE) Autopilot
• Prisma Cloud Enterprise Edition (SaaS)

• GKE Autopilot is a mode of operation in GKE in which Google manages the cluster configuration, including nodes, scaling, security, and other preconfigured settings.  This automation can affect the deployment of nodes and their resource allocations.

• GKE Autopilot limits some administrative functions that affect all workload Pods, including Pods managed by DaemonSets. DaemonSets that perform administrative functions on nodes using elevated privileges, such as the privileged security context, won't run on Autopilot clusters unless explicitly allowed by GKE.

Best practices when deploying DaemonSets on Autopilot:

• Ensure GKE Autopilot option is selected in Manage > Defenders > Deploy > Advanced Settings

• Deploy DaemonSets before any other workloads.
• Set a higher PriorityClass on DaemonSets than regular Pods.
  • The higher PriorityClass lets GKE evict lower-priority Pods to accommodate DaemonSet pods if the node can accommodate those pods.
  • This helps to ensure that the DaemonSet is present on each node. 
  • This requires manual addition to the deployment yaml file per the referenced documentation .

• Autopilot Overview 
• Setting Priority Class 
• Install-gke-autopilot