Migrating from PSPs to Pod Security Admission

Step 1: Check that all pods run under the ibm-privileged-psp PSP

Complete the following steps ensure that all pods run under the ibm-privileged-psp PSP.

One difference between Pod Security Admission and PodSecurityPolicies is that Pod Security Admission is validating while the PodSecurityPolicies can be mutating. This makes it important that pods use the ibm-privileged-psp, which is not a mutating PSP, before you upgrade. Because both Pod Security Admission and the ibm-privileged-psp are validating, pods work the same under either one.

For example, if a pod is currently running under ibm-restricted-psp, it might set the fsGroup and supplementalGroups to 1 in the pod securityContext section, based on the MustRunAs range in the PSP. In other words, the ibm-restricted-psp can change the pod securityContext. Such changes are visible as a difference between the securityContext of the running pod and the securityContext in the pod template of the deployment or similar resource that creates the pod. The ibm-privileged-psp is not mutating, so a pod running under it might run with different groups and might not be able to access data stored in an existing PVC.

1. Get the details of all pods and check their PSPs.

   kubectl get pods -A -o jsonpath="{.items[*].metadata.annotations.kubernetes\.io\/psp}" | tr " " "\n" | sort -u
   

2. Review the command output for any PSPs that are not ibm-privileged-psp. If your pods are using a different PSP, you must update your application to use the ibm-privileged-psp before upgrading your cluster. If no other PSPs are listed, you can continue with the upgrade.

Upgrading to 1.25 is not recommended if the output lists a PSP other than the ibm-privileged-psp PSP.

Step 2: Verify the privileged-psp-user cluster role binding uses the default configuration

Complete the following steps to verify that the privileged-psp-user cluster role binding uses the default configuration. This ensures that all service accounts and users are authorized to use the ibm-restricted-psp through the privileged-psp-user cluster role binding.

Your upgrade to 1.25 fails if the cluster role binding does not have the default role and subjects exactly as shown in the following example.

1. Get the details privileged-psp-user cluster role binding.

   kubectl get clusterrolebinding privileged-psp-user -o yaml
   

2. Review the role and subjects in the output. If the output is different than the following example, do not upgrade to 1.25. If your privileged-psp-user cluster role binding matches the following example, you can continue with the upgrade.

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     creationTimestamp: "2022-10-06T20:12:36Z"
     name: privileged-psp-user
     resourceVersion: "151862"
     uid: 15014736-94d2-4cba-a3a8-92dd36de453b
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: ibm-privileged-psp-user
   subjects:
   - apiGroup: rbac.authorization.k8s.io
     kind: Group
     name: system:masters
   - apiGroup: rbac.authorization.k8s.io
     kind: Group
     name: system:nodes
   - apiGroup: rbac.authorization.k8s.io
     kind: Group
     name: system:serviceaccounts
   - apiGroup: rbac.authorization.k8s.io
     kind: Group
     name: system:authenticated
   

Step 3: Verify the restricted-psp-user cluster role binding uses the default configuration

Complete the following steps to verify that the restricted-psp-user cluster role binding uses the default configuration. This ensures that all service accounts and users are authorized to use the ibm-restricted-psp through the restricted-psp-user cluster role binding.

Your upgrade to 1.25 fails if the cluster role binding does not include the default role and subjects as shown in the following example.

1. Get the details of the restricted-psp-user cluster role binding

   kubectl get clusterrolebinding restricted-psp-user -o yaml
   

2. Review the command output. Do not upgrade your cluster to version 1.25 if the cluster role binding does not include default settings for the role and subjects as shown in the following example.

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     creationTimestamp: "2022-10-06T20:13:10Z"
     name: restricted-psp-user
     resourceVersion: "151890"
     uid: 4edb362f-9933-48d7-95e2-f41cd9f4dead
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: ibm-restricted-psp-user
   subjects:
   - apiGroup: rbac.authorization.k8s.io
     kind: Group
     name: system:masters
   - apiGroup: rbac.authorization.k8s.io
     kind: Group
     name: system:nodes
   - apiGroup: rbac.authorization.k8s.io
     kind: Group
     name: system:serviceaccounts
   - apiGroup: rbac.authorization.k8s.io
     kind: Group
     name: system:authenticated
   

Step 4: Checking for non-IBM PSPs

Complete the steps to check whether your cluster uses non-IBM PSPs.

Upgrading to version 1.25 fails if the command output includes additional PSPs to those in the following example. You might have additional PSP from third party applications and require updates to the applications or working with the application vendors to determine the proper upgrade strategy to use for those applications.

1. List your pod security policies.

   kubectl get podsecuritypolicies -o name
   

2. Review the command output.

   Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
   podsecuritypolicy.policy/ibm-anyuid-hostaccess-psp
   podsecuritypolicy.policy/ibm-anyuid-hostpath-psp
   podsecuritypolicy.policy/ibm-anyuid-psp
   podsecuritypolicy.policy/ibm-privileged-psp
   podsecuritypolicy.policy/ibm-restricted-psp
   

3. Update your apps to use the IBM PSPs.

Step 1: Enable Pod Security admission in your 1.24 cluster

Enable Pod Security admission in your 1.24 cluster. This command updates the cluster master to use the new Pod Security admission configuration. It may take a few minutes for the cluster master to update.

ibmcloud ks cluster master pod-security set --cluster <CLUSTER>

Step 2: Review namespace permissions

Review the namespace permissions in the external Kubernetes documentation. If your Kubernetes permissions are managed by IAM service roles, the Manager service role is required to create or edit namespaces and to set pod security labels.

• Review namespace permissions

Step 3: Simplify and standardize PSPs

Clean up your Pod Security configuration by following the steps in the external Kubernetes documentation. Do not modify or delete any of the following PSPs: ibm-privileged-psp, ibm-anyuid-psp, ibm-anyuid-hostpath-psp, ibm-anyuid-hostaccess-psp and ibm-restricted-psp.

• Simplify & standardize PodSecurityPolicies

Step 4: Update the namespaces in your cluster

Update the namespaces in your cluster by following the steps in the external Kubernetes documentation. These steps must be performed on every namespace in the cluster that is not managed by IBM Cloud. Note the exceptions included in this section.

Do not delete or modify the Pod Security labels for the following namespaces that are managed by IBM Cloud: kube-system, ibm-system, ibm-operators.

In step 3.d. Bypass PodSecurity Policy of the external documentation linked in this step, do not create the suggested privileged PSP. If you create this additional PSP, it must be deleted before you upgrade to version 1.25, as detailed in the upgrade requirements. Instead, use the following command to create a RoleBinding to the ibm-privileged-psp-user cluster role: kubectl create -n $NAMESPACE rolebinding disable-psp --clusterrole ibm-privileged-psp-user --group system:serviceaccounts:$NAMESPACE.

• Update namespaces

Step 5: Review the namespace creation process

Review the information in the external Kubernetes documentation to ensure that the Pod Security profile is applied to any new namespaces that are created in your cluster.

• Review namespace creation processes

Step 6: Optional. Disable the PSP feature in the cluster

1. Disable the PodSecurityPolicy admission controller in the cluster. This command updates the cluster master to use the new configuration.

   ibmcloud ks cluster master pod-security policy disable --cluster <cluster>
   

2. Wait a few minutes for the cluster master update to complete.
3. Delete your PSPs and any associated Roles, ClusterRoles, RoleBindings, and ClusterRoleBindings. Make sure that the components you delete do not grant any other unrelated permissions that you might need elsewhere. Do not delete the following PSPs: ibm-privileged-psp, ibm-anyuid-psp, ibm-anyuid-hostpath-psp, ibm-anyuid-hostaccess-psp and ibm-restricted-psp. Do not delete the following ClusterRoles and ClusterRoleBindings: privileged-psp-user and restricted-psp-user.

If you need to re-enable PSPs in your 1.24 cluster, run ibmcloud ks cluster master pod-security policy enable --cluster <cluster>. This command updates the cluster master to use the PSP configuration.

Step 7: Optional. Upgrade your cluster

Upgrade your cluster to version 1.25 to use Pod Security admission. Or, keep your cluster at version 1.24 with Pod Security admission enabled until you are ready to upgrade.