Adding File Storage for VPC to apps

Quick start for File Storage for VPC with dynamic provisioning

Create a persistent volume claim (PVC) to dynamically provision File Storage for VPC for your cluster. Dynamic provisioning automatically creates the matching persistent volume (PV) and orders the file share in your account.

1. Access your Red Hat OpenShift cluster.

2. Save the following code to a file called my-pvc.yaml. This code creates a claim that is named my-pvc by using the ibmc-vpc-file-dp2 storage class, billed monthly, with a gigabyte size of 10Gi.

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: my-pvc # Enter a name for your PVC.
   spec:
     accessModes:
     - ReadWriteMany # The PVC can be mounted by multiple pods. All pods can read from and write to the volume.
     resources:
       requests:
         storage: 10Gi # Enter the size of the storage in gigabytes (Gi).
     storageClassName: ibmc-vpc-file-dp2 # Enter the name of the storage class that you want to use.
   

3. Create the PVC.

   oc apply -f my-pvc.yaml
   

4. Verify that your PVC is created and bound to the PV. This process can take a few minutes.

   oc describe pvc my-pvc
   

   Example output

   Name:        my-pvc
   Namespace:    default
   StorageClass:    ""
   Status:        Bound
   Volume:        pvc-0d787071-3a67-11e7-aafc-eef80dd2dea2
   Labels:        <none>
   Capacity:    20Gi
   Access Modes:    RWX
   Events:
       FirstSeen    LastSeen    Count    From                                SubObjectPath    Type        Reason            Message
       ---------    --------    -----    ----                                -------------    --------    ------            -------
       1m        1m        1    {ibm.io/ibmc-vpc-file 31898035-3011-11e7-a6a4-7a08779efd33 }            Normal        ProvisioningSucceeded    Successfully provisioned volume pvc-0d787071-3a67-11e7-aafc-eef80dd2dea2
   
   

5. Save the following deployment configuration to file called deployment.yaml and reference the PVC that you created in the previous step.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: my-deployment
     labels:
       app: my-deployment
   spec:
     selector:
       matchLabels:
         app: busybox
     template:
       metadata:
         labels:
           app: busybox
       spec:
         containers:
         - name: busybox
           image: busybox:1.28 # The name of the container image that you want to use. 
           command: [ "sh", "-c", "sleep 1h" ]
           volumeMounts:
           - name: my-vol
             mountPath: /data/demo
         volumes:
         - name: my-vol
           persistentVolumeClaim:
             claimName: my-pvc
   

   mountPath

   In the container volume mounts section, enter the absolute path of the directory to where the volume is mounted inside the container. Data that is written to the mount path is stored under the root directory in your physical File Storage for VPC instance. If you want to share a volume between different apps, you can specify volume sub paths for each of your apps.

   volumeMounts.name

   In the container volume mounts section, enter the name of the volume to mount to your pod.

   volume.name

   In the volumes section, enter the name of the volume to mount to your pod. Typically this name is the same as volumeMounts.name.

   claimName

   In the volumes persistent volume claim section, enter the name of the PVC that binds the PV that you want to use.

6. Create the deployment.

   oc apply -f deployment.yaml
   

7. Verify that the PV is successfully mounted.

   oc describe deployment my-deployment
   

   The mount point is in the Volume Mounts field and the volume is in the Volumes field.

   Volume Mounts:
       /var/run/secrets/kubernetes.io/serviceaccount from default-token-tqp61 (ro)
       /volumemount from myvol (rw)
   ...
   Volumes:
   myvol:
       Type:    PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
       ClaimName:    mypvc
       ReadOnly:    false
   

Attaching existing file storage to an app

Create a persistent volume claim (PVC) to statically provision File Storage for VPC for your cluster. Static provisioning allows cluster administrators to make existing storage devices available to a cluster.

Before you can create a persistent volume (PV), you have to retrieve details about your file share.

1. Get the ID for your file share.

   ibmcloud is shares
   

2. Get the details of your share. Make a note of the mount targets.

   ibmcloud is share SHARE-ID
   

3. Get the nfsServerPath, also called the Mount Path.

   ibmcloud is share-mount-target SHARE-ID SHARE-TARGET-ID
   

4. Create a PV configuration file called static-file-share.yaml that references your file share.

   apiVersion: v1
   kind: PersistentVolume
   metadata:
   name: static-file-share
   spec:
   mountOptions:
     - hard
     - nfsvers=4.1
     - sec=sys
   accessModes:
   - ReadWriteMany
   capacity:
     storage: 10Gi
   csi:
     volumeAttributes:
       nfsServerPath:  NFS-SERVER-PATH
     driver: vpc.file.csi.ibm.io
     volumeHandle: FILE-SHARE-ID:SHARE-TARGET-ID
   

5. Create the PV.

   oc apply -f static-file-share.yaml
   

6. Create a PVC.

     apiVersion: v1
     kind: PersistentVolumeClaim
     metadata:
       name: pvc-static
     spec:
       accessModes:
       - ReadWriteMany
       resources:
         requests:
           storage: 10Gi
       storageClassName: "" #Leave the storage class blank.
   

7. Create the PVC to bind your PV.

   oc apply -f pvc-static.yaml
   

8. Create a deployment file name testpod.yaml to attach your fileshare to the desired application pod.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: testpod
     labels:
       app: testpod
   spec:
     securityContext:
       fsgroup: 0
     replicas: 1
     selector:
       matchLabels:
         app: testpod
     template:
       metadata:
         labels:
           app: testpod
       spec:
         containers:
         - image: IMAGE
           name: CONTAINER-NAME
           volumeMounts:
           - mountPath: /myvol  
             name: pvc-name 
         volumes:
         - name: pvc-name 
           persistentVolumeClaim:
             claimName: pvc-static # The name of the PVC that you created earlier
   

   spec.containers.image

   The name of the container image that you want to use. To list available images in your IBM Cloud Container Registry account, run ibmcloud cr image-list.

   spec.containers.name

   The name of the container that you want to deploy to your cluster.

   spec.containers.volumeMounts.mountPath

   Enter the absolute path of the directory to where the volume is mounted inside the container. Data that is written to the mount path is stored under the root directory in your physical File Storage for VPC instance. If you want to share a volume between different apps, you can specify volume sub paths for each of your apps.

   volumeMounts.name

   Enter the name of the volume to mount to your pod.

   volume.name

   Enter the name of the volume to mount to your pod. Typically this name is the same as volumeMounts.name.

   volumes.persistentVolumeClaim.claimName

   Enter the name of the PVC that binds the PV that you want to use.

   securityContext

   Enter the securityContext that your app needs, for example runAsUser or runAsGroup that matches uid or gid of your file share. Note that fsGroup: 0 is used in the example to provide root permissions. For more information about security context, see Configure a Security Context for a Pod or Container.

9. Create the deployment.

   oc apply -f testpod.yaml
   

Deploying an app that runs as non-root

If your app needs to run as non-root, or if your cluster resource group is different than the VPC/Subnet resource group, you must create your own customized storage class. To get started, check out the customized storage class samples.

You can run your app as non-root by first creating a custom storage class that contains the group ID (gid) or user ID (uid) that you want to use.

1. Create a custom storage class and specify the group ID (gid) or user ID (uid) that you want to use for your app.

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
       name: ibmc-vpc-file-dp-eni-default
   provisioner: vpc.file.csi.ibm.io
   mountOptions:
       - hard
       - nfsvers=4.1
       - sec=sys
   parameters:
       profile: "dp2"
       iops: "100" # Default IOPS.
       billingType: "hourly" # The default billing policy used.
       encrypted: "false" # By default, all PVC using this class will only be provider managed encrypted.
       gid: "3000" # The default is 0 which runs as root. You can optionally specify a non-root group ID.
       uid: "1000" # The default is 0 which runs as root. You can optionally specify a non-root user ID.
       classVersion: "1"
   reclaimPolicy: "Delete"
   allowVolumeExpansion: true
   

2. Save the following code to a file called my-pvc.yaml. This code creates a claim that is named my-pvc by using the ibmc-vpc-file-dp2 storage class, billed monthly, with a gigabyte size of 10Gi.

   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: my-pvc # Enter a name for your PVC.
   spec:
     accessModes:
     - ReadWriteMany # The PVC can be mounted by multiple pods. All pods can read from and write to the volume.
     resources:
       requests:
         storage: 10Gi # Enter the size of the storage in gigabytes (Gi). After your storage is provisioned, you can't change the size. Make sure to specify a size that matches the amount of data that you want to store.
     storageClassName: ibmc-vpc-file-dp-eni-default # Enter the name of the storage class that you want to use.
   

3. Create the PVC.

   kubectl apply -f my-pvc.yaml
   

4. Create a pod that mounts the PVC.

   apiVersion: v1
   kind: Pod
   metadata:
     name: security-context-demo
   spec:
     securityContext:
       runAsUser: 1000
       runAsGroup: 3000
     volumes:
     - name: sec-ctx-vol
       emptyDir: {}
     containers:
     - name: sec-ctx-demo
       image: busybox:1.28
       command: [ "sh", "-c", "sleep 1h" ]
       volumeMounts:
       - name: sec-ctx-vol
         mountPath: /data/demo
       securityContext:
         allowPrivilegeEscalation: false
       persistentVolumeClaim:
         claimName: my-pvc # The name of the PVC that you created earlier
   

Creating a custom storage class

Create your own customized storage class with the preferred settings for your File Storage for VPC instance.

1. Review the Storage class reference to determine the profile that you want to use for your storage class.

   If you want to use a preinstalled storage class as a template, you can get the details of a storage class by using the oc get sc STORAGECLASS -o yaml command.

2. Create a custom storage class configuration file.

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
       name: ibmc-vpc-file-dp-eni-default
   provisioner: vpc.file.csi.ibm.io
   mountOptions:
       - hard
       - nfsvers=4.1
       - sec=sys
   parameters:
       profile: "dp2" # The VPC Storage profile used.
       iops: "100" # Default IOPS.
       billingType: "hourly" # The default billing policy used.
       encrypted: "false" # By default, all PVC using this class will only be provider managed encrypted.
       gid: "0" # The default is 0 which runs as root. You can optionally specify a non-root group ID.
       uid: "0" # The default is 0 which runs as root. You can optionally specify a non-root user ID.
       encryptionKey: "" # If encrypted is true, then you must specify the CRN of the encryption key that you want to use from your associated KP/HPCS instance.
       resourceGroup: "" # By default, the cluster resource group is used. If your VPC or subnets are in a different resource group than the cluster, enter the VPC or subnet resource group here.
       region: "" # By default, the region will be populated from the worker node topology.
       zone: "" # By default, the storage vpc driver automatically selects a zone.
       tags: "" # A list of tags "a, b, c" that will be created when the volume is created.
       isENIEnabled: "true" # VPC File Share uses the VNI feature.
       securityGroupIDs: "" # Specify a comma-separated list of security group IDs.
       subnetID: "" # Specify the subnet ID in which the ENI/VNI is created. If not specified, the subnet used are the VPC subnets in the same zone of the cluster.
       zone: "" # By default, the storage VPC driver selects a zone. The user can override this default.
       primaryIPID: "" # Existing ID of reserved IP address from the same subnet as the file share zone. The subnet-id is not mandatory for this field.
       primaryIPAddress: "" # IP address for VNI to be created in the subnet of the zone. The subnetID parameter is also required for this field.
       classVersion: "1"
   reclaimPolicy: "Delete"
   allowVolumeExpansion: true
   

3. Create the customized storage class in your cluster.

   oc apply -f custom-storageclass.yaml
   

4. Verify that your storage class is available in the cluster.

   oc get storageclasses
   

   Example output

   NAME                                          PROVISIONER
   ibmc-vpc-file-dp-eni-default                  vpc.file.csi.ibm.io
   

Limiting file share access by worker node, zone, or worker pool

The default behavior for File Storage for VPC cluster add-on is that pods on a given node can access file shares.

In versions 1.2 of and later of the File Storage for VPC cluster add-on, you can control more granularly how pods access your file shares. For example, you might limit file share access to only pods on a specific node, in a specific zone, on a specific worker pool, or in a specific subnet. Review the following scenarios for how you can configure pod access to your file shares.

If you use VNI to limit pod access to your file shares, you will not have high availability as file share access and in case a subnet is down where the VNI is created, you will not have access to your file share which also impacts reading and writing to the file share.