Lab 04 - Shared Volumes

In this section, we will create a Portworx volume (PVC) for NGINX.

Why NGINX?

In this lab, we are using NGINX to demonstrate how shared volumes can be utilized to deploy and scale stateless applications effectively. NGINX is an ideal example for testing the behavior of shared volumes, as its data must remain consistent across multiple instances. This approach will help us understand how Portworx shared volumes work in an OpenShift environment.

Create StorageClass

Take a look at the StorageClass definition for Portworx and create the storage class.

cat <<EOF | oc apply -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
    name: px-shared-sc
provisioner: pxd.portworx.com
parameters:
   repl: "3"
   sharedv4: "true"
EOF

The parameters are declarative policies for your storage volume. See here for a full list of supported parameters. In our case, the key parameter is sharedv4 = true.

Create PersistentVolumeClaim

Take a look at the PersistentVolumeClaim and create it.

cat <<EOF | oc apply -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
   name: px-shared-pvc
spec:
  storageClassName: px-shared-sc
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
EOF

Now that we have the volume created, let’s deploy a few NGINX instances and see how the shared volume works!

In this step, we will deploy the NGINX application using the PersistentVolumeClaim created before.

Notice in the specification below, we have set the securityContext.seLinuxOptions. Without this setting, the pods may be assigned random SELinux labels, in which case only the last pod to come online would have access to the shared volume.

Deploy 3 Instances of NGINX

cat <<EOF | oc apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp1
  labels:
    app: webapp1
spec:
  selector:
    matchLabels:
      app: webapp1
  replicas: 1
  template:
    metadata:
      labels:
        app: webapp1
        group: webapp
    spec:
      securityContext:
        runAsNonRoot: true
        seLinuxOptions:
          level: "s0:c1,c0"
        seccompProfile:
          type: RuntimeDefault
      containers:
      - name: webapp1
        securityContext:
          allowPrivilegeEscalation: false
          seLinuxOptions:
            level: "s0:c1,c0"
          capabilities:
            drop: ["ALL"]
        image: nginxinc/nginx-unprivileged
        ports:
        - containerPort: 8080
        volumeMounts:
        - mountPath: /usr/share/nginx/html
          name: shared-data
      volumes:
      - name: shared-data
        persistentVolumeClaim:
          claimName: px-shared-pvc
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp2
  labels:
    app: webapp2
spec:
  selector:
    matchLabels:
      app: webapp2
  replicas: 1
  template:
    metadata:
      labels:
        app: webapp2
        group: webapp
    spec:
      securityContext:
        runAsNonRoot: true
        seLinuxOptions:
          level: "s0:c1,c0"
        seccompProfile:
          type: RuntimeDefault
      containers:
      - name: webapp2
        securityContext:
          allowPrivilegeEscalation: false
          seLinuxOptions:
            level: "s0:c1,c0"
          capabilities:
            drop: ["ALL"]
        image: nginxinc/nginx-unprivileged
        ports:
        - containerPort: 8080
        volumeMounts:
        - mountPath: /usr/share/nginx/html
          name: shared-data
      volumes:
      - name: shared-data
        persistentVolumeClaim:
          claimName: px-shared-pvc
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp3
  labels:
    app: webapp3
spec:
  selector:
    matchLabels:
      app: webapp3
  replicas: 1
  template:
    metadata:
      labels:
        app: webapp3
        group: webapp
    spec:
      securityContext:
        runAsNonRoot: true
        seLinuxOptions:
          level: "s0:c1,c0"
        seccompProfile:
          type: RuntimeDefault
      containers:
      - name: webapp3
        securityContext:
          allowPrivilegeEscalation: false
          seLinuxOptions:
            level: "s0:c1,c0"
          capabilities:
            drop: ["ALL"]
        image: nginxinc/nginx-unprivileged
        ports:
        - containerPort: 8080
        volumeMounts:
        - mountPath: /usr/share/nginx/html
          name: shared-data
      volumes:
      - name: shared-data
        persistentVolumeClaim:
          claimName: px-shared-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: webapp1-svc
  labels:
    app: webapp1
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: webapp1
---
apiVersion: v1
kind: Service
metadata:
  name: webapp2-svc
  labels:
    app: webapp2
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: webapp2
---
apiVersion: v1
kind: Service
metadata:
  name: webapp3-svc
  labels:
    app: webapp3
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: webapp3
EOF

Observe the volumeMounts and volumes sections where we mount the PVC.

Verify NGINX Pods Are Ready

Run the command below and wait until all three NGINX pods are in a ready state.

watch oc get pods -l group=webapp -o wide

When all three pods are in the Running state, press ctrl-c to clear the screen. Be patient. If they stay in the Pending state for a while, it is because each node must fetch the Docker image.

Inspect the Portworx Volume

Portworx ships with a pxctl command line tool that can be used to manage Portworx.

Below, we will use pxctl to inspect the underlying volume for our PVC.

pxctl volume inspect $(oc get pvc | grep px-shared-pvc | awk '{print $3}')

Status: Indicates that the volume is attached and shows the node on which it is attached. For shared volumes, this is the transaction coordinator node that all other nodes use to write data.
HA: Displays the number of configured replicas for this volume (shared volumes can also be replicated; you can test this by modifying the storage class in step 2).
Shared: Shows if the volume is shared.
IO Priority: Displays the relative priority of the volume’s IO (high, medium, or low).
Volume consumers: Shows which pods are accessing the volume.

With our shared volume successfully created and mounted across all three NGINX containers, we can now write data into the html folder of NGINX and verify that all three containers can read the data.

Confirm Our NGINX Servers Are Up

Run the following command:

oc run test-webapp1 --image nginx --restart=Never --rm -ti -- curl webapp1-svc

You should see the following:

<html>
<head><title>403 Forbidden</title></head>
<body bgcolor="white">
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/xxx</center>
</body>
</html>

Create `index.html` in the NGINX HTML Folder on `webapp1`

Copy index.html into the webapp1 pod:

cat <<"EOF" > /tmp/index.html
 /$$$$$$$                       /$$
| $$__  $$                     | $$
| $$  \ $$ /$$$$$$   /$$$$$$  /$$$$$$   /$$  /$$  /$$  /$$$$$$   /$$$$$$  /$$   /$$
| $$$$$$$//$$__  $$ /$$__  $$|_  $$_/  | $$ | $$ | $$ /$$__  $$ /$$__  $$|  $$ /$$/
| $$____/| $$  \ $$| $$  \__/  | $$    | $$ | $$ | $$| $$  \ $$| $$  \__/ \  $$$$/
| $$     | $$  | $$| $$        | $$ /$$| $$ | $$ | $$| $$  | $$| $$        >$$  $$
| $$     |  $$$$$$/| $$        |  $$$$/|  $$$$$/$$$$/|  $$$$$$/| $$       /$$/\  $$
|__/      \______/ |__/         \___/   \_____/\___/  \______/ |__/      |__/  \__/
EOF

POD=$(oc get pods -l app=webapp1 | grep Running | awk '{print $1}')
oc cp /tmp/index.html $POD:/usr/share/nginx/html/index.html

Now, let’s access all three URLs and verify that the "Hello World" message appears on each. This happens because all three containers are attached to the same volume, so any updates made to one are reflected across all.

oc run test-webapp1 --image nginx --restart=Never --rm -ti -- curl webapp1-svc

oc run test-webapp2 --image nginx --restart=Never --rm -ti -- curl webapp2-svc

oc run test-webapp3 --image nginx --restart=Never --rm -ti -- curl webapp3-svc

Summary

Congratulations! You have successfully created a shared volume using Portworx and deployed multiple NGINX instances to demonstrate how data consistency is maintained across multiple pods. We verified that each instance of NGINX can access the same shared data, showcasing the power of shared storage in an OpenShift environment.

With our shared volumes successfully created and mounted across all three NGINX containers, we were able to modify the html folder of webapp1, and see the changes reflected in all NGINX instances. This demonstrated how shared volumes facilitate consistent data across multiple application pods.