Basic Setup:

1. AWS EKS Cluster
2. AWS EFS for volumes
3. Jira Service Management
4. Confluence

I have a pod that needs two statically provisioned AWS EFS volumes attached to function. This is an instance of confluence that uses a volume each for the local and shared home directories.

The shared home directory EFS will attach without issue, but the local home directory volume will not. I get the provisioner error "Waiting for a volume to be created either by the external provisioner 'efs.csi.aws'" and the PVC will not attach.

Note that the shared home and local home spec are exactly the same, so I don't see why one will work and other wont. The SGs, networking, and everything else I am aware of, are also identical.

Local Home PV

kind: PersistentVolume
apiVersion: v1
metadata:
  name: confluence-local-efs
  labels:
    type: efs
spec:
  storageClassName: efs-local-sc
  claimRef:
    name: confluence-local-home
    namespace: app-confluence
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  accessModes:
    - ReadWriteMany
  csi:
    driver: efs.csi.aws
    volumeHandle: fs-0ee787b1d3c2ce898

Shared Home PV

kind: PersistentVolume
apiVersion: v1
metadata:
  name: confluence-shared-efs
  labels:
    type: efs
spec:
  storageClassName: efs-shared-sc
  claimRef:
    name: confluence-shared-home
    namespace: app-confluence
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  accessModes:
    - ReadWriteMany
  csi:
    driver: efs.csi.aws
    volumeHandle: fs-0904d99352b0a2515

Storage Class

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: efs-shared-sc
provisioner: efs.csi.aws
parameters:
  provisioningMode: efs-ap
  fileSystemId: fs-**::fsap-***

CSI Driver

apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
  name: efs.csi.aws
spec:
  attachRequired: false

The PVCs are being generated via helm, the values are:

volumes:
  localHome:
    persistentVolumeClaim:
      create: true
      storageClassName: efs-local-sc
  sharedHome:
    persistentVolumeClaim:
      create: true
      storageClassName: efs-shared-sc

Results of describe on PVC

Name:          local-home-confluence-0
Namespace:     app-confluence
StorageClass:  efs-local-sc
Status:        Pending
Volume:
Labels:        app.kubernetes.io/instance=confluence
               app.kubernetes.io/name=confluence
Annotations:   volume.beta.kubernetes.io/storage-provisioner: efs.csi.aws
               volume.kubernetes.io/storage-provisioner: efs.csi.aws
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Used By:       confluence-0
Events:
  Type    Reason                Age                From                         Message
  ----    ------                ----               ----                         -------
  Normal  ExternalProvisioning  0s (x63 over 15m)  persistentvolume-controller  Waiting for a volume to be created either by the external provisioner 'efs.csi.aws' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered.

Again, a near identical volume is being bound to the same pod, so it is baffling that this is happening on only one.

Any ideas would be greatly appreciated.