Tip: Change status of PVC from Released to Available

Symptoms:

    When users delete PVC in Kubernetes, the PV status stays on "Released".  Users would like to recreate the PVC with the same PV but failed. new PVC status always stays on "Pending".

Solution:

   We need to manually clear the status to make it "Available" via below command

 kubectl patch pv  <pv name> -p '{"spec":{"claimRef": null}}'

A few simple Tips on Kubernetes

• Core DNS is deployment and Flannel is DaemonSet
• Persistent Volume (PV)  is not under the namespace
• Persistent Volume Claim(PVC) is under the namespace
• Delete PVC will delete PV automatically by default (Oracle Kubernetes Engine). Change retain policy if necessary
• Drain the node before rebooting worker node
• Both TCP and UDP must be open in the worker node subnet.
• If UDP is open after VMs are up and running, we may need to recreate VMs to let docker daemon to work with new settings

How To Fix Pod Mounted PV Permission Issues on OKE

Symptom:

  We are building DB services on OKE . By default OKE(Oracle Kubernete Engine) storageclass is oci which is OCI block volume.  If we don't specify storageclass in yaml file, OKE would automatically create block volumes as persistent volumes and attach to pods for us which is very convenient.
However we hit permission issue , by default the filesystem created by OKE is owned by root , the docker images user is oracle with id 54321 . It fails on creating DB.

Solution:

  It is not a good practice for a Dockerfile to modify parent host mounted file permission. We can use yaml to tell OKE to mount the volume with correct permission.  More details refer Kubernetes security context doc
Add below in the spec of the yaml file, in this case 54321 is the id

securityContext:
         runAsUser: 54321
         fsGroup: 54321

Change the Reclaim Policy of a PersistentVolume In OKE

Symptom:

  By default OKE(Oracle Kubernete Engine)  storageclass is oci which is OCI block volume
  If we don't specify storageclass in yaml file,  OKE would automatically create block volumes as persistent volumes and attach to pods for us which is very convenient.
 However reclaim policy of persistent volumes is "DELETE"
 It means if we delete pv and pvc OKE created, OKE would delete block volumes in OCI as well.

Solution:

  To prevent potential data loss due to reclaim policy " DELETE" , we can update it to be "RETAIN"

kubectl patch pv <your-pv-name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

Please refer kubernete doc for more details