Apex Ords Operator for Kubernetes

Requirement:

We often need to provision Apex and Ords for Dev, Stage, Prod. 

This is the operator to automate Apex Oracle Application Express 19.1 and Ords oracle rest data service via Kubernetes CRD, it creates a brand new Oracle 19c database statefulset, apex, ords deployment plus load balancer in the Kubernetes cluster

Solution:

Full details and source codes are on GitHub repository

Demo:

Kubectl Plugin for Oracle Database

 Requirement:

We often need to provision new oracle databases for developers

This is the kubectl plugin to automate the creation of oracle database statefulset in the Kubernetes cluster

Solution:

Full details and source codes are on the GitHub repository

Demo:

Tip: what are the GVK GVR CRD CR Scheme in Kuberentes Core API

GVK:

• GVK stands for Group Version Kind 
• Each Kind in K8S has Group and Version. i.e. Kind "Pod" is in Group "core" , Version "v1". Refer to official API doc
• GVK  is defined to associate Group, Version and Kind
• Each GVK map to a given root Go type in the package
• Source code definition is  here 

GVR:

• GVR stands for Group Version Resource
• GVR is a "use" or "instance" of GVK in the K8S API
• The command "kubectl api-resources"  gives us a list of GVR in the K8S cluster

CRD:

• CRD stands for Custom Resource Definition
• Each CRD is like Kind in K8S, so it also has Group, Version
• CRD is the extension of the K8S API. Refer to official doc
• Once it is defined, it acts like GVK in K8S API.

CR:

• CR stands for Custom Resource
• CR is a "use" or "instance" of CRD in the K8S API
• Once it is instantiated, it acts like GVR in K8S API.
• The command "kubectl api-resources"  gives us a list including both GVR and CR in the cluster.

Scheme:

• The scheme is defined to keep track of a given GO type mapping to a given GVK. 
• For example, we define   myexample.io/api/v1.mykind{}
• The scheme is going to map it to the API group we defined in CRD: batchv1.myexample.io/v1
• {
      kind:  mykind
      apiVersion: batchv1.myexample.io/v1
  }
• Source code definition is here.

Tip: How to get Go Client in Kubernetes Operator Reconcile()

Requirement:

      When we build a K8S operator via kubebuilder, we often need to interact with Control Plane. By default, we use controller-runtime client 

 However, when we use this client to fulfil some functions of kubectl i.e. drain, we hit an error:

 cannot use r.Client (variable of type client.Client) as kubernetes.Interface value in struct literal: missing method AdmissionregistrationV1

Solution:

The error indicates the controller-runtime client does not implement method AdmissionregistrationV1, so we can't use it, instead, we init a new GO Client in the reconcile(). Sample code is like

"k8s.io/client-go/kubernetes"

ctrlconfig "sigs.k8s.io/controller-runtime/pkg/client/config"

cfg, err := ctrlconfig.GetConfig()

if err != nil {

log.Log.Error(err, "unable to get kubeconfig")

return err

}

kubeclientset, err := kubernetes.NewForConfig(cfg)

if err != nil {

return err

}

Automation Tool to Create Http Ords and Loadbalancer in K8S

Requirement:

A kubectl plugin that create http and ords( Oracle Rest Data Services) based on Apex (oracle application express) 19.1

Once we have Apex ready . We often need to provision  http and ords for it. We would like to automate http ords and loadbalancer deployment in K8S. Once we have db hostname, port , sys password , apex /ords password. We can deployment a brand new http ords and loadbalancer deployment env via 1 command. We can also delete it via 1 command. ords image is based on docker images of oracle github.

Solution:

Full details and source codes are on github repository

Automation Tool to Create Database 19.2 in K8S

Requirement:

A kubectl plugin that create statefulset of oracle database 19.2 in your Kubernetes cluster or minikube.You get the full power of oracle database 19.2 in about 10-20 min (need more time of first time run to download docker image) and you can access it from laptop (assume ports are open)

Solution:

Full details and source codes are on github repository

Automation Tool to Create Apex 19.1 in K8S

Requirement:

A kubectl plugin to provision Apex(Oracle Application Express).  Apex is the foundation of many applications .  We often need to provision a apex for test, stage and prod. We would like to automate apex 19.1 deployment on a Oracle DB. 
This database can be a  DB in Cloud(AWS, Azure, GCP,OCI)  , it can be a DB in a VM, it can be DB pod in K8S.  Once we have db hostname, port , sys password , we can deployment a brand new Apex 19.1  env via 1 command.  We can also delete it via 1 command.

Solution:

Full details and source codes are on github repository

Error: expected ';', found '{' in Golang

Symptom:

When we write go code for kubernetes OwnerReference , we get such error

expected ';', found '{' 

code is like

var oradbownerref = []metav1.ObjectMeta.OwnerReference{{
Kind:       apexords.TypeMeta.Kind,
APIVersion: apexords.TypeMeta.APIVersion,
Name:       apexords.ObjectMeta.Name,
UID:        apexords.ObjectMeta.UID,
}}

Solution:

It is due to OwnerReference  is on metav1 level ,not metav1.ObjectMeta level.
Correct code is

var oradbownerref = []metav1.OwnerReference{{
Kind:       apexords.TypeMeta.Kind,
APIVersion: apexords.TypeMeta.APIVersion,
Name:       apexords.ObjectMeta.Name,
UID:        apexords.ObjectMeta.UID,
}}

Example of Pod Struct with ConfigMap ImagePullSecrets in Client-GO

typeMetadata := metav1.TypeMeta{
Kind:       "Pod",
APIVersion: "v1",
}
objectMetadata := metav1.ObjectMeta{
Name: "ordspod",
Namespace:    o.UserSpecifiedNamespace,
}
configmapvolume := &corev1.ConfigMapVolumeSource{
LocalObjectReference: corev1.LocalObjectReference{Name: "test-configmap"},
}
podSpecs := corev1.PodSpec{
ImagePullSecrets: []corev1.LocalObjectReference{{
Name: "test-secret",
}},
Volumes:  []corev1.Volume{{
Name: "ords-config",
VolumeSource: corev1.VolumeSource{
ConfigMap: configmapvolume,
},
}},
Containers:    []corev1.Container{{
Name: "ordspod",
Image: "ords:v19",
VolumeMounts: []corev1.VolumeMount{{
Name: "ords-config",
MountPath: "/mnt/k8s",
}},
}},
}
pod := corev1.Pod{
TypeMeta:   typeMetadata,
ObjectMeta: objectMetadata,
Spec:       podSpecs,
}

Tip to Get Output From A Command Running Inside Pod

Symptom:

  We write a client-go program to run a simple command ie pwd inside a pod.  It runs fine, No error  but I don't see pwd output to the standard OS output. Partial sample code is like

func ExecPodCmd(o *KubeOperations,Podname,Podnamespace string,SimpleCommand []string) error {
       SimpleCommand := []string{"pwd"}

execReq := o.clientset.CoreV1().RESTClient().Post().
    Resource("pods").
Name(Podname).
Namespace(Podnamespace).
SubResource("exec")
    execReq.VersionedParams(&corev1.PodExecOptions{
Command:   PsqlCommand,
Stdin:     true,
Stdout:    true,
Stderr:    true,
}, scheme.ParameterCodec)
exec, err := remotecommand.NewSPDYExecutor(o.restConfig, "POST", execReq.URL())
if err != nil {
return fmt.Errorf("error while creating Executor: %v", err)
}
err = exec.Stream(remotecommand.StreamOptions{
Stdin:  os.Stdin,
Stdout: os.Stdout,
Stderr: os.Stderr,
Tty:    false,
})
if err != nil {
return fmt.Errorf("error in Stream: %v", err)
} else {
return nil
}
}

Solution:

It turns out we need to use bash -c to run the simplecommand to get proper os output.
Replace   SimpleCommand := []string{"pwd"} with SimpleCommand := []string{"/bin/sh", "-c", "pwd"}

Error: unable to upgrade connection: you must specify at least 1 of stdin, stdout, stderr

Symptom:

  We would like to run a simple comand ie pwd in a pod via client-go.  It error on below part of code:

err = exec.Stream(remotecommand.StreamOptions{
Stdin:  stdin,
Stdout: stdout,
Stderr: stderr,
Tty:    false,
})

Solution:

  We can simply to use os.Stdin os.Stdout os.Stderr to get the comand output from pod to our standard os output.  Correct code is like:

err = exec.Stream(remotecommand.StreamOptions{
Stdin:  os.Stdin,
Stdout: os.Stdout,
Stderr: os.Stderr,
Tty:    false,
})

Error: cannot use "k8s.io/api/core/v1".ConfigMapVolumeSource literal (type "k8s.io/api/core/v1".ConfigMapVolumeSource) as type *"k8s.io/api/core/v1".ConfigMapVolumeSource in field value

Symptom:

Related code:
o.pgreplicamaster.Spec.Template.Spec.Volumes = []corev1.Volume{
{Name: "pgreplica-config",
VolumeSource: corev1.VolumeSource{ 
   ConfigMap: corev1.ConfigMapVolumeSource{
    LocalObjectReference: corev1.LocalObjectReference{Name: o.UserSpecifiedCM},
   },
   },
    
},
}
 Compile Error
cannot use "k8s.io/api/core/v1".ConfigMapVolumeSource literal (type "k8s.io/api/core/v1".ConfigMapVolumeSource) as type *"k8s.io/api/core/v1".ConfigMapVolumeSource in field value

Solution:

    Refer VolumeSource k8s doc. ConfigMap suppose to have pointer  *ConfigMapVolumeSource, not value of  ConfigMapVolumeSource , We need to add &.
Meanwhile we can't mix field value and value , refer stackflow link
Correct code is:
o.pgreplicamaster.Spec.Template.Spec.Volumes = []corev1.Volume{
{Name: "pgreplica-config",
 VolumeSource: corev1.VolumeSource{ 
   ConfigMap: &corev1.ConfigMapVolumeSource{
    LocalObjectReference: corev1.LocalObjectReference{Name: o.UserSpecifiedCM},
   },
   },
 
},
}

Error: spec.volumes[1].configMap.name: Required value, spec.containers[0].volumeMounts[1].name: Not found

Symptom:

  We use partial yaml below to create a statefulset with mount a volume for configmap.It is successful
apiVersion: apps/v1
kind: StatefulSet
......  
    spec:
      volumes:
        - name: pgreplica-config
          configMap:
            name: pgconfigmap
........
When we try to use client-go to do the same thing.
Related codes
o.pgreplicamaster.Spec.Template.Spec.Volumes = []corev1.Volume{
{Name: "pgreplica-config",
 VolumeSource: corev1.VolumeSource{ 
   ConfigMap: &corev1.ConfigMapVolumeSource{
    Items: corev1.KeyToPath{{Key: "name",Path: pgconfigmap}},
   },
   },
    
},
}
We hit error
Error: spec.volumes[1].configMap.name: Required value, spec.containers[0].volumeMounts[1].name: Not found

Solution

Refer k8s doc of ConfigMapVolumeSource
There is LocalObjectReference which we should use, instead of using Items
Update code as below to make it work
o.pgreplicamaster.Spec.Template.Spec.Volumes = []corev1.Volume{
{Name: "pgreplica-config",
VolumeSource: corev1.VolumeSource{
   ConfigMap: &corev1.ConfigMapVolumeSource{
    LocalObjectReference: corev1.LocalObjectReference{Name: "pgconfigmap"},
   },
   },
   
},
}

Example of Updating ConfigMap name in Client-go

Option 1:

o.pgreplicamaster.Spec.Template.Spec.Volumes[0].VolumeSource.ConfigMap.LocalObjectReference = corev1.LocalObjectReference{Name: o.UserSpecifiedCM}

is doing the same thing as

Option 2:

o.pgreplicamaster.Spec.Template.Spec.Volumes = []corev1.Volume{
{Name: "pgreplica-config",
VolumeSource: corev1.VolumeSource{
   ConfigMap: &corev1.ConfigMapVolumeSource{
    LocalObjectReference: corev1.LocalObjectReference{Name: o.UserSpecifiedCM},
   },
   },
 
},
}

fmt.Printf("%#v\n",o.pgreplicamaster.Spec.Template.Spec.Volumes[0].VolumeSource.ConfigMap.LocalObjectReference)

Go Exercise: How To use decode yaml into k8s client go data struct

Please see code on github link