Apex Ords Operator for Kubernetes

Requirement:

We often need to provision Apex and Ords for Dev, Stage, Prod. 

This is the operator to automate Apex Oracle Application Express 19.1 and Ords oracle rest data service via Kubernetes CRD, it creates a brand new Oracle 19c database statefulset, apex, ords deployment plus load balancer in the Kubernetes cluster

Solution:

Full details and source codes are on GitHub repository

Demo:

Tip: what are the GVK GVR CRD CR Scheme in Kuberentes Core API

GVK:

• GVK stands for Group Version Kind 
• Each Kind in K8S has Group and Version. i.e. Kind "Pod" is in Group "core" , Version "v1". Refer to official API doc
• GVK  is defined to associate Group, Version and Kind
• Each GVK map to a given root Go type in the package
• Source code definition is  here 

GVR:

• GVR stands for Group Version Resource
• GVR is a "use" or "instance" of GVK in the K8S API
• The command "kubectl api-resources"  gives us a list of GVR in the K8S cluster

CRD:

• CRD stands for Custom Resource Definition
• Each CRD is like Kind in K8S, so it also has Group, Version
• CRD is the extension of the K8S API. Refer to official doc
• Once it is defined, it acts like GVK in K8S API.

CR:

• CR stands for Custom Resource
• CR is a "use" or "instance" of CRD in the K8S API
• Once it is instantiated, it acts like GVR in K8S API.
• The command "kubectl api-resources"  gives us a list including both GVR and CR in the cluster.

Scheme:

• The scheme is defined to keep track of a given GO type mapping to a given GVK. 
• For example, we define   myexample.io/api/v1.mykind{}
• The scheme is going to map it to the API group we defined in CRD: batchv1.myexample.io/v1
• {
      kind:  mykind
      apiVersion: batchv1.myexample.io/v1
  }
• Source code definition is here.

Tip: How to get Go Client in Kubernetes Operator Reconcile()

Requirement:

      When we build a K8S operator via kubebuilder, we often need to interact with Control Plane. By default, we use controller-runtime client 

 However, when we use this client to fulfil some functions of kubectl i.e. drain, we hit an error:

 cannot use r.Client (variable of type client.Client) as kubernetes.Interface value in struct literal: missing method AdmissionregistrationV1

Solution:

The error indicates the controller-runtime client does not implement method AdmissionregistrationV1, so we can't use it, instead, we init a new GO Client in the reconcile(). Sample code is like

"k8s.io/client-go/kubernetes"

ctrlconfig "sigs.k8s.io/controller-runtime/pkg/client/config"

cfg, err := ctrlconfig.GetConfig()

if err != nil {

log.Log.Error(err, "unable to get kubeconfig")

return err

}

kubeclientset, err := kubernetes.NewForConfig(cfg)

if err != nil {

return err

}

Kubebuilder Controller-runtime client_go_adapter.go Error

Symptom:

   We hit below error when we build an operator via kubebuilder 3.1 + controller runtime v0.8.3 + kuberentes 1.20.2

./../../pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/metrics/client_go_adapter.go:134:3: cannot use &latencyAdapter{...} (type *latencyAdapter) as type metrics.LatencyMetric in field value:

*latencyAdapter does not implement metrics.LatencyMetric (wrong type for Observe method)

have Observe(string, url.URL, time.Duration)

want Observe(context.Context, string, url.URL, time.Duration)

../../../pkg/mod/sigs.k8s.io/controller-runtime@v0.8.3/pkg/metrics/client_go_adapter.go:135:3: cannot use &resultAdapter{...} (type *resultAdapter) as type metrics.ResultMetric in field value:

*resultAdapter does not implement metrics.ResultMetric (wrong type for Increment method)

have Increment(string, string, string)

want Increment(context.Context, string, string, string)

Solution:

 It is related to controller runtime v0.8.3 has compatible issues with Kubernetes 0.21 modules. See link

To fix it, find go.mod, change the module from 0.21 to 0.20.2

Error: the server could not find the requested resource

Symptom:

We follow the https://book.kubebuilder.io/ and create an example
It error out when we do "make run"
the server could not find the requested resource (put cronjobs.batch.mycrontab cronjob-sample)"}

Solution:

After i check cronjob_types.go , add below, issue is fixed

// +kubebuilder:subresource:status
type CronJob struct {

Tips of How to Understand CustomResourceDefinition in K8S

In Kubernetes world, CustomResourceDefinition (CRD) will play a big role when we extend K8S cluster.  In the future, CRD may include even core K8S components like pod, deployment, statefulset......etc.  see CNCF youtube link
The basic logic is :

• We define what CRD we would like to use , ie mysql-operator in github, postgres-operator.... 

Once they are defined, they will be stored in Etcd.  They are purely text file, We can define whatever we want. K8S just regard them as text config file and put them in Etcd. K8S won't check if the CRD has been implemented or not. It won't error out if CRD is not implemented.  Related kubectl commands:

lubectl get crd <name>  -o yaml    kubectl api-resources   kubectl api-versions

• We implement the business logic via GO

This part will implement how we handle events related to the CRD we define.  K8S will send events to this part, this part will do business logic for the events. K8S cluster is events driven system. All events are put in different queues inside K8S core. 

• We create yaml files to use the CRD we define

We create yaml to generate some events (create, update, delete....)  to K8S,  our 2nd part above will receive these events and do the business logic we implement.

Fortunately kubebuilder  does lots of scaffolding work for us. More details Refer kubebuilder book