Showing posts with label kubectl. Show all posts
Showing posts with label kubectl. Show all posts

Wednesday, February 09, 2022

Tip: kubectl apply --dry-run=client server RBAC role

 kubectl apply --dry-run is very useful to test manifests.

There are differences of RBAC requirement with kubectl apply --dry-run=client and server.

Both need a role in fetching CRD to go through the validation admission chain and the mutating admission chain.

We need READ ONLY  role for kubectl apply --dry-run=client and READ WRITE role for kubectl apply --dry-run=server

Saturday, August 21, 2021

Kubectl Plugin for Oracle Database

 Requirement:

We often need to provision new oracle databases for developers
This is the kubectl plugin to automate the creation of oracle database statefulset in the Kubernetes cluster

Solution:

Full details and source codes are on the GitHub repository

Demo:



Thursday, July 01, 2021

Pass CKA Exam Tips

On Thursday, I passed the CKA exam with a 93 score mark and get the certificate. I share some tips on how I achieve that.

  • 17 questions in 2 hours. 
  • Don't worry about copy and paste. You can copy it by clicking it when your mouse hovers on the "important text".
  • Read each question carefully. Always understand questions before starting to do it. Check the weight of each question. The high weight means more mark points in the question. 
  • Skip the difficult questions and make sure you get easy marks.  Only a 66 score mark is needed to pass the exam. 
  • Practise and create examples for each test point in the CKA curriculum
  • Strongly recommend this udemy course.  The practice and mock exams are great to prepare for CKA exams.
  • Practise all commands in the kubectl cheatsheet

Tuesday, February 02, 2021

Tip: A Command to get all resources and subresources in Kuberentes Cluster

 list=($(kubectl get --raw / | jq -r '.paths[] | select(. | startswith("/api"))')); for tgt in ${list[@]}; do aruyo=$(kubectl get --raw ${tgt} | jq .resources); if [ "x${aruyo}" != "xnull" ]; then echo; echo "===${tgt}==="; kubectl get --raw ${tgt} | jq -r ".resources[] | .name,.verbs"; fi; done

Sunday, August 30, 2020

Tip: Impersonate users on kubectl

We can impersonate users with the --as= and the --as-group= flags.

kubectl auth can-i create pods --as=me

Wednesday, May 20, 2020

Kubectl Stop Working After Upgrade Ubuntu

Symptom:

   After we upgrade ubuntu to 20.04, kubectl stop working with OKE (Oracle Kubernetes Engine).  Error is like
$kubectl version
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: initfsencoding: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007f2d902e0740 (most recent call first):
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: getting credentials: exec: signal: aborted (core dumped)

Solution:

  The issue is related to Kube config of OKE. In OKE, we need to use oci to authenticate users to access OKE control plane. The upgrade of ubuntu somehow breaks the python env that oci cli depends.  To fix it, we need to re-install oracle cloud oci cli. Please refer link

Monday, August 05, 2019

Automation Tool to Create Http Ords and Loadbalancer in K8S

Requirement:

A kubectl plugin that create http and ords( Oracle Rest Data Services) based on Apex (oracle application express) 19.1
Once we have Apex ready . We often need to provision  http and ords for it. We would like to automate http ords and loadbalancer deployment in K8S. Once we have db hostname, port , sys password , apex /ords password. We can deployment a brand new http ords and loadbalancer deployment env via 1 command. We can also delete it via 1 command. ords image is based on docker images of oracle github.
Solution:
Full details and source codes are on github repository

Monday, June 10, 2019

Tip to Verify KubeDNS is Working From Host

Requirement:

Sometimes we need to verify KubeDNS is working from host OS( ie VM ).

Solution:

  • Find KubeDNS service IP address via command
kubectl run -i --tty busybox --image=busybox --restart=Never -- cat /etc/resolv.conf
  • install nc if necessary. ie yum install nc
  •  ie the IP address is 10.96.5.5 , then run while loop to check each KubeDNS pod is responding. You may get timeout or can't resolve error if one of the pods is not working
while true;do nc -vz 10.96.5.5 53;sleep 3; done
  • More debug details please refer K8S doc 

Tuesday, February 26, 2019

Change the Reclaim Policy of a PersistentVolume In OKE

Symptom:

  By default OKE(Oracle Kubernete Engine)  storageclass is oci which is OCI block volume
  If we don't specify storageclass in yaml file,  OKE would automatically create block volumes as persistent volumes and attach to pods for us which is very convenient.
 However reclaim policy of persistent volumes is "DELETE"
 It means if we delete pv and pvc OKE created, OKE would delete block volumes in OCI as well.

Solution:

  To prevent potential data loss due to reclaim policy " DELETE" , we can update it to be "RETAIN"

kubectl patch pv <your-pv-name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

Please refer kubernete doc for more details

Saturday, February 16, 2019

Tips of How to Understand CustomResourceDefinition in K8S

In Kubernetes world, CustomResourceDefinition (CRD) will play a big role when we extend K8S cluster.  In the future, CRD may include even core K8S components like pod, deployment, statefulset......etc.  see CNCF youtube link
The basic logic is :
Once they are defined, they will be stored in Etcd.  They are purely text file, We can define whatever we want. K8S just regard them as text config file and put them in Etcd. K8S won't check if the CRD has been implemented or not. It won't error out if CRD is not implemented.  Related kubectl commands:
lubectl get crd <name>  -o yaml    kubectl api-resources   kubectl api-versions
  • We implement the business logic via GO
This part will implement how we handle events related to the CRD we define.  K8S will send events to this part, this part will do business logic for the events. K8S cluster is events driven system. All events are put in different queues inside K8S core. 
  • We create yaml files to use the CRD we define
We create yaml to generate some events (create, update, delete....)  to K8S,  our 2nd part above will receive these events and do the business logic we implement.

Fortunately kubebuilder  does lots of scaffolding work for us. More details Refer kubebuilder book

Thursday, February 14, 2019

How To Put Binary or Text Config Files into K8s ConfigMap

Requirement:

  We often have all kind of config files for our Apps.  From binary wallet files to simple text config files, we need to store them somewhere where apps can access.  It is not a good idea to put these config files inside docker images or PV(persistent volume) as it would make the pod less portable and hard to scale and migrate. Fortunately from K8S v1.10 , Configmap would support both binary and text config files .  Here are some examples how we utilize configmap to achieve that. Mounted configmap are updated automatically. More details please refer kubernetes configmap doc

Solution:

We plan to create 2 configmaps . One for binary files like ewallet and one for text config files  like sysctl.conf, init.ora....

  • Use kubectl create configmap walletconfigmap --from-file=ewallet1 --from-file=ewallet2
  • Use kubectl create configmap textconfigmap --from-file=sysctl.conf  --from-file=jdbc.xml  --from-file=init.ora  
  • Use kubectl get configmap walletconfigmap -o yaml . You would see ewallet would be stored as binaryData

apiVersion: v1
items:
- apiVersion: v1
  binaryData:
    ewallet: sdfsweffewg.....
........
  • Mount wallet configmap as a volume in the pod with correct config file path . ie wallet file is on /etc/oracle/tde
  • Mount text configmap as a volume in the pod with correct config file path . ie text file is on /opt/oracle/dbs
volumes:
- name: wallet-volume
  configMap:
          name: walletconfigmap 
- name: textconfig-volume
   configMap:
          name: textconfigmap 

Under container section of yaml file
volumeMounts:
- name: wallet-volume
  mountPath: /etc/oracle/tde
- name: textconfig-volume
  mountPath: /opt/oracle/dbs

  • In this way, when we into the pod via kubectl exec -it  <pod name> /bin/bash , we would see ewallet1 ewallet2 in /etc/oracle/tde  and  sysctl.conf jdbc.xml and init.ora are in /opt/oracle/dbs.
  • If we do some updates on configmap , K8S will sync them periodically.

Monday, February 04, 2019

Tip to Get K8S API Server Startup Parameters

If the API server is running in a pod, to find out details of startup parameters of K8S API
$ kubectl get po -n kube-system kube-apiserver -o yaml
.......
- command:
    - kube-apiserver
    - --secure-port=6443
    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
    - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --insecure-port=0
    - --requestheader-username-headers=X-Remote-User
    - --requestheader-allowed-names=front-proxy-client
    - --service-cluster-ip-range=1.1.1.1/12
    - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
    - --enable-bootstrap-token-auth=true
    - --requestheader-extra-headers-prefix=X-Remote-Extra-
    - --advertise-address=1.1.1.1
    - --service-account-key-file=/etc/kubernetes/pki/sa.pub
    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
    - --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota
    - --allow-privileged=true
    - --requestheader-group-headers=X-Remote-Group
    - --authorization-mode=Node,RBAC
    - --etcd-servers=https://127.0.0.1:2379
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    image: container-registry.oracle.com/kubernetes_developer/kube-apiserver-amd64:v1.10.5
..........

Sunday, February 03, 2019

Tip to Check K8S Core API Https Header and Contents via Kubectl

As we know, K8S core API are Https calls. We can use kubectl --v=8 to get  debug details each https call of the core API. With same way, we can other kubectl commands debug info. ie

$ kubectl --v=8 create  -f ship_v1beta1_frigate.yaml
I0203 23:37:32.108940   21540 loader.go:357] Config loaded from file /home/k8suser/.kube/config
I0203 23:37:32.111815   21540 round_trippers.go:383] GET https://1.1.1.1:6443/openapi/v2
I0203 23:37:32.111862   21540 round_trippers.go:390] Request Headers:
I0203 23:37:32.111884   21540 round_trippers.go:393]     Accept: application/com.github.proto-openapi.spec.v2@v1.0+protobuf
I0203 23:37:32.111906   21540 round_trippers.go:393]     User-Agent: kubectl/v1.10.5+2.0.2.el7 
.................
frigate.ship.example.com "frigate-sample-test" created