Tuesday, August 27, 2019

Tip: Clean evicted pods and dangling docker images

Clean evicted pods

kubectl get pods --all-namespaces -o json | jq '.items[] | select(.status.reason!=null) | select(.status.reason | contains("Evicted")) | "kubectl delete pods \(.metadata.name) -n \(.metadata.namespace)"' | xargs -n 1 bash -c

Clean dangling docker images. 

A dangling image is one that is not tagged and is not referenced by any container.
docker image prune -a -f --filter "until=24h"

Wednesday, August 14, 2019

Error:no kind is registered in scheme pkg/runtime/scheme.go:101

Symptom:

  When we create controller operator via kubebuilder 2.0  we add deployment type in our controller. But it error out when we "make run"
"no kind is registered for the type v1.Deployment in scheme \"pkg/runtime/scheme.go:101\"

Solution:

   Per kubebuilder 2.0 , "Every set of controllers needs a Scheme, which provides mappings between Kinds and their corresponding Go types."
   We need to add deployment type and all other related to scheme.  Then we can use those objects in our controller.
sample codes :

import (
"flag"
"os"
theapexordsv1 "apexords-operator/api/v1"
"apexords-operator/controllers"
appsv1beta1 "k8s.io/api/apps/v1beta1"
corev1 "k8s.io/api/core/v1"
appsv1 "k8s.io/api/apps/v1"
"k8s.io/apimachinery/pkg/runtime"
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/log/zap"

)
var (
scheme   = runtime.NewScheme()
setupLog = ctrl.Log.WithName("setup")
)
func init() {
appsv1beta1.AddToScheme(scheme)
appsv1.AddToScheme(scheme)
corev1.AddToScheme(scheme)
theapexordsv1.AddToScheme(scheme)

}

Error finding current repository: could not determine repository path from module data, package data, or by initializing a module: go

Symptom:

When we run kubebuilder init,  we get below error
$kubebuilder init --domain my.domain
2019/05/29 16:23:45 error finding current repository: could not determine repository path from module data, package data, or by initializing a module: go: cannot determine module path for source directory /home/henryxie/go/kubebuilder-src/my.domain/ (outside GOPATH, no import comments)

Solution:

It is due to go mod init is not working properly.
To fix it
run go mod init  <directory>    ie   go mod init myfirstcontroller
then kubebuilder init --domain my.domain

Tuesday, August 06, 2019

Tip: How to Git Push Passwordless via SSH for Multiple Internal and External Repositories

Requirement:

   Sometimes we have projects. We need to git commit changes for multiple git repositories. Some repositories are internal, some are external. We would like to setup passwordless for them as well

Solution:


  • Setup ssh key for all the git repositories , thus we can git commit passwordless . Refer github doc
  • Setup ssh config to use proxy to ssh external git repositories if we are behind proxy in intranet,  ie github.com
  • vi .ssh/config , example below

Host=github.com
ProxyCommand=socat - PROXY:your.proxy.ip:%h:%p,proxyport=3128,proxyauth=user:pwd

  • git remote -v
  • git remote set-url --add --push origin git+ssh://original/repo.git
  • git remote set-url --add --push origin git+ssh://another/repo.git



Monday, August 05, 2019

Automation Tool to Create Http Ords and Loadbalancer in K8S

Requirement:

A kubectl plugin that create http and ords( Oracle Rest Data Services) based on Apex (oracle application express) 19.1
Once we have Apex ready . We often need to provision  http and ords for it. We would like to automate http ords and loadbalancer deployment in K8S. Once we have db hostname, port , sys password , apex /ords password. We can deployment a brand new http ords and loadbalancer deployment env via 1 command. We can also delete it via 1 command. ords image is based on docker images of oracle github.
Solution:
Full details and source codes are on github repository

Automation Tool to Create Database 19.2 in K8S

Requirement:

A kubectl plugin that create statefulset of oracle database 19.2 in your Kubernetes cluster or minikube.You get the full power of oracle database 19.2 in about 10-20 min (need more time of first time run to download docker image) and you can access it from laptop (assume ports are open)

Solution:

Full details and source codes are on github repository

Automation Tool to Create Apex 19.1 in K8S

Requirement:

A kubectl plugin to provision Apex(Oracle Application Express).  Apex is the foundation of many applications .  We often need to provision a apex for test, stage and prod. We would like to automate apex 19.1 deployment on a Oracle DB. 
This database can be a  DB in Cloud(AWS, Azure, GCP,OCI)  , it can be a DB in a VM, it can be DB pod in K8S.  Once we have db hostname, port , sys password , we can deployment a brand new Apex 19.1  env via 1 command.  We can also delete it via 1 command.

Solution:

Full details and source codes are on github repository

Tip: ClusterFirst vs ClusterFirstWithHostNet in Kubernetes Pod DNS config

Symptom:

 We got below error when we start 2 pods in the same Host. 1pod starts successfully and 1 pod is on pending status , can't startup running.  Error is like below
0/2 nodes are available: 2 node(s) didn't have free ports for the requested pod ports.

Reason:

 We have 2 entries in the deployment yaml files.  It means the pod is using host network, the first pod uses the certain port and the 2nd pod can't use the same port in the host, thus we see above error. After we remove these 2 entries , restart deployment, the issue is fixed.  Default dnsPolicy is  ClusterFirst.  Refer pod dns config settings doc
dnsPolicy: ClusterFirstWithHostNet
hostNetwork: true