Monday, November 26, 2018

Kubectl: Unable to connect to the server: x509: certificate is valid for ..... , not ..... in K8S

Symptom:

     When we setup kubectl on local workstation to access remote Kubernete Cluster.  The remote public IP of K8S API server access point is 52.64.132.188.  Port 6443 is open. We obtain the ca.pem file locally and run below to generete kubeconfig file locally.
kubectl config set-cluster kubernetes-the-hard-way \
  --certificate-authority=ca.pem \
  --embed-certs=true \
  --server=https://52.64.132.188:6443
After that, we try to run kubectl get node but get Unable to connect to the server: x509: certificate error. Details like
$ kubectl get node
Unable to connect to the server: x509: certificate is valid for 10.32.0.1, 172.31.44.176, 172.31.2.170, 172.31.3.17, 127.0.0.1, not 52.64.132.188

Diagnosis:

The reason of  "Unable to connect to the server: x509: certificate is valid for ..... , not ....." is quite likely the  K8S API server does not have "52.64.132.188" in its CA authority host list. We need to go back and check what cert hosts were added into the kubernetes.pem when K8S cluster was initiated.
In my case, I ran 

cfssl gencert \
  -ca=ca.pem \
  -ca-key=ca-key.pem \
  -config=ca-config.json \
  -hostname=10.32.0.1, 172.31.44.176, 172.31.2.170, 172.31.3.17, 127.0.0.1,test.testdomain.com \
  -profile=kubernetes \
  kubernetes-csr.json | cfssljson -bare kubernetes
 
I used test.testdomain.com  not ip address "52.64.132.188" because public ip can be changed later.K8S CA has "test.testdomain.com" in the CA list, not ip address.  That is the reason why K8S API server does not think "52.64.132.188" is a valid client to access the API.

Solution:

    To solve it, we need to update our local kubeconfig file to use test.testdomain.com not IP address.
    kubectl config set-cluster kubernetes-the-hard-way \
  --certificate-authority=ca.pem \
  --embed-certs=true \
  --server=https://test.testdomain.com:6443

No comments: