pod 部署不上

[root@infra-k8s01 ~]# kb get po -n dev
NAME                                READY     STATUS        RESTARTS   AGE
recommend-system-5944d76fc7-hl242   0/1       Pending       0          43m

查看 pod 信息

[root@infra-k8s01 ~]# kb describe po recommend-system-5944d76fc7-hl242 -n dev
Name:               recommend-system-5944d76fc7-hl242
Namespace:          dev
...
Events:
  Type     Reason            Age                From               Message
  ----     ------            ----               ----               -------
  Warning  FailedScheduling  43m (x2 over 43m)  default-scheduler  0/20 nodes are available: 20 node(s) were not ready, 20 node(s) were out of disk space.

查看 node 信息

[root@infra-k8s01 ~]# kb get node
NAME           STATUS     ROLES     AGE       VERSION
192.168.1.14   NotReady   <none>    1y        v1.11.5
192.168.1.17   NotReady   <none>    1y        v1.11.5
192.168.1.21   NotReady   <none>    1y        v1.11.5
192.168.1.24   NotReady   <none>    1y        v1.11.5
...

发现 kubelet 一直在重启，查看 kubelet 日志，发现认证问题

[root@infra-k8s01 ~]# journalctl -u kubelet
...
k8s.io/kubernetes/pkg/kubelet/kubelet.go:464: Failed to list *v1.Node: Unauthorized
k8s.io/kubernetes/pkg/kubelet/kubelet.go:455: Failed to list *v1.Service: Unauthorized
k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:47: Failed to list *v1.Pod: Unauthorized
k8s.io/kubernetes/pkg/kubelet/kubelet.go:464: Failed to list *v1.Node: Unauthorized
k8s.io/kubernetes/pkg/kubelet/kubelet.go:455: Failed to list *v1.Service: Unauthorized
...

查看 kube-apiserver 日志，发现证书过期

[root@infra-k8s01 ~]# journalctl  -u kube-apiserver
...
Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid]
...

下面解决证书过期问题：删除 node 节点上过去的 kubelet 相关证书以及配置文件，然后重启 kubelet，kubele 会向 apiserver 发起一个 csr，在 master 节点 kubectl certificate approve

node 节点删除 kubelet 相关证书

mkdir /opt/kubelet-cert
mv /etc/kubernetes/kubelet.kubeconfig /opt/kubelet-cert
mv /etc/kubernetes/ssl/kubelet* /opt/kubelet-cert"
systemctl restart kubelet

master 节点执行，查看 csr 并同意

[root@infra-k8s01 ~]# kb get csr
NAME                                                   AGE       REQUESTOR           CONDITION
node-csr-1S_Xf9s1tqmOAMc5Eja75ranGCy4vtM2Ba3IZuxhMT4   10s       kubelet-bootstrap   Pending
node-csr-4AZXXqYtQ1-5R27oEWE7YBRyorEuz6hzDzEWuoEmgGc   10s       kubelet-bootstrap   Pending
...

[root@infra-k8s01 ~]# kb get csr | tail -n +2 | awk '{print $1}' | xargs -I {} kubectl certificate approve {}
certificatesigningrequest.certificates.k8s.io/node-csr-1S_Xf9s1tqmOAMc5Eja75ranGCy4vtM2Ba3IZuxhMT4 approved
certificatesigningrequest.certificates.k8s.io/node-csr-4AZXXqYtQ1-5R27oEWE7YBRyorEuz6hzDzEWuoEmgGc approved
...

查看 node 状态

[root@infra-k8s01 ~]# kb get node
NAME           STATUS    ROLES     AGE       VERSION
192.168.1.14   Ready     <none>    1y        v1.11.5
192.168.1.17   Ready     <none>    1y        v1.11.5
...

也可以设置证书自动续期，参考https://blog.csdn.net/feifei3851/article/details/88390425