基于Docker搭建多主多从K8s高可用集群 - 在线方式 - 永久证书

主机名IP地址角色操作系统硬件配置
ansible10.62.158.200同步工具节点CentOS 72 Core/4G Memory
master0110.62.158.201管理节点01CentOS 72 Core/4G Memory
master0210.62.158.202管理节点02CentOS 72 Core/4G Memory
master0310.62.158.203管理节点03CentOS 72 Core/4G Memory
node0110.62.158.204工作节点01CentOS 71 Core/2G Memory
node0210.62.158.205工作节点02CentOS 71 Core/2G Memory
k8s-ha0110.62.158.206主代理节点CentOS 71 Core/2G Memory
k8s-ha0210.62.158.207备用代理节点CentOS 71 Core/2G Memory

查看当前集群状态

[root@master01 ~]# kubectl get node
NAME       STATUS   ROLES                  AGE   VERSION
master01   Ready    control-plane,master   22d   v1.23.0
master02   Ready    control-plane,master   22d   v1.23.0
master03   Ready    control-plane,master   22d   v1.23.0
node01     Ready    <none>                 22d   v1.23.0
node02     Ready    <none>                 22d   v1.23.0

查看证书过期时间

[root@master01 ~]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Apr 17, 2025 04:39 UTC   342d                                    no      
apiserver                  Apr 17, 2025 04:39 UTC   342d            ca                      no      
apiserver-etcd-client      Apr 17, 2025 04:39 UTC   342d            etcd-ca                 no      
apiserver-kubelet-client   Apr 17, 2025 04:39 UTC   342d            ca                      no      
controller-manager.conf    Apr 17, 2025 04:39 UTC   342d                                    no      
etcd-healthcheck-client    Apr 17, 2025 04:39 UTC   342d            etcd-ca                 no      
etcd-peer                  Apr 17, 2025 04:39 UTC   342d            etcd-ca                 no      
etcd-server                Apr 17, 2025 04:39 UTC   342d            etcd-ca                 no      
front-proxy-client         Apr 17, 2025 04:39 UTC   342d            front-proxy-ca          no      
scheduler.conf             Apr 17, 2025 04:39 UTC   342d                                    no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Apr 15, 2034 04:39 UTC   9y              no      
etcd-ca                 Apr 15, 2034 04:39 UTC   9y              no      
front-proxy-ca          Apr 15, 2034 04:39 UTC   9y              no

GoLang环境安装 - go1.22.0.linux-amd64.tar.gz

# 下载环境文件
[root@master01 ~]# wget https://studygolang.com/dl/golang/go1.17.6.linux-amd64.tar.gz
[root@master01 ~]# ls
anaconda-ks.cfg  calico.yaml  docker  docker-20.10.tar.gz  go1.17.6.linux-amd64.tar.gz  kubeadm-config.yml  nginx.yml  sysconfigure.sh
# 文件解压
[root@master01 ~]# tar -xvf go1.17.6.linux-amd64.tar.gz -C /usr/local/
# 配置环境变量
[root@master01 ~]# echo "export PATH=$PATH:/usr/local/go/bin" >>/etc/profile
[root@master01 ~]# source /etc/profile
# 验证go环境
[root@master01 ~]# go version
go version go1.17.6 linux/amd64

下载kubernetes源码 - kubernetes-1.23.0.tar.gz

# 查看下当前已安装的k8s版本
[root@master01 ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.0", GitCommit:"ab69524f795c42094a6630298ff53f3c3ebab7f4", GitTreeState:"clean", BuildDate:"2021-12-07T18:16:20Z", GoVersion:"go1.17.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.0", GitCommit:"ab69524f795c42094a6630298ff53f3c3ebab7f4", GitTreeState:"clean", BuildDate:"2021-12-07T18:09:57Z", GoVersion:"go1.17.3", Compiler:"gc", Platform:"linux/amd64"}
# 下载对应的源码
[root@master01 ~]# wget https://github.com/kubernetes/kubernetes/archive/v1.23.0.tar.gz
[root@master01 ~]# ls
anaconda-ks.cfg  calico.yaml  docker  docker-20.10.tar.gz  go1.17.6.linux-amd64.tar.gz  kubeadm-config.yml  kubernetes-1.23.0.tar.gz  nginx.yml  sysconfigure.sh
# 源码解压
[root@master01 ~]# tar -zxvf kubernetes-1.23.0.tar.gz
[root@master01 ~]# ls
anaconda-ks.cfg  calico.yaml  docker  docker-20.10.tar.gz  go1.17.6.linux-amd64.tar.gz  kubeadm-config.yml  kubernetes-1.23.0  kubernetes-1.23.0.tar.gz  nginx.yml  sysconfigure.sh

修改证书有效期

[root@master01 ~]# cd kubernetes-1.23.0
[root@master01 kubernetes-1.23.0]# vim ./cmd/kubeadm/app/constants/constants.go
CertificateValidity = time.Hour * 24 * 365 * 100
[root@master kubernetes-1.23.0]# vim staging/src/k8s.io/client-go/util/cert/cert.go
NotAfter:              now.Add(duration365d * 100).UTC(),

编译源代码文件

[root@master01 kubernetes-1.23.0]# make WHAT=cmd/kubeadm GOFLAGS=-v

注意：若出现 ./hack/run-in-gopath.sh:行34: _output/bin/prerelease-lifecycle-gen: 权限不够 问题时，需要添加权限后再编译源码文件

[root@master01 kubernetes-1.23.0]# yum install rsync jq -y
[root@master01 bin]# chmod +x _output/bin/prerelease-lifecycle-gen
[root@master01 bin]# chmod +x _output/bin/deepcopy-gen

查看编译后kubeadm二进制文件 - kubeadm

[root@master01 kubernetes-1.23.0]# ls -l _output/bin/
总用量 79012
-rwxr-xr-x 1 root root  6275072 5月   9 18:19 conversion-gen
-rwxr-xr-x 1 root root  5996544 5月   9 18:19 deepcopy-gen
-rwxr-xr-x 1 root root  6000640 5月   9 18:19 defaulter-gen
-rwxr-xr-x 1 root root  3376695 5月   9 18:19 go2make
-rwxr-xr-x 1 root root 45170688 5月   9 18:43 kubeadm
-rwxr-xr-x 1 root root  8114176 5月   9 18:19 openapi-gen
-rwxr-xr-x 1 root root  5971968 5月   9 18:19 prerelease-lifecycle-gen

备份原有的kubeadm文件

[root@master01 kubernetes-1.23.0]# cp /usr/bin/kubeadm /usr/bin/kubeadm_bak20240509

替换新编译的kubeadm文件覆盖旧的kubeadm文件

# 集群初始化时仅使用kubeadm二进制文件即可，需要赋予可执行权限
[root@master01 kubernetes-1.23.0]# cp /root/kubernetes-1.23.0/_output/bin/kubeadm /usr/bin/

备份pki证书文件

[root@master01 kubernetes-1.23.0]# cd /etc/kubernetes
[root@master01 kubernetes]# cp -R pki/ pki_bak20240509 -- 未初始化不需要这个

证书更新

# CA根证书不生效，需要集群初始化时才生效
[root@master01 kubernetes]# kubeadm certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed

Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates.

查看证书是否为100年

[root@master01 kubernetes]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Apr 15, 2124 11:45 UTC   99y                                     no      
apiserver                  Apr 15, 2124 11:45 UTC   99y             ca                      no      
apiserver-etcd-client      Apr 15, 2124 11:45 UTC   99y             etcd-ca                 no      
apiserver-kubelet-client   Apr 15, 2124 11:45 UTC   99y             ca                      no      
controller-manager.conf    Apr 15, 2124 11:45 UTC   99y                                     no      
etcd-healthcheck-client    Apr 15, 2124 11:45 UTC   99y             etcd-ca                 no      
etcd-peer                  Apr 15, 2124 11:45 UTC   99y             etcd-ca                 no      
etcd-server                Apr 15, 2124 11:45 UTC   99y             etcd-ca                 no      
front-proxy-client         Apr 15, 2124 11:45 UTC   99y             front-proxy-ca          no      
scheduler.conf             Apr 15, 2124 11:45 UTC   99y                                     no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Apr 15, 2034 04:39 UTC   9y              no      
etcd-ca                 Apr 15, 2034 04:39 UTC   9y              no      
front-proxy-ca          Apr 15, 2034 04:39 UTC   9y              no
上传01管理节点中编译后的kubeadm二进制文件到02管理节点和03管理节点，02管理节点与03管理节点证书有效期调整，操作步骤一致，以下为02管理节点操作过程

上传kubeadm二进制文件

[root@master02 ~]# ls
anaconda-ks.cfg  docker  docker-20.10.tar.gz  kubeadm  sysconfigure.sh

添加执行权限

[root@master02 ~]# chmod +x kubeadm

备份原有的kubeadm文件

[root@master02 ~]# cp /usr/bin/kubeadm /usr/bin/kubeadm_bak20240509

替换新编译的kubeadm文件覆盖旧的kubeadm文件

[root@master kubernetes-1.23.0]# cp /root/kubeadm /usr/bin/

备份pki证书文件

[root@master02 ~]# cd /etc/kubernetes
[root@master02 kubernetes]# cp -R pki/ pki_bak20240509

证书更新

[root@master02 kubernetes]# kubeadm certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed

Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates.

查看证书是否为100年

[root@master02 kubernetes]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Apr 15, 2124 11:56 UTC   99y                                     no      
apiserver                  Apr 15, 2124 11:56 UTC   99y             ca                      no      
apiserver-etcd-client      Apr 15, 2124 11:56 UTC   99y             etcd-ca                 no      
apiserver-kubelet-client   Apr 15, 2124 11:56 UTC   99y             ca                      no      
controller-manager.conf    Apr 15, 2124 11:56 UTC   99y                                     no      
etcd-healthcheck-client    Apr 15, 2124 11:56 UTC   99y             etcd-ca                 no      
etcd-peer                  Apr 15, 2124 11:56 UTC   99y             etcd-ca                 no      
etcd-server                Apr 15, 2124 11:56 UTC   99y             etcd-ca                 no      
front-proxy-client         Apr 15, 2124 11:56 UTC   99y             front-proxy-ca          no      
scheduler.conf             Apr 15, 2124 11:56 UTC   99y                                     no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Apr 15, 2034 04:39 UTC   9y              no      
etcd-ca                 Apr 15, 2034 04:39 UTC   9y              no      
front-proxy-ca          Apr 15, 2034 04:39 UTC   9y              no
# kubernetes # 容器 # 云原生
Logo
K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 |  韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号​前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。​长夜有穷，真水无香。领略过了 K8s“身在江

avatar K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub

avatar K8S/Kubernetes

k8s 火了！

2020，上云之年，产品云端化成为一种趋势。在一线城市，很多公司都已经构建了自己的私有云环境，比如阿里云、网易云、华为云等。而Kubernetes 作为基于容器编排领域的王者，具备扩展...

avatar K8S/Kubernetes