使用外部 etcd 时,kubeadm 不再负责 etcd 的生老病死,只做一件事:将 kube-apiserver 配置为外部 etcd 的客户端。核心在于提前备好 etcd 的 CA 与客户端证书,并在 ClusterConfiguration 中声明 etcd.external,让控制平面顺利“连库”启动。

1、配置介绍

  • ClusterConfiguration,etcd.localetcd.external 是互斥的,只能二选一
  • 写了 etcd.local → kubeadm 负责 etcd 全生命周期
  • 写了 etcd.external → kubeadm 完全不管 etcd,只当它是“外部依赖”

1.1、etcd.local

# local 配置
etcd:
  local:
    dataDir: /data/etcd

kubeadm 会自动完成以下所有事情:

阶段

kubeadm 行为

证书

生成 etcd CA + server/peer/client 证书

Static Pod

/etc/kubernetes/manifests/etcd.yaml 写 etcd Pod

kubelet

通过 Static Pod 机制拉起 etcd

apiserver

自动注入 etcd 相关参数

升级

kubeadm upgrade 会升级 etcd 版本

备份

kubeadm init phase etcd 可单独操作

1.2、etcd.external

# external 配置
etcd:
  external:
    endpoints:
      - https://10.0.0.11:2379
      - https://10.0.0.12:2379
      - https://10.0.0.13:2379
    caFile: /etc/kubernetes/pki/etcd/ca.crt
    certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
    keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key

kubeadm 只做一件事:把 apiserver 配置成 etcd 的客户端

2、借 Kubeadm 生成外部 ETCD 证书

2.1、准备 local 配置(只用来生证书)

# kubeadm-local-init.yaml
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
nodeRegistration:
  name: master01
  criSocket: unix:///run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  kubeletExtraArgs:
    root-dir: /data/kubelet
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
clusterName: kubernetes
kubernetesVersion: 1.24.17
controlPlaneEndpoint: "172.21.143.53:6443"   # VIP:6443
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
etcd:
  local:
    dataDir: /data/etcd
    serverCertSANs:
      - "172.21.143.37"                       # etcd 集群 ip
      - "172.21.143.38"
      - "172.21.143.39"
    peerCertSANs:
      - "172.21.143.37"                       # etcd 集群 ip
      - "172.21.143.38"
      - "172.21.143.39"
apiServer:
  timeoutForControlPlane: 10m0s
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: "systemd"

2.2、执行 init 生成证书

# 此处仅使用配置生成相关证书
kubeadm init phase certs all --config=kubeadm-init-config.yaml

ls /etc/kubernetes/pki/etcd/
# 你会看到 kubeadm 用自己 CA 签好的完整 etcd PKI
# ca.crt  ca.key
# server.crt  server.key
# peer.crt  peer.key
# apiserver-etcd-client.crt  apiserver-etcd-client.key
# healthcheck-client.crt  healthcheck-client.key

# 将该证书给 etcd 使用(详细步骤可见博客步骤4:https://blog.csdn.net/mm1234556/article/details/160848692)

3、初始化集群

  • 待 etcd 集群状态正常后,执行下方步骤

3.1、准备 external 配置

# kubeadm-external-init.yaml
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
nodeRegistration:
  name: master01
  criSocket: unix:///run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  kubeletExtraArgs:
    root-dir: /data/kubelet
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
clusterName: kubernetes
kubernetesVersion: 1.24.17
controlPlaneEndpoint: "172.21.143.53:6443"   # VIP:6443
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
etcd:
  external:
    endpoints:
      - https://172.21.143.37:2379
      - https://172.21.143.38:2379
      - https://172.21.143.39:2379
    caFile:   /etc/kubernetes/pki/etcd/ca.crt                       # 配置 2.2 步骤生成的证书
    certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
    keyFile:  /etc/kubernetes/pki/apiserver-etcd-client.key
apiServer:
  timeoutForControlPlane: 10m0s
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: "systemd"

3.2、初始化集群

# 步骤 2.2 生成的证书仅保留 etcd 相关证书
## /etc/kubernetes/pki/etcd 目录 + /etc/kubernetes/pki/apiserver-etcd-client.crt + 

# 初始化集群
kubeadm init --config kubeadm-external-init.yaml --upload-certs

更多推荐