又来更新部署了，这次为配合新集群应用，需要部署V1.21.5版本，供学习使用

一、环境准备

三台虚机：

操作系统： CentOS Linux release 7.9.2009 (Core)   

内核版本：3.10.0-1160.88.1.el7.x86_64

k8s-master01: 192.168.66.200   

k8s-master02: 192.168.66.201

k8s-node01:    192.168.66.250

本次安装k8s 1.18.20版本

二、服务器初始化配置

服务器初始化配置可以参考原先我写的1.15.0版本部署方案 

K8s 1.15.0 版本部署安装-CSDN博客

三、安装docker

3.1、安装指定的Docker版本
yum list docker-ce.x86_64 --showduplicates | sort -r
yum install -y containerd.io-1.6.18 docker-ce-23.0.1 docker-ce-cli-23.0.1
 
3.2、创建docker daemon.json文件
mkdir /etc/docker
cat > /etc/docker/daemon.json <<EOF
{
  "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF
 
3.3 启动docker
systemctl enable docker; systemctl start docker;systemctl status docker
 
3.4 查看安装后的docker版本
docker --version

四、安装软件包

4.1、安装软件
yum install -y kubelet-1.21.5 kubeadm-1.21.5 kubectl-1.21.5 ipvsadm
 
4.2、配置kubelet
cat >  /etc/sysconfig/kubelet <<EOF
KUBELET_CGROUP_ARGS="--cgroup-driver=systemd"
KUBELET_EXTRA_ARGS="--fail-swap-on=false"
EOF
 
4.3、启动kubelet并设置开机启动
systemctl enable kubelet && systemctl start kubelet
 
4.4、检查版本
kubeadm version
确认是所安装的版本

[root@k8s-master01 ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.5", GitCommit:"aea7bbadd2fc0cd689de94a54e5b7b758869d691", GitTreeState:"clean", BuildDate:"2021-09-15T21:09:27Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}

五、部署集群

以下操作在k8s-master01节点上操作

5.1 、生成初始化配置
kubeadm config print init-defaults > kubeadm.yaml

[root@k8s-master01 ~]# cat kubeadm.yaml
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.66.199     ###修改为VIP地址
  bindPort: 16443                       ###修改为VIP端口
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: k8s-master01                   ###修改为主机名
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
#imageRepository: k8s.gcr.io
imageRepository: registry.aliyuncs.com/google_containers   ###修改为阿里云的
kind: ClusterConfiguration
kubernetesVersion: 1.21.0
controlPlaneEndpoint: 192.168.66.199:16443        ###新增
apiServer:                                        ###新增
  certSANs:                                       ###新增
  - 192.168.66.200                                ###新增
  - 192.168.66.201                                ###新增
  - 192.168.66.199                                ###新增
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
  podSubnet: 10.2.0.0/16                          ###新增
scheduler: {}

---                                               ###新增
apiVersion: kubeproxy.config.k8s.io/v1alpha1      ###新增
kind: KubeProxyConfiguration                      ###新增
mode: ipvs                                        ###新增

5.2 、执行初始化配置
kubeadm init --config kubeadm.yaml
成功会显示如下信息

[root@k8s-master01 ~]# kubeadm init --config kubeadm.yaml
W0514 15:40:29.345211   70060 strict.go:54] error unmarshaling configuration schema.GroupVersionKind{Group:"kubeadm.k8s.io", Version:"v1beta2", Kind:"ClusterConfiguration"}: error converting YAML to JSON: yaml: unmarshal errors:
  line 18: key "apiServer" already set in map
[init] Using Kubernetes version: v1.21.0
[preflight] Running pre-flight checks
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 20.10
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.66.199 192.168.66.200 192.168.66.201]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master01 localhost] and IPs [192.168.66.199 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master01 localhost] and IPs [192.168.66.199 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 10.505320 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.21" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node k8s-master01 as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node k8s-master01 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: abcdef.0123456789abcdef
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:

  kubeadm join 192.168.66.199:16443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:8133b9f1d5c51395887ff55d2cbd851658b52d02c5a559ad1ecd50adde734cee \
        --control-plane 

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.66.199:16443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:8133b9f1d5c51395887ff55d2cbd851658b52d02c5a559ad1ecd50adde734cee

5.3 按照要求执行生成文件

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

5.4 查看组件状态

kubectl get cs

[root@k8s-master01 manifests]# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS      MESSAGE                                                                                       ERROR
controller-manager   Unhealthy   Get "http://127.0.0.1:10252/healthz": dial tcp 127.0.0.1:10252: connect: connection refused   
scheduler            Unhealthy   Get "http://127.0.0.1:10251/healthz": dial tcp 127.0.0.1:10251: connect: connection refused   
etcd-0               Healthy     {"health":"true"}

该状态其实不影响整个集群使用，是因为默认的kube-scheduler.yaml和kube-controller-manager.yaml 中默认使用了10251和10252的端口。可以将对应的配置文件中- --port=0 给注释掉

所有master节点上执行
[root@k8s-master01 ~]# sed -i 's/- --port=0/#&/' /etc/kubernetes/manifests/kube-scheduler.yaml
[root@k8s-master01 ~]# sed -i 's/- --port=0/#&/' /etc/kubernetes/manifests/kube-controller-manager.yaml

Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                  
scheduler            Healthy   ok                  
etcd-0               Healthy   {"health":"true"}

5.5、部署calico组件

calico版本

链接：https://pan.baidu.com/s/1AC9cVZFR2kv9gGBqkCa3EQ 
提取码：zuao

下载完成后应用

kubectl apply -f calico.yaml

[root@k8s-master01 ~]# kubectl apply -f /root/calico/calico.yaml 
configmap/calico-config created
Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created
customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created
daemonset.apps/calico-node created
serviceaccount/calico-node created
deployment.apps/calico-kube-controllers created
serviceaccount/calico-kube-controllers created

查看pod和node节点

kubectl get pod -n kube-system

kubectl get node

[root@k8s-master01 ~]# kubectl get pod -n kube-system
NAME                                       READY   STATUS    RESTARTS   AGE
calico-kube-controllers-7676785684-kqchs   1/1     Running   0          10m
calico-node-wrtvz                          1/1     Running   0          10m
coredns-59d64cd4d4-jtq99                   1/1     Running   0          11m
coredns-59d64cd4d4-tdfln                   1/1     Running   0          11m
etcd-k8s-master01                          1/1     Running   0          11m
kube-apiserver-k8s-master01                1/1     Running   0          11m
kube-controller-manager-k8s-master01       1/1     Running   0          6m36s
kube-proxy-mrmwl                           1/1     Running   0          11m
kube-scheduler-k8s-master01                1/1     Running   0          6m45s
[root@k8s-master01 ~]# kubectl get node 
NAME           STATUS   ROLES                  AGE   VERSION
k8s-master01   Ready    control-plane,master   11m   v1.21.5

六、部署node节点

将k8s-node01节点添加到集群中

[root@k8s-node01 ~]# kubeadm join 192.168.66.199:16443 --token abcdef.0123456789abcdef \
>         --discovery-token-ca-cert-hash sha256:310dee45f00d289b43661943cfc74391727bf9823ecb2744e3ac3abb8ad04f2f

[root@k8s-node01 ~]# kubeadm join 192.168.66.199:16443 --token abcdef.0123456789abcdef \
>         --discovery-token-ca-cert-hash sha256:310dee45f00d289b43661943cfc74391727bf9823ecb2744e3ac3abb8ad04f2f 
[preflight] Running pre-flight checks
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 20.10
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

[root@k8s-node01 ~]# 

登录master01节点检查所有组件运行情况

kubectl get pod -A -owide

kubectl get node 

[root@k8s-master01 ~]# kubectl get pod -n kube-system
NAME                                       READY   STATUS    RESTARTS   AGE
calico-kube-controllers-7676785684-kqchs   1/1     Running   0          11m
calico-node-l6w4t                          1/1     Running   0          22s
calico-node-wrtvz                          1/1     Running   0          11m
coredns-59d64cd4d4-jtq99                   1/1     Running   0          12m
coredns-59d64cd4d4-tdfln                   1/1     Running   0          12m
etcd-k8s-master01                          1/1     Running   0          12m
kube-apiserver-k8s-master01                1/1     Running   0          12m
kube-controller-manager-k8s-master01       1/1     Running   0          7m31s
kube-proxy-khmrc                           1/1     Running   0          22s
kube-proxy-mrmwl                           1/1     Running   0          12m
kube-scheduler-k8s-master01                1/1     Running   0          7m40s
[root@k8s-master01 ~]# kubectl get node 
NAME           STATUS   ROLES                  AGE   VERSION
k8s-master01   Ready    control-plane,master   12m   v1.21.5
k8s-node01     Ready    <none>                 23s   v1.21.5

master02节点添加集群

从master01上将证书文件copy到master02上

scp /etc/kubernetes/pki/ca.* 192.168.66.201:/etc/kubernetes/pki/

scp /etc/kubernetes/pki/sa.* 192.168.66.201:/etc/kubernetes/pki/

scp /etc/kubernetes/pki/front-proxy-ca* 192.168.66.201:/etc/kubernetes/pki/

scp /etc/kubernetes/pki/etcd/ca.* 192.168.66.201:/etc/kubernetes/pki/etcd/

[root@k8s-master01 pki]# scp /etc/kubernetes/pki/ca.* 192.168.66.201:/etc/kubernetes/pki/
root@192.168.66.201's password: 
ca.crt                                                                                                                  100% 1066     1.1MB/s   00:00    
ca.key                                                                                                                  100% 1679   689.1KB/s   00:00    
[root@k8s-master01 pki]# scp /etc/kubernetes/pki/sa.* 192.168.66.201:/etc/kubernetes/pki/ 
root@192.168.66.201's password: 
sa.key                                                                                                                  100% 1679     1.9MB/s   00:00    
sa.pub                                                                                                                  100%  451   415.3KB/s   00:00    
[root@k8s-master01 pki]# scp /etc/kubernetes/pki/front-proxy-ca* 192.168.66.201:/etc/kubernetes/pki/    
root@192.168.66.201's password: 
front-proxy-ca.crt                                                                                                      100% 1078     1.3MB/s   00:00    
front-proxy-ca.key                                                                                                      100% 1679     1.7MB/s   00:00    
[root@k8s-master01 pki]# scp /etc/kubernetes/pki/etcd/ca.* 192.168.66.201:/etc/kubernetes/pki/etcd/
root@192.168.66.201's password: 
ca.crt                                                                                                                  100% 1058   456.0KB/s   00:00    
ca.key                                                                                                                  100% 1675     1.1MB/s   00:00    
[root@k8s-master01 pki]# 

集群添加命令master02节点

  kubeadm join 192.168.66.199:16443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:8133b9f1d5c51395887ff55d2cbd851658b52d02c5a559ad1ecd50adde734cee \
        --control-plane 

[root@k8s-master02 pki]#   kubeadm join 192.168.66.199:16443 --token abcdef.0123456789abcdef \
>         --discovery-token-ca-cert-hash sha256:8133b9f1d5c51395887ff55d2cbd851658b52d02c5a559ad1ecd50adde734cee \
>         --control-plane 
[preflight] Running pre-flight checks
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 23.0.1. Latest validated version: 20.10
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks before initializing the new control plane instance
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [k8s-master02 localhost] and IPs [192.168.66.201 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [k8s-master02 localhost] and IPs [192.168.66.201 127.0.0.1 ::1]
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [k8s-master02 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.66.201 192.168.66.199 192.168.66.200]
[certs] Generating "front-proxy-client" certificate and key
[certs] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[certs] Using the existing "sa" key
[kubeconfig] Generating kubeconfig files
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[kubeconfig] Writing "admin.conf" kubeconfig file
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[endpoint] WARNING: port specified in controlPlaneEndpoint overrides bindPort in the controlplane address
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[check-etcd] Checking that the etcd cluster is healthy
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
[etcd] Announced new etcd member joining to the existing etcd cluster
[etcd] Creating static Pod manifest for "etcd"
[etcd] Waiting for the new etcd member to join the cluster. This can take up to 40s
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[mark-control-plane] Marking the node k8s-master02 as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node k8s-master02 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]

This node has joined the cluster and a new control plane instance was created:

* Certificate signing request was sent to apiserver and approval was received.
* The Kubelet was informed of the new secure connection details.
* Control plane (master) label and taint were applied to the new node.
* The Kubernetes control plane instances scaled up.
* A new etcd member was added to the local/stacked etcd cluster.

To start administering your cluster from this node, you need to run the following as a regular user:

        mkdir -p $HOME/.kube
        sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
        sudo chown $(id -u):$(id -g) $HOME/.kube/config

Run 'kubectl get nodes' to see this node join the cluster.

[root@k8s-master02 pki]# 

按照操作

        mkdir -p $HOME/.kube
        sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
        sudo chown $(id -u):$(id -g) $HOME/.kube/config

登录master01节点检查所有组件运行情况

kubectl get pod -A -owide

kubectl get node 

kubectl get cs

[root@k8s-master01 ~]# kubectl get node 
NAME           STATUS   ROLES                  AGE   VERSION
k8s-master01   Ready    control-plane,master   22m   v1.21.5
k8s-master02   Ready    control-plane,master   66s   v1.21.5
k8s-node01     Ready    <none>                 10m   v1.21.5
[root@k8s-master01 ~]# 
[root@k8s-master01 ~]# kubectl get pod -n kube-system
NAME                                       READY   STATUS    RESTARTS   AGE
calico-kube-controllers-7676785684-kqchs   1/1     Running   0          21m
calico-node-l6w4t                          1/1     Running   0          10m
calico-node-v9qvw                          1/1     Running   0          68s
calico-node-wrtvz                          1/1     Running   0          21m
coredns-59d64cd4d4-jtq99                   1/1     Running   0          22m
coredns-59d64cd4d4-tdfln                   1/1     Running   0          22m
etcd-k8s-master01                          1/1     Running   0          22m
etcd-k8s-master02                          1/1     Running   0          63s
kube-apiserver-k8s-master01                1/1     Running   0          22m
kube-apiserver-k8s-master02                1/1     Running   0          67s
kube-controller-manager-k8s-master01       1/1     Running   1          18m
kube-controller-manager-k8s-master02       1/1     Running   0          66s
kube-proxy-bt4nb                           1/1     Running   0          68s
kube-proxy-khmrc                           1/1     Running   0          10m
kube-proxy-mrmwl                           1/1     Running   0          22m
kube-scheduler-k8s-master01                1/1     Running   1          18m
kube-scheduler-k8s-master02                1/1     Running   0          66s

[root@k8s-master01 ~]# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                  
scheduler            Healthy   ok                  
etcd-0               Healthy   {"health":"true"}