1、所有节点需要设置一下系统参数

cat <<EOF >/etc/sysctl.d/k8s.conf

net.ipv4.ip_forward = 1

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

EOF

sysctl -p /etc/sysctl.d/k8s.conf

2.、更新k8s的镜像源

curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

cat << EOF >/etc/apt/sources.list.d/kubernetes.list

deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main

EOF

sudo apt-get update

3、安装 kubelet 、kubeadm 、kubectl

sudo apt-get install -y kubelet kubeadm kubectl

sudo systemctl enable kubelet

4、创建安装配置文件

$ cat <<EOF > kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.20.4
controlPlaneEndpoint: "10.12.70.130:8443"  //vip地址
networking:
   podSubnet: "22.244.0.0/16"    //pod分配的地址段
   serviceSubnet: "22.96.0.0/12"   //service分配的地址段

EOF

//使用kubeadm 安装节点
sudo kubeadm init --config=kubeadm-config.yaml --upload-certs

安装成功会显示如下内容：

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

  kubeadm join 10.12.70.130:8443 --token zjlnlj.ks1dca7a6wu3ppuq \
    --discovery-token-ca-cert-hash sha256:98a766ad94977fbf0321095abcbfb473371062bcd244a877a48e2cb11591b83c \
    --control-plane --certificate-key 80249a72d42714d13e038568e828f613a3a80568bb3e5fe8bbfa02a133923dc9

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.12.70.130:8443 --token zjlnlj.ks1dca7a6wu3ppuq \
    --discovery-token-ca-cert-hash sha256:98a766ad94977fbf0321095abcbfb473371062bcd244a877a48e2cb11591b83c

黄色部分是kubectl命令的操作权限的配置文件，按照显示的内容将文件拷贝到用户目录下，就可以使用kubectl命令进行操作了。若使用sudo kubectl ...命令必须将配置文件放到/root/.kube/目录下。

绿色部分是需要复制记录下来的添加其他节点进入集群的密钥。其中第一段是添加worker节点的，第二段是添加master节点的。最好将其复制到文本文件中，以备下一步使用。

通过ss -tl命令或netstat命令检查服务端口是否已启动

$ netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 172.22.132.10:8443      0.0.0.0:*               LISTEN      11218/haproxy
tcp        0      0 0.0.0.0:9090            0.0.0.0:*               LISTEN      11218/haproxy
tcp        0      0 127.0.0.1:44551         0.0.0.0:*               LISTEN      9237/kubelet
tcp        0      0 127.0.0.1:10248         0.0.0.0:*               LISTEN      9237/kubelet
tcp        0      0 127.0.0.1:10249         0.0.0.0:*               LISTEN      11669/kube-proxy
tcp        0      0 172.22.132.11:2379      0.0.0.0:*               LISTEN      10367/etcd
tcp        0      0 172.22.132.11:2380      0.0.0.0:*               LISTEN      10367/etcd
tcp        0      0 127.0.0.1:10257         0.0.0.0:*               LISTEN      10460/kube-controll
tcp        0      0 127.0.0.1:10259         0.0.0.0:*  

使用kubectl get nodes命令检查初始化的节点安装状态

NAME     STATUS     ROLES    AGE   VERSION
k8s-m1   NotReady   master   31s   v1.21.2
这里状态为NotReady是因为没有安装网络插件，需要安装calico或者flannel网络插件

我安装的是calico插件

curl https://docs.projectcalico.org/manifests/custom-resources.yaml -o calico.yaml

下载下来以后将calico.yaml中的网络段改为你要设置的集群网络段

sed -i 's/192.168.0.0\/16/10.244.0.0\/16/g' calico.yaml

安装calico.yaml

kubectl apply -f calico.yaml

安装之后再使用kubectl get  nodes 命令查看状态

NAME       STATUS   ROLES                  AGE   VERSION
k8s-m1   Ready    control-plane,master   93d   v1.21.2

status列显示Ready就说明安装正常了

接下来就是使用之前保存下来的两个token密钥加入其它节点

首先加入master节点，就是之前规划的安装有keepalived的节点，使用如下命令

kubeadm join 10.12.70.130:8443 --token zjlnlj.ks1dca7a6wu3ppuq \
    --discovery-token-ca-cert-hash sha256:98a766ad94977fbf0321095abcbfb473371062bcd244a877a48e2cb11591b83c \
    --control-plane --certificate-key 80249a72d42714d13e038568e828f613a3a80568bb3e5fe8bbfa02a133923dc9

标红的部分是你自己的密钥值，之前kubeadm init 安装成功后回显的提示信息。

添加完后master节点，同样也需要在master节点上安装calico，和上述步骤一样，在此不再赘述。

其次加入worker节点，这些节点是k8s运行中自动部署pod的服务器，使用如下命令：

kubeadm join 10.12.70.130:8443 --token zjlnlj.ks1dca7a6wu3ppuq \
    --discovery-token-ca-cert-hash sha256:98a766ad94977fbf0321095abcbfb473371062bcd244a877a48e2cb11591b83c

可以看到添加worker节点少了 --control-plane --certificate-key这个参数配置项。

安装成功使用kubectl get nodes 查看状态

NAME       STATUS   ROLES                  AGE   VERSION
k8s70131   Ready    control-plane,master   93d   v1.21.2
k8s70132   Ready    control-plane,master   93d   v1.21.2
k8s70133   Ready    control-plane,master   93d   v1.21.2
k8s70134   Ready    <none>                 18d   v1.20.4
k8s70136   Ready    <none>                 38h   v1.21.2

显示类似这样就说明添加成功了。