本实验记录安装Kubernetes集群的经过，目的是在个人电脑上安装Kubernetes集群，以一个Master和两个Worker节点为例。

实验环境

项目	内容	备注
虚拟平台	Workstation 16 Pro	17966106
OS	CentOS 7	2009
Docker	20.10.8	docker-ce-stable
kubeadm	1.21.3
kubelet	1.21.3
kubectl	1.21.3
coredns	1.8.0	k8s.gcr.io/coredns/coredns
pause	3.4.1
etcd	3.4.13

安装步骤

安装虚拟机

在Workstation环境使用虚拟机，采用NAT模式，

查看或修改虚拟机网络情况，编辑–>虚拟网络编辑器

我们配置使用192.168.21.0/24子网，打开NAT设置可以看到网关为192.168.21.2

以上信息在安装虚拟机是用于手动配置地址

主机名	角色	IP地址
mater-01	Master Node	192.168.21.168
worker-01	Worker Node	192.168.21.11
worker-02	Worker Node	192.168.21.12

安装docker

依然选用dockers作为kubernetes的runtime，本步骤三台虚拟都需要

1. Update Docker Package Database

yum check-update

2. Update yum

yum update

3. 如果没有执行第二步，需要安装依赖

yum install -y yum-utils device-mapper-persistent-data lvm2

4. Add the Docker Repository to CentOS（国内）
   这里使用阿里云地址：

yum-config-manager  --add-repo  http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

5. 更新yum软件包索引

yum makecache fast

6. Install the selected Docker version with the command:

yum install docker-ce-<VERSION STRING>
systemctl enable docker
systemctl start docker

7. 修改daemon.json
   kubernetes文件驱动默认由systemd, 而我们安装的docker使用的文件驱动是cgroupfs, 造成不一致, 会导致在初始化Kubernetes时镜像无法启动。

可以通过docker info查看。

修改或创建/etc/docker/daemon.json，加入下面的内容：

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

重启docker:

systemctl restart docker
systemctl status docker

要是已经做了kubeadm init不成功，修改好cgroupdriver后，需要

[root@master01 ~]# rm -rf /etc/kubernetes /var/lib/kubelet

再次初始化（先kubeadm reset）

安装Kubeadm/kubelet和Kubectl

本步骤适合所有Node

环境准备

1. 配置Kubernetes 国内仓库

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
 http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

可以过通过以下命令查版本：

yum list kubelet kubeadm kubectl --showduplicates|sort -r

然后再用yum install 按版本安装

[root@master01 ~]# yum list kubelet kubeadm kubectl  --showduplicates|sort -r
……
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
 * updates: centos-distro.cavecreek.net
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Loading mirror speeds from cached hostfile
Loaded plugins: fastestmirror, langpacks
……
kubelet.x86_64                       1.8.3-0                          kubernetes
kubelet.x86_64                       1.8.2-1                          kubernetes
kubelet.x86_64                       1.8.2-0                          kubernetes
kubelet.x86_64                       1.8.15-0                         kubernetes
kubelet.x86_64                       1.8.14-0                         kubernetes
kubelet.x86_64                       1.8.13-0                         kubernetes
kubelet.x86_64                       1.8.12-0                         kubernetes
kubelet.x86_64                       1.8.1-1                          kubernetes
kubelet.x86_64                       1.8.11-0                         kubernetes
kubelet.x86_64                       1.8.1-0                          kubernetes
kubelet.x86_64                       1.8.10-0                         kubernetes
kubelet.x86_64                       1.8.0-1                          kubernetes
kubelet.x86_64                       1.8.0-0                          kubernetes
……

2. Configure Firewall

On the Master Node enter:

sudo firewall-cmd --permanent --add-port=6443/tcp
sudo firewall-cmd --permanent --add-port=2379-2380/tcp
sudo firewall-cmd --permanent --add-port=10250/tcp
sudo firewall-cmd --permanent --add-port=10251/tcp
sudo firewall-cmd --permanent --add-port=10252/tcp
sudo firewall-cmd --permanent --add-port=10255/tcp
sudo firewall-cmd --reload

On the Worker Node enter:

sudo firewall-cmd --permanent --add-port=10251/tcp
sudo firewall-cmd --permanent --add-port=10255/tcp
sudo firewall-cmd --reload

3. Update Iptables Settings
   Set the net.bridge.bridge-nf-call-iptables to ‘1’ in your sysctl config file.
   This ensures that packets are properly processed by IP tables during filtering and port forwarding.

cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system # 查看结果

4. Disable SELinux
   The containers need to access the host filesystem. SELinux needs to be set to permissive or disabled mode, which effectively disables its security functions.
   Use following commands to disable SELinux:

sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

5. Disable SWAP
   Lastly, we need to disable SWAP to enable the kubelet to work properly:

sudo sed -i '/swap/d' /etc/fstab
sudo swapoff -a

下载安装所需images

本步骤适合所有Node

1. 查看所需images

[root@master-01 ~]# kubeadm config images list
I0810 11:52:33.618380   16253 version.go:254] remote version is much newer: v1.22.0; falling back to: stable-1.21
k8s.gcr.io/kube-apiserver:v1.21.3
k8s.gcr.io/kube-controller-manager:v1.21.3
k8s.gcr.io/kube-scheduler:v1.21.3
k8s.gcr.io/kube-proxy:v1.21.3
k8s.gcr.io/pause:3.4.1
k8s.gcr.io/etcd:3.4.13-0
k8s.gcr.io/coredns/coredns:v1.8.0

2. 使用脚本下载

#!/bin/bash
images=(
    kube-apiserver:v1.21.3
    kube-controller-manager:v1.21.3
    kube-scheduler:v1.21.3
    kube-proxy:v1.21.3
    pause:3.4.1
    etcd:3.4.13-0
    coredns:1.8.0
)

for imageName in ${images[@]} ; do
    docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
    docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
    docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
done

需要注意的是k8s.gcr.io/coredns/coredns:v1.8.0的名字和下载后tag不一致：k8s.gcr.io/coredns:v1.8.0，需要重新tag

安装

sudoapt-get update

sudoapt-get install -y kubelet=1.21.3 kubeadm=1.21.3 kubectl=1.21.3

初始化Kubernets

1. kubeadm init

kubeadm init --pod-network-cidr=10.211.0.0/16 --service-cidr=10.96.0.0/16 --kubernetes-version=1.21.3

2. Manage Cluster as Regular User
   To start using the cluster you need to run it as a regular user by typing:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

3. Join Worker Node to Cluster
   As indicated in Step 1, use the kubeadm join command on each worker node to connect it to the cluster.

kubeadm join --discovery-token cfgrty.1234567890jyrfgd --discovery-token-ca-cert-hash sha256:1234..cdef 1.2.3.4:6443

Replace the codes with the ones from master server. Repeat this action for each worker node on cluster.

CNI

Use Antrea CNI

kubectl apply -f https://raw.githubusercontent.com/antrea-io/antrea/main/build/yamls/antrea.yml

安装好antrea后，可以看到：

[root@master-01 ~]# kubectl get node
NAME        STATUS   ROLES                  AGE    VERSION
master-01   Ready    control-plane,master   3d9h   v1.21.3
worker-01   Ready    <none>                 3d8h   v1.21.3
worker-02   Ready    <none>                 3d8h   v1.21.3

Node —>Ready

以上