一、环境准备

1、准备3台干净的机器

主机名	地址	角色	配置
kub-k8s-master	192.168.96.134	主节点	2核4G
kub-k8s-node1	192.168.96.135	工作节点	1核2G
kub-k8s-node2	192.168.96.136	工作节点	1核2G

 2、关闭防火墙&selinux

[root@master ~]# systemctl stop firewalld
[root@master ~]# systemctl enable firewalld
[root@master ~]# vim /etc/sysconfig/selinux
[root@master ~]# setenforce 0
或
[root@master ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/sysconfig/selinux

3、时间同步

[root@docker ~]# ntpdate ntp.aliyun.com
或
[root@docker ~]# ntpdate time.windows.com

4、配置静态ip

[root@docker ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33

5、修改主机名

hostnamectl set-hostname master(node1,node2)

6、本地解析

echo >> /etc/hosts <<EOF
192.168.96.134 master
192.168.96.135 node1
192.168.96.136 node2
EOF

7、关闭系统Swap 

sed -i 's/.*swap.*/#&/' /etc/fstab

二、安装docker

1、清理环境（如果是新的机器就不用执行）

 yum remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-selinux \
docker-engine-selinux \
docker-engine

2、配置阿里云Docker Yum源

yum install -y yum-utils device-mapper-persistent-data lvm2 git
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

3、安装docker

3.1安装较旧版本（比如Docker 17.03.2) ：
 需要指定完整的rpm包的包名，并且加上--setopt=obsoletes=0 参数：
 yum install -y --setopt=obsoletes=0 \
 docker-ce-17.03.2.ce-1.el7.centos.x86_64 \
 docker-ce-selinux-17.03.2.ce-1.el7.centos.noarch

3.2安装新版本

yum install docker-ce -y
systemctl enable docker
systemctl start docker

报错1：
    docker info的时候报如下错误
    bridge-nf-call-iptables is disabled

解决1：
    追加如下配置,然后重启系统
    # vim /etc/sysctl.conf   
    net.bridge.bridge-nf-call-ip6tables = 1
    net.bridge.bridge-nf-call-iptables = 1
    net.bridge.bridge-nf-call-arptables = 1

问题2：
    虚拟机ping百度也能ping通，但是需要等好几秒才出结果，关键是下载镜像一直报错如下
    # docker pull daocloud.io/library/nginx
    Using default tag: latest
    Error response from daemon: Get https://daocloud.io/v2/: dial tcp: lookup daocloud.io on 192.168.1.2:53: read udp   192.168.1.189:41335->192.168.1.2:53: i/o timeout

解决2：
    我的虚拟机用的网关和dns都是虚拟机自己的.1或者.2，把DNS改成8.8.8.8问题就解决了，ping百度也秒出结果

4、生产docker的环境配置

sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://pilvpemn.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

#注意：一定注意编码问题，出现错误---查看命令：journalctl -amu docker 即可发现错误

三、安装Kubeadm包

1、配置阿里云源

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

 2、下载kube镜像

docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.22.0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.22.0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.22.0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.22.0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:v1.8.4
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.0-0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5

# 下载完了之后需要将aliyun下载下来的所有镜像打成k8s.gcr.io/kube-controller-manager:v1.22.0这样的tag
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.22.0 k8s.gcr.io/kube-controller-manager:v1.22.0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.22.0 k8s.gcr.io/kube-proxy:v1.22.0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.22.0 k8s.gcr.io/kube-apiserver:v1.22.0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.22.0 k8s.gcr.io/kube-scheduler:v1.22.0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:v1.8.4 k8s.gcr.io/coredns/coredns:v1.8.4
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.0-0 k8s.gcr.io/etcd:3.5.0-0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5 k8s.gcr.io/pause:3.5    

# 可以清理掉aliyun的镜像标签
docker rmi -f `docker images --format {{.Repository}}:{{.Tag}} | grep aliyun`

3、安装依赖包及常用软件包

 yum install -y conntrack ntpdate ntp ipvsadm ipset jq iptables curl sysstat libseccomp wget vim net-tools git iproute lrzsz bash-completion tree bridge-utils unzip bind-utils gcc

3、安装对应版本

 yum install -y kubelet-1.22.0-0.x86_64 kubeadm-1.22.0-0.x86_64 kubectl-1.22.0-0.x86_64

4、上传kubelet打好的包

注:已经做过打包处理过的安装包，仅做参考

5、加载ipvs相关内核模块

 cat <<EOF > /etc/modules-load.d/ipvs.conf 
ip_vs
ip_vs_lc
ip_vs_wlc
ip_vs_rr
ip_vs_wrr
ip_vs_lblc
ip_vs_lblcr
ip_vs_dh
ip_vs_sh
ip_vs_nq
ip_vs_sed
ip_vs_ftp
ip_vs_sh
nf_conntrack_ipv4
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip
EOF

6、配置转发相关参数，否则可能会出错

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF

7、重启服务器使配置生效

reboot

如果net.bridge.bridge-nf-call-iptables报错，加载br_netfilter模块
# modprobe br_netfilter
# modprobe ip_conntrack
# sysctl -p /etc/sysctl.d/k8s.conf
8、查看是否加载成功

 lsmod | grep ip_vs

四、配置启动kubelet

1.配置kubelet使用pause镜像

 DOCKER_CGROUPS=`docker info |grep 'Cgroup' | awk ' NR==1 {print $3}'`

2、配置kubelet的cgroups

 cat >/etc/sysconfig/kubelet<<EOF
KUBELET_EXTRA_ARGS="--cgroup-driver=$DOCKER_CGROUPS --pod-infra-container-image=k8s.gcr.io/pause:3.5"
EOF

3、启动

systemctl daemon-reload
systemctl enable kubelet && systemctl restart kubelet

注：以上的操作三台服务器都要操作

五、配置master节点[在master上执行]

1、运行初始化过程

 kubeadm init --kubernetes-version=v1.22.0 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.96.10

注：
apiserver-advertise-address=192.168.96.10    ---master的ip地址。
--kubernetes-version=v1.22.0   --更具具体版本进行修改

 如果在node上也运行初始化过程，请重置

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
 [root@kub-k8s-master ~] kubeadm reset --force

六、配置使用网络插件[在master上执行]

curl -L https://docs.projectcalico.org/v3.22/manifests/calico.yaml -O
kubectl apply -f  calico.yaml

七、node加入集群[在两台node上执行]

kubeadm join 192.168.96.134:6443 --token pd2k8x.jlgxrsht2prsgkhc \
        --discovery-token-ca-cert-hash sha256:68a049dbfb67d5682a6a439f8f6f5e9041ccb7ab59965cec9bf614e5077ad1f2 

八、后续检查[master]

1、查看pods

kubectl get pods -n kube-system
kubectl get pod -A

2、查看节点：

 kubectl get nodes

Kubeadm 方式部署集群部署完成