部署k8s 1.30.x-底层用docker cri 集群管理Rancher全链路部署指南
·
本文按照环境规划、系统准备、容器运行时安装(Docker CRI)、Kubernetes集群部署、Rancher管理平台部署的完整流程,提供详细的部署方案。
第一章:环境规划与系统准备
1.1 集群架构设计
采用 3 Master + 2 Worker 的高可用架构,与参考博文保持一致,但运行时替换为Docker CRI,管理平台替换为Rancher。
| 节点类型 | 主机名 | IP 地址 | 角色 | 特殊说明 |
|---|---|---|---|---|
| Master | master01 | 192.168.1.10 | control-plane, etcd | 同时部署Rancher Server |
| Master | master02 | 192.168.1.20 | control-plane, etcd | 高可用节点 |
| Master | master03 | 192.168.1.30 | control-plane, etcd | 高可用节点 |
| Worker | worker01 | 192.168.1.41 | node | 应用工作节点 |
| Worker | worker02 | 192.168.1.42 | node | 应用工作节点 |
网络规划:
- Pod CIDR:10.244.0.0/16
- Service CIDR:10.96.0.0/12
- 节点网络:192.168.1.0/24
- Rancher域名:rancher.k8s.local(需配置本地DNS或hosts解析)
1.2 系统初始化(所有节点执行)
# 1. 禁用Swap
sudo swapoff -a
sudo sed -i '/ swap / s/^/#/' /etc/fstab
# 2. 设置主机名和hosts(按节点分别执行)
# 在master01节点执行:
sudo hostnamectl set-hostname master01
# 在master02节点执行:
sudo hostnamectl set-hostname master02
# 其他节点类似
# 编辑/etc/hosts文件(所有节点相同配置)
cat <<EOF | sudo tee -a /etc/hosts
192.168.1.10 master01
192.168.1.20 master02
192.168.1.30 master03
192.168.1.41 worker01
192.168.1.42 worker02
EOF
# 3. 时间同步
sudo timedatectl set-timezone Asia/Shanghai
sudo apt update
sudo apt install -y chrony
sudo systemctl enable --now chrony
# 4. 加载内核模块
sudo modprobe overlay
sudo modprobe br_netfilter
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
# 5. 内核参数优化
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
sudo sysctl --system
# 6. 安装基础工具
sudo apt update && sudo apt install -y \
apt-transport-https \
ca-certificates \
curl \
gnupg \
lsb-release \
net-tools \
jq \
wget \
vim
第二章:Docker CRI运行时安装与配置
2.1 Docker安装(替代Containerd)
# 1. 卸载旧版本Docker(如果存在)
sudo apt-get remove -y docker docker-engine docker.io containerd runc
# 2. 安装依赖包
sudo apt-get update
sudo apt-get install -y \
ca-certificates \
curl \
gnupg \
lsb-release
# 3. 添加Docker官方GPG密钥
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
# 4. 设置Docker稳定版仓库
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# 5. 安装Docker引擎(指定兼容版本)
sudo apt-get update
sudo apt-get install -y docker-ce=5:24.0.7-1~ubuntu.22.04~jammy \
docker-ce-cli=5:24.0.7-1~ubuntu.22.04~jammy \
containerd.io \
docker-buildx-plugin \
docker-compose-plugin
# 6. 配置Docker CRI
sudo mkdir -p /etc/docker
cat <<EOF | sudo tee /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m",
"max-file": "3"
},
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
],
"registry-mirrors": [
"https://docker.mirrors.ustc.edu.cn",
"https://hub-mirror.c.163.com"
],
"live-restore": true,
"iptables": false,
"bridge": "none"
}
EOF
# 7. 重启Docker服务
sudo systemctl daemon-reload
sudo systemctl restart docker
sudo systemctl enable docker
# 8. 验证Docker安装
sudo docker version
sudo docker run hello-world
2.2 Docker CRI验证
# 1. 检查Docker CRI socket
ls -la /var/run/docker.sock
# 2. 验证cgroup驱动
docker info | grep -i cgroup
# 3. 测试容器运行
sudo docker run -d --name nginx-test nginx:alpine
sudo docker ps
sudo docker exec nginx-test nginx -v
第三章:Kubernetes集群部署(基于Docker CRI)
3.1 安装Kubernetes组件
# 1. 添加Kubernetes APT源
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
# 2. 安装指定版本组件
sudo apt update
sudo apt install -y \
kubelet=1.30.1-00 \
kubeadm=1.30.1-00 \
kubectl=1.30.1-00
sudo apt-mark hold kubelet kubeadm kubectl
3.2 配置kubelet使用Docker CRI
# 在所有节点配置kubelet使用Docker CRI
cat <<EOF | sudo tee /etc/default/kubelet
KUBELET_EXTRA_ARGS="--container-runtime=remote --container-runtime-endpoint=unix:///var/run/dockershim.sock --cgroup-driver=systemd"
EOF
# 重启kubelet
sudo systemctl daemon-reload
sudo systemctl restart kubelet
3.3 高可用集群初始化
首节点初始化(在master01执行)
# 创建kubeadm配置文件
cat <<EOF > kubeadm-config-docker.yaml
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
nodeRegistration:
criSocket: unix:///var/run/dockershim.sock
kubeletExtraArgs:
cgroup-driver: systemd
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: v1.30.1
controlPlaneEndpoint: "192.168.1.10:6443"
networking:
podSubnet: "10.244.0.0/16"
serviceSubnet: "10.96.0.0/12"
imageRepository: "registry.aliyuncs.com/google_containers"
apiServer:
extraArgs:
service-node-port-range: 30000-32767
controllerManager:
extraArgs:
node-cidr-mask-size: "24"
scheduler:
extraArgs:
bind-address: "0.0.0.0"
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
maxPods: 110
EOF
# 初始化集群
sudo kubeadm init --config kubeadm-config-docker.yaml --upload-certs
# 配置kubectl
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
# 保存加入命令(重要!)
kubeadm token create --print-join-command > join-command.txt
kubeadm init phase upload-certs --upload-certs | grep certificate-key >> join-command.txt
配置kubectl自动补全
# 配置bash自动补全
echo 'source <(kubectl completion bash)' >> ~/.bashrc
echo 'alias k=kubectl' >> ~/.bashrc
echo 'complete -F __start_kubectl k' >> ~/.bashrc
source ~/.bashrc
3.4 Calico网络插件部署
# 下载Calico manifest
curl https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml -O
# 修改Pod CIDR以匹配配置
sed -i 's|# - name: CALICO_IPV4POOL_CIDR|- name: CALICO_IPV4POOL_CIDR|g' calico.yaml
sed -i 's|# value: "192.168.0.0/16"| value: "10.244.0.0/16"|g' calico.yaml
# 部署Calico
kubectl apply -f calico.yaml
# 验证Calico部署
kubectl get pods -n kube-system -l k8s-app=calico-node
3.5 添加其他Master节点
# 在master02和master03上执行(使用master01生成的命令)
# 示例命令格式:
sudo kubeadm join 192.168.1.10:6443 \
--token <your-token> \
--discovery-token-ca-cert-hash sha256:<your-hash> \
--control-plane \
--certificate-key <your-cert-key>
3.6 Worker节点加入集群
# 在worker01和worker02上执行
sudo kubeadm join 192.168.1.10:6443 \
--token <your-token> \
--discovery-token-ca-cert-hash sha256:<your-hash>
第四章:Rancher管理平台部署
4.1 Rancher Server部署(单节点Docker方式)
# 在master01节点上部署Rancher Server
# 1. 创建Rancher数据目录
sudo mkdir -p /opt/rancher
sudo chmod 755 /opt/rancher
# 2. 使用Docker运行Rancher(推荐使用稳定版本)
sudo docker run -d \
--restart=unless-stopped \
--name rancher-server \
-p 80:80 -p 443:443 \
--privileged \
-v /opt/rancher:/var/lib/rancher \
rancher/rancher:v2.8.2
# 3. 查看Rancher启动日志
sudo docker logs -f rancher-server
# 4. 等待Rancher启动(约1-2分钟),然后访问:
# https://192.168.1.10
4.2 初始Rancher配置
- 首次访问:浏览器打开
https://192.168.1.10 - 设置管理员密码:按照提示设置admin用户密码
- 配置服务器URL:设置为
https://192.168.1.10 - 跳过添加集群:选择"Skip"跳过初始集群添加
4.3 导入现有Kubernetes集群到Rancher
# 1. 在Rancher UI中创建集群
# 登录Rancher -> 集群管理 -> 创建 -> 导入现有集群
# 2. 设置集群名称(如:k8s-docker-cluster)
# 3. 点击"创建"
# 4. 获取集群注册命令
# 在Rancher生成的页面中,复制以下格式的命令:
# kubectl apply -f https://192.168.1.10/v3/import/xxxxx.yaml
# 5. 在master01节点执行注册命令
kubectl apply -f https://192.168.1.10/v3/import/xxxxx.yaml
# 6. 验证Rancher Agent状态
kubectl get pods -n cattle-system
kubectl get pods -n cattle-system -w # 监控启动过程
4.4 Rancher多节点高可用部署(可选,生产环境推荐)
# 1. 安装Helm
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
# 2. 添加Rancher Helm仓库
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable
helm repo update
# 3. 创建命名空间
kubectl create namespace cattle-system
# 4. 安装cert-manager
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.3/cert-manager.crds.yaml
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.13.3
# 5. 安装Rancher(高可用模式)
helm install rancher rancher-stable/rancher \
--namespace cattle-system \
--set hostname=rancher.k8s.local \
--set replicas=3 \
--set bootstrapPassword=Admin@123456 \
--set ingress.tls.source=rancher
第五章:集群验证与功能测试
5.1 集群健康检查
# 1. 检查节点状态
kubectl get nodes -o wide
# 2. 检查核心组件
kubectl get pods -n kube-system
# 3. 检查Docker CRI状态
kubectl describe node master01 | grep -A5 -B5 "Container Runtime"
# 4. 创建测试Pod验证Docker运行时
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: docker-cri-test
spec:
containers:
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
EOF
# 检查Pod状态
kubectl get pod docker-cri-test -o wide
kubectl describe pod docker-cri-test | grep -i runtime
5.2 Rancher功能验证
-
登录Rancher Dashboard:
https://192.168.1.10 -
查看集群状态:确保集群状态为"Active"
-
测试工作负载部署:
- 在Rancher UI中创建Deployment
- 验证Pod调度到Worker节点
- 检查使用Docker作为运行时
-
测试Rancher多集群管理(如有多个集群):
- 添加新集群测试
- 验证统一监控和日志
5.3 网络与存储测试
# 1. 部署测试应用
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-app
spec:
replicas: 2
selector:
matchLabels:
app: test-app
template:
metadata:
labels:
app: test-app
spec:
containers:
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: test-app-service
spec:
selector:
app: test-app
ports:
- port: 80
targetPort: 80
type: ClusterIP
EOF
# 2. 验证服务发现
kubectl get svc test-app-service
kubectl run test-curl --image=curlimages/curl -it --rm -- curl test-app-service
第六章:安全加固与优化建议
6.1 Docker CRI安全配置
# 1. 配置Docker守护进程安全参数
sudo tee -a /etc/docker/daemon.json <<EOF
{
"userns-remap": "default",
"no-new-privileges": true,
"selinux-enabled": true,
"icc": false,
"live-restore": true,
"userland-proxy": false
}
EOF
# 2. 重启Docker
sudo systemctl restart docker
# 3. 配置Docker镜像扫描
# 安装Trivy扫描工具
sudo apt install -y wget apt-transport-https
----
## 参考来源
- [超详细教程:Ubuntu 22.04 搭建 Kubernetes 1.30.1 集群并部署 Dashboard 2.7.0 + Ingress-Nginx 1.10.1](https://blog.csdn.net/hanhanduizhang/article/details/149200427)
- [如何通过rancher部署k8s集群](https://blog.csdn.net/monarch91/article/details/122763156)
- [rancher2.6部署k8s集群示例](https://blog.csdn.net/monarch91/article/details/124596535)
- [rancher 安装 部署 k8s](https://blog.csdn.net/weixin_47752736/article/details/125018358)
- [rancher2.7.5部署kubernetesv1.26版本集群](https://blog.csdn.net/weixin_38924998/article/details/135826685)
- [Rancher部署K8S容器集群及NFS存储](https://blog.csdn.net/weixin_44029834/article/details/140545524)
更多推荐
所有评论(0)