K8s 1.20.11 Ansible 自动化部署完整方案

2402_88311944

235人浏览 · 2026-08-24 22:00:06

2402_88311944 · 2026-08-24 22:00:06 发布

一、目录结构设计

plaintext

/opt/k8s-ansible/
├── ansible.cfg
├── inventory/
│   └── hosts
├── group_vars/
│   └── all.yml
├── roles/
│   ├── common/
│   │   ├── tasks/
│   │   │   └── main.yml
│   │   └── templates/
│   │       └── hosts.j2
│   ├── docker/
│   │   ├── tasks/
│   │   │   └── main.yml
│   │   └── templates/
│   │       └── daemon.json.j2
│   ├── k8s-base/
│   │   └── tasks/
│   │       └── main.yml
│   ├── master/
│   │   └── tasks/
│   │       └── main.yml
│   ├── worker/
│   │   └── tasks/
│   │       └── main.yml
│   └── network/
│       └── tasks/
│           └── main.yml
├── playbooks/
│   ├── 01-init-env.yml
│   ├── 02-install-docker.yml
│   ├── 03-install-k8s.yml
│   ├── 04-init-master.yml
│   ├── 05-join-worker.yml
│   ├── 06-install-network.yml
│   └── deploy-all.yml
└── README.md

二、创建目录命令（在 master01 10.8.0.10 上执行）

bash

mkdir -p /opt/k8s-ansible/{inventory,group_vars,playbooks}
mkdir -p /opt/k8s-ansible/roles/common/{tasks,templates}
mkdir -p /opt/k8s-ansible/roles/docker/{tasks,templates}
mkdir -p /opt/k8s-ansible/roles/k8s-base/tasks
mkdir -p /opt/k8s-ansible/roles/master/tasks
mkdir -p /opt/k8s-ansible/roles/worker/tasks
mkdir -p /opt/k8s-ansible/roles/network/tasks
cd /opt/k8s-ansible

三、各配置文件内容

1. ansible.cfg

bash

vim /opt/k8s-ansible/ansible.cfg

ini

[defaults]
inventory = ./inventory/hosts
host_key_checking = False
remote_user = root
log_path = ./ansible.log
retry_files_enabled = False
gathering = smart
fact_caching = jsonfile
fact_caching_connection = ./facts_cache
fact_caching_timeout = 86400

[privilege_escalation]
become = True
become_method = sudo
become_user = root
become_ask_pass = False

2. inventory/hosts

bash

vim /opt/k8s-ansible/inventory/hosts

ini

[masters]
master01 ansible_host=10.8.0.10

[workers]
node01 ansible_host=10.8.0.12
node02 ansible_host=10.8.0.13

[k8s:children]
masters
workers

[k8s:vars]
ansible_ssh_pass=你的root密码

把 你的root密码 替换成实际密码，或配置 SSH 免密后去掉此行。

3. group_vars/all.yml

bash

vim /opt/k8s-ansible/group_vars/all.yml

yaml

k8s_version: "1.20.11"
k8s_cidr: "10.244.0.0/16"
k8s_apiserver: "10.8.0.10"
docker_registry_mirrors:
  - "https://docker.mirrors.ustc.edu.cn"
  - "https://hub-mirror.c.163.com"
pause_image: "registry.aliyuncs.com/google_containers/pause:3.2"
kube_proxy_image: "registry.aliyuncs.com/google_containers/kube-proxy:v1.20.11"
flannel_image: "quay.io/coreos/flannel:v0.14.0"

四、Roles 各任务文件

Role 1: common（基础环境初始化）

bash

vim /opt/k8s-ansible/roles/common/tasks/main.yml

yaml

- name: 关闭防火墙
  systemd:
    name: firewalld
    state: stopped
    enabled: no

- name: 关闭selinux
  selinux:
    state: disabled

- name: 临时关闭selinux
  command: setenforce 0
  ignore_errors: yes

- name: 关闭swap
  command: swapoff -a

- name: 注释fstab中的swap
  replace:
    path: /etc/fstab
    regexp: '^([^#].*swap.*)$'
    replace: '# \1'

- name: 配置主机名解析
  template:
    src: hosts.j2
    dest: /etc/hosts

- name: 加载ipvs内核模块
  modprobe:
    name: "{{ item }}"
    state: present
  loop:
    - ip_vs
    - ip_vs_rr
    - ip_vs_wrr
    - ip_vs_sh
    - nf_conntrack_ipv4
    - br_netfilter

- name: 配置内核参数
  sysctl:
    name: "{{ item.key }}"
    value: "{{ item.value }}"
    sysctl_file: /etc/sysctl.d/k8s.conf
    reload: yes
  loop:
    - { key: "net.bridge.bridge-nf-call-iptables", value: "1" }
    - { key: "net.bridge.bridge-nf-call-ip6tables", value: "1" }
    - { key: "net.ipv4.ip_forward", value: "1" }
    - { key: "vm.swappiness", value: "0" }

- name: 安装基础工具
  yum:
    name:
      - wget
      - curl
      - vim
      - net-tools
      - conntrack-tools
      - ipset
      - ipvsadm
      - socat
      - yum-utils
      - device-mapper-persistent-data
      - lvm2
    state: present

bash

vim /opt/k8s-ansible/roles/common/templates/hosts.j2

plaintext

127.0.0.1   localhost localhost.localdomain
::1         localhost localhost.localdomain
10.8.0.10   master01
10.8.0.12   node01
10.8.0.13   node02

Role 2: docker（安装配置 docker）

bash

vim /opt/k8s-ansible/roles/docker/tasks/main.yml

yaml

- name: 添加docker yum源
  yum_repository:
    name: docker-ce
    description: Docker CE Stable
    baseurl: https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/stable
    gpgcheck: yes
    gpgkey: https://mirrors.aliyun.com/docker-ce/linux/centos/gpg

- name: 安装docker
  yum:
    name:
      - docker-ce-19.03.15
      - docker-ce-cli-19.03.15
      - containerd.io
    state: present

- name: 创建docker配置目录
  file:
    path: /etc/docker
    state: directory

- name: 配置docker daemon.json
  template:
    src: daemon.json.j2
    dest: /etc/docker/daemon.json

- name: 启动docker并设置开机自启
  systemd:
    name: docker
    state: started
    enabled: yes
    daemon_reload: yes

- name: 预拉取pause镜像
  command: docker pull {{ pause_image }}

- name: 给pause镜像打标签
  command: docker tag {{ pause_image }} k8s.gcr.io/pause:3.2

bash

vim /opt/k8s-ansible/roles/docker/templates/daemon.json.j2

json

{
  "registry-mirrors": {{ docker_registry_mirrors | to_json }},
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}

Role 3: k8s-base（安装 kubeadm/kubelet/kubectl）

bash

vim /opt/k8s-ansible/roles/k8s-base/tasks/main.yml

yaml

- name: 添加k8s阿里云yum源
  yum_repository:
    name: kubernetes
    description: Kubernetes
    baseurl: https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
    gpgcheck: no
    enabled: yes

- name: 安装k8s组件
  yum:
    name:
      - kubelet-{{ k8s_version }}
      - kubeadm-{{ k8s_version }}
      - kubectl-{{ k8s_version }}
    state: present
    disable_excludes: kubernetes

- name: 启动kubelet并设置开机自启
  systemd:
    name: kubelet
    state: started
    enabled: yes
    daemon_reload: yes

Role 4: master（初始化主节点）

bash

vim /opt/k8s-ansible/roles/master/tasks/main.yml

yaml

- name: 检查是否已初始化
  stat:
    path: /etc/kubernetes/admin.conf
  register: k8s_init_check

- name: kubeadm初始化主节点
  command: >
    kubeadm init --apiserver-advertise-address={{ k8s_apiserver }}
    --image-repository=registry.aliyuncs.com/google_containers
    --kubernetes-version=v{{ k8s_version }}
    --service-cidr=10.96.0.0/12
    --pod-network-cidr={{ k8s_cidr }}
  when: not k8s_init_check.stat.exists

- name: 创建root的kube目录
  file:
    path: /root/.kube
    state: directory

- name: 复制admin.conf到root目录
  copy:
    src: /etc/kubernetes/admin.conf
    dest: /root/.kube/config
    remote_src: yes
    owner: root
    group: root

- name: 获取join命令
  command: kubeadm token create --print-join-command
  register: join_command
  changed_when: false

- name: 保存join命令到本地文件
  copy:
    content: "{{ join_command.stdout }}"
    dest: /opt/k8s-ansible/join-command.sh
  delegate_to: localhost
  become: no

- name: 修改kube-proxy镜像为阿里云
  command: >
    kubectl set image daemonset/kube-proxy
    kube-proxy={{ kube_proxy_image }}
    -n kube-system
  ignore_errors: yes

Role 5: worker（从节点加入集群）

bash

vim /opt/k8s-ansible/roles/worker/tasks/main.yml

yaml

- name: 检查是否已加入集群
  stat:
    path: /etc/kubernetes/kubelet.conf
  register: k8s_join_check

- name: 读取join命令
  slurp:
    src: /opt/k8s-ansible/join-command.sh
  register: join_cmd_file
  delegate_to: localhost
  become: no

- name: 节点加入集群
  command: "{{ join_cmd_file.content | b64decode }}"
  when: not k8s_join_check.stat.exists

Role 6: network（安装 flannel 网络插件）

bash

vim /opt/k8s-ansible/roles/network/tasks/main.yml

yaml

- name: 下载flannel yaml
  get_url:
    url: https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
    dest: /opt/kube-flannel.yml
    force: yes
  ignore_errors: yes

- name: 应用flannel网络插件
  command: kubectl apply -f /opt/kube-flannel.yml
  ignore_errors: yes

- name: 等待flannel pod启动
  command: kubectl get pods -n kube-flannel
  register: flannel_status
  until: flannel_status.stdout.find("Running") != -1
  retries: 30
  delay: 10
  ignore_errors: yes

五、Playbooks 编排文件

01-init-env.yml

bash

vim /opt/k8s-ansible/playbooks/01-init-env.yml

yaml

- hosts: k8s
  roles:
    - common

02-install-docker.yml

bash

vim /opt/k8s-ansible/playbooks/02-install-docker.yml

yaml

- hosts: k8s
  roles:
    - docker

03-install-k8s.yml

bash

vim /opt/k8s-ansible/playbooks/03-install-k8s.yml

yaml

- hosts: k8s
  roles:
    - k8s-base

04-init-master.yml

bash

vim /opt/k8s-ansible/playbooks/04-init-master.yml

yaml

- hosts: masters
  roles:
    - master

05-join-worker.yml

bash

vim /opt/k8s-ansible/playbooks/05-join-worker.yml

yaml

- hosts: workers
  roles:
    - worker

06-install-network.yml

bash

vim /opt/k8s-ansible/playbooks/06-install-network.yml

yaml

- hosts: masters
  roles:
    - network

deploy-all.yml（一键部署总入口）

bash

vim /opt/k8s-ansible/playbooks/deploy-all.yml

yaml

- import_playbook: 01-init-env.yml
- import_playbook: 02-install-docker.yml
- import_playbook: 03-install-k8s.yml
- import_playbook: 04-init-master.yml
- import_playbook: 05-join-worker.yml
- import_playbook: 06-install-network.yml

六、部署执行流程

前置：master01 安装 ansible

bash

yum install -y epel-release
yum install -y ansible
ansible --version

步骤 1：配置 SSH 免密（master01 执行）

bash

ssh-keygen -t rsa -N "" -f /root/.ssh/id_rsa
ssh-copy-id root@10.8.0.10
ssh-copy-id root@10.8.0.12
ssh-copy-id root@10.8.0.13

返回这个代表成功

步骤 2：测试 ansible 连通性

bash

cd /opt/k8s-ansible
ansible all -m ping

预期三台机器全部返回 SUCCESS。

步骤 3：分步执行（推荐，便于排错）

bash

cd /opt/k8s-ansible

ansible-playbook playbooks/01-init-env.yml
ansible-playbook playbooks/02-install-docker.yml
ansible-playbook playbooks/03-install-k8s.yml
ansible-playbook playbooks/04-init-master.yml
ansible-playbook playbooks/05-join-worker.yml
ansible-playbook playbooks/06-install-network.yml

步骤 4：一键部署（确认无误后使用）

bash

cd /opt/k8s-ansible
ansible-playbook playbooks/deploy-all.yml

步骤 5：验证集群

bash

kubectl get nodes
kubectl get pods -n kube-system
kubectl get pods -n kube-flannel

Ansible 部署 K8s v1.20.11 踩坑总结

一、Ansible 配置问题

表格

问题	根因	修复
role 找不到	playbook 在 `playbooks/` 子目录，默认从 playbook 所在目录找 roles	`ansible.cfg` 加 `roles_path = /opt/k8s-ansible/roles`（绝对路径）
变量未定义 `docker_registry_mirrors`	`group_vars/` 应与 inventory 文件同级，不是项目根目录	`mv group_vars inventory/`
`setenforce 0` 报错	selinux 已是 disabled，再执行返回非零	加 `ignore_errors: yes`（已忽略，不影响）

二、镜像源问题（国内环境核心坑）

表格

问题	根因	修复
flannel yaml 下载超时	`raw.githubusercontent.com` 国内 SSL 握手超时	改用 template 生成本地 yaml，不联网下载
flannel 镜像 `ErrImagePull`	`coreos/flannel` 在 docker.io，加速器无缓存，回源超时	改成 `quay.io/coreos/flannel:v0.14.0`（quay.io 国内可访问）
阿里云镜像地址不存在	`registry.cn-beijing.aliyuncs.com/imcto/flannel` 是我编的，实际不存在	放弃该地址，用 quay.io 官方镜像

三、CNI 网络问题（最耗时的坑）

表格

问题	根因	修复
节点全部 NotReady	flannel v0.14.0 把 CNI 插件二进制拆到独立镜像 `flannelcni/flannel-cni-plugin`，国内拉不到，`/opt/cni/bin/flannel` 缺失	用 `bridge` 插件替代，手动创建 `cni0` 网桥 + 读取 `subnet.env` 生成配置
coredns 一直 ContainerCreating	CNI 配置的 `subnet` 字段写成了 `10.244.1.1/24`（带主机位），host-local IPAM 要求网络地址 `10.244.1.0/24`	subnet 改成 `10.244.x.0/24`，cni0 网桥 IP 保持 `10.244.x.1`

四、最终结果

plaintext

master01  Ready   v1.20.11
node01    Ready   v1.20.11
node02    Ready   v1.20.11

kube-system 下 etcd、apiserver、controller-manager、scheduler、kube-proxy×3、flannel×3、coredns×2 全部 Running。

五、核心教训

国内部署 K8s，所有镜像必须提前确认可拉取，docker.io 冷门镜像基本超时，优先 quay.io 或阿里云
flannel v0.14+ 拆分了 CNI 插件，要么用 v0.13 及以前版本，要么手动补 bridge 配置
CNI subnet 必须是网络地址（.0），不能是网关地址（.1）
Ansible 的 roles_path 和 group_vars 路径容易踩坑，必须用绝对路径或 inventory 同级目录

亚马逊云科技技术品牌专区

更多推荐

基于STM32的全屋环境智能监测与预警终端设计

亚马逊云科技技术品牌专区

Keras：面向人类的深度学习库

Keras3支持多后端（TensorFlow、PyTorch、JAX），通过设置环境变量切换，兼容指定版本。pip安装后可用Sequential API快速构建CNN模型，如MNIST分类示例，实

亚马逊云科技技术品牌专区

AI行业日报｜2026-09-25：5个热点事件

本期行业动态聚焦于端侧算力重构、自主智能体工作流、开发工具经济模型、网络隔离架构设计与公共部门合规审查。以下结合一线技术场景进行结构化梳理，供 AI 研究员与软件工程团队参考。 PrismML 正将其轻量级语言模型适配至搭载高通处理器的智能眼镜终端 1 。该平台的核心战略指向开源权重模型的本地化运行，旨在最大化挖掘穿戴设备现有算力的利用效率 1 。对于嵌入式与边缘计算开发者而言，此类微型模型向低功