声明:
如果您有更好的技术与作者分享,或者商业合作;
请访问作者个人网站 http://www.esqabc.com/view/message.html 留言给作者。
如果该案例触犯您的专利,请在这里:http://www.esqabc.com/view/message.html 留言给作者说明原由
作者一经查实,马上删除。

.
.

1、Etcd集群各节点的名称和ip如下:

前提提条件、服务器,请查看这个地址:https://blog.csdn.net/esqabc/article/details/102726771

内网IP名称
172.26.16.249k8s-01
172.26.16.250k8s-02
172.26.16.251k8s-03

2、下载etcd

a、官方下载地址:https://github.com/etcd-io/etcd/releases/tag/v3.3.17/etcd-v3.3.17-linux-amd64.tar.gz
下载后上传到:/opt/k8s/work

[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 ~]# tar -xvf etcd-v3.3.17-linux-amd64.tar.gz

b、分发二进制文件到集群节点

[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 ~]# source /opt/k8s/bin/environment.sh

for node_ip in ${ETCD_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp etcd-v3.3.17-linux-amd64/etcd* root@${node_ip}:/opt/k8s/bin
    ssh root@${node_ip} "chmod +x /opt/k8s/bin/*"
  done

3、创建etcd证书和私钥

a、创建证书json

[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 ~]# cat > etcd-csr.json <<EOF
添加下面内容:

{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "172.26.16.249",
    "172.26.16.250",
    "172.26.16.251"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "4Paradigm"
    }
  ]
}
EOF

说明一下:

  • hosts:字段指定授权使用该证书的etcd节点IP或域名列表,需要将etcd集群的3个节点都添加其中

b、生成证书和私钥

[root@k8s-01 ~]# cd /opt/k8s/work

cfssl gencert -ca=/opt/k8s/work/ca.pem \
    -ca-key=/opt/k8s/work/ca-key.pem \
    -config=/opt/k8s/work/ca-config.json \
    -profile=kubernetes etcd-csr.json | cfssljson -bare etcd

[root@k8s-01 work]# ls etcd*pem
在这里插入图片描述

c、分发证书和私钥到etcd各个节点

[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 ~]# source /opt/k8s/bin/environment.sh

for node_ip in ${ETCD_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p /etc/etcd/cert"
    scp etcd*.pem root@${node_ip}:/etc/etcd/cert/
  done

在这里插入图片描述

d、创建etcd的启动文件

[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 ~]# cat > etcd.service.template <<EOF

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos
[Service]
Type=notify
WorkingDirectory=${ETCD_DATA_DIR}
ExecStart=/opt/k8s/bin/etcd \\
  --data-dir=${ETCD_DATA_DIR} \\
  --wal-dir=${ETCD_WAL_DIR} \\
  --name=##NODE_NAME## \\
  --cert-file=/etc/etcd/cert/etcd.pem \\
  --key-file=/etc/etcd/cert/etcd-key.pem \\
  --trusted-ca-file=/etc/kubernetes/cert/ca.pem \\
  --peer-cert-file=/etc/etcd/cert/etcd.pem \\
  --peer-key-file=/etc/etcd/cert/etcd-key.pem \\
  --peer-trusted-ca-file=/etc/kubernetes/cert/ca.pem \\
  --peer-client-cert-auth \\
  --client-cert-auth \\
  --listen-peer-urls=https://##NODE_IP##:2380 \\
  --initial-advertise-peer-urls=https://##NODE_IP##:2380 \\
  --listen-client-urls=https://##NODE_IP##:2379,http://127.0.0.1:2379 \\
  --advertise-client-urls=https://##NODE_IP##:2379 \\
  --initial-cluster-token=etcd-cluster-0 \\
  --initial-cluster=${ETCD_NODES} \\
  --initial-cluster-state=new \\
  --auto-compaction-mode=periodic \\
  --auto-compaction-retention=1 \\
  --max-request-bytes=33554432 \\
  --quota-backend-bytes=6442450944 \\
  --heartbeat-interval=250 \\
  --election-timeout=2000
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF

说明一下:

  • WorkDirectory、–data-dir 指定etcd工作目录和数据存储为${ETCD_DATA_DIR},需要在启动前创建这个目录
  • –wal-dir 指定wal目录,为了提高性能,一般使用SSD和–data-dir不同的盘
  • –name
    指定节点名称,当–initial-cluster-state值为new时,–name的参数值必须位于–initial-cluster列表中
  • –cert-file、–key-file ETCD server与client通信时使用的证书和私钥
  • –trusted-ca-file 签名client证书的CA证书,用于验证client证书
  • –peer-cert-file、–peer-key-file ETCD与peer通信使用的证书和私钥
  • –peer-trusted-ca-file 签名peer证书的CA证书,用于验证peer证书

e、分发启动文件到各个节点

(1)、分发会将配置文件中的#替换成ip

[root@k8s-01 ~]# cd /opt/k8s/work

for (( i=0; i < 3; i++ ))
  do
    sed -e "s/##NODE_NAME##/${MASTER_NAMES[i]}/" -e "s/##NODE_IP##/${ETCD_IPS[i]}/" etcd.service.template > etcd-${ETCD_IPS[i]}.service 
  done

[root@k8s-01 ~]# ls *.service
在这里插入图片描述

(2)、分发会将配置文件中的#替换成ip

[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 ~]# source /opt/k8s/bin/environment.sh

for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp etcd-${node_ip}.service root@${node_ip}:/etc/systemd/system/etcd.service
  done

4、启动etcd服务

a、重命名etcd启动文件并启动etcd服务

[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 ~]# source /opt/k8s/bin/environment.sh

for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p ${ETCD_DATA_DIR} ${ETCD_WAL_DIR}"
    ssh root@${node_ip} "systemctl daemon-reload && systemctl enable etcd && systemctl restart etcd " &
  done

b、检查etcd启动结果

for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "systemctl status etcd|grep Active"
  done

如果启动正常,如下图:
在这里插入图片描述
如果etcd集群状态不是active (running),请使用下面命令查看etcd日志

[root@k8s-01 ~]# journalctl -fu etcd

c、验证ETCD集群状态,在任意etcd节点执行

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ETCDCTL_API=3 /opt/k8s/bin/etcdctl \
    --endpoints=https://${node_ip}:2379 \
    --cacert=/etc/kubernetes/cert/ca.pem \
    --cert=/etc/etcd/cert/etcd.pem \
    --key=/etc/etcd/cert/etcd-key.pem endpoint health
  done

正常状态,如下图:
在这里插入图片描述
d、查看当前etcd集群leader

[root@k8s-01 ~]# cd /opt/k8s/work

ETCDCTL_API=3 /opt/k8s/bin/etcdctl \
  -w table --cacert=/etc/kubernetes/cert/ca.pem \
  --cert=/etc/etcd/cert/etcd.pem \
  --key=/etc/etcd/cert/etcd-key.pem \
  --endpoints=${ETCD_ENDPOINTS} endpoint status

正常,如下图:
在这里插入图片描述
.

5、部署Flannel网络

a、下载分发flanneld二进制文件

[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 work]# mkdir flannel
[root@k8s-01 work]# wget http://down.i4t.com/k8s1.14/flannel-v0.11.0-linux-amd64.tar.gz
[root@k8s-01 work]# tar -xzvf flannel-v0.11.0-linux-amd64.tar.gz -C flannel

b、分发二进制文件到所有集群的节点

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp flannel/{flanneld,mk-docker-opts.sh} root@${node_ip}:/opt/k8s/bin/
    ssh root@${node_ip} "chmod +x /opt/k8s/bin/*"
  done

c、创建Flannel证书和私钥

[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 ~]# cat > flanneld-csr.json <<EOF
添加下面内容:

{
  "CN": "flanneld",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "4Paradigm"
    }
  ]
}
EOF

d、生成证书和私钥

[root@k8s-01 ~]# cd /opt/k8s/work

cfssl gencert -ca=/opt/k8s/work/ca.pem \
  -ca-key=/opt/k8s/work/ca-key.pem \
  -config=/opt/k8s/work/ca-config.json \
  -profile=kubernetes flanneld-csr.json | cfssljson -bare flanneld

[root@k8s-01 ~]# ls flanneld*pem
在这里插入图片描述

e、将生成的证书和私钥分发到所有节点

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p /etc/flanneld/cert"
    scp flanneld*.pem root@${node_ip}:/etc/flanneld/cert
  done

f、向etcd写入Pod网段信息

[root@k8s-01 ~]# cd /opt/k8s/work

etcdctl \
  --endpoints=${ETCD_ENDPOINTS} \
  --ca-file=/opt/k8s/work/ca.pem \
  --cert-file=/opt/k8s/work/flanneld.pem \
  --key-file=/opt/k8s/work/flanneld-key.pem \
  mk ${FLANNEL_ETCD_PREFIX}/config '{"Network":"'${CLUSTER_CIDR}'", "SubnetLen": 21, "Backend": {"Type": "vxlan"}}'

g、创建flanneld的启动文件

[root@k8s-01 ~]# cd /opt/k8s/work
[root@k8s-01 work]# cat > flanneld.service << EOF

[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
After=network-online.target
Wants=network-online.target
After=etcd.service
Before=docker.service
[Service]
Type=notify
ExecStart=/opt/k8s/bin/flanneld \\
  -etcd-cafile=/etc/kubernetes/cert/ca.pem \\
  -etcd-certfile=/etc/flanneld/cert/flanneld.pem \\
  -etcd-keyfile=/etc/flanneld/cert/flanneld-key.pem \\
  -etcd-endpoints=${ETCD_ENDPOINTS} \\
  -etcd-prefix=${FLANNEL_ETCD_PREFIX} \\
  -iface=${IFACE} \\
  -ip-masq
ExecStartPost=/opt/k8s/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker
Restart=always
RestartSec=5
StartLimitInterval=0
[Install]
WantedBy=multi-user.target
RequiredBy=docker.service
EOF

h、分发启动文件到所有节点

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp flanneld.service root@${node_ip}:/etc/systemd/system/
  done

i、启动flanneld服务

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "systemctl daemon-reload && systemctl enable flanneld && systemctl restart flanneld"
  done

j、检查启动结果

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "systemctl status flanneld|grep Active"
  done

正常结果:
在这里插入图片描述
k、检查分配给flanneld的Pod网段信息

[root@k8s-01 ~]# cd /opt/k8s/work

etcdctl \
  --endpoints=${ETCD_ENDPOINTS} \
  --ca-file=/etc/kubernetes/cert/ca.pem \
  --cert-file=/etc/flanneld/cert/flanneld.pem \
  --key-file=/etc/flanneld/cert/flanneld-key.pem \
  get ${FLANNEL_ETCD_PREFIX}/config

l、查看已分配的Pod子网网段列表

[root@k8s-01 ~]# cd /opt/k8s/work

etcdctl \
  --endpoints=${ETCD_ENDPOINTS} \
  --ca-file=/etc/kubernetes/cert/ca.pem \
  --cert-file=/etc/flanneld/cert/flanneld.pem \
  --key-file=/etc/flanneld/cert/flanneld-key.pem \
  ls ${FLANNEL_ETCD_PREFIX}/subnets

m、查看节点flannel网络信息

[root@k8s-01 ~]# ip addr show
在这里插入图片描述

n、检查是否创建了 flannel 接口

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh ${node_ip} "/usr/sbin/ip addr show flannel.1|grep -w inet"
  done

在这里插入图片描述
.
.
.

也可以参考这篇文章:https://i4t.com/4253.html

Logo

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐