Flannel网络是在以太网的基础上再封装的一个包含容器IP地址的虚拟网络。

在master节点上建一个文件夹

mkdir flannel

cd flannel

下载安装包

wget https://github.com/coreos/flannel/releases/download/v0.9.1/flannel-v0.9.1-linux-amd64.tar.gz

解压

tar -xzvf flannel-v0.9.1-linux-amd64.tar.gz

将解压后的两个文件flanneld和mk-docker-opts.sh分别拷贝到node节点上

因为我的hosts文件映射为

172.18.98.46    host1
172.18.98.47    host2
172.18.98.48    master

scp flanneld mk-docker-opts.sh root@host1:/opt/kubernetes/bin/

scp flanneld mk-docker-opts.sh root@host2:/opt/kubernetes/bin/

在host1和host2中分别添加网段

cd /opt/kubernetes/ssl

# etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379" set /coreos.com/network/config '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'

查看

# etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379" get /coreos.com/network/config
{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}

cd /opt/kubernetes/cfg

touch flanneld

vim flanneld

内容如下

FLANNEL_OPTIONS="--etcd-endpoints=https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379 -etcd-cafile=/opt/kubernetes/ssl/ca.pem -etcd-certfile=/opt/kubernetes/ssl/server.pem -etcd-keyfile=/opt/kubernetes/ssl/server-key.pem"

cd /usr/lib/systemd/system

touch flanneld.service

vim flanneld.service

内容如下

[Unit]
Description=Flanneld overlay address etcd agent
After=network-online.target network.target
Before=docker.service

[Service]
Type=notify
EnvironmentFile=/opt/kubernetes/cfg/flanneld
ExecStart=/opt/kubernetes/bin/flanneld --ip-masq $FLANNEL_OPTIONS
ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
Restart=on-failure

[Install]
WantedBy=multi-user.target

启动flanneld.service

service flanneld start

成功启动，查看进程

# ps -ef | grep flanneld
root     24305     1  0 14:28 ?        00:00:01 /opt/kubernetes/bin/flanneld --ip-masq --etcd-endpoints=https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379 -etcd-cafile=/opt/kubernetes/ssl/ca.pem -etcd-certfile=/opt/kubernetes/ssl/server.pem -etcd-keyfile=/opt/kubernetes/ssl/server-key.pem

通过ifconfig,我们可以看到多了一个flannel.1的虚拟网卡

flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 172.17.27.0  netmask 255.255.255.255  broadcast 0.0.0.0
        ether 8a:00:81:c6:2a:a1  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 11 overruns 0  carrier 0  collisions 0

查看subnet.env文件

cd /run/flannel/

# cat subnet.env 
DOCKER_OPT_BIP="--bip=172.17.27.1/24"
DOCKER_OPT_IPMASQ="--ip-masq=false"
DOCKER_OPT_MTU="--mtu=1450"
DOCKER_NETWORK_OPTIONS=" --bip=172.17.27.1/24 --ip-masq=false --mtu=1450"

这里面就是分配了一个子网。

让docker使用该网络，修改docker.service

cd /usr/lib/systemd/system

vim docker.service

修改内容

[Service]
Type=notify
EnvironmentFile=/run/flannel/subnet.env
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS

红色部分为修改内容

重启docker

systemctl daemon-reload & systemctl restart docker

再次执行ifconfig，可以看到flannel和docker的网桥在同一个网段，表示开始生效

docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 172.17.27.1  netmask 255.255.255.0  broadcast 172.17.27.255
        ether 02:42:ff:c9:b9:9a  txqueuelen 0  (Ethernet)
        RX packets 9430218  bytes 10206182292 (9.5 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 7696548  bytes 2199505782 (2.0 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 172.17.27.0  netmask 255.255.255.255  broadcast 0.0.0.0
        ether 8a:00:81:c6:2a:a1  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 11 overruns 0  carrier 0  collisions 0

在两台node都装好的情况下，可以互相ping一下对方的flannel的网关

比如我在host2上ping host1的flannel网关

# ping 172.17.27.1
PING 172.17.27.1 (172.17.27.1) 56(84) bytes of data.
64 bytes from 172.17.27.1: icmp_seq=1 ttl=64 time=0.440 ms
64 bytes from 172.17.27.1: icmp_seq=2 ttl=64 time=0.379 ms
64 bytes from 172.17.27.1: icmp_seq=3 ttl=64 time=0.333 ms
64 bytes from 172.17.27.1: icmp_seq=4 ttl=64 time=0.363 ms
64 bytes from 172.17.27.1: icmp_seq=5 ttl=64 time=0.377 ms

完全没有问题

查看所有node的flannel网段（在node节点上，任意节点）

cd /opt/kubernetes/ssl

# etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379" ls /coreos.com/network/subnets
/coreos.com/network/subnets/172.17.27.0-24
/coreos.com/network/subnets/172.17.94.0-24

这个是我的两个node节点的网段。

获取某一个节点的key

# etcdctl --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem --endpoints="https://172.18.98.48:2379,https://172.18.98.47:2379,https://172.18.98.46:2379" get /coreos.com/network/subnets/172.17.27.0-24
{"PublicIP":"172.18.98.46","BackendType":"vxlan","BackendData":{"VtepMAC":"8a:00:81:c6:2a:a1"}}

查看路由

# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         gateway         0.0.0.0         UG    0      0        0 eth0
link-local      0.0.0.0         255.255.0.0     U     1002   0        0 eth0
172.17.27.0     172.17.27.0     255.255.255.0   UG    0      0        0 flannel.1
172.17.94.0     0.0.0.0         255.255.255.0   U     0      0        0 docker0
172.18.96.0     0.0.0.0         255.255.240.0   U     0      0        0 eth0