Docker Macvlan网络实验
Docker Macvlan网络实验实验环境虚机IP角色OSdocker1190.100.100.41docker宿主机CentOS7.5docker2190.100.100.42docker宿主机CentOS7.5linux190.100.100.40软路由CentOS7.5以上操作系统环境全部为vmware workstation运行的虚机本次实验由易到难共分3个阶段,分别是网卡独占模式、子接
实验环境
虚机 | IP | 角色 | OS |
---|---|---|---|
docker1 | 190.100.100.41 | docker宿主机 | CentOS7.5 |
docker2 | 190.100.100.42 | docker宿主机 | CentOS7.5 |
linux | 190.100.100.40 | 软路由 | CentOS7.5 |
以上操作系统环境全部为vmware workstation运行的虚机
本次实验由易到难共分3个阶段,分别是网卡独占模式、子接口模式(为了共享物理网卡)以及网关模式(目的是为了打通二层的vlan)
一:网卡独占模式
此方法是最原始也最简单的工作模式
优点:配置简单
缺点:在物理网卡有限的情况下,不能复用物理资源,导致资源浪费;另外也不利于容器之间的隔离
本次实验使用docker1和docker2的ens37网卡
Docker1配置
- 创建macvlan网络
docker network create -d macvlan --subnet=166.66.66.0/24 -o parent=ens37 mac_net1
参数说明:
-d macvlan —指定网络类型为macvlan
–subnet=166.66.66.0/24 —指定网络CIDR
-o parent=ens37 —指定网卡(即使网卡上存在IP也不影响创建macvlan)
mac_net1 —给创建的macvlan网络起一个名字
- 启动虚机并指定IP
docker run -itd --name macvlan --ip=166.66.66.41 --network mac_net1 centos
创建了一个名称为macvlan的容器,并分配IP地址为:166.66.66.41
Docker2配置
- 创建macvlan网络
docker network create -d macvlan --subnet=166.66.66.0/24 -o parent=ens37 mac_net1
参数说明:
-d macvlan —指定网络类型为macvlan
–subnet=166.66.66.0/24 —指定网络CIDR
-o parent=ens37 —指定网卡(即使网卡上存在IP也不影响创建macvlan)
mac_net1 —给创建的macvlan网络起一个名字
- 启动虚机并指定IP
docker run -itd --name macvlan --ip=166.66.66.42 --network mac_net1 centos
测试网络连通
从docker1 ping docker2
从docker2 ping docker1
至此,第1个试验测试完成,停止虚机并删除macvlan配置,进行下一个实验。
二:子接口macvlan
Docker1虚机配置
生成网卡子接口配置
配置文件内容如下:
ens37.10子接口配置
cd /etc/sysconfig/network-scripts/
cat ifcfg-ens37.10 ifcfg-ens37.20
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens37.10
DEVICE=ens37.10
ONBOOT=yes
VLAN=yes
ens37.20子接口配置
cd /etc/sysconfig/network-scripts/
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens37.20
DEVICE=ens37.20
ONBOOT=yes
VLAN=yes
启用网卡
[root@docker1 network-scripts]# ifup ens37.10 ens37.20
INFO : [ipv6_wait_tentative] Waiting for interface ens37.10 IPv6 address(es) to leave the 'tentative' state
INFO : [ipv6_wait_tentative] Waiting for interface ens37.10 IPv6 address(es) to leave the 'tentative' state
[root@docker1 network-scripts]#
创建子接口macvlan
docker network create -d macvlan --subnet=166.66.10.0/24 -o parent=ens37.10 mac_net10
docker network create -d macvlan --subnet=166.66.20.0/24 -o parent=ens37.20 mac_net20
[root@docker1 network-scripts]# docker network create -d macvlan --subnet=166.66.10.0/24 -o parent=ens37.10 mac_net10
1c121ffa87321ae0ffc0f506c910e4a372dc628ec4dd10b10a82fd437fe82262
[root@docker1 network-scripts]# docker network create -d macvlan --subnet=166.66.20.0/24 -o parent=ens37.20 mac_net20
a2e8b07cf461f0d4e795686d38f0d9367fbc8f1534c082acc9ef26cf5392908e
[root@docker1 network-scripts]# docker network ls
NETWORK ID NAME DRIVER SCOPE
ad0eafeaea63 bridge bridge local
d5776b3e90d3 host host local
1c121ffa8732 mac_net10 macvlan local
a2e8b07cf461 mac_net20 macvlan local
ea06123ffce9 my_net bridge local
fea15909e181 none null local
[root@docker1 network-scripts]#
启动虚机并分配IP
docker run -itd --name macvlan10 --ip=166.66.10.41 --network mac_net10 centos
docker run -itd --name macvlan20 --ip=166.66.20.41 --network mac_net20 centos
[root@docker1 network-scripts]# docker run -itd --name macvlan10 --ip=166.66.10.41 --network mac_net10 centos
6d32db88635ab15facf7864c5c2f915cb5afda93490697457b85c428d772ca0e
[root@docker1 network-scripts]# docker run -itd --name macvlan20 --ip=166.66.20.41 --network mac_net20 centos
438b2d2adc3989181ffb679564eb9d030a33ccd62071280e7cf53239e713ca0a
[root@docker1 network-scripts]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
438b2d2adc39 centos "/bin/bash" 3 seconds ago Up 2 seconds macvlan20
6d32db88635a centos "/bin/bash" 8 seconds ago Up 7 seconds macvlan10
[root@docker1 network-scripts]#
查看启动虚机的IP地址
[root@docker1 network-scripts]# docker exec macvlan10 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
21: eth0@if20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:a6:42:0a:29 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 166.66.10.41/24 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:a6ff:fe42:a29/64 scope link
valid_lft forever preferred_lft forever
[root@docker1 network-scripts]# docker exec macvlan20 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
22: eth0@if18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:a6:42:14:2a brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 166.66.20.41/24 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:a6ff:fe42:142a/64 scope link
valid_lft forever preferred_lft forever
[root@docker1 network-scripts]#
Docker2虚机配置
生成网卡子接口配置
配置文件内容如下:
ens37.10子接口配置
cd /etc/sysconfig/network-scripts/
cat ifcfg-ens37.10 ifcfg-ens37.20
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens37.10
DEVICE=ens37.10
ONBOOT=yes
VLAN=yes
ens37.20子接口配置
cd /etc/sysconfig/network-scripts/
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens37.20
DEVICE=ens37.20
ONBOOT=yes
VLAN=yes
启用网卡
[root@docker1 network-scripts]# ifup ens37.10 ens37.20
INFO : [ipv6_wait_tentative] Waiting for interface ens37.10 IPv6 address(es) to leave the 'tentative' state
INFO : [ipv6_wait_tentative] Waiting for interface ens37.10 IPv6 address(es) to leave the 'tentative' state
[root@docker1 network-scripts]#
创建子接口macvlan
docker network create -d macvlan --subnet=166.66.10.0/24 -o parent=ens37.10 mac_net10
docker network create -d macvlan --subnet=166.66.20.0/24 -o parent=ens37.20 mac_net20
[root@docker2 network-scripts]# docker network create -d macvlan --subnet=166.66.10.0/24 -o parent=ens37.10 mac_net10
98d7afec9599fd4fce4dbad4fcccbcd9e8f7d0f0711facd9c78e8372c1098c30
[root@docker2 network-scripts]# docker network create -d macvlan --subnet=166.66.20.0/24 -o parent=ens37.20 mac_net20
1ce3c91470c9c846174eed5b5d8ab7528db41ec4c9ff36a238330c81b26cc306
[root@docker2 network-scripts]# docker network ls
NETWORK ID NAME DRIVER SCOPE
566b523b514b bridge bridge local
2d057d9a3e0d host host local
98d7afec9599 mac_net10 macvlan local
1ce3c91470c9 mac_net20 macvlan local
8f5a595f3fd3 none null local
[root@docker2 network-scripts]#
启动虚机并分配IP
docker run -itd --name macvlan10 --ip=166.66.10.42 --network mac_net10 centos
docker run -itd --name macvlan20 --ip=166.66.20.42 --network mac_net20 centos
[root@docker2 network-scripts]# docker run -itd --name macvlan10 --ip=166.66.10.42 --network mac_net10 centos
4e01f3d7394fd27c144728cd90173dea25148a4c15357bbdf2f4b4abc6e6a30b
[root@docker2 network-scripts]# docker run -itd --name macvlan20 --ip=166.66.20.42 --network mac_net20 centos
ae0b21064547c56d8c045399089b77696976448ce002b376df3854bc1f183afe
[root@docker2 network-scripts]# docker network ls
NETWORK ID NAME DRIVER SCOPE
566b523b514b bridge bridge local
2d057d9a3e0d host host local
98d7afec9599 mac_net10 macvlan local
1ce3c91470c9 mac_net20 macvlan local
8f5a595f3fd3 none null local
[root@docker2 network-scripts]#
查看启动虚机的IP地址
[root@docker2 network-scripts]# docker exec macvlan10 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
15: eth0@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:a6:42:0a:2a brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 166.66.10.42/24 brd 166.66.10.255 scope global eth0
valid_lft forever preferred_lft forever
[root@docker2 network-scripts]# docker exec macvlan20 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
16: eth0@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:a6:42:14:2a brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 166.66.20.42/24 brd 166.66.20.255 scope global eth0
valid_lft forever preferred_lft forever
[root@docker2 network-scripts]#
网络连通性试验
至此,共创建了4个虚机:
- docker1-macvlan10-166.66.10.41
- docker1-macvlan20-166.66.20.41
- docker2-macvlan10-166.66.10.42
- docker2-macvlan20-166.66.20.42
docker1-macvlan10 ping docker2-macvlan10
[root@docker1 network-scripts]# docker exec macvlan10 ping 166.66.10.42 -c 5
PING 166.66.10.42 (166.66.10.42) 56(84) bytes of data.
64 bytes from 166.66.10.42: icmp_seq=1 ttl=64 time=1.08 ms
64 bytes from 166.66.10.42: icmp_seq=2 ttl=64 time=0.779 ms
64 bytes from 166.66.10.42: icmp_seq=3 ttl=64 time=0.756 ms
64 bytes from 166.66.10.42: icmp_seq=4 ttl=64 time=0.762 ms
64 bytes from 166.66.10.42: icmp_seq=5 ttl=64 time=0.750 ms
--- 166.66.10.42 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 16ms
rtt min/avg/max/mdev = 0.750/0.826/1.083/0.128 ms
[root@docker1 network-scripts]#
docker1-macvlan20 ping docker2-macvlan20
[root@docker1 network-scripts]# docker exec macvlan20 ping 166.66.20.42 -c 5
PING 166.66.20.42 (166.66.20.42) 56(84) bytes of data.
64 bytes from 166.66.20.42: icmp_seq=1 ttl=64 time=0.906 ms
64 bytes from 166.66.20.42: icmp_seq=2 ttl=64 time=0.709 ms
64 bytes from 166.66.20.42: icmp_seq=3 ttl=64 time=0.746 ms
64 bytes from 166.66.20.42: icmp_seq=4 ttl=64 time=0.762 ms
64 bytes from 166.66.20.42: icmp_seq=5 ttl=64 time=0.722 ms
--- 166.66.20.42 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 14ms
rtt min/avg/max/mdev = 0.709/0.769/0.906/0.070 ms
[root@docker1 network-scripts]#
docker2-macvlan10 ping docker1-macvlan10
[root@docker2 network-scripts]# docker exec macvlan10 ping 166.66.10.41 -c 5
PING 166.66.10.41 (166.66.10.41) 56(84) bytes of data.
64 bytes from 166.66.10.41: icmp_seq=1 ttl=64 time=0.584 ms
64 bytes from 166.66.10.41: icmp_seq=2 ttl=64 time=0.564 ms
64 bytes from 166.66.10.41: icmp_seq=3 ttl=64 time=0.593 ms
64 bytes from 166.66.10.41: icmp_seq=4 ttl=64 time=0.400 ms
64 bytes from 166.66.10.41: icmp_seq=5 ttl=64 time=0.683 ms
--- 166.66.10.41 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 11ms
rtt min/avg/max/mdev = 0.400/0.564/0.683/0.096 ms
[root@docker2 network-scripts]#
docker2-macvlan20 ping docker1-macvlan20
[root@docker2 network-scripts]# docker exec macvlan20 ping 166.66.20.41 -c 5
PING 166.66.20.41 (166.66.20.41) 56(84) bytes of data.
64 bytes from 166.66.20.41: icmp_seq=1 ttl=64 time=0.642 ms
64 bytes from 166.66.20.41: icmp_seq=2 ttl=64 time=0.581 ms
64 bytes from 166.66.20.41: icmp_seq=3 ttl=64 time=0.593 ms
64 bytes from 166.66.20.41: icmp_seq=4 ttl=64 time=0.579 ms
64 bytes from 166.66.20.41: icmp_seq=5 ttl=64 time=0.517 ms
--- 166.66.20.41 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 11ms
rtt min/avg/max/mdev = 0.517/0.582/0.642/0.045 ms
[root@docker2 network-scripts]#
测试结论
通过测试发现,2个macvlan10(即使跨虚机)之间可以互通,2个macvlan20之间可以互通。但是macvlan10和macvlan20之间不通(即使在同一个虚机)。
不通的原因是由于macvlan10出去的网络包是带vlan 10的tag,而macvlan20出去的包是带vlan20的tag,如果没有三层(路由)做处理的话,是无法相互通讯的。
下面第3个实验就是借助第3台虚机做软路由实现跨网段、跨vlan通讯。
三:网关(路由)模式
通过上面的实验已经实现了容器跨主机访问,但是此时仅仅是二层通讯,对于不同的vlan(比如vlan10和vlan20)之间还不能通,需要借助三层功能才可以实现通讯。本次实验我们使用190.100.100.40这个服务器充当软路由来实现三层功能。
开启ip_forward
首先确保已经开启了ip_forward选项
[root@docker-manager ~]# sysctl -a|grep net.ipv4.ip_forward
net.ipv4.ip_forward = 1
如没打开,可以直接编辑/etc/sysctl.conf
- 写入net.ipv4.ip_forward = 1
- 执行sysctl -p使配置生效
建立子网文件
同样在/etc/sysconfig/network-scripts目录下创建2个子网接口文件
ifcfg-ens37.10
对于此处的IP地址166.66.10.1即为docker1和docker2的vlan10用到的网关
[root@docker-manager network-scripts]# cat ifcfg-ens37.10
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens37.10
DEVICE=ens37.10
ONBOOT=yes
VLAN=yes
IPADDR=166.66.10.1
NETMASK=255.255.255.0
ifcfg-ens37.20
对于此处的IP地址166.66.20.1即为docker1和docker2用到的网关
[root@docker-manager network-scripts]# cat ifcfg-ens37.20
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens37.20
DEVICE=ens37.20
ONBOOT=yes
VLAN=yes
IPADDR=166.66.20.1
NETMASK=255.255.255.0
添加iptables策略
添加如下iptables策略,转发数据包
iptables -t nat -A POSTROUTING -o ens37.10 -j MASQUERADE
iptables -t nat -A POSTROUTING -o ens37.20 -j MASQUERADE
iptables -A FORWARD -i ens37.10 -o ens37.20 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i ens37.20 -o ens37.10 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i ens37.10 -o ens37.20 -j ACCEPT
iptables -A FORWARD -i ens37.20 -o ens37.10 -j ACCEPT
查看nat表规则
[root@docker-manager network-scripts]# iptables -t nat -vnxL
Chain PREROUTING (policy ACCEPT 37 packets, 3401 bytes)
pkts bytes target prot opt in out source destination
2170 124690 DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT 27 packets, 2271 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all -- * !docker0 172.17.0.0/16 0.0.0.0/0
0 0 MASQUERADE tcp -- * * 172.17.0.2 172.17.0.2 tcp dpt:8500
1 84 MASQUERADE all -- * ens37.10 0.0.0.0/0 0.0.0.0/0
7 588 MASQUERADE all -- * ens37.20 0.0.0.0/0 0.0.0.0/0
Chain DOCKER (2 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0
52 2704 DNAT tcp -- !docker0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8500 to:172.17.0.2:8500
[root@docker-manager network-scripts]#
至此就可以实现跨vlan访问了。
更多推荐
所有评论(0)