Docker实现 LVS + Keepalived + bind 二级负载均衡主从模式集群

Docker实现 LVS + Keepalived + bind 二级负载均衡主从模式集群1. 概述本文利用docker快速搭建一个由17个小节点构成的二层负载均衡集群。第一层负载均衡DNS采用Bind实现，第二层负载均衡由LVS实现，同时主从切换用Keepalived来实现。本文主要是验证二层负载均衡+主从切换的方案，由于虚拟机占用资源较大，故采用docker来验证。关于LVS，Keepaliv

xzsww123

2334人浏览 · 2021-01-08 14:13:27

xzsww123 · 2021-01-08 14:13:27 发布

Docker实现 LVS + Keepalived + bind 二级负载均衡主从模式集群

1. 概述

本文利用docker快速搭建一个由17个小节点构成的二层负载均衡集群。第一层负载均衡DNS采用Bind
实现，第二层负载均衡由LVS实现，同时主从切换用Keepalived来实现。

本文主要是验证二层负载均衡+主从切换的方案，由于虚拟机占用资源较大，故采用docker来验证。

关于LVS，Keepalived和bind的相关原理和各自的搭建步骤，可以参考其他的文档。

2. Docker环境安装配置

2.1 宿主机环境安装

yum install -y ipvsadm
ipvsadm

2.2 Docker环境安装配置

2.2.1 安装docker

yum install -y docker

2.1.2 配置Docker镜像的源

打开 /etc/docker

vim /etc/docker/daemon.json

将daemon.json改成如下

{
"registry-mirrors":["https://almtd3fa.mirror.aliyuncs.com"]
}

2.2.3 启动docker

systemctl start docker.service
systemctl enable docker.service # 设置开机启动

2.2.4 登录docker

注册了docker账号后可以直接登录

docker login

3. 搭建集群

3.1 集群节点信息

本集群总共由17个节点构成，具体的地址映射关系如下所示

节点的名称	DIP	VIP
DNS-Server	172.18.0.2	172.18.0.100
----	----	----
DS-1-Master	172.18.1.2	172.18.1.100
DS-1-Slave	172.18.1.3	172.18.1.100
RS-1-Master	172.18.1.4	172.18.1.101
RS-1-Slave	172.18.1.5	172.18.1.101
RS-2-Master	172.18.1.6	172.18.1.102
RS-2-Slave	172.18.1.7	172.18.1.102
RS-3-Master	172.18.1.8	172.18.1.103
RS-3-Slave	172.18.1.9	172.18.1.103
----	----	----
DS-2-Master	172.18.2.2	172.18.2.100
DS-2-Slave	172.18.2.3	172.18.2.100
RS-4-Master	172.18.2.4	172.18.2.101
RS-4-Slave	172.18.2.5	172.18.2.101
RS-5-Master	172.18.2.6	172.18.2.102
RS-5-Slave	172.18.2.7	172.18.2.102
RS-6-Master	172.18.2.8	172.18.2.103
RS-6-Slave	172.18.2.9	172.18.2.103

DS-1和DS-2分别作为两个小集群的网关，每个网关掌握着3个RS节点。

外面的客户端通过DNS-Server获取DS-1和DS-2的地址，并由DS-1和DS-2进行消息的路由和转发。

除了DNS-Server节点外，所有的节点均拥有镜像节点，作为从节点Slave随时顶替主节点master。

整体架构图如下所示
在这里插入图片描述

3.2 制作单个节点的镜像

docker创建自定义网络，这是由于docker设置固定的ip需要用户自己设置subnet，在此例中
172.18.0.0/16是集群的子网。lvs-keepalived-bind-Cluster是集群所用子网的名称。

docker network create --subnet=172.18.0.0/16 lvs-keepalived-bind-Cluster

节点都采用centos7作为基础的镜像，在此基础上不断加入需要的组件。

# 启动DNS-Server
docker run -itd \
  --name DNS-Server \
  --privileged=true \
  --network lvs-keepalived-bind-Cluster \
  --ip 172.18.0.2 \
  --hostname DNS-Server \
  centos:centos7 /usr/sbin/init

进入DNS-Server

docker exec -it DNS-Server /bin/bash

在DNS-Server上进行环境安装

# 必需组件
yum install -y net-tools    # 安装ifconfig
yum install -y keepalived   # 安装keepalived
yum install -y httpd        # 安装apache服务
systemctl restart httpd     # 启动apache服务
systemctl enable httpd      # 开机启动apache服务
echo "I am DNS-Server -VIP: 172.18.0.2 -DIP: 172.18.0.100 " > /var/www/html/index.html

# 个性化设置
sed -i 's/PS1=.*$/PS1="\[\\\[\\e\[32;1m\\\]\\u\\\[\\e\[37;0m\\\]@\\h \\\[\\e\[36;1m\\\]\\w\\\[\\e\[0m\\\]\]\\\\$ \"/g' /etc/bashrc

提交本地docker镜像

# -a :提交的镜像作者；
# -c :使用Dockerfile指令来创建镜像；
# -m :提交时的说明文字；
# -p :在commit时，将容器暂停。
docker commit -a "Kotori" -m "cluster node for lvs + keepalived + bind project" DNS-Server xzsww123/cluster-node:v1.0

提交远端docker镜像

# docker push 注册用户名/镜像名
docker push xzsww123/cluster-node

3.3 启动其他节点

基于DNS-SERVER的镜像，建立并启动其他节点的容器

3.3.1 节点DS-1-Master的设置

############# 节点DS-1-Master的设置 #############
docker run -itd --name DS-1-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.2 --hostname DS-1-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it DS-1-Master /bin/bash

### 设置LVS-DS的相关
echo "I am DS-1-Master -VIP: 172.18.1.100 -DIP: 172.18.1.2" > /var/www/html/index.html
systemctl restart httpd
yum -y install ipvsadm
# 创建虚拟网卡和路由规则的开机脚本
echo "
ifconfig eth0:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
route add -host 172.18.1.100 dev eth0
ipvsadm -C
ipvsadm -A -t 172.18.1.100:80 -s rr
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.101 -g
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.102 -g
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.103 -g
ipvsadm-save
" > /root/lvs-script-ds.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-ds.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-100
}

vrrp_instance VI_1 {  
    state MASTER      
    interface eth0
    mcast_src_ip 172.18.1.2
    virtual_router_id 100
    priority 200     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.1.100
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of DS-1-Master #############

3.3.2 节点DS-1-Slave的设置

############# 节点DS-1-Slave的设置 #############
docker run -itd --name DS-1-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.3 --hostname DS-1-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it DS-1-Slave /bin/bash

### 设置LVS-DS的相关
echo "I am DS-1-Slave -VIP: 172.18.1.100 -DIP: 172.18.1.3" > /var/www/html/index.html
systemctl restart httpd
yum -y install ipvsadm
# 创建虚拟网卡和路由规则的开机脚本
echo "
ifconfig eth0:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
route add -host 172.18.1.100 dev eth0:0
ipvsadm -C
ipvsadm -A -t 172.18.1.100:80 -s rr
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.101 -g
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.102 -g
ipvsadm -a -t 172.18.1.100:80 -r 172.18.1.103 -g
ipvsadm-save
" > /root/lvs-script-ds.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-ds.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-100
}

vrrp_instance VI_1 {  
    state BACKUP
    interface eth0    
    mcast_src_ip 172.18.1.3
    virtual_router_id 100
    priority 100     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.1.100
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of DS-1-Slave #############

3.3.3 节点RS-1-Master的设置

############# 节点RS-1-Master的设置 #############
docker run -itd --name RS-1-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.4 --hostname RS-1-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-1-Master /bin/bash

### 设置LVS-RS的相关
echo "I am RS-1-Master -VIP: 172.18.1.101 -DIP: 172.18.1.4" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-101
}

vrrp_instance VI_1 {  
    state MASTER
    interface eth0    
    mcast_src_ip 172.18.1.4
    virtual_router_id 101
    priority 200     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.1.101
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-1-Master #############

3.3.4 节点RS-1-Slave的设置

############# 节点RS-1-Slave的设置 #############
docker run -itd --name RS-1-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.5 --hostname RS-1-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-1-Slave /bin/bash

### 设置LVS-RS的相关
echo "I am RS-1-Slave -VIP: 172.18.1.101 -DIP: 172.18.1.5" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" > /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-101
}

vrrp_instance VI_1 {  
    state BACKUP
    interface eth0    
    mcast_src_ip 172.18.1.5
    virtual_router_id 101
    priority 100     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.1.101
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-1-Slave #############

3.3.5 节点RS-2-Master的设置

############# 节点RS-2-Master的设置 #############
docker run -itd --name RS-2-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.6 --hostname RS-2-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-2-Master /bin/bash
### 设置LVS-RS的相关
echo "I am RS-2-Master -VIP: 172.18.1.102 -DIP: 172.18.1.6" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-102
}

vrrp_instance VI_1 {  
    state MASTER
    interface eth0    
    mcast_src_ip 172.18.1.6
    virtual_router_id 102
    priority 200     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.1.102
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit    
############# end of RS-2-Master #############

3.3.6 节点RS-2-Slave的设置

############# 节点RS-2-Slave的设置 #############
docker run -itd --name RS-2-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.7 --hostname RS-2-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-2-Slave /bin/bash
### 设置LVS-RS的相关
echo "I am RS-2-Slave -VIP: 172.18.1.102 -DIP: 172.18.1.7" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-102
}

vrrp_instance VI_1 {  
    state BACKUP
    interface eth0    
    mcast_src_ip 172.18.1.7
    virtual_router_id 102
    priority 100     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.1.102
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit    
############# end of RS-2-Slave #############

3.3.7 节点RS-3-Master的设置

############# 节点RS-3-Master的设置 #############
docker run -itd --name RS-3-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.8 --hostname RS-3-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-3-Master /bin/bash
### 设置LVS-RS的相关
echo "I am RS-3-Master -VIP: 172.18.1.103 -DIP: 172.18.1.8" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-103
}

vrrp_instance VI_1 {  
    state MASTER
    interface eth0    
    mcast_src_ip 172.18.1.8
    virtual_router_id 103
    priority 200     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.1.103
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-3-Master #############

3.3.8 节点RS-3-Slave的设置

############# 节点RS-3-Slave的设置 #############
docker run -itd --name RS-3-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.1.9 --hostname RS-3-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-3-Slave /bin/bash
### 设置LVS-RS的相关
echo "I am RS-3-Slave -VIP: 172.18.1.103 -DIP: 172.18.1.9" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.1.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-103
}

vrrp_instance VI_1 {  
    state BACKUP
    interface eth0    
    mcast_src_ip 172.18.1.9
    virtual_router_id 103
    priority 100     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.1.103
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-3-Slave #############

3.3.9 节点DS-2-Master的设置

############# 节点DS-2-Master的设置 #############
docker run -itd --name DS-2-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.2 --hostname DS-2-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it DS-2-Master /bin/bash

### 设置LVS-DS的相关
echo "I am DS-2-Master -VIP: 172.18.2.100 -DIP: 172.18.2.2" > /var/www/html/index.html
systemctl restart httpd
yum -y install ipvsadm
# 创建虚拟网卡和路由规则的开机脚本
echo "
ifconfig eth0:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
route add -host 172.18.2.100 dev eth0
ipvsadm -C
ipvsadm -A -t 172.18.2.100:80 -s rr
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.101 -g
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.102 -g
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.103 -g
ipvsadm-save
" > /root/lvs-script-ds.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-ds.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-100
}

vrrp_instance VI_1 {  
    state MASTER      
    interface eth0
    mcast_src_ip 172.18.2.2
    virtual_router_id 200
    priority 200     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.2.100
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of DS-2-Master #############

3.3.10 节点DS-2-Slave的设置

############# 节点DS-2-Slave的设置 #############
docker run -itd --name DS-2-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.3 --hostname DS-2-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it DS-2-Slave /bin/bash

### 设置LVS-DS的相关
echo "I am DS-2-Slave -VIP: 172.18.2.100 -DIP: 172.18.2.3" > /var/www/html/index.html
systemctl restart httpd
yum -y install ipvsadm
# 创建虚拟网卡和路由规则的开机脚本
echo "
ifconfig eth0:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
route add -host 172.18.2.100 dev eth0:0
ipvsadm -C
ipvsadm -A -t 172.18.2.100:80 -s rr
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.101 -g
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.102 -g
ipvsadm -a -t 172.18.2.100:80 -r 172.18.2.103 -g
ipvsadm-save
" > /root/lvs-script-ds.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-ds.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-100
}

vrrp_instance VI_1 {  
    state BACKUP
    interface eth0    
    mcast_src_ip 172.18.2.3
    virtual_router_id 200
    priority 100     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.2.100
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of DS-2-Slave #############

3.3.11 节点RS-4-Master的设置

############# 节点RS-4-Master的设置 #############
docker run -itd --name RS-4-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.4 --hostname RS-4-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-4-Master /bin/bash

### 设置LVS-RS的相关
echo "I am RS-4-Master -VIP: 172.18.2.101 -DIP: 172.18.2.4" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-101
}

vrrp_instance VI_1 {  
    state MASTER
    interface eth0    
    mcast_src_ip 172.18.2.4
    virtual_router_id 201
    priority 200     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.2.101
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-4-Master #############

3.3.12 节点RS-4-Slave的设置

############# 节点RS-1-Slave的设置 #############
docker run -itd --name RS-4-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.5 --hostname RS-4-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-4-Slave /bin/bash

### 设置LVS-RS的相关
echo "I am RS-4-Slave -VIP: 172.18.2.101 -DIP: 172.18.2.5" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.1.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" > /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-101
}

vrrp_instance VI_1 {  
    state BACKUP
    interface eth0    
    mcast_src_ip 172.18.2.5
    virtual_router_id 201
    priority 100     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.2.101
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-5-Slave #############

3.3.13 节点RS-5-Master的设置

############# 节点RS-2-Master的设置 #############
docker run -itd --name RS-5-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.6 --hostname RS-5-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-5-Master /bin/bash
### 设置LVS-RS的相关
echo "I am RS-5-Master -VIP: 172.18.2.102 -DIP: 172.18.2.6" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-102
}

vrrp_instance VI_1 {  
    state MASTER
    interface eth0    
    mcast_src_ip 172.18.2.6
    virtual_router_id 202
    priority 200     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.2.102
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit    
############# end of RS-5-Master #############

3.3.14 节点RS-5-Slave的设置

############# 节点RS-2-Slave的设置 #############
docker run -itd --name RS-5-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.7 --hostname RS-5-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-5-Slave /bin/bash
### 设置LVS-RS的相关
echo "I am RS-5-Slave -VIP: 172.18.2.102 -DIP: 172.18.2.7" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-102
}

vrrp_instance VI_1 {  
    state BACKUP
    interface eth0    
    mcast_src_ip 172.18.2.7
    virtual_router_id 202
    priority 100     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.2.102
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit    
############# end of RS-5-Slave #############

3.3.15 节点RS-6-Master的设置

############# 节点RS-6-Master的设置 #############
docker run -itd --name RS-6-Master --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.8 --hostname RS-6-Master xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-6-Master /bin/bash
### 设置LVS-RS的相关
echo "I am RS-6-Master -VIP: 172.18.2.103 -DIP: 172.18.2.8" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-103
}

vrrp_instance VI_1 {  
    state MASTER
    interface eth0    
    mcast_src_ip 172.18.2.8
    virtual_router_id 203
    priority 200     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.2.103
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-6-Master #############

3.3.16 节点RS-6-Slave的设置

############# 节点RS-6-Slave的设置 #############
docker run -itd --name RS-6-Slave --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.2.9 --hostname RS-6-Slave xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it RS-6-Slave /bin/bash
### 设置LVS-RS的相关
echo "I am RS-6-Slave -VIP: 172.18.2.103 -DIP: 172.18.2.9" > /var/www/html/index.html
systemctl restart httpd
# 开机时创建虚拟网卡
echo "
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
ifconfig lo:0 172.18.2.100 broadcast 172.18.2.100 netmask 255.255.255.255 up
" > /root/lvs-script-rs.sh
# 将开机脚本加入到开机启动
echo "/bin/sh /root/lvs-script-rs.sh" >> /etc/rc.d/rc.local
# 改变开机脚本权限
chmod +x /etc/rc.d/rc.local
source /etc/rc.d/rc.local

### 设置keepalived的相关
yum install -y keepalived
echo "
global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from keepalived@localhost      
   smtp_server 127.0.0.1    
   smtp_connect_timeout 30  
   router_id vip-103
}

vrrp_instance VI_1 {  
    state BACKUP
    interface eth0    
    mcast_src_ip 172.18.2.9
    virtual_router_id 203
    priority 100     
    advert_int 1     
    authentication {
        auth_type PASS
        auth_pass MrUse
    }
    virtual_ipaddress {
        172.18.2.103
    }
}
" > /etc/keepalived/keepalived.conf
systemctl daemon-reload             #重新加载配置
systemctl start keepalived.service  #启动keepalived服务
systemctl enable keepalived.service #开机启动keepalived服务
systemctl status keepalived.service #查看当前状态
exit
############# end of RS-6-Slave #############

4. 创建DNS服务器容器

直接用已经生成的镜像创建DNS服务器的容器，如果已存在这个容器就先删除

docker run -itd --name DNS-Server --privileged=true --network lvs-keepalived-bind-Cluster --ip 172.18.0.2 --hostname DNS-Server xzsww123/cluster-node:v1.0 /usr/sbin/init
docker exec -it DNS-Server /bin/bash

# 安装和启动bind
yum install -y bind bind-utils bind-devel bind-libs bind-chroot
systemctl start named
systemctl enable named
systemctl status named

# 修改域主服务器设置 /etc/named.conf
vi /etc/named.conf

# 改成如下
options {
	listen-on port 53 { 172.18.0.2; };
	listen-on-v6 port 53 { ::1; };
  rrset-order { order random; };
	directory "/var/named";
	dump-file "/var/named/data/cache_dump.db";
	statistics-file "/var/named/data/named_stats.txt";
	memstatistics-file "/var/named/data/named_mem_stats.txt";
	recursing-file  "/var/named/data/named.recursing";
	secroots-file   "/var/named/data/named.secroots";
	allow-query { any; };                 
	recursion yes;  
	dnssec-enable yes;
	dnssec-validation yes;
	bindkeys-file "/etc/named.root.key"; #大大
	managed-keys-directory "/var/named/dynamic";
	pid-file "/run/named/named.pid";
	session-keyfile "/run/named/session.key";
};

logging {
        channel default_debug {
                file "data/named.run";
                severity dynamic;
        };
};

zone "." IN {
	type hint;
	file "named.ca";
};

include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";


# 修改区域配置文件设置 /etc/named.rfc1912.zones
vi /etc/named.rfc1912.zones

# 添加如下代码
zone "Odyssey.com" IN {
type master;           
file "Odyssey.com.zone";
};

# 创建自己的zone文件
vi  /var/named/Odyssey.com.zone
# 改成如下
$TTL 600
@   IN  SOA Odyssey.com.   root.qq.com(
                            1811201955   
                            1H           
                            1M          
                            1W           
                            10M)         

@   IN  NS  ns1
ns1 IN A 172.18.0.2

@   IN  A   172.18.1.100
    IN  A   172.18.2.100

www IN  A   172.18.1.100
www IN  A   172.18.2.100

*   IN  A   255.255.255.255

# 重新加载服务
rndc reload

5. 整体方案测试

以上集群搭建完毕，接下来进行简单的测试

先在宿主机上安装nscd，用来刷新DNS缓存

yum install -y nscd

先在宿主机上改变dns （最好断掉外网，因为外网上也有Odyssey.com这个域名）

echo "
DNS1=172.18.0.2
" >> /etc/sysconfig/network-scripts/ifcfg-ens33

service network restart

然后执行

host -t a www.Odyssey.com 172.18.0.2

得到如下结果

Using domain server:
Name: 172.18.0.2
Address: 172.18.0.2#53
Aliases:

www.Odyssey.com has address 172.18.1.100
www.Odyssey.com has address 172.18.2.100

然后直接访问www.Odyssey.com

curl www.Odyssey.com

可以看到访问请求轮流在RS间循环，但是DNS轮询机制会受到多方面的影响，如：A记录的TTL时间长
短的影响；别的 DNS 服务器 Cache 的影响；windows 客户端也有一个DNS Cache。这些都会影响
DNS轮询的效果。因此 DNS 的轮询机制并不能做为一个 load balancing的解决方案，只能作为一
个 load distribution 方案。因此执行上面的指令未必会切换DS服务器，这是根据每台服务器上的
DNS过期时间等都有关系。

附录

docker stop $(docker ps -a | grep xzsww123 | awk '{print $1 }')   #停止所有已经退出的容器
docker rm $(docker ps -a | grep xzsww123 | awk '{print $1 }')     #删除所有已经退出的容器

参考资料

[1] LVS负载均衡之DR模式部署

[2] keepalived 配置文件解析

[3] Linux运维实战之DNS（bind）服务器的安装与配置

[4] linux实现DNS轮询实现负载平衡

向您推荐>>Eolink开发者社区

权威｜前沿｜技术｜干货｜国内首个API全生命周期开发者社区

更多推荐

ELK实现containerd的容器日志采集展示【基于logging的全栈监测】

企业级ELK Stack构建介绍

云原生

深入理解 Mocha 测试框架：从零实现一个 Mocha

前言什么是自动化测试自动化测试在很多团队中都是Devops环节中很难执行起来的一个环节，主要原因在于测试代码的编写工作很难抽象，99%的场景都需要和业务强绑定，而且写测试代码的编写工作量往往比编写实际业务代码的工作量更多。在一些很多业务场景中投入产出比很低，适合写自动化测试的应该是那些中长期业务以及一些诸如组件一样的基础库。自动化测试是个比较大的概念，其中分类也比较多，比如单元测试，端对端测试，集

云原生

(20200916 Solved)docker-compose up创建容器自动退出

问题描述如题，创建容器后自动退出了。并且docker start container无效解决方案原因是缺失了控制终端的配置，需要在docker-compose.yml中增加tty:true ，有时候这样也不行，需要再增加一个command:/bin/bash，命令不一定是这个，需要是一个不会退出的命令，然后用-d后台启动容器。Referencesdocker-compose启动容器后自动退出...

云原生

所有评论(0)

查看更多评论

xzsww123

@xzsww123

已为社区贡献1条内容