1、机器规划

1.1 机器分配

文档是测试，使用了两台机器，实际可以是两台或者三台机器

IP	功能	配置	说明
10.11.123.11	squid	4C8G
10.11.123.12	squid	4C8G
10.11.123.13	Keepalived	2C4G
10.11.123.14	Keepalived	2C4G
10.11.123.15	Vip

1.2 前置操作

所有机器都操作

1.2.1 关闭防火墙

systemctl stop firewalld ; systemctl disable firewalld
setenforce 0
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

1.2.2 关闭 swap 分区

swapoff -a && sysctl -w vm.swappiness=0
sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab

1.2.3 安装命令

yum install net-tools -y

2、keepalived安装

2.1 master节点安装

10.11.123.13机器

2.1.1 软件安装

yum install -y keepalived

2.1.2 配置修改

! Configuration File for keepalived
global_defs {
   notification_email {
     testemail@qq.com
   }
   notification_email_from testemail@qq.com
   smtp_server 172.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER
    interface ens192
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.11.123.15
    }
}

virtual_server 10.11.123.15 3128 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 10.11.123.11 3128 {
        weight 1
        #SSL_GET {
        #    url {
        #      path /
        #      digest ff20ad2481f97b1754ef3e12ecd3a9cc
        #    }
        #    url {
        #      path /mrtg/
        #      digest 9b3a0c85a887a256d6939da88aabd8cd
        #    }
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3128
        }
    }

    real_server 10.11.123.12 3128 {
        weight 1 
        #SSL_GET {
        #    url {
        #      path / 
        #      digest ff20ad2481f97b1754ef3e12ecd3a9cc
        #    }
        #    url {
        #      path /mrtg/
        #      digest 9b3a0c85a887a256d6939da88aabd8cd
        #    }
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3128
        }
    }
}

2.1.3 启动服务

systemctl start keepalived
systemctl enable keepalived
systemctl restart keepalived
systemctl status keepalived

2.2 backup节点安装

10.11.123.14机器

2.2.1 软件安装

yum install -y curl gcc openssl-devel libnl3-devel net-snmp-devel
yum install -y keepalived

2.2.2 配置修改

! Configuration File for keepalived
global_defs {
   notification_email {
     testemail@qq.com
   }
   notification_email_from testemail@qq.com
   smtp_server 172.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens192
    virtual_router_id 51
    priority 99
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.11.123.15
    }
}

virtual_server 10.11.123.15 3128 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 10.11.123.11 3128 {
        weight 1
        #SSL_GET {
        #    url {
        #      path /
        #      digest ff20ad2481f97b1754ef3e12ecd3a9cc
        #    }
        #    url {
        #      path /mrtg/
        #      digest 9b3a0c85a887a256d6939da88aabd8cd
        #    }
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3128
        }
    }

    real_server 10.11.123.12 3128 {
        weight 1 
        #SSL_GET {
        #    url {
        #      path / 
        #      digest ff20ad2481f97b1754ef3e12ecd3a9cc
        #    }
        #    url {
        #      path /mrtg/
        #      digest 9b3a0c85a887a256d6939da88aabd8cd
        #    }
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 3128
        }
    }
}

2.2.3 启动服务

systemctl start keepalived
systemctl enable keepalived
systemctl restart keepalived
systemctl status keepalived

3、代理服务安装

3.1 安装squid

两个节点安装方式一样,这里使用Squid,使用yum方式安装Squid

第一步：安装
yum install -y squid

第二步：修改代理端口
vim /etc/squid/squid.conf
acl localnet src 192.168.0.0/16 #配置运行访问代理服务器的源
http_port 3128 #设置代理服务器监听端口，默认3128，建议修改，如果启用了防火墙需要开放监听端口的安全策略。

第三步：查看服务端口
netstat -tnpl |grep 3128

第四步：启动代理服务
systemctl start squid && systemctl enable squid 

3.2 配置vip

10.11.123.11和10.11.123.12都进行配置

vim /etc/init.d/realserver
chmod 755 /etc/init.d/realserver
service realserver start

realserver文件

SNS_VIP=10.11.123.15
/etc/rc.d/init.d/functions
case "$1" in
start)
       ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
       /sbin/route add -host $SNS_VIP dev lo:0
       echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
       sysctl -p >/dev/null 2>&1
       echo "RealServer Start OK"
       ;;
stop)
       ifconfig lo:0 down
       route del $SNS_VIP >/dev/null 2>&1
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
       echo "RealServer Stoped"
       ;;
*)
       echo "Usage: $0 {start|stop}"
       exit 1
esac
exit 0

4、应用容器配置

4.1 配置环境变量

创建工作负载的时候设置环境变量，下面使用nginx为例

ip为keepalived的ip，端口为squid的端口

4.2 验证代理

查看pod的环境变量

环境变量中显示：

http_proxy=http://10.11.123.15:3128

https_proxy=https://10.11.123.15:3128

测试访问外网是否走代理

状态码为200，并且走代理，返回界面

4.3 验证代理高可用

1、curl baidu.com看返回结果

2、停止一台机器的squid服务，在curl看下结果

3、结论

当2台中的一台squid停止服务，keepalived会进行健康检查，发现服务不通回调用另一个机器的服务，实现高可用。