TCP长连接在k8s中基于nginx负载均衡
TCP长连接在k8s中基于nginx负载均衡
nodeport其实也是一种实现方式,但是nodeport是有一些缺陷的:
第一:占用worker主机的port,项目多,团队多,容易port冲突
第二:如果pod没有落到workerA上,但是请求是从workerA发起,当workerA宕机,请求就没法转发。
使用nginx ingress+k8s service去服务发现和负载均衡,算是比较合理的操作。(还是会存在上面第二个问题,需要在集群外的ngxin+keepalive去做高可用)
集群环境需要修改一点东西:
最新版的k8s ingress-nginx默认集群安装的时候是不带tcp和udp配置映射的。需要自己准备。
准备:tcp-services-configmap.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: kube-system
data:
17000: "default/echoserver:4040" #测试的tcpserver服务名称和对外端口
准备:udp-services-configmap.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: kube-system
通过集群导入yaml功能导入
修改rke2-ingress-nginx-controller的入口参数:
编辑并添加:
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
保存配置,等待ingress-nginx重启
项目准备:
使用supersocket 写了一个简单的tpc服务控制台程序:当客户端发送消息过来时,就返回当前服务ip
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using SuperSocket;
using SuperSocket.ProtoBase;
using System;
using System.Linq;
using System.Net;
using System.Net.NetworkInformation;
using System.Net.Sockets;
using System.Text;
using System.Threading.Tasks;
namespace EchoServer
{
internal class Program
{
static async Task Main(string[] args)
{
var host = SuperSocketHostBuilder.Create<TextPackageInfo, LinePipelineFilter>(args)
.UsePackageHandler(async (s, p) =>
{
await s.SendAsync(Encoding.UTF8.GetBytes(GetIP() + "\r\n"));
})
.ConfigureSuperSocket(options =>
{
options.Name = "Echo Server";
options.AddListener(new ListenOptions
{
Ip = "Any",
Port = 4040
}
);
})
.ConfigureLogging((hostCtx, loggingBuilder) =>
{
loggingBuilder.AddConsole();
})
.Build();
await host.RunAsync();
}
private static string GetIP()
{
string localIp = NetworkInterface.GetAllNetworkInterfaces()
.Select(p => p.GetIPProperties())
.SelectMany(p => p.UnicastAddresses)
.FirstOrDefault(p => p.Address.AddressFamily == AddressFamily.InterNetwork && !IPAddress.IsLoopback(p.Address))?.Address.ToString();
return localIp;
}
}
}
准备流水线文件:.rancher-pipeline.yml
stages:
- name: build
steps:
- publishImageConfig:
dockerfilePath: ./EchoServer/Dockerfile
buildContext: .
tag: 192.168.21.4:8081/test/echoserver:${CICD_GIT_BRANCH}-${CICD_GIT_COMMIT}
pushRemote: true
registry: 192.168.21.4:8081
env:
PLUGIN_DEBUG: "true"
PLUGIN_INSECURE: "true"
- name: deploy
steps:
- applyYamlConfig:
path: ./deployment.yaml
timeout: 60
notification: {}
准备部署文件:deployment.yaml 与之前项目不同的地方已经注释出来了
apiVersion: v1
kind: Service
metadata:
name: echoserver
namespace: default
labels:
app: echoserver
service: echoserver
spec:
ports:
- port: 4040 #容器端口
targetPort: 4040 #暴露对外端口
protocol: TCP #tcp协议
name: tcp-rt
selector:
app: echoserver
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: echoserver
namespace: default
labels:
app: echoserver
version: v1
spec:
replicas: 2
selector:
matchLabels:
app: echoserver
version: v1
template:
metadata:
labels:
app: echoserver
version: v1
spec:
containers:
- name: echoserver
image: 192.168.21.4:8081/test/echoserver:${CICD_GIT_BRANCH}-${CICD_GIT_COMMIT}
readinessProbe:
tcpSocket: #监控检查用tcp
port: 4040
initialDelaySeconds: 10
periodSeconds: 5
ports:
- containerPort: 4040
imagePullSecrets:
- name: mydockerhub
在流水线中生成和部署。
准备客户端:(也是基于supersocket写的客户端控制台程序,连接到服务端后循环发消息,并打印收到的服务消息)注意连接用的ip和端口。ip就是任意一个worker节点,端口则是tcp-services-configmap.yaml中的17000
using SuperSocket.Client;
using SuperSocket.ProtoBase;
using System;
using System.Net;
using System.Text;
using System.Threading.Tasks;
namespace EchoClient
{
internal class Program
{
static async Task Main(string[] args)
{
var client = new EasyClient<TextPackageInfo>(new LinePipelineFilter()).AsClient();
if (!await client.ConnectAsync(new IPEndPoint(IPAddress.Parse("192.168.21.234"), 17000)))
{
Console.WriteLine("Failed to connect the target server.");
return;
}
_ = Task.Run(async () =>
{
while (true)
{
await client.SendAsync(Encoding.UTF8.GetBytes("hello" + "\r\n"));
await Task.Delay(2000);
}
});
while (true)
{
var p = await client.ReceiveAsync();
if (p == null) // connection dropped
break;
Console.WriteLine(p.Text);
}
}
}
}
启动多个客户端:
更多推荐
所有评论(0)