我目前再RedHat工作,负责OpenShift 的测试工作,分享下OpenShift的Router是怎么工作的
首先说下k8s的service,大家都知道,k8s proxy 进程的目的是做service->pod的代理,支持两种方式,一种是在node上添加ip table rule, 为每一个service 启动一个tcp port 做监听,在node上写一条ip table rule ,将service Ip 的请求nat 到kube-proxy 为其代理的端口,然后代理请求servive的IP 通过一定算法将请求转发到Pod ,一种是直接将到Service ip的请求转发到pod, 通过iptable 的方式,为每一个endpoint添加ip table.
要想通过域名解析的方式访问pod的话,可以在DNS中将域名解析到service ip.
openshift 的解决方案是利用haproxy 作为router,在每一个Node上启动一个haproxy 的pod,监听53端口,在DNS 服务器中将所有的域名解析请求转发到router pod 的node上,然后router pod,也就是haproxy再将请求转发到pod,这个过程完全无需人工操作。大概步骤如下:
1. 创建pod
2. 创建service
3.创建route,没个route都对应一个service,对应一个域名,可以直接从DeploymentConfig expose出来: oc expose dc/<dc_name>
4.openshift router controller,当然这个controller 运行在 haproxy的pod上,会监听service endpoint(service 对应的POD)的变化动态的修改haproxy 配置,当新创建一个route的时候,会再haproxy中添加对应的配置,当service endpoint增加或者删除或者状态不是ready的时候会修改ha-proxy配置以确保转发到健康的pod上面。
5. 目前每一个ha-proxy的配置文件都包括全网的域名解析的配置,有
user story
在做 route sharding,某一个router pod只解析某部分域名,从而减小route的配置文件的大小和满足不同域名需要不同的解析方式的需求
6. 当然,每一次pod的增减都去跟改配置和刷新haproxy配置文件会频繁的刷新haproxy 配置,目前也支持延迟刷新的功能,在一个周期内去刷新一次。
7. openshift不仅支持haproxy router, 还支持h5,详细请参考
https://docs.openshift.org/lat ... .html
8.更多route相关新功能请参考
https://trello.com/b/TV5P9gKe/networking
以下为一些resource的template 定义,贴到这作为参考。
[vagrant@ose ~]$ oc get service nodejs-example -o json
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "nodejs-example",
"namespace": "haowang",
"selfLink": "/api/v1/namespaces/haowang/services/nodejs-example",
"uid": "1f832486-ef2e-11e5-bcc1-fa163efe3ad5",
"resourceVersion": "43606",
"creationTimestamp": "2016-03-21T06:28:28Z",
"labels": {
"template": "nodejs-example"
},
"annotations": {
"description": "Exposes and load balances the application pods"
}
},
"spec": {
"ports": [
{
"name": "web",
"protocol": "TCP",
"port": 8080,
"targetPort": 8080
}
],
"selector": {
"name": "nodejs-example"
},
"portalIP": "172.31.174.255",
"clusterIP": "172.31.174.255",
"type": "ClusterIP",
"sessionAffinity": "None"
},
"status": {
"loadBalancer": {}
}
}
[vagrant@ose ~]$ oc get route -o json
{
"kind": "List",
"apiVersion": "v1",
"metadata": {},
"items": [
{
"kind": "Route",
"apiVersion": "v1",
"metadata": {
"name": "nodejs-example",
"namespace": "haowang",
"selfLink": "/oapi/v1/namespaces/haowang/routes/nodejs-example",
"uid": "1f80762b-ef2e-11e5-bcc1-fa163efe3ad5",
"resourceVersion": "43607",
"creationTimestamp": "2016-03-21T06:28:28Z",
"labels": {
"template": "nodejs-example"
},
"annotations": {
"openshift.io/host.generated": "true"
}
},
"spec": {
"host": "nodejs-example-haowang.0319-j7r.qe.rhcloud.com",
"to": {
"kind": "Service",
"name": "nodejs-example"
}
},
"status": {
"ingress": [
{
"host": "nodejs-example-haowang.0319-j7r.qe.rhcloud.com",
"routerName": "router",
"conditions": [
{
"type": "Admitted",
"status": "True",
"lastTransitionTime": "2016-03-21T06:28:28Z"
}
]
}
]
}
}
]
}
[vagrant@ose ~]$ oc get endpoints -o json
{
"kind": "List",
"apiVersion": "v1",
"metadata": {},
"items": [
{
"kind": "Endpoints",
"apiVersion": "v1",
"metadata": {
"name": "nodejs-example",
"namespace": "haowang",
"selfLink": "/api/v1/namespaces/haowang/endpoints/nodejs-example",
"uid": "1f8d0087-ef2e-11e5-bcc1-fa163efe3ad5",
"resourceVersion": "44576",
"creationTimestamp": "2016-03-21T06:28:28Z",
"labels": {
"template": "nodejs-example"
}
},
"subsets": [
{
"addresses": [
{
"ip": "10.2.2.18",
"targetRef": {
"kind": "Pod",
"namespace": "haowang",
"name": "nodejs-example-2-qagll",
"uid": "a73aedef-ef2f-11e5-bcc1-fa163efe3ad5",
"resourceVersion": "44570"
}
}
],
"ports": [
{
"name": "web",
"port": 8080,
"protocol": "TCP"
}
]
}
]
}
]
}
所有评论(0)