原生环境kubernetes ingress与Citrix Netscaler VPX集成+Citrix IPAM Controller实现VIP自动获取+域名A记录自动更新
·
环境:
k8s v1.36
netscaler vpx 14.1
1、配置VPX ADC ip地址
Configuration-System-Network-IPs中添加subnet ip sip是与后端通信的地址，无需添加vip，由citrix IPAM controller自动分配
在netscaler命令行中指定ip地址提供域名解析，之后客户端或其他dns服务器将指定域的请求转发到该地址可以解析ingress域名
add service adns_svc 192.168.255.111 ADNS 532、通过k8s secret 创建 citrix netscaler 管理凭证 该凭证需要有对netscaler 的api访问权限，将<username> <password>替换为实际用户名密码
kubectl create secret generic nslogin --from-literal=username=<username> --from-literal=password=<password>3、部署Netscaler Ingress Controller
下载清单，helm部署请查看官方ingress controller部署文档https://docs.netscaler.com/en-us/netscaler-k8s-ingress-controller/deploy-nsic
curl -Lo netscaler-ingress-controller.yaml https://raw.githubusercontent.com/netscaler/netscaler-k8s-ingress-controller/master/deployment/baremetal/citrix-k8s-ingress-controller.yaml修改清单文件deployment中 vpx ip地址参数
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: cic-k8s-role
rules:
- apiGroups: [""]
resources: ["endpoints", "pods", "secrets", "nodes", "routes", "namespaces", "configmaps", "services"]
verbs: ["get", "list", "watch"]
# services/status is needed to update the loadbalancer IP in service status for integrating
# service of type LoadBalancer with external-dns
- apiGroups: [""]
resources: ["services/status"]
verbs: ["patch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create"]
- apiGroups: ["extensions"]
resources: ["ingresses", "ingresses/status"]
verbs: ["get", "list", "watch", "patch"]
- apiGroups: ["networking.k8s.io"]
resources: ["ingresses", "ingresses/status", "ingressclasses"]
verbs: ["get", "list", "watch", "patch"]
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "list", "watch"]
- apiGroups: ["citrix.com"]
resources: ["rewritepolicies", "authpolicies", "ratelimits", "listeners", "httproutes", "continuousdeployments", "apigatewaypolicies", "wafs", "bots", "corspolicies", "appqoepolicies"]
verbs: ["get", "list", "watch", "create", "delete", "patch"]
- apiGroups: ["citrix.com"]
resources: ["rewritepolicies/status", "continuousdeployments/status", "authpolicies/status", "ratelimits/status", "listeners/status", "httproutes/status", "wafs/status", "apigatewaypolicies/status", "bots/status", "corspolicies/status", "appqoepolicies/status"]
verbs: ["patch"]
- apiGroups: ["citrix.com"]
resources: ["vips"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: ["route.openshift.io"]
resources: ["routes"]
verbs: ["get", "list", "watch"]
- apiGroups: ["config.openshift.io"]
resources: ["networks"]
verbs: ["get", "list"]
- apiGroups: ["network.openshift.io"]
resources: ["hostsubnets"]
verbs: ["get", "list", "watch"]
- apiGroups: ["crd.projectcalico.org"]
resources: ["ipamblocks"]
verbs: ["get", "list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: cic-k8s-role
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cic-k8s-role
subjects:
- kind: ServiceAccount
name: cic-k8s-role
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: cic-k8s-ingress-controller
spec:
selector:
matchLabels:
app: cic-k8s-ingress-controller
replicas: 1
template:
metadata:
name: cic-k8s-ingress-controller
labels:
app: cic-k8s-ingress-controller
annotations:
spec:
serviceAccountName: cic-k8s-role
containers:
- name: cic-k8s-ingress-controller
image: "quay.io/netscaler/netscaler-k8s-ingress-controller:4.0.16"
env:
# Set NetScaler NSIP/SNIP, SNIP in case of HA (mgmt has to be enabled)
- name: "NS_IP"
value: "X.X.X.X" #配置为vpx的web管理地址
# Set username for Nitro
- name: "NS_USER"
valueFrom:
secretKeyRef:
name: nslogin
key: username
- name: "LOGLEVEL"
value: "INFO"
# Set user password for Nitro
- name: "NS_PASSWORD"
valueFrom:
secretKeyRef:
name: nslogin
key: password
# Set log level
- name: "EULA"
value: "yes"
- name: NS_CONFIG_DNS_REC #添加该键值对实现ingress或service的vip自动在netscaler dns中的记录自动更新
value: "true"
args:
- --ingress-classes
citrix #要和之后创建的ingressclass名称一致
- --feature-node-watch #该项为true，可以根据pod变化添加静态路由，本案例中service使用nodeport netscaler不需要和pod通信
false
- --NS-CONFIG-SAVE-INTERVAL #可以添加该参数控制器定时向vpx api发送保存配置请求
30
- --update-ingress-status #添加该参数实现ingress将ip写回到ADDRESS字段 需要通过该字段更新dns a记录
yes
- --ipam
citrix-ipam-controller #IPAM控制器的deployment名称
imagePullPolicy: Always部署vpx ingress controller
kubectl apply -f netscaler-ingress-controller.yaml部署后观察pod启动日志，ingress controller已经可以和vpx api交互了
[root@k8s-master1 ~]# kubectl logs deploy/cic-k8s-ingress-controller
User has accepted EULA. Starting Triton
The default port for Citrix ingress controller to communicate with Citrix ADC has been changed from 80 to 443 and the protocol has been changed from HTTP to HTTPS
Citrix Ingress Controller version: 4.0.16, build: Fri Mar 13 09:32:58 IST 2026 B:ce3d1216a5e863a2865d6adc6534b39d7fd45728
2026-05-03 07:36:53,182 - INFO - [nstriton.py:initalize_k8s_client:366] (MainThread) Initialize the k8s client interface and k8sclienthelper
2026-05-03 07:36:53,186 - INFO - [client.py:__init__:233] (MainThread) Kubernetes parameters:
2026-05-03 07:36:53,187 - INFO - [client.py:__init__:234] (MainThread) Cluster:
2026-05-03 07:36:53,187 - INFO - [client.py:__init__:235] (MainThread) Kube API Server URL: https://10.96.0.1:443
2026-05-03 07:36:53,188 - INFO - [client.py:__init__:240] (MainThread) Setting Connect_timeout to 5 sec and Read_timeout to 10 sec for all requests methods.
2026-05-03 07:36:53,189 - INFO - [nstriton.py:main:913] (MainThread) configured NS_APPS_NAME_PREFIX as "k8s-"
2026-05-03 07:36:53,189 - INFO - [nstriton.py:main:932] (MainThread) configured NS_MC_PREFIX as "mc-"
2026-05-03 07:36:53,189 - INFO - [nstriton.py:main:965] (MainThread) ==============> Netscaler Ingress Controller with RBAC scope: cluster <================
2026-05-03 07:36:53,189 - INFO - [nstriton.py:main:992] (MainThread) NS is running as standalone
2026-05-03 07:36:53,190 - INFO - [nstriton.py:main:1002] (MainThread) Default VIP is $NS-VIP$. NS IP is 192.168.255.110
2026-05-03 07:36:53,485 - INFO - [nstriton.py:main:1083] (MainThread) Desired service state is configured as False
2026-05-03 07:36:55,309 - INFO - [baseconfigmapvar.py:_load_data:60] (MainThread) Data loaded successful for LOGLEVEL config variable value: info
2026-05-03 07:36:55,316 - INFO - [baseconfigmapvar.py:_load_data:60] (MainThread) Data loaded successful for JSONLOG config variable value: false
2026-05-03 07:36:55,324 - INFO - [baseconfigmapvar.py:_load_data:60] (MainThread) Data loaded successful for NS_PROTOCOL config variable value: https
2026-05-03 07:36:55,328 - INFO - [baseconfigmapvar.py:_load_data:60] (MainThread) Data loaded successful for NS_PORT config variable value: 443
2026-05-03 07:36:55,338 - INFO - [baseconfigmapvar.py:_load_data:60] (MainThread) Data loaded successful for IGNORE_NODE_EXTERNAL_IP config variable value: false
2026-05-03 07:36:55,348 - INFO - [baseconfigmapvar.py:_load_data:60] (MainThread) Data loaded successful for POD_IPS_FOR_SERVICEGROUP_MEMBERS config variable value: false
2026-05-03 07:36:55,352 - INFO - [baseconfigmapvar.py:_load_data:60] (MainThread) Data loaded successful for NS_CONFIG_DNS_REC config variable value: false
2026-05-03 07:36:55,362 - INFO - [nitrointerface.py:register_ns_cfg_vars:1675] (MainThread) register_ns_cfg_vars: CIC session variables updated successfully
2026-05-03 07:36:55,369 - INFO - [nitrointerface.py:apply_session_vars:1644] (MainThread) CIC session variables ns_protocol:https ns_port:443 updated successfully
2026-05-03 07:36:55,371 - INFO - [nitrointerface.py:__init__:1546] (MainThread) Netscaler ADC ip address 192.168.255.110, NS vip = $NS-VIP$ , Lbrole = server User nsroot Password *******
2026-05-03 07:36:55,372 - INFO - [config_dispatcher.py:run:192] (Dispatcher) Starting Dispatcher thread
2026-05-03 07:36:55,578 - INFO - [kubeeventwriter.py:write:119] (MainThread) CIC Event: CONNECTED: Citrix ADC:192.168.255.110:443
2026-05-03 07:36:55,946 - INFO - [nitrointerface.py:log_ns_info:1621] (MainThread) Netscaler ADC parameters:
2026-05-03 07:36:55,947 - INFO - [nitrointerface.py:log_ns_info:1622] (MainThread) ADC URL: https://192.168.255.110:443
2026-05-03 07:36:55,947 - INFO - [nitrointerface.py:log_ns_info:1623] (MainThread) ADC Entity Prefix is set: k8s-
2026-05-03 07:36:55,948 - INFO - [nitrointerface.py:log_ns_info:1624] (MainThread) Embedded ADC: False
2026-05-03 07:36:55,948 - INFO - [nitrointerface.py:log_ns_info:1625] (MainThread) Enable Certificate Validation: False
2026-05-03 07:36:55,949 - INFO - [nitrointerface.py:log_ns_info:1626] (MainThread) Monitor Netscaler ADC: auto
2026-05-03 07:36:55,949 - INFO - [nitrointerface.py:log_ns_info:1627] (MainThread) Monitor ADC probe frequency: 10
2026-05-03 07:36:56,087 - INFO - [nitrointerface.py:is_cert_bundle_supported:10489] (MainThread) ADC version major:14, minor:1, mr_major:43, mr_minor:50.
2026-05-03 07:36:56,202 - INFO - [nitrointerface.py:enable_ns_features:8277] (MainThread) ADC version major:14, minor:1, mr_major:43, mr_minor:50.
为citrix ingress controller创建ingress class
cat > netscaler-ingressclass.yaml << EOF
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
name: citrix
spec:
controller: citrix.com/ingress-controller
EOF
#应用清单文件
kubectl apply -f netscaler-ingressclass.yaml
#查看ingress
[root@k8s-master1 ~]# kubectl get ingressclass
NAME CONTROLLER PARAMETERS AGE
citrix citrix.com/ingress-controller <none> 38h4、部署Citrix IPAM Controller
创建VIP CRD
kubectl create -f https://raw.githubusercontent.com/netscaler/netscaler-k8s-ingress-controller/master/crd/vip/vip.yaml下载Citrix IPAM Controller清单
curl -Lo citrix-ipam-controller.yaml https://raw.githubusercontent.com/netscaler/netscaler-k8s-ingress-controller/master/example/ipam-controller/citrix-ipam-controller.yaml修改VIP_RANGE环境变量
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: citrix-ipam-controller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: citrix-ipam-controller
rules:
- apiGroups:
- citrix.com
resources:
- vips
verbs:
- '*'
- apiGroups:
- apiextensions.k8s.io
resources:
- customresourcedefinitions
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: citrix-ipam-controller
subjects:
- kind: ServiceAccount
name: citrix-ipam-controller
namespace: kube-system
roleRef:
kind: ClusterRole
apiGroup: rbac.authorization.k8s.io
name: citrix-ipam-controller
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: citrix-ipam-controller
name: citrix-ipam-controller
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: citrix-ipam-controller
template:
metadata:
labels:
app: citrix-ipam-controller
spec:
serviceAccountName: citrix-ipam-controller
containers:
- image: quay.io/netscaler/netscaler-ipam-controller:2.1.2
imagePullPolicy: IfNotPresent
name: citrix-ipam-controller
env:
# This IPAM controller takes environment variable VIP_RANGE. IPs in this range are used to assign values for IP range
- name: VIP_RANGE
value: '[{"ingress-range": ["192.168.255.130-192.168.255.150"]}]' #配置ip范围 值的字典键名也就是ingress-range是被ingress注解中键ingress.citrix.com/ipam-range:引用的5、部署ingress、service和deployment测试ingress是否正常
cat > test.yaml << EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
ingress.citrix.com/ipam-range: "ingress-range" #使用ipam分配ingress的vip地址，值需要与citrix ipam controller中变量VIP_RANGE值中的字典键一致
spec:
ingressClassName: citrix #指向刚才创建的ingressclass名称
rules:
- host: test.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: test-service
port:
number: 80
---
apiVersion: v1
kind: Service
metadata:
name: test-service
spec:
selector:
app: test-deploy
ports:
- name: http
port: 80
targetPort: 80
nodePort: 30080
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-deploy
spec:
replicas: 2
selector:
matchLabels:
app: test-deploy
template:
metadata:
labels:
app: test-deploy
spec:
containers:
- name: nginx
image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/nginx:latest
ports:
- containerPort: 80
EOFcurl命令测试对ingress是否转发到后端pod
[root@k8s-master1 ~]# curl http://test.example.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
citrix ingress controller在vpx adc中创建了servicegroup对象对应了k8s endpoint对象
load balancing virtual servers对应k8s service对象
content swithching virtual servers对应k8s ingress对象
更多推荐
7
0- 0
已为社区贡献2条内容
所有评论(0)