kubernetes中ingress控制器traefik获取真实客户源IP
traefik获取真实客户源IP
·
一.现象
公司kubenetes生产环境使用的ingress控制器是traefik，默认是通过deployment部署的，现在研发上反馈不能获取客户的真实源IP地址，通过x_forward_for获取的IP地址都是kubernetes集群内部的IP地址。
二.解决思路
通过查找traefik的官方文档
Traefik EntryPoints Documentation - Traefik
traefik传输客户源地址到后端需要配置Forwarded Headers参数，用于x_forwarded_for保存客户源IP地址
具体配置如下
yaml格式，用于启用的时候指定配置文件
## Static configuration
entryPoints:
web:
address: ":80"
forwardedHeaders:
insecure: trueTOML格式，用于traefik启动的时候指定配置文件，这个文件在kubernetes中可以以comfigmap存在
## Static configuration
[entryPoints]
[entryPoints.web]
address = ":80"
[entryPoints.web.forwardedHeaders]
insecure = true命名行格式
--entryPoints.web.address=:80
--entryPoints.web.forwardedHeaders.insecure三.问题处理
1.公司现有环境配置
---
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
meta.helm.sh/release-name: traefik
meta.helm.sh/release-namespace: default
labels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: traefik
helm.sh/chart: traefik-9.11.0
name: traefik
namespace: default
resourceVersion: '505763774'
spec:
progressDeadlineSeconds: 600
replicas: 6
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/name: traefik
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: traefik
helm.sh/chart: traefik-9.11.0
spec:
containers:
- args:
- '--global.checknewversion'
- '--global.sendanonymoususage'
- '--entryPoints.traefik.address=:9000/tcp'
- '--entryPoints.web.address=:8000/tcp'
- '--entryPoints.websecure.address=:8443/tcp'
- '--api.dashboard=true'
- '--ping=true'
- '--providers.kubernetescrd'
- '--providers.kubernetesingress'
image: 'traefik:2.3.3'
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /ping
port: 9000
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 2
name: traefik
ports:
- containerPort: 9000
name: traefik
protocol: TCP
- containerPort: 8000
name: web
protocol: TCP
- containerPort: 8443
name: websecure
protocol: TCP
readinessProbe:
failureThreshold: 1
httpGet:
path: /ping
port: 9000
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 2
resources: {}
securityContext:
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsGroup: 65532
runAsNonRoot: true
runAsUser: 65532
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /data
name: data
- mountPath: /tmp
name: tmp
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 65532
serviceAccount: traefik
serviceAccountName: traefik
terminationGracePeriodSeconds: 60
volumes:
- emptyDir: {}
name: data
- emptyDir: {}
name: tmp
status:
availableReplicas: 6
conditions:
- lastTransitionTime: '2022-10-10T07:58:50Z'
lastUpdateTime: '2022-10-10T07:58:50Z'
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: 'True'
type: Available
- lastTransitionTime: '2020-11-25T22:53:59Z'
lastUpdateTime: '2022-11-17T10:44:40Z'
message: ReplicaSet "traefik-54bf67c74d" has successfully progressed.
reason: NewReplicaSetAvailable
status: 'True'
type: Progressing
observedGeneration: 13
readyReplicas: 6
replicas: 6
updatedReplicas: 6
---
apiVersion: v1
kind: Service
metadata:
annotations:
meta.helm.sh/release-name: traefik
meta.helm.sh/release-namespace: default
labels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: traefik
helm.sh/chart: traefik-9.11.0
name: traefik
namespace: default
resourceVersion: '505762379'
spec:
clusterIP: 10.96.252.108
externalTrafficPolicy: Cluster
ports:
- name: web
nodePort: 30079
port: 80
protocol: TCP
targetPort: web
selector:
app.kubernetes.io/instance: traefik
app.kubernetes.io/name: traefik
sessionAffinity: None
type: NodePort
status:
loadBalancer: {}
发现trafik启动的参数里面没有指定配置文件，也没有以comfigmap形式存在，所以只能考虑通过添加启动参数的方式（即命令行方式）解决问题
2.解决问题
在deployment部署中的traefik启动参数中添加 - '--entrypoints.web.forwardedHeaders.insecure'和- '--entrypoints.websecure.forwardedHeaders.insecure'启动参数，其中标红的web和websecure根据--entryPoints.web.address=:8000/tcp和--entryPoints.websecure.address=:8443/tcp要保持一致，有可能不一样。
修改后的derloyment的yaml文件如下：
---
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
meta.helm.sh/release-name: traefik
meta.helm.sh/release-namespace: default
labels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: traefik
helm.sh/chart: traefik-9.11.0
name: traefik
namespace: default
resourceVersion: '505763774'
spec:
progressDeadlineSeconds: 600
replicas: 6
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/name: traefik
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: traefik
helm.sh/chart: traefik-9.11.0
spec:
containers:
- args:
- '--global.checknewversion'
- '--global.sendanonymoususage'
- '--entryPoints.traefik.address=:9000/tcp'
- '--entryPoints.web.address=:8000/tcp'
- '--entryPoints.websecure.address=:8443/tcp'
- '--api.dashboard=true'
- '--ping=true'
- '--providers.kubernetescrd'
- '--providers.kubernetesingress'
- '--entrypoints.web.forwardedHeaders.insecure'
- '--entrypoints.websecure.forwardedHeaders.insecure'
image: 'traefik:2.3.3'
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /ping
port: 9000
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 2
name: traefik
ports:
- containerPort: 9000
name: traefik
protocol: TCP
- containerPort: 8000
name: web
protocol: TCP
- containerPort: 8443
name: websecure
protocol: TCP
readinessProbe:
failureThreshold: 1
httpGet:
path: /ping
port: 9000
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 2
resources: {}
securityContext:
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
runAsGroup: 65532
runAsNonRoot: true
runAsUser: 65532
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /data
name: data
- mountPath: /tmp
name: tmp
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 65532
serviceAccount: traefik
serviceAccountName: traefik
terminationGracePeriodSeconds: 60
volumes:
- emptyDir: {}
name: data
- emptyDir: {}
name: tmp
status:
availableReplicas: 6
conditions:
- lastTransitionTime: '2022-10-10T07:58:50Z'
lastUpdateTime: '2022-10-10T07:58:50Z'
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: 'True'
type: Available
- lastTransitionTime: '2020-11-25T22:53:59Z'
lastUpdateTime: '2022-11-17T10:44:40Z'
message: ReplicaSet "traefik-54bf67c74d" has successfully progressed.
reason: NewReplicaSetAvailable
status: 'True'
type: Progressing
observedGeneration: 13
readyReplicas: 6
replicas: 6
updatedReplicas: 6
---
apiVersion: v1
kind: Service
metadata:
annotations:
meta.helm.sh/release-name: traefik
meta.helm.sh/release-namespace: default
labels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: traefik
helm.sh/chart: traefik-9.11.0
name: traefik
namespace: default
resourceVersion: '505762379'
spec:
clusterIP: 10.96.252.108
externalTrafficPolicy: Cluster
ports:
- name: web
nodePort: 30079
port: 80
protocol: TCP
targetPort: web
selector:
app.kubernetes.io/instance: traefik
app.kubernetes.io/name: traefik
sessionAffinity: None
type: NodePort
status:
loadBalancer: {}
3.验证测试
通过和研发和业务验证测试，通过x_forwarded_for获取的客户端源Ip地址格式如下
其中第一个公网IP地址是客户的真实IP地址，其他几个是经过的k8s节点和pod的IP地址，至此解决业务上获取真实客户IP地址的需求。
开源、云原生的融合云平台
更多推荐
3
0- 0
已为社区贡献10条内容
所有评论(0)