K8s-Ingress Nginx-Day 08
本章主要讲ingress-nginx的安装和基本使用
·
1. 什么是Ingress
官方文档：https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/#what-is-ingress
Ingress 是 kubernetes API 中的标准资源类型之一，主要是k8s官方在维护。
2. Ingress的作用
Ingress 提供从集群外部到集群内服务的 HTTP 和 HTTPS 路由。 流量路由由 Ingress 资源所定义的规则来控制。
ingress 实现的功能是在应用层对客户端请求的 host 名称或请求的 URL 路径把请求转发到指定的 service 资源的规则，即用
于将 kubernetes 集群外部的请求资源转发之集群内部的 service，再被 service 转发之 pod
处理客户端的请求。
说简单点就是另一种对外暴露服务的方式。
Ingress本身其实就是nginx二次开发的一个东西。
3. Ingress-controller
为了让 Ingress 资源工作，集群必须有一个正在运行的 Ingress 控制器。
Ingress 资源需要指定监听地址、请求的 host 和 URL 等配置，然后根据这些规则的匹配机制将客户端的请求进行转发，这种能够为 ingress 配置资源监听并转发流量的组件称为ingress 控制器(ingress controller)，ingress controller 是 kubernetes 的一个附件，类似于dashboard 或者 flannel 一样，需要单独部署。
4. 其他类型的Ingres控制器
https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress-controllers/#additional-controllers
5. 部署Ingress Nginx
官方文档：https://kubernetes.github.io/ingress-nginx/deploy/
官方文档：https://github.com/kubernetes/ingress-nginx/blob/main/docs/deploy/index.md
可以把官方提供的这个yaml下载下面，改改部署也可以
5.1 部署注意事项
官方文档：https://github.com/kubernetes/ingress-nginx?tab=readme-ov-file#supported-versions-table
部署Ingress-Nginx的时候一定要注意和k8s版本兼容性的问题
5.2 部署方式
Ingress-nginx有2种部署方式，一个是deployment，一个是demonset，区别：
（1）deployment方式部署
默认只有一个ingress控制器，在被调度到的节点上监听默认的80和443端口，如果有请求是从其他节点过来的，需要先经过路由转发才能到我ingress控制器所在的这个节点，就增加了一个转发的性能开销。
（2）demonset方式部署
在每个k8s节点上都部署一个ingress控制器，同时也在每个节点上都监听了80和443端口，该方式性能更好。
5.3 部署Ingress Nginx v1.3.0
因为我本地也无法从外网拉取docker镜像，所以改用了改好的1.3.0，我的集群是1.24.0的，刚好1.3.0能用
5.3.1 yaml配置
5.3.1.1 deployment
[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# cat 1.ingress-nginx-controller-v1.3.0_deployment.yaml
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resourceNames:
- ingress-controller-leader
resources:
- configmaps
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- coordination.k8s.io
resourceNames:
- ingress-controller-leader
resources:
- leases
verbs:
- get
- update
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
- namespaces
verbs:
- list
- watch
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: v1
data:
allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
nodePort: 50080
- appProtocol: http #kubernetes v1.20 stable,appProtocol字段提供了一种为每个Service端口指定应用协议的方式,此字段的取值会被映射到对应的Endpoints
name: prometheus-metrics-port
port: 10254
protocol: TCP
targetPort: 10254 #ingress-nginx-controller内置的指标数据采集端口
nodePort: 50254
# - name: metrics-port
# port: 10254
# targetPort: 10254
# nodePort: 50254
# protocol: TCP
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
nodePort: 50443
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: NodePort
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
- appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
spec:
containers:
- args:
- /nginx-ingress-controller
- --election-id=ingress-controller-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/ingress-nginx-controller:v1.3.0
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
- containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: true
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
runAsUser: 101
volumeMounts:
- mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
nodeSelector:
kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
- name: webhook-cert
secret:
secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-create
spec:
containers:
- args:
- create
- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
- --namespace=$(POD_NAMESPACE)
- --secret-name=ingress-nginx-admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-patch
spec:
containers:
- args:
- patch
- --webhook-name=ingress-nginx-admission
- --namespace=$(POD_NAMESPACE)
- --patch-mutating=false
- --secret-name=ingress-nginx-admission
- --patch-failure-policy=Fail
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: nginx
spec:
controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- ingresses
sideEffects: None
5.3.1.2 demonset
[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# cat 2.ingress-nginx-controller-v1.3.0_daemonset.yaml
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resourceNames:
- ingress-controller-leader
resources:
- configmaps
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- coordination.k8s.io
resourceNames:
- ingress-controller-leader
resources:
- leases
verbs:
- get
- update
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
- namespaces
verbs:
- list
- watch
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: v1
data:
allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
nodePort: 30080
- appProtocol: http #kubernetes v1.20 stable,appProtocol字段提供了一种为每个Service端口指定应用协议的方式,此字段的取值会被映射到对应的Endpoints
name: prometheus-metrics-port
port: 10254
protocol: TCP
targetPort: 10254 #ingress-nginx-controller内置的指标数据采集端口
nodePort: 30254
# - name: metrics-port
# port: 10254
# targetPort: 10254
# nodePort: 50254
# protocol: TCP
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
nodePort: 30443
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: NodePort
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
- appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
spec:
#hostNetwork: true #使用宿主机网络。使用宿主机网络一定要修改dns策略为ClusterFirstWithHostNet，否则ingress控制器的默认dns策略ClusterFirst会因为pod使用了hostnetwork而退回default模式，从而直接使用宿主机dns配置。
#hostPID: true #使用宿主机Pid
containers:
- args:
- /nginx-ingress-controller
- --election-id=ingress-controller-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/ingress-nginx-controller:v1.3.0
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
- containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: true
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
runAsUser: 101
volumeMounts:
- mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
tolerations:
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoSchedule"
nodeSelector:
kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
- name: webhook-cert
secret:
secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-create
spec:
containers:
- args:
- create
- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
- --namespace=$(POD_NAMESPACE)
- --secret-name=ingress-nginx-admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-patch
spec:
containers:
- args:
- patch
- --webhook-name=ingress-nginx-admission
- --namespace=$(POD_NAMESPACE)
- --patch-mutating=false
- --secret-name=ingress-nginx-admission
- --patch-failure-policy=Fail
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: nginx
spec:
controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- ingresses
sideEffects: None
5.3.2 demonset方式部署ingress nginx 控制器
[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# kubectl apply -f 2.ingress-nginx-controller-v1.3.0_daemonset.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
daemonset.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# kubectl get po,svc -n ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-admission-create-bbxkb 0/1 Completed 0 26m
pod/ingress-nginx-admission-patch-z7zqf 0/1 Completed 0 26m
pod/ingress-nginx-controller-5ch6s 1/1 Running 0 26m
pod/ingress-nginx-controller-d9mtz 1/1 Running 0 26m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller NodePort 10.200.238.16 <none> 80:30080/TCP,10254:30254/TCP,443:30443/TCP 26m
service/ingress-nginx-controller-admission ClusterIP 10.200.118.113 <none> 443/TCP 22m
6. 基于域名实现单host及多host的ingress
6.1 部署测试用的web服务
[root@master1 Ingress-case-20230611-backup]# cat tomcat-app1.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
app: tomcat-app1-deployment-label
name: tomcat-app1-deployment
spec:
replicas: 1
selector:
matchLabels:
app: tomcat-app1-selector
template:
metadata:
labels:
app: tomcat-app1-selector
spec:
containers:
- name: tomcat-app1-container
image: tomcat:7.0.94-alpine
#command: ["/apps/tomcat/bin/run_tomcat.sh"]
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
protocol: TCP
name: http
env:
- name: "password"
value: "123456"
- name: "age"
value: "18"
resources:
limits:
cpu: 1
memory: "512Mi"
requests:
cpu: 500m
memory: "512Mi"
---
kind: Service
apiVersion: v1
metadata:
labels:
app: tomcat-app1-service-label
name: tomcat-app1-service
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 8080
selector:
app: tomcat-app1-selector
[root@master1 Ingress-case-20230611-backup]# kubectl apply -f tomcat-app1.yaml
deployment.apps/tomcat-app1-deployment created
service/tomcat-app1-service created
[root@master1 Ingress-case-20230611-backup]# cat tomcat-app2.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
app: tomcat-app2-deployment-label
name: tomcat-app2-deployment
spec:
replicas: 1
selector:
matchLabels:
app: tomcat-app2-selector
template:
metadata:
labels:
app: tomcat-app2-selector
spec:
containers:
- name: tomcat-app2-container
image: tomcat:7.0.94-alpine
#command: ["/apps/tomcat/bin/run_tomcat.sh"]
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
protocol: TCP
name: http
env:
- name: "password"
value: "123456"
- name: "age"
value: "18"
resources:
limits:
cpu: 1
memory: "512Mi"
requests:
cpu: 500m
memory: "512Mi"
---
kind: Service
apiVersion: v1
metadata:
labels:
app: tomcat-app2-service-label
name: tomcat-app2-service
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 8080
selector:
app: tomcat-app2-selector
[root@master1 Ingress-case-20230611-backup]# kubectl apply -f tomcat-app2.yaml
deployment.apps/tomcat-app2-deployment created
service/tomcat-app2-service created
[root@master1 Ingress-case-20230611-backup]# kubectl get po |grep tomcat
tomcat-app1-deployment-749f777dbf-mz9ws 1/1 Running 0 4m10s
tomcat-app2-deployment-6999ff4c89-brmrp 1/1 Running 0 2m48s
6.2 环境初始化
6.2.1 tomcat-1
[root@master1 Ingress-case-20230611-backup]# kubectl exec -it tomcat-app1-deployment-749f777dbf-mz9ws -- /bin/bash
bash-4.4# pwd
/usr/local/tomcat/webapps
bash-4.4# mkdir app1
bash-4.4# echo 'pc web test page' > app1/index.jsp
bash-4.4# exit
6.2.2 tomcat-2
[root@master1 Ingress-case-20230611-backup]# kubectl exec -it tomcat-app2-deployment-6999ff4c89-brmrp -- /bin/bash
bash-4.4# pwd
/usr/local/tomcat/webapps
bash-4.4# mkdir app2
bash-4.4# echo 'mobile app test page' > app2/index.jsp
bash-4.4# exit
6.3 配置单域名访问的ingress规则
6.3.1 配置ingress规则
[root@master1 Ingress-case-20230611-backup]# cat 2.1.ingress_single-mobile.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: mobile.test.com # 用域名作为ingress名称，方便识别
namespace: default # 注意nginx规则要和后端pod在同一个ns下
annotations: # 注解，类似于指令，相当于配置nginx服务的一些功能
kubernetes.io/ingress.class: "nginx" ##指定Ingress Controller的类型
nginx.ingress.kubernetes.io/use-regex: "true" ##指定后面rules定义的path可以使用正则表达式
nginx.ingress.kubernetes.io/proxy-connect-timeout: "600" ##连接超时时间,默认为5s
nginx.ingress.kubernetes.io/proxy-send-timeout: "600" # 设置代理服务器向后端服务器发送数据的超时时间，单位为秒，默认60s，如果超时Ingress控制器将终止连接并返回适当的错误。
nginx.ingress.kubernetes.io/proxy-read-timeout: "600" ##读取超时时间,默认为60s
# 上面的三个proxy超时时间，建议设置长一点时间。
nginx.ingress.kubernetes.io/proxy-body-size: "50m" ##客户端上传文件，最大大小，默认为20m
#nginx.ingress.kubernetes.io/rewrite-target: / ##URL重写
nginx.ingress.kubernetes.io/app-root: /index.html # 定义上下文中 / 必须重定向的应用程序根目录
spec:
rules: # 路由规则
- host: mobile.test.com # 客户端访问的host域名
http:
paths:
- pathType: Prefix # 前缀匹配，且区分大小写。另外还有Exact精确匹配和ImplementationSpecific。
path: "/" # 匹配路径，类似nginx的location
backend: # 后端配置
service:
name: tomcat-app2-service # 后端pod的service名称
port:
number: 80 # service端口
[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 2.1.ingress_single-mobile.yaml
ingress.networking.k8s.io/mobile.test.com created
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
mobile.test.com <none> mobile.test.com 80 3s
6.3.2 访问测试
6.4 配置多域名访问的ingress规则
6.4.1 配置ingress规则
# 清理环境
[root@master1 Ingress-case-20230611-backup]# kubectl delete -f 2.1.ingress_single-mobile.yaml
# 配置并应用新的ingress规则
[root@master1 Ingress-case-20230611-backup]# cat 2.2.ingress_multi-host.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-web
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
#nginx.ingress.kubernetes.io/rewrite-target: / ##URL重写
nginx.ingress.kubernetes.io/app-root: /index.html
spec:
rules:
- host: pc.test.com # 域名一
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: tomcat-app1-service
port:
number: 80
- host: mobile.test.com # 域名二
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: tomcat-app2-service
port:
[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 2.2.ingress_multi-host.yaml
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-web <none> pc.test.com,mobile.test.com 192.168.10.11,192.168.10.12 80 15m
6.4.2 访问测试
6.4.3 清理环境
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-web <none> pc.test.com,mobile.test.com 192.168.10.11,192.168.10.12 80 15m
[root@master1 Ingress-case-20230611-backup]# kubectl delete ingress nginx-web
ingress.networking.k8s.io "nginx-web" deleted
6.5 配置基于URL访问的ingress规则
就是和nginx location配置差不多的道理，通过不同的URL location匹配并转发到不同的后端service。
6.5.1 ingress规则配置
[root@master1 Ingress-case-20230611-backup]# cat 3.1.ingress-url.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-web
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
#nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/app-root: /index.html
spec:
rules:
- host: pc.test.com
http:
paths:
- pathType: Prefix
path: "/app1"
backend:
service:
name: tomcat-app1-service
port:
number: 80
- pathType: Prefix
path: "/app2"
backend:
service:
name: tomcat-app2-service
port:
number: 80
[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 3.1.ingress-url.yaml
ingress.networking.k8s.io/nginx-web created
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-web <none> pc.test.com 80 3s
6.5.2 访问测试
6.5.3 清理环境
[root@master1 Ingress-case-20230611-backup]# kubectl delete -f 3.1.ingress-url.yaml
ingress.networking.k8s.io "nginx-web" deleted
7. 单域名和多域名的https证书配置
7.1 单域名https证书配置
7.1.1 自签证书
7.1.1.1 签发ca证书
[root@master1 ~]# mkdir certs
[root@master1 ~]# cd certs
[root@master1 certs]# openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 3560 -nodes -subj '/CN=pc.test.com'
Generating a 4096 bit RSA private key
...........++
..............................................................................++
writing new private key to 'ca.key'
-----
[root@master1 certs]# ll
total 8
-rw-r--r-- 1 root root 1793 Feb 19 10:37 ca.crt
-rw-r--r-- 1 root root 3272 Feb 19 10:37 ca.key
7.1.1.2 签发域名证书-公钥和csr
[root@master1 certs]# openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=pc.test.com'
Generating a 4096 bit RSA private key
............................................................................................................................................................................................................++
.........................................................................................................................................++
writing new private key to 'server.key'
-----
[root@master1 certs]# ll -rt
total 16
-rw-r--r-- 1 root root 3272 Feb 19 10:37 ca.key
-rw-r--r-- 1 root root 1793 Feb 19 10:37 ca.crt
-rw-r--r-- 1 root root 3272 Feb 19 10:38 server.key
-rw-r--r-- 1 root root 1586 Feb 19 10:38 server.csr
[root@master1 certs]#
7.1.1.3 签发域名证书-私钥
[root@master1 certs]# openssl x509 -req -sha256 -days 3650 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt
Signature ok
subject=/CN=pc.test.com
Getting CA Private Key
[root@master1 certs]# ll -rt
total 20
-rw-r--r-- 1 root root 3272 Feb 19 10:37 ca.key
-rw-r--r-- 1 root root 1793 Feb 19 10:37 ca.crt
-rw-r--r-- 1 root root 3272 Feb 19 10:38 server.key
-rw-r--r-- 1 root root 1586 Feb 19 10:38 server.csr
-rw-r--r-- 1 root root 1663 Feb 19 10:40 server.crt
7.1.2 上传证书到k8s集群中
[root@master1 certs]# kubectl create secret tls tls-secret-pc --cert=server.crt --key=server.key -n default
secret/tls-secret-pc created
[root@master1 certs]# kubectl get secret # 证书上传完毕后，内容就会通过base64加密
NAME TYPE DATA AGE
tls-secret-pc kubernetes.io/tls 2 5s
7.1.3 配置ingress规则
[root@master1 Ingress-case-20230611-backup]# cat 4.1.ingress-https-magedu_single-host.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-web
annotations:
kubernetes.io/ingress.class: "nginx" ##指定Ingress Controller的类型
nginx.ingress.kubernetes.io/ssl-redirect: 'true' #SSL重定向，即将http请求强制重定向至https，等于nginx中的全站https
spec:
tls:
- hosts:
- pc.test.com
secretName: tls-secret-pc
rules:
- host: pc.test.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: tomcat-app1-service
port:
number: 80
[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 4.1.ingress-https-magedu_single-host.yaml
ingress.networking.k8s.io/nginx-web created
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-web <none> pc.test.com 80, 443 5s
7.1.4 访问测试
7.2 多域名https证书配置
这里只展示配置，其他的不再演示
7.2.1 配置ingress规则
[root@master1 Ingress-case-20230611-backup]# cat 4.2.ingress-https-magedu_multi-host.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-web-mobile
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: 'true'
spec:
tls: # 就是在tls下配置不同的hosts就行了，指定不同的secret证书
- hosts:
- mobile.test.com
secretName: tls-secret-mobile
- hosts:
- pc.test.com
secretName: tls-secret-pc
rules:
- host: pc.test.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: tomcat-app1-service
port:
number: 80
- host: mobile.test.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: tomcat-app2-service
port:
number: 80
8. ingress 证书更新案例
8.1 确认老证书的过期时间
8.2 签发新证书
实际工作中，使用商业证书都是不需要自己手动签发的。
[root@master1 certs-new]# openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=pc.test.com'
Generating a 4096 bit RSA private key
......................................................................................................................++
.........++
writing new private key to 'server.key'
-----
[root@master1 certs-new]# ls
server.csr server.key
[root@master1 certs-new]# openssl x509 -req -sha256 -days 36500 -in server.csr -CA ../certs/ca.crt -CAkey ../certs/ca.key -set_serial 01 -out server.crt # 这里还是基于之前的ca签发新证书，这里的证书有效期我配置了100年 -days 36500
Signature ok
subject=/CN=pc.test.com
Getting CA Private Key
[root@master1 certs-new]# ll -rt
total 12
-rw-r--r-- 1 root root 3268 Feb 19 11:40 server.key
-rw-r--r-- 1 root root 1586 Feb 19 11:40 server.csr
-rw-r--r-- 1 root root 1667 Feb 19 11:41 server.crt
[root@master1 certs-new]#
8.3 base64加密新证书内容
8.3.1 加密公钥.crt文件内容
# 加密后的内容：
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVvakNDQW9vQ0FRRXdEUVlKS29aSWh2Y05BUUVMQlFBd0ZqRVVNQklHQTFVRUF3d0xjR011ZEdWemRDNWoKYjIwd0lCY05NalF3TWpFNU1ETTBNVE0xV2hnUE1qRXlOREF4TWpZd016UXhNelZhTUJZeEZEQVNCZ05WQkFNTQpDM0JqTG5SbGMzUXVZMjl0TUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUFzRlFuClovOXlLOVZBYnZiNVQ0bm5oSTNBMXUzMEVnYzFadlZRS1Q3L29uMFpWYnVZTFVNdDc3bHJNMjRRaUtGS0JjaXIKYjFrT0xJNDAvV1lmdVBUMDVIK2J5MlFxR3Zkb2dOWXBza2E5SWsxcnM1bzNVVjltNDBNSnhSOE9IOEJoUThNLwpzZVZGMWp5SXEwb0ZLUHFFSDVkMnhZTXdMekMrT3ArNGNNZUsxU3hoNlNNbm82TXhhcU00aGJlTU5NbmFPY1BjCkVHT3FqZHVMbmtPRXd4UDFmWDZrQkpEaW5WditKYUJwWG8zdHJJaFg4RWIxNGpqL21qemFON1pObm43d3grVGcKTHl3cUhEZWdHVmoxays4V2hqOUdXNWhXL3F1ZXM1L3lDeUdDR1ZYODNxN3NvRWxjR0ovRU8yRnZPTWJQTUJRdwplZjdvTkU5WFhzdXZMZVVMUXNOZ3FvR3RicEpaTlFxLzh1QnBIbUhsNjljQmtuVkVnSlJZMC9pK3VEaGFpQ1lWCkJPZURVb3hFOFZQSWlMMWJ2KzhIZ2lWVm5LL3lmazhsU3J1RldnbzlkTkhDTWFEQ014cE5KYTA5N2xYQ1c5N1cKMGNUNEF6WGdXL3B0VlR6ZktWZUd4VCtGem9jUGlZbk1EeXEwcXVTMXNYeWpjK0xZTjZWWmRPcGRwZzVZczg3Tgp5V25EVUl1OWg4NldRbnJMZDd1ZTY1VU9EWmJjb2QzTVVoMUVNcFNwTHE3bEZwWTdZRmVONklBRWRobnRzWVhECkFzU3ZWdVNSL1hkRThZYVNFYXNhMEYrWVczZWlGQzNobEFzZDZVVG5UK0E4Z09zemhpQVorRE43NDROOVVVdEgKeWlKYXVkTlVxRXA1VjE3V1gzN25oZWJ1SzdaUlRMOWRmcThYSzk4Q0F3RUFBVEFOQmdrcWhraUc5dzBCQVFzRgpBQU9DQWdFQVVzR09qUG5Sb2NOaFRqUjNoa1dQYmNDT3AwenJDbndEY1Y4ZG11QzVGbm5PTUdJc2hZelV5ZHJUCjZsOEFmRlhibUNtZ2xvbjFxUEpPR3lMT3A5YlRRdlQ3bDY4elRSb0p5R1JxdVZCRGoxb1JCUUI3TER3ZUN4THQKU2NGR0JFdi9wWFc3NnVWbXovZTVRRmlIY0pna1F1bmsyR0JvZUUwM2lLR3ZabW9PcnM3b2VMRW1XM3prb2JxbgpiTlV1MFMraHRJcHlnZEYrM1pjL0RVTmsyaWEwS3JFVnVlMWdDL0YveldiNVR5eDBQVEhnNlIwKzRKeERMRlF0Cnh0N3IrNXVQeUg0QVlsWFNwenNnM3VvL1BmNkZtOHA0VlJDcGNCYUlaUCtPRlZzSzN2SFhyODJuNHdXOU8vRCsKNFJ1cnpXWnRGK0hmNHJVY3VpN2gwbmk5d0pUUUVUZDJuTEZxcHM0ZENuNCs1L1FtY3JVUDdkNlk0OExEYktLawp3UWd2NzB1aUp6alhRNjRKa1dMSDhWVTRRdEpGQSs1QkFraHFaRFZEWXoxMEV3QlJvRWNuWFA5MXpaUyszOTVICkZFNjB0Vkx0YmVnSktxb0pUSTJ0WThyWWNhSVBWbjNESi9KbTdLTWpXZGwxK3BKNEhVNzgzRW9sRzMyYkZiNTUKZm9LQzE2T0tkQ2x4ZTZrOCtyQWxBRnV1Z1UxZzBFVXVycjNHeERtbjBaNG5hZEFKcmYvK1p6NXZXUndPWEwwSgpGRFhUTkRVRE9qcnIxTlJ1bHVTOERROEVIZTM4N1dtM2ExMSttOUZVYUhpdHI4U21DZG5lK2RzQkswNUxVVEtoCmJEL3pOQzI3SzJIQUxmaUlrRmZZTFN4TGl1Q1c3QXpoQ3oyeHp2UVdUS0RQRGRaZ1BCMD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQ==
8.3.2 加密私钥.key文件内容
8.3 更新证书
在线更新secrets内容是实时生效的，但是生产环境变更时，一定不要在线编辑，先导出旧的secrets yaml到本地，备份后，在文件中修改应用，这样就算出了问题，也可以第一时间恢复正常。
但是由于secrets创建后，无法通过apply -f 更新内容，所以先导出旧的内容备份，然后edit在线编辑实时生效。
或者备份好后，先delete 旧的secrets，再apply -f创建新的，但是这种方式没有第一种稳妥。
8.3.1 修改对应证书secrets的内容
[root@master1 certs-new]# kubectl get secrets
NAME TYPE DATA AGE
tls-secret-pc kubernetes.io/tls 2 73m
[root@master1 certs-new]# kubectl edit secrets tls-secret-pc
8.3.2 验证证书更新结果
9. 配置ingress控制器日志格式为json
参考：https://github.com/zhangshijle/ingress-files/blob/main/2.ingress-configmap.yaml
[root@master1 Ingress-case-20230611-backup]# kubectl get cm -n ingress-nginx|grep ingress
ingress-controller-leader 0 18h
ingress-nginx-controller 1 18h
# 调整内容展示
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
# 下面这个注解的作用记录了最后一次应用的配置信息。
kubectl.kubernetes.io/last-applied-configuration: >
{"apiVersion":"v1","kind":"ConfigMap","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/part-of":"ingress-nginx"},"name":"nginx-configuration","namespace":"ingress-nginx"}}
data:
use-forwarded-headers: "true" # 启用使用转发的HTTP头信息。
compute-full-forwarded-for: "true" # 启用计算完整的转发的HTTP头中的"X-Forwarded-For"字段。
use-proxy-protocol: 'True' # 启用使用代理协议（Proxy Protocol），但是要后端支持4层才行。
real-ip-header: proxy_protocol # 指定真实IP的头信息名称为"proxy_protocol"。
set-real-ip-from: 172.26.0.0/16 # 公有云的话要修改为当前kubernetes环境的当前使用的VPC的CIDR格式的网络信息（掩码），然后日志里面会显示是从哪个网段转发过来的请求。私有云不用加。
# 日志格式
log-format-upstream: '{"nginx_timestamp":"$time_iso8601","tcp_xff":"$proxy_protocol_addr","clientip":"$remote_addr","nginx_host":"$server_addr","host":"$http_host","request":"$request","url":"$request_uri","upstreamhost":"$upstream_addr","status":"$status","body_bytes_sent":"$body_bytes_sent","request_time":"$request_time","upstream_response_time":"$upstream_response_time","xff":"$http_x_forwarded_for","referer":"$http_referer","http_user_agent":"$http_user_agent","request_length":"$request_length","request_method":"$request_method"}'
K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容
更多推荐
24
0- 0
已为社区贡献11条内容
所有评论(0)