前言：

 

ingress直译：进口；入口；初切；进入；进入资格；进入权。在kubernetes中，它指的是网络入口。

ingress概述：

通俗来讲，Ingress和之前提到的Service、Deployment等类似，也是一 个Kubernetes的资源对象，Deployment是用来部署应用的，Ingress就是实现 用域名的方式访问应用。Ingress实现的方式有很多，比如Nginx、HAProxy、 Treafik等（也就是控制器），就Nginx而言，和上述提到的传统服务架构用Nginx类似。 Ingress控制器在每个符合条件的宿主机上部署一个Pod，这个Pod里面运行的 就是Nginx进程，里面的实现逻辑和宿主机部署Nginx的方式并无太大区别， 关键区别是宿主机部署的Nginx需要更改Nginx的配置文件配置域名，而 Ingress则和其他Kubernetes资源文件一样，使用YAML文件进行配置，之后 Ingress控制器根据YAML文件定义的内容自动生成对应的配置文件。 在Kubernetes v1.1版中正式引用Ingress class的概念，用于从集群外部到集 群内部Service的HTTP和HTTPS路由，可以配置提供服务外部访问的URL、负载 均衡和终止SSL，并提供基于域名的虚拟主机。流量从Internet到Ingress再 到Services最后到Pod上，通常情况下，Ingress部署在所有的Node节点上， 暴露443和80端口（一般通过hostNetwork的方式部署Ingress），之后再通过 F5或公有云LB代理到对应的Ingress节点上，之后将域名解析到F5或公有云LB 即可实现基于域名的服务发布。

OK，为什么要用ingress呢？这个问题我先给打个10分。

确实，我们希望任何东西都是简单可靠可用的，但一台服务器上的端口是有限的，总共是65535好像，并且如果一个服务器如果像一面墙一样打满了孔洞，光管理这些端口就是一个非常大的工作量了，何况这些孔洞还有安全方面的风险（所以喽，有的安全性要求高的服务器会安装防火墙，就是堵这些孔洞的）。

回到k8s集群内，我们知道service的网络模式可以是NodePort或者EXTERNAL-IP或者是CLUSTER-IP或者是什么都没有的无头SERVICE（CLUSTER-IP 没有值）。

而想要把服务发布到集群外无外乎就是NodePort和EXTERNAL-IP两种方式了，CLUSTER-IP只能在集群内部使用，毕竟虚拟出来的IP地址嘛。

NodePort会在集群内的所有节点开辟一个随机的唯一的不重复的端口，并且这个端口还是由kube-apiserver限定的2700多个端口，关键的地方是随机和唯一不重复。

kube-apiserver的端口限定通常是它的配置文件（30000到32767）：

KUBE_APISERVER_OPTS="--v=2 \
--logtostderr=false \
--service-node-port-range=30000-32767 \
--log-dir=/opt/kubernetes/logs \
--etcd-servers=https://192.168.217.16:2379,https://192.168.217.17:2379,https://192.168.217.18:2379 \
。。。。省略
--service-cluster-ip-range=10.0.0.0/16 \
。。。。省略

 那么，如果有几十个service需要发布到集群外，使用NodePort就会有几十个上百个随机的端口，是不是十分的不安全和不方便呢（要查询出随机的端口，并记住的哦）。

EXTERNAL-IP 是使用互联网的IP，而互联网的IP资源是比较少的，这一条就把服务发布的路堵的差不多了。

而ingress 是在集群内创建一个service，此service可以绑定N个自定义的域名，并且它使用的端口是一个固定的端口，除非你重新生成这个service，而这就给了我们的无数可能，也因此，借由这个service，我们可以实现服务发布的特定流程，比如红蓝发布。

例如我这个ingress controller的service：

[root@k8s-master ~]# k get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.0.191.241   <none>        80:32296/TCP,443:30766/TCP   9h

只要利用ingress资源清单文件绑定一个域名+32296这个端口就可以在集群外访问到了，如果是tls类型的域名，那么就是域名+30766端口在集群外就可以访问到。可以看到，端口是固定了的两个，那么孔洞也就十分少了，安全性自然是提高了嘛，剩下的管理工作就是管理好这些ingress清单文件就可以了，这就自然得达到了服务治理的目的。 

总结一哈，ingress可以实现服务治理（管理集群内的每一个service），服务发布（比如红蓝，金丝雀等等都是基于ingress），网络黑白名单，https支持等等功能（主要的还是服务service）。

简单的流程说一哈：

（1）某一个web应用（例如，门户网站，WordPress这样的cms等等，此时的表现形式是一个pod实例）---->基于web应用创建一个需要发布到集群外的service（这个service一般不需要使用NodePort或者EXTERNAL-IP，因为如果是这两个类型已经可以发布service到集群外了，在使用ingress就是脱裤子放P了）

（2）ingress资源清单文件（此文件绑定上面的web应用的service和一个自定义域名）---->ingress 控制器绑定一个ingress class（指定使用哪个控制器，如果有多个controller的话）---->ingress service（ingress 控制器发出一个service（通常这个service是NodePort或者EXTERNAL-IP的网络类型，这个服务非常重要，是总入口，是在部署ingress控制器的时候生成）---->ingress service通过控制器处理web应用衍生的那个service，也就是（1）的service

一，

ingress-nginx的部署：

ingress-nginx是一种网络插件，部署比较简单，说实话，部署简单，用好ingress真的非常不容易

版本问题：

由于kubernetes的版本众多，因此，ingress作为其扩展功能插件是也随着kubernetes的版本有更新，并且ingress资源定义也随着版本变迁有一定的改变。

在本例中，使用的kubernetes版本是1.18，经测试ingress是和其匹配的，但如果用在高版本的kubernetes会有问题，在此提醒一哈（具体表现是在高版本kubernetes内部署不会成功，高版本指的是1.22及以后的版本）。

Support Versions table

Ingress-nginx versionk8s supported versionAlpine VersionNginx Version
v1.1.11.23, 1.22, 1.21, 1.20, 1.193.14.21.19.9†
v1.1.01.22, 1.21, 1.20, 1.193.14.21.19.9†
v1.0.51.22, 1.21, 1.20, 1.193.14.21.19.9†
v1.0.41.22, 1.21, 1.20, 1.193.14.21.19.9†
v1.0.31.22, 1.21, 1.20, 1.193.14.21.19.9†
v1.0.21.22, 1.21, 1.20, 1.193.14.21.19.9†
v1.0.11.22, 1.21, 1.20, 1.193.14.21.19.9†
v1.0.01.22, 1.21, 1.20, 1.193.13.51.20.1
v0.50.01.21, 1.20, 1.193.14.21.19.9†
v0.49.31.21, 1.20, 1.193.14.21.19.9†
v0.49.21.21, 1.20, 1.193.14.21.19.9†
v0.49.11.21, 1.20, 1.193.14.21.19.9†
v0.49.01.21, 1.20, 1.193.13.51.20.1
v0.48.11.21, 1.20, 1.193.13.51.20.1
v0.47.01.21, 1.20, 1.193.13.51.20.1
v0.46.01.21, 1.20, 1.193.13.21.19.6

#######附：

在Kubernetes 1.19之前，仅支持v1beta1 Ingress资源-从Kubernetes 1.19到1.21，同时支持v1beta1和v1 Ingress资源-在Kubernetes 1.22及更高版本中，仅支持v1 Ingress资源 

yaml文件的下载：

版本号：GitHub - kubernetes/ingress-nginx: Ingress-NGINX Controller for Kubernetes

例如，下载1.1.1版本的yaml文件，如果是其它版本替换版本号就可以了 

https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/cloud/deploy.yaml

vim deploy-ingress.yaml

这么长的文件不用说了，很多同学看完必定会说：我的头要裂开了。没错，我也是这样的。

此文件主要有几个地方需要注意，第一，修改部署方式为daemonsets，第二，镜像部分我做了处理都是国内可以直接pull的，也就是本地化，第三，如果要部署的话，建议master节点打effect为NoScheduler的污点，毕竟ingress是一个重要服务，放到master和apiserver抢资源不太合适。

apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
 
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx
  namespace: ingress-nginx
automountServiceAccountToken: true
---
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
data:
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
  name: ingress-nginx
rules:
  - apiGroups:
      - ''
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ''
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ''
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ''
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - extensions
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingressclasses
    verbs:
      - get
      - list
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
  - kind: ServiceAccount
    name: ingress-nginx
    namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ''
    resources:
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ''
    resources:
      - configmaps
      - pods
      - secrets
      - endpoints
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ''
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - networking.k8s.io   # k8s 1.14+
    resources:
      - ingressclasses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ''
    resources:
      - configmaps
    resourceNames:
      - ingress-controller-leader-nginx
    verbs:
      - get
      - update
  - apiGroups:
      - ''
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ''
    resources:
      - events
    verbs:
      - create
      - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
  - kind: ServiceAccount
    name: ingress-nginx
    namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  type: ClusterIP
  ports:
    - name: https-webhook
      port: 443
      targetPort: webhook
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
---
apiVersion: v1
kind: Service
metadata:
  annotations:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/component: controller
  revisionHistoryLimit: 10
  minReadySeconds: 0
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/component: controller
    spec:
      dnsPolicy: ClusterFirstWithHostNet
      containers:
        - name: controller
          image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v0.50.0
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          args:
            - /nginx-ingress-controller
            - --election-id=ingress-controller-leader
            - --ingress-class=nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            runAsUser: 101
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          livenessProbe:
            failureThreshold: 5
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
            - name: webhook
              containerPort: 8443
              protocol: TCP
          volumeMounts:
            - name: webhook-cert
              mountPath: /usr/local/certificates/
              readOnly: true
          resources:
            requests:
              cpu: 100m
              memory: 90Mi
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
        - name: webhook-cert
          secret:
            secretName: ingress-nginx-admission
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  name: ingress-nginx-admission
webhooks:
  - name: validate.nginx.ingress.kubernetes.io
    matchPolicy: Equivalent
    rules:
      - apiGroups:
          - networking.k8s.io
        apiVersions:
          - v1beta1
        operations:
          - CREATE
          - UPDATE
        resources:
          - ingresses
    failurePolicy: Fail
    sideEffects: None
    admissionReviewVersions:
      - v1
      - v1beta1
    clientConfig:
      service:
        namespace: ingress-nginx
        name: ingress-nginx-controller-admission
        path: /networking/v1beta1/ingresses
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
rules:
  - apiGroups:
      - admissionregistration.k8s.io
    resources:
      - validatingwebhookconfigurations
    verbs:
      - get
      - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
  - kind: ServiceAccount
    name: ingress-nginx-admission
    namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ''
    resources:
      - secrets
    verbs:
      - get
      - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: ingress-nginx-admission
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
  - kind: ServiceAccount
    name: ingress-nginx-admission
    namespace: ingress-nginx
---
apiVersion: batch/v1
kind: Job
metadata:
  name: ingress-nginx-admission-create
  annotations:
    helm.sh/hook: pre-install,pre-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  namespace: ingress-nginx
spec:
  template:
    metadata:
      name: ingress-nginx-admission-create
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
    spec:
      containers:
        - name: create
          image: jettech/kube-webhook-certgen:v1.5.1
          imagePullPolicy: IfNotPresent
          args:
            - create
            - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
            - --namespace=$(POD_NAMESPACE)
            - --secret-name=ingress-nginx-admission
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx-admission
      securityContext:
        runAsNonRoot: true
        runAsUser: 2000
---
apiVersion: batch/v1
kind: Job
metadata:
  name: ingress-nginx-admission-patch
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    helm.sh/chart: ingress-nginx-3.33.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.47.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: admission-webhook
  namespace: ingress-nginx
spec:
  template:
    metadata:
      name: ingress-nginx-admission-patch
      labels:
        helm.sh/chart: ingress-nginx-3.33.0
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 0.47.0
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: admission-webhook
    spec:
      containers:
        - name: patch
          image: docker.io/jettech/kube-webhook-certgen:v1.5.1
          imagePullPolicy: IfNotPresent
          args:
            - patch
            - --webhook-name=ingress-nginx-admission
            - --namespace=$(POD_NAMESPACE)
            - --patch-mutating=false
            - --secret-name=ingress-nginx-admission
            - --patch-failure-policy=Fail
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
      restartPolicy: OnFailure
      serviceAccountName: ingress-nginx-admission
      securityContext:
        runAsNonRoot: true
        runAsUser: 2000

apply上述文件后，将会出现这么些pod：

[root@master ~]# k get po -n  ingress-nginx -owide
NAME                                      READY   STATUS      RESTARTS   AGE    IP                NODE         NOMINATED NODE   READINESS GATES
ingress-nginx-admission-create-xc2z4      0/1     Completed   0          4d2h   192.168.169.133   k8s-node2    <none>           <none>
ingress-nginx-admission-patch-7xgst       0/1     Completed   3          4d2h   192.168.235.197   k8s-master   <none>           <none>
ingress-nginx-controller-987b747f-xrzn8   1/1     Running     17         4d2h   10.244.169.139    k8s-node2    <none>           <none>
ingress-nginx-controller-ph46w            1/1     Running     0          163m   10.244.36.93      k8s-node1    <none>           <none>
ingress-nginx-controller-t2nxd            1/1     Running     0          163m   10.244.169.163    k8s-node2    <none>           <none>

将会有这么两个service：

注意31702这个端口，这个端口以后会常用的，这两个service是部署文件里产生的。

[root@master ~]# k get svc -n ingress-nginx 
NAME                                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.0.0.102   <none>        80:31702/TCP,443:31675/TCP   4d2h
ingress-nginx-controller-admission   ClusterIP   10.0.0.12    <none>        443/TCP                      4d2h

节点情况如下：

[root@master ~]# k get no -owide
NAME         STATUS   ROLES    AGE   VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION               CONTAINER-RUNTIME
k8s-master   Ready    <none>   36d   v1.18.3   192.168.217.16   <none>        CentOS Linux 7 (Core)   5.16.9-1.el7.elrepo.x86_64   docker://20.10.7
k8s-node1    Ready    <none>   36d   v1.18.3   192.168.217.17   <none>        CentOS Linux 7 (Core)   5.16.9-1.el7.elrepo.x86_64   docker://20.10.7
k8s-node2    Ready    <none>   36d   v1.18.3   192.168.217.18   <none>        CentOS Linux 7 (Core)   5.16.9-1.el7.elrepo.x86_64   docker://20.10.7

二，

编写ingress资源清单文件

vim ingress-http.yaml

（两个自定义域名nginx.test.com和tomcat.test.com 绑定到了nginx-service和tomcat-service 这两服务上了）

这里要注意了，为什么两个servicePort都是80了，因为上面的ingress-nginx-controller是80端口嘛，由于我的kubernetes版本是1.18，因此还是使用注解方式。

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-http
  namespace: dev
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: nginx.test.com
    http:
      paths:
      - path: /
        backend:
          serviceName: nginx-service
          servicePort: 80
  - host: tomcat.test.com
    http:
      paths:
      - path: /
        backend:
          serviceName: tomcat-service
          servicePort: 80

此文件执行过后，ingress的情况如下：

可以看到此ingress是绑定到了192.168.217.18也就是node2节点了，绑定了两个域名，ADDRESS是node2节点的IP。

[root@master ~]# k get ing -A
NAMESPACE   NAME           CLASS    HOSTS                            ADDRESS          PORTS   AGE
dev         ingress-http   <none>   nginx.test.com,tomcat.test.com   192.168.217.18   80      3h23m

建立namespace：

[root@master ~]# cat tomcat-nginx-ns.yaml 
apiVersion: v1
kind: Namespace
metadata:
  name: dev
---

vim tomcat-nginx.yaml

建立两个deployment的pod，可提供web功能的，一个nginx 一个tomcat，两个都做了node选择，和ingress-nginx-controller处于同一个节点。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-pod
  template:
    metadata:
      labels:
        app: nginx-pod
    spec:
      containers:
      - name: nginx
        image: nginx:1.17.1
        ports:
        - containerPort: 80
      nodeName: k8s-node2
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deployment
  namespace: dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tomcat-pod
  template:
    metadata:
      labels:
        app: tomcat-pod
    spec:
      containers:
      - name: tomcat
        image: tomcat:8.5-jre10-slim
        ports:
        - containerPort: 8080
      nodeName: k8s-node2

vim tomcat-nginx-svc.yaml

这里又需要注意了，两个service一个无头service，一个普通的clusterip，一会使用了ingress清单文件就可以将这两个服务发布到集群外了。

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  namespace: dev
spec:
  ports:
    - port: 80
      name: nginx
  clusterIP: None
  selector:
    app: nginx-pod
---
apiVersion: v1
kind: Service
metadata:
  name: tomcat-service
  namespace: dev
spec:
  selector:
    app: tomcat-pod
  type: ClusterIP
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080

三，

OK，以上文件都apply后，就可以看结果了：

宿主机做hosts域名解析：

 

OK，如果nginx部署到node1节点会怎么样呢？

报错504

 这就说明一个问题，kubectl get ingress -A 查询出来的那个IP地址也就是ingress的节点和要发布的service对应的pod要在同一个节点下，和service的类型没有关系，即使service是无头的也是OK的，ingress-nginx-controller会帮我们自己处理好的。并且多个service都是通过同一个端口发不出来的，只是域名不同而已。

四，

改造成https也就是使用ssl的域名（实验性质，当然还是使用自签的证书，实际生产环境肯定是使用备案过的证书哦）

a，

生成自签证书

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=tomcat.test.com/ST=BeiJing/L=BeiJing/O=devops/OU=unicorn"

可以看到，生成了这么两个玩意 

[root@master ~]# ls tls*
tls.crt  tls.key

b，

生成secret，证书存放到secret里

注意，这里要指定namespace，要不ingress controller找不到证书

kubectl create secret tls tls-secret --key=tls.key --cert tls.crt -n dev

c，

编写ingress文件

vim ingress-https.yaml

主要就是添加了tls相关，域名还是不变的以及一个注解，并且引用了前面打入的证书

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test-ingress3
  namespace: dev
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    kubernetes.io/ingress.class: nginx
#    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/ssl-redirect: 'true'
#    #    nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
  tls:
    - hosts:
      - tomcat.test.com
      secretName: tls-secret
  rules:
  - host: tomcat.test.com
    http:
      paths:
      - path: /
        backend:
          serviceName: tomcat-service
          servicePort: 80

查看ingress，可以看到多了一个443，还是绑定的node2节点（ingress 控制器前面部署的时候搞错了，就部署在了一个节点，daemonset没有使用的。）

[root@master ~]# k get ing -A
NAMESPACE   NAME            CLASS    HOSTS                            ADDRESS          PORTS     AGE
dev         ingress-http    <none>   nginx.test.com,tomcat.test.com   192.168.217.18   80        11h
dev         test-ingress3   <none>   tomcat.test.com                  192.168.217.18   80, 443   4m16s

d，

验证；

需要先查询一哈ingress的service提供的端口，查询出端口是31675

[root@master ~]# vim ingress-https.yaml
[root@master ~]# k get svc -A
NAMESPACE       NAME                                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                      AGE
default         kubernetes                           ClusterIP   10.0.0.1     <none>        443/TCP                      37d
dev             nginx-service                        ClusterIP   None         <none>        80/TCP                       16h
dev             tomcat-service                       ClusterIP   10.0.92.37   <none>        80/TCP                       16h
ingress-nginx   ingress-nginx-controller             NodePort    10.0.0.102   <none>        80:31702/TCP,443:31675/TCP   4d14h
ingress-nginx   ingress-nginx-controller-admission   ClusterIP   10.0.0.12    <none>        443/TCP                      4d14h
kube-system     coredns                              ClusterIP   10.0.0.2     <none>        53/UDP,53/TCP                36d

OK，https证书启用成功，此网站的证书只是没有注册的自产证书，但功能是完好的。 

总结：

那么现在这个ingress controller插件是可以使用的，ingress统一了要发布服务的端口，可以看到即使多个门户，也可以简单的以域名来区分，端口是统一的31702（http）或者31675（https），从而达到了服务治理的目的（其它功能，比如黑白名单，重定向，二级域名跳转等等留待以后研究哈）：

[root@master ~]# k get ing -A
NAMESPACE   NAME            CLASS    HOSTS                            ADDRESS          PORTS     AGE
dev         ingress-http    <none>   nginx.test.com,tomcat.test.com   192.168.217.18   80        11h
dev         test-ingress3   <none>   tomcat.test.com                  192.168.217.18   80, 443   4m16s

ingress-nginx-controller截取的部分日志：

192.168.217.18 - - [03/Oct/2022:09:03:03 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://nginx.test.com:31702/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36 Core/1.94.175.400 QQBrowser/11.1.5155.400" 423 0.001 [dev-nginx-service-80] [] 10.244.36.97:80 555 0.001 404 3b851fa39d2268e5fbd230fc5f8d1d59
192.168.217.18 - - [03/Oct/2022:09:03:05 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36 Core/1.94.175.400 QQBrowser/11.1.5155.400" 581 0.001 [dev-nginx-service-80] [] 10.244.36.97:80 0 0.001 304 21366d34e3103a50236738d6b1dd00e7
192.168.217.18 - - [03/Oct/2022:09:03:07 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36 Core/1.94.175.400 QQBrowser/11.1.5155.400" 581 0.002 [dev-nginx-service-80] [] 10.244.36.97:80 0 0.002 304 69bcb2182681ea3be696fe3b449e286c
192.168.217.18 - - [03/Oct/2022:09:03:09 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36 Core/1.94.175.400 QQBrowser/11.1.5155.400" 581 0.001 [dev-nginx-service-80] [] 10.244.36.97:80 0 0.000 304 3622a7bf35abbdf08c43084c89fd0110
192.168.217.18 - - [03/Oct/2022:09:06:15 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36 Core/1.94.175.400 QQBrowser/11.1.5155.400" 500 0.003 [dev-nginx-service-80] [] 10.244.36.97:80 612 0.003 200 5f6ba8d7070c9b7984acf2012fa57a5b
192.168.217.18 - - [03/Oct/2022:09:06:17 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Safari/537.36 Core/1.94.175.400 QQBrowser/11.1.5155.400" 581 0.002 [dev-nginx-service-80] [] 10.244.36.97:80 0 0.001 304 12d114732628f18df5988661bf79fc83
192.168.217.18 - - [03/Oct/2022:09:16:42 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36" 460 0.001 [dev-nginx-service-80] [] 10.244.36.97:80 612 0.001 200 298738d5c747741af42d8f13fb4c4566

可以看到我是使用的QQ浏览器（也用了谷歌105版本），192.168.217.18:31702 代理了10.244.36.97:80

NAMESPACE       NAME                                       READY   STATUS      RESTARTS   AGE     IP                NODE         NOMINATED NODE   READINESS GATES
dev             nginx-deployment-b785b4498-5r8jn           1/1     Running     0          16m     10.244.36.97      k8s-node1    <none>           <none>

# kubernetes # 云原生 # nginx # 运维 # 容器
Logo
Cloudpods

开源、云原生的融合云平台

更多推荐

面向未来的 IT 基础设施管理架构——融合云（Unified IaaS）

随着数字化时代的到来，IT系统已成为人类社会正常运转不可或缺的组成部分。不远的未来，智能制造，5G和人工智能等技术将成为推动生产力发展的重要引擎，人类社会将面临前所未有的全面彻底的数字化浪潮。IT基础设施作为IT系统运行的平台和载体，是实现数字化的基石。在这场数字化浪潮中，企业必须积极拥抱云计算技术，采用符合技术发展趋势、面向未来的IT基础构架，才能在未来的竞争中赢得先机。 一、云计算历经十余年

avatar Cloudpods

Cloudpods负载均衡的功能介绍

作者:周有松 今天的内容会从以下几个方面展开： 负载均衡产品简介。主要介绍负载均衡作为一个云上产品，它的功能模型是怎样的，日常使用中会遇到的业务词汇负载均衡的功能与典型应用场景。这部分主要结合业务词汇，对负载均衡服务中常见的一些功能选项进行介绍，并举例介绍一些典型的应用场景最后，我们做一下总结，讨论一下负载均衡产品相比传统方式的优点 一、产品简介​ 1. 以NGINX为例​ 提到负载均衡，我们以

avatar Cloudpods

使用Linux vfio将Nvidia GPU透传给QEMU虚拟机

Linux 上虚拟机 GPU 透传需要使用 vfio 的方式。主要是因为在 vfio 方式下对虚拟设备的权限和 DMA 隔离上做的更好。但是这么做也有个缺点，这个物理设备在主机和其他虚拟机都不能使用了。 qemu 直接使用物理设备本身命令行是很简单的，关键在于事先在主机上对系统、内核和物理设备的一些配置。 单纯从 qemu 的命令行来看，其实和普通虚拟机启动就差了最后那个-device的选项。这

avatar Cloudpods