1、ingress介绍

K8s集群对外暴露服务的方式目前只有三种:
LoadblancerNodeportingress
前两种熟悉起来比较快,而且使用起来也比较方便,在此就不进行介绍了。

下面详细讲解下ingress这个服务,ingress由两部分组成:

  1. ingress controller:将新加入的Ingress转化成Nginx的配置文件并使之生效
  2. ingress服务:将Nginx的配置抽象成一个Ingress对象,每添加一个新的服务只需写一个新的Ingress的yaml文件即可

其中ingress controller目前主要有两种:基于nginx服务的ingress controller基于traefik的ingress controller
而其中traefik的ingress controller,目前支持http和https协议。由于对nginx比较熟悉,而且需要使用TCP负载,所以在此我们选择的是基于nginx服务的ingress controller。
但是基于nginx服务的ingress controller根据不同的开发公司,又分为k8s社区的ingres-nginxnginx公司的nginx-ingress
在此根据github上的活跃度和关注人数,我们选择的是k8s社区的ingres-nginx

k8s社区提供的ingress,github地址如下:https://github.com/kubernetes/ingress-nginx
nginx社区提供的ingress,github地址如下:https://github.com/nginxinc/kubernetes-ingress

2、ingress的工作原理

ingress具体的工作原理如下:
step1:ingress contronler通过与k8s的api进行交互,动态的去感知k8s集群中ingress服务规则的变化,然后读取它,并按照定义的ingress规则,转发到k8s集群中对应的service。
step2:而这个ingress规则写明了哪个域名对应k8s集群中的哪个service,然后再根据ingress-controller中的nginx配置模板,生成一段对应的nginx配置。
step3:然后再把该配置动态的写到ingress-controller的pod里,该ingress-controller的pod里面运行着一个nginx服务,控制器会把生成的nginx配置写入到nginx(/etc/nginx.conf)的配置文件中,然后reload一下,使其配置生效,以此来达到域名分配置及动态更新的效果。

3、ingress可以解决的问题

1)动态配置服务

如果按照传统方式, 当新增加一个服务时, 我们可能需要在流量入口加一个反向代理指向我们新的k8s服务. 而如果用了Ingress, 只需要配置好这个服务, 当服务启动时, 会自动注册到Ingress的中, 不需要而外的操作。

2)减少不必要的端口暴露

配置过k8s的都清楚, 第一步是要关闭防火墙的, 主要原因是k8s的很多服务会以NodePort方式映射出去, 这样就相当于给宿主机打了很多孔, 既不安全也不优雅. 而Ingress可以避免这个问题, 除了Ingress自身服务可能需要映射出去, 其他服务都不要用NodePort方式。

4、部署ingress(deployment的方式)

1)配置文件准备(粘贴下面网址的yanl文件)

github连接地址

https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml

2)配置文件介绍和设置ingress的网络模式

1> namespace.yaml

创建一个独立的命名空间 ingress-nginx。

2> configmap.yaml

ConfigMap是存储通用的配置变量的,类似于配置文件,使用户可以将分布式系统中用于不同模块的环境变量统一到一个对象中管理;而它与配置文件的区别在于它是存在集群的“环境”中的,并且支持K8S集群中所有通用的操作调用方式。
从数据角度来看,ConfigMap的类型只是键值组,用于存储被Pod或者其他资源对象(如RC)访问的信息。这与secret的设计理念有异曲同工之妙,主要区别在于ConfigMap通常不用于存储敏感信息,而只存储简单的文本信息。
ConfigMap可以保存环境变量的属性,也可以保存配置文件。
创建pod时,对configmap进行绑定,pod内的应用可以直接引用ConfigMap的配置。相当于configmap为应用/运行环境封装配置。
pod使用ConfigMap,通常用于:设置环境变量的值、设置命令行参数、创建配置文件。

3> default-backend.yaml  这里没有这个文件,需要我们自己添加,在下面会有添加

如果外界访问的域名不存在的话,则默认转发到default-http-backend这个Service,其会直接返回404。

4> rbac.yaml

负责Ingress的RBAC授权的控制,其创建了Ingress用到的ServiceAccount、ClusterRole、Role、RoleBinding、ClusterRoleBinding。

5> with-rbac.yaml

是Ingress的核心,用于创建ingress-controller。前面提到过,ingress-controller的作用是将新加入的Ingress进行转化为Nginx的配置。

6> 各文件的作用:

configmap.yaml:提供configmap可以在线更新nginx的配置
default-backend.yaml:提供一个缺省的后台错误页面 404
namespace.yaml:创建一个独立的命名空间 ingress-nginx
rbac.yaml:创建对应的role rolebinding 用于rbac
tcp-services-configmap.yaml:修改L4负载均衡配置的configmap
udp-services-configmap.yaml:修改L4负载均衡配置的configmap
with-rbac.yaml:有应用rbac的nginx-ingress-controller组件

 

部署完整文件mandatory.yaml这个没有defaultbackend-amd64

[root@localhost ingress]# cat mandatory.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: Deployment
#kind: DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      #hostNetwork: true
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        kubernetes.io/os: linux
        #custem/ingress-controller-ready: "true"
      containers:
        - name: nginx-ingress-controller
          image: harbor.superred.com/kubernetes/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
              #hostPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              #hostPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

apiVersion: v1
kind: LimitRange
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  limits:
  - min:
      memory: 90Mi
      cpu: 100m
    type: Container

---

apiVersion: v1
kind: Service
metadata:
  namespace: ingress-nginx
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http #服务为http
      port: 80 #service端口为80
      targetPort: 80 #容器端口为80
      protocol: TCP
      nodePort: 30080
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
      nodePort: 30443
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  #externalTrafficPolicy: Cluster|Locale

部署完整文件mandatory.yaml这个有defaultbackend-amd64   添加- --default-backend-service=$(POD_NAMESPACE)/default-http-backend 参数

[root@localhost ingress]# cat mandatory.yaml.all 
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: Deployment
#kind: DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      #hostNetwork: true
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        kubernetes.io/os: linux
        #custem/ingress-controller-ready: "true"
      containers:
        - name: nginx-ingress-controller
          image: harbor.superred.com/kubernetes/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
              #hostPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              #hostPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

apiVersion: v1
kind: LimitRange
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  limits:
  - min:
      memory: 90Mi
      cpu: 100m
    type: Container

---

apiVersion: v1
kind: Service
metadata:
  namespace: ingress-nginx
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http #服务为http
      port: 80 #service端口为80
      targetPort: 80 #容器端口为80
      protocol: TCP
      nodePort: 30080
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
      nodePort: 30443
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  #externalTrafficPolicy: Cluster
---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    app: default-http-backend
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: default-http-backend
  template:
    metadata:
      labels:
       app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        image:  harbor.superred.com/kubernetes/defaultbackend-amd64:1.5
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
         # 这里调整了cpu和memory的大小,可能不同集群限制的最小值不同,看部署失败的原因就清楚
          limits:
            cpu: 100m
            memory: 100Mi
          requests:
            cpu: 100m
            memory: 100Mi
---
apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  # namespace: ingress-nginx
  namespace: ingress-nginx
  labels:
    app: default-http-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: default-http-backend

 直接部署在裸机上时要额外加上一项叫做service-nodeport.yaml,如果不加上这一项会发现ingress controller部署完以后在集群内部可以被访问,在集群外部是无法被访问到的,因为ingress controller无法接入外部的流量,如果要接入流量则需要再部署一个nodePort的service,或者也可以将ingress controller部署为直接共享节点网络名称空间的方式,但这个时候我们需要手动改造with-rbac.yaml这个文件中的kind 类型由Deployent改为DaemonSet,然后去掉replicas,然后在pod template上的spec中加一项hostnetwork用于共享宿主机的网络名称空间,当然需要保证监听的端口不被冲突。

[root@localhost ingress]# cat service-nodeport.yaml 
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http #服务为http
      port: 80 #service端口为80
      targetPort: 80 #容器端口为80
      protocol: TCP
      nodePort: 30080
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
      nodePort: 30443
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  #externalTrafficPolicy: Cluster|Locale

DaemonSet方式的

[root@localhost ingress]# cat mandatory.yaml.DaemonSet
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
#kind: Deployment
kind: DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
#  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      hostNetwork: true
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        kubernetes.io/os: linux
        custem/ingress-controller-ready: "true"
      containers:
        - name: nginx-ingress-controller
          image: harbor.superred.com/kubernetes/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
              hostPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              hostPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

apiVersion: v1
kind: LimitRange
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  limits:
  - min:
      memory: 90Mi
      cpu: 100m
    type: Container

---

apiVersion: v1
kind: Service
metadata:
  namespace: ingress-nginx
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http #服务为http
      port: 80 #service端口为80
      targetPort: 80 #容器端口为80
      protocol: TCP
      nodePort: 30080
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
      nodePort: 30443
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  #externalTrafficPolicy: Cluster|Locale

部署ingress(DaemonSet的方式)

官方原始文件使用的是deployment,replicate 为 1,这样将会在某一台节点上启动对应的nginx-ingress-controller pod。外部流量访问至该节点,由该节点负载分担至内部的service。测试环境考虑防止单点故障,改为DaemonSet然后删掉replicate ,配合亲和性部署在制定节点上启动nginx-ingress-controller pod,确保有多个节点启动nginx-ingress-controller pod,后续将这些节点加入到外部硬件负载均衡组实现高可用性

1)添加hostNetwork

true:添加该字段,暴露nginx-ingress-controller pod的服务端口(80)

2)添加亲和性属性

增加亲和性部署,有custom/ingress-controller-ready 标签的节点才会部署该DaemonSet

3)设置节点的label

kubectl label nodes 10.10.3.170 custem/ingress-controller-ready=true
kubectl label nodes 10.10.3.167 custem/ingress-controller-ready=true

4)指定运行节点(需要打标签)

nginx-ingress-controller会随意选择一个node节点运行pod,为此需要我们把nginx-ingress-controller运行到指定的node节点上
首先需要给需要运行nginx-ingress-controller的node节点打标签,在此我们把nginx-ingress-controller运行在指定的node节点上

kubectl get nodes –show-labels         #先查看各个node上的lables信息
kubectl label nodes 10.10.3.170 nginx=nginx  #给node3打标签
kubectl get nodes –show-labels         #查看node3上的lables

kubectl label nodes 10.10.3.170 nginx-  删除标签

 5)

在这里插入图片描述

6)修改配置文件

设置 hostNetwork: true
由于ingress 使用到物理机的80/443 端口,所以需要设置为hostNetwork模式
在这里插入图片描述

 

 

部署

[root@localhost ingress]# kubectl create -f mandatory.yaml
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
limitrange/ingress-nginx created
service/ingress-nginx created

部署情况

Every 2.0s: kubectl get pods,svc,ingress,secrets -o wide --all-namespaces                                                                                                              localhost.localdomain: Fri Feb 19 10:28:35 2021

NAMESPACE              NAME                                             READY   STATUS    RESTARTS   AGE   IP             NODE          NOMINATED NODE   READINESS GATES
ingress-nginx          pod/nginx-ingress-controller-68bb844f96-d5x4q    1/1     Running   0          25s   10.244.20.18   10.10.3.167   <none>           <none>
ingress-nginx          pod/nginx-ingress-controller-68bb844f96-p7rdj    1/1     Running   0          25s   10.244.40.9    10.10.3.170   <none>           <none>
kube-system            pod/coredns-654979db4b-hxq9g              	1/1     Running   0          9d    10.244.38.2    10.10.3.178   <none>           <none>
kube-system            pod/coredns-654979db4b-vknrv                     1/1     Running   0          9d    10.244.45.2    10.10.3.177   <none>           <none>
kube-system            pod/metrics-server-v0.3.6-785d474994-npkhr       2/2     Running   0          8d    10.244.40.4    10.10.3.170   <none>           <none>
kubernetes-dashboard   pod/dashboard-metrics-scraper-775b89678b-77rgh   1/1     Running   0          9d    10.244.40.2    10.10.3.170   <none>           <none>
kubernetes-dashboard   pod/kubernetes-dashboard-66d54d4cd7-6fgn9        1/1     Running   0          9d    10.244.20.9    10.10.3.167   <none>           <none>

NAMESPACE              NAME                                TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                      AGE   SELECTOR
default                service/kubernetes                  ClusterIP   10.0.0.1     <none>        443/TCP                      9d    <none>
ingress-nginx          service/ingress-nginx               NodePort    10.0.0.41    <none>        80:30080/TCP,443:30443/TCP   25s   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
kube-system            service/kube-dns                    ClusterIP   10.0.0.2     <none>        53/UDP,53/TCP,9153/TCP       9d    k8s-app=kube-dns
kube-system            service/metrics-server              ClusterIP   10.0.0.53    <none>        443/TCP                      8d    k8s-app=metrics-server
kubernetes-dashboard   service/dashboard-metrics-scraper   NodePort    10.0.0.130   <none>        8000:30002/TCP               9d    k8s-app=dashboard-metrics-scraper
kubernetes-dashboard   service/kubernetes-dashboard        NodePort    10.0.0.156   <none>        443:30001/TCP                9d    k8s-app=kubernetes-dashboard

NAMESPACE              NAME                                              TYPE                                  DATA   AGE
default                secret/default-token-th2qw                        kubernetes.io/service-account-token   3      9d
default                secret/tomcat-secret                              kubernetes.io/tls                     2      19m
ingress-nginx          secret/default-token-7ll9f                        kubernetes.io/service-account-token   3      25s
ingress-nginx          secret/nginx-ingress-serviceaccount-token-gsp2m   kubernetes.io/service-account-token   3      25s
kube-node-lease        secret/default-token-rg6l9                        kubernetes.io/service-account-token   3      9d
kube-public            secret/default-token-w8rxx                        kubernetes.io/service-account-token   3      9d
kube-system            secret/coredns-token-jt5vg                        kubernetes.io/service-account-token   3      9d
kube-system            secret/dashboard-admin-token-gcvkm                kubernetes.io/service-account-token   3      9d
kube-system            secret/default-token-wdtw8                        kubernetes.io/service-account-token   3      9d
kube-system            secret/metrics-server-token-r57vr                 kubernetes.io/service-account-token   3      8d
kubernetes-dashboard   secret/default-token-xvfcr                        kubernetes.io/service-account-token   3      9d
kubernetes-dashboard   secret/kubernetes-dashboard-certs                 Opaque                                0      9d
kubernetes-dashboard   secret/kubernetes-dashboard-csrf                  Opaque                                1      9d
kubernetes-dashboard   secret/kubernetes-dashboard-key-holder            Opaque                                2      9d
kubernetes-dashboard   secret/kubernetes-dashboard-token-wr68s           kubernetes.io/service-account-token   3      9d
monitoring             secret/default-token-r2fw2                        kubernetes.io/service-account-token   3      8d
monitoring             secret/prometheus-operator-token-nbn8k            kubernetes.io/service-account-token   3      8d

验证 ingress 是否配置成功 (访问一个不存在的规则) 

此时访问我们节点端口显示  404 ,此时表示我们调度器已经正常工作 ,提示404,这个因为当前ingress-nginx服务现在还没有后端服务,这是正常的,到此Ingress控制器部署成功!!!

二、部署Tomcat后端服务来验证Ingress

1、创建tomcat部署文件

[root@localhost ingress]# cat tomcat-deploy.yaml 
apiVersion: v1
kind: Service
metadata:
  name: tomcat
  namespace: default
spec:
  selector:
    app: tomcat
    release: canary
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  - name: ajp
    port: 8009
    targetPort: 8009

---
apiVersion: apps/v1
kind: Deployment
metadata: 
  name: tomcat-deploy
spec:
  replicas: 3
  selector: 
    matchLabels:
      app: tomcat
      release: canary
  template:
    metadata:
      labels:
        app: tomcat
        release: canary
    spec:
      containers:
      - name: tomcat
        image: harbor.superred.com/superredtools/tomcat:7-alpine
        ports:
        - name: httpd
          containerPort: 8080
        - name: ajp
          containerPort: 8009

 

[root@localhost ingress]# kubectl create -f tomcat-deploy.yaml 
service/tomcat created
deployment.apps/tomcat-deploy created




[root@localhost ingress]# kubectl get pods,svc,ingress,secrets -o wide --all-namespaces
NAMESPACE              NAME                                             READY   STATUS    RESTARTS   AGE     IP             NODE          NOMINATED NODE   READINESS GATES
default                pod/tomcat-deploy-7b6c9df96d-49x9l               1/1     Running   0          39s     10.244.45.6    10.10.3.177   <none>           <none>
default                pod/tomcat-deploy-7b6c9df96d-6nr4w               1/1     Running   0          39s     10.244.38.10   10.10.3.178   <none>           <none>
default                pod/tomcat-deploy-7b6c9df96d-jg4bz               1/1     Running   0          39s     10.244.20.19   10.10.3.167   <none>           <none>
ingress-nginx          pod/nginx-ingress-controller-68bb844f96-d5x4q    1/1     Running   0          6m53s   10.244.20.18   10.10.3.167   <none>           <none>
ingress-nginx          pod/nginx-ingress-controller-68bb844f96-p7rdj    1/1     Running   0          6m53s   10.244.40.9    10.10.3.170   <none>           <none>
kube-system            pod/coredns-654979db4b-hxq9g                     1/1     Running   0          9d      10.244.38.2    10.10.3.178   <none>           <none>
kube-system            pod/coredns-654979db4b-vknrv                     1/1     Running   0          9d      10.244.45.2    10.10.3.177   <none>           <none>
kube-system            pod/metrics-server-v0.3.6-785d474994-npkhr	2/2     Running   0          8d      10.244.40.4    10.10.3.170   <none>           <none>
kubernetes-dashboard   pod/dashboard-metrics-scraper-775b89678b-77rgh   1/1     Running   0          9d      10.244.40.2    10.10.3.170   <none>           <none>
kubernetes-dashboard   pod/kubernetes-dashboard-66d54d4cd7-6fgn9        1/1     Running   0          9d      10.244.20.9    10.10.3.167   <none>           <none>

NAMESPACE              NAME                                TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                      AGE     SELECTOR
default                service/kubernetes                  ClusterIP   10.0.0.1     <none>        443/TCP                      9d      <none>
default                service/tomcat                      ClusterIP   10.0.0.141   <none>        8080/TCP,8009/TCP            39s     app=tomcat,release=canary
ingress-nginx          service/ingress-nginx               NodePort    10.0.0.41    <none>        80:30080/TCP,443:30443/TCP   6m53s   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx

2、Http访问验证

2.1、将tomcat添加至ingress-nginx中

[root@localhost ingress]# cat ingress-tomcat.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat
  namespace: default
  annotations: 
    kubernets.io/ingress.class: "nginx"
spec:
  rules:
  - host: tomcat.cclinux.com.cn  #将域名与node IP 绑定写入访问节点hosts文件
    http:
      paths:
      - path: 
        backend:
          serviceName: tomcat
          servicePort: 8080
[root@localhost ingress]# kubectl apply -f ingress-tomcat.yaml  #运行Yaml文件
[root@localhost ingress]#  kubectl get ingress  #检查生成的ingress信息

NAMESPACE   NAME                                CLASS    HOSTS          ADDRESS   PORTS   AGE
default     ingress.extensions/ingress-tomcat   <none>   tomcat.cclinux.com.cn   10.0.0.36   80      2m52s

2.2、将域名解析写入访问节点hosts文件

 

2.3、验证tomcat服务

浏览器打开http://tomcat.cclinux.com.cn:30080

 

http访问成功!!!

3、Https访问验证

3.1、创建私有证书及secret

root@ingress-nginx]# openssl genrsa -out tls.key 2048  #执行成功后,该目录下会生成tls.key文件

[root@k8s-node01]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=BJ/L=BJ/O=k8s/CN=tomcat.cclinux.com.cn   #注意域名要和服务的域名一致 ,域名必须为提供服务的域名, 执行成功后,该目录下会生成tls.crt文件
[root@localhost ingress]# ls
ingress-tomcat-tls.yaml  ingress-tomcat.yaml  mandatory.yaml  mandatory.yaml.all  mandatory.yaml.back  service-nodeport.yaml  tls.crt  tls.key  tomcat-deploy.yaml

tls.crt:公钥
tls.key:私钥

您可以通过指定包含TLS私钥和证书的秘密来保护ingress。目前,入口只支持一个TLS端口443,并假设TLS终端。如果一个入口中的TLS配置部分指定了不同的主机,那么它们将根据通过SNI TLS扩展指定的主机名在同一个端口上进行多路复用(前提是入口控制器支持SNI)。TLS密钥必须包含名为TLS的密钥。crt和tls。包含用于TLS的证书和私钥的密钥,例如 

[root@ingress-nginx]# kubectl create secret tls tomcat-secret --cert=tls.crt --key=tls.key  #创建名称为tomcat-secret的secret文件
[root@ingress-nginx]# kubectl get secrets   #检查创建的secret文件:

default                secret/tomcat-secret                              kubernetes.io/tls                     2      31m

或 

apiVersion: v1
kind: Secret
metadata:
  name: tomcat-secret
  namespace: default
data:
  tls.crt: base64 encoded cert
  tls.key: base64 encoded key
type: kubernetes.io/tls

 

3.2、将证书应用至tomcat服务中:

在一个Ingress中引用这个secret将告诉Ingress控制器使用TLS保护从客户机到负载均衡器的通道。您需要确保您创建的TLS secret来自一个包含tomcat.cclinux.com.cn CN的证书。 域名cclinux.com.cn已经在华为云申请完毕,可以使用 一年50多元钱

[root@localhost ingress]# cat ingress-tomcat-tls.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat-tls
  namespace: default
  annotations: 
    kubernets.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - tomcat.cclinux.com.cn      #与secret证书的域名需要保持一致
    secretName: tomcat-secret  #secret证书的名称
  rules:
  - host: tomcat.cclinux.com.cn  #将域名与node IP 绑定写入访问节点hosts文件
    http:
      paths:
      - path: 
        backend:
          serviceName: tomcat
          servicePort: 8080
[root@ingress-nginx]# kubectl apply -f ingress-tomcat-tls.yaml   #运行该文件
[root@ingress-nginx]# kubectl get ingress  #检查生成的ingress信息


NAMESPACE   NAME                                    CLASS    HOSTS          ADDRESS     PORTS     AGE
default     ingress.extensions/ingress-tomcat       <none>   www.bobo.com   10.0.0.41   80        6m3s
default     ingress.extensions/ingress-tomcat-tls   <none>   www.bobo.com   10.0.0.41   80, 443   70s

3.3、Https访问tomcat服务,

 

https访问成功!!!如果想不提示不安全字样,需要用到https://blog.csdn.net/Michaelwubo/article/details/113307318 文件介绍

nginx部署

[root@localhost ingress]# cat nginx-deploy.yaml 
apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  selector:
    app: nginx
  ports:
  - name: http
    port: 80
    targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata: 
  name: nginx-deploy
spec:
  replicas: 1
  selector: 
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: harbor.superred.com/superredtools/nginx:wubo1
        ports:
        - name: httpd
          containerPort: 80

nginx容器是修改过的,修改内容如下 

root@nginx-deploy-5c8dccb448-c2bxc:/# cat /etc/nginx/conf.d/default.conf 
server {
    listen       80;
    listen  [::]:80;
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }
    location /wubo {
        root   /usr/share/nginx/html;
        index  wubo.html;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}

root@nginx-deploy-5c8dccb448-c2bxc:/# cat /usr/share/nginx/html/wubo/wubo.html 
wubo

 ingress文件 path:后面的路径一定是app真实的访问路径才可以

[root@localhost ingress]# cat ingress-tomcat.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat
  namespace: default
  annotations: 
    kubernetes.io/ingress.class: "nginx"
    #ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: tomcat.cclinux.com.cn  #将域名与node IP 绑定写入访问节点hosts文件
    http:
      paths:
      - path: /
        backend:
          serviceName: tomcat
          servicePort: 8080
      - path: /wubo
        backend:
          serviceName: nginx
          servicePort: 80

浏览器访问可以了,同一个域名通过不同的path区分不同的app应用 

 

介绍

 

Kubernetes 提供了两种内建的云端负载均衡机制( cloud load balancing )用于发布公共应用, 一种是工作于传输层的 Service 资源,它实现的是 TCP 负载均衡器”,另一种是Ingress 资源,它 现的是“ HTTP(S )负载均衡器”

 

( 1) TCP 载均衡器

无论是 iptables 还是ipvs 模型的 Service 资源都配置于 Linux 内核中的 Netfilter 之上进行四层调度,是一种类型更为通用的调度器, 支持调度 HTTP、MySQL 应用层服务。不过,也正是由于 作于传输层从而使得它无法做到类似卸载 HTTPS 中的 SSL 会话等一类作,也不支持基于 URL 的请求调度机制,而且, Kubernetes 也不支持为此类负载均衡器配置任何类型的健康状态检查机制

( 2) HTTP(S )负载均衡器

HTTP(S )负载均衡器是应用层负载均衡机制的 种,支持根据环境做出更好的调度决策。与传输层调度器相比,它提供了诸如可自定义 URL 映射和 TLS 卸载 功能,并支持多种类型 的后端服务器健康状态检查机制

 

Ingress和Ingess Controller

 

Kubernetes 中, Service 资源和 Pod 资源的 IP 地址仅能用于集群网络内部的通信,所有的网络流量都无法穿透边界路由器( Edge Rout )以 现集群内外通信。尽管可以为Service 使用 NodePort LoadBalancer 类型通过节点引人外部流 ,但它依然是4层流量流发,可用的负载均衡器也为传输层负载均衡机制。

 

Ingress是Kubernetes API 的标准资源类型之一 ,它其实就是一组基于DNS 名称(host)或URL 路径把请求转发至指定的 Service 资源的规则,用于将集群外部的请求流 转发至集群内部完成服务发布,然而 Ingress 资源自身并不能进行“流量穿透”,它仅是一组路由规则的集合,这些规则要想真正发挥作用还需要其他功能的辅助,如监听某套接字,然后根据这些规则的匹配机制路由请求流量,这种能够为 Ingress 资源监听套接字并转发流量的组件称为 Ingress 控制器( Ingress Controller)

注意:

不同于 Deployment 控制器等,Ingress 控制器并不直接运行为 kube-controller-rnanager 的一部分,它是Kubemetes 集群的一个重要附件,类似于 CoreDNS,需要在集群上单独部署

 

 

Ingress 控制器可以由任何具有反向代理( HTTP HTTPS )功能的服务程序 现,如Nginx,Envoy,HAProxy ,Vulcand,Traefik 等。 Ingress 控制器自身也是运行于集群中Pod 资源对象,它与被代理的运行为 Pod 资源的应用运行于同一网络中,如图 ingress-nginx与pod1,pod3 等的关系所示:

另一方面,使用 Ingress 资源进行流量分发时, Ingress 控制器可基于某 Ingress 资源定义的规则将客户端的请求流 直接转发至与 Service 应的后端 Pod 资源之上,这种转发机制会绕过 service 资源,从而省去了由 kube proxy实现的端口代理开销 如上图所示,Ingress 规则需要由一个 Service 资源对象辅助识别相关的所有 Pod 象,但 ingress-nginx控制器可经由 api.ilinux.io 规则的定义直接将请求流量调度至 pod3或者pod4 ,而无须经由Service对象API 再次转发, WAP关规则的作用方式与此类同。

创建 Ingress 资源

Ingress 资源是基于 HTTP 虚拟主机或 URL 的转发规则, 它在资源配置清单的 spec中嵌套了rules、backend、tls 等字段进行定义。下面的示例中定义了 Ingress 资源,它包含了一个转发规则,把发往 www. ilinux.io 请求代理给名为 myapp-svc的Service 资源:

上面资源 单中的 annotations 用于识别其所属的 Ingress 控制器的类别,这一点在集群上部署有多个Ingress 控制器时尤为重要。Ingress Spec 中的字段定义 Ingress 资源的核心组成部分,它主要嵌套如下三个字段:

  •  rules <Object> :用于定义当前Ingress 资源的转发规则列表;未由 rules 定义规则,或者没有匹配到任何规则时,所有流量都会转发到由 backend 定义的默认后端
  • backend <Object>: 默认的后端用于服务那些没有匹配到任何规则的请求;定义Ingress 源时,至少应该定义 backend或者rules 两者之一;此字段用于让负载均衡器指定一个全局默认的后端。
  • tls <Object> : TLS ,目前仅支持通过 认端口 443 提供服务;如果要配置指定的列表成员指向了不同的主机,则必须通过 SNI TLS 扩展机制来支持此功能

 

backend 对象的定义由两个必选的内嵌 段组成: serviceName和servicePort ,分别用于指定流量转发的后端目标 Service 资源的名称和端口。

rules 对象由一系列配置Ingress资源的 host 规则组成 这些 host 规则用于将一个主机上的某个URL 路径映射至相关的后端 Service对象, 它的定义格式如下:

注意 .spec.rules.host属性值目前不 持使用 IP 地址,也不 持后跟“:PORT ”格端口号,且此字段值留空表示通配所有的主机名

tis 对象由两个内嵌字段组成,仅在定义 TLS 主机的转发规则时才需要定义此 对象。

  • host :包含于使用的 TLS 证书之内的主机名称字符串列表,因此,此处使用的主机

名必须匹配 tlsSecret 中的名称

  • secretName :用于引用 SSL 会话的 secret 象名 称,在基于 SNI 实现多主机路由的

场景中,此字段为可选

Ingress 资源类型

 

基于 HTTP 暴露的每个 Service 资源均 发布于一个独立的 FQDN 主机名之上,如”www.ik8s.io”;也可发布于某主机的 URL 路径之上,从而将它们整合到同一个 Web 点,如“ www.ik8s.io grafana “,至于是否需要发布为 HTTPS 类型 的应用则取决于用户的业务需求。

1.单Service 资源型 Ingress

暴露单个服务的方法有很多种 ,如服务类型中的 NodePort,LoadBalancer 等,不过一样可以考虑使用 Ingress 来暴露服务,此时只需要为 Ingress 指定“ default backend” 即可如下面的 例:

Ingress 控制器会为其分配一个 IP 地址接入请求流量,并将它们转至示例中的 my-svc后端

2.基于URL 路径进行流量分发

垂直拆分或微服务架构中,每个小的应用都有其专用的 Service 资源暴露服务,但在对外开放的站点上,它们可能是财经,新闻,电商、无线端或 API 接口 等一类的独立应用,可通过主域名 URL 路径( path )分别接入,例如, www.ilinux.io/api www.ilinux.io/wap,用 于发布集群内名称为 API和WAP的Services 资源。于是,可对应地创建一个如下的Ingress 资源, 它将对www.ilinux.io/api 的请求统统转发至 API Service 资源,将对www.ilinux.io/wap 的请求转发至 WAP Service 资源. 前提是真实的访问路径必须是带有wap或api的(也就是web项目的访问路径 )context path一定有wap或api

3.基于主机名称的虚拟主机

上面类型2中描述的需求,也可以将每个应用分别以独立的 FQDN 主机名进行输出,如:wap.ik8s.io和api.ik8s.io ,这两个主机名解析到 external LB (如上图所示)的 IP 地址之上,分别用于发布集群 部的 WAP和API 两个 Servic 资源 这种实现方案其实就是Web 站点部署中的 “基于主 机名的虚拟主机”,将多个 FQDN 解析至同一个 IP 地址, 然后根据“主机头”( Host header )进行转发。下面是以独立 FQDN 主机形式发布服务的 Ingress资源示例:

 

4.TLS 类型的 Ingress 资源

这种类型用于以 HTTPS发布Service 资源, 基于一个含有私钥和证书的 Secret对象,即可配 TLS 协议的 Ingress 资源,目前来说, Ingress 资源仅支持单 TLS 端口, 并且还会卸载 TLS会话。在 Ingress 资源中引用此 Secret 即可让 Ingress控制器加载并配置为 HTTPS服务。

下面是一个简单的 TLS 型的 Ingress 资源示例:

部署 Ingress 控制器( Nginx)

 

Ingress 控制器自身是运行 Pod 中的容器应用,一般是 Nginx或Envoy -类的具有代理及负载均衡功能的守护进程,它监视着来自于API Server的Ingress 对象状态,并以其规则生成相应的应用程序专有格式的配置文件并通过重载或开启守护进程而使新配置生效。例如,对于 Nginx 来说,Ingress 规则需要转换为 Nginx 的配置信息,简单来说, Ingress 控制器其实就是托管于Kubemetes 系统之上的用于实现在应用层发布服务的Pod 资源,它将跟跟踪Ingress 资源并实时生成配置规则。那么,同样运行为 Pod 资源的 Ingress 控制器进程又该如何接入外部的请求流量那? 常用的解决方案有如下两种:

  1. 以 Deployment 控制器管理Ingress 控制器的 Pod 资源,并通过 NodePort或Load Balancer 类型的 Service 对象为其接入集群外部的请求流量,这就意味着,定义一个Ingress 控制器时,必须在其前端定义一个专用的 Service 资源:如图 所示

2.借助于 DaemonSet 控制器,将 Ingress控制器的 Pod 资源各自以单一实例的方式运行于集群的所有或部分工作节点之上,并配置这类Pod对象以hostPort(如图a)或 hostNetwork (如图b)的方式在当前节点接入外部流量

 

 

 

 

 

 

 

 

 

 

 

https://blog.csdn.net/m0_37939350/article/details/108087673

https://www.cnblogs.com/panwenbin-logs/p/9915927.html

https://www.cnblogs.com/crazymagic/p/11267303.html

https://blog.csdn.net/weixin_44729138/article/details/105978555

https://www.cnblogs.com/zisefeizhu/p/13478746.html

http://www.kendd.cn/?p=2492

 

Logo

开源、云原生的融合云平台

更多推荐