k8s 7层路由机制:Ingress

根据前面对 Service 的使用说明，我们知道 Service 的表现形式为IP:Port，工作在TCP/IP层.对于基于 HTTP 的服务来说，不同的URL地址经常对应到不同的后端服务或者虚拟服务器，这些应用层的转发机制仅通过kubernetes的Service机制是无法实现的。kubernetes V1.1版本中新增Ingress将不同URL的访问请求转发到后端不同的Service，以实现H

zhangshaohuas

1340人浏览 · 2020-07-26 19:44:11

zhangshaohuas · 2020-07-26 19:44:11 发布

根据前面对 Service 的使用说明，我们知道 Service 的表现形式为IP:Port，工作在TCP/IP层.

对于基于 HTTP 的服务来说，不同的URL地址经常对应到不同的后端服务或者虚拟服务器，这些应用层的转发机制仅通过kubernetes的Service机制是无法实现的。

kubernetes V1.1版本中新增Ingress将不同URL的访问请求转发到后端不同的Service，以实现HTTP层的业务路由机制。

Ingress由两部分组成：Ingress Controller 和 Ingress 策略定义。
在这里插入图片描述

对 https://mywebsite.com/api的访问将被路由到后端名为"api"的 Service。
对https://mywebsite.com/web的访问将被路由到后端名为"web"的 Service。
对https:/mywebsite.com/doc的访问将被路由到后端名为"doc"的 Service。

下面通过一个例子分三步说明Ingress Controller和Ingress 策略和客户端如何访问 Ingress 提供的服务。

1 创建Ingress Controller和默认的backend服务

在定义Ingress策略之前，需要先部署Ingress Controller，为所有后端Service都提供一个统一的入口。

Ingress Controller需要实现基于不同HTTP URL向后转发的负载分发规则，并可以灵活设置7层负载分发策略。

如果公有云服务商能够提供该类型的HTTP路由LoadBalancer，则也可设置其为Ingress Controller。

在Kubernetes中，Ingress Controller将以Pod的形式运行，监控API Server的/ingress接口后端的backend services，如果Service发生变化，则Ingress Controller应自动更新其转发规则。

下面的例子使用Nginx来实现一个Ingress Controller，需要实现的基本逻辑如下。

监听API Server，获取全部Ingress的定义
基于Ingress的定义，生成Nginx所需的配置文件/etc/nginx/nginx.conf。
执行nginx -s reload命令，重新加载nginx.conf配置文件的内容。

本例使用谷歌提供的nginx-ingress-controller镜像来创建Ingress Controller。该Ingress Controller以daemonset的形式进行创建，在每个Node上都将启动一个Nginx服务。

这里为Nginx容器设置了hostNetwork:true，将容器应用监听的80和443端口号映射到物理机上，使得客户端应用可以通过URL地址“ http://物理机IP:80 ”或“ https://物理机IP:443 ”来访问该Ingress Controller。

最新版本地址：https://github.com/kubernetes/ingress-nginx/tree/master/deploy

指定版本号下载：https://github.com/kubernetes/ingress-nginx/tree/nginx-0.23.0/deploy

部署文件介绍：

namespace.yaml
创建独立的命名空间 ingress-nginx
configmap.yaml
rbac.yaml
负责Ingress的RBAC授权的控制，其创建了Ingress用到的ServiceAccount、ClusterRole、Role、RoleBinding、ClusterRoleBinding
with-rbac.yaml
使用带rbac的方式创建ingress-controller，是整个ingress的核心部署文件。
mandatory.yaml
该文件整合了前面四项文件的内容，是用于实际部署ingress服务的yaml文件。即只需要使用该文件就可以完成ingress-controller的全部部署工作。

下载ingress相关的yaml

-rw-r--r--. 1 root root      7868 7月  27 07:51 mandatory.yaml

vim mandatory.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    app.kubernetes.io/name: default-http-backend
    app.kubernetes.io/part-of: ingress-nginx
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: default-http-backend
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: default-http-backend
        app.kubernetes.io/part-of: ingress-nginx
    spec:
      terminationGracePeriodSeconds: 60
      containers:
        - name: default-http-backend
          # Any image is permissible as long as:
          # 1. It serves a 404 page at /
          # 2. It serves 200 on a /healthz endpoint
          image: registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/defaultbackend-amd64:1.5
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            timeoutSeconds: 5
          ports:
            - containerPort: 8080
          resources:
            limits:
              cpu: 10m
              memory: 20Mi
            requests:
              cpu: 10m
              memory: 20Mi

---
apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: default-http-backend
    app.kubernetes.io/part-of: ingress-nginx
spec:
  ports:
    - port: 80
      targetPort: 8080
  selector:
    app.kubernetes.io/name: default-http-backend
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      hostNetwork: true ##重点
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: registry.cn-qingdao.aliyuncs.com/kubernetes_xingej/nginx-ingress-controller:0.20.0
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 33
            runAsUser: 33
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80  ##重点
            - name: https
              containerPort: 443  ##重点
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1

---

执行mandatory.yaml:

[root@k8s-master ingress-nginx]# kubectl apply -f mandatory.yaml

节点端口监听：

[root@k8s-node1 tmp]# netstat  -lptnu|egrep -w "80|443"
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      95925/nginx: master 
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      95925/nginx: master 
tcp6       0      0 :::443                  :::*                    LISTEN      95925/nginx: master 
tcp6       0      0 :::80                   :::*                    LISTEN      95925/nginx: master

查看资源创建情况：

[root@k8s-master src]# kubectl get pod -n ingress-nginx
NAME                                    READY   STATUS    RESTARTS   AGE
default-http-backend-6c7b756d58-chg9b   1/1     Running   0          28m
nginx-ingress-controller-jrhxt          1/1     Running   0          28m
nginx-ingress-controller-wxfz9          1/1     Running   0          28m

[root@k8s-master src]# kubectl get svc -n ingress-nginx
NAME                   TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
default-http-backend   ClusterIP   10.1.13.163   <none>        80/TCP    29m

[root@k8s-node2 ~]# curl 10.1.13.163
default backend - 404

打开浏览器验证，正常:

在这里插入图片描述

创建后端服务

这里我们已nginx为服务为例，创建一个nginx和跟nginx对应的service，这里要注意metadata.name要和后面创建的ingress中的serviceName一致，切记！

[root@master myself]# cat mypod.yaml 
apiVersion: v1
kind: Service
metadata:
  name: service-nginx
  namespace: default
spec:
  selector:
    app: mynginx
  ports:
  - name: http
    port: 80
    targetPort: 80

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mydepoy
  namespace: default
spec:
  replicas: 5
  selector:
    matchLabels:
      app: mynginx
  template:
    metadata:
      labels:
        app: mynginx
    spec:
      containers:
      - name: mycontainer
        image: lizhaoqwe/nginx:v1
        imagePullPolicy: IfNotPresent
        ports:
        - name: nginx 
          containerPort: 80

[root@k8s-master src]# kubectl get pod 
NAME                      READY   STATUS    RESTARTS   AGE
mydepoy-5c657dbc7-4hft6   1/1     Running   0          161m
mydepoy-5c657dbc7-b5krn   1/1     Running   0          161m
mydepoy-5c657dbc7-kpj8c   1/1     Running   0          161m
mydepoy-5c657dbc7-pnscj   1/1     Running   0          161m
mydepoy-5c657dbc7-rdsv4   1/1     Running   0          161m

创建ingress规则

[root@master myself]# cat my-ingress.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-mynginx
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: mynginx.fengzi.com
    http:
      paths:
      - path:
        backend:
          serviceName: service-nginx
          servicePort: 80

我们可以去查看nginx的配置文件，去查看我们所创建的规则有没有注入到ingress中

[root@k8s-master src]# kubectl get ingress 
NAME              HOSTS                ADDRESS   PORTS   AGE
ingress-mynginx   mynginx.fengzi.com             80      22m

查看ingress 规则详情：

[root@k8s-master src]# kubectl describe ing ingress-mynginx
Name:             ingress-mynginx
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host                Path  Backends
  ----                ----  --------
  mynginx.fengzi.com  
                         service-nginx:80 (10.244.1.18:80,10.244.1.19:80,10.244.1.20:80 + 2 more...)

[root@k8s-master src]# curl 10.244.1.18:80
v1

主机hosts绑定：

192.168.1.7 k8s-master mynginx.fengzi.com
192.168.1.8 k8s-node1 mynginx.fengzi.com
192.168.1.9 k8s-node2 mynginx.fengzi.com

验证：

[root@k8s-node1 tmp]# curl mynginx.fengzi.com
v1

实验二：
访问 ingress.wg007.com/demo的请求将被转发到集群中的Service webapp 即 webapp:8080/demo

前提条件：成功部署tomcat+mysql

设置ingress 规则：

vim tomcat-ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata: 
  name: ingress.wg007.com   
spec:       
  rules:    # 重点
  - host: ingress.wg007.com   
    http:  
      paths: 
      - path: /demo 
        backend:
          serviceName: myweb
          servicePort: 8080

hosts绑定：

192.168.1.7 ingress.wg007.com
192.168.1.8 ingress.wg007.com

浏览器访问：（先hosts绑定哦）
在这里插入图片描述
成功了！！

K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 | 韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。长夜有穷，真水无香。领略过了 K8s“身在江

K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub