概念

ingress: 是k8s内部的一个资源对象
ingress controller ingress控制器： 是k8s里启动的一个pod，运行的是nginx的镜像，实现k8s内部的service（ClusterIP类型）的负载均衡
ingress 和ingress controller 的关系:
ingress controller 本质上是一个nginx软件，用来做负载均衡ingress 是k8s内部管理nginx配置（nginx.conf）的组件，用来给ingress controller传参
参考安装部署文档：
https://kubernetes.github.io/ingress-nginx/deploy/
使用旧版本ingress controller v1.1完成安装

准备工作

需要提前上传下面的这些镜像和yaml文件到k8s集群里的linux系统里，建议存放到master节点上，然后再scp到node节点上

[root@master .kube]# mkdir /ingress
[root@master .kube]# cd /ingress/
[root@master ingress]# ls
ingress-controller-deploy.yaml         nfs-pvc.yaml                       sc-ingress.yaml
ingress-nginx-controllerv1.1.0.tar.gz  nfs-pv.yaml                        sc-nginx-svc-1.yaml
kube-webhook-certgen-v1.1.0.tar.gz     nginx-deployment-nginx-svc-2.yaml

ingress-controller-deploy.yaml ：是部署ingress controller使用的yaml文件
ingress-nginx-controllerv1.1.0.tar.gz ： ingress-nginx-controller镜像
kube-webhook-certgen-v1.1.0.tar.gz ：kube-webhook-certgen镜像
sc-ingress.yaml 创建ingress的配置文件
sc-nginx-svc-1.yaml ： 启动sc-nginx-svc服务和相关pod的yaml
nginx-deployment-nginx-svc-2.yaml ： 启动sc-nginx-svc-2服务和相关pod的yaml

步骤1： 安装ingress controller

1.将镜像scp到所有的node节点服务器上

[root@master ingress]# scp ingress-nginx-controllerv1.1.0.tar.gz node-1:/root
ingress-nginx-controllerv1.1.0.tar.gz                                               100%  276MB  42.7MB/s   00:06    
[root@master ingress]# scp ingress-nginx-controllerv1.1.0.tar.gz node-2:/root
ingress-nginx-controllerv1.1.0.tar.gz                                               100%  276MB  45.7MB/s   00:06    
[root@master ingress]# scp kube-webhook-certgen-v1.1.0.tar.gz node-2:/root
kube-webhook-certgen-v1.1.0.tar.gz                                                  100%   47MB  40.5MB/s   00:01    
[root@master ingress]# scp kube-webhook-certgen-v1.1.0.tar.gz node-1:/root
kube-webhook-certgen-v1.1.0.tar.gz                                                  100%   47MB  47.1MB/s   00:00    
[root@master ingress]#

2.导入镜像，在所有的节点服务器（node-1和node-2）上进行

[root@k8smaster ingress]# docker load -i ingress-nginx-controllerv1.1.0.tar.gz 
[root@k8smaster ingress]# docker load -i kube-webhook-certgen-v1.1.0.tar.gz
[root@k8snode2 ~]# docker images
REPOSITORY                                                                     TAG        IMAGE ID       CREATED         SIZE
registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller   v1.1.0     ae1a7201ec95   16 months ago   285MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen       v1.1.1     c41e9fcadf5a   17 months ago   47.7MB
[root@k8smaster new]#

3.使用ingress-controller-deploy.yaml 文件去启动ingress controller

[root@k8smaster 4-4]# ls
ingress-controller-deploy.yaml         kube-webhook-certgen-v1.1.0.tar.gz  sc-ingress.yaml
ingress-nginx-controllerv1.1.0.tar.gz  nginx-deployment-nginx-svc-2.yaml   sc-nginx-svc-1.yaml
[root@k8smaster 4-4]#

4.执行ingress-controller-deploy.yaml 文件，去启动ingress controller

[root@k8smaster 4-4]# kubectl apply -f ingress-controller-deploy.yaml 
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created

5.查看ingress controller的相关命名空间

[root@k8smaster 4-4]# kubectl get ns
NAME              STATUS   AGE
default           Active   11d
ingress-nginx     Active   52s
kube-node-lease   Active   11d
kube-public       Active   11d
kube-system       Active   11d
sanchuang         Active   10d
[root@k8smaster 4-4]#

6.查看ingress controller的相关service

[root@k8smaster 4-4]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.99.160.10   <none>        80:30092/TCP,443:30263/TCP   91s
ingress-nginx-controller-admission   ClusterIP   10.99.138.23   <none>        443/TCP                      91s
[root@k8smaster 4-4]#

7.查看ingress controller的相关pod

[root@k8smaster 4-4]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-k69t2        0/1     Completed   0          119s
ingress-nginx-admission-patch-zsrk8         0/1     Completed   1          119s
ingress-nginx-controller-6c8ffbbfcf-bt94p   1/1     Running     0          119s
ingress-nginx-controller-6c8ffbbfcf-d49kx   1/1     Running     0          119s
[root@k8smaster 4-4]#

步骤2： 创建pod和暴露pod的服务

1.执行sc-nginx-svc-1.yaml文件

root@k8smaster 4-4]# cat sc-nginx-svc-1.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sc-nginx-deploy
  labels:
    app: sc-nginx-feng
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sc-nginx-feng
  template:
    metadata:
      labels:
        app: sc-nginx-feng
    spec:
      containers:
      - name: sc-nginx-feng
        image: nginx
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name:  sc-nginx-svc
  labels:
    app: sc-nginx-svc
spec:
  selector:
    app: sc-nginx-feng
  ports:
  - name: name-of-service-port
    protocol: TCP
    port: 80
    targetPort: 80
[root@k8smaster 4-4]# 
[root@k8smaster 4-4]# kubectl apply -f sc-nginx-svc-1.yaml 
deployment.apps/sc-nginx-deploy created
service/sc-nginx-svc created
[root@k8smaster 4-4]# kubectl get svc
NAME           TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes     ClusterIP   10.96.0.1      <none>        443/TCP   11d
sc-nginx-svc   ClusterIP   10.109.14.56   <none>        80/TCP    4s
[root@k8smaster 4-4]#

2.查看服务器的详细信息，查看Endpoints对应的pod的ip和端口是否正常

[root@k8smaster 4-4]# kubectl describe svc sc-nginx-svc
Name:              sc-nginx-svc
Namespace:         default
Labels:            app=sc-nginx-svc
Annotations:       <none>
Selector:          app=sc-nginx-feng
Type:              ClusterIP
IP Families:       <none>
IP:                10.109.14.56
IPs:               10.109.14.56
Port:              name-of-service-port  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.185.253:80,10.244.185.254:80,10.244.249.24:80
Session Affinity:  None
Events:            <none>
[root@k8smaster 4-4]# curl 10.109.14.56  访问服务暴露的ip
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@k8smaster 4-4]#

步骤3： 启用ingress 关联ingress controller 和service

1.执行sc-ingress.yaml文件

[root@k8smaster 4-4]# cat sc-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sc-ingress
  annotations:
    kubernets.io/ingress.class: nginx #注释 这个ingress 是关联ingress controller的
spec:
  ingressClassName: nginx  #关联ingress controller
  rules:
  - host: www.feng.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: sc-nginx-svc
            port:
              number: 80
  - host: www.zhang.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: sc-nginx-svc-2
            port:
              number: 80
[root@k8smaster 4-4]# 
[root@k8smaster 4-4]# kubectl apply -f sc-ingress.yaml   执行文件
ingress.networking.k8s.io/sc-ingress created
[root@k8smaster 4-4]# kubectl get ingress  查看效果
NAME         CLASS   HOSTS                        ADDRESS   PORTS   AGE
sc-ingress   nginx   www.feng.com,www.zhang.com             80      8s
[root@k8smaster 4-4]# 
过几分钟可以看到 有宿主机的ip地址
[root@k8smaster 4-4]# kubectl get ingress
NAME         CLASS   HOSTS                        ADDRESS                       PORTS   AGE
sc-ingress   nginx   www.feng.com,www.zhang.com   192.168.2.222,192.168.2.223   80      52s
[root@k8smaster 4-4]#

步骤4： 查看ingress controller 里的nginx.conf 文件里是否有ingress对应的规则

1.查看业务Pod的名字

[root@k8smaster 4-4]# kubectl get pod -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-k69t2        0/1     Completed   0          25m
ingress-nginx-admission-patch-zsrk8         0/1     Completed   1          25m
ingress-nginx-controller-6c8ffbbfcf-bt94p   1/1     Running     0          25m
ingress-nginx-controller-6c8ffbbfcf-d49kx   1/1     Running     0          25m
[root@k8smaster 4-4]#

2.进入ingress controller对应的pod里查看nginx.conf的配置

[root@k8smaster 4-4]# kubectl exec -n ingress-nginx -it ingress-nginx-controller-6c8ffbbfcf-bt94p -- bash
bash-5.1$ 
bash-5.1$ cat nginx.conf|grep zhang.com
	## start server www.zhang.com
		server_name www.zhang.com ;
	## end server www.zhang.com
bash-5.1$ cat nginx.conf|grep feng.com
	## start server www.feng.com
		server_name www.feng.com ;
	## end server www.feng.com

bash-5.1$ cat nginx.conf|grep -C3 upstream_balancer
	error_log  /var/log/nginx/error.log notice;
	
	upstream upstream_balancer {
		server 0.0.0.1:1234; # placeholder
		
		balancer_by_lua_block {

3.获取ingress controller对应的service暴露宿主机的端口

访问宿主机和相关端口，就可以验证ingress controller是否能进行负载均衡

[root@k8smaster 4-4]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.99.160.10   <none>        80:30092/TCP,443:30263/TCP   37m
ingress-nginx-controller-admission   ClusterIP   10.99.138.23   <none>        443/TCP                      37m
[root@k8smaster 4-4]#

在其他的宿主机（nfs服务器上）或者windows机器上使用域名进行访问

[root@nfs-server ~]# vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.2.222 www.feng.com
192.168.2.223 www.zhang.com

4.访问域名

因为我们是基于域名做的负载均衡的配置，所有必须要在浏览器里使用域名去访问，不能使用ip地址
同时ingress controller做负载均衡的时候是基于http协议的，7层负载均衡

[root@nfs-server ~]# curl  www.feng.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@nfs-server ~]# 

访问www.zhang.com出现异常，503错误，是nginx内部错误

[root@nfs-server ~]# curl  www.zhang.com   
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx</center>
</body>
</html>
[root@nfs-server ~]#

步骤5：启动第2个服务和pod，使用了pv+pvc+nfs

需要提前准备好nfs服务器

1.创建pv和pvc

[root@k8smaster 4-4]# ls
ingress-controller-deploy.yaml         nfs-pvc.yaml                       sc-ingress.yaml
ingress-nginx-controllerv1.1.0.tar.gz  nfs-pv.yaml                        sc-nginx-svc-1.yaml
kube-webhook-certgen-v1.1.0.tar.gz     nginx-deployment-nginx-svc-2.yaml
[root@k8smaster 4-4]# 

[root@k8smaster 4-4]# cat nfs-pv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: sc-nginx-pv
  labels:
    type: sc-nginx-pv
spec:
  capacity:
    storage: 10Gi 
  accessModes:
    - ReadWriteMany
  storageClassName: nfs
  nfs:
    path: "/web"       #nfs共享的目录
    server: 192.168.203.131   #nfs服务器的ip地址
    readOnly: false
[root@k8smaster 4-4]#

提醒：需要检查下nfs服务器是否启动nfs服务，并且设置nfs服务开机启动

[root@nfs-server ~]# service nfs restart
Redirecting to /bin/systemctl restart nfs.service
[root@nfs-server ~]# systemctl enable nfs
[root@nfs-server ~]# 

[root@k8smaster 4-4]# kubectl apply -f nfs-pv.yaml 
persistentvolume/sc-nginx-pv configured
[root@k8smaster 4-4]# 
创建pvc
[root@k8smaster 4-4]# cat nfs-pvc.yaml 
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: sc-nginx-pvc
spec:
  accessModes:
  - ReadWriteMany      
  resources:
     requests:
       storage: 1Gi
  storageClassName: nfs #使用nfs类型的pv
[root@k8smaster 4-4]# 

[root@k8smaster 4-4]# kubectl apply -f nfs-pvc.yaml 
persistentvolumeclaim/sc-nginx-pvc unchanged
[root@k8smaster 4-4]# 

[root@k8smaster 4-4]# kubectl get pv
NAME             CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                   STORAGECLASS   REASON   AGE
sc-nginx-pv      10Gi       RWX            Retain           Bound    default/sc-nginx-pvc    nfs                     20h
task-pv-volume   10Gi       RWO,RWX        Retain           Bound    default/task-pv-claim   manual                  24h
[root@k8smaster 4-4]# kubectl get pvc
NAME            STATUS   VOLUME           CAPACITY   ACCESS MODES   STORAGECLASS   AGE
sc-nginx-pvc    Bound    sc-nginx-pv      10Gi       RWX            nfs            20h
task-pv-claim   Bound    task-pv-volume   10Gi       RWO,RWX        manual         24h
[root@k8smaster 4-4]#

2.准备启动第2个service和pod

[root@k8smaster 4-4]# cat nginx-deployment-nginx-svc-2.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sc-nginx-feng-2
  template:
    metadata:
      labels:
        app: sc-nginx-feng-2
    spec:
      volumes:
        - name: sc-pv-storage-nfs
          persistentVolumeClaim:
            claimName: sc-nginx-pvc
      containers:
        - name: sc-pv-container-nfs
          image: nginx
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 80
              name: "http-server"
          volumeMounts:
            - mountPath: "/usr/share/nginx/html"
              name: sc-pv-storage-nfs
---
apiVersion: v1
kind: Service
metadata:
  name:  sc-nginx-svc-2
  labels:
    app: sc-nginx-svc-2
spec:
  selector:
    app: sc-nginx-feng-2
  ports:
  - name: name-of-service-port
    protocol: TCP
    port: 80
    targetPort: 80
[root@k8smaster 4-4]#

3.应用

[root@k8smaster 4-4]# kubectl apply -f nginx-deployment-nginx-svc-2.yaml 
deployment.apps/nginx-deployment created
service/sc-nginx-svc-2 created
[root@k8smaster 4-4]#

[root@k8smaster 4-4]# kubectl get svc
NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes       ClusterIP   10.96.0.1        <none>        443/TCP   11d
sc-nginx-svc     ClusterIP   10.109.14.56     <none>        80/TCP    48m
sc-nginx-svc-2   ClusterIP   10.104.244.189   <none>        80/TCP    10s
[root@k8smaster 4-4]# 

[root@k8smaster 4-4]# kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.99.160.10   <none>        80:30092/TCP,443:30263/TCP   58m
ingress-nginx-controller-admission   ClusterIP   10.99.138.23   <none>        443/TCP                      58m
[root@k8smaster 4-4]# kubectl get ingress
NAME         CLASS   HOSTS                        ADDRESS                       PORTS   AGE
sc-ingress   nginx   www.feng.com,www.zhang.com   192.168.2.222,192.168.2.223   80      36m
[root@k8smaster 4-4]#

访问宿主机暴露的端口号30092或者80都可以
使用ingress controller暴露服务，感觉不需要使用30000以上的端口访问，可以直接访问80或者443，比使用service 暴露服务还是有点优势的

[root@nfs-server ~]# curl  www.zhang.com
welcome to sanchuang
welcome to sanchuang  teacherfeng nfs  pv pvc
[root@nfs-server ~]# 

[root@nfs-server ~]# curl  www.feng.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@nfs-server ~]#

搞定

# kubernetes # 负载均衡 # 容器
Logo
K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 |  韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号​前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。​长夜有穷，真水无香。领略过了 K8s“身在江

avatar K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub

avatar K8S/Kubernetes

k8s 火了！

2020，上云之年，产品云端化成为一种趋势。在一线城市，很多公司都已经构建了自己的私有云环境，比如阿里云、网易云、华为云等。而Kubernetes 作为基于容器编排领域的王者，具备扩展...

avatar K8S/Kubernetes