coredns的部署

基于二进制kubernetes集群中的CoreDNS部署

wget -O coredns.yaml.sed https://raw.githubusercontent.com/coredns/deployment/master/kubernetes/coredns.yaml.sed

vim coredns.yaml
apiVersion:v1
kind:ServiceAccount
metadata:
name:coredns
namespace:kube-system
---
apiVersion:rbac.authorization.k8s.io/v1
kind:ClusterRole
metadata:
labels:
kubernetes.io/bootstrapping:rbac-defaults
name:system:coredns
rules:
-apiGroups:
-""
resources:
-endpoints
-services
-pods-namespaces
verbs:
-list
-watch
-apiGroups:-discovery.k8s.io
resources:
-endpointslices
verbs:
-list
-watch
---
apiVersion:rbac.authorization.k8s.io/v1
kind:ClusterRoleBinding
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate:"true"
labels:
kubernetes.io/bootstrapping:rbac-defaults
name:system:coredns
roleRef:
apiGroup:rbac.authorization.k8s.io
kind:ClusterRole
name:system:corednssubjects:
-kind:ServiceAccount
name:coredns
namespace:kube-system
---
apiVersion:v1
kind:ConfigMap
metadata:
name:coredns
namespace:kube-system
data:
Corefile:|
.:53{
errors
health{
lameduck5s
}
readyready
#此处CLUSTER_DOMAIN修改cluster.local,REVERSE_CIDRS修改为in-addr.arpa
ip6.arpa
kubernetes cluster.local in-addr.arpa ip6.arpa { #【修改1】替换CLUSTER_DOMAIN REVERSE_CIDRS
fallthroughin-addr.arpaip6.arpa
}
prometheus:9153
forward . /etc/resolv.conf { #【修改2】替换UPSTREAMNAMESERVER
#此处UPSTREAMNAMESERVER修改为
/etc/resolv.conf
max_concurrent1000
}
cache30
loop
reload
loadbalance
}STUBDOMAINS
---
apiVersion:apps/v1
kind:Deployment
metadata:
name:coredns
namespace:kube-systemlabels:
k8s-app:kube-dns
kubernetes.io/name:"CoreDNS"
spec:
#replicas:notspecifiedhere:
#1.Defaultis1.
#2.WillbetunedinrealtimeifDNShorizontalauto-scalingisturnedon.
replicas:2#添加此行为多副本,默认为1
strategy:
type:RollingUpdate
rollingUpdate:
maxUnavailable:1
selector:
matchLabels:
k8s-app:kube-dns
template:
metadata:
labels:
k8s-app:kube-dnsspec:
priorityClassName:system-cluster-critical
serviceAccountName:coredns
tolerations:
-key:"CriticalAddonsOnly"
operator:"Exists"
nodeSelector:
kubernetes.io/os:linux
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
-labelSelector:
matchExpressions:
-key:k8s-app
operator:In
values:["kube-dns"]
topologyKey:kubernetes.io/hostnamecontainers:
-name:coredns
image:coredns/coredns:1.9.4
imagePullPolicy:IfNotPresent
resources:
limits:
memory:170Mi#此处的资源限制修改为合适的值,比如:4096Mi
requests:
cpu:100m#此处的资源限制修改为合适的值
memory:70Mi#此处的资源限制修改为合适的值
args:["-conf","/etc/coredns/Corefile"]
volumeMounts:
-name:config-volume
mountPath:/etc/coredns
readOnly:true
ports:
-containerPort:53
name:dnsprotocol:UDP
-containerPort:53
name:dns-tcp
protocol:TCP-containerPort:9153
name:metrics
protocol:TCP
securityContext:
allowPrivilegeEscalation:false
capabilities:
add:
-NET_BIND_SERVICE
drop:
-all
readOnlyRootFilesystem:true
livenessProbe:
httpGet:
path:/health
port:8080
scheme:HTTP
initialDelaySeconds:60
timeoutSeconds:5successThreshold:1
failureThreshold:5
readinessProbe:
httpGet:
path:/ready
port:8181
scheme:HTTP
dnsPolicy:Default
volumes:
-name:config-volume
configMap:
name:coredns
items:
-key:Corefile
path:Corefile---
apiVersion:v1
kind:Service
metadata:
name:kube-dns
namespace:kube-system
annotations:
prometheus.io/port:"9153"
prometheus.io/scrape:"true"
labels:
k8s-app:kube-dns
kubernetes.io/cluster-service:"true"
kubernetes.io/name:"CoreDNS"
spec:
selector:
k8s-app:kube-dns
clusterIP:10.96.0.2 #【修改3】替换CLUSTER_DNS_IP
#修改此处为kube-dnsSVC的地址,比如:10.96.0.10,可通过查看
Pod的/etc/resolv.conf获取ports:
-name:dns
port:53
protocol:UDP
-name:dns-tcp
port:53
protocol:TCP-name:metrics
port:9153
protocol:TCP

#应用创建coredns的Pod

kubectl apply -f coredns.yaml

coredns解析内部服务的原理解析

1.Pod 域名请求发至 kube-dns ServiceIP，路由到 CoreDNS Pod。

2.CoreDNS 依靠 RBAC 权限持续监听 Service、EndpointSlice 资源。

3.kubernetes插件拦截.cluster.local域名：

     匹配内部服务域名，直接返回 ClusterIP / 后端 PodIP；

    fallthrough：反向解析查不到数据，则流转下一插件。

4.非集群域名（外网域名），流转至forward转发上游 DNS。

5.cache 缓存解析结果，loadbalance 随机打乱多记录实现简易轮询

pod配置

.:53 {
    errors
    health {
      lameduck 5s
    }
    ready
    kubernetes cluster.local in-addr.arpa ip6.arpa {
      fallthrough in-addr.arpa ip6.arpa
    }
    prometheus :9153
    forward . /etc/resolv.conf {
      max_concurrent 1000
    }
    cache 30
    loop
    reload
    loadbalance
}

.:53：监听全部域名，53 端口提供 DNS 服务

errors：打印 DNS 查询错误日志

health：提供健康检测接口；lameduck 5s，关闭前等待 5s 处理存量请求

ready：提供就绪探测接口

kubernetes：处理集群内部域名；cluster.local 集群域名后缀；支持正反解析；fallthrough 反向解析未命中交给后续插件

prometheus :9153：9153 端口输出监控指标

forward：外部域名转发上游 DNS，读取宿主机 resolv.conf；限制最大并发查询 1000

cache 30：解析结果缓存 30 秒

loop：检测阻断 DNS 循环转发

reload：Corefile 变更自动加载

loadbalance：多条 A 记录随机返回，实现简易轮询

案例

配置CoreDNS的域名解析转发至指定的DNS Server

#创建集群外的DNS

安装部署dns

#!/bin/bash
# install_dns.sh 安装bind9主DNS服务器 Ubuntu24.04
set -e

DOMAIN="wang.org"
NS_MASTER="master"
NS_IP="10.0.0.200"

# 1.安装软件
apt update
apt install -y bind9 bind9utils bind9-doc

# 2.修改全局配置 /etc/bind/named.conf.local
cat > /etc/bind/named.conf.local <<EOF
zone "$DOMAIN" {
        type master;
        file "/etc/bind/$DOMAIN.zone";
};
EOF

# 3.生成区域解析文件
cat > /etc/bind/${DOMAIN}.zone <<EOF
\$TTL 1D
@       IN SOA  $NS_MASTER admin (
        1       ; serial
        1D      ; refresh
        1H      ; retry
        1W      ; expire
        3H )    ; minimum
        NS      $NS_MASTER
$NS_MASTER     A       $NS_IP
www     A       10.0.0.200
harbor  A       10.0.0.100
test    A       6.6.6.6
a       A       1.1.1.1
b       A       2.2.2.2
EOF

# 4.权限
chown bind:bind /etc/bind/${DOMAIN}.zone

# 5.语法校验
named-checkconf
named-checkzone $DOMAIN /etc/bind/${DOMAIN}.zone

# 6.重启服务
systemctl restart bind9
systemctl enable bind9

echo "DNS bind9 部署完成！"
echo "测试命令：nslookup www.$DOMAIN 127.0.0.1"

bashinstall_dns.sh

cat > /etc/bind/wang.org.zone <<'EOF'
$TTL 1D
@       IN SOA  master admin (
    1       ; serial
    1D      ; refresh
    1H      ; retry
    1W      ; expire
    3H )    ; minimum
        NS      master
master  A       10.0.0.200
www     A       10.0.0.200
harbor  A       10.0.0.100
test    A       6.6.6.6
a       A       1.1.1.1
b       A       2.2.2.2
EOF
chown bind:bind /etc/bind/wang.org.zone

生效

rndc reload

.:53{
errors
health{
lameduck5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure  #【新增】允许解析pod主机域名
fallthrough in-addr.arpa ip6.arpa
ttl 30         #【新增】内部域名解析TTL设置30秒
}
prometheus :9153
forward wang.org 10.0.0.200{ #【新增】wang.org域名转发至自建DNS10.0.0.200
prefer_udp
}
forward . /etc/resolv.conf{
max_concurrent 1000
}
cache 30{ #【修改】扩展cache配置
disable success cluster.local  #关闭集群域名正向缓存
disable denial cluster.local    #关闭集群域名否定缓存
}
loop
reload
loadbalance
}

重新创建coredns的Pod生效

kubectl rollout restart deployment -n kube-system coredns

kubectl run client-test-$RANDOM --image registry.cn-beijing.aliyuncs.com/wangxiaochun/admin-box:v0.1 --restart=Never --rm -it --command -- /bin/bash

root@client-test-908/# host a.wang.org
a.wang.org has address 1.1.1.1
root@client-test-908/# host b.wang.org
b.wang.org has address 2.2.2.2
#复现异常
Host b.wang.org not found: 3(NXDOMAIN)
Host a.wang.org not found: 3(NXDOMAIN)

部署ingress nginx控制器

基于YAML部署

基于kubectl apply部署

[root@master1~]#VERSION=1.15.1
wget https://raw.githubusercontent.com/kubernetes/ingress-
nginx/controller-v${VERSION}/deploy/static/provider/cloud/deploy.yaml

#修改三处image地址指向国内镜像地址

[root@master1~]#vim deploy.yaml

#1 controller
#image:registry.k8s.io/ingress-nginx/controller:v11.14.3@sha256:ac444cd9515af325ba577b596fe4f27a34be1aa330538e8b317ad9d6c8fb94ee
image:registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.14.3

#2 certgen第一处
#image:registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.6.7@sha256:7a38cf0f8480775baaee71ab519c7465fd1dfeac66c421f28f087786e631456e
image:registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.6.7

#3 certgen第二处
#image:registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.6.7@sha256:7a38cf0f8480775baaee71ab519c7465fd1dfeac66c421f28f087786e631456e
image:registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.6.7

应用资源配置文件

[root@master1~]#kubectl apply -f deploy.yaml

查看ingress-nginx-controller的Pod运行在哪个节点上

[root@master1~]#kubectl get pod -A -o wide | grep ingress-nginx-controller

确认效果

[root@master1~]#kubectl get ns

生成IngressClass

[root@master1~]#kubectl get ingressclass

查看secret

[root@master1~]#kubectl get secrets -n ingress-nginx

[root@master1~]#kubectl get secrets -n ingress-nginx ingress-nginx-admission -
o yaml

kubectl get ep -n ingress-nginx
NAME                                          ENDPOINTS                               AGE
ingress-nginx-controller                      10.244.1.126:443,10.244.1.126:80        13m
ingress-nginx-controller-admission            10.244.1.126:8443                       13m

curl 10.0.0.101:32699
#返回：400 Bad Request，明文HTTP请求访问HTTPS端口，流量到达ingress-nginx

直接获取状态码

curl -I -o /dev/null -s -w %{http_code}"\n" 10.0.0.101:32453
返回状态码：404
#流量抵达ingress-nginx，无匹配Ingress规则

查看访问日志

kubectl logs -f -n ingress-nginx ingress-nginx-controller-548b8cb9b8-tsxpl
#作用：实时查看ingress-nginx控制器运行日志，排查访问异常

案例

子URL访问

命令式实现单域名支持子URL

#添加/*，支持子URL，如果有URL则转发至Pod对应相同的URL

kubectl create ingress demo-ingress \
--rule="hao123.com/*=my-svc:80" \
--class=nginx \
-o yaml --dry-run=client

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  creationTimestamp: null
  name: demo-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: hao123.com
    http:
      paths:
      - path: /(.*)
        pathType: Prefix
        backend:
          service:
            name: my-svc
            port:
              number: 80
status:
  loadBalancer: {}

创建

kubectl create ingress demo-ingress \
--rule="hao123.com/*=pod-test1:80" \
--class=nginx

kubectl get ingress -n default

查看

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  creationTimestamp: "2021-05-28T01:36:20Z"
  generation: 1
  name: demo-ingress
  namespace: default
  resourceVersion: "1502354"
  uid: 0938aeda-0ea6-4235-a032-3a6adb0e2174
spec:
  ingressClassName: nginx
  rules:
  - host: hao123.com
    http:
      paths:
      - backend:
          service:
            name: pod-test1
            port:
              number: 80
        path: /(.*)
        pathType: Prefix
status:
  loadBalancer: {}

kubectl get ingress demo-ingress -o yaml

客户端访问测试

curl -H "host:hao123.com" 10.0.0.99
curl -H "host:hao123.com" 10.0.0.99/hostname

蓝绿发布

基于 Service Selector 的蓝绿发布

当前在线版本：蓝（Blue）

新版本：绿（Green）

部署新版本 Pod（绿），和老版本（蓝）同时运行

测试验证绿色版本业务正常

修改 Service，流量全部切到绿色

业务稳定后，按需删除蓝色旧版本

核心：靠 Service 标签选择器切换流量，无需 Ingress 改动

环境说明

现有业务：pod-test1（蓝色老版本） 新建绿色版本：pod-test2（新版本）

Service 名称：svc-test Ingress：

demo-ingress 绑定 svc-test

蓝部署 yaml blue.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: pod-test-blue
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: pod-test
      version: blue
  template:
    metadata:
      labels:
        app: pod-test
        version: blue
    spec:
      containers:
      - name: pod-test
        image: xxx:v1   # 老镜像v1
        ports:
        - containerPort: 80

部署

kubectl apply -f blue.yaml

Service（初始指向蓝版本）svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: svc-test
  namespace: default
spec:
  selector:
    app: pod-test
    version: blue    # 当前流量走向：蓝
  ports:
  - port: 80
    targetPort: 80
  type: ClusterIP

kubectl apply -f svc.yaml

Ingress（之前配置好的，不用改动）

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-ingress
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: hao123.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: svc-test
            port:
              number: 80

kubectl apply -f ingress-demo.yaml

客户端 → Ingress → svc-test → label version=blue Pod（老版本）

部署绿色新版本 green.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: pod-test-green
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: pod-test
      version: green
  template:
    metadata:
      labels:
        app: pod-test
        version: green
    spec:
      containers:
      - name: pod-test
        image: xxx:v2   # 新版本镜像v2
        ports:
        - containerPort: 80

kubectl apply -f green.yaml

此时：蓝 Pod、绿 Pod同时运行，但是流量依旧全部走蓝。

单独测试绿色版本

流量切换：修改 Service 标签，切到绿色

编辑 svc.yaml，selector 改为 version: green

spec:
  selector:
    app: pod-test
    version: green

更新生效

kubectl apply -f svc.yaml

验证业务

curl -H "Host:hao123.com" 10.0.0.101:32453

两种情况

情况 A：新版本一切正常 → 清理旧版本

kubectl delete -f blue.yaml

情况 B：新版本出问题 → 快速回滚

kubectl apply -f svc.yaml

快速查看状态

# 查看两个deployment
kubectl get deploy
# 查看service标签选择器
kubectl describe svc svc-test
# 查看哪些Pod被service选中
kubectl get endpoints svc-test

Ingress 切换后端实现蓝绿发布

逻辑：两套独立 Deployment + 两个独立 Service svc-blue（老版本 / 蓝）、svc-green（新版本 / 绿）Service、Deployment 不动，只修改 Ingress 后端指向谁实现流量切换

Ingress 始终绑定同一个域名，切换不用改客户端

客户端 → hao123.com(Ingress) →【切换目标】svc-blue /svc-green → Pod

蓝环境（老版本）blue.yaml

apiVersion: apps/v1
kind: Deployment
name: pod-test-blue
spec:
  replicas: 2
  selector:
    matchLabels:
      app: pod-test
      ver: blue
  template:
    metadata:
      labels:
        app: pod-test
        ver: blue
    spec:
      containers:
      - name: web
        image: v1镜像
        ports:
        - containerPort:80

kubectl apply -f blue.yaml

蓝对应的 Service svc-blue.yaml

apiVersion: v1
kind: Service
name: svc-blue
spec:
  selector:
    app: pod-test
    ver: blue
  ports:
  - port:80
    targetPort:80

kubectl apply -f svc-blue.yaml

绿环境（新版本）green.yaml

apiVersion: apps/v1
kind: Deployment
name: pod-test-green
spec:
  replicas: 2
  selector:
    matchLabels:
      app: pod-test
      ver: green
  template:
    metadata:
      labels:
        app: pod-test
        ver: green
    spec:
      containers:
      - name: web
        image: v2新版本镜像
        ports:
        - containerPort:80

kubectl apply -f green.yaml

绿对应的 Service svc-green.yaml

apiVersion: v1
kind: Service
name: svc-green
spec:
  selector:
    app: pod-test
    ver: green
  ports:
  - port:80
    targetPort:80

kubectl apply -f svc-green.yaml

初始 Ingress：流量指向蓝（正式对外使用）ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
name: demo-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: hao123.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: svc-blue   # 当前指向蓝
            port:
              number: 80

验证

curl -H "Host:hao123.com" 10.0.0.101:32453

测试新版本

kubectl create ingress test-green --rule="test.hao123.com/*=svc-green:80" --class=nginx

流量全量切换到绿色（修改 Ingress service 名称）

修改 ingress.yaml 中 service.name: svc-blue → svc-green

backend:
  service:
    name: svc-green
    port:
      number: 80

更新生效

kubectl apply -f ingress.yaml

验证流量已经切新版本

curl -H "Host:hao123.com" 10.0.0.101:32453

故障回滚操作

把 Ingress 改回 svc-blue，再次 apply，立刻切回老版本

业务稳定后清理

kubectl delete -f blue.yaml
kubectl delete -f svc-blue.yaml

两种蓝绿方案对比

修改 Service selector 切换流量

优点：Ingress 固定不变
缺点：只能一个 Service

修改 Ingress 后端切换 Service

优点：两套 Service 完全隔离；多域名、多路径场景更灵活
缺点：需要维护两个 Service