52.k8s的controller管理-4
·
第4部：controller核心管理
控制器controller
1：controller作用及分类
controller用于控制pod
官方文档：https://kubernetes.io/zh-cn/docs/concepts/workloads/controllers/
控制器主要分为:
- Deployments 部署无状态应用，控制pod升级,回退
- ReplicaSet 副本集,控制pod扩容,裁减
- ReplicationController (相当于ReplicaSet的老版本,现在建议使用Deployments加ReplicaSet替代 RC)
- StatefulSets 部署有状态应用，结合Service、存储等实现对有状态应用部署
- DaemonSet 守护进程集，运行在所有集群节点(包括master),比如使用filebeat,node_exporter
- Jobs 一次性
- Cronjob 周期性
2：Deployment
Replicaset控制器的功能
- 支持新的基于集合的selector(以前的rc里没有这种功能)
- 通过改变Pod副本数量实现Pod的扩容和缩容
Deployment控制器的功能
- Deployment集成了上线部署、滚动升级、创建副本、回滚等功能
- Deployment里包含并使用了ReplicaSet
Deployment用于部署无状态应用
无状态应用的特点：
- 所有pod无差别
- 所有pod中容器运行同一个image
- 所有pod可以运行在集群中任意node上
- 所有pod无启动顺序先后之分
- 随意pod数量扩容或缩容
创建deployment类型应用
1：准备YAML文件 获取帮助
[root@docker-master pod.dir]# kubectl explain deployment
GROUP: apps
KIND: Deployment
VERSION: v1
......
编写文件
[root@docker-master pod.dir]# vim nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deploy #deployment名称
namespace: web-test
spec:
# 副本控制
replicas: 2 #副本集，deployment里使用了replicaset
selector:
matchLabels:
app: nginx # 匹配模版标签,表示deployment和rs控制器控制带有此标签的pod
template: #代表pod的配置模板
metadata:
labels:
app: nginx # pod的标签
spec:
restartPolicy: Always
containers: #pod中容器定义
- name: c1
image: nginx:1.26-alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
# 资源限制
resources:
requests:
cpu: 100m
memory: 200Mi
limits:
cpu: 150m
memory: 300Mi
# 存活探针
livenessProbe:
httpGet:
port: http
path: /index.html
2：应用YAML文件
[root@docker-master pod.dir]# kubectl create ns web-test
namespace/web-test created
[root@docker-master pod.dir]# kubectl apply -f nginx-deploy.yaml
deployment.apps/nginx-deploy created
3：验证
[root@docker-master pod.dir]# kubectl get pods -n web-test
NAME READY STATUS RESTARTS AGE
nginx-deploy-84b89f5f96-6vbsv 1/1 Running 0 13m
nginx-deploy-84b89f5f96-bbqsc 1/1 Running 0 13m
查看deployment名称
[root@docker-master pod.dir]# kubectl get deployment -n web-test
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 2/2 2 2 34s
查看replicaset名称
[root@docker-master pod.dir]# kubectl get replicaset -n web-test
NAME DESIRED CURRENT READY AGE
nginx-deploy-84b89f5f96 2 2 2 14m
访问deployment
1：查看pod的IP地址
[root@docker-master pod.dir]# kubectl get pod -o wide -n web-test
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deploy-84b89f5f96-6vbsv 1/1 Running 0 14m 10.244.248.173 docker-node2 <none> <none>
nginx-deploy-84b89f5f96-bbqsc 1/1 Running 0 14m 10.244.148.223 docker-node1 <none> <none>
2：查看所有集群节点的网卡
[root@docker-master pod.dir]# ifconfig tunl0 | head -2
tunl0: flags=193<UP,RUNNING,NOARP> mtu 1480
inet 10.244.219.64 netmask 255.255.255.255
[root@docker-node1 ~]# ifconfig tunl0 | head -2
tunl0: flags=193<UP,RUNNING,NOARP> mtu 1480
inet 10.244.148.192 netmask 255.255.255.255
[root@docker-node2 ~]# ifconfig tunl0 | head -2
tunl0: flags=193<UP,RUNNING,NOARP> mtu 1480
inet 10.244.248.128 netmask 255.255.255.255
总结：可以看到所有集群节点的IP为10.244.0.0/16这个大网段内的子网
3：在任意集群节点上都可以访问此deploy里的pod
[root@docker-master pod.dir]# ping -c 2 10.244.248.173
PING 10.244.248.173 (10.244.248.173) 56(84) bytes of data.
64 bytes from 10.244.248.173: icmp_seq=1 ttl=63 time=0.886 ms
64 bytes from 10.244.248.173: icmp_seq=2 ttl=63 time=0.695 ms
--- 10.244.248.173 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1003ms
rtt min/avg/max/mdev = 0.695/0.790/0.886/0.099 ms
[root@docker-master pod.dir]# curl http://10.244.248.173
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
删除deployment中的pod
1：强制删除pod（注意：是删除deployment中的pod）
[root@docker-master pod.dir]# kubectl delete pod nginx-deploy-84b89f5f96-6vbsv -n web-test --grace-period=0 --force
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "nginx-deploy-84b89f5f96-6vbsv" force deleted
2：再次查看，发现又重新启动了一个pod（节点发生变化，IP发生变化）
[root@docker-master pod.dir]# kubectl get pod -o wide -n web-test
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deploy-84b89f5f96-bbqsc 1/1 Running 0 21m 10.244.148.223 docker-node1 <none> <none>
nginx-deploy-84b89f5f96-nsb95 1/1 Running 0 5s 10.244.248.174 docker-node2 <none> <none>
总结：pod的IP不是固定的，就算把整个集群关闭再启动，pod也会自动启动，但是IP地址会发生变化 既然IP地址不是固定的，所以需要一个固定的访问endpoint给用户，那么这种方式就是service
pod版本升级
查看帮助
[root@docker-master pod.dir]# kubectl set -h
Configure application resources.
These commands help you make changes to existing application resources.
Available Commands:
env Update environment variables on a pod template
image Update the image of a pod template
resources 使用 Pod 模板更新对象的资源请求/限制
selector 为资源设置选择器
serviceaccount Update the service account of a resource
subject Update the user, group, or service account in a role binding or cluster role
binding
Usage:
kubectl set SUBCOMMAND [options]
Use "kubectl set <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).
升级前验证nginx版本
[root@docker-master pod.dir]# kubectl get pod -n web-test
NAME READY STATUS RESTARTS AGE
nginx-deploy-84b89f5f96-bbqsc 1/1 Running 1 (6m24s ago) 17h
nginx-deploy-84b89f5f96-nsb95 1/1 Running 1 (6m19s ago) 17h
[root@docker-master pod.dir]# kubectl describe pods nginx-deploy-84b89f5f96-bbqsc -n web-test | grep Image
Image: nginx:1.26-alpine
Image ID: docker-pullable://nginx@sha256:1eadbb07820339e8bbfed18c771691970baee292ec4ab2558f1453d26153e22d
[root@docker-master pod.dir]# kubectl exec nginx-deploy-84b89f5f96-bbqsc -n web-test -- nginx -v
nginx version: nginx/1.26.3
升级为1.29版本
# 查看控制器名称
[root@docker-master pod.dir]# kubectl get deploy -n web-test
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 2/2 2 2 18h
[root@docker-master pod.dir]# kubectl set image deployment nginx-deploy c1=nginx:1.29-alpine -n web-test --record
Flag --record has been deprecated, --record will be removed in the future
deployment.apps/nginx-deploy image updated
[root@docker-master pod.dir]# kubectl get pods -n web-test
NAME READY STATUS RESTARTS AGE
nginx-deploy-5bcb87f6cf-4f54f 0/1 ContainerCreating 0 20s
nginx-deploy-84b89f5f96-bbqsc 1/1 Running 1 (27m ago) 18h
nginx-deploy-84b89f5f96-nsb95 1/1 Running 1 (26m ago) 17h
[root@docker-master pod.dir]# kubectl get pods -n web-test
NAME READY STATUS RESTARTS AGE
nginx-deploy-5bcb87f6cf-4f54f 1/1 Running 0 2m8s
nginx-deploy-5bcb87f6cf-zqxrt 1/1 Running 0 88s
升级原则：Pod先创建再删除
[root@docker-master pod.dir]# kubectl exec nginx-deploy-5bcb87f6cf-4f54f -n web-test -- nginx -v
nginx version: nginx/1.29.4
说明：
- deployment nginx-deploy代表名为nginx-deploy的deployment
- c1=nginx:1.29-a1pine前面的c1为容器名
- –record 表示会记录
容器名怎么查看？
kubectl describe pod pod名查看kubectl edit deployment deployment名来查看容器名kubectl get deployment deployment名 -o yaml来查看容器名
验证
如果升级的pod数量较多，则需要一定时间，可通过下面命令查看是否已经成功
[root@docker-master pod.dir]# kubectl rollout status deployment nginx-deploy -n web-test
deployment "nginx-deploy" successfully rolled out
pod版本回退
1：查看版本历史信息
[root@docker-master pod.dir]# kubectl rollout history deployment nginx-deploy -n web-test
deployment.apps/nginx-deploy
REVISION CHANGE-CAUSE
1 <none>
2 kubectl set image deployment nginx-deploy c1=nginx:1.29-alpine --namespace=web-test --record=true
2：定义要回退的版本（还需要执行才是真的回退版本）
[root@docker-master pod.dir]# kubectl rollout history deployment nginx-deploy -n web-test --revision=1
deployment.apps/nginx-deploy with revision #1
Pod Template:
Labels: app=nginx
pod-template-hash=84b89f5f96
Containers:
c1:
Image: nginx:1.26-alpine #可以看到编号1代表执行到1.26版本
Port: 80/TCP
Host Port: 0/TCP
Limits:
cpu: 150m
memory: 300Mi
Requests:
cpu: 100m
memory: 200Mi
Liveness: http-get http://:http/index.html delay=0s timeout=1s period=10s #success=1 #failure=3
Environment: <none>
Mounts: <none>
Volumes: <none>
3：执行回退
[root@docker-master pod.dir]# kubectl rollout undo deployment nginx-deploy -n web-test --to-revision=1
deployment.apps/nginx-deploy rolled back
4：验证
回到了1.26版本，但version的ID 变了
[root@docker-master pod.dir]# kubectl rollout history deployment nginx-deploy -n web-test
deployment.apps/nginx-deploy
REVISION CHANGE-CAUSE
2 kubectl set image deployment nginx-deploy c1=nginx:1.29-alpine --namespace=web-test --record=true
3 <none>
[root@docker-master pod.dir]# kubectl get pods -n web-test
NAME READY STATUS RESTARTS AGE
nginx-deploy-84b89f5f96-dts94 1/1 Running 0 50s
nginx-deploy-84b89f5f96-xgxjm 1/1 Running 0 48s
[root@docker-master pod.dir]# kubectl describe pods nginx-deploy-84b89f5f96-dts94 -n web-test | grep Image
Image: nginx:1.26-alpine
Image ID: docker-pullable://nginx@sha256:1eadbb07820339e8bbfed18c771691970baee292ec4ab2558f1453d26153e22d
[root@docker-master pod.dir]# kubectl exec nginx-deploy-84b89f5f96-dts94 -n web-test -- nginx -v
nginx version: nginx/1.26.3
副本扩容
获取帮助
[root@docker-master pod.dir]# kubectl scale -h
1：扩容为3个副本
[root@docker-master pod.dir]# kubectl scale deployment nginx-deploy -n web-test --replicas=3
deployment.apps/nginx-deploy scaled
若故障现象：一直处于ContainerCreating状态
原因：由于网络插件 （Calico）未能正确设置 Pod 网络，导致创建 Pod 时出现了 FailedCreatePodSandBox 错误。
解决方法：由于 Calico 插件出现暂时性故障，可以尝试重启 Calico 组件来解决
[root@docker-master pod.dir]# kubectl rollout restart daemonset calico-node -n kube-system
2：查看
[root@docker-master pod.dir]# kubectl get pods -o wide -n web-test
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deploy-84b89f5f96-2pnql 1/1 Running 0 59s 10.244.248.178 docker-node2 <none> <none>
nginx-deploy-84b89f5f96-dts94 1/1 Running 0 34m 10.244.248.177 docker-node2 <none> <none>
nginx-deploy-84b89f5f96-xgxjm 1/1 Running 0 34m 10.244.148.229 docker-node1 <none> <none>
副本裁减
1：指定副本数为1进行裁减
[root@docker-master pod.dir]# kubectl scale deployment nginx-deploy -n web-test --replicas=1
deployment.apps/nginx-deploy scaled
2：查看验证
[root@docker-master pod.dir]# kubectl get pods -o wide -n web-test
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deploy-84b89f5f96-xgxjm 1/1 Running 0 37m 10.244.148.229 docker-node1 <none> <none>
副本数量可以降为0，有需要就启动，无需要就停止
[root@docker-master pod.dir]# kubectl scale deployment nginx-deploy -n web-test --replicas=0
deployment.apps/nginx-deploy scaled
查看
[root@docker-master pod.dir]# kubectl get deploy -n web-test
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deploy 0/0 0 0 19h
[root@docker-master pod.dir]# kubectl get pods -o wide
No resources found in default namespace.
多副本滚动更新
1：先扩容多副本
[root@docker-master pod.dir]# kubectl scale deployment nginx-deploy -n web-test --replicas=16
deployment.apps/nginx-deploy scaled
2：验证
[root@docker-master pod.dir]# kubectl get pods -n web-test
NAME READY STATUS RESTARTS AGE
nginx-deploy-84b89f5f96-248qv 1/1 Running 0 25s
nginx-deploy-84b89f5f96-2kdsc 1/1 Running 0 25s
nginx-deploy-84b89f5f96-5hqzj 1/1 Running 0 25s
nginx-deploy-84b89f5f96-6zkdt 1/1 Running 0 25s
nginx-deploy-84b89f5f96-85hg6 1/1 Running 0 25s
nginx-deploy-84b89f5f96-b2m4r 1/1 Running 0 25s
nginx-deploy-84b89f5f96-c4585 1/1 Running 0 25s
nginx-deploy-84b89f5f96-c4bx7 1/1 Running 0 25s
nginx-deploy-84b89f5f96-clnzl 1/1 Running 0 25s
nginx-deploy-84b89f5f96-gc2s2 1/1 Running 0 25s
nginx-deploy-84b89f5f96-j52gz 1/1 Running 0 25s
nginx-deploy-84b89f5f96-mvk25 1/1 Running 0 25s
nginx-deploy-84b89f5f96-phk8k 1/1 Running 0 25s
nginx-deploy-84b89f5f96-sjrwb 1/1 Running 0 25s
nginx-deploy-84b89f5f96-w59gb 1/1 Running 0 25s
nginx-deploy-84b89f5f96-z2cnq 1/1 Running 0 25s
3：滚动更新
[root@docker-master pod.dir]# kubectl set image deployment nginx-deploy c1=nginx:1.29-alpine -n web-test --record
Flag --record has been deprecated, --record will be removed in the future
deployment.apps/nginx-deploy image updated
4：验证
# 查看Pod名称
[root@docker-master pod.dir]# kubectl get pod -n web-test
[root@docker-master pod.dir]# kubectl describe pod nginx-deploy-5bcb87f6cf-27qkt -n web-test | grep Image
Image: nginx:1.29-alpine
Image ID: docker-pullable://nginx@sha256:b0f7830b6bfaa1258f45d94c240ab668ced1b3651c8a222aefe6683447c7bf55
[root@docker-master pod.dir]# kubectl rollout status deployment nginx-deploy -n web-test
deployment "nginx-deploy" successfully rolled out
删除deployment
删除deployment，里面的pod也会被自动删除
[root@docker-master pod.dir]# kubectl delete deployment nginx-deploy -n web-test
deployment.apps "nginx-deploy" deleted
# 也可以通过删除命名空间全部删除
[root@docker-master pod.dir]# kubectl delete ns web-test
[root@docker-master pod.dir]# kubectl get pod -n web-test
No resources found in web-test namespace.
3：ReplicaSet
提醒：replicaset控制器很少单独使用，不建议脱离deployment
1：编写YAML文件
[root@docker-master controllers_dir]# vim nginx-replicas.yaml
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: nginx-rc
namespace: default
spec:
replicas: 4
selector:
matchLabels:
app: nginx
template:
metadata:
name: nginx
labels:
app: nginx
spec:
containers:
- name: c1
image: nginx:1.26-alpine
ports:
- name: http
containerPort: 80
2：应用YAML文件
[root@docker-master controllers_dir]# kubectl apply -f nginx-replicas.yaml
replicaset.apps/nginx-rc created
3：验证
[root@docker-master controllers_dir]# kubectl get rs
NAME DESIRED CURRENT READY AGE
nginx-rc 4 4 4 24s
[root@docker-master controllers_dir]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-rc-4m8xc 1/1 Running 0 34s
nginx-rc-4n45p 1/1 Running 0 34s
nginx-rc-sx8nv 1/1 Running 0 34s
nginx-rc-zqkrw 1/1 Running 0 34s
[root@docker-master controllers_dir]# kubectl get deployment
No resources found in default namespace.
说明：找不到deployment，说明创建rs并没有创建deployment
是否能扩容，这里先获取帮助信息
[root@docker-master controllers_dir]# kubectl scale -h
进行扩容
[root@docker-master controllers_dir]# kubectl scale replicaset nginx-rc --replicas=2
replicaset.apps/nginx-rc scaled
查看副本数量
[root@docker-master controllers_dir]# kubectl get rs
NAME DESIRED CURRENT READY AGE
nginx-rc 2 2 2 72m
[root@docker-master controllers_dir]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-rc-sx8nv 1/1 Running 0 2m12s
nginx-rc-zqkrw 1/1 Running 0 2m12s
版本升级
[root@docker-master controllers_dir]# kubectl describe pod nginx-rc-sx8nv | grep Image:
Image: nginx:1.26-alpine
[root@docker-master controllers_dir]# kubectl set image replicaset nginx-rc c1=nginx:1.29-alpine --record
Flag --record has been deprecated, --record will be removed in the future
replicaset.apps/nginx-rc image updated
检查，版本并没发生变化，replicaset无法实现版本更新
[root@docker-master controllers_dir]# kubectl describe pod nginx-rc-sx8nv | grep Image:
Image: nginx:1.26-alpine
控制器controller进阶
1：DaemonSet
DaemonSet介绍
- Daemonset能够让所有(或者特定)的节点运行同一个pod。
- 当节点加入到K8s集群中，pod会被(DaemonSet)调度到该节点上运行，当节点从K8S集群中被移 除，被Daemonset调度的pod会被移除。
- 如果删除DaemonSet，所有跟这个DaemonSet相关的pods都会被删除。
- 如果一个DaemonSet的Pod被杀死、停止、或者崩溃，那么DaemonSet将会重新创建一个新的副 本在这台计算节点上。
- DaemonSet一般应用于日志收集、监控采集、分布式存储守护进程等。
DaemonSet应用案例
1：编写YAML文件
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: nginx-ds
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
# 添加污点,master节点不创建守护进程Pod
tolerations: #代表容忍,保证在所有节点运行
- key: node-role.kubernetes.io/master #能容忍的污点key
effect: NoSchedule #能容忍的污点effect
containers:
- name: c1
image: nginx:1.26-alpine
imagePullPolicy: IfNotPresent
resources: #资源限制是为了防止节点资源被占太多（根据实际情况配置）
limits:
memory: 100Mi
requests:
memory: 100Mi
说明：
tolerations:
- key：污点的唯一标识，用于精确匹配节点对应的taint key,格式：node-role.kubernetes.io/<角色名>
- effect：污点的限制行为
- NoSchedule：只限制新的Pod，已运行的Pod不受影响
- PreferNoSchedule：软限制，尽量不调度，集群资源不足时，可以调度
- NoExecute：最严格，既不允许新的Pod调度，还会驱逐已运行的不匹配的Pod
2：应用YAML文件
[root@docker-master controllers_dir]# kubectl apply -f nginx-ds.yaml
daemonset.apps/nginx-ds created
3：验证
没有设置副本数量为什么会创建2个Pod
daemonset作用：确保每个副本至少运行一个Pod副本。该集群有2个node节点，故创建了2个Pod
[root@docker-master controllers_dir]# kubectl get daemonset
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
nginx-ds 2 2 2 2 2 <none> 21s
[root@docker-master controllers_dir]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ds-ljlfr 1/1 Running 0 41s 10.244.148.248 docker-node1 <none> <none>
nginx-ds-xnzv7 1/1 Running 0 41s 10.244.248.135 docker-node2 <none> <none>
无法进行扩缩容
[root@docker-master controllers_dir]# kubectl scale daemonset nginx-ds --replicas=4
Error from server (NotFound): the server could not find the requested resource
可以进行版本升级
[root@docker-master controllers_dir]# kubectl describe pods nginx-ds-ljlfr | grep Image:
Image: nginx:1.26-alpine
升级操作
[root@docker-master controllers_dir]# kubectl set image daemonset nginx-ds c1=nginx:1.29-alpine
daemonset.apps/nginx-ds image updated
验证，更新到1.29版本
[root@docker-master controllers_dir]# kubectl describe pods nginx-ds-s689j | grep Image
Image: nginx:1.29-alpine
2：Job
job介绍
- 对于ReplicaSet而言，它希望pod保持预期数目、持久运行下去，除非用户明确删除，否则这些对 象一直存在，它们针对的是耐久性任务如web服务等。
- 对于非耐久性任务，比如压缩文件，任务完成后，pod需要结束运行，不需要pod继续保持在系统 中，这个时候就要用到job。
- job负责批量处理短暂的一次性任务(short lived one-of tasks)，即仅执行一次的任务，它保证批处 理任务的一个或多个Pod成功结束。
job应用案例
计算圆周率2000位
1：编写YAML文件
[root@docker-master controllers_dir]# vim job-pi.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: pod-pi
spec:
template:
metadata:
name: pi
spec:
nodeName: docker-node2
containers:
- name: c1
image: perl
imagePullPolicy: IfNotPresent
command: ["perl","-Mbignum=bpi","-wle","print bpi(2000)"]
restartPolicy: Never
2：应用YAML文件创建job
[root@docker-master controllers_dir]# kubectl apply -f job-pi.yaml
job.batch/pod-pi created
3：验证
[root@docker-master controllers_dir]# kubectl get pod
NAME READY STATUS RESTARTS AGE
mysql-dump-gsf2x 0/1 Completed 0 49m
mysql-pod-0 1/1 Running 0 75m
pod-pi-g6d74 0/1 Completed 0 9s
[root@docker-master controllers_dir]# kubectl logs pod-pi-g6d74
3.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066063155881748815209209628292540917153643678925903600113305305488204665213841469519415116094330572703657595919530921861173819326117931051185480744623799627495673518857527248912279381830119491298336733624406566430860213949463952247371907021798609437027705392171762931767523846748184676694051320005681271452635608277857713427577896091736371787214684409012249534301465495853710507922796892589235420199561121290219608640344181598136297747713099605187072113499999983729780499510597317328160963185950244594553469083026425223082533446850352619311881710100031378387528865875332083814206171776691473035982534904287554687311595628638823537875937519577818577805321712268066130019278766111959092164201989380952572010654858632788659361533818279682303019520353018529689957736225994138912497217752834791315155748572424541506959508295331168617278558890750983817546374649393192550604009277016711390098488240128583616035637076601047101819429555961989467678374494482553797747268471040475346462080466842590694912933136770289891521047521620569660240580381501935112533824300355876402474964732639141992726042699227967823547816360093417216412199245863150302861829745557067498385054945885869269956909272107975093029553211653449872027559602364806654991198818347977535663698074265425278625518184175746728909777727938000816470600161452491921732172147723501414419735685481613611573525521334757418494684385233239073941433345477624168625189835694855620992192221842725502542568876717904946016534668049886272327917860857843838279679766814541009538837863609506800642251252051173929848960841284886269456042419652850222106611863067442786220391949450471237137869609563643719172874677646575739624138908658326459958133904780275901
创建固定次数job
1：编写YAML
[root@docker-master controllers_dir]# vim job-num.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: job-hello
spec:
completions: 10 # 执行job的次数
parallelism: 1 # 执行job的并发数
template:
metadata:
name: hello
spec:
containers:
- name: c1
image: busybox
imagePullPolicy: IfNotPresent
command: ["echo","hello"]
restartPolicy: Never
2：应用YAML文件创建job
[root@docker-master controllers_dir]# kubectl apply -f job-num.yaml
job.batch/job-hello created
3：验证
[root@docker-master controllers_dir]# kubectl get pod
NAME READY STATUS RESTARTS AGE
job-hello-2svsk 0/1 Completed 0 59s
job-hello-82sft 0/1 Completed 0 68s
job-hello-9ck4q 0/1 Completed 0 62s
job-hello-d8xrs 0/1 Completed 0 50s
job-hello-hqc7q 0/1 Completed 0 53s
job-hello-ng6hh 0/1 Completed 0 47s
job-hello-rmbjr 0/1 Completed 0 65s
job-hello-sb7nh 0/1 Completed 0 44s
job-hello-vwmkr 0/1 Completed 0 56s
job-hello-w75cf 0/1 Completed 0 72s
[root@docker-master controllers_dir]# kubectl get job
NAME COMPLETIONS DURATION AGE
job-hello 10/10 31s 83s
查看输出内容
[root@docker-master controllers_dir]# kubectl logs job-hello-2svsk
hello
logs后面跟Pod名称，查看的是容器日志
一次性备份MySQL数据库
通过job控制器创建应用备份MySQL数据库
1：MySQL数据库准备
[root@docker-master controllers_dir]# vim mysql.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql-svc
spec:
ports:
- port: 3306
name: mysql
clusterIP: None #无头服务
selector:
app: mysql-dump #匹配pod名称
---
apiVersion: apps/v1 #有状态控制器
kind: StatefulSet
metadata:
name: mysql-pod
spec:
selector:
matchLabels:
app: mysql-dump
serviceName: mysql-svc
template:
metadata:
labels:
app: mysql-dump #pod名称
spec:
nodeName: docker-node2 #资源创建到docker-node2
containers:
- name: c1 #容器名称
image: mysql:5.7
imagePullPolicy: IfNotPresent
env:
- name: MYSQL_ROOT_PASSWORD # 变量名 MySQL管理员密码设置
value: "abc123" # 变量值
ports:
- containerPort: 3306
volumeMounts: #数据库中数据挂载
- mountPath: "/var/lib/mysql"
name: mysql-data
volumes:
- name: mysql-data
hostPath:
path: /opt/mysqldata
创建用于实现任务的资源清单文件
[root@docker-master controllers_dir]# kubectl apply -f mysql.yaml
service/mysql-svc unchanged
statefulset.apps/mysql-pod created
查看资源
[root@docker-master controllers_dir]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
mysql-pod-0 1/1 Running 0 28s 10.244.248.146 docker-node2 <none> <none>
[root@docker-master controllers_dir]# kubectl get statefulsets
NAME READY AGE
mysql-pod 1/1 46s
运行在docker-node2节点，可以查看docker-node2节点数据库目录
[root@docker-node2 ~]# ls /opt/mysqldata
auto.cnf client-cert.pem ibdata1 ibtmp1 performance_schema server-cert.pem
ca-key.pem client-key.pem ib_logfile0 mysql private_key.pem server-key.pem
ca.pem ib_buffer_pool ib_logfile1 mysql.sock public_key.pem sys
创建用于实现任务的资源清单文件
[root@docker-master controllers_dir]# vim mysql-dump.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: mysql-dump
spec:
template:
metadata:
name: mysql-bk
spec:
restartPolicy: Never
nodeName: docker-node1
containers:
- name: c1
image: mysql:5.7
imagePullPolicy: IfNotPresent
command: ["/bin/sh","-c","mysqldump --host=mysql-svc -uroot -pabc123 --databases mysql > /root/mysql_bak.sql"]
volumeMounts:
- mountPath: "/root"
name: mysql-dump-data01
volumes:
- name: mysql-dump-data
hostPath:
path: /opt/mysqldump
应用文件
[root@docker-master controllers_dir]# kubectl apply -f mysql-dump.yaml
job.batch/mysql-dump created
验证
[root@docker-master controllers_dir]# kubectl get job
NAME COMPLETIONS DURATION AGE
mysql-dump 1/1 4s 12s
在docker-node1节点上查看备份sql文件
[root@docker-node1 ~]# ls /opt/mysqldump
mysql_bak.sql
3：cronjob
类似于Linux系统的crontab，在指定的时间周期运行相关的任务
案例：
周期性输出字符
1：编写YAML文件
[root@docker-master controllers_dir]# vim cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: pod-cronjob
spec:
schedule: "* * * * *" #分时日月周
jobTemplate:
spec:
template:
spec:
nodeName: docker-node2
containers:
- name: c1
image: busybox
imagePullPolicy: IfNotPresent
args:
- /bin/sh
- -c
- date;echo hello kubernetes
restartPolicy: OnFailure
2：应用YAML文件创建cronjob
[root@docker-master controllers_dir]# kubectl apply -f cronjob.yaml
cronjob.batch/pod-cronjob created
3：查看验证
[root@docker-master controllers_dir]# kubectl get cj # cj：cronjob的缩写
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
pod-cronjob * * * * * False 0 34s 2m25s
[root@docker-master controllers_dir]# kubectl get pod
NAME READY STATUS RESTARTS AGE
pod-cronjob-29482643-r64nm 0/1 Completed 0 78s # 每隔一分钟创建一个Pod
pod-cronjob-29482644-tvmhj 0/1 Completed 0 18s
[root@docker-master controllers_dir]# kubectl delete -f cronjob.yaml
cronjob.batch "pod-cronjob" deleted
周期性备份MySQL数据库
MySQL数据库准备
1：准备YAML文件
[root@docker-master controllers_dir]# vim mysql.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql-svc
spec:
ports:
- port: 3306
name: mysql
clusterIP: None
selector:
app: mysql-dump
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mysql-pod
spec:
selector:
matchLabels:
app: mysql-dump
serviceName: mysql-svc
template:
metadata:
labels:
app: mysql-dump
spec:
nodeName: docker-node2
containers:
- name: c1
image: mysql:5.7
imagePullPolicy: IfNotPresent
env:
- name: MYSQL_ROOT_PASSWORD
value: "abc123"
ports:
- containerPort: 3306
volumeMounts:
- mountPath: "/var/lib/mysql"
name: mysql-data
volumes:
- name: mysql-data
hostPath:
path: /opt/mysqldata
2：运行YAML文件
[root@docker-master controllers_dir]# kubectl apply -f mysql.yaml
service/mysql-svc created
statefulset.apps/mysql-pod created
3：验证
[root@docker-master controllers_dir]# kubectl get pod
NAME READY STATUS RESTARTS AGE
mysql-pod-0 1/1 Running 0 49m
4：cronjob控制器类型应用资源清单文件
[root@docker-master controllers_dir]# vim mysql-cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: mysql-cronjob
spec:
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
nodeName: docker-node1
containers:
- name: c1
image: mysql:5.7
imagePullPolicy: IfNotPresent
command: ["/bin/sh","-c","mysqldump --host=mysql-svc -uroot -pabc123 --databases mysql > /root/mysql`date +%Y%m%d%H%M`.sql"]
volumeMounts:
- name: mysql-cronjob-data
mountPath: "/root"
restartPolicy: Never
volumes:
- name: mysql-cronjob-data
hostPath:
path: /opt/mysql_cronjob
schedule: "*/1 * * * *"表示每一分钟，如果是 schedule: "1 * * * *"表示第一分钟
5：执行应用资源文件
[root@docker-master controllers_dir]# kubectl apply -f mysql-cronjob.yaml
cronjob.batch/mysql-cronjob created
6：查看结果
[root@docker-master controllers_dir]# kubectl get cj
\NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
mysql-cronjob */1 * * * * False 0 14s 35s
[root@docker-master controllers_dir]# kubectl get pod
NAME READY STATUS RESTARTS AGE
mysql-cronjob-29482672-nzglz 0/1 Completed 0 27s
mysql-pod-0 1/1 Running 0 26m
[root@docker-master controllers_dir]# kubectl logs mysql-cronjob-29482672-nzglz
mysqldump: [Warning] Using a password on the command line interface can be insecure.
在docker-node1节点查看mysql备份文件
[root@docker-node1 ~]# ls /opt/mysql_cronjob
mysql202601210207.sql mysql202601210208.sql
控制器controller之StatefulSet
statefulset控制器作用
- Statefulset 是用来管理有状态应用的控制器。
- Statefulset 用来管理应用的部署、扩缩容及滚动更新。
- Statefulset 为 Pod 提供持久存储和持久标识符。
参考文档：https://kubernetes.io/zh-cn/docs/concepts/workloads/controllers/statefulset/
无状态应用与有状态应用
1：无状态应用
- 如nginx
- 请求本身包含了响应端为响应这一请求所需的全部信息。每一个请求都像首次执行一样，不会依赖 之前的数据进行响应。
- 不需要持久化的数据
- 无状态应用的多个实例之间互不依赖，可以无序的部署、删除或伸缩
2：有状态应用
- 如MySQL
- 前后请求有关联与依赖
- 需要持久化的数据
- 有状态应用的多个实例之间有依赖，不能相互替换:无论怎么调度，每个 Pod 都有一个永久不变的 ID。
StatefulSet的特点
- 稳定的、唯一的网络标识符。(无clusterIP通过headless服务实现)
- 稳定的、持久的存储。(通过静态存储PV，PVC，动态存储storageclass实现)
- 有序的、优雅的部署和缩放
- 有序的、自动的滚动更新（从最大数更新，例如：编号2，编号1，编号0）。
StatefulSet的YAML组成
需要三个组成部分:
- headless service: 实现稳定，唯一的网络标识。
- statefulset类型资源: 写法和deployment几乎一致，就是类型不一样。
- volumeClaimTemplate:指定存储卷。
补充：无头服务是指没有cluster-IP，其通过域名解析
创建StatefulSet应用
参考文档：https://kubernetes.io/zh-cn/docs/tutorials/stateful-application/basic-stateful-set/
网络存储NFS部署
1：nfs服务器，地址192.168.108.33
CentOS 7 模版创建
[root@nfs ~]# yum install -y nfs-utils
[root@nfs ~]# mkdir -p /data/nfs
[root@nfs ~]# vim /etc/exports
# 解析：*允许所有用户访问 rw可读写 no_root_squash禁用安全降权规则。
/data/nfs *(rw,no_root_squash,sync)
[root@nfs ~]# systemctl restart nfs-server.service
[root@nfs ~]# systemctl enable nfs-server.service
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
[root@nfs ~]# showmount -e
Export list for nfs:
/data/nfs *
2：所有node节点安装nfs客户端软件包
[root@docker-node1 ~]# yum install -y nfs-utils
[root@docker-node2 ~]# yum install -y nfs-utils
3：验证nfs可用性
[root@docker-node1 ~]# showmount -e 192.168.108.33
Export list for 192.168.108.33:
/data/nfs *
[root@docker-node2 ~]# showmount -e 192.168.108.33
Export list for 192.168.108.33:
/data/nfs *
4：master节点上创建yaml文件
[root@docker-master controllers_dir]# vim nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deploy
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: c1
image: nginx:1.26-alpine
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
# 挂载
volumeMounts:
- name: nfs-data
mountPath: /usr/share/nginx/html # 容器内网页目录
restartPolicy: Always
volumes:
- name: nfs-data
nfs:
path: /data/nfs # 共享目录
server: 192.168.108.33
5：应用yaml文件
[root@docker-master controllers_dir]# kubectl apply -f nginx-deployment.yaml
deployment.apps/nginx-deploy created
6：查看存储pod
[root@docker-master controllers_dir]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deploy-767f8b4f98-g2zkd 1/1 Running 0 30s
nginx-deploy-767f8b4f98-jfjs5 1/1 Running 0 30s
7：在nfs服务器上创建index文件
[root@nfs ~]# cd /data/nfs
[root@nfs nfs]# ls
[root@nfs nfs]# echo "hello nfs storage" > index.html
8：在master上可以直接验证查看
[root@docker-master controllers_dir]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE
nginx-deploy-767f8b4f98-g2zkd 1/1 Running 0 2m3s 10.244.148.202 docker-node1 <none> <none>
nginx-deploy-767f8b4f98-jfjs5 1/1 Running 0 2m3s 10.244.248.157 docker-node2 <none> <none>
[root@docker-master controllers_dir]# curl http://10.244.148.202
hello nfs storage
[root@docker-master controllers_dir]# curl http://10.244.248.157
hello nfs storage
PV（持久化存储卷）与 PVC（持久存储卷声明）
kubernetes存储卷的分类太丰富了,每种类型都要写相应的接口与参数才行，这就让维护与管理难度加 大。
kubernetes支持分类查看
参考文档：https://kubernetes.io/zh-cn/docs/concepts/storage/
[root@docker-master controllers_dir]# kubectl explain pod.spec.volumes
persistenvolume(PV)是配置好的一段存储(可以是任意类型的存储卷)
- 也就是说将网络存储共享出来,配置定义成PV。
PersistentVolumeClaim(PVC)是用户pod使用PV的申请请求。
- 用户不需要关心具体的volume实现细节,只需要关心使用需求。
pv与pvc之间的关系
- pv提供存储资源(生产者)
- pvc使用存储资源(消费者)
- 使用pvc绑定pv
实现nfs类型pv与pvc
1：编写pv的yaml文件
[root@docker-master controllers_dir]# vim pv-nfs.yaml
apiVersion: v1
kind: PersistentVolume #类型pv
metadata:
name: pv-nfs #名称
spec:
# 容量
capacity:
storage: 10Gi
# 访问模式
accessModes:
- ReadWriteMany
nfs:
path: /data/nfs #nfs共享目录
server: 192.168.108.33 #nfs服务器IP
访问模式3种：参考文档： https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes
- ReadWriteOnce 单节点读写挂载
- ReadOnlyMany 多节点只读挂载
- ReadWriteMany 多节点读写挂载
- ReadWriteOncePod 卷可以被单个 Pod 以读写方式挂载。（1.29新特性）
我们要实现多个nginx跨节点之间的数据共享,所以选择ReadWriteMany模式。
2：创建pv并验证
[root@docker-master controllers_dir]# kubectl apply -f pv-nfs.yaml
persistentvolume/pv-nfs created
[root@docker-master controllers_dir]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv-nfs 10Gi RWX Retain Available
说明：
- RWX为ReadWriteMany的简写
- Retain是回收策略
- Retain表示不使用了需要手动回收
- 参考: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#reclaim-policy
3：创建pvc的yaml文件
[root@docker-master controllers_dir]# vim pvc-nfs.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-nfs
spec:
accessModes:
- ReadWriteMany # 类型与pv一致，否则无法绑定
resources:
requests:
storage: 10Gi #大小与pv一致
4：应用yaml
[root@docker-master controllers_dir]# kubectl apply -f pvc-nfs.yaml
persistentvolumeclaim/pvc-nfs created
5：验证
[root@docker-master controllers_dir]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-nfs Bound pv-nfs 10Gi RWX 11s
查看pv，绑定成功
[root@docker-master controllers_dir]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv-nfs 10Gi RWX Retain Bound default/pvc-nfs 4m59s
6：创建nginx应用yaml文件
[root@docker-master controllers_dir]# vim nginx-pvc.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-pvc
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
restartPolicy: Always
containers:
- name: c1
image: nginx:1.26-alpine
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumes:
- name: www
persistentVolumeClaim:
claimName: pvc-nfs
7：运行yaml文件
[root@docker-master controllers_dir]# kubectl apply -f nginx-pvc.yaml
deployment.apps/nginx-pvc created
8：验证
[root@docker-master controllers_dir]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-pvc-77699f8c7c-9tw2c 1/1 Running 0 12s 10.244.248.156 docker-node2 <none> <none>
nginx-pvc-77699f8c7c-bbj72 1/1 Running 0 12s 10.244.148.198 docker-node1 <none> <none>
[root@docker-master controllers_dir]# curl http://10.244.248.156
hello nfs storage
[root@docker-master controllers_dir]# curl http://10.244.148.198
hello nfs storage
在nfs中修改index.html首页内容
[root@nfs nfs]# vim index.html
hello, this is nfs storage
再次查看首页
[root@docker-master controllers_dir]# curl http://10.244.148.198
hello, this is nfs storage
[root@docker-master controllers_dir]# curl http://10.244.248.156
hello, this is nfs storage
动态供给
每次使用存储要先创建pv,再创建pvc，真累!所以我们可以实现使用存储的动态供给特性。
- 静态存储需要用户申请PVC时保证容量和读写类型与预置PV的容量及读写类型完全匹配,而动态存储 则无需如此。
- 管理员无需预先创建大量的PV作为存储资源。
Kubernetes从1.4版起引入了一个新的资源对象StorageClass,可用于将存储资源定义为具有显著特性的 类(Class)而不是具体的PV。用户通过PVC直接向意向的类别发出申请，匹配由管理员事先创建的PV，或 者由其按需为用户动态创建PV，这样就免去了需要先创建PV的过程。
NFS文件系统创建存储动态供给
官方插件不支持NFS动态供给，可以使用第三方插件来实现
第三方插件地址：https://github.com/kubernetes-retired/external-storage
点开deploy目录
在master节点创建下载目录
[root@docker-master controllers_dir]# mkdir nfs-cli
[root@docker-master controllers_dir]# cd nfs-cli/
1：下载class插件文件
[root@docker-master nfs-cli]# wget https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/class.yaml
--2026-01-21 14:51:47-- https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/class.yaml
正在解析主机 raw.githubusercontent.com (raw.githubusercontent.com)... 198.18.0.6
正在连接 raw.githubusercontent.com (raw.githubusercontent.com)|198.18.0.6|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：246 [text/plain]
正在保存至: “class.yaml”
100%[========================================================================>] 246 --.-K/s 用时 0s
2026-01-21 14:51:48 (5.61 MB/s) - 已保存 “class.yaml” [246/246])
[root@docker-master nfs-cli]# vim class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass #类型
metadata:
name: nfs-client #名称，要使用就需要调用此名称
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner #动态供给插件
parameters:
archiveOnDelete: "false" #删除数据时是否存档，false表示不存档，true表示存档
应用yaml文件
[root@docker-master nfs-cli]# kubectl apply -f class.yaml
storageclass.storage.k8s.io/nfs-client created
查看资源
[root@docker-master nfs-cli]# kubectl get storageclass
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
nfs-client k8s-sigs.io/nfs-subdir-external-provisioner Delete Immediate false 18s
- RECLAIMPOLICY pv回收策略，pod或者pvc被删除后，pv是否删除还是保留
- VOLUMEBINDINGMODE Immediate模式下pvc与pv立即绑定，主要是不等待相关pod调度完 成，不关心其运行节点，直接完成
- ALLOWVOLUMEEXPANSION pvc扩容 false不允许，true允许
2：下载并创建rbac
用于apiserver给storage提供自动创建的授权
因为storage自动创建pv需要经过kube-apiserver，所以需要授权。
[root@docker-master nfs-cli]# wget https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/rbac.yaml
--2026-01-21 14:54:50-- https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/rbac.yaml
正在解析主机 raw.githubusercontent.com (raw.githubusercontent.com)... 198.18.0.6
正在连接 raw.githubusercontent.com (raw.githubusercontent.com)|198.18.0.6|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：1900 (1.9K) [text/plain]
正在保存至: “rbac.yaml”
100%[========================================================================>] 1,900 --.-K/s 用时 0s
2026-01-21 14:54:51 (51.8 MB/s) - 已保存 “rbac.yaml” [1900/1900])
应用yaml文件
[root@docker-master nfs-cli]# kubectl apply -f rbac.yaml
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
3：创建动态供给的deployment
需要一个deployment来专门实现pv与pvc的自动创建
下载对应的deployment文件
[root@docker-master nfs-cli]# wget https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/deployment.yaml
--2026-01-21 14:55:56-- https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/deployment.yaml
正在解析主机 raw.githubusercontent.com (raw.githubusercontent.com)... 198.18.0.6
正在连接 raw.githubusercontent.com (raw.githubusercontent.com)|198.18.0.6|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：1069 (1.0K) [text/plain]
正在保存至: “deployment.yaml”
100%[========================================================================>] 1,069 --.-K/s 用时 0s
2026-01-21 14:55:57 (97.9 MB/s) - 已保存 “deployment.yaml” [1069/1069])
编辑yaml文件
[root@docker-master nfs-cli]# vim deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-client-provisioner
labels:
app: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: registry.cn-beijing.aliyuncs.com/pylixm/nfs-subdir-external-provisioner:v4.0.0 #替换国内镜像源
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: k8s-sigs.io/nfs-subdir-external-provisioner
- name: NFS_SERVER
value: 192.168.108.33 #nfs服务器地址
- name: NFS_PATH
value: /data/nfs #共享目录
volumes:
- name: nfs-client-root
nfs:
server: 192.168.108.33 #nfs服务器地址
path: /data/nfs #共享目录
应用yaml文件
[root@docker-master nfs-cli]# kubectl apply -f deployment.yaml
deployment.apps/nfs-client-provisioner created
验证
[root@docker-master nfs-cli]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-56fc4bcc87-hkk89 1/1 Running 0 58s
4：nginx应用使用动态供给
1.创建statefulset应用来调用名为nfs-client的storageclass，以实现动态供给
[root@docker-master nfs-cli]# vim nginx-storageclass.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-svc #service名称
spec:
ports:
- port: 80
name: web
selector:
app: nginx #service关联pod标签
clusterIP: None #无头服务
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nginx-storageclass #名称+编号=pod名称
spec:
replicas: 2
selector:
matchLabels:
app: nginx #控制器关联pod标签
serviceName: "nginx-svc" #关联service名称
template:
metadata:
labels:
app: nginx #pod标签
spec:
containers:
- name: c1 # 容器名
image: nginx:1.26-alpine
imagePullPolicy: IfNotPresent
ports:
- name: web
containerPort: 80
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
# 动态持久化
volumeClaimTemplates: #pvc模板
- metadata:
name: www #构成pvc名称
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: "nfs-client"
resources:
requests:
storage: 1Gi
2：应用YAML文件
[root@docker-master nfs-cli]# kubectl apply -f nginx-storageclass.yaml
service/nginx-svc created
statefulset.apps/nginx-storageclass created
3：查看资源
[root@docker-master nfs-cli]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-56fc4bcc87-hkk89 1/1 Running 0 33m
nginx-storageclass-0 1/1 Running 0 3s
nginx-storageclass-1 0/1 Pending 0 1s
[root@docker-master nfs-cli]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7d23h
nginx-svc ClusterIP None <none> 80/TCP 21s
[root@docker-master nfs-cli]# kubectl get statefulset
NAME READY AGE
nginx-storageclass 2/2 41s
pvc状态
[root@docker-master nfs-cli]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-nfs Bound pv-nfs 10Gi RWX 96m
www-nginx-storageclass-0 Bound pvc-80f1db0e-34ab-4fc5-a193-a97732fa99d5 1Gi RWO nfs-client 2m41s
www-nginx-storageclass-1 Bound pvc-1c8f1684-1dd8-410b-b985-72d98bbd8756 1Gi RWO nfs-client 56s
pv状态
[root@docker-master nfs-cli]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv-nfs 10Gi RWX Retain Bound default/pvc-nfs 101m
pvc-1c8f1684-1dd8-410b-b985-72d98bbd8756 1Gi RWO Delete Bound default/www-nginx-storageclass-1 nfs-client 68s
pvc-80f1db0e-34ab-4fc5-a193-a97732fa99d5 1Gi RWO Delete Bound default/www-nginx-storageclass-0 nfs-client 2m53s
到nfs-server中，查看/data/nfs目录，会发现有两个文件夹，保存两个pod网站数据
[root@nfs nfs]# cd /data/nfs/
[root@nfs nfs]# ls
default-www-nginx-storageclass-0-pvc-80f1db0e-34ab-4fc5-a193-a97732fa99d5 index.html
default-www-nginx-storageclass-1-pvc-1c8f1684-1dd8-410b-b985-72d98bbd8756
往两个目录中分别写入index.html内容进行查看
[root@nfs nfs]# echo "this is storage01" > default-www-nginx-storageclass-0-pvc-80f1db0e-34ab-4fc5-a193-a97732fa99d5/index.html
[root@nfs nfs]# echo "this is storage02" > default-www-nginx-storageclass-1-pvc-1c8f1684-1dd8-410b-b985-72d98bbd8756/index.html
[root@docker-master nfs-cli]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nfs-client-provisioner-56fc4bcc87-hkk89 1/1 Running 0 36m 10.244.248.159 docker-node2 <none> <none>
nginx-storageclass-0 1/1 Running 0 3m43s 10.244.248.165 docker-node2 <none> <none>
nginx-storageclass-1 1/1 Running 0 3m41s 10.244.148.203 docker-node1 <none> <none>
[root@docker-master nfs-cli]# curl http://10.244.248.165
this is storage01
[root@docker-master nfs-cli]# curl http://10.244.148.203
this is storage02
使用kube-dns解析无头服务域名，可以直接看到后端pod的地址
[root@docker-master nfs-cli]# kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 7d23h
metrics-server ClusterIP 10.111.168.59 <none> 443/TCP 7d
[root@docker-master nfs-cli]# dig -t a nginx-svc.default.svc.cluster.local. @10.96.0.10
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.16 <<>> -t a nginx-svc.default.svc.cluster.local. @10.96.0.10
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15668
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;nginx-svc.default.svc.cluster.local. IN A
;; ANSWER SECTION:
nginx-svc.default.svc.cluster.local. 30 IN A 10.244.148.203
nginx-svc.default.svc.cluster.local. 30 IN A 10.244.248.165
;; Query time: 3 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: 三 1月 21 15:42:00 CST 2026
;; MSG SIZE rcvd: 166
解析pod域名
[root@docker-master nfs-cli]# dig -t a nginx-storageclass-0.nginx-svc.default.svc.cluster.local. @10.96.0.10
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.16 <<>> -t a nginx-storageclass-0.nginx-svc.default.svc.cluster.local. @10.96.0.10
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17527
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;nginx-storageclass-0.nginx-svc.default.svc.cluster.local. IN A
;; ANSWER SECTION:
nginx-storageclass-0.nginx-svc.default.svc.cluster.local. 30 IN A 10.244.248.165
;; Query time: 0 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: 三 1月 21 15:42:27 CST 2026
;; MSG SIZE rcvd: 157
[root@docker-master nfs-cli]# dig -t a nginx-storageclass-1.nginx-svc.default.svc.cluster.local. @10.96.0.10
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.16 <<>> -t a nginx-storageclass-1.nginx-svc.default.svc.cluster.local. @10.96.0.10
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23052
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;nginx-storageclass-1.nginx-svc.default.svc.cluster.local. IN A
;; ANSWER SECTION:
nginx-storageclass-1.nginx-svc.default.svc.cluster.local. 30 IN A 10.244.148.203
;; Query time: 0 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: 三 1月 21 15:42:48 CST 2026
;; MSG SIZE rcvd: 157
在集群里创建pod访问域名，观察负载均衡
[root@docker-master nfs-cli]# kubectl run -it centos --image=centos:7 --image-pull-policy=IfNotPresent
[root@docker-master nfs-cli]# kubectl exec -it centos -- /bin/bash
[root@centos /]# curl http://nginx-svc.default.svc.cluster.local.
this is storage01
[root@centos /]# curl http://nginx-svc.default.svc.cluster.local.
this is storage02
直接访问pod域名
[root@centos /]# curl http://nginx-storageclass-1.nginx-svc.default.svc.cluster.local.
this is storage02
[root@centos /]# curl http://nginx-storageclass-0.nginx-svc.default.svc.cluster.local.
this is storage01
金丝雀发布更新（灰度发布）
它将按照与 Pod 终止相同的顺序(从最大序号到最小序号)进行，每次更新一个 Pod。
Statefulset可以使用partition参数来实现金丝雀更新，partition参数可以控制StatefulSet控制器更新的 Pod，下面，就进行StatefulSet控制器的金丝雀更新实战。
[root@docker-master nfs-cli]# kubectl get sts nginx-storageclass -o yaml
......
updateStrategy:
rollingUpdate:
partition: 0 #默认分区号0，表示更新0及大于0的所有pod
type: RollingUpdate
......
说明：
使用patch参数来指定了statefu1set控制器的partition参数为1，表示当更新时，只有Pod的编号大于等 于1的才更新。
修改更新
[root@docker-master nfs-cli]# kubectl patch sts nginx-storageclass -p '{"spec":{"updateStrategy":{"rollingUpdate":{"partition":1}}}}'
再次观察，值设置为1
updateStrategy:
rollingUpdate:
partition: 1
查看更新前版本
[root@docker-master nfs-cli]# kubectl describe pod nginx-storageclass-0 | grep Image:
Image: nginx:1.26-alpine
[root@docker-master nfs-cli]# kubectl describe pod nginx-storageclass-1 | grep Image:
Image: nginx:1.26-alpine
更新版本
[root@docker-master nfs-cli]# kubectl set image sts/nginx-storageclass c1=nginx:1.29-alpine
statefulset.apps/nginx-storageclass image updated
再次查看版本，发现只有web-1进行了更新
[root@docker-master nfs-cli]# kubectl describe pod nginx-storageclass-0 | grep Image:
Image: nginx:1.26-alpine
[root@docker-master nfs-cli]# kubectl describe pod nginx-storageclass-1 | grep Image:
Image: nginx:1.29-alpine
直观查看pod版本列表
# 以列的形式查看
[root@docker-master nfs-cli]# kubectl get pod -o custom-columns=Name:metadata.name,Image:spec.containers[0].image
Name Image
centos centos:7
nfs-client-provisioner-56fc4bcc87-hkk89 registry.cn-beijing.aliyuncs.com/pylixm/nfs-subdir-external-provisioner:v4.0.0
nginx-storageclass-0 nginx:1.26-alpine
nginx-storageclass-1 nginx:1.29-alpine
扩容数量为4，观察版本更新
[root@docker-master nfs-cli]# kubectl scale sts nginx-storageclass --replicas=4
statefulset.apps/nginx-storageclass scaled
[root@docker-master nfs-cli]# kubectl get pod -o custom-columns=Name:metadata.name,Image:spec.containers[0].image
Name Image
centos centos:7
nfs-client-provisioner-56fc4bcc87-hkk89 registry.cn-beijing.aliyuncs.com/pylixm/nfs-subdir-external-provisioner:v4.0.0
nginx-storageclass-0 nginx:1.26-alpine
nginx-storageclass-1 nginx:1.29-alpine
nginx-storageclass-2 nginx:1.29-alpine
nginx-storageclass-3 nginx:1.29-alpine
发现：新增的pod都使用的新版本
缩容数量为1，观察版本变化
[root@docker-master nfs-cli]# kubectl scale sts nginx-storageclass --replicas=1
statefulset.apps/nginx-storageclass scaled
[root@docker-master nfs-cli]# kubectl get pod -o custom-columns=Name:metadata.name,Image:spec.containers[0].image
Name Image
centos centos:7
nfs-client-provisioner-56fc4bcc87-hkk89 registry.cn-beijing.aliyuncs.com/pylixm/nfs-subdir-external-provisioner:v4.0.0
nginx-storageclass-0 nginx:1.26-alpine
发现，就剩下原来的老版本的pod，说明位置没变，不可变基础设施，再次扩容，观察版本
[root@docker-master nfs-cli]# kubectl scale sts nginx-storageclass --replicas=6
statefulset.apps/nginx-storageclass scaled
[root@docker-master nfs-cli]# kubectl get pod -o custom-columns=Name:metadata.name,Image:spec.containers[0].image
Name Image
centos centos:7
nfs-client-provisioner-56fc4bcc87-hkk89 registry.cn-beijing.aliyuncs.com/pylixm/nfs-subdir-external-provisioner:v4.0.0
nginx-storageclass-0 nginx:1.26-alpine
nginx-storageclass-1 nginx:1.29-alpine
nginx-storageclass-2 nginx:1.29-alpine
nginx-storageclass-3 nginx:1.29-alpine
nginx-storageclass-4 nginx:1.29-alpine
nginx-storageclass-5 nginx:1.29-alpine
扩容依旧是按照partition的位置部署新版本
实现全部更新
[root@docker-master nfs-cli]# kubectl patch sts nginx-storageclass -p '{"spec":{"updateStrategy":{"rollingUpdate":{"partition":0}}}}'
statefulset.apps/nginx-storageclass patched
[root@docker-master nfs-cli]# kubectl get pod -o custom-columns=Name:metadata.name,Image:spec.containers[0].image
Name Image
centos centos:7
nfs-client-provisioner-56fc4bcc87-hkk89 registry.cn-beijing.aliyuncs.com/pylixm/nfs-subdir-external-provisioner:v4.0.0
nginx-storageclass-0 nginx:1.29-alpine
nginx-storageclass-1 nginx:1.29-alpine
nginx-storageclass-2 nginx:1.29-alpine
nginx-storageclass-3 nginx:1.29-alpine
nginx-storageclass-4 nginx:1.29-alpine
nginx-storageclass-5 nginx:1.29-alpine
控制器功能对比
| 控制器 | 版本升级/回退 | 副本扩容/缩减 |
|---|---|---|
| Deployment | ✅ | ✅ |
| ReplicaSet | ❌ | ✅ |
| DaemonSet | ✅ | ❌ |
| StatefulSet | ✅ | ✅ |
| Job | ❌ | ❌ |
| CronJob | ❌ | ❌ |
更多推荐
20
0- 0
已为社区贡献14条内容
所有评论(0)