Pod控制器
k8s Pod控制器
Pod控制器
Pod时k8s的最小管理单元,按照Pod创建方式可分为两类:
- 自助式:k8s直接创建Pod,Pod删除后就销毁,不会重建
- 控制器创建:k8s通过控制器创建Pod,删除后会自动重建,满足用户需求
不同场景下k8s控制器使用:
- ReplicationController:比较原始的pod控制器,已经被废弃,由ReplicaSet替代
- ReplicaSet:保证副本数量一直维持在期望值,并支持pod数量扩缩容,镜像版本升级
- Deployment:通过控制ReplicaSet来控制Pod,并支持滚动升级、回退版本
- Horizontal Pod Autoscaler:可以根据集群负载自动水平调整Pod的数量,实现削峰填谷
- DaemonSet:在集群中的指定Node上运行且仅运行一个副本,一般用于守护进程类的任务
- Job:它创建出来的pod只要完成任务就立即退出,不需要重启或重建,用于执行一次性任务
- Cronjob:它创建的Pod负责周期性任务控制,不需要持续后台运行
- StatefulSet:管理有状态应用
ReplicaSet
ReplicaSet 的目的是维护一组在任何时候都处于运行状态的 Pod 副本的稳定集合。 因此,它通常用来保证给定数量的、完全相同的 Pod 的可用性。
支持:
- Pod数量扩缩容
- 镜像版本升降级
资源清单:
apiVersion: apps/v1 # 版本号
kind: ReplicaSet # 类型
metadata: # 元数据
name: # rs名称
namespace: # 所属命名空间
labels: #标签
controller: rs
spec: # 详情描述
replicas: 3 # 副本数量
selector: # 选择器,通过它指定该控制器管理哪些pod
matchLabels: # Labels匹配规则
app: nginx-pod
matchExpressions: # Expressions匹配规则
- {key: app, operator: In, values: [nginx-pod]}
template: # 模板,当副本数量不足时,会根据下面的模板创建pod副本
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
创建pc-replicaset.yaml文件:
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: pc-replicaset
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
#创建名称空间dev
[root@master ~]# kubectl create ns dev
namespace/dev created
[root@master ~]# vim pc-replicaset.yaml
#创建rs
[root@master ~]# kubectl create -f pc-replicaset.yaml
replicaset.apps/pc-replicaset created
#查看Pod控制器
[root@master ~]# kubectl get rs pc-replicaset -n dev -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-replicaset 3 3 3 117s nginx nginx:1.17.1 app=nginx-pod
# DESIRED:期望副本数量
# CURRENT:当前副本数量
# READY:已经准备好提供服务的副本数量
#查看Pod
[root@master ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
pc-replicaset-5snls 1/1 Running 0 3m20s
pc-replicaset-lbvqf 1/1 Running 0 3m20s
pc-replicaset-x77fz 1/1 Running 0 3m20s
扩缩容
#将rc yaml文件修改副本数量为5
[root@master ~]# kubectl edit rs pc-replicaset -n dev
replicaset.apps/pc-replicaset edited
....
spec:
replicas: 5
...
#查看Pod数量
[root@master ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
pc-replicaset-5snls 1/1 Running 0 11m
pc-replicaset-lbvqf 1/1 Running 0 11m
pc-replicaset-wj8b5 1/1 Running 0 5m34s
pc-replicaset-x4cwn 1/1 Running 0 5m34s
pc-replicaset-x77fz 1/1 Running 0 11m
#也可以使用命令来实现
[root@master ~]# kubectl scale rs pc-replicaset --replicas=2 -n dev
replicaset.apps/pc-replicaset scaled
[root@master ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
pc-replicaset-5snls 1/1 Running 0 13m
pc-replicaset-lbvqf 1/1 Running 0 13m
镜像版本升降级
[root@master ~]# kubectl get rs pc-replicaset -n dev -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-replicaset 2 2 2 21m nginx nginx:1.17.1 app=nginx-pod
#修改配置文件
[root@master ~]# kubectl edit rs pc-replicaset -n dev
...
spec:
containers:
- image: nginx:1.17.2
...
#升级为1.17.2
[root@master ~]# kubectl get rs pc-replicaset -n dev -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-replicaset 2 2 2 23m nginx nginx:1.17.2 app=nginx-pod
#命令格式
[root@master ~]# kubectl set image rs pc-replicaset nginx=nginx:1.17.1 -n dev
replicaset.apps/pc-replicaset image updated
[root@master ~]# kubectl get rs pc-replicaset -n dev -o wide
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
pc-replicaset 2 2 2 30m nginx nginx:1.17.1 app=nginx-pod
删除RS
#kubernetes删除RS前,会将RS的replicasclear调整为0,等待所有的Pod被删除后,在执行RS对象的删除
[root@master ~]# kubectl delete rs pc-replicaset -n dev
replicaset.apps "pc-replicaset" deleted
[root@master ~]# kubectl get pod -n dev -o wide
No resources found in dev namespace.
[root@master ~]# kubectl get pod -n dev
No resources found in dev namespace.
# 如果希望仅仅删除RS对象(保留Pod),可以使用kubectl delete命令时添加--cascade=false选项(不推荐)。
# 也可以使用yaml直接删除(推荐)
[root@master ~]# kubectl delete -f pc-replicaset.yaml
replicaset.apps "pc-replicaset" deleted
Deployment
一个 Deployment 为 Pod 和 ReplicaSet 提供声明式的更新能力。
你负责描述 Deployment 中的 目标状态,而 Deployment 控制器(Controller) 以受控速率更改实际状态, 使其变为期望状态。你可以定义 Deployment 以创建新的 ReplicaSet,或删除现有 Deployment, 并通过新的 Deployment 收养其资源。
- 通过管理RS来间接管理Pod
- 支持RS所有功能
- 支持发布停止,继续
- 支持版本滚动更新和版本回退
apiVersion: apps/v1 # 版本号
kind: Deployment # 类型
metadata: # 元数据
name: # rs名称
namespace: # 所属命名空间
labels: #标签
controller: deploy
spec: # 详情描述
replicas: 3 # 副本数量
revisionHistoryLimit: 3 # 保留历史版本
paused: false # 暂停部署,默认是false
progressDeadlineSeconds: 600 # 部署超时时间(s),默认是600
strategy: # 策略
type: RollingUpdate # 滚动更新策略
rollingUpdate: # 滚动更新
maxSurge: 30% # 最大额外可以存在的副本数,可以为百分比,也可以为整数
maxUnavailable: 30% # 最大不可用状态的 Pod 的最大值,可以为百分比,也可以为整数
selector: # 选择器,通过它指定该控制器管理哪些pod
matchLabels: # Labels匹配规则
app: nginx-pod
matchExpressions: # Expressions匹配规则
- {key: app, operator: In, values: [nginx-pod]}
template: # 模板,当副本数量不足时,会根据下面的模板创建pod副本
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
创建pc-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: pc-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
#创建deployment
[root@master ~]# vim pc-deployment.yaml
[root@master ~]# kubectl create -f pc-deployment.yaml
deployment.apps/pc-deployment created
[root@master ~]# kubectl get deploy -n dev -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
pc-deployment 3/3 3 3 22s nginx nginx:1.17.1 app=nginx-pod
#查看deployment管理的rs
[root@master ~]# kubectl get rs -n dev
NAME DESIRED CURRENT READY AGE
pc-deployment-5d89bdfbf9 3 3 3 49s
[root@master ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
pc-deployment-5d89bdfbf9-27mps 1/1 Running 0 2m8s
pc-deployment-5d89bdfbf9-65kdh 1/1 Running 0 2m8s
pc-deployment-5d89bdfbf9-c4tqv 1/1 Running 0 2m8s
扩缩容
# 变更副本数量为5个
[root@master ~]# kubectl scale deploy pc-deployment --replicas=5 -n dev
deployment.apps/pc-deployment scaled
[root@master ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
pc-deployment-5d89bdfbf9-27mps 1/1 Running 0 14m
pc-deployment-5d89bdfbf9-65kdh 1/1 Running 0 14m
pc-deployment-5d89bdfbf9-c4tqv 1/1 Running 0 14m
pc-deployment-5d89bdfbf9-j2lw7 1/1 Running 0 2m28s
pc-deployment-5d89bdfbf9-rrndt 1/1 Running 0 2m28s
# 编辑deployment的副本数量,修改spec:replicas: 3即可
[root@master ~]# kubectl edit deploy pc-deployment -n dev
deployment.apps/pc-deployment edited
[root@master ~]# kubectl get pod -n dev
NAME READY STATUS RESTARTS AGE
pc-deployment-5d89bdfbf9-27mps 1/1 Running 0 15m
pc-deployment-5d89bdfbf9-65kdh 1/1 Running 0 15m
pc-deployment-5d89bdfbf9-c4tqv 1/1 Running 0 15m
镜像更新
两种策略:
- 重建更新
- 滚动更新(默认)
strategy:指定新的Pod替换旧的Pod的策略, 支持两个属性:
type:指定策略类型,支持两种策略
Recreate:在创建出新的Pod之前会先杀掉所有已存在的Pod
RollingUpdate:滚动更新,就是杀死一部分,就启动一部分,在更新过程中,存在两个版本Pod
rollingUpdate:当type为RollingUpdate时生效,用于为RollingUpdate设置参数,支持两个属性:
maxUnavailable:用来指定在升级过程中不可用Pod的最大数量,默认为25%。
maxSurge: 用来指定在升级过程中可以超过期望的Pod的最大数量,默认为25%。
- 编辑pc-deployment.yaml ,在spec节点下添加更新策略
spec:
strategy: # 策略
type: Recreate # 重建更新
-
操作
#应用 [root@master ~]# kubectl apply -f pc-deployment.yaml Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply deployment.apps/pc-deployment configured #更新镜像版本 [root@master ~]# kubectl get deploy -n dev -o wide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELEC TOR pc-deployment 3/3 3 3 51m nginx nginx:1.17.1 app=n ginx-pod [root@master ~]# kubectl set image deployment pc-deployment nginx=nginx:1.17.2 -n dev deployment.apps/pc-deployment image updated #新开一个窗口,观察Pod创建过程 [root@master ~]# kubectl get pod -n dev -w NAME READY STATUS RESTARTS AGE pc-deployment-5d89bdfbf9-8k2ht 1/1 Running 0 3m15s pc-deployment-5d89bdfbf9-s5jmd 1/1 Running 0 3m11s pc-deployment-5d89bdfbf9-w9xk7 1/1 Running 0 3m13s pc-deployment-5d89bdfbf9-w9xk7 1/1 Terminating 0 3m59s pc-deployment-5d89bdfbf9-8k2ht 1/1 Terminating 0 4m1s pc-deployment-5d89bdfbf9-s5jmd 1/1 Terminating 0 3m57s pc-deployment-5d89bdfbf9-s5jmd 0/1 Terminating 0 3m59s pc-deployment-5d89bdfbf9-8k2ht 0/1 Terminating 0 4m3s pc-deployment-5d89bdfbf9-w9xk7 0/1 Terminating 0 4m1s pc-deployment-5d89bdfbf9-8k2ht 0/1 Terminating 0 4m4s pc-deployment-5d89bdfbf9-8k2ht 0/1 Terminating 0 4m4s pc-deployment-5d89bdfbf9-w9xk7 0/1 Terminating 0 4m2s pc-deployment-5d89bdfbf9-w9xk7 0/1 Terminating 0 4m2s pc-deployment-5d89bdfbf9-s5jmd 0/1 Terminating 0 4m pc-deployment-5d89bdfbf9-s5jmd 0/1 Terminating 0 4m pc-deployment-675d469f8b-prks5 0/1 Pending 0 0s pc-deployment-675d469f8b-prks5 0/1 Pending 0 0s pc-deployment-675d469f8b-2b88f 0/1 Pending 0 0s pc-deployment-675d469f8b-sgm7x 0/1 Pending 0 0s pc-deployment-675d469f8b-2b88f 0/1 Pending 0 0s pc-deployment-675d469f8b-sgm7x 0/1 Pending 0 0s pc-deployment-675d469f8b-prks5 0/1 ContainerCreating 0 1s pc-deployment-675d469f8b-2b88f 0/1 ContainerCreating 0 1s pc-deployment-675d469f8b-sgm7x 0/1 ContainerCreating 0 2s pc-deployment-675d469f8b-sgm7x 1/1 Running 0 3s pc-deployment-675d469f8b-2b88f 1/1 Running 0 3s pc-deployment-675d469f8b-prks5 1/1 Running 0 3s #先删除Pod在重建 #查看版本 [root@master ~]# kubectl get deploy -n dev -o wide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR pc-deployment 3/3 3 3 55m nginx nginx:1.17.2 app=nginx-pod
版本回退
#删除原来的控制器创建新控制器
[root@master ~]# kubectl create -f pc-deployment.yaml --record
deployment.apps/pc-deployment created
[root@master ~]# kubectl get deploy,rs,pod -n dev
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/pc-deployment 3/3 3 3 26s
NAME DESIRED CURRENT READY AGE
replicaset.apps/pc-deployment-5d89bdfbf9 3 3 3 26s
NAME READY STATUS RESTARTS AGE
pod/pc-deployment-5d89bdfbf9-bjgfg 1/1 Running 0 26s
pod/pc-deployment-5d89bdfbf9-fk7kp 1/1 Running 0 26s
pod/pc-deployment-5d89bdfbf9-wbzfm 1/1 Running 0 26s
#更新镜像
[root@master ~]# kubectl set image deployment pc-deployment nginx=nginx:1.17.2 -n dev
deployment.apps/pc-deployment image updated
#rs现在有两个
[root@master ~]# kubectl get rs -n dev
NAME DESIRED CURRENT READY AGE
pc-deployment-5d89bdfbf9 0 0 0 4m5s
pc-deployment-675d469f8b 3 3 3 72s
#rs更新过程
[root@master ~]# kubectl get rs -n dev -w
NAME DESIRED CURRENT READY AGE
pc-deployment-5d89bdfbf9 3 3 3 2m24s
pc-deployment-5d89bdfbf9 0 3 3 2m43s
pc-deployment-5d89bdfbf9 0 3 3 2m43s
pc-deployment-5d89bdfbf9 0 0 0 2m43s
pc-deployment-675d469f8b 3 0 0 0s
pc-deployment-675d469f8b 3 0 0 0s
pc-deployment-675d469f8b 3 3 0 0s
pc-deployment-675d469f8b 3 3 1 3s
pc-deployment-675d469f8b 3 3 2 3s
pc-deployment-675d469f8b 3 3 3 3s
#Pod重建更新过程
[root@master ~]# kubectl get pod -n dev -w
NAME READY STATUS RESTARTS AGE
pc-deployment-5d89bdfbf9-bjgfg 1/1 Running 0 2m36s
pc-deployment-5d89bdfbf9-fk7kp 1/1 Running 0 2m36s
pc-deployment-5d89bdfbf9-wbzfm 1/1 Running 0 2m36s
pc-deployment-5d89bdfbf9-fk7kp 1/1 Terminating 0 2m43s
pc-deployment-5d89bdfbf9-bjgfg 1/1 Terminating 0 2m43s
pc-deployment-5d89bdfbf9-wbzfm 1/1 Terminating 0 2m43s
pc-deployment-5d89bdfbf9-bjgfg 0/1 Terminating 0 2m45s
pc-deployment-5d89bdfbf9-fk7kp 0/1 Terminating 0 2m45s
pc-deployment-5d89bdfbf9-wbzfm 0/1 Terminating 0 2m45s
pc-deployment-5d89bdfbf9-wbzfm 0/1 Terminating 0 2m53s
pc-deployment-5d89bdfbf9-wbzfm 0/1 Terminating 0 2m53s
pc-deployment-5d89bdfbf9-fk7kp 0/1 Terminating 0 2m53s
pc-deployment-5d89bdfbf9-fk7kp 0/1 Terminating 0 2m53s
pc-deployment-5d89bdfbf9-bjgfg 0/1 Terminating 0 2m53s
pc-deployment-5d89bdfbf9-bjgfg 0/1 Terminating 0 2m53s
pc-deployment-675d469f8b-d989b 0/1 Pending 0 0s
pc-deployment-675d469f8b-t6qfc 0/1 Pending 0 0s
pc-deployment-675d469f8b-d989b 0/1 Pending 0 0s
pc-deployment-675d469f8b-rf7pd 0/1 Pending 0 0s
pc-deployment-675d469f8b-rf7pd 0/1 Pending 0 0s
pc-deployment-675d469f8b-t6qfc 0/1 Pending 0 0s
pc-deployment-675d469f8b-d989b 0/1 ContainerCreating 0 0s
pc-deployment-675d469f8b-rf7pd 0/1 ContainerCreating 0 1s
pc-deployment-675d469f8b-t6qfc 0/1 ContainerCreating 0 1s
pc-deployment-675d469f8b-rf7pd 1/1 Running 0 3s
pc-deployment-675d469f8b-d989b 1/1 Running 0 3s
pc-deployment-675d469f8b-t6qfc 1/1 Running 0 3s
两个RS旧的RS是为了能够版本回退
deployment支持版本升级过程中的暂停、继续功能以及版本回退等诸多功能,下面具体来看.
kubectl rollout: 版本升级相关功能,支持下面的选项:
-
status 显示当前升级状态
-
history 显示 升级历史记录
-
pause 暂停版本升级过程
-
resume 继续已经暂停的版本升级过程
-
restart 重启版本升级过程
-
undo 回滚到上一级版本(可以使用–to-revision回滚到指定版本)
#查看当前版本状态
[root@master ~]# kubectl rollout status deploy pc-deployment -n dev
deployment "pc-deployment" successfully rolled out
# 查看升级历史记录
[root@master ~]# kubectl rollout history deploy pc-deployment -n dev
deployment.apps/pc-deployment
REVISION CHANGE-CAUSE
1 kubectl create --filename=pc-deployment.yaml --record=true
2 kubectl create --filename=pc-deployment.yaml --record=true
#升级
[root@master ~]# kubectl set image deployment pc-deployment nginx=nginx:1.17.3 -n dev
deployment.apps/pc-deployment image updated
[root@master ~]# kubectl rollout history deploy pc-deployment -n dev
deployment.apps/pc-deployment
REVISION CHANGE-CAUSE
1 kubectl create --filename=pc-deployment.yaml --record=true
2 kubectl create --filename=pc-deployment.yaml --record=true
3 kubectl create --filename=pc-deployment.yaml --record=true
#回退版本 这里直接使用--to-revision=1回滚到了1版本, 如果省略这个选项,就是回退到上个版本,就是2版本
[root@master ~]# kubectl get deploy -n dev -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
pc-deployment 3/3 3 3 20m nginx nginx:1.17.3 app=nginx-pod
#查看rs
[root@master ~]# kubectl get rs -n dev
NAME DESIRED CURRENT READY AGE
pc-deployment-5d89bdfbf9 0 0 0 20m
pc-deployment-675d469f8b 0 0 0 17m
pc-deployment-7865c58bdf 3 3 3 2m12s
[root@master ~]# kubectl rollout undo deployment pc-deployment --to-revision=1 -n dev
deployment.apps/pc-deployment rolled back
[root@master ~]# kubectl get rs -n dev
NAME DESIRED CURRENT READY AGE
pc-deployment-5d89bdfbf9 3 3 3 21m
pc-deployment-675d469f8b 0 0 0 18m
pc-deployment-7865c58bdf 0 0 0 3m
HPA(HorizontalPodAutoscaler)
在 Kubernetes 中,HorizontalPodAutoscaler 自动更新工作负载资源 (例如 Deployment 或者 StatefulSet), 目的是自动扩缩工作负载以满足需求。
水平扩缩意味着对增加的负载的响应是部署更多的 Pod。 这与 “垂直(Vertical)” 扩缩不同,对于 Kubernetes, 垂直扩缩意味着将更多资源(例如:内存或 CPU)分配给已经为工作负载运行的 Pod。
如果负载减少,并且 Pod 的数量高于配置的最小值, HorizontalPodAutoscaler 会指示工作负载资源(Deployment、StatefulSet 或其他类似资源)缩减。
- 安装metrics-server手机集群中资源使用情况
# 安装git
[root@master ~]# yum install git -y
# 获取metrics-server, 注意使用的版本
[root@master ~]# git clone -b v0.3.6 https://github.com/kubernetes-incubator/metrics-server
# 修改deployment, 注意修改的是镜像和初始化参数
[root@master ~]# cd /root/metrics-server/deploy/1.8+/
[root@master 1.8+]# vim metrics-server-deployment.yaml
按图中添加下面选项
hostNetwork: true
image: registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6
args:
- --kubelet-insecure-tls
- --kubelet-preferred-address-types=InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP
#安装metrics-server
[root@master 1.8+]# kubectl apply -f ./
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
serviceaccount/metrics-server created
deployment.apps/metrics-server created
service/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
#查看Pod运行情况
[root@master 1.8+]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-6955765f44-ffpr7 1/1 Running 0 6h22m
coredns-6955765f44-rjt82 1/1 Running 0 6h6m
etcd-master 1/1 Running 1 28h
kube-apiserver-master 1/1 Running 1 28h
kube-controller-manager-master 1/1 Running 1 28h
kube-proxy-lqdks 1/1 Running 0 28h
kube-proxy-wcn2x 1/1 Running 0 28h
kube-proxy-x78bv 1/1 Running 1 28h
kube-scheduler-master 1/1 Running 1 28h
metrics-server-6b976979db-g84tb 1/1 Running 0 25s
[root@master ~]# kubectl top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
master 339m 16% 1102Mi 30%
node1 49m 2% 582Mi 33%
node2 118m 5% 594Mi 34%
- 准备deploy和service
- service对外暴露接口
- deploy部署Pod
#创建deployment
[root@master 1.8+]# kubectl run nginx --image=nginx:1.17.1 --requests=cpu=100m -n dev
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/nginx created
[root@master 1.8+]# kubectl get deploy,pod -n dev
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 1/1 1 1 57s
NAME READY STATUS RESTARTS AGE
pod/nginx-778cb5fb7b-vh9qr 1/1 Running 0 57s
#创建service
[root@master 1.8+]# kubectl expose deployment nginx --type=NodePort --port=80 -n dev
service/nginx exposed
[root@master 1.8+]# kubectl get svc -n dev
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx NodePort 10.99.103.181 <none> 80:30243/TCP 13s
- 部署HPA
创建pc-hpa.yaml
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
name: pc-hpa
namespace: dev
spec:
minReplicas: 1 #最小pod数量
maxReplicas: 10 #最大pod数量
targetCPUUtilizationPercentage: 3 # CPU使用率指标
scaleTargetRef: # 指定要控制的nginx信息
apiVersion: apps/v1
kind: Deployment
name: nginx
[root@master ~]# kubectl create -f pc-hpa.yaml
horizontalpodautoscaler.autoscaling/pc-hpa created
[root@master ~]# kubectl get hpa -n dev
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
pc-hpa Deployment/nginx <unknown>/3% 1 10 0 10s
新开多个窗口检测Pod及其他变化
使用postman来测试:
保存这个测试:
发出10000条请求
查看各个窗口动态变化情况:
#Pod
[root@master ~]# kubectl get pod -n dev -w
NAME READY STATUS RESTARTS AGE
nginx-778cb5fb7b-vh9qr 1/1 Running 0 14m
nginx-778cb5fb7b-26w69 0/1 Pending 0 0s
nginx-778cb5fb7b-q28z6 0/1 Pending 0 0s
nginx-778cb5fb7b-pgl7d 0/1 Pending 0 0s
nginx-778cb5fb7b-26w69 0/1 Pending 0 0s
nginx-778cb5fb7b-q28z6 0/1 Pending 0 0s
nginx-778cb5fb7b-pgl7d 0/1 Pending 0 0s
nginx-778cb5fb7b-26w69 0/1 ContainerCreating 0 0s
nginx-778cb5fb7b-q28z6 0/1 ContainerCreating 0 0s
nginx-778cb5fb7b-pgl7d 0/1 ContainerCreating 0 0s
nginx-778cb5fb7b-26w69 1/1 Running 0 3s
nginx-778cb5fb7b-pgl7d 1/1 Running 0 3s
nginx-778cb5fb7b-q28z6 1/1 Running 0 3s
nginx-778cb5fb7b-rd6nh 0/1 Pending 0 1s
nginx-778cb5fb7b-rd6nh 0/1 Pending 0 1s
nginx-778cb5fb7b-h5t9t 0/1 Pending 0 0s
nginx-778cb5fb7b-h5t9t 0/1 Pending 0 0s
nginx-778cb5fb7b-rd6nh 0/1 ContainerCreating 0 1s
nginx-778cb5fb7b-h5t9t 0/1 ContainerCreating 0 0s
nginx-778cb5fb7b-rd6nh 1/1 Running 0 3s
nginx-778cb5fb7b-h5t9t 1/1 Running 0 3s
nginx-778cb5fb7b-z89l9 0/1 Pending 0 0s
nginx-778cb5fb7b-z89l9 0/1 Pending 0 0s
nginx-778cb5fb7b-h6jm6 0/1 Pending 0 0s
nginx-778cb5fb7b-854jz 0/1 Pending 0 0s
nginx-778cb5fb7b-56jq5 0/1 Pending 0 0s
nginx-778cb5fb7b-h6jm6 0/1 Pending 0 0s
nginx-778cb5fb7b-854jz 0/1 Pending 0 0s
nginx-778cb5fb7b-56jq5 0/1 Pending 0 0s
nginx-778cb5fb7b-z89l9 0/1 ContainerCreating 0 0s
nginx-778cb5fb7b-h6jm6 0/1 ContainerCreating 0 0s
nginx-778cb5fb7b-854jz 0/1 ContainerCreating 0 0s
nginx-778cb5fb7b-56jq5 0/1 ContainerCreating 0 1s
nginx-778cb5fb7b-z89l9 1/1 Running 0 7s
nginx-778cb5fb7b-854jz 1/1 Running 0 8s
nginx-778cb5fb7b-56jq5 1/1 Running 0 9s
nginx-778cb5fb7b-h6jm6 1/1 Running 0 9s
[root@master ~]# kubectl get pod -n dev -w
NAME READY STATUS RESTARTS AGE
nginx-778cb5fb7b-26w69 1/1 Running 0 104s
nginx-778cb5fb7b-56jq5 1/1 Running 0 59s
nginx-778cb5fb7b-854jz 1/1 Running 0 59s
nginx-778cb5fb7b-h5t9t 1/1 Running 0 89s
nginx-778cb5fb7b-h6jm6 1/1 Running 0 59s
nginx-778cb5fb7b-pgl7d 1/1 Running 0 104s
nginx-778cb5fb7b-q28z6 1/1 Running 0 104s
nginx-778cb5fb7b-rd6nh 1/1 Running 0 90s
nginx-778cb5fb7b-vh9qr 1/1 Running 0 54m
nginx-778cb5fb7b-z89l9 1/1 Running 0 59s
^C[root@master ~]# kubectl get pod -n dev -w
NAME READY STATUS RESTARTS AGE
nginx-778cb5fb7b-26w69 1/1 Running 0 2m27s
nginx-778cb5fb7b-56jq5 1/1 Running 0 102s
nginx-778cb5fb7b-854jz 1/1 Running 0 102s
nginx-778cb5fb7b-h5t9t 1/1 Running 0 2m12s
nginx-778cb5fb7b-h6jm6 1/1 Running 0 102s
nginx-778cb5fb7b-pgl7d 1/1 Running 0 2m27s
nginx-778cb5fb7b-q28z6 1/1 Running 0 2m27s
nginx-778cb5fb7b-rd6nh 1/1 Running 0 2m13s
nginx-778cb5fb7b-vh9qr 1/1 Running 0 54m
nginx-778cb5fb7b-z89l9 1/1 Running 0 102s
nginx-778cb5fb7b-h5t9t 1/1 Terminating 0 6m21s
nginx-778cb5fb7b-56jq5 1/1 Terminating 0 5m51s
nginx-778cb5fb7b-h6jm6 1/1 Terminating 0 5m52s
nginx-778cb5fb7b-rd6nh 1/1 Terminating 0 6m23s
nginx-778cb5fb7b-854jz 1/1 Terminating 0 5m52s
nginx-778cb5fb7b-z89l9 1/1 Terminating 0 5m52s
nginx-778cb5fb7b-854jz 0/1 Terminating 0 5m55s
nginx-778cb5fb7b-h5t9t 0/1 Terminating 0 6m26s
nginx-778cb5fb7b-z89l9 0/1 Terminating 0 5m56s
nginx-778cb5fb7b-h6jm6 0/1 Terminating 0 5m56s
nginx-778cb5fb7b-rd6nh 0/1 Terminating 0 6m28s
nginx-778cb5fb7b-h6jm6 0/1 Terminating 0 5m57s
nginx-778cb5fb7b-z89l9 0/1 Terminating 0 5m57s
nginx-778cb5fb7b-h5t9t 0/1 Terminating 0 6m27s
nginx-778cb5fb7b-56jq5 0/1 Terminating 0 5m57s
nginx-778cb5fb7b-854jz 0/1 Terminating 0 5m57s
nginx-778cb5fb7b-854jz 0/1 Terminating 0 5m58s
nginx-778cb5fb7b-854jz 0/1 Terminating 0 5m58s
nginx-778cb5fb7b-rd6nh 0/1 Terminating 0 6m30s
nginx-778cb5fb7b-rd6nh 0/1 Terminating 0 6m30s
nginx-778cb5fb7b-56jq5 0/1 Terminating 0 5m59s
nginx-778cb5fb7b-56jq5 0/1 Terminating 0 5m59s
nginx-778cb5fb7b-h6jm6 0/1 Terminating 0 6m4s
nginx-778cb5fb7b-h6jm6 0/1 Terminating 0 6m4s
nginx-778cb5fb7b-h5t9t 0/1 Terminating 0 6m35s
nginx-778cb5fb7b-h5t9t 0/1 Terminating 0 6m35s
nginx-778cb5fb7b-z89l9 0/1 Terminating 0 6m5s
nginx-778cb5fb7b-z89l9 0/1 Terminating 0 6m5s
nginx-778cb5fb7b-pgl7d 1/1 Terminating 0 7m38s
nginx-778cb5fb7b-q28z6 1/1 Terminating 0 7m39s
nginx-778cb5fb7b-26w69 1/1 Terminating 0 7m39s
nginx-778cb5fb7b-pgl7d 0/1 Terminating 0 7m55s
nginx-778cb5fb7b-q28z6 0/1 Terminating 0 7m55s
nginx-778cb5fb7b-26w69 0/1 Terminating 0 7m55s
nginx-778cb5fb7b-26w69 0/1 Terminating 0 7m55s
nginx-778cb5fb7b-26w69 0/1 Terminating 0 7m55s
nginx-778cb5fb7b-pgl7d 0/1 Terminating 0 7m56s
nginx-778cb5fb7b-pgl7d 0/1 Terminating 0 7m56s
nginx-778cb5fb7b-q28z6 0/1 Terminating 0 7m56s
nginx-778cb5fb7b-q28z6 0/1 Terminating 0 7m56s
#hpa
[root@master ~]# kubectl get hpa -n dev -w
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
pc-hpa Deployment/nginx 0%/3% 1 10 1 40m
pc-hpa Deployment/nginx 17%/3% 1 10 1 41m
pc-hpa Deployment/nginx 17%/3% 1 10 4 41m
pc-hpa Deployment/nginx 17%/3% 1 10 6 41m
pc-hpa Deployment/nginx 5%/3% 1 10 6 42m
pc-hpa Deployment/nginx 5%/3% 1 10 10 42m
pc-hpa Deployment/nginx 1%/3% 1 10 10 43m
pc-hpa Deployment/nginx 0%/3% 1 10 10 44m
pc-hpa Deployment/nginx 0%/3% 1 10 10 47m
pc-hpa Deployment/nginx 0%/3% 1 10 4 48m
pc-hpa Deployment/nginx 0%/3% 1 10 4 48m
pc-hpa Deployment/nginx 0%/3% 1 10 1 49m
#deploy
[root@master ~]# kubectl get deploy -n dev -w
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 10/10 10 10 55m
nginx 10/4 10 10 58m
nginx 10/4 10 10 58m
nginx 4/4 4 4 59m
nginx 4/1 4 4 60m
nginx 4/1 4 4 60m
nginx 1/1 1 1 60m
观察发现Pod随着请求的多少在动态扩缩容
DaemonSet
DaemonSet 确保全部(或者某些)节点上运行一个 Pod 的副本。 当有节点加入集群时, 也会为他们新增一个 Pod 。 当有节点从集群移除时,这些 Pod 也会被回收。删除 DaemonSet 将会删除它创建的所有 Pod。
DaemonSet 的一些典型用法:
- 在每个节点上运行集群守护进程
- 在每个节点上运行日志收集守护进程
- 在每个节点上运行监控守护进程
apiVersion: apps/v1 # 版本号
kind: DaemonSet # 类型
metadata: # 元数据
name: # rs名称
namespace: # 所属命名空间
labels: #标签
controller: daemonset
spec: # 详情描述
revisionHistoryLimit: 3 # 保留历史版本
updateStrategy: # 更新策略
type: RollingUpdate # 滚动更新策略
rollingUpdate: # 滚动更新
maxUnavailable: 1 # 最大不可用状态的 Pod 的最大值,可以为百分比,也可以为整数
selector: # 选择器,通过它指定该控制器管理哪些pod
matchLabels: # Labels匹配规则
app: nginx-pod
matchExpressions: # Expressions匹配规则
- {key: app, operator: In, values: [nginx-pod]}
template: # 模板,当副本数量不足时,会根据下面的模板创建pod副本
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
创建一个yaml文件:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: pc-daemonset
namespace: dev
spec:
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
在master节点之间,观察是否在每个节点上都运行了一个
Job
Job 会创建一个或者多个 Pod,并将继续重试 Pod 的执行,直到指定数量的 Pod 成功终止。 随着 Pod 成功结束,Job 跟踪记录成功完成的 Pod 个数。 当数量达到指定的成功个数阈值时,任务(即 Job)结束。 删除 Job 的操作会清除所创建的全部 Pod。 挂起 Job 的操作会删除 Job 的所有活跃 Pod,直到 Job 被再次恢复执行。
Job资源清单
apiVersion: batch/v1 # 版本号
kind: Job # 类型
metadata: # 元数据
name: # rs名称
namespace: # 所属命名空间
labels: #标签
controller: job
spec: # 详情描述
completions: 1 # 指定job需要成功运行Pods的次数。默认值: 1
parallelism: 1 # 指定job在任一时刻应该并发运行Pods的数量。默认值: 1
activeDeadlineSeconds: 30 # 指定job可运行的时间期限,超过时间还未结束,系统将会尝试进行终止。
backoffLimit: 6 # 指定job失败后进行重试的次数。默认是6
manualSelector: true # 是否可以使用selector选择器选择pod,默认是false
selector: # 选择器,通过它指定该控制器管理哪些pod
matchLabels: # Labels匹配规则
app: counter-pod
matchExpressions: # Expressions匹配规则
- {key: app, operator: In, values: [counter-pod]}
template: # 模板,当副本数量不足时,会根据下面的模板创建pod副本
metadata:
labels:
app: counter-pod
spec:
restartPolicy: Never # 重启策略只能设置为Never或者OnFailure
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 2;done"]
关于重启策略设置的说明:
如果指定为OnFailure,则job会在pod出现故障时重启容器,而不是创建pod,failed次数不变
如果指定为Never,则job会在pod出现故障时创建新的pod,并且故障pod不会消失,也不会重启,failed次数加1
如果指定为Always的话,就意味着一直重启,意味着job任务会重复去执行了,当然不对,所以不能设置为Always
创建yaml文件
apiVersion: batch/v1
kind: Job
metadata:
name: pc-job
namespace: dev
spec:
manualSelector: true
selector:
matchLabels:
app: counter-pod
template:
metadata:
labels:
app: counter-pod
spec:
restartPolicy: Never
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 3;done"]
查看过程
#创建JOb
[root@master ~]# kubectl create -f pc-job.yaml
job.batch/pc-job created
#查看JOb
[root@master ~]# kubectl get job -n dev -w
NAME COMPLETIONS DURATION AGE
pc-job 0/1 0s
pc-job 0/1 0s 0s
pc-job 1/1 31s 31s
#查看Pod
[root@master ~]# kubectl get pod -n dev -w
NAME READY STATUS RESTARTS AGE
pc-job-m2qpk 0/1 Pending 0 0s
pc-job-m2qpk 0/1 Pending 0 0s
pc-job-m2qpk 0/1 ContainerCreating 0 0s
pc-job-m2qpk 1/1 Running 0 4s
pc-job-m2qpk 0/1 Completed 0 31s
# 接下来,调整下pod运行的总数量和并行数量 即:在spec下设置下面两个选项
# completions: 6 # 指定job需要成功运行Pods的次数为6
# parallelism: 3 # 指定job并发运行Pods的数量为3
# 然后重新运行job,观察效果,此时会发现,job会每次运行3个pod,总共执行了6个pod
#Pod过程
[root@master ~]# kubectl get pod -n dev -w
NAME READY STATUS RESTARTS AGE
pc-job-7rt9n 0/1 Pending 0 0s
pc-job-7rt9n 0/1 Pending 0 0s
pc-job-4zxls 0/1 Pending 0 0s
pc-job-qt8b2 0/1 Pending 0 0s
pc-job-4zxls 0/1 Pending 0 0s
pc-job-qt8b2 0/1 Pending 0 0s
pc-job-7rt9n 0/1 ContainerCreating 0 0s
pc-job-4zxls 0/1 ContainerCreating 0 0s
pc-job-qt8b2 0/1 ContainerCreating 0 0s
pc-job-qt8b2 1/1 Running 0 2s
pc-job-4zxls 1/1 Running 0 2s
pc-job-7rt9n 1/1 Running 0 2s
pc-job-4zxls 0/1 Completed 0 29s
pc-job-gw2wq 0/1 Pending 0 0s
pc-job-gw2wq 0/1 Pending 0 0s
pc-job-7rt9n 0/1 Completed 0 30s
pc-job-gw2wq 0/1 ContainerCreating 0 0s
pc-job-8wttt 0/1 Pending 0 0s
pc-job-8wttt 0/1 Pending 0 0s
pc-job-qt8b2 0/1 Completed 0 30s
pc-job-tcnkp 0/1 Pending 0 0s
pc-job-tcnkp 0/1 Pending 0 0s
pc-job-8wttt 0/1 ContainerCreating 0 0s
pc-job-tcnkp 0/1 ContainerCreating 0 0s
pc-job-gw2wq 1/1 Running 0 3s
pc-job-tcnkp 1/1 Running 0 3s
pc-job-8wttt 1/1 Running 0 3s
pc-job-tcnkp 0/1 Completed 0 30s
pc-job-8wttt 0/1 Completed 0 31s
pc-job-gw2wq 0/1 Completed 0 31s
#Job过程
[root@master ~]# kubectl get job -n dev -w
NAME COMPLETIONS DURATION AGE
pc-job 0/6 0s
pc-job 0/6 0s 0s
pc-job 1/6 30s 30s
pc-job 2/6 30s 30s
pc-job 3/6 30s 30s
pc-job 4/6 60s 60s
pc-job 5/6 61s 61s
pc-job 6/6 61s 61s
CronJob
CronJob控制器以Job控制器资源为其管控对象,并借助它管理pod资源对象,Job控制器定义的作业任务在其控制器资源创建之后便会立即执行,但CronJob可以以类似于Linux操作系统的周期性任务作业计划的方式控制其运行时间点及重复运行的方式。也就是说,CronJob可以在特定的时间点(反复的)去运行job任务。
资源清单文件:
apiVersion: batch/v1beta1 # 版本号
kind: CronJob # 类型
metadata: # 元数据
name: # rs名称
namespace: # 所属命名空间
labels: #标签
controller: cronjob
spec: # 详情描述
schedule: # cron格式的作业调度运行时间点,用于控制任务在什么时间执行
concurrencyPolicy: # 并发执行策略,用于定义前一次作业运行尚未完成时是否以及如何运行后一次的作业
failedJobHistoryLimit: # 为失败的任务执行保留的历史记录数,默认为1
successfulJobHistoryLimit: # 为成功的任务执行保留的历史记录数,默认为3
startingDeadlineSeconds: # 启动作业错误的超时时长
jobTemplate: # job控制器模板,用于为cronjob控制器生成job对象;下面其实就是job的定义
metadata:
spec:
completions: 1
parallelism: 1
activeDeadlineSeconds: 30
backoffLimit: 6
manualSelector: true
selector:
matchLabels:
app: counter-pod
matchExpressions: 规则
- {key: app, operator: In, values: [counter-pod]}
template:
metadata:
labels:
app: counter-pod
spec:
restartPolicy: Never
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 20;done"]
Cron 时间表语法
.spec.schedule
字段是必需的。该字段的值遵循 Cron 语法:
# ┌───────────── 分钟 (0 - 59)
# │ ┌───────────── 小时 (0 - 23)
# │ │ ┌───────────── 月的某天 (1 - 31)
# │ │ │ ┌───────────── 月份 (1 - 12)
# │ │ │ │ ┌───────────── 周的某天 (0 - 6)(周日到周一;在某些系统上,7 也是星期日)
# │ │ │ │ │ 或者是 sun,mon,tue,web,thu,fri,sat
# │ │ │ │ │
# │ │ │ │ │
# * * * * *
需要重点解释的几个选项:
schedule: cron表达式,用于指定任务的执行时间
*/1 * * * *
<分钟> <小时> <日> <月份> <星期>
分钟 值从 0 到 59.
小时 值从 0 到 23.
日 值从 1 到 31.
月 值从 1 到 12.
星期 值从 0 到 6, 0 代表星期日
多个时间可以用逗号隔开; 范围可以用连字符给出;*可以作为通配符; /表示每...
concurrencyPolicy:
Allow: 允许Jobs并发运行(默认)
Forbid: 禁止并发运行,如果上一次运行尚未完成,则跳过下一次运行
Replace: 替换,取消当前正在运行的作业并用新作业替换它
新建yaml文件:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: pc-cronjob
namespace: dev
labels:
controller: cronjob
spec:
schedule: "*/1 * * * *"
jobTemplate:
metadata:
spec:
template:
spec:
restartPolicy: Never
containers:
- name: hello
image: busybox:1.30
command:
- /bin/sh
- -c
- date; echo Hello from the Kubernetes cluster
每分钟开始输出时间信息和欢迎信息
#创建cronjob
[root@master ~]# kubectl create -f pc-cronjob.yaml
cronjob.batch/pc-cronjob created
#删除
[root@master ~]# kubectl delete -f pc-cronjob.yaml
cronjob.batch "pc-cronjob" deleted
#与此同时
[root@master ~]# kubectl get cronjob -n dev -w
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
pc-cronjob */1 * * * * False 0 <none> 0s
pc-cronjob */1 * * * * False 1 1s 23s
pc-cronjob */1 * * * * False 0 11s 33s
pc-cronjob */1 * * * * False 1 1s 83s
pc-cronjob */1 * * * * False 0 12s 94s
pc-cronjob */1 * * * * False 1 2s 2m24s
pc-cronjob */1 * * * * False 0 12s 2m34s
pc-cronjob */1 * * * * False 0 14s 2m36s
[root@master ~]# kubectl get pod -n dev -w
NAME READY STATUS RESTARTS AGE
pc-cronjob-1685543700-jv7n6 0/1 Pending 0 0s
pc-cronjob-1685543700-jv7n6 0/1 Pending 0 0s
pc-cronjob-1685543700-jv7n6 0/1 ContainerCreating 0 0s
pc-cronjob-1685543700-jv7n6 0/1 Completed 0 3s
pc-cronjob-1685543760-qdmwj 0/1 Pending 0 0s
pc-cronjob-1685543760-qdmwj 0/1 Pending 0 0s
pc-cronjob-1685543760-qdmwj 0/1 ContainerCreating 0 1s
pc-cronjob-1685543760-qdmwj 0/1 Completed 0 3s
pc-cronjob-1685543820-65xfc 0/1 Pending 0 0s
pc-cronjob-1685543820-65xfc 0/1 Pending 0 0s
pc-cronjob-1685543820-65xfc 0/1 ContainerCreating 0 0s
pc-cronjob-1685543820-65xfc 0/1 Completed 0 2s
pc-cronjob-1685543820-65xfc 0/1 Terminating 0 13s
pc-cronjob-1685543700-jv7n6 0/1 Terminating 0 2m14s
pc-cronjob-1685543760-qdmwj 0/1 Terminating 0 74s
pc-cronjob-1685543820-65xfc 0/1 Terminating 0 13s
pc-cronjob-1685543700-jv7n6 0/1 Terminating 0 2m14s
pc-cronjob-1685543760-qdmwj 0/1 Terminating 0 74s
[root@master ~]# kubectl get job -n dev -w
NAME COMPLETIONS DURATION AGE
pc-cronjob-1685543700 0/1 0s
pc-cronjob-1685543700 0/1 0s 0s
pc-cronjob-1685543700 1/1 3s 3s
pc-cronjob-1685543760 0/1 0s
pc-cronjob-1685543760 0/1 0s 0s
pc-cronjob-1685543760 1/1 3s 3s
pc-cronjob-1685543820 0/1 0s
pc-cronjob-1685543820 0/1 0s 0s
pc-cronjob-1685543820 1/1 2s 2s
pc-cronjob-1685543820 1/1 2s 13s
pc-cronjob-1685543760 1/1 3s 74s
pc-cronjob-1685543700 1/1 3s 2m14s
更多推荐
所有评论(0)