系列文章目录

第一章：✨ k8s入门：裸机部署 k8s 集群
第二章：✨ k8s入门：部署应用到 k8s 集群
第三章：✨ k8s入门：service 简单使用
第四章：✨ k8s入门：StatefulSet 简单使用
第五章：✨ k8s入门：存储（storage）
第六章：✨ K8S 配置 storageclass 使用 nfs 动态申领本地磁盘空间
第七章：✨ k8s入门：配置 ConfigMap & Secret
第八章：✨ k8s入门：k8s入门：Helm 构建 MySQL
第九章：✨ k8s入门：kubernetes-dashboard 安装
第十章：✨ k8s入门：kube-prometheus-stack 全家桶搭建（Grafana + Prometheus）

文章目录

一、简介

本篇文章使用 PersistentVolumeClaim （pvc） 结合资源对象 StorageClass （SC） 去动态申领 PersistentVolume （PV）本地 nfs 共享文件系统存储空间，SC 的根本作用就是根据 pvc 的定义来动态创建pv，不仅节省了我们管理员的时间，还可以封装不同类型的存储供 pvc 选用。

每个 StorageClass 都包含以下三个重要的字段，这些字段会在sc需要动态分配pv时会使用到：

  • 每个 StorageClass 都有一个制备器（Provisioner），用来决定使用哪个卷插件制备 PV。 该字段必须指定。

  • 回收策略（reclaimPolicy）可以是 Delete 或者 Retain，如果 StorageClass 对象创建时没有指定 reclaimPolicy，它将默认为 Delete。

  • PersistentVolume 可以配置为可扩展。将此功能设置为 true 时，允许用户通过编辑相应的 PVC 对象来调整卷大小。

注意： 首先保证你的 nfs 共享文件系统可用，没有安装的可参考 https://blog.csdn.net/qq_41538097/article/details/125215989 步骤七

配置文件可参考：https://github.com/kubernetes-retired/external-storage/tree/master/nfs-client/deploy
StorageClass 详细使用可参考： https://kubernetes.io/zh-cn/docs/concepts/storage/storage-classes

二、配置文件

配置文件可以从该链接获取 https://kubernetes.io/zh-cn/docs/concepts/storage/storage-classes

也可以命令将整个项目拉取下来git clone https://github.com/kubernetes-incubator/external-storage.git

我的配置文件如下

[root@master nfs]# pwd
/root/StorageClass/nfs
[root@master nfs]# ls
class.yaml  deployment.yaml  rbac.yaml  test-claim.yaml  test-pod.yaml
三、rbac.yaml

创建 rbac（基于角色的访问控制）配置文件 rbac.yaml，就是用户通过角色与权限进行关联。

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

部署 rbac 配置文件

[root@master nfs]# kubectl apply -f rbac.yaml 
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
四、deployment.yaml

新增 nfs 的 Deployment 配置 deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: quay.io/external_storage/nfs-client-provisioner:latest
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: nfs-diy # StorageClass 三个重要字段之一 Provisioner，名字自己指定
            - name: NFS_SERVER
              value: 192.168.25.100 # nfs 服务器地址
            - name: NFS_PATH
              value: /home/data # nfs 共享文件夹
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.25.100 # nfs 服务器地址
            path: /home/data # nfs 共享文件夹

部署 nfs 的 Deployment

[root@master nfs]# kubectl apply -f deployment.yaml 
deployment.apps/nfs-client-provisioner created
[root@master nfs]# kubectl get deployment
NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
nfs-client-provisioner   1/1     1            1           10s
[root@master nfs]# kubectl get pod -o wide
NAME                                      READY   STATUS    RESTARTS        AGE    IP            NODE    NOMINATED NODE   READINESS GATES
nfs-client-provisioner-6b4c9588c6-9hs2q   1/1     Running   0               23s    10.244.2.46   node2   <none>           <none>
五、class.yaml

定义了 存储类 class.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storage
provisioner: nfs-diy # 和 deployment.yaml 中 env.PROVISIONER_NAME 保持一致
reclaimPolicy: Retain
parameters:
  archiveOnDelete: "false"

部署 SC

[root@master nfs]# kubectl apply -f class.yaml 
storageclass.storage.k8s.io/nfs-storage created
[root@master nfs]# kubectl get sc
NAME          PROVISIONER   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-storage   nfs-diy       Retain          Immediate           false                  15s
六、test-claim.yaml

使用 PVC 申领本地磁盘

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
  annotations:
    volume.beta.kubernetes.io/storage-class: "nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi

部署 PVC

[root@master nfs]# kubectl apply -f test-claim.yaml 
persistentvolumeclaim/test-claim created
[root@master nfs]# kubectl get pvc
NAME                 STATUS    VOLUME                CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-claim           Pending                                                   nfs-storage    9s
[root@master nfs]# kubectl describe pvc/test-claim
Name:          test-claim
Namespace:     default
StorageClass:  nfs-storage
Status:        Pending
Volume:        
Labels:        <none>
Annotations:   volume.beta.kubernetes.io/storage-class: nfs-storage
               volume.beta.kubernetes.io/storage-provisioner: nfs-diy
               volume.kubernetes.io/storage-provisioner: nfs-diy
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Used By:       <none>
Events:
  Type    Reason                Age               From                         Message
  ----    ------                ----              ----                         -------
  Normal  ExternalProvisioning  6s (x3 over 15s)  persistentvolume-controller  waiting for a volume to be created, either by external provisioner "nfs-diy" or manually created by system administrator

如上可发现创建失败，报错 waiting for a volume to be created, either by external provisioner "nfs-diy" or manually created by system administrator

注意：如果是 v1.20 版本以上 apiserver 默认禁止使用 selfLink，需要手动配置- --feature-gates=RemoveSelfLink=false 开启。

修改kube-apiserver.yaml 文件：

[root@master nfs]# cat /etc/kubernetes/manifests/kube-apiserver.yaml
apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 192.168.25.100:6443
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=192.168.25.100
    - --feature-gates=RemoveSelfLink=false   # 添加这条信息
    - --allow-privileged=true

systemctl restart kubelet重启 kubelet 服务，然后查看 PVC，PV 信息，可以看到 PVC 申领 PV 卷成功。
注意： 如果不成功，仍然报这个错

  • 可以使用 kubectl get pod -n kube-system 查看 kube-apiserver 是否启动成功
  • 查看上面 deployment 创建的 pod 是否启动失败
[root@master nfs]# systemctl restart kubelet
[root@master nfs]# kubectl describe pvc/test-claim 
Name:          test-claim
Namespace:     default
StorageClass:  nfs-storage
Status:        Bound
Volume:        pvc-c2ae61f8-7c40-4393-9110-114a0455afa1
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-class: nfs-storage
               volume.beta.kubernetes.io/storage-provisioner: nfs-diy
               volume.kubernetes.io/storage-provisioner: nfs-diy
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      1Mi
Access Modes:  RWX
VolumeMode:    Filesystem
Used By:       <none>
Events:
  Type    Reason                 Age                From                                                                                  Message
  ----    ------                 ----               ----                                                                                  -------
  Normal  ExternalProvisioning   12s (x5 over 72s)  persistentvolume-controller                                                           waiting for a volume to be created, either by external provisioner "nfs-diy" or manually created by system administrator
  Normal  Provisioning           5s                 nfs-diy_nfs-client-provisioner-5c58f4c86f-ndffx_15778b3c-f0a6-11ec-894c-ceb09bae6415  External provisioner is provisioning volume for claim "default/test-claim"
  Normal  ProvisioningSucceeded  5s                 nfs-diy_nfs-client-provisioner-5c58f4c86f-ndffx_15778b3c-f0a6-11ec-894c-ceb09bae6415  Successfully provisioned volume pvc-c2ae61f8-7c40-4393-9110-114a0455afa1
[root@master nfs]# kubectl get pvc
NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-claim           Bound    pvc-c2ae61f8-7c40-4393-9110-114a0455afa1   1Mi        RWX            nfs-storage    107s
[root@master nfs]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                        STORAGECLASS   REASON   AGE
pvc-c2ae61f8-7c40-4393-9110-114a0455afa1   1Mi        RWX            Retain           Bound    default/test-claim           nfs-storage             109s
[root@master nfs]# ls /home/data
default-test-claim-pvc-c2ae61f8-7c40-4393-9110-114a0455afa1

PVC 动态持久卷在 nfs 服务器持久卷路径被配置为 ​​共享文件夹目录/${namespace}-${pvcName}-${pvName}​​，比如：/home/data/default-test-claim-pvc-c2ae61f8-7c40-4393-9110-114a0455afa1

七、test-pod.yaml

创建 pod 测试

kind: Pod
apiVersion: v1
metadata:
  name: test-pod-nginx
spec:
  containers:
    - name: test-pod-nginx-containers
      image: nginx:latest
      ports:
        - containerPort: 80
          name: "http-server"
          hostPort: 80
      volumeMounts:
        - name: test-pod-nginx-storage
          mountPath: "/usr/share/nginx/html"
  restartPolicy: "Never"
  volumes:
    - name: test-pod-nginx-storage
      persistentVolumeClaim:
        claimName: test-claim

部署 pod

[root@master nfs]# kubectl apply -f test-pod.yaml 
pod/test-pod-nginx created
[root@master nfs]# kubectl get pod -o wide
NAME                                      READY   STATUS    RESTARTS        AGE     IP            NODE    NOMINATED NODE   READINESS GATES
test-pod-nginx                            1/1     Running   0               33s     10.244.1.46   node1   <none>           <none>
[root@master data]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                        STORAGECLASS   REASON   AGE
pvc-c2ae61f8-7c40-4393-9110-114a0455afa1   1Mi        RWX            Retain           Bound    default/test-claim           nfs-storage             13m
[root@master nfs]# echo "K8S configures storageclass to dynamically claim local disk space using nfs" > /home/data/default-test-claim-pvc-c2ae61f8-7c40-4393-9110-114a0455afa1/index.html

因为上述 pod 部署到了 node1 节点（192.168.1.101），访问 http://192.168.25.101/ 测试成功

在这里插入图片描述

八、动态新增 PV

test-claim2.yaml，通过 nfs-storage 动态申领 100M PV 本地存储

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim2
  annotations:
    volume.beta.kubernetes.io/storage-class: "nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Mi

部署 PVC，可以看到申请的 PV 卷名字 pvc-9385f11d-2587-469b-bd13-3a756fe61acb，大小为 100 M，挂载目录：/home/data/default-test-claim2-pvc-9385f11d-2587-469b-bd13-3a756fe61acb

[root@master nfs]# kubectl apply -f test-claim2.yaml 
persistentvolumeclaim/test-claim2 created
[root@master nfs]# kubectl get pvc
NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-claim           Bound    pvc-c2ae61f8-7c40-4393-9110-114a0455afa1   1Mi        RWX            nfs-storage    10h
test-claim2          Bound    pvc-9385f11d-2587-469b-bd13-3a756fe61acb   100Mi      RWX            nfs-storage    5s
[root@master nfs]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                        STORAGECLASS   REASON   AGE
pvc-9385f11d-2587-469b-bd13-3a756fe61acb   100Mi      RWX            Retain           Bound    default/test-claim2          nfs-storage             17s
pvc-c2ae61f8-7c40-4393-9110-114a0455afa1   1Mi        RWX            Retain           Bound    default/test-claim           nfs-storage             10h
# java # kubernetes # 开发语言
Logo
K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 |  韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号​前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。​长夜有穷，真水无香。领略过了 K8s“身在江

avatar K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub

avatar K8S/Kubernetes

k8s 火了！

2020，上云之年，产品云端化成为一种趋势。在一线城市，很多公司都已经构建了自己的私有云环境，比如阿里云、网易云、华为云等。而Kubernetes 作为基于容器编排领域的王者，具备扩展...

avatar K8S/Kubernetes