Kubernetes部署NFS PV数据持久化存储

准备所有k8s节点需要安装nfs服务一台nfs服务器提供存储根据需求创建数据共享目录image.png配置对所有k8s节点的授权检查是否已经授权showmount -e 192.168.100.204image.png之后不需要挂载，直接开始使用nfs的存储创建PV和PVC。PersistentVolume（PV）是指由集群管理员配置提供的某存储系统上的段存储空间，它是对底层共享存储的抽象，将共

m0_67505608

507人浏览 · 2022-03-16 16:08:41

m0_67505608 · 2022-03-16 16:08:41 发布

准备

所有k8s节点需要安装nfs服务
一台nfs服务器提供存储
根据需求创建数据共享目录

image.png

配置对所有k8s节点的授权
检查是否已经授权
showmount -e 192.168.100.204

image.png

之后不需要挂载，直接开始使用nfs的存储创建PV和PVC。

PersistentVolume（PV）是指由集群管理员配置提供的某存储系统上的段存储空间，它是对底层共享存储的抽象，将共享存储作为种可由用户申请使的资源，实现了“存储消费”机制。通过存储插件机制，PV支持使用多种网络存储系统或云端存储等多种后端存储系统，例如，NFS、RBD和Cinder等。PV是集群级别的资源，不属于任何名称空间，用户对PV资源的使需要通过PersistentVolumeClaim（PVC）提出的使申请（或称为声明）来完成绑定，是PV资源的消费者，它向PV申请特定大小的空间及访问模式（如rw或ro），从创建出PVC存储卷，后再由Pod资源通过PersistentVolumeClaim存储卷关联使，如下图：

 二 下载NFS插件
GitHub地址：https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client/deploy
1、下载所需文件
for file in class.yaml deployment.yaml rbac.yaml; do wget https://raw.githubusercontent.com/kubernetes-incubator/external-storage/master/nfs-client/deploy/$file; done

2、创建RBAC授权
# cat rbac.yaml
kind: ServiceAccount
apiVersion: v1
metadata:
  name: nfs-client-provisioner
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

3、创建Storageclass类
# cat class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

4、创建NFS的deployment，修改相应的NFS服务器IP及挂载路径
# cat deployment.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs-client-provisioner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-client-provisioner
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: quay.io/external_storage/nfs-client-provisioner:v2.0.0
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: fuseim.pri/ifs
            - name: NFS_SERVER
              value: 192.168.1.100
            - name: NFS_PATH
              value: /huoban/k8s
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.1.100
            path: /huoban/k8s

三、创建一个PV动态供给应用实例
例如：创建一个nginx动态获取PV

# cat nginx.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: nginx
  serviceName: "nginx"
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      imagePullSecrets:
      - name: nginx
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "managed-nfs-storage"
      resources:
        requests:
          storage: 1Gi

启动之后我们可以看到一下信息
# kubectl get pod,pv,pvc
pod/coredns-68c964545b-mszkt               0/1     Evicted   0          11d
pod/coredns-68c964545b-xzszx               1/1     Running   14         11d
pod/kube-apiserver-ha-proxy-k8s-master-1   1/1     Running   19         23d
pod/kube-apiserver-ha-proxy-k8s-node-1     1/1     Running   17         23d
pod/kube-apiserver-ha-proxy-k8s-node-2     1/1     Running   8          23d
pod/kube-flannel-ds-amd64-588x5            1/1     Running   1          3d19h
pod/kube-flannel-ds-amd64-jfv4h            1/1     Running   36         23d
pod/kube-flannel-ds-amd64-mkjqk            1/1     Running   5          10d
pod/metrics-server-8459f8db8c-h8bn7        2/2     Running   55         13d
pod/my-web-0                               1/1     Running   1          2d18h
pod/my-web-1                               1/1     Running   1          2d18h
pod/my-web-2                               1/1     Running   1          2d18h
pod/my-web-3                               1/1     Running   1          2d18h

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                             STORAGECLASS          REASON   AGE
persistentvolume/pvc-122aef78-c633-4eb0-8edf-4d9a5fa633ac   1Gi        RWO            Delete           Bound    kube-system/www-my-web-0          managed-nfs-storage            2d18h
persistentvolume/pvc-6d3698d5-5e4d-48f8-bb86-c0a300c25b2a   1Gi        RWO            Delete           Bound    kube-system/www-my-web-2          managed-nfs-storage            2d18h
persistentvolume/pvc-77c8a432-7ee5-4b0d-afae-94a88e0a6113   5Gi        RWX            Delete           Bound    kube-system/managed-nfs-storage   managed-nfs-storage            2d18h
persistentvolume/pvc-77e21151-70cc-4185-ad4b-12cf39c4128c   1Gi        RWO            Delete           Bound    kube-system/www-my-web-1          managed-nfs-storage            2d18h
persistentvolume/pvc-cf0c4a85-4d09-4918-b63c-a40c11c1b658   1Gi        RWO            Delete           Bound    kube-system/www-my-web-3          managed-nfs-storage            2d18h

NAME                                        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
persistentvolumeclaim/managed-nfs-storage   Bound    pvc-77c8a432-7ee5-4b0d-afae-94a88e0a6113   5Gi        RWX            managed-nfs-storage   2d18h
persistentvolumeclaim/www-my-web-0          Bound    pvc-122aef78-c633-4eb0-8edf-4d9a5fa633ac   1Gi        RWO            managed-nfs-storage   2d18h
persistentvolumeclaim/www-my-web-1          Bound    pvc-77e21151-70cc-4185-ad4b-12cf39c4128c   1Gi        RWO            managed-nfs-storage   2d18h
persistentvolumeclaim/www-my-web-2          Bound    pvc-6d3698d5-5e4d-48f8-bb86-c0a300c25b2a   1Gi        RWO            managed-nfs-storage   2d18h
persistentvolumeclaim/www-my-web-3          Bound    pvc-cf0c4a85-4d09-4918-b63c-a40c11c1b658   1Gi        RWO            managed-nfs-storage   2d18h
现在，我们在NFS服务器上也可以看到自动生成了3个挂载目录，单pod删除之后数据还会存在
[10:41:55root@k8s-master-1 /nfs/data]#ll
total 0
drwxrwxrwx 2 root root  6 Mar 26 16:30 kube-system-managed-nfs-storage-pvc-77c8a432-7ee5-4b0d-afae-94a88e0a6113
drwxrwxrwx 2 root root 24 Mar 26 16:33 kube-system-www-my-web-0-pvc-122aef78-c633-4eb0-8edf-4d9a5fa633ac
drwxrwxrwx 2 root root 24 Mar 26 16:33 kube-system-www-my-web-1-pvc-77e21151-70cc-4185-ad4b-12cf39c4128c
drwxrwxrwx 2 root root 24 Mar 26 16:33 kube-system-www-my-web-2-pvc-6d3698d5-5e4d-48f8-bb86-c0a300c25b2a
drwxrwxrwx 2 root root 24 Mar 26 16:33 kube-system-www-my-web-3-pvc-cf0c4a85-4d09-4918-b63c-a40c11c1b658

StatefulSet应用有以下特点：
1.唯一的网络标识
2.域名访问(<statefulsetName-index>.<service-name>.svc.cluster.local) 如：web-0.nginx.default.svc.cluster.local
3.独立的持久存储
4.有序的部署和删除

[13:45:46root@k8s-master-1 ~/nfs/lk]#cat nfs-pvc.yaml 
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: managed-nfs-storage
spec:
  storageClassName: "managed-nfs-storage"
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi
[13:46:18root@k8s-master-1 ~/nfs/lk]#cat nginx_deployment.yaml 
apiVersion: v1
kind: Service
metadata:
  name: my-nginx-service #服务名
  labels:
    app: my-nginx  #标签名
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None    #Headless 无头服务配置
  selector:
    app: my-nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: my-web
spec:
  selector:
    matchLabels:
      app: my-nginx
  serviceName: "my-nginx-service"
  replicas: 5    #部署4个副本
  template:
    metadata:
      labels:
        app: my-nginx
    spec:
      containers:
      - name: nginx
        image: registry.aliyuncs.com/google_containers/nginx-slim:0.8
        ports:
        - containerPort: 80
          name: my-web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "managed-nfs-storage"
      resources:
        requests:
          storage: 1Gi

大部分内容来自：Kubernetes v1.18.6部署NFS PV数据持久化存储 - 简书

作者：前浪浪奔浪流
链接：https://www.jianshu.com/p/db3aa6544f50