有状态、无状态服务的概念参见：kubernetes原理及核心组件里的StatefullSet介绍。

1 存储

• PV：PersistentVolume（持久卷，简称PV），是对存储系统的抽象，将存储系统抽象为k8s的一个资源，便于访问。
• PVC： PersistentVolumeClaim（持久卷声明，简称PVC），用户（一般是开发人员，开发人员在发布服务时要指定pod挂载Volume的空间大小、存储类型等）的一种存储请求，用户不需要知道存储的实现细节，只需要通过PVC告知存储的需求（比如：多大的空间、什么样的存储系统等），PVC会自动匹配PV（PV一般是又运维人员创建的）。一个PVC对应一个PV。
• StorageClass：在集群环境中，PVC的数量可能会非常多，导致运维人员维护PV的工作异常繁重，通过StorageClass可以实现对PV的自动创建和维护。

2 Headless Services

Headless Services是用来对部署有状态服务提供的一种Service，与StatefullSet一起使用。Deployment也有一个对应的Service，与Service不同的是，Headless没有VIP（虚拟IP），因为Headless不需要要与外界通信。有状态服务的负载均衡一般是由客户端来完成的，比如redis，会在配置文件中配置redis的一组IP地址（如果是与Spring集成的，属性名可能是：spring.redis.host），然后由redis的client实现负载均衡，所以Headless不用对外提供服务，但是要保证客户端能够通过IP直接访问到Pod。

3 实战

以NFS为例来进行演示。需要搭建一台nfs服务器
安装nfs命令

yum install nfs-utils -y

创建目录，并将目录暴露给其他服务器

mkdir /opt/k8s
vim /etc/exports
/opt/k8s 192.168.75.0/24(rw,no_root_squash)

systemctl start nfs

需要共享上面目录的节点也需要安装nfs，否则在创建NFS provisioner时会报以下错误

下载nfs-client-provisioner镜像，然后push到harbor仓库

创建权限

#创建ServiceAccount，用来管控NFS provisioner在k8s集群中运行的权限
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  namespace: default
---
#以下apiVersion不能更改
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
#角色匹配操作权限
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

创建StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: jmgao1983-nfs-storage
parameters:
   archiveOnDelete: "false"

创建NFS provisioner

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-client-provisioner
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: www.harbor.com/harbor/nfs-client-provisioner:v1
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: jmgao1983-nfs-storage
            - name: NFS_SERVER
              value: 192.168.75.13
            - name: NFS_PATH
              value: /opt/k8s
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.75.13
            path: /opt/k8s

方式一：手动创建PV

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: manual-claim
  #通过注解的方式绑定storage class，由storage class来创建PV
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  #访问模式
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      #空间大小
      storage: 1Mi
---
kind: Pod
apiVersion: v1
metadata:
  name: nginx
spec:
  containers:
    - name: nginx
      image: www.harbor.com/harbor/nginx:v1
      command:
        - "/bin/sh"
      args:
        - "-c"
        - "touch /mnt/demo && exit 0 || exit 1"
      volumeMounts:
        - name: nfs-pvc
          mountPath: "/mnt"
  restartPolicy: "Never"
  #挂载PVC
  volumes:
    - name: nfs-pvc
      persistentVolumeClaim:
        claimName: manual-claim

方式二：StatefulSet+volumeClaimTemplates自动创建PV

apiVersion: v1
kind: Service
metadata:
  name: nginx-headless
  labels:
    app: nginx
spec:
  ports:
    - port: 80
      name: web
  #StatefullSet对应的Service是Headless Service，所以clusterIP为None
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: web
spec:
  serviceName: "nginx"
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: www.harbor.com/harbor/nginx:v1
          ports:
            - containerPort: 80
              name: web
          volumeMounts:
            - name: auto-pv
              mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
    - metadata:
        name: auto-pv
        annotations:
          volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
      spec:
        accessModes: [ "ReadWriteOnce" ]
        resources:
          requests:
            storage: 1Gi