需求

  1. k8s(Kubernetes)中安装Redis集群
  2. 使用nfs存储
  3. 部署Redis集群 3主3从
  4. 服务器内部ip：192.168.0.254

必备环境安装

ubuntu20.x 服务器 nfs服务安装

https://foxwho.blog.csdn.net/article/details/116994098

#创建nfs redis 存储目录 共 6个
mkdir -p /nfs/redis/pv{1..6}

#编辑 nfs 存储配置文件
vim /etc/exports

增加或修改如下

/nfs/redis/pv1 *(ro,sync,no_root_squash,no_subtree_check)
/nfs/redis/pv2 *(ro,sync,no_root_squash,no_subtree_check)
/nfs/redis/pv3 *(ro,sync,no_root_squash,no_subtree_check)
/nfs/redis/pv4 *(ro,sync,no_root_squash,no_subtree_check)
/nfs/redis/pv5 *(ro,sync,no_root_squash,no_subtree_check)
/nfs/redis/pv6 *(ro,sync,no_root_squash,no_subtree_check)

重新 刷新配置共享配置文件中的共享目录

exportfs -rv

查看 nfs共享目录

showmount -e 192.168.0.254

输出

Export list for 192.168.0.254:
/nfs/redis/pv6 *
/nfs/redis/pv5 *
/nfs/redis/pv4 *
/nfs/redis/pv3 *
/nfs/redis/pv2 *
/nfs/redis/pv1 *

k8s 安装略

请自行搜索 k8s 安装

Kubernetes部署redis集群

创建ConfigMap

redis-cm.yml

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-cluster
data:
  update-node.sh: |
    #!/bin/sh
    REDIS_NODES="/data/nodes.conf"
    sed -i -e "/myself/ s/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/${POD_IP}/" ${REDIS_NODES}
    exec "$@"
  redis.conf: |+
    cluster-enabled yes
    cluster-require-full-coverage no
    cluster-node-timeout 15000
    cluster-config-file /data/nodes.conf
    cluster-migration-barrier 1
    dir /data
    appendonly yes
    protected-mode no

执行创建命令

kubectl apply -f redis-cm.yml

查看ConfigMap

kubectl get cm

输出

NAME               DATA   AGE
kube-root-ca.crt   1      42h
redis-cluster      2      16h

创建pv

redis-pv.yml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv1
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /nfs/redis/pv1
    server: 192.168.0.254
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv2
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /nfs/redis/pv2
    server: 192.168.0.254
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv3
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /nfs/redis/pv3
    server: 192.168.0.254
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv4
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /nfs/redis/pv4
    server: 192.168.0.254
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv5
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /nfs/redis/pv5
    server: 192.168.0.254
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv6
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /nfs/redis/pv6
    server: 192.168.0.254

执行创建命令

kubectl apply -f redis-pv.yml

输出

persistentvolume/redis-pv1 created
persistentvolume/redis-pv2 created
persistentvolume/redis-pv3 created
persistentvolume/redis-pv4 created
persistentvolume/redis-pv5 created
persistentvolume/redis-pv6 created

查看pv

kubectl get pv

输出

NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                          STORAGECLASS    REASON   AGE
redis-pv1   5Gi        RWO            Recycle          Bound    default/data-redis-cluster-0   redis-cluster            16h
redis-pv2   5Gi        RWO            Recycle          Bound    default/data-redis-cluster-1   redis-cluster            16h
redis-pv3   5Gi        RWO            Recycle          Bound    default/data-redis-cluster-2   redis-cluster            16h
redis-pv6   5Gi        RWO            Recycle          Bound    default/data-redis-cluster-3   redis-cluster            16h
redis-pv4   5Gi        RWO            Recycle          Bound    default/data-redis-cluster-4   redis-cluster            16h
redis-pv5   5Gi        RWO            Recycle          Bound    default/data-redis-cluster-5   redis-cluster            16h

创建 StatefulSet

redis-sts.yml

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-cluster
spec:
  serviceName: redis-cluster
  replicas: 6
  selector:
    matchLabels:
      app: redis-cluster
  template:
    metadata:
      labels:
        app: redis-cluster
    spec:
      containers:
      - name: redis
        image: redis:6.2-alpine
        ports:
        - containerPort: 6379
          name: client
        - containerPort: 16379
          name: gossip
        command: ["/conf/update-node.sh", "redis-server", "/conf/redis.conf"]
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        volumeMounts:
        - name: conf
          mountPath: /conf
          readOnly: false
        - name: data
          mountPath: /data
          readOnly: false
      volumes:
      - name: conf
        configMap:
          name: redis-cluster
          defaultMode: 0755
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 5Gi
      storageClassName: redis-cluster

执行创建命令

kubectl apply -f redis-sts.yml

输出

statefulset.apps/redis-cluster created

查看

kubectl get pods -l app=redis-cluster

输出

NAME              READY   STATUS    RESTARTS   AGE
redis-cluster-0   1/1     Running   0          16h
redis-cluster-1   1/1     Running   0          16h
redis-cluster-2   1/1     Running   0          16h
redis-cluster-3   1/1     Running   0          16h
redis-cluster-4   1/1     Running   0          16h
redis-cluster-5   1/1     Running   0          16h

创建 svc

redis-svc.yml

---
apiVersion: v1
kind: Service
metadata:
  name: redis-cluster
spec:
  type: ClusterIP
  ports:
  - port: 6379
    targetPort: 6379
    name: client
  - port: 16379
    targetPort: 16379
    name: gossip
  selector:
    app: redis-cluster

执行创建命令

kubectl apply -f redis-svc.yml

输出

service/redis-cluster created

查看 svc redis-cluster

kubectl get svc redis-cluster

输出

NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)              AGE
redis-cluster   ClusterIP   10.43.216.117   <none>        6379/TCP,16379/TCP   17h

初始化 Redis Cluster

在命令执行中有Can I set the above configuration? (type 'yes' to accept):时，要输入yes创建集群

kubectl exec -it redis-cluster-0 -- redis-cli --cluster create --cluster-replicas 1 $(kubectl get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 {end}')

输出

····
Can I set the above configuration? (type 'yes' to accept): yes
····

验证集群部署

kubectl exec -it redis-cluster-0 -- redis-cli cluster info

输出

····
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:57351
cluster_stats_messages_pong_sent:55991
cluster_stats_messages_sent:113342
cluster_stats_messages_ping_received:55986
cluster_stats_messages_pong_received:57351
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:113342
····

查看各个redis的主从状态

for x in $(seq 0 5); do echo "redis-cluster-$x"; kubectl exec redis-cluster-$x -- redis-cli role; echo; done

输出

redis-cluster-0
master
80332
10.42.0.19
6379
80332

redis-cluster-1
master
80332
10.42.0.20
6379
80332

redis-cluster-2
master
80332
10.42.0.18
6379
80332

redis-cluster-3
slave
10.42.0.17
6379
connected
80332

redis-cluster-4
slave
10.42.0.15
6379
connected
80332

redis-cluster-5
slave
10.42.0.16
6379
connected
80346

参考

https://github.com/zuxqoj/kubernetes-redis-cluster

https://github.com/llmgo/redis-sts.git

https://rancher.com/blog/2019/deploying-redis-cluster/

https://www.cnblogs.com/winstom/p/11881882.html

https://blog.csdn.net/zhutongcloud/article/details/90768390

# kubernetes # redis
Logo
K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 |  韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号​前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。​长夜有穷，真水无香。领略过了 K8s“身在江

avatar K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub

avatar K8S/Kubernetes

k8s 火了！

2020，上云之年，产品云端化成为一种趋势。在一线城市，很多公司都已经构建了自己的私有云环境，比如阿里云、网易云、华为云等。而Kubernetes 作为基于容器编排领域的王者，具备扩展...

avatar K8S/Kubernetes