RabbitMQ 集群部署：基于 K8s StatefulSet 实现高可用与持久化

一、核心目标

1. 高可用：通过集群消除单点故障
2. 持久化：消息和队列状态持久存储
3. 自动恢复：节点故障后自动重建
4. 水平扩展：按需增减节点

二、架构设计要点

graph LR
A[Headless Service] --> B[StatefulSet Pod-0]
A --> C[StatefulSet Pod-1]
A --> D[StatefulSet Pod-2]
B --> E[PV-0]
C --> F[PV-1]
D --> G[PV-2]

三、部署步骤

1. 准备持久化存储（StorageClass）

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: rabbitmq-storage
provisioner: kubernetes.io/aws-ebs # 根据云平台调整
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer

2. 创建 Headless Service

apiVersion: v1
kind: Service
metadata:
  name: rabbitmq
spec:
  clusterIP: None
  ports:
  - name: epmd
    port: 4369
  - name: amqp
    port: 5672
  - name: management
    port: 15672
  selector:
    app: rabbitmq

3. 配置 StatefulSet

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: rabbitmq
spec:
  serviceName: "rabbitmq"
  replicas: 3
  selector:
    matchLabels:
      app: rabbitmq
  template:
    metadata:
      labels:
        app: rabbitmq
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: rabbitmq
        image: rabbitmq:3.12-management
        env:
        - name: RABBITMQ_ERLANG_COOKIE
          value: "SECRET_COOKIE" # 集群通信密钥
        - name: RABBITMQ_NODENAME
          value: "rabbit@$(HOSTNAME).rabbitmq"
        ports:
        - containerPort: 4369
        - containerPort: 5672
        - containerPort: 15672
        volumeMounts:
        - name: data
          mountPath: /var/lib/rabbitmq
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "rabbitmq-storage"
      resources:
        requests:
          storage: 10Gi

四、集群初始化脚本

#!/bin/sh
# 自动加入集群
if [ "$HOSTNAME" != "rabbitmq-0" ]; then
  rabbitmqctl stop_app
  rabbitmqctl join_cluster rabbit@rabbitmq-0.rabbitmq
  rabbitmqctl start_app
fi

# 设置镜像策略（所有队列镜像到所有节点）
rabbitmqctl set_policy ha-all ".*" '{"ha-mode":"all"}'

五、高可用验证

1. 节点状态检查：

   kubectl exec rabbitmq-0 -- rabbitmqctl cluster_status
   

   预期输出包含所有节点：

   Running Nodes: [rabbit@rabbitmq-0, rabbit@rabbitmq-1, rabbit@rabbitmq-2]
   

2. 故障测试：

   • 删除节点：kubectl delete pod rabbitmq-1
   • 观察：StatefulSet 自动重建 Pod，队列服务不中断

六、关键优化项

1. 网络策略：

   # 限制只允许特定命名空间访问
   ingress:
   - from:
     - namespaceSelector:
         matchLabels:
           project: microservices
   

2. 资源限制：

   resources:
     limits:
       memory: 2Gi
       cpu: "1"
     requests:
       memory: 1Gi
       cpu: "0.5"
   

3. 监控集成：

   • Prometheus 指标端点：rabbitmq_prometheus:15692
   • Grafana 仪表板 ID：10991

七、灾难恢复方案

1. 定期快照：

   # 备份元数据
   kubectl exec rabbitmq-0 -- rabbitmqctl export_definitions > backup.json
   

2. 存储卷恢复：
   • 通过 PVC 快照恢复数据卷
   • 重新挂载到新 Pod

最佳实践：生产环境建议部署 3/5/7 奇数节点集群，结合 x-ha-policy 参数控制队列镜像策略，平衡性能与可靠性。