k8s学习(二十九) RocketMq集群(双主双从-同步)
确认安装前已经安装好StorageClass,可参考:https://blog.csdn.net/u011943534/article/details/1008875301、准备jdk镜像,并拷贝到服务器docker pull java:8-alpine2、制作rocketmq镜像下载地址:https://mirrors.tuna.tsinghua.edu.cn/apache/rocketmq/将
确认安装前已经安装好StorageClass,可参考:https://blog.csdn.net/u011943534/article/details/100887530
1、准备jdk镜像,并拷贝到服务器
docker pull java:8-alpine
2、制作rocketmq镜像
下载地址:https://mirrors.tuna.tsinghua.edu.cn/apache/rocketmq/
将”rocketmq-all-4.8.0-bin-release.zip”
拷贝至服务器,解压
修改bin/runbroker.sh
,将内存调整为适当(比如调小)
JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn512m"
编写Dockerfile-rocketmq:
FROM java:8-alpine
RUN rm -f /etc/localtime \
&& ln -sv /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezone
ENV LANG en_US.UTF-8
ADD rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-all-4.8.0-bin-release
RUN mv /usr/local/rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-4.8.0 \
&& mkdir -p /data/rocketmq/store
CMD ["/bin/bash"]
docker build -f Dockerfile-rocketmq -t 172.16.10.190:80/library/rocketmq:4.8.0 .
docker push 172.16.10.190:80/library/rocketmq:4.8.0
3、制作nameserver镜像
使用第三步的rocketmq-all-4.8.0-bin-release
在conf/下添加namesrv.properties
内容为:
listenPort=20901
编写Dokerfile-namesrv:
FROM java:8-alpine
RUN rm -f /etc/localtime \
&& ln -sv /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezone
ENV LANG en_US.UTF-8
ADD rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-all-4.8.0-bin-release
RUN mv /usr/local/rocketmq-all-4.8.0-bin-release /usr/local/rocketmq-4.8.0
WORKDIR /usr/local/rocketmq-4.8.0
CMD ["/usr/local/rocketmq-4.8.0/bin/mqnamesrv","-c","/usr/local/rocketmq-4.8.0/conf/namesrv.properties"]
docker build -f Dockerfile-nameserver -t 172.16.10.190:80/library/rocketmq-namesrv:4.8.0 .
docker push 172.16.10.190:80/library/rocketmq-namesrv:4.8.0
4、制作rocketmq-ui镜像
下载源码:
git clone https://github.com/apache/rocketmq-externals.git
下载maven:
wget http://mirrors.hust.edu.cn/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
tar -zvxf rocketmq-externals.tar.gz
将本目录下的”application.properties“中内容替换到rocketmq-externals/rocketmq-console/src/main/resources/application.properties中
编写Dockerfile-ui
FROM java:8-alpine
RUN rm -f /etc/localtime \
&& ln -sv /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
&& echo "Asia/Shanghai" > /etc/timezone
ENV LANG en_US.UTF-8
ENV MAVEN_HOME /usr/local/maven
ENV PATH $PATH:$MAVEN_HOME/bin
ADD rocketmq-externals /usr/local/rocketmq-externals
ADD apache-maven-3.3.9 /usr/local/apache-maven-3.3.9
RUN mv /usr/local/apache-maven-3.3.9 /usr/local/maven
WORKDIR /usr/local/rocketmq-externals/rocketmq-console/
RUN mvn clean package -Dmaven.test.skip=true
WORKDIR /usr/local/rocketmq-externals/rocketmq-console/target
EXPOSE 8080
CMD ["nohup","java","-jar","rocketmq-console-ng-1.0.0.jar","&"]
5、准备configmap
broker-a.properties:
brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=0
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=MASTER
broker-a-s.properties:
brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=1
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=SLAVE
flushDiskType=SYNC_FLUSH
broker-b.properties:
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=0
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=MASTER
flushDiskType=SYNC_FLUSH
broker-b-s.properties:
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=1
namesrvAddr=mq-namesrv-0.mq-namesrv.default.svc.cluster.local:20901
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=20911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88
storePathRootDir=/data/rocketmq/store
maxMessageSize=65536
brokerRole=SLAVE
flushDiskType=SYNC_FLUSH
根据以上文件生成configmap:
kubectl create configmap rocketmq-config --from-file=broker-a.properties --from-file=broker-b.properties --from-file=broker-a-s.properties --from-file=broker-b-s.properties -n default
6、准备服务文件
broker-a-statefulset.yaml:
apiVersion: v1
kind: Service
metadata:
labels:
app: broker-a
name: broker-a
namespace: default
spec:
type: NodePort
ports:
- port: 20911
targetPort: 20911
name: broker-port
nodePort: 30911
selector:
app: broker-a
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: broker-a
namespace: default
spec:
serviceName: broker-a
replicas: 1
selector:
matchLabels:
app: broker-a
template:
metadata:
labels:
app: broker-a
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app"
operator: In
values:
- broker-a
topologyKey: "kubernetes.io/hostname"
imagePullSecrets:
- name: harborsecret
containers:
- name: broker-a
image: 172.16.10.190:80/library/rocketmq:4.8.0
imagePullPolicy: Always
command: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker -c /usr/local/rocketmq-4.8.0/conf/broker-a.properties"]
volumeMounts:
- mountPath: /root/logs
name: rocketmq-data
subPath: mq-brokeroptlogs
- mountPath: /data/rocketmq
name: rocketmq-data
subPath: mq-brokeroptstore
- name: broker-config
mountPath: /usr/local/rocketmq-4.8.0/conf/broker-a.properties
subPath: broker-a.properties
- name: time
mountPath: /etc/localtime
lifecycle:
postStart:
exec:
command: ["/bin/sh","-c","touch /tmp/health"]
livenessProbe:
exec:
command: ["test","-e","/tmp/health"]
initialDelaySeconds: 5
timeoutSeconds: 5
periodSeconds: 10
readinessProbe:
tcpSocket:
port: 20911
initialDelaySeconds: 15
timeoutSeconds: 5
periodSeconds: 20
volumes:
- name: broker-config
configMap:
name: rocketmq-config
- name: time
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
volumeClaimTemplates:
- metadata:
name: rocketmq-data
annotations:
volume.beta.kubernetes.io/storage-class: "course-nfs-storage"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 4Gi
broker-a-s-statefulset.yaml:
apiVersion: v1
kind: Service
metadata:
labels:
app: broker-a-s
name: broker-a-s
namespace: default
spec:
type: NodePort
ports:
- port: 20911
targetPort: 20911
name: broker-port
nodePort: 30912
selector:
app: broker-a-s
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: broker-a-s
namespace: default
spec:
serviceName: broker-a-s
replicas: 1
selector:
matchLabels:
app: broker-a-s
template:
metadata:
labels:
app: broker-a-s
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app"
operator: In
values:
- broker-a-s
topologyKey: "kubernetes.io/hostname"
imagePullSecrets:
- name: harborsecret
containers:
- name: broker-a-s
image: 172.16.10.190:80/library/rocketmq:4.8.0
imagePullPolicy: Always
command: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker -c /usr/local/rocketmq-4.8.0/conf/broker-a-s.properties"]
volumeMounts:
- mountPath: /root/logs
name: rocketmq-data
subPath: mq-brokeroptlogs
- mountPath: /data/rocketmq
name: rocketmq-data
subPath: mq-brokeroptstore
- name: broker-config
mountPath: /usr/local/rocketmq-4.8.0/conf/broker-a-s.properties
subPath: broker-a-s.properties
- name: time
mountPath: /etc/localtime
lifecycle:
postStart:
exec:
command: ["/bin/sh","-c","touch /tmp/health"]
livenessProbe:
exec:
command: ["test","-e","/tmp/health"]
initialDelaySeconds: 5
timeoutSeconds: 5
periodSeconds: 10
readinessProbe:
tcpSocket:
port: 20911
initialDelaySeconds: 15
timeoutSeconds: 5
periodSeconds: 20
volumes:
- name: broker-config
configMap:
name: rocketmq-config
items:
- key: broker-a-s.properties
path: broker-a-s.properties
- name: time
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
volumeClaimTemplates:
- metadata:
name: rocketmq-data
annotations:
volume.beta.kubernetes.io/storage-class: "course-nfs-storage"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 4Gi
broker-b-statefulset.yaml:
apiVersion: v1
kind: Service
metadata:
labels:
app: broker-b
name: broker-b
namespace: default
spec:
type: NodePort
ports:
- port: 20911
targetPort: 20911
name: broker-port
nodePort: 30913
selector:
app: broker-b
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: broker-b
namespace: default
spec:
serviceName: broker-b
replicas: 1
selector:
matchLabels:
app: broker-b
template:
metadata:
labels:
app: broker-b
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app"
operator: In
values:
- broker-b
topologyKey: "kubernetes.io/hostname"
imagePullSecrets:
- name: harborsecret
containers:
- name: broker-b
image: 172.16.10.190:80/library/rocketmq:4.8.0
imagePullPolicy: Always
command: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker -c /usr/local/rocketmq-4.8.0/conf/broker-b.properties"]
volumeMounts:
- mountPath: /root/logs
name: rocketmq-data
subPath: mq-brokeroptlogs
- mountPath: /data/rocketmq
name: rocketmq-data
subPath: mq-brokeroptstore
- name: broker-config
mountPath: /usr/local/rocketmq-4.8.0/conf/broker-b.properties
subPath: broker-b.properties
- name: time
mountPath: /etc/localtime
lifecycle:
postStart:
exec:
command: ["/bin/sh","-c","touch /tmp/health"]
livenessProbe:
exec:
command: ["test","-e","/tmp/health"]
initialDelaySeconds: 5
timeoutSeconds: 5
periodSeconds: 10
readinessProbe:
tcpSocket:
port: 20911
initialDelaySeconds: 15
timeoutSeconds: 5
periodSeconds: 20
volumes:
- name: broker-config
configMap:
name: rocketmq-config
- name: time
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
volumeClaimTemplates:
- metadata:
name: rocketmq-data
annotations:
volume.beta.kubernetes.io/storage-class: "course-nfs-storage"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 4Gi
broker-b-s-statefulset.yaml:
apiVersion: v1
kind: Service
metadata:
labels:
app: broker-b-s
name: broker-b-s
namespace: default
spec:
type: NodePort
ports:
- port: 20911
targetPort: 20911
name: broker-port
nodePort: 30914
selector:
app: broker-b-s
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: broker-b-s
namespace: default
spec:
serviceName: broker-b-s
replicas: 1
selector:
matchLabels:
app: broker-b-s
template:
metadata:
labels:
app: broker-b-s
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app"
operator: In
values:
- broker-b-s
topologyKey: "kubernetes.io/hostname"
imagePullSecrets:
- name: harborsecret
containers:
- name: broker-b-s
image: 172.16.10.190:80/library/rocketmq:4.8.0
imagePullPolicy: Always
command: ["sh","-c","/usr/local/rocketmq-4.8.0/bin/mqbroker -c /usr/local/rocketmq-4.8.0/conf/broker-b-s.properties"]
volumeMounts:
- mountPath: /root/logs
name: rocketmq-data
subPath: mq-brokeroptlogs
- mountPath: /data/rocketmq
name: rocketmq-data
subPath: mq-brokeroptstore
- name: broker-config
mountPath: /usr/local/rocketmq-4.8.0/conf/broker-b-s.properties
subPath: broker-b-s.properties
- name: time
mountPath: /etc/localtime
lifecycle:
postStart:
exec:
command: ["/bin/sh","-c","touch /tmp/health"]
livenessProbe:
exec:
command: ["test","-e","/tmp/health"]
initialDelaySeconds: 5
timeoutSeconds: 5
periodSeconds: 10
readinessProbe:
tcpSocket:
port: 20911
initialDelaySeconds: 15
timeoutSeconds: 5
periodSeconds: 20
volumes:
- name: broker-config
configMap:
name: rocketmq-config
items:
- key: broker-b-s.properties
path: broker-b-s.properties
- name: time
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
volumeClaimTemplates:
- metadata:
name: rocketmq-data
annotations:
volume.beta.kubernetes.io/storage-class: "course-nfs-storage"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 4Gi
namesrv-statefulset.yaml:
apiVersion: v1
kind: Service
metadata:
labels:
app: mq-namesrv
name: mq-namesrv
namespace: default
spec:
type: NodePort
ports:
- port: 20901
targetPort: 20901
name: namesrv-port
nodePort: 30915
selector:
app: mq-namesrv
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mq-namesrv
namespace: default
spec:
serviceName: mq-namesrv
replicas: 1
selector:
matchLabels:
app: mq-namesrv
template:
metadata:
labels:
app: mq-namesrv
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app"
operator: In
values:
- mq-namesrv
topologyKey: "kubernetes.io/hostname"
imagePullSecrets:
- name: harborsecret
containers:
- name: mq-namesrv
image: 172.16.10.190:80/library/rocketmq-namesrv:4.8.0
imagePullPolicy: Always
volumeMounts:
- name: time
mountPath: /etc/localtime
lifecycle:
postStart:
exec:
command: ["/bin/sh","-c","touch /tmp/health"]
livenessProbe:
exec:
command: ["test","-e","/tmp/health"]
initialDelaySeconds: 5
timeoutSeconds: 5
periodSeconds: 10
readinessProbe:
tcpSocket:
port: 20901
initialDelaySeconds: 15
timeoutSeconds: 5
periodSeconds: 20
volumes:
- name: time
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
rocketmq-externals-deployment.yaml:
apiVersion: v1
kind: Service
metadata:
labels:
app: mq-externals
name: mq-externals
namespace: default
spec:
type: NodePort
ports:
- port: 8080
targetPort: 8080
name: console-port
nodePort: 30916
selector:
app: mq-externals
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mq-externals
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: mq-externals
template:
metadata:
labels:
app: mq-externals
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app"
operator: In
values:
- mq-externals
topologyKey: "kubernetes.io/hostname"
imagePullSecrets:
- name: harborsecret
containers:
- name: mq-externals
image: 172.16.10.190:80/library/rocketmq-externals:v3
imagePullPolicy: Always
volumeMounts:
- name: time
mountPath: /etc/localtime
lifecycle:
postStart:
exec:
command: ["/bin/sh","-c","touch /tmp/health"]
livenessProbe:
exec:
command: ["test","-e","/tmp/health"]
initialDelaySeconds: 5
timeoutSeconds: 5
periodSeconds: 10
readinessProbe:
tcpSocket:
port: 8080
initialDelaySeconds: 15
volumes:
- name: time
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
7、部署服务
将6步的文件拷贝至K8s
kubectl apply -f .
8、查看
http://172.16.10.160:30916
更多推荐
所有评论(0)