k8s上zookeeper和kafka部署
helm安装kafka1、helm安装不做介绍2、helm下载kafka和zk[root@cce-master1 soft]# helm search hub kafkaURLCHART VERSIONAPP VERSIONDESCRIPTIONhttps://hub.helm.sh/charts/riftbit/kafka14.2.02.8.1
·
helm安装kafka
1、helm安装
不做介绍
2、helm下载kafka和zk
[root@cce-master1 soft]# helm search hub kafka
URL CHART VERSION APP VERSION DESCRIPTION
https://hub.helm.sh/charts/riftbit/kafka 14.2.0 2.8.1 Apache Kafka is a distributed streaming platform.
https://hub.helm.sh/charts/bitnami/kafka 14.9.0 2.8.1 Apache Kafka is a distributed streaming platform.
https://hub.helm.sh/charts/bitnami-aks/kafka 14.9.0 2.8.1 Apache Kafka is a distributed streaming platform.
helm pull bitnami/kafka
helm pull bitnami/zookeeper
3、安装zookeeper
[root@cce-master1 soft]# helm install -n zookeeper zookeeper bitnami/zookeeper --set persistence.storageClass=copaddon-zk-kafka,persistence.size=1Gi,replicaCount=3
helm install -f zookeeper/values.yaml --name zookeeper-cluster bitnami/zookeeper --namespace zookeeper
[root@cce-master1 soft]# helm install -n zookeeper zookeeper bitnami/zookeeper --set persistence.storageClass=copaddon-zk-kafka,persistence.size=1Gi,replicaCount=3
NAME: zookeeper
LAST DEPLOYED: Wed Jan 12 16:50:23 2022
NAMESPACE: zookeeper
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: zookeeper
CHART VERSION: 7.6.1
APP VERSION: 3.7.0
** Please be patient while the chart is being deployed **
ZooKeeper can be accessed via port 2181 on the following DNS name from within your cluster:
zookeeper.zookeeper.svc.cluster.local
To connect to your ZooKeeper server run the following commands:
export POD_NAME=$(kubectl get pods --namespace zookeeper -l "app.kubernetes.io/name=zookeeper,app.kubernetes.io/instance=zookeeper,app.kubernetes.io/component=zookeeper" -o jsonpath="{.items[0].metadata.name}")
kubectl exec -it $POD_NAME -- zkCli.sh
To connect to your ZooKeeper server from outside the cluster execute the following commands:
kubectl port-forward --namespace zookeeper svc/zookeeper 2181:2181 &
zkCli.sh 127.0.0.1:2181
或者是修该values.yaml文件,修改相应参数
persistence.storageClass=copaddon-zk-kafka
persistence.size=1Gi,replicaCount=3
global.imagePullSecrets= ["default-secret"]
image仓库
helm install -f zookeeper/values.yaml --name zookeeper-cluster bitnami/zookeeper --namespace zookeeper
[root@cce-master3 ~]# kubectl get pod -n zookeeper
NAME READY STATUS RESTARTS AGE
zookeeper-0 1/1 Running 0 51s
zookeeper-1 1/1 Running 0 51s
zookeeper-2 1/1 Running 0 49s
卸载
helm uninstall -n zookeeper zookeeper
4、安装kafka
zookeeper.zookeeper.svc.cluster.local
helm install -n kafka kafka bitnami/kafka --set zookeeper.enabled=false --set replicaCount=3 --set externalZookeeper.servers=zookeeper.zookeeper.svc.cluster.local --set persistence.storageClass=copaddon-zk-kafka
helm install -f kafka/values.yaml --name kafka-cluster bitnami/kafka --namespace kafka
[root@cce-master1 soft]# helm install -n kafka kafka bitnami/kafka --set zookeeper.enabled=false --set replicaCount=3 --set externalZookeeper.servers=zookeeper.zookeeper.svc.cluster.local --set persistence.storageClass=copaddon-zk-kafka
NAME: kafka
LAST DEPLOYED: Wed Jan 12 17:33:16 2022
NAMESPACE: kafka
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: kafka
CHART VERSION: 14.9.0
APP VERSION: 2.8.1
** Please be patient while the chart is being deployed **
Kafka can be accessed by consumers via port 9092 on the following DNS name from within your cluster:
kafka.kafka.svc.cluster.local
Each Kafka broker can be accessed by producers via port 9092 on the following DNS name(s) from within your cluster:
kafka-0.kafka-headless.kafka.svc.cluster.local:9092
kafka-1.kafka-headless.kafka.svc.cluster.local:9092
kafka-2.kafka-headless.kafka.svc.cluster.local:9092
To create a pod that you can use as a Kafka client run the following commands:
kubectl run kafka-client --restart='Never' --image docker.io/bitnami/kafka:2.8.1-debian-10-r73 --namespace kafka --command -- sleep infinity
kubectl exec --tty -i kafka-client --namespace kafka -- bash
PRODUCER:
kafka-console-producer.sh \
--broker-list kafka-0.kafka-headless.kafka.svc.cluster.local:9092,kafka-1.kafka-headless.kafka.svc.cluster.local:9092,kafka-2.kafka-headless.kafka.svc.cluster.local:9092 \
--topic test
CONSUMER:
kafka-console-consumer.sh \
--bootstrap-server kafka.kafka.svc.cluster.local:9092 \
--topic test \
--from-beginning
或者修改values.yaml文件,修改成桉树
replicaCount=3
externalZookeeper.servers=zookeeper.zookeeper.svc.cluster.local
persistence.storageClass=copaddon-zk-kafka
global.imagePullSecrets= ["default-secret"]
image修改
helm install -f kafka/values.yaml --name kafka-cluster bitnami/kafka --namespace kafka
5、zookeeper测试
kubectl get pods --namespace zookeeper -l "app.kubernetes.io/name=zookeeper,app.kubernetes.io/instance=zookeeper,app.kubernetes.io/component=zookeeper" -o jsonpath="{.items[0].metadata.name}"
export POD_NAME=$(kubectl get pods --namespace zookeeper -l "app.kubernetes.io/name=zookeeper,app.kubernetes.io/instance=zookeeper,app.kubernetes.io/component=zookeeper" -o jsonpath="{.items[0].metadata.name}")
kubectl exec -it $POD_NAME -n zookeeper -- zkCli.sh #可以查看zk中相关的目录
6、kafka测试
kubectl run kafka-client --restart='Never' --image docker.io/bitnami/kafka:2.8.1-debian-10-r73 --namespace kafka --command -- sleep infinity
kubectl get pod -n kafka
kubectl exec --tty -i kafka-client --namespace kafka -- bash
PRODUCER:
kafka-console-producer.sh --broker-list kafka-0.kafka-headless.kafka.svc.cluster.local:9092,kafka-1.kafka-headless.kafka.svc.cluster.local:9092,kafka-2.kafka-headless.kafka.svc.cluster.local:9092 --topic test
CONSUMER
kafka-console-consumer.sh --bootstrap-server kafka.kafka.svc.cluster.local:9092 --topic test --from-beginning
7、kafka-manager安装
[root@cce-master1 soft]# helm repo add stable http://mirror.azure.cn/kubernetes/charts
[root@cce-master1 soft]# helm search repo kafka-manager
NAME CHART VERSION APP VERSION DESCRIPTION
stable/kafka-manager 2.3.5 1.3.3.22 DEPRECATED - A tool for managing Apache Kafka.
[root@cce-master1 soft]# helm fetch stable/kafka-manager
[root@cce-master1 soft]# cat kafka-manager-values.yaml
image:
repository: cce.zhanghsn/com/zhanghsn/kafka-manager
tag: 1.3.3.22
zkHosts: zookeeper:2181
basicAuth:
enabled: true
username: admin
password: admin
ingress:
enabled: true
hosts:
- km.hongda.com
helm install --name kafka-manager --namespace kafka -f kafka-manager-values.yaml stable/kafka-manager
[root@cce-master1 soft]# kubectl get pod,svc,ing -nkafka
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAME READY STATUS RESTARTS AGE
pod/kafka-0 1/1 Running 0 104m
pod/kafka-1 1/1 Running 0 104m
pod/kafka-2 1/1 Running 0 104m
pod/kafka-client 1/1 Running 0 94m
pod/kafka-manager-7869d5f76b-jtp8f 1/1 Running 0 7m42s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kafka ClusterIP 10.247.42.103 <none> 9092/TCP 17h
service/kafka-headless ClusterIP None <none> 9092/TCP,9093/TCP 17h
service/kafka-manager ClusterIP 10.247.149.131 <none> 9000/TCP 7m42s
service/kafka-manager-nodeport NodePort 10.247.43.39 <none> 9000:32065/TCP 21m
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.extensions/kafka-manager <none> km.hongda.com 10.11.95.235,10.11.95.236,10.11.95.237 8 0 7m42s
[root@cce-master1 soft]#
kafka-manager添加kafka集群
更多推荐
已为社区贡献5条内容
所有评论(0)