安装准备

• 树莓派k8s集群

root@pi4-master01:~# kubectl get nodes -o wideNAME           STATUS   ROLES    AGE     VERSION    INTERNAL-IP    EXTERNAL-IP   OS-IMAGE           KERNEL-VERSION     CONTAINER-RUNTIMEpi4-master01   Ready    master   4d18h   v1.15.10   192.168.5.18   <none>        Ubuntu 20.04 LTS   5.4.0-1011-raspi   docker://18.9.9pi4-node01     Ready    node     4d17h   v1.15.10   192.168.5.19   <none>        Ubuntu 20.04 LTS   5.4.0-1011-raspi   docker://18.9.9pi4-node02     Ready    node     4d17h   v1.15.10   192.168.5.20   <none>        Ubuntu 20.04 LTS   5.4.0-1011-raspi   docker://18.9.9

• 树莓派k8s集群已安装helm

root@pi4-master01:~/k8s/cluster-monitoring-0.37.0# helm versionClient: &version.Version{SemVer:"v2.15.0", GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750", GitTreeState:"clean"}Server: &version.Version{SemVer:"v2.15.0+unreleased", GitCommit:"9668ad4d90c5e95bd520e58e7387607be6b63bb6", GitTreeState:"dirty"}

• 树莓派k8s集群已安装存储类，并设置为默认存储

root@pi4-master01:~/k8s/cluster-monitoring-0.37.0# kubectl get storageclassNAME                   PROVISIONER             AGElocal-path (default)   rancher.io/local-path   4d15h

helm安装kafka集群

这里我们选择用helm方式安装，我们选择的是官方的chart地址：https://github.com/helm/charts/tree/master/incubator/kafka, 这个chart默认在 http://storage.googleapis.com/kubernetes-charts-incubator 上。

root@pi4-master01:~# helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator"incubator" has been added to your repositoriesroot@pi4-master01:~# helm repo listNAME       URL                                                      local      http://127.0.0.1:8879/charts                             stable     http://mirror.azure.cn/kubernetes/charts/                incubator  http://storage.googleapis.com/kubernetes-charts-incubatorroot@k8s-master:~# helm search kafkaNAME                       CHART VERSION  APP VERSION  DESCRIPTION                                                 incubator/kafka            0.21.2         5.0.1        Apache Kafka is publish-subscribe messaging rethought as ...stable/kafka-manager       2.3.1          1.3.3.22     A tool for managing Apache Kafka.                           incubator/schema-registry  1.2.0          5.0.1        Schema Registry provides a serving layer for your metadat...stable/schema-registry-ui  0.4.2          v0.9.5       This is a web tool for the confluentinc/schema-registry i...

incubator/kafka就是我们需要安装的chart。这个chart还以子chart的形式引入了zookeeper的chart，默认情况安装，不仅会安装3个节点的kafka集群，也会安装3个节点的zookeeper集群。

注意：incubator/kafka从版本0.17.0开始，内置的zookeeper版本切换成了zookeeper:3.5.5，这个版本的arm架构镜像目前社区内还没有，自己构建暂时不成功，所以暂时使用版本0.16.4来安装，0.16.4版本用的是google的k8szk，具体镜像是参考k8szk的Dockerfile构建而成，封装版本是3.4.14。

• 使用helm安装kakfka集群命令如下

helm install -n kafka \incubator/kafka --version 0.16.4 \--set image=iecedge/cp-kafka-arm64 \--set imageTag=5.0.1 \--set zookeeper.image.repository=pi4k8s/k8szk \--set zookeeper.image.tag=v3.4.14

通过上述命令，将以默认的配置在Kubernetes中部署kafka。默认情况下，chart会安装部署3个kafka Pod和3个 zookeeper Pod。其中3个zookeeper Pod组成zookeeper集群，另外3个kafka Pod依赖该zookeeper集群组成kafka集群。

该chart中涉及的镜像均是x86的，只要把相关的镜像换成arm64v8架构的镜像即可。另外，该chart默认开启了kafka的持久化存储，不指定storageclass的话，将使用k8s的默认存储。因为我们的树莓派k8s集群已经设置了local-path为默认storageclass，所以不需要关闭持久化存储既可正常安装。

• 查看创建的pod

root@pi4-master01:~# kubectl get pod | grep kafkakafka-0                                          1/1     Running   2          9m55skafka-1                                          1/1     Running   0          7m10skafka-2                                          1/1     Running   0          4m17skafka-zookeeper-0                                1/1     Running   0          9m55skafka-zookeeper-1                                1/1     Running   0          9m31skafka-zookeeper-2                                1/1     Running   0          7m20s

• 查看创建的svc

root@pi4-master01:~# kubectl get svc|grep kafkakafka                           ClusterIP   10.102.254.197   <none>         9092/TCP                     94mkafka-headless                  ClusterIP   None             <none>         9092/TCP                     94mkafka-zookeeper                 ClusterIP   10.105.26.83     <none>         2181/TCP                     94mkafka-zookeeper-headless        ClusterIP   None             <none>         2181/TCP,3888/TCP,2888/TCP   94m

• 查看创建的pv，pvc

root@pi4-master01:~# kubectl get pvc,pv | grep kafkapersistentvolumeclaim/datadir-kafka-0                           Bound    pvc-35d461a8-da86-404b-9654-12416ee312e4   1Gi        RWO            local-path     152mpersistentvolumeclaim/datadir-kafka-1                           Bound    pvc-8042ba75-3504-4c55-ad18-0de74badf2bd   1Gi        RWO            local-path     7m59spersistentvolumeclaim/datadir-kafka-2                           Bound    pvc-bf828ff4-fbf4-4d91-a042-6d6f8b4bf563   1Gi        RWO            local-path     5m6spersistentvolume/pvc-35d461a8-da86-404b-9654-12416ee312e4   1Gi        RWO            Delete           Bound    default/datadir-kafka-0                           local-path              152mpersistentvolume/pvc-8042ba75-3504-4c55-ad18-0de74badf2bd   1Gi        RWO            Delete           Bound    default/datadir-kafka-1                           local-path              7m56spersistentvolume/pvc-bf828ff4-fbf4-4d91-a042-6d6f8b4bf563   1Gi        RWO            Delete           Bound    default/datadir-kafka-2                           local-path              5m

验证

验证kafka集群

该chart默认使用的kafka镜像是confluentinc/cp-kafka:5.0.1，这个版本对应Apache Kafka的2.0.X系列。Confluent Platform Kafka(简称CP Kafka)提供了一些Apache Kafka没有的高级特性，例如跨数据中心备份、Schema注册中心以及集群监控工具等。
如下是cp-kafka和apache kafka对应关系

• 进入kafka-0容器查看当前kafka版本

root@pi4-master01:~# kubectl exec -it kafka-0 bashroot@kafka-0:/# ls /usr/share/java/kafka | grep kafkakafka-clients-2.0.1-cp1.jarkafka-log4j-appender-2.0.1-cp1.jarkafka-streams-2.0.1-cp1.jarkafka-streams-examples-2.0.1-cp1.jarkafka-streams-scala_2.11-2.0.1-cp1.jarkafka-streams-test-utils-2.0.1-cp1.jarkafka-tools-2.0.1-cp1.jarkafka.jarkafka_2.11-2.0.1-cp1-javadoc.jarkafka_2.11-2.0.1-cp1-scaladoc.jarkafka_2.11-2.0.1-cp1-sources.jarkafka_2.11-2.0.1-cp1-test-sources.jarkafka_2.11-2.0.1-cp1-test.jarkafka_2.11-2.0.1-cp1.jar

可以看到对应apache kafka的版本号是2.11-2.0.1，前面2.11是Scala编译器的版本，Kafka的服务器端代码是使用Scala语言开发的，后边2.0.1是Kafka的版本。

• 进入kafka-0容器创建一个Topic test1

root@pi4-master01:~# kubectl exec -it kafka-0 bashroot@kafka-0:/# kafka-topics --zookeeper kafka-zookeeper:2181 --topic test1 --create --partitions 3 --replication-factor 3Created topic "test1".

• 进入kafka-1容器查看当前Topic

root@pi4-master01:~# kubectl exec -it kafka-1 bashroot@kafka-1:/# kafka-topics --zookeeper kafka-zookeeper:2181 --listtest1

• 进入kafka-0容器创建一个生产者向Topic test1并发消息

root@pi4-master01:~# kubectl exec -it kafka-0 bashroot@kafka-0:/#  kafka-console-producer --broker-list kafka:9092 -topic test1>hello>world

• 同时进入kafka-1容器创建一个消费者监听Topic test1

root@pi4-master01:~# kubectl exec -it kafka-0 bash root@kafka-1:/# kafka-console-consumer --bootstrap-server kafka:9092 -topic test1helloworld

验证zookeeper集群

root@pi4-master01:~# kubectl exec -it kafka-zookeeper-0 zkServer.sh statusZooKeeper JMX enabled by defaultZooKeeper remote JMX Port set to 1099ZooKeeper remote JMX authenticate set to falseZooKeeper remote JMX ssl set to falseZooKeeper remote JMX log4j set to trueUsing config: /usr/bin/../etc/zookeeper/zoo.cfgMode: followerroot@pi4-master01:~# kubectl exec -it kafka-zookeeper-1 zkServer.sh statusZooKeeper JMX enabled by defaultZooKeeper remote JMX Port set to 1099ZooKeeper remote JMX authenticate set to falseZooKeeper remote JMX ssl set to falseZooKeeper remote JMX log4j set to trueUsing config: /usr/bin/../etc/zookeeper/zoo.cfgMode: leaderroot@pi4-master01:~# kubectl exec -it kafka-zookeeper-2 zkServer.sh statusZooKeeper JMX enabled by defaultZooKeeper remote JMX Port set to 1099ZooKeeper remote JMX authenticate set to falseZooKeeper remote JMX ssl set to falseZooKeeper remote JMX log4j set to trueUsing config: /usr/bin/../etc/zookeeper/zoo.cfgMode: follower

可以看到3个pod中会有一个leader,2个follower，集群状态OK。

其它配置

基于helm在树莓派k8s集群安装kafka集群还是很方便的，只需要使用arm64v8架构的镜像即可。但是如果想要部署的kafka集群更具有个性，还是很有必要研究一下该chart的其它配置项，这些配置项我们可以通过helm inspect命令来查看

root@pi4-master01:~# helm inspect values incubator/kafka --version 0.16.4# ------------------------------------------------------------------------------# Kafka:# ------------------------------------------------------------------------------## The StatefulSet installs 3 pods by defaultreplicas: 3## The kafka image repositoryimage: "confluentinc/cp-kafka"## The kafka image tagimageTag: "5.0.1"  # Confluent image for Kafka 2.0.0## Specify a imagePullPolicy## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-imagesimagePullPolicy: "IfNotPresent"## Configure resource requests and limits## ref: http://kubernetes.io/docs/user-guide/compute-resources/resources: {}  # limits:  #   cpu: 200m  #   memory: 1536Mi  # requests:  #   cpu: 100m  #   memory: 1024MikafkaHeapOptions: "-Xmx1G -Xms1G"## Optional Container Security contextsecurityContext: {}## The StatefulSet Update Strategy which Kafka will use when changes are applied.## ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategiesupdateStrategy:  type: "OnDelete"## Start and stop pods in Parallel or OrderedReady (one-by-one.)  Note - Can not change after first release.## ref: https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#pod-management-policypodManagementPolicy: OrderedReady## Useful if using any custom authorizer## Pass in some secrets to use (if required)# secrets:# - name: myKafkaSecret#   keys:#     - username#     - password#   # mountPath: /opt/kafka/secret# - name: myZkSecret#   keys:#     - user#     - pass#   mountPath: /opt/zookeeper/secret## The subpath within the Kafka container's PV where logs will be stored.## This is combined with `persistence.mountPath`, to create, by default: /opt/kafka/data/logslogSubPath: "logs"## Use an alternate scheduler, e.g. "stork".## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/### schedulerName:## Use an alternate serviceAccount## Useful when using images in custom repositories# serviceAccountName:## Pod scheduling preferences (by default keep pods within a release on separate nodes).## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity## By default we don't set affinityaffinity: {}## Alternatively, this typical example defines:## antiAffinity (to keep Kafka pods on separate pods)## and affinity (to encourage Kafka pods to be collocated with Zookeeper pods)# affinity:#   podAntiAffinity:#     requiredDuringSchedulingIgnoredDuringExecution:#     - labelSelector:#         matchExpressions:#         - key: app#           operator: In#           values:#           - kafka#       topologyKey: "kubernetes.io/hostname"#   podAffinity:#     preferredDuringSchedulingIgnoredDuringExecution:#      - weight: 50#        podAffinityTerm:#          labelSelector:#            matchExpressions:#            - key: app#              operator: In#              values:#                - zookeeper#          topologyKey: "kubernetes.io/hostname"## Node labels for pod assignment## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselectornodeSelector: {}## Readiness probe config.## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/##readinessProbe:  initialDelaySeconds: 30  periodSeconds: 10  timeoutSeconds: 5  successThreshold: 1  failureThreshold: 3## Period to wait for broker graceful shutdown (sigterm) before pod is killed (sigkill)## ref: https://kubernetes-v1-4.github.io/docs/user-guide/production-pods/#lifecycle-hooks-and-termination-notice## ref: https://kafka.apache.org/10/documentation.html#brokerconfigs controlled.shutdown.*terminationGracePeriodSeconds: 60# Tolerations for nodes that have taints on them.# Useful if you want to dedicate nodes to just run kafka# https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/tolerations: []# tolerations:# - key: "key"#   operator: "Equal"#   value: "value"#   effect: "NoSchedule"## Headless service.##headless:  # annotations:  # targetPort:  port: 9092## External access.##external:  enabled: false  # type can be either NodePort or LoadBalancer  type: NodePort  # annotations:  #  service.beta.kubernetes.io/openstack-internal-load-balancer: "true"  dns:    useInternal: false    useExternal: true  # If using external service type LoadBalancer and external dns, set distinct to true below.  # This creates an A record for each statefulset pod/broker. You should then map the  # A record of the broker to the EXTERNAL IP given by the LoadBalancer in your DNS server.  distinct: false  servicePort: 19092  firstListenerPort: 31090  domain: cluster.local  loadBalancerIP: []  init:    image: "lwolf/kubectl_deployer"    imageTag: "0.4"    imagePullPolicy: "IfNotPresent"# Annotation to be added to Kafka podspodAnnotations: {}# Labels to be added to Kafka podspodLabels: {}  # service: broker  # team: developers## Configuration Overrides. Specify any Kafka settings you would like set on the StatefulSet## here in map format, as defined in the official docs.## ref: https://kafka.apache.org/documentation/#brokerconfigs##configurationOverrides:  "confluent.support.metrics.enable": false  # Disables confluent metric submission  # "auto.leader.rebalance.enable": true  # "auto.create.topics.enable": true  # "controlled.shutdown.enable": true  # "controlled.shutdown.max.retries": 100  ## Options required for external access via NodePort  ## ref:  ## - http://kafka.apache.org/documentation/#security_configbroker  ## - https://cwiki.apache.org/confluence/display/KAFKA/KIP-103%3A+Separation+of+Internal+and+External+traffic  ##  ## Setting "advertised.listeners" here appends to "PLAINTEXT://${POD_IP}:9092,", ensure you update the domain  ## If external service type is Nodeport:  # "advertised.listeners": |-  #   EXTERNAL://kafka.cluster.local:$((31090 + ${KAFKA_BROKER_ID}))  ## If external service type is LoadBalancer and distinct is true:  # "advertised.listeners": |-  #   EXTERNAL://kafka-$((${KAFKA_BROKER_ID})).cluster.local:19092  ## If external service type is LoadBalancer and distinct is false:  # "advertised.listeners": |-  #   EXTERNAL://${LOAD_BALANCER_IP}:31090  ## Uncomment to define the EXTERNAL Listener protocol  # "listener.security.protocol.map": |-  #   PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT## set extra ENVs#   key: "value"envOverrides: {}## A collection of additional ports to expose on brokers (formatted as normal containerPort yaml)# Useful when the image exposes metrics (like prometheus, etc.) through a javaagent instead of a sidecaradditionalPorts: {}## Persistence configuration. Specify if and how to persist data to a persistent volume.##persistence:  enabled: true  ## The size of the PersistentVolume to allocate to each Kafka Pod in the StatefulSet. For  ## production servers this number should likely be much larger.  ##  size: "1Gi"  ## The location within the Kafka container where the PV will mount its storage and Kafka will  ## store its logs.  ##  mountPath: "/opt/kafka/data"  ## Kafka data Persistent Volume Storage Class  ## If defined, storageClassName:   ## If set to "-", storageClassName: "", which disables dynamic provisioning  ## If undefined (the default) or set to null, no storageClassName spec is  ##   set, choosing the default provisioner.  (gp2 on AWS, standard on  ##   GKE, AWS & OpenStack)  ##  # storageClass:jmx:  ## Rules to apply to the Prometheus JMX Exporter.  Note while lots of stats have been cleaned and exposed,  ## there are still more stats to clean up and expose, others will never get exposed.  They keep lots of duplicates  ## that can be derived easily.  The configMap in this chart cleans up the metrics it exposes to be in a Prometheus  ## format, eg topic, broker are labels and not part of metric name. Improvements are gladly accepted and encouraged.  configMap:    ## Allows disabling the default configmap, note a configMap is needed    enabled: true    ## Allows setting values to generate confimap    ## To allow all metrics through (warning its crazy excessive) comment out below `overrideConfig` and set    ## `whitelistObjectNames: []`    overrideConfig: {}      # jmxUrl: service:jmx:rmi:///jndi/rmi://127.0.0.1:5555/jmxrmi      # lowercaseOutputName: true      # lowercaseOutputLabelNames: true      # ssl: false      # rules:      # - pattern: ".*"    ## If you would like to supply your own ConfigMap for JMX metrics, supply the name of that    ## ConfigMap as an `overrideName` here.    overrideName: ""  ## Port the jmx metrics are exposed in native jmx format, not in Prometheus format  port: 5555  ## JMX Whitelist Objects, can be set to control which JMX metrics are exposed.  Only whitelisted  ## values will be exposed via JMX Exporter.  They must also be exposed via Rules.  To expose all metrics  ## (warning its crazy excessive and they aren't formatted in a prometheus style) (1) `whitelistObjectNames: []`  ## (2) commented out above `overrideConfig`.  whitelistObjectNames:  # []  - kafka.controller:*  - kafka.server:*  - java.lang:*  - kafka.network:*  - kafka.log:*## Prometheus Exporters / Metrics##prometheus:  ## Prometheus JMX Exporter: exposes the majority of Kafkas metrics  jmx:    enabled: false    ## The image to use for the metrics collector    image: solsson/kafka-prometheus-jmx-exporter@sha256    ## The image tag to use for the metrics collector    imageTag: a23062396cd5af1acdf76512632c20ea6be76885dfc20cd9ff40fb23846557e8    ## Interval at which Prometheus scrapes metrics, note: only used by Prometheus Operator    interval: 10s    ## Timeout at which Prometheus timeouts scrape run, note: only used by Prometheus Operator    scrapeTimeout: 10s    ## Port jmx-exporter exposes Prometheus format metrics to scrape    port: 5556    resources: {}      # limits:      #   cpu: 200m      #   memory: 1Gi      # requests:      #   cpu: 100m      #   memory: 100Mi  ## Prometheus Kafka Exporter: exposes complimentary metrics to JMX Exporter  kafka:    enabled: false    ## The image to use for the metrics collector    image: danielqsj/kafka-exporter    ## The image tag to use for the metrics collector    imageTag: v1.2.0    ## Interval at which Prometheus scrapes metrics, note: only used by Prometheus Operator    interval: 10s    ## Timeout at which Prometheus timeouts scrape run, note: only used by Prometheus Operator    scrapeTimeout: 10s    ## Port kafka-exporter exposes for Prometheus to scrape metrics    port: 9308    ## Resource limits    resources: {}#      limits:#        cpu: 200m#        memory: 1Gi#      requests:#        cpu: 100m#        memory: 100Mi    # Tolerations for nodes that have taints on them.    # Useful if you want to dedicate nodes to just run kafka-exporter    # https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/    tolerations: []    # tolerations:    # - key: "key"    #   operator: "Equal"    #   value: "value"    #   effect: "NoSchedule"    ## Pod scheduling preferences (by default keep pods within a release on separate nodes).    ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity    ## By default we don't set affinity    affinity: {}    ## Alternatively, this typical example defines:    ## affinity (to encourage Kafka Exporter pods to be collocated with Kafka pods)    # affinity:    #   podAffinity:    #     preferredDuringSchedulingIgnoredDuringExecution:    #      - weight: 50    #        podAffinityTerm:    #          labelSelector:    #            matchExpressions:    #            - key: app    #              operator: In    #              values:    #                - kafka    #          topologyKey: "kubernetes.io/hostname"    ## Node labels for pod assignment    ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector    nodeSelector: {}  operator:    ## Are you using Prometheus Operator?    enabled: false    serviceMonitor:      # Namespace Prometheus is installed in      namespace: monitoring      ## Defaults to whats used if you follow CoreOS [Prometheus Install Instructions](https://github.com/coreos/prometheus-operator/tree/master/helm#tldr)      ## [Prometheus Selector Label](https://github.com/coreos/prometheus-operator/blob/master/helm/prometheus/templates/prometheus.yaml#L65)      ## [Kube Prometheus Selector Label](https://github.com/coreos/prometheus-operator/blob/master/helm/kube-prometheus/values.yaml#L298)      selector:        prometheus: kube-prometheus## Kafka Config job configuration##configJob:  ## Specify the number of retries before considering kafka-config job as failed.  ## https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#pod-backoff-failure-policy  backoffLimit: 6## Topic creation and configuration.## The job will be run on a deployment only when the config has been changed.## - If 'partitions' and 'replicationFactor' are specified we create the topic (with --if-not-exists.)## - If 'partitions', 'replicationFactor' and 'reassignPartitions' are specified we reassign the partitions to## increase the replication factor of an existing topic.## - If 'partitions' is specified we 'alter' the number of partitions. This will## silently and safely fail if the new setting isn’t strictly larger than the old (i.e. a NOOP.) Do be aware of the## implications for keyed topics (ref: https://docs.confluent.io/current/kafka/post-deployment.html#admin-operations)## - If 'defaultConfig' is specified it's deleted from the topic configuration. If it isn't present,## it will silently and safely fail.## - If 'config' is specified it's added to the topic configuration.#### Note: To increase the 'replicationFactor' of a topic, 'reassignPartitions' must be set to true (see above).##topics: []  # - name: myExistingTopicConfig  #   config: "cleanup.policy=compact,delete.retention.ms=604800000"  # - name: myExistingTopicReassignPartitions  #   partitions: 8  #   replicationFactor: 5  #   reassignPartitions: true  # - name: myExistingTopicPartitions  #   partitions: 8  # - name: myNewTopicWithConfig  #   partitions: 8  #   replicationFactor: 3  #   defaultConfig: "segment.bytes,segment.ms"  #   config: "cleanup.policy=compact,delete.retention.ms=604800000"  # - name: myAclTopicPartitions  #   partitions: 8  #   acls:  #     - user: read  #       operations: [ Read ]  #     - user: read_and_write  #       operations:  #         - Read  #         - Write  #     - user: all  #       operations: [ All ]# ------------------------------------------------------------------------------# Zookeeper:# ------------------------------------------------------------------------------zookeeper:  ## If true, install the Zookeeper chart alongside Kafka  ## ref: https://github.com/kubernetes/charts/tree/master/incubator/zookeeper  enabled: true  ## Configure Zookeeper resource requests and limits  ## ref: http://kubernetes.io/docs/user-guide/compute-resources/  resources: ~  ## Environmental variables to set in Zookeeper  env:    ## The JVM heap size to allocate to Zookeeper    ZK_HEAP_SIZE: "1G"  persistence:    enabled: false    ## The amount of PV storage allocated to each Zookeeper pod in the statefulset    # size: "2Gi"  ## Specify a Zookeeper imagePullPolicy  ## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images  image:    PullPolicy: "IfNotPresent"  ## If the Zookeeper Chart is disabled a URL and port are required to connect  url: ""  port: 2181  ## Pod scheduling preferences (by default keep pods within a release on separate nodes).  ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity  ## By default we don't set affinity:  affinity: {}  # Criteria by which pod label-values influence scheduling for zookeeper pods.  # podAntiAffinity:  #   requiredDuringSchedulingIgnoredDuringExecution:  #     - topologyKey: "kubernetes.io/hostname"  #       labelSelector:  #         matchLabels:  #           release: zookeeper

显然，配置项很丰富，从jvm参数设置、是否开启监控、是否对k8s集群外部内开启服务都能支持。

开启监控

开启监控前，需要在集群部署prometheus，否则不会有效果。可以参考树莓派k8s集群安装监控prometheus进行部署。这个chart中kafka的监控，有两种，分别是jmx和kafka两种类型，这里我们只开启kafka类型监控，需要设置prometheus.kafka.enabled为true，且prometheus.kafka.image和prometheus.kafka.imageTag设置为支持arm64v8架构的镜像监控才能使用，这里选择的镜像是iecedge/kafka-exporter_arm64:v1.2.0。

此外，还需要设置prometheus.operator.enabled为true，才能监控到kafka指标。

使用helm安装开启监控的kafka集群命令如下

helm install -n kafka \incubator/kafka --version 0.16.4 \--set image=iecedge/cp-kafka-arm64 \--set imageTag=5.0.1 \--set prometheus.kafka.enabled=true \--set prometheus.kafka.image=iecedge/kafka-exporter_arm64 \--set prometheus.kafka.imageTag=v1.2.0 \--set prometheus.operator.enabled=true \--set zookeeper.image.repository=pi4k8s/k8szk \--set zookeeper.image.tag=v3.4.14

确认安装状态

root@pi4-master01:~# kubectl get pod | grep kafkakafka-0                                          0/1     Running   0          65skafka-exporter-66f7ddcf5d-phdcp                  0/1     Error     3          76skafka-zookeeper-0                                1/1     Running   0          76skafka-zookeeper-1                                1/1     Running   0          47skafka-zookeeper-2                                0/1     Running   0          18sroot@pi4-master01:~# kubectl get pod | grep kafkakafka-0                                          1/1     Running   1          4m29skafka-1                                          1/1     Running   0          3m6skafka-2                                          1/1     Running   0          2m32skafka-exporter-66f7ddcf5d-phdcp                  1/1     Running   4          4m40skafka-zookeeper-0                                1/1     Running   0          4m40skafka-zookeeper-1                                1/1     Running   0          4m11skafka-zookeeper-2                                1/1     Running   0          3m42sroot@pi4-master01:~# kubectl get svc|grep kafkakafka                           ClusterIP   10.103.216.68    9092/TCP                     4m55skafka-exporter                  ClusterIP   10.97.79.253     9308/TCP                     4m55skafka-headless                  ClusterIP   None             9092/TCP                     4m56skafka-zookeeper                 ClusterIP   10.105.120.146   2181/TCP                     4m55skafka-zookeeper-headless        ClusterIP   None             2181/TCP,3888/TCP,2888/TCP   4m56sroot@pi4-master01:~#  kubectl get servicemonitor --all-namespaces |grep kafkamonitoring   kafka-exporter            5m14s

在prometheus查看exporter是否生效

生效后在grafana系统配置kafka监控面板

• kafka监控面板 ：https://grafana.com/grafana/dashboards/12460

部署步骤如下：

点"Upload .json file"按钮，选择已经下载好的kafka监控面板文件 ,进入面板设置界面如下

 设置好相应参数后，点"Import"按钮，继续导入，引导入kafka监控页面如下

总结

本实践简单介绍了如何使用chart在树莓派k8s集群用helm部署kafka并配置对于kafka的监控，读者可以基于自己的需要自行调整kafka参数以便适用更多场景。

参考：

• 树莓派 k8s 集群入坑指南(点击阅读原文直达链接)

• 树莓派k8s集群安装监控prometheus