kafka3.4.0版本升级--helm部署

使用helm将kafka升级至3.4.0版本，修复漏洞。

小仲码还是要敲代码

1768人浏览 · 2023-03-06 22:55:16

小仲码还是要敲代码 · 2023-03-06 22:55:16 发布

kafka3.4.0版本升级–helm部署

前言

最近由于kafka的漏洞需要升级至3.4.0版本，之前由于不是helm部署，升级起来出现了权限问题、挂盘问题，在k8s搞了许久都搞不定，狠下心来，直接来一波helm安装，在2月份的时候，helm官网已推出chart-21.0.1包（https://artifacthub.io/packages/helm/bitnami/kafka）。用chart-21.0.1包准备开搞。

helm 安装kafka

helm 安装kafka比较简单，便不多说，官网有相应的说明，简单来说就是下载个chart包，在helm环境下执行helm安装即可。

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-kafka bitnami/kafka --version 21.0.1

当然，这样安装是默认value.yaml的配置，但是对于实际应用，并非如此简单，我们要看懂实际部署的需求，比如，挂盘要求，service、端口等都是要根据实际生产或者说之前版本调整的。所以接下来要对value.yaml进行配置，配置属于自己“独一无二”的kafka。

“独一无二”的value.yaml

先说明下kafka实际要求：

3个pod
storageClass为：xxx-xxx-xxx
需要外界访问

修改默认的value.yaml部分配置：

全局配置修改：

global:
  imageRegistry: ""
  ## E.g.
  ## imagePullSecrets:
  ##   - myRegistryKeySecretName
  ##
  imagePullSecrets: []
  storageClass: "xxx-xxx-xxx "

修改镜像
这个比较简单，一般单位的服务器是没法直接连外网的，所以都会有自己的hub镜像仓库，所以先得从外网拉取镜像到自己仓库，然后将下列镜像改为自己的镜像。需要修改kakfa（bitnami/kafka:3.4.0-debian-11-r2）、zookeeper（zookeeper:3.8.1-debian-11-r0）、kubectl（kubectl:1.25.6-debian-11-r10）镜像：以kakfa修改为例：

image:
  registry: hubtest.xxx.com.cn
  repository: xxx/
  tag: bitnami/kafka:3.4.0-debian-11-r2
  digest: ""  #无需填

设置副本数

replicaCount: 3

补充资源配置
一定要在resources这一步补充资源配置，不然pod是启动不了的，那就蛋疼了。

resources:
  requests:
    memory: "64Mi"
    cpu: "250m"
  limits:
    memory: "1280Mi"
    cpu: "500m"

启动集群外部访问kafka–开启externalAccess
这一步非常关键，因为一般应用日志会往kafka上吐，这就是给应用或者filebeat提供吐的入口。做了以下修改：
externalAccess中enabled 设置为 true，autoDiscovery:设置为true
kubectl镜像设置为本地仓库镜像
service type设置为： NodePort
（因为有F5作为负载均衡，所以就不用k8s service自带的LoadBalancer，只要求暴露端口就行，所以就采用nodeport方式。）
添加resource资源
nodePorts:
- 30001
- 30002
- 30003
  注意：个数要和副本数（3）一样
  有个巨坑得和大家说说说，配置中有这么一句话Note: RBAC might be required
  也就是当你启动externalAccess，启动helm的要带上一个参数，不然会报关于RBAC的错误，加上以下一行就行。
  –set rbac.create=true
  如果是rancher 自带的helm可视化部署，直接在命令那一行输入rbac.create=true
  访问方式：
  :9094 或者:30001
  具体修改如下：

## External Access to Kafka brokers configuration
##
externalAccess:
  ## @param externalAccess.enabled Enable Kubernetes external cluster access to Kafka brokers
  ##
  enabled: true
  ## External IPs auto-discovery configuration
  ## An init container is used to auto-detect LB IPs or node ports by querying the K8s API
  ## Note: RBAC might be required
  ##
  autoDiscovery:
    ## @param externalAccess.autoDiscovery.enabled Enable using an init container to auto-detect external IPs/ports by querying the K8s API
    ##
    enabled: true
    ## Bitnami Kubectl image
    ## ref: https://hub.docker.com/r/bitnami/kubectl/tags/
    ## @param externalAccess.autoDiscovery.image.registry Init container auto-discovery image registry
    ## @param externalAccess.autoDiscovery.image.repository Init container auto-discovery image repository
    ## @param externalAccess.autoDiscovery.image.tag Init container auto-discovery image tag (immutable tags are recommended)
    ## @param externalAccess.autoDiscovery.image.digest Kubectl image digest in the way sha256:aa.... Please note this parameter, if set, will override the tag
    ## @param externalAccess.autoDiscovery.image.pullPolicy Init container auto-discovery image pull policy
    ## @param externalAccess.autoDiscovery.image.pullSecrets Init container auto-discovery image pull secrets
    ##
    image:
      registry: hubtest.xxx.com.cn
      repository: bitnami/kubectl
      tag: 1.25.6-debian-11-r10
      digest: ""
      ## Specify a imagePullPolicy
      ## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
      ## ref: https://kubernetes.io/docs/user-guide/images/#pre-pulling-images
      ##
      pullPolicy: IfNotPresent
      ## Optionally specify an array of imagePullSecrets (secrets must be manually created in the namespace)
      ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
      ## e.g:
      ## pullSecrets:
      ##   - myRegistryKeySecretName
      ##
      pullSecrets: []
    ## Init Container resource requests and limits
    ## ref: https://kubernetes.io/docs/user-guide/compute-resources/
    ## @param externalAccess.autoDiscovery.resources.limits The resources limits for the auto-discovery init container
    ## @param externalAccess.autoDiscovery.resources.requests The requested resources for the auto-discovery init container
    ##
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "1280Mi"
        cpu: "500m"
  ## Parameters to configure K8s service(s) used to externally access Kafka brokers
  ## Note: A new service per broker will be created
  ##
  service:
    ## @param externalAccess.service.type Kubernetes Service type for external access. It can be NodePort, LoadBalancer or ClusterIP
    ##
    type: NodePort
    ## @param externalAccess.service.ports.external Kafka port used for external access when service type is LoadBalancer
    ##
    ports:
      external: 9094
    ## @param externalAccess.service.loadBalancerIPs Array of load balancer IPs for each Kafka broker. Length must be the same as replicaCount
    ## e.g:
    ## loadBalancerIPs:
    ##   - X.X.X.X
    ##   - Y.Y.Y.Y
    ##
    loadBalancerIPs: []
    ## @param externalAccess.service.loadBalancerNames Array of load balancer Names for each Kafka broker. Length must be the same as replicaCount
    ## e.g:
    ## loadBalancerNames:
    ##   - broker1.external.example.com
    ##   - broker2.external.example.com
    ##
    loadBalancerNames: []
    ## @param externalAccess.service.loadBalancerAnnotations Array of load balancer annotations for each Kafka broker. Length must be the same as replicaCount
    ## e.g:
    ## loadBalancerAnnotations:
    ##   - external-dns.alpha.kubernetes.io/hostname: broker1.external.example.com.
    ##   - external-dns.alpha.kubernetes.io/hostname: broker2.external.example.com.
    ##
    loadBalancerAnnotations: []
    ## @param externalAccess.service.loadBalancerSourceRanges Address(es) that are allowed when service is LoadBalancer
    ## ref: https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service
    ## e.g:
    ## loadBalancerSourceRanges:
    ## - 10.10.10.0/24
    ##
    loadBalancerSourceRanges: []
    ## @param externalAccess.service.nodePorts Array of node ports used for each Kafka broker. Length must be the same as replicaCount
    ## e.g:
    ## nodePorts:
    ##   - 30001
    ##   - 30002
    ##
    nodePorts: 
      - 30001
      - 30002
      - 30003
    ## @param externalAccess.service.useHostIPs Use service host IPs to configure Kafka external listener when service type is NodePort
    ##
    useHostIPs: false
    ## @param externalAccess.service.usePodIPs using the MY_POD_IP address for external access.
    ##
    usePodIPs: false
    ## @param externalAccess.service.domain Domain or external ip used to configure Kafka external listener when service type is NodePort or ClusterIP
    ## NodePort: If not specified, the container will try to get the kubernetes node external IP
    ## ClusterIP: Must be specified, ingress IP or domain where tcp for external ports is configured
    ##
    domain: ""
    ## @param externalAccess.service.publishNotReadyAddresses Indicates that any agent which deals with endpoints for this Service should disregard any indications of ready/not-ready
    ## ref: https://kubernetes.io/docs/reference/kubernetes-api/service-resources/service-v1/
    publishNotReadyAddresses: false
    ## @param externalAccess.service.labels Service labels for external access
    ##
    labels: {}
    ## @param externalAccess.service.annotations Service annotations for external access
    ##
    annotations: {}
    ## @param externalAccess.service.extraPorts Extra ports to expose in the Kafka external service
    ##
    extraPorts: []