Kubernetes的API流量查看器-Kubeshark

Kubeshark 由 2021 年 UP9 公司开源的 K8s API 流量查看器 Mizu 发展而来，试图成为一款 K8s 全过程流量监控工具。Kubeshark 被叫做 kubernetes 的 API 流量查看器，它提供对进出 Kubernetes 集群内容器和 pod 的所有 API 流量和负载的深度可见性和监控。类似于针对 Kubernetes 而重新发明的 TCPDump 和 Wir

MyySophia

752人浏览 · 2023-04-07 09:21:07

MyySophia · 2023-04-07 09:21:07 发布

kubeshark-serviceMesh

问了一下chatGPT感觉在胡说

参考

什么是kubeshark

Kubeshark 由 2021 年 UP9 公司开源的 K8s API 流量查看器 Mizu 发展而来，试图成为一款 K8s 全过程流量监控工具。

Kubeshark 被叫做 kubernetes 的 API 流量查看器，它提供对进出 Kubernetes 集群内容器和 pod 的所有 API 流量和负载的深度可见性和监控。类似于针对 Kubernetes 而重新发明的 TCPDump 和 Wireshark。地址：https://github.com/kubeshark/kubeshark

Kubeshark 也被叫做 Kubernetes 的可观测性工具，可以对微服务进行动态分析，检测异常并在运行时出现某些模式时触发功能。

可以将 Kubeshark 视为 Wireshark、BPF 编译器集合 (BCC) 工具等的 Kubernetes 感知组合。
Kubeshark 可以嗅探集群中的部分或所有 TCP 流量，将其记录到 PCAP 文件中并剖析。
Kubeshark 使用 eBPF 来跟踪内核空间和用户空间中的函数调用。

安装kubeshark

安装之前需要对其架构有些了解

一共由三部分组成分别是:

CLI，它是客户端的二进制文件，通过 K8s API 与集群通信。
Hub，它协调 worker 部署，接收来自每个 worker 的嗅探和剖析，并收集到一个中心位置。它还提供一个Web界面，用于在浏览器上显示收集到的流量。
Work，作为 DaemonSet 部署到集群中，以确保集群中的每个节点都被 Kubeshark 覆盖。

启动后三个组件的运行情况，一定要对其限制资源，否则可能会拖垮集群。

启动kubeshark

监控所有namespace的pod流量

CLI启动

$ ./kubeshark.exe --docker-registry 10.50.10.185/kubeshark/docker.io/kubeshark/ --debug tap -A
2023-04-07T08:56:20+08:00 DBG config.go:79 > Init config is finished. config={"Config":{"Regenerate":false},"DumpLogs":false,"HeadlessMode":false,"Kube":{"ConfigPathStr":"","Context":""},"License":"","Logs":{"FileStr":""},"ResourceLabels":{},"Scripting":{"Env":null,"Source":"","WatchScripts":true},"Tap":{"AllNamespaces":true,"Debug":true,"Docker":{"ImagePullPolicy":"Always","ImagePullSecrets":null,"Registry":"10.50.10.185/kubeshark/docker.io/kubeshark/","Tag":"latest"},"DryRun":false,"Namespaces":null,"PacketCapture":"libpcap","Pcap":"","PodRegexStr":".*","Proxy":{"Front":{"DstPort":80,"SrcPort":8899},"Host":"127.0.0.1","Hub":{"DstPort":80,"SrcPort":8898},"Worker":{"DstPort":8897,"SrcPort":8897}},"Resources":{"Hub":{"CpuLimit":"750m","CpuRequests":"50m","MemoryLimit":"1Gi","MemoryRequests":"50Mi"},"Worker":{"CpuLimit":"750m","CpuRequests":"50m","MemoryLimit":"1Gi","MemoryRequests":"50Mi"}},"SelfNamespace":"kubeshark","ServiceMesh":true,"StorageLimit":"200MB","Tls":true}}
2023-04-07T08:56:20+08:00 INF tapRunner.go:53 > Using Docker: registry=10.50.10.185/kubeshark/docker.io/kubeshark/ tag=latest
2023-04-07T08:56:20+08:00 INF tapRunner.go:61 > Kubeshark will store the traffic up to a limit (per node). Oldest TCP streams will be removed once the limit is reached. limit=200MB
2023-04-07T08:56:20+08:00 INF versionCheck.go:23 > Checking for a newer version...
2023-04-07T08:56:20+08:00 DBG provider.go:83 > K8s client config. api-path= host=https://10.50.10.108:6443 user-agent=
2023-04-07T08:56:20+08:00 INF common.go:69 > Using kubeconfig: path="C:\\Users\\00625\\.kube\\config"
2023-04-07T08:56:20+08:00 INF tapRunner.go:82 > Targeting pods in: namespaces=[""]
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: busybox
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: dnsutils
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: myapp-6c84fdb964-q9jgg
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: myapp-6c84fdb964-zx8sq
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: nfs-deploy-187-sc-77d958d57d-7k82m
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: nginx-web-2ztjg
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: nginx-web-jl78g
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: rpt-etl-loadmesdefectdata-meqmsgp-final-deployment-54b557fzzlzc
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: tea-797c7b5d89-26clj
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: tea-797c7b5d89-h7m6j
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: tea-797c7b5d89-xv8qq
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: configmap-demo-pod
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: etl-loadedcdata-oc-test-deployment-765bdcbb57-jqr96
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: nginx-deployment-776bf658dc-5qmwv
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: nginx-deployment-776bf658dc-dpl77
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: nginx-deployment-776bf658dc-vgwmq
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: nginx-web-bcrvx
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: nginx-web-tb9b9
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: pc-deployment-5dbbbc49b9-pdnkn
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: pc-deployment-5dbbbc49b9-sjthh
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: pod-configmap
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: pod-liveness-httpget
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: qms-deploy-7f5ccdcc8b-dhn6g
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: tomcat-deployment-869c6bb5d-b9vgp
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: tomcat-deployment-869c6bb5d-ckptc
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: tomcat-deployment-869c6bb5d-ngd2k
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: web-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: web-1
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: cloud-eureka-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: db-example-7f5b967b5b-hcl7b
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: gateway-example-97868d74-mpxwl
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: svc-example-57c4bcd684-nsrrg
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: web-example-74cf6f548c-njlnm
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-gitlab-runner-minio-gitlab-runner-75f87cfdbf-d89z6
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: gitlab-8d977cc89-bj76w
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: gitlib-db-postgresql-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: gitlib-redis-node-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: ingress-nginx-controller-7466f577fb-kd6r8
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: zentao-9d66c8d6c-4fkwv
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kafka-client
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kafka-cluster-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: zookeeper-cluster-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: calico-kube-controllers-7ff9f9d7ff-xw775
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: calico-node-45t6m
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: calico-node-8qxkq
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: calico-node-9twcw
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: calico-node-bfntl
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: calico-node-dwxj9
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: calico-node-ms4pr
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: coredns-66dbf4b6cd-2zk95
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: coredns-66dbf4b6cd-qlzbk
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: coredns-66dbf4b6cd-qmjcn
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: eip-nfs-chot-nas-500gb-6bbfddcc9b-mt9jj
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: dashboard-metrics-scraper-549b9db55d-wc7m8
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kubernetes-dashboard-6c57ff6878-45czg
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-loki-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-loki-grafana-54fc8c475-qrh2g
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-promtail-b8clp
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-promtail-dkj42
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-promtail-gl9xm
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-promtail-h7m8h
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-promtail-znr8g
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-pv-browser-bmrd4
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-pv-browser-fxlfz
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-pv-browser-g8fw5
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-pv-browser-pnxjb
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: kuboard-pv-browser-vpggp
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-minio-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-minio-1
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-minio-2
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-minio-3
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: mysql-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-nexus-nexus-repository-manager-6595d7c79b-q7znf
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: nfs-client-provisioner-sc179-8447d45564-tfxfd
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: alertmanager-chot-prometheus-kube-prome-alertmanager-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-prometheus-grafana-7fccf797f5-jrqsx
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-prometheus-kube-prome-operator-5754f74d75-lfngs
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-prometheus-kube-state-metrics-89975d948-r6dx6
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: prometheus-chot-prometheus-kube-prome-prometheus-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-rabbitmq-dev-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-rabbitmq-dev-1
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-rabbitmq-0
2023-04-07T08:56:21+08:00 INF tapRunner.go:142 > Targeted pod: chot-rabbitmq-1
2023-04-07T08:56:21+08:00 INF tapRunner.go:92 > Waiting for the creation of Kubeshark resources...
2023-04-07T08:56:21+08:00 INF createResources.go:111 > Successfully created a pod. pod=kubeshark-hub
2023-04-07T08:56:21+08:00 INF createResources.go:123 > Successfully created a pod. pod=kubeshark-front
2023-04-07T08:56:21+08:00 INF createResources.go:72 > Successfully created a service. service=kubeshark-hub
2023-04-07T08:56:21+08:00 INF createResources.go:79 > Successfully created a service. service=kubeshark-front
2023-04-07T08:56:21+08:00 DBG wait.go:13 > Waiting to finish...
2023-04-07T08:56:21+08:00 INF tapRunner.go:264 > Added: pod=kubeshark-front
2023-04-07T08:56:21+08:00 INF tapRunner.go:173 > Added: pod=kubeshark-hub
2023-04-07T08:56:21+08:00 DBG tapRunner.go:281 > Watching pod. containers-statuses=[{"image":"10.50.10.185/kubeshark/docker.io/kubeshark/front:latest","imageID":"","lastState":{},"name":"kubeshark-front","ready":false,"restartCount":0,"started":false,"state":{"waiting":{"reason":"ContainerCreating"}}}] phase=Pending pod=kubeshark-front
2023-04-07T08:56:21+08:00 DBG versionCheck.go:38 > Fetched the latest release: elapsed-time=435.195 local-version=39.5 upstream-version=39.5
2023-04-07T08:56:22+08:00 DBG tapRunner.go:281 > Watching pod. containers-statuses=[{"image":"10.50.10.185/kubeshark/docker.io/kubeshark/front:latest","imageID":"","lastState":{},"name":"kubeshark-front","ready":false,"restartCount":0,"started":false,"state":{"waiting":{"reason":"ContainerCreating"}}}] phase=Pending pod=kubeshark-front
2023-04-07T08:56:22+08:00 DBG tapRunner.go:190 > Watching pod. containers-statuses=[{"image":"10.50.10.185/kubeshark/docker.io/kubeshark/hub:latest","imageID":"","lastState":{},"name":"kubeshark-hub","ready":false,"restartCount":0,"started":false,"state":{"waiting":{"reason":"ContainerCreating"}}}] phase=Pending pod=kubeshark-hub
2023-04-07T08:56:22+08:00 DBG tapRunner.go:369 > Watching events. event=kubeshark-hub.1753810db14a1df9 kind=Pod name=kubeshark-hub note="Pulling image \"10.50.10.185/kubeshark/docker.io/kubeshark/hub:latest\"" pod=kubeshark-hub reason=Pulling
2023-04-07T08:56:22+08:00 DBG tapRunner.go:369 > Watching events. event=kubeshark-hub.1753810dbb96fbca kind=Pod name=kubeshark-hub note="Successfully pulled image \"10.50.10.185/kubeshark/docker.io/kubeshark/hub:latest\" in 172.792597ms" pod=kubeshark-hub reason=Pulled
2023-04-07T08:56:22+08:00 DBG tapRunner.go:369 > Watching events. event=kubeshark-hub.1753810dbd12f28b kind=Pod name=kubeshark-hub note="Created container kubeshark-hub" pod=kubeshark-hub reason=Created
2023-04-07T08:56:22+08:00 DBG tapRunner.go:369 > Watching events. event=kubeshark-hub.1753810dc34b23fe kind=Pod name=kubeshark-hub note="Started container kubeshark-hub" pod=kubeshark-hub reason=Started
2023-04-07T08:56:23+08:00 DBG tapRunner.go:281 > Watching pod. containers-statuses=[{"containerID":"docker://05b0472bc9b3ebbed8df9c299a48b04221bfba87e09e88a2f5374e86db887d40","image":"10.50.10.185/kubeshark/docker.io/kubeshark/front:latest","imageID":"docker-pullable://10.50.10.185/kubeshark/docker.io/kubeshark/front@sha256:80a9dee0a3e473d960f90230862767012ba3b32f8ee13b322ee59f35b3eddbb0","lastState":{},"name":"kubeshark-front","ready":false,"restartCount":0,"started":true,"state":{"running":{"startedAt":"2023-04-07T00:56:21Z"}}}] phase=Running pod=kubeshark-front
2023-04-07T08:56:23+08:00 DBG tapRunner.go:281 > Watching pod. containers-statuses=[{"containerID":"docker://05b0472bc9b3ebbed8df9c299a48b04221bfba87e09e88a2f5374e86db887d40","image":"10.50.10.185/kubeshark/docker.io/kubeshark/front:latest","imageID":"docker-pullable://10.50.10.185/kubeshark/docker.io/kubeshark/front@sha256:80a9dee0a3e473d960f90230862767012ba3b32f8ee13b322ee59f35b3eddbb0","lastState":{},"name":"kubeshark-front","ready":true,"restartCount":0,"started":true,"state":{"running":{"startedAt":"2023-04-07T00:56:21Z"}}}] phase=Running pod=kubeshark-front
2023-04-07T08:56:23+08:00 DBG tapRunner.go:190 > Watching pod. containers-statuses=[{"containerID":"docker://5872d7590936c6b43c2b06cd47a027651e6b6ae14e61d488bb8af4bd40b6bc3d","image":"10.50.10.185/kubeshark/docker.io/kubeshark/hub:latest","imageID":"docker-pullable://10.50.10.185/kubeshark/docker.io/kubeshark/hub@sha256:de70ee57cd66b6225e43d8500ef95e4f8ef2cd5e8ad7646a11220c87d83ed145","lastState":{},"name":"kubeshark-hub","ready":true,"restartCount":0,"started":true,"state":{"running":{"startedAt":"2023-04-07T00:56:22Z"}}}] phase=Running pod=kubeshark-hub
2023-04-07T08:56:23+08:00 INF proxy.go:29 > Starting proxy... namespace=kubeshark service=kubeshark-hub src-port=8898
2023-04-07T08:56:23+08:00 DBG hub.go:45 > Connection test passed successfully. url=http://localhost:8898
2023-04-07T08:56:23+08:00 INF workers.go:33 > Creating the worker DaemonSet...
2023-04-07T08:56:23+08:00 DBG provider.go:648 > Applying worker DaemonSets. daemonset-name=kubeshark-worker-daemon-set image=10.50.10.185/kubeshark/docker.io/kubeshark/worker:latest namespace=kubeshark pod=kubeshark-worker
2023-04-07T08:56:23+08:00 INF workers.go:52 > Successfully created the worker DaemonSet.
2023-04-07T08:56:23+08:00 INF tapRunner.go:436 > Waiting for worker containers...
2023-04-07T08:56:29+08:00 DBG hub.go:112 > Reported storage limit to Hub: limit=200000000
2023-04-07T08:56:29+08:00 DBG hub.go:145 > Reported pod regex to Hub: namespaces=[""] regex=.*
2023-04-07T08:56:29+08:00 DBG hub.go:344 > Reported POST scripts done to Hub.
2023-04-07T08:56:29+08:00 INF tapRunner.go:471 > Hub is available at: url=http://localhost:8898
2023-04-07T08:56:29+08:00 INF proxy.go:29 > Starting proxy... namespace=kubeshark service=kubeshark-front src-port=8899
2023-04-07T08:56:29+08:00 DBG hub.go:45 > Connection test passed successfully. url=http://localhost:8899
2023-04-07T08:56:29+08:00 INF tapRunner.go:492 > Kubeshark is available at: url=http://localhost:8899

前台展示

过滤出失败的API 调用

http and response.status >= 400

kubeshark-serviceMesh

部署完成后，Kubeshark CLI 将在 http://localhost:8899 打开 UI 单击右上角名为 Service Map 的按钮打开服务依赖关系图。该图根据网络流量显示 Pod 以及它们之间的关系。

问了一下chatGPT感觉在胡说

参考

The Kubernetes Network Analyzer - Kubeshark

一文解决Kubernetes的API流量查看器-Kubeshark - 知乎

K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 | 韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。长夜有穷，真水无香。领略过了 K8s“身在江

K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub