检查k8s集群pod网络是否异常
网络超时 k8s集群 kafka区块链
·
事由,生产环境的k8s集群故障,orderer通信超时,kafka链接故障。对k8s高可用集群网络进行排查
Overlay 网络上意味着 NGINX Ingress Controller 需要能够将请求从NODE_1路由到NODE_2。如果 Overlay 网络不起作用,可能导致 NGINX Ingress Controller 无法将请求路由到 Pod
一、新建yml文件
vim overlaytest.yml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: overlaytest
spec:
selector:
matchLabels:
name: overlaytest
template:
metadata:
labels:
name: overlaytest
spec:
tolerations:
- operator: Exists
containers:
- image: rancherlabs/swiss-army-knife #无网环境可能会提示无法拉取镜像。可以将导入的镜像上传至harbor,改为harbor镜像路径
imagePullPolicy: Always
name: overlaytest
command: ["sh", "-c", "tail -f /dev/null"]
terminationMessagePath: /dev/termination-log
二、启动
kubectl create -f overlaytest.yml
#查看状态
kubectl rollout status ds/overlaytest -w
返回
daemon set "overlaytest" successfully rolled out
三、编写执行脚本
确保pod正常启动后
vim 1.sh
#!/bin/bash
echo "=> Start network overlay test"
kubectl get pods -l name=overlaytest -o jsonpath='{range .items[*]}{@.metadata.name}{" "}{@.spec.nodeName}{"\n"}{end}' |
while read spod shost
do kubectl get pods -l name=overlaytest -o jsonpath='{range .items[*]}{@.status.podIP}{" "}{@.spec.nodeName}{"\n"}{end}' |
while read tip thost
do kubectl --request-timeout='10s' exec $spod -c overlaytest -- /bin/sh -c "ping -c2 $tip > /dev/null 2>&1"
RC=$?
if [ $RC -ne 0 ]
then echo FAIL: $spod on $shost cannot reach pod IP $tip on $thost
else echo $shost can reach $thost
fi
done
done
echo "=> End network overlay test"
bash 1.sh
返回正确
=> Start network overlay test
xxx cannot reach xxx
xxx cannot reach xxx
xxx cannot reach xxx
…
…
=> End network overlay test
返回错误
=> Start network overlay test
command terminated with exit code 1
xxx cannot reach xxx
…
…
=> End network overlay test
详情可以阅读官方文档
http://docs.rancher.cn/docs/rancher2.5/troubleshooting/networking/_index
更多推荐
已为社区贡献2条内容
所有评论(0)