Istio 从网关访问服务错误 503 NC cluster_not_found
背景:是这样的,我使用默认的 httpbin gateway 来测试一些服务,但是使用之后从网关访问不符合我的预期,所以简单看了一下原因。这里的 yaml 是官方提供的(我小改了一些修改了 vs 的范围,以及添加了 host)apiVersion: networking.istio.io/v1alpha3kind: Gatewaymetadata:name: httpbin-gatewayspec
背景:是这样的,我使用默认的 httpbin gateway 来测试一些服务,但是使用之后从网关访问不符合我的预期,所以简单看了一下原因。
这里的 yaml 是官方提供的(我小改了一些修改了 vs 的范围,以及添加了 host)
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: httpbin-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "httpbin-arm.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- "httpbin-arm.com"
- "mesh"
gateways:
- httpbin-gateway
http:
- route:
- destination:
host: httpbin.book-system
port:
number: 8000
执行后,从网关访问没有任何输出
curl -v -H "Host: httpbin-arm.com" http://192.168.2.240:32445
查看网关的日志发现错误
[2022-05-06T07:07:07.890Z] "GET / HTTP/1.1" 503 NC cluster_not_found - "-" 0 0 0 - "192.168.2.240" "curl/7.66.0" "b05d1bcb-f67c-9f17-aa91-587fd8f50e0c" "httpbin-arm.com" "-" - - 10.250.0.193:8080 192.168.2.240:6117 - -
[2022-05-06T07:09:13.744Z] "GET / HTTP/1.1" 503 NC cluster_not_found - "-" 0 0 0 - "192.168.2.240" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15" "2945db3e-928e-96a1-a4cf-2f871410a6db" "httpbin-arm.com:32445" "-" - - 10.250.0.193:8080 192.168.2.240:9720 - -
[2022-05-06T07:09:13.856Z] "GET /favicon.ico HTTP/1.1" 503 NC cluster_not_found - "-" 0 0 0 - "192.168.2.240" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15" "c932aa92-84bb-9de0-ba96-46b974d8edea" "httpbin-arm.com:32445" "-" - - 10.250.0.193:8080 192.168.2.240:48728 - -
[2022-05-06T07:09:21.786Z] "GET / HTTP/1.1" 503 NC cluster_not_found - "-" 0 0 0 - "192.168.2.240" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15" "8582ad7b-6cdc-9f23-9254-ee7a11b46d5e" "httpbin-arm.com:32445" "-" - - 10.250.0.193:8080 192.168.2.240:59931 - -
[2022-05-06T07:09:29.161Z] "GET / HTTP/1.1" 503 NC cluster_not_found - "-" 0 0 0 - "192.168.2.240" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15" "29ce549d-38c1-9aa7-9864-946a2c9b354e" "httpbin-arm.com:32445" "-" - - 10.250.0.193:8080 192.168.2.240:59931 - -
[2022-05-06T07:09:56.839Z] "GET / HTTP/1.1" 503 NC cluster_not_found - "-" 0 0 0 - "192.168.2.240" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15" "29b737ab-0ac1-96fd-aff5-ef3283e8ff68" "httpbin-arm.com:32445" "-" - - 10.250.0.193:8080 192.168.2.240:59931 - -
有错误就比较简单了,我们接下来就是寻找为什么会出现这个错误
1: 查看网关的 route 配置
从下面返回的结果我们就可以发现,httpbin-arm.com
host 路由到了 outbound|8000||httpbin.book-system
cluster,这个 cluster 是我们在 vs 中使用的。并没有直接使用 destinationrule 来定义 cluster。
./istioctl pc route istio-ingressgateway-667f585f87-75dxv.istio-system -oyaml
- name: http.80
validateClusters: false
virtualHosts:
- domains:
- httpbin-arm.com
- httpbin-arm.com:*
includeRequestAttemptCount: true
name: httpbin-arm.com:80
routes:
- decorator:
operation: httpbin.book-system:8000/*
match:
prefix: /
metadata:
filterMetadata:
istio:
config: /apis/networking.istio.io/v1alpha3/namespaces/book-system/virtual-service/httpbin
route:
cluster: outbound|8000||httpbin.book-system
maxGrpcTimeout: 0s
retryPolicy:
hostSelectionRetryMaxAttempts: "5"
numRetries: 2
retriableStatusCodes:
- 503
retryHostPredicate:
- name: envoy.retry_host_predicates.previous_hosts
retryOn: connect-failure,refused-stream,unavailable,cancelled,retriable-status-codes
timeout: 0s
2: 查看网关的 cluster 配置
以下是网关的部分 cluster 配置。
从输出的内容可以看到,httpbin 的 cluster 只有一个并且叫做 outbound|8000||httpbin.book-system.svc.cluster.local
./istioctl pc cluster istio-ingressgateway-667f585f87-75dxv.istio-system -oyaml
.....
filters:
- name: istio.metadata_exchange
typedConfig:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
typeUrl: type.googleapis.com/envoy.tcp.metadataexchange.config.MetadataExchange
value:
protocol: istio-peer-exchange
metadata:
filterMetadata:
istio:
default_original_port: 8000
services:
- host: httpbin.book-system.svc.cluster.local
name: httpbin
namespace: book-system
name: outbound|8000||httpbin.book-system.svc.cluster.local
transportSocketMatches:
- match:
tlsMode: istio
name: tlsMode-istio
....
结论:我们在 vs 中使用的 destination
host 是 httpbin.book-system
,所以网关去找一个叫做 outbound|8000||httpbin.book-system
的 cluster,但是实际底层的 httpbin destinationrule 是 fqdn 配置即:全域名 name.namespaces.svc.cluster.local
所以就没有找到 cluster 导致初选了错误。
两种修复方法
- 更新 vs 中 host 的配置,更新成 fqdn 即可
- 使用 destinationrule 来定义 httpbin cluster,然后配置 vs subset
这两种方法都可以解决问题。
更多推荐
所有评论(0)