Tungsten Fabri(4):TOR模式对接vMX

参考:http://nakadaisuke.blogspot.com/2018/08/how-to-configure-contrailtungstenfabric.html准备工作基于EVE-NG环境通过deployer部署TF&k8s集成环境(master01/node01/node02)deployer主机作为一台裸金属服务器在每个node和裸金属服务器上都有一台vM...

ljyfree

431人浏览 · 2020-04-08 16:41:50

ljyfree · 2020-04-08 16:41:50 发布

参考:http://nakadaisuke.blogspot.com/2018/08/how-to-configure-contrailtungstenfabric.html

准备工作

基于EVE-NG环境
通过deployer部署TF&k8s集成环境(master01/node01/node02)
deployer主机作为一台裸金属服务器
在每个node和裸金属服务器上都有一台vMX做TOR
vMX-1的lo0的IP为1.1.1.1，以此类推到vMX-2/vMX-3，lo0作为管理IP和VTEP IP
master01和vMX-x的ge-0/0/1的接口IP在同网段
master01上添加静态路由到各台vMX的lo0的IP
vMX设置用户名/密码，开启netconf（参考这里）

TF的WebUI上的配置

下面以vMX-1为例，对于vMX-2/vMX-3都是如法炮制。

注册Leaf交换机
Configure->Physical Devices->Physical Routers,添加NetConf Management Physical Router
配置BGP Router
Configure > Infrastructure > BGP Routers
配置VTEP信息
Configure > Physical Devices > Physical Routers

检查

安装参考文章，此时部署在master01上的contrail应该通过netconf对三台Leaf交换机下发配置"group __contrail__"
但是检查后发现，vMX上并没有新配置下发
实际上在WebUI做添加操作的过程中，一直在master01的接口上针对830端口进行抓包，并没有报文交互
结论：验证失败，控制器没有通过netconf向vMX下发配置

记录一些信息

device-manager-zk.log

[root@master01 contrail]# cat /var/log/contrail/device-manager-zk.log   
04/07/2020 03:02:38 AM [device-manager]: Connecting to 192.168.122.177:2181
04/07/2020 03:02:38 AM [device-manager]: Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=400000, session_id=0, passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None)
04/07/2020 03:02:39 AM [device-manager]: Zookeeper connection established, state: CONNECTED
04/07/2020 03:02:50 AM [device-manager]: Sending request(xid=1): Exists(path='/device-manager', watcher=None)
04/07/2020 03:02:50 AM [device-manager]: Sending request(xid=2): Exists(path='/', watcher=None)
04/07/2020 03:02:50 AM [device-manager]: Received response(xid=2): ZnodeStat(czxid=0, mzxid=0, ctime=0, mtime=0, version=0, cversion=4, aversion=0, ephemeralOwner=0, dataLength=0, numChildren=6, pzxid=19)
04/07/2020 03:02:50 AM [device-manager]: Sending request(xid=3): Create(path='/device-manager', data='', acl=[ACL(perms=31, acl_list=['ALL'], id=Id(scheme='world', id='anyone'))], flags=0)
04/07/2020 03:02:50 AM [device-manager]: Received response(xid=3): u'/device-manager'
04/07/2020 03:02:50 AM [device-manager]: Sending request(xid=4): Create(path='/device-manager/2f94ac307252466999db07d2f40c9945__lock__', data='1', acl=[ACL(perms=31, acl_list=['ALL'], id=Id(scheme='world', id='anyone'))], flags=3)
04/07/2020 03:02:50 AM [device-manager]: Received response(xid=4): u'/device-manager/2f94ac307252466999db07d2f40c9945__lock__0000000000'
04/07/2020 03:02:50 AM [device-manager]: Sending request(xid=5): GetChildren(path='/device-manager', watcher=None)
04/07/2020 03:02:50 AM [device-manager]: Received response(xid=5): [u'2f94ac307252466999db07d2f40c9945__lock__0000000000']
04/07/2020 03:04:21 AM [device-manager]: Connection dropped: socket connection broken
04/07/2020 03:04:21 AM [device-manager]: Transition to CONNECTING
04/07/2020 03:04:21 AM [device-manager]: Zookeeper connection lost
04/07/2020 03:04:22 AM [device-manager]: Connecting to 192.168.122.177:2181
04/07/2020 03:04:22 AM [device-manager]: Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=59, time_out=400000, session_id=72058492531376131, passwd='Y\xae\xd2C\xe9>\xd5G5\x9b;\x1e\xa7\x138\xb1', read_only=None)
04/07/2020 03:04:22 AM [device-manager]: Session has expired
04/07/2020 03:04:22 AM [device-manager]: Zookeeper session lost, state: EXPIRED_SESSION
04/07/2020 03:04:37 AM [device-manager]: Connecting to 192.168.122.177:2181
04/07/2020 03:04:37 AM [device-manager]: Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=400000, session_id=0, passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None)
04/07/2020 03:04:37 AM [device-manager]: Zookeeper connection established, state: CONNECTED
04/07/2020 03:26:32 AM [device-manager]: Connecting to 192.168.122.177:2181
04/07/2020 03:26:32 AM [device-manager]: Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=400000, session_id=0, passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None)
04/07/2020 03:26:32 AM [device-manager]: Zookeeper connection established, state: CONNECTED
04/07/2020 03:26:35 AM [device-manager]: Sending request(xid=1): Exists(path='/device-manager', watcher=None)
04/07/2020 03:26:35 AM [device-manager]: Received response(xid=1): ZnodeStat(czxid=58, mzxid=58, ctime=1586242970680, mtime=1586242970680, version=0, cversion=2, aversion=0, ephemeralOwner=0, dataLength=0, numChildren=0, pzxid=235)
04/07/2020 03:26:35 AM [device-manager]: Sending request(xid=2): Create(path='/device-manager/db86b0955a8c42fbbb66bdf56e730d5d__lock__', data='1', acl=[ACL(perms=31, acl_list=['ALL'], id=Id(scheme='world', id='anyone'))], flags=3)
04/07/2020 03:26:35 AM [device-manager]: Received response(xid=2): u'/device-manager/db86b0955a8c42fbbb66bdf56e730d5d__lock__0000000001'
04/07/2020 03:26:35 AM [device-manager]: Sending request(xid=3): GetChildren(path='/device-manager', watcher=None)
04/07/2020 03:26:35 AM [device-manager]: Received response(xid=3): [u'db86b0955a8c42fbbb66bdf56e730d5d__lock__0000000001']
04/07/2020 03:28:18 AM [device-manager]: Sending request(xid=4): GetChildren(path='/id/pnf/network_id', watcher=None)
04/07/2020 03:28:18 AM [device-manager]: Received error(xid=4) NoNodeError()
[root@master01 contrail]#

contrail-device-manager.log

04/07/2020 03:02:54 AM [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = master01.local  process_status = [ <<  module_id = contrail-device-manager  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 192.168.122.177:2181,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 192.168.122.177:9161,  ]  status = Up  description =   >>, <<  type = ApiServer  name = ApiServer  server_addrs = [ 192.168.122.177:8082,  ]  status = Down  description = ApiServer Connection State updated  >>, <<  type = Collector  name = Collector  server_addrs = [ 192.168.122.177:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 192.168.122.177:5673,  ]  status = Up  description =   >>,  ]  description = ApiServer:ApiServer[ApiServer Connection State updated], Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
04/07/2020 03:02:58 AM [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail
...
 /usr/lib/python2.7/site-packages/vnc_api/vnc_api.py in crud(self=<vnc_api.vnc_api.ApiServerSession object>, method='post', url=u'http://192.168.122.177:8082/feature-configs', *args=(), **kwargs={'data': u'{"feature-config":{"parent_type": "role-defini...nt-leaf", "assisted-replicator"], "uuid": null}}', 'headers': {'Content-type': 'application/json; charset="UTF-8"', 'X-Contrail-Useragent': 'master01:/usr/bin/contrail-device-manager'}, 'timeout': 5})
  230                     self.logger.log(op=method, url=url,
  231                             data=data, headers=headers)
  232                 result = crud_method(url, *args, **kwargs)
  233                 if self.logger:
  234                     self.logger.log_response(result)
result undefined
crud_method = <bound method Session.post of <requests.sessions.Session object>>
url = u'http://192.168.122.177:8082/feature-configs'
args = ()
kwargs = {'data': u'{"feature-config":{"parent_type": "role-defini...nt-leaf", "assisted-replicator"], "uuid": null}}', 'headers': {'Content-type': 'application/json; charset="UTF-8"', 'X-Contrail-Useragent': 'master01:/usr/bin/contrail-device-manager'}, 'timeout': 5}
...

04/07/2020 10:33:18 PM [contrail-device-manager] [WARNING]: It is not recommended to use 'get' or 'multiget' pycassa methods. It's better to use 'xget' or 'get_range' methods due to thrift limitations
04/07/2020 10:33:19 PM [contrail-device-manager] [WARNING]: It is not recommended to use 'get' or 'multiget' pycassa methods. It's better to use 'xget' or 'get_range' methods due to thrift limitations
04/07/2020 10:33:19 PM [contrail-device-manager] [WARNING]: No features found for vMX-3(b476a04c-8d35-479c-bb71-ef63965519de)
04/07/2020 10:33:19 PM [contrail-device-manager] [WARNING]: Could not push config to the device: b476a04c-8d35-479c-bb71-ef63965519de, vMX-3; Fabric Object not yet available for this Physical Router
04/07/2020 10:33:19 PM [contrail-device-manager] [WARNING]: It is not recommended to use 'get' or 'multiget' pycassa methods. It's better to use 'xget' or 'get_range' methods due to thrift limitations
04/07/2020 10:33:19 PM [contrail-device-manager] [ERROR]: Exception: argument of type 'NoneType' is not iterableTraceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/device_manager/db.py", line 670, in nc_handler
    self.push_config()
  File "/usr/lib/python2.7/site-packages/device_manager/db.py", line 1015, in push_config
    feature_configs=feature_configs)
  File "/usr/lib/python2.7/site-packages/device_manager/overlay_conf.py", line 38, in push_conf
    self.set_common_config()
  File "/usr/lib/python2.7/site-packages/device_manager/ansible_role_common.py", line 1478, in set_common_config
    self.build_bgp_config()
  File "/usr/lib/python2.7/site-packages/device_manager/ansible_conf.py", line 574, in build_bgp_config
    peer.params, attr, external, peer)
  File "/usr/lib/python2.7/site-packages/device_manager/ansible_conf.py", line 497, in add_bgp_peer
    if peer_pr and "Route-Reflector" in peer_pr.routing_bridging_roles\
TypeError: argument of type 'NoneType' is not iterable

记录当前状态

[root@master01 ~]# pip install lxml prettytable
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Looking in indexes: https://mirrors.aliyun.com/pypi/simple
Requirement already satisfied: lxml in /usr/lib64/python2.7/site-packages (4.5.0)
Collecting prettytable
  Downloading https://mirrors.aliyun.com/pypi/packages/ef/30/4b0746848746ed5941f052479e7c23d2b56d174b82f4fd34a25e389831f5/prettytable-0.7.2.tar.bz2 (21 kB)
Installing collected packages: prettytable
    Running setup.py install for prettytable ... done
Successfully installed prettytable-0.7.2
[root@master01 ~]# 
[root@master01 ~]# curl -O https://raw.githubusercontent.com/vcheny/contrail-introspect-cli/master/ist.py
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 80020  100 80020    0     0   5555      0  0:00:14  0:00:14 --:--:-- 15885
[root@master01 ~]# ist.py ctr status
-bash: ist.py: command not found
[root@master01 ~]python ist.py ctr status
module_id: contrail-control
state: Functional
description
+-----------+-----------+------------------------+--------+----------------------------------+
| type      | name      | server_addrs           | status | description                      |
+-----------+-----------+------------------------+--------+----------------------------------+
| Collector | n/a       |   192.168.122.177:8086 | Up     | Established                      |
| Database  | Cassandra |   192.168.122.177:9041 | Up     | Established Cassandra connection |
| Database  | RabbitMQ  |   192.168.122.177:5673 | Up     | RabbitMQ connection established  |
+-----------+-----------+------------------------+--------+----------------------------------+
[root@master01 ~]# 


[root@master01 ~]# python ist.py ctr nei     
+--------------+-----------------+----------+----------+-----------+-------------+------------+------------+-----------------------------+
| peer         | peer_address    | peer_asn | encoding | peer_type | state       | send_state | flap_count | flap_time                   |
+--------------+-----------------+----------+----------+-----------+-------------+------------+------------+-----------------------------+
| vMX-1        | 1.1.1.1         | 64512    | BGP      | internal  | Established | in sync    | 0          | n/a                         |
| vMX-2        | 2.2.2.2         | 64512    | BGP      | internal  | Established | in sync    | 0          | n/a                         |
| vMX-3        | 3.3.3.3         | 64512    | BGP      | internal  | Established | in sync    | 0          | n/a                         |
| node01.local | 192.168.122.104 | 0        | XMPP     | internal  | Established | in sync    | 1          | 2020-Apr-07 09:20:22.750483 |
| node02.local | 192.168.122.31  | 0        | XMPP     | internal  | Established | in sync    | 1          | 2020-Apr-07 09:19:56.687904 |
+--------------+-----------------+----------+----------+-----------+-------------+------------+------------+-----------------------------+
[root@master01 ~]# 

[root@master01 ~]# python ist.py ctr route tables  
name: default-domain:default-project:__link_local__:__link_local__.inet.0
name: default-domain:default-project:dci-network:__default__.inet.0
name: default-domain:default-project:dci-network:dci-network.inet.0
name: default-domain:default-project:default-virtual-network:default-virtual-network.inet.0
name: inet.0
name: default-domain:default-project:ip-fabric:ip-fabric.inet.0
name: default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network.inet.0
name: default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network.inet.0
name: default-domain:k8s-test-ns1:k8s-ns1-pod-net01:k8s-ns1-pod-net01.inet.0
name: default-domain:k8s-test-ns1:k8s-ns1-pod-net02:k8s-ns1-pod-net02.inet.0
[root@master01 ~]# 

[root@master01 ~]# python ist.py ctr route show

bgp.ermvpn.0: 10 destinations, 10 routes (0 primary, 10 secondary, 0 infeasible)

1-192.168.122.104:1-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:14:25.307309, last_modified: 2020-Apr-07 09:32:27.422957
    [Local|None] age: 1 day 16:14:25.351363, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

1-192.168.122.104:2-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:12:32.102169, last_modified: 2020-Apr-07 09:34:20.628097
    [Local|None] age: 1 day 16:12:32.147148, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

1-192.168.122.104:3-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:12:32.107789, last_modified: 2020-Apr-07 09:34:20.622477
    [Local|None] age: 1 day 16:12:32.155649, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

1-192.168.122.104:4-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:28.032166, last_modified: 2020-Apr-07 09:25:24.698100
    [Local|None] age: 1 day 16:21:28.082375, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

1-192.168.122.104:5-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:22.776488, last_modified: 2020-Apr-07 09:25:29.953778
    [Local|None] age: 1 day 16:21:22.828326, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

2-192.168.122.104:1-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:27.999505, last_modified: 2020-Apr-07 09:25:24.730761
    [Local|None] age: 1 day 16:21:28.052441, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

2-192.168.122.104:2-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:28.049753, last_modified: 2020-Apr-07 09:25:24.680513
    [Local|None] age: 1 day 16:21:28.104431, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

2-192.168.122.104:3-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:28.041954, last_modified: 2020-Apr-07 09:25:24.688312
    [Local|None] age: 1 day 16:21:28.097763, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

2-192.168.122.104:4-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:28.031771, last_modified: 2020-Apr-07 09:25:24.698495
    [Local|None] age: 1 day 16:21:28.095397, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

2-192.168.122.104:5-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:22.775790, last_modified: 2020-Apr-07 09:25:29.954476
    [Local|None] age: 1 day 16:21:22.840033, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

bgp.evpn.0: 6 destinations, 6 routes (0 primary, 6 secondary, 0 infeasible)

2-192.168.122.31:1-0-00:50:00:00:02:00,0.0.0.0, age: 1 day 16:14:25.354883, last_modified: 2020-Apr-07 09:32:27.375383
    [XMPP|node02.local] age: 1 day 16:14:25.420034, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None

2-192.168.122.31:1-0-00:50:00:00:02:00,192.168.122.31, age: 1 day 16:14:25.319118, last_modified: 2020-Apr-07 09:32:27.411148
    [XMPP|node02.local] age: 1 day 16:14:25.384883, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None

2-192.168.122.104:1-0-00:50:00:00:03:00,0.0.0.0, age: 1 day 16:21:27.999176, last_modified: 2020-Apr-07 09:25:24.731090
    [XMPP|node01.local] age: 1 day 16:21:28.065414, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None

2-192.168.122.104:1-0-00:50:00:00:03:00,192.168.122.104, age: 1 day 16:21:27.999121, last_modified: 2020-Apr-07 09:25:24.731145
    [XMPP|node01.local] age: 1 day 16:21:28.065956, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None

3-192.168.122.31:1-2-192.168.122.31, age: 1 day 16:14:25.308468, last_modified: 2020-Apr-07 09:32:27.421798
    [Local|None] age: 1 day 16:14:25.375908, localpref: 100, nh: 192.168.122.31, encap: ['vxlan'], label: 2, AS path: None

3-192.168.122.104:1-2-192.168.122.104, age: 1 day 16:21:28.009947, last_modified: 2020-Apr-07 09:25:24.720319
    [Local|None] age: 1 day 16:21:28.078199, localpref: 100, nh: 192.168.122.104, encap: ['vxlan'], label: 2, AS path: None

bgp.l3vpn.0: 7 destinations, 7 routes (0 primary, 7 secondary, 0 infeasible)

192.168.122.31:1:192.168.122.31/32, age: 1 day 16:14:25.318958, last_modified: 2020-Apr-07 09:32:27.411308
    [XMPP (interface)|node02.local] age: 1 day 16:14:25.388152, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'udp', 'native'], label: 16, AS path: None

192.168.122.31:3:10.10.20.1/32, age: 1 day 16:12:31.896036, last_modified: 2020-Apr-07 09:34:20.834230
    [XMPP (interface)|node02.local] age: 1 day 16:12:31.965847, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'udp'], label: 27, AS path: None

192.168.122.104:1:192.168.122.104/32, age: 1 day 16:21:27.998632, last_modified: 2020-Apr-07 09:25:24.731634
    [XMPP (interface)|node01.local] age: 1 day 16:21:28.069228, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp', 'native'], label: 16, AS path: None

192.168.122.104:3:10.10.10.1/32, age: 1 day 16:21:27.665918, last_modified: 2020-Apr-07 09:25:25.064348
    [XMPP (interface)|node01.local] age: 1 day 16:21:27.737561, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 39, AS path: None

192.168.122.104:4:10.47.255.249/32, age: 1 day 16:21:22.758790, last_modified: 2020-Apr-07 09:25:29.971476
    [XMPP (interface)|node01.local] age: 1 day 16:21:22.830930, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 29, AS path: None

192.168.122.104:4:10.47.255.250/32, age: 1 day 16:21:19.131104, last_modified: 2020-Apr-07 09:25:33.599162
    [XMPP (interface)|node01.local] age: 1 day 16:21:19.203858, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 34, AS path: None

192.168.122.104:5:10.96.0.10/32, age: 1 day 16:21:19.122871, last_modified: 2020-Apr-07 09:25:33.607395
    [XMPP (interface)|node01.local] age: 1 day 16:21:19.196802, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 46, AS path: None

bgp.rtarget.0: 13 destinations, 23 routes (23 primary, 0 secondary, 0 infeasible)

64512:target:64512:7999999, age: 1 day 16:14:25.526767, last_modified: 2020-Apr-07 09:32:27.203499
    [XMPP|node02.local] age: 1 day 16:14:25.601843, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None
    [XMPP|node01.local] age: 1 day 16:21:28.235319, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:64512:8000002, age: 1 day 16:14:25.525796, last_modified: 2020-Apr-07 09:32:27.204470
    [XMPP|node02.local] age: 1 day 16:14:25.601631, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None
    [XMPP|node01.local] age: 1 day 16:21:28.169681, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:64512:8000004, age: 1 day 16:14:25.525778, last_modified: 2020-Apr-07 09:32:27.204488
    [XMPP|node02.local] age: 1 day 16:14:25.602631, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None
    [XMPP|node01.local] age: 1 day 16:21:28.171357, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:64512:8000005, age: 1 day 16:14:25.525774, last_modified: 2020-Apr-07 09:32:27.204492
    [XMPP|node02.local] age: 1 day 16:14:25.604275, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None
    [XMPP|node01.local] age: 1 day 16:21:28.172209, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:64512:8000006, age: 1 day 16:12:32.153586, last_modified: 2020-Apr-07 09:34:20.576680
    [XMPP|node02.local] age: 1 day 16:12:32.232845, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None
    [XMPP|node01.local] age: 1 day 16:21:28.207038, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:64512:8000007, age: 1 day 16:12:32.133745, last_modified: 2020-Apr-07 09:34:20.596521
    [XMPP|node02.local] age: 1 day 16:12:32.213910, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None
    [XMPP|node01.local] age: 1 day 16:21:28.240694, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:64512:8000008, age: 1 day 16:12:32.153579, last_modified: 2020-Apr-07 09:34:20.576687
    [XMPP|node02.local] age: 1 day 16:12:32.234633, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None
    [XMPP|node01.local] age: 1 day 16:21:28.241618, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:192.168.122.177:0, age: 1 day 18:20:17.413007, last_modified: 2020-Apr-07 07:26:35.317259
    [Local|None] age: 1 day 18:20:17.494997, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:192.168.122.177:3, age: 1 day 16:14:25.525770, last_modified: 2020-Apr-07 09:32:27.204496
    [XMPP|node02.local] age: 1 day 16:14:25.608312, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None
    [XMPP|node01.local] age: 1 day 16:21:28.176205, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:192.168.122.177:6, age: 1 day 16:21:28.093395, last_modified: 2020-Apr-07 09:25:24.636871
    [XMPP|node01.local] age: 1 day 16:21:28.176662, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:192.168.122.177:7, age: 1 day 16:21:22.811733, last_modified: 2020-Apr-07 09:25:29.918533
    [XMPP|node01.local] age: 1 day 16:21:22.895784, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:192.168.122.177:8, age: 1 day 16:12:32.153561, last_modified: 2020-Apr-07 09:34:20.576705
    [XMPP|node02.local] age: 1 day 16:12:32.238184, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None
    [XMPP|node01.local] age: 1 day 16:21:28.212252, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

64512:target:192.168.122.177:9, age: 1 day 16:12:32.133739, last_modified: 2020-Apr-07 09:34:20.596527
    [XMPP|node02.local] age: 1 day 16:12:32.220344, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None
    [XMPP|node01.local] age: 1 day 16:21:28.247335, localpref: 100, nh: 192.168.122.177, encap: [], label: 0, AS path: None

inet.3: 2 destinations, 2 routes (2 primary, 0 secondary, 0 infeasible)

192.168.122.31/32, age: 1 day 16:14:25.289969, last_modified: 2020-Apr-07 09:32:27.440297
    [XMPP (interface)|node02.local] age: 1 day 16:14:25.377991, localpref: 100, nh: 192.168.122.31, encap: [], label: 3, AS path: None

192.168.122.104/32, age: 1 day 16:21:28.034825, last_modified: 2020-Apr-07 09:25:24.695441
    [XMPP (interface)|node01.local] age: 1 day 16:21:28.123418, localpref: 100, nh: 192.168.122.104, encap: [], label: 3, AS path: None

default-domain:default-project:ip-fabric:ip-fabric.ermvpn.0: 4 destinations, 4 routes (4 primary, 0 secondary, 0 infeasible)

0-192.168.122.31:1-0.0.0.0,255.255.255.255,0.0.0.0, age: 1 day 16:14:25.290128, last_modified: 2020-Apr-07 09:32:27.440138
    [XMPP|node02.local] age: 1 day 16:14:25.379319, localpref: 100, nh: 192.168.122.31, encap: ['gre', 'udp'], label: 0, AS path: None

0-192.168.122.104:1-0.0.0.0,255.255.255.255,0.0.0.0, age: 1 day 16:21:28.015179, last_modified: 2020-Apr-07 09:25:24.715087
    [XMPP|node01.local] age: 1 day 16:21:28.104953, localpref: 100, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 0, AS path: None

1-0:0-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:14:25.307871, last_modified: 2020-Apr-07 09:32:27.422395
    [Local|None] age: 1 day 16:14:25.398456, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

2-0:0-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:14:25.306421, last_modified: 2020-Apr-07 09:32:27.423845
    [Local|None] age: 1 day 16:14:25.397488, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

default-domain:default-project:ip-fabric:ip-fabric.evpn.0: 8 destinations, 8 routes (8 primary, 0 secondary, 0 infeasible)

2-0:0-0-00:50:00:00:02:00,0.0.0.0, age: 1 day 16:14:25.355146, last_modified: 2020-Apr-07 09:32:27.375120
    [XMPP|node02.local] age: 1 day 16:14:25.447923, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None

2-0:0-0-00:50:00:00:02:00,192.168.122.31, age: 1 day 16:14:25.319334, last_modified: 2020-Apr-07 09:32:27.410932
    [XMPP|node02.local] age: 1 day 16:14:25.413454, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None

2-0:0-0-00:50:00:00:03:00,0.0.0.0, age: 1 day 16:21:27.999207, last_modified: 2020-Apr-07 09:25:24.731059
    [XMPP|node01.local] age: 1 day 16:21:28.094021, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None

2-0:0-0-00:50:00:00:03:00,192.168.122.104, age: 1 day 16:21:27.999748, last_modified: 2020-Apr-07 09:25:24.730518
    [XMPP|node01.local] age: 1 day 16:21:28.096822, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'mpls-o-gre', 'udp'], label: 20, AS path: None

2-192.168.122.31:1-2-ff:ff:ff:ff:ff:ff,0.0.0.0, age: 1 day 16:14:25.321337, last_modified: 2020-Apr-07 09:32:27.408929
    [XMPP|node02.local] age: 1 day 16:14:25.420807, localpref: 100, nh: 192.168.122.31, encap: ['vxlan'], label: 2, AS path: None

2-192.168.122.104:1-2-ff:ff:ff:ff:ff:ff,0.0.0.0, age: 1 day 16:21:28.010269, last_modified: 2020-Apr-07 09:25:24.719997
    [XMPP|node01.local] age: 1 day 16:21:28.110317, localpref: 100, nh: 192.168.122.104, encap: ['vxlan'], label: 2, AS path: None

3-192.168.122.31:1-2-192.168.122.31, age: 1 day 16:14:25.309044, last_modified: 2020-Apr-07 09:32:27.421222
    [Local|None] age: 1 day 16:14:25.409701, localpref: 100, nh: 192.168.122.31, encap: ['vxlan'], label: 2, AS path: None

3-192.168.122.104:1-2-192.168.122.104, age: 1 day 16:21:28.010062, last_modified: 2020-Apr-07 09:25:24.720204
    [Local|None] age: 1 day 16:21:28.112911, localpref: 100, nh: 192.168.122.104, encap: ['vxlan'], label: 2, AS path: None

default-domain:default-project:ip-fabric:ip-fabric.inet.0: 5 destinations, 5 routes (2 primary, 3 secondary, 0 infeasible)

10.47.255.249/32, age: 1 day 16:21:22.754137, last_modified: 2020-Apr-07 09:25:29.976129
    [XMPP (interface)|node01.local] age: 1 day 16:21:22.857772, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 29, AS path: None

10.47.255.250/32, age: 1 day 16:21:19.131071, last_modified: 2020-Apr-07 09:25:33.599195
    [XMPP (interface)|node01.local] age: 1 day 16:21:19.235177, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 34, AS path: None

10.96.0.10/32, age: 1 day 16:21:19.122804, last_modified: 2020-Apr-07 09:25:33.607462
    [XMPP (interface)|node01.local] age: 1 day 16:21:19.227504, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 46, AS path: None

192.168.122.31/32, age: 1 day 16:14:25.319305, last_modified: 2020-Apr-07 09:32:27.410961
    [XMPP (interface)|node02.local] age: 1 day 16:14:25.426408, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'udp', 'native'], label: 16, AS path: None

192.168.122.104/32, age: 1 day 16:21:27.998746, last_modified: 2020-Apr-07 09:25:24.731520
    [XMPP (interface)|node01.local] age: 1 day 16:21:28.106575, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp', 'native'], label: 16, AS path: None

default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network.ermvpn.0: 3 destinations, 3 routes (3 primary, 0 secondary, 0 infeasible)

0-192.168.122.104:4-0.0.0.0,255.255.255.255,0.0.0.0, age: 1 day 16:21:28.032462, last_modified: 2020-Apr-07 09:25:24.697804
    [XMPP|node01.local] age: 1 day 16:21:28.141592, localpref: 100, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 0, AS path: None

1-0:0-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:28.032288, last_modified: 2020-Apr-07 09:25:24.697978
    [Local|None] age: 1 day 16:21:28.142144, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

2-0:0-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:28.031895, last_modified: 2020-Apr-07 09:25:24.698371
    [Local|None] age: 1 day 16:21:28.142628, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network.inet.0: 5 destinations, 5 routes (2 primary, 3 secondary, 0 infeasible)

10.47.255.249/32, age: 1 day 16:21:22.758839, last_modified: 2020-Apr-07 09:25:29.971427
    [XMPP (interface)|node01.local] age: 1 day 16:21:22.870314, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 29, AS path: None

10.47.255.250/32, age: 1 day 16:21:19.131145, last_modified: 2020-Apr-07 09:25:33.599121
    [XMPP (interface)|node01.local] age: 1 day 16:21:19.243330, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 34, AS path: None

10.96.0.10/32, age: 1 day 16:21:19.122835, last_modified: 2020-Apr-07 09:25:33.607431
    [XMPP (interface)|node01.local] age: 1 day 16:21:19.236818, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 46, AS path: None

192.168.122.31/32, age: 1 day 16:14:25.318909, last_modified: 2020-Apr-07 09:32:27.411357
    [XMPP (interface)|node02.local] age: 1 day 16:14:25.433492, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'udp', 'native'], label: 16, AS path: None

192.168.122.104/32, age: 1 day 16:21:27.998609, last_modified: 2020-Apr-07 09:25:24.731657
    [XMPP (interface)|node01.local] age: 1 day 16:21:28.113649, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp', 'native'], label: 16, AS path: None

default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network.ermvpn.0: 3 destinations, 3 routes (3 primary, 0 secondary, 0 infeasible)

0-192.168.122.104:5-0.0.0.0,255.255.255.255,0.0.0.0, age: 1 day 16:21:22.777989, last_modified: 2020-Apr-07 09:25:29.952277
    [XMPP|node01.local] age: 1 day 16:21:22.893796, localpref: 100, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 0, AS path: None

1-0:0-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:22.776576, last_modified: 2020-Apr-07 09:25:29.953690
    [Local|None] age: 1 day 16:21:22.892847, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

2-0:0-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:21:22.775930, last_modified: 2020-Apr-07 09:25:29.954336
    [Local|None] age: 1 day 16:21:22.892807, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network.inet.0: 5 destinations, 5 routes (1 primary, 4 secondary, 0 infeasible)

10.47.255.249/32, age: 1 day 16:21:22.758756, last_modified: 2020-Apr-07 09:25:29.971510
    [XMPP (interface)|node01.local] age: 1 day 16:21:22.876368, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 29, AS path: None

10.47.255.250/32, age: 1 day 16:21:19.131081, last_modified: 2020-Apr-07 09:25:33.599185
    [XMPP (interface)|node01.local] age: 1 day 16:21:19.249151, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 34, AS path: None

10.96.0.10/32, age: 1 day 16:21:19.122994, last_modified: 2020-Apr-07 09:25:33.607272
    [XMPP (interface)|node01.local] age: 1 day 16:21:19.242847, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 46, AS path: None

192.168.122.31/32, age: 1 day 16:14:25.318883, last_modified: 2020-Apr-07 09:32:27.411383
    [XMPP (interface)|node02.local] age: 1 day 16:14:25.439389, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'udp', 'native'], label: 16, AS path: None

192.168.122.104/32, age: 1 day 16:21:27.998600, last_modified: 2020-Apr-07 09:25:24.731666
    [XMPP (interface)|node01.local] age: 1 day 16:21:28.119562, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp', 'native'], label: 16, AS path: None

default-domain:k8s-test-ns1:k8s-ns1-pod-net01:k8s-ns1-pod-net01.ermvpn.0: 4 destinations, 4 routes (4 primary, 0 secondary, 0 infeasible)

0-192.168.122.31:2-0.0.0.0,255.255.255.255,0.0.0.0, age: 1 day 16:12:32.109415, last_modified: 2020-Apr-07 09:34:20.620851
    [XMPP|node02.local] age: 1 day 16:12:32.231231, localpref: 100, nh: 192.168.122.31, encap: ['gre', 'udp'], label: 0, AS path: None

0-192.168.122.104:3-0.0.0.0,255.255.255.255,0.0.0.0, age: 1 day 16:21:28.049318, last_modified: 2020-Apr-07 09:25:24.680948
    [XMPP|node01.local] age: 1 day 16:21:28.171572, localpref: 100, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 0, AS path: None

1-0:0-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:12:32.108131, last_modified: 2020-Apr-07 09:34:20.622135
    [Local|None] age: 1 day 16:12:32.230943, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

2-0:0-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:12:32.103512, last_modified: 2020-Apr-07 09:34:20.626754
    [Local|None] age: 1 day 16:12:32.226756, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

default-domain:k8s-test-ns1:k8s-ns1-pod-net01:k8s-ns1-pod-net01.inet.0: 2 destinations, 2 routes (1 primary, 1 secondary, 0 infeasible)

10.10.10.1/32, age: 1 day 16:21:27.665965, last_modified: 2020-Apr-07 09:25:25.064301
    [XMPP (interface)|node01.local] age: 1 day 16:21:27.790039, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 39, AS path: None

10.10.20.1/32, age: 1 day 16:12:31.895992, last_modified: 2020-Apr-07 09:34:20.834274
    [XMPP (interface)|node02.local] age: 1 day 16:12:32.020713, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'udp'], label: 27, AS path: None

default-domain:k8s-test-ns1:k8s-ns1-pod-net02:k8s-ns1-pod-net02.ermvpn.0: 4 destinations, 4 routes (4 primary, 0 secondary, 0 infeasible)

0-192.168.122.31:3-0.0.0.0,255.255.255.255,0.0.0.0, age: 1 day 16:12:32.102992, last_modified: 2020-Apr-07 09:34:20.627274
    [XMPP|node02.local] age: 1 day 16:12:32.228379, localpref: 100, nh: 192.168.122.31, encap: ['gre', 'udp'], label: 0, AS path: None

0-192.168.122.104:2-0.0.0.0,255.255.255.255,0.0.0.0, age: 1 day 16:21:28.053722, last_modified: 2020-Apr-07 09:25:24.676544
    [XMPP|node01.local] age: 1 day 16:21:28.179558, localpref: 100, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 0, AS path: None

1-0:0-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:12:32.102824, last_modified: 2020-Apr-07 09:34:20.627442
    [Local|None] age: 1 day 16:12:32.229695, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

2-0:0-192.168.122.177,255.255.255.255,0.0.0.0, age: 1 day 16:12:32.101927, last_modified: 2020-Apr-07 09:34:20.628339
    [Local|None] age: 1 day 16:12:32.229561, localpref: 100, nh: 192.168.122.177, encap: ['gre', 'udp'], label: 0, AS path: None

default-domain:k8s-test-ns1:k8s-ns1-pod-net02:k8s-ns1-pod-net02.inet.0: 2 destinations, 2 routes (1 primary, 1 secondary, 0 infeasible)

10.10.10.1/32, age: 1 day 16:21:27.665882, last_modified: 2020-Apr-07 09:25:25.064384
    [XMPP (interface)|node01.local] age: 1 day 16:21:27.794101, localpref: 200, nh: 192.168.122.104, encap: ['gre', 'udp'], label: 39, AS path: None

10.10.20.1/32, age: 1 day 16:12:31.896098, last_modified: 2020-Apr-07 09:34:20.834168
    [XMPP (interface)|node02.local] age: 1 day 16:12:32.025273, localpref: 200, nh: 192.168.122.31, encap: ['gre', 'udp'], label: 27, AS path: None
[root@master01 ~]#

K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 | 韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。长夜有穷，真水无香。领略过了 K8s“身在江

K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub