EVPN+VxLAN 中未标记的 VLAN

网络工程 思科 转变 网络 vxlan evpn
2022-03-05 14:03:54

我正在处理非常奇怪的问题，但想咨询专家我是否在做任何愚蠢的事情。我有小型 EVPN+VxLAN clos 网络（OSFP+iBGP+Multicast+Arp-suppression（没有 anycastgateway）在我的设置中）非常标准的配置。我在 vPC 中寻求重新定位。我的叶子连接到HP c7000 bladecenterswitch 6120XG。

在此处输入图像描述

问题：在刀片服务器上，我有两个 10G 网卡（nic1 连接到刀片交换机 A，而 nic2 连接到刀片交换机 B）所以我配置了绑定主动备份，因为 6120XG 不支持 MLAG。在这一点上一切都很好。我有 Linux PXE kickstart 服务器，我有专用的 PXE VLAN 70，我在 6120XG 交换机上没有标记（因为 PXE 不支持 VLAN 标记）。当我重新启动刀片服务器并进入 PXE 启动以启动时，我可以看到我的 PXE 从 DHCP 获取 IP 地址，但之后它停止 ping 该 IP 并且 kickstart 失败说没有网络连接（简而言之，我无法 ping 那个 pxe IP）。

我已为 arp 抑制配置了所有 VLAN/VNI。如果我删除了 PXE VLAN/VNI 的 arp-suppression，那么一切正常，PXE 成功地启动了我的所有服务器，但是一旦我添加了 arp-suppression，它就停止了工作。知道 EVPN+VxLAN 如何处理未标记的 VLAN 吗？

叶 NVE1 接口配置

interface nve1
  no shutdown
  description ** VTEP/NVE Interface **
  host-reachability protocol bgp
  source-interface loopback1
  member vni 10064
    suppress-arp
    mcast-group 239.1.1.1
  member vni 10065
    suppress-arp
    mcast-group 239.1.1.1
  member vni 10070
    suppress-arp     <---------- if i remove this it then PXE works. 
    mcast-group 239.1.1.1
  member vni 10100
    suppress-arp
    mcast-group 239.1.1.1
  member vni 10555 associate-vrf

Bladecenter 交换机未标记 VLAN 70

vlan 70
   name "pxe"
   untagged 1-16
   tagged Trk1
   no ip address
   exit

更新 - 1

调试后，我发现 PXE 引导负载开始在特定位置加载 CentOS linux，LinuxVLAN ID 0在接口上设置为使其本机以允许访问所有 VLAN，此时我注意到它在该特定 VLAN 上使用 ARP 广播淹没了我的网络，并且这导致网络上的数据包丢失并且我的安装卡住了。

arp suppression由于不确定原因，看起来这个未标记的框架进入了循环。我也有其他 VLAN，它们都标有 VLAN，工作正常，没有任何问题。我没有看到他们有任何问题。

[root@pxe-server ~]# tcpdump -i bond0.70 -nn not port 22 and host 10.70.0.112
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on bond0.70, link-type EN10MB (Ethernet), capture size 262144 bytes
13:13:23.115377 ARP, Request who-has 10.70.0.112 (ff:ff:ff:ff:ff:ff) tell 0.0.0.0, length 46
13:13:23.115539 ARP, Request who-has 10.70.0.112 (ff:ff:ff:ff:ff:ff) tell 0.0.0.0, length 46
13:13:23.115974 ARP, Request who-has 10.70.0.112 (ff:ff:ff:ff:ff:ff) tell 0.0.0.0, length 46
13:13:23.116110 ARP, Request who-has 10.70.0.112 (ff:ff:ff:ff:ff:ff) tell 0.0.0.0, length 46
13:13:23.215306 ARP, Request who-has 10.70.0.112 (ff:ff:ff:ff:ff:ff) tell 0.0.0.0, length 46
13:13:23.215312 ARP, Request who-has 10.70.0.112 (ff:ff:ff:ff:ff:ff) tell 0.0.0.0, length 46
13:13:23.215823 ARP, Request who-has 10.70.0.112 (ff:ff:ff:ff:ff:ff) tell 0.0.0.0, length 46

打破这种广播风暴的唯一方法是suppress-arp从 VLAN 70 中删除选项。我认为这在 Cisco 软件中可能很重要，因为不确定 arp 与非标记帧的关系是什么。

这是我的 Leaf-1-1 配置，Leaf-1-2 是第二个具有非常相似配置的 vPC 对等体。它的网络非常简单，不是花哨的东西，根据图表，我只有 2 个 vPC 对 Leaf。

问：我suppress-arp在 VNI 上启用了它没有的纯 L2VNI，anycast-gateway因为我的 Cisco ASA 是所有 VLAN 的网关。（我假设 suppress-arp 仅在没有分布式网关的情况下使用 L2VNI）

leaf-1-1# show run

cfs eth distribute
nv overlay evpn
feature ospf
feature bgp
feature pim
feature fabric forwarding
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature vpc
feature nv overlay

ip domain-lookup
system qos
  service-policy type network-qos jumboframes

fabric forwarding anycast-gateway-mac 0000.dead.beef
ip pim rp-address 10.255.0.123 group-list 239.0.0.0/8
ip pim ssm range 232.0.0.0/8
vlan 1,64-68,70,100,444,555
vlan 64
  name inside
vlan 65
  vn-segment 10065
vlan 66
  vn-segment 10066
vlan 67
  vn-segment 10067
vlan 68
  vn-segment 10068
vlan 70
  name pxe_boot
  vn-segment 10070
vlan 100
  name public
  vn-segment 10100
vlan 444
  name BACKUP_VLAN_ROUTING_VPC
vlan 555
  name L3VNI
  vn-segment 10555

spanning-tree port type edge bpduguard default
spanning-tree loopguard default
spanning-tree vlan 64-68,70,100,555 priority 8192
route-map DIRECT-PERMIT-ALL permit 10
  description ** Route-Map for BGP to redist route **
vrf context CUST1
  vni 10555
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management
  ip route 0.0.0.0/0 172.30.0.1
hardware access-list tcam region vacl 0
hardware access-list tcam region arp-ether 256
vpc domain 1
  peer-switch
  role priority 10
  peer-keepalive destination 172.30.0.32 source 172.30.0.31
  delay restore 90
  peer-gateway
  delay restore interface-vlan 30
  ip arp synchronize


interface Vlan1
  no ip redirects
  no ipv6 redirects

interface Vlan100
  description ** Anycast Gateway For Public  **
  no shutdown
  mtu 9216
  vrf member CUST1
  ip address 60.25.124.1/23
  fabric forwarding mode anycast-gateway

interface Vlan444
  description ** Underlay Backup over vPC Peer-Link **
  no shutdown
  ip address 192.168.1.1/30
  ip ospf authentication-key 3 fa3ab8e90610229c
  ip ospf network point-to-point
  ip router ospf UNDERLAY-NET area 0.0.0.0
  ip pim sparse-mode

interface Vlan555
  description ** L3VNI-For-IRB **
  no shutdown
  mtu 9216
  vrf member CUST1
  ip forward

interface port-channel111
  description ** Link to enc-k001-1-a **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 111

interface port-channel112
  description ** Link to enc-k001-1-b **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 112

interface port-channel121
  description ** Link to enc-k001-2-a **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 121

interface port-channel122
  description ** Link to enc-k001-2-b **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 122

interface port-channel211
  description ** Link to enc-k002-1-a **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 211

interface port-channel212
  description ** Link to enc-k002-1-b **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 212

interface port-channel221
  description ** Link to enc-k002-2-a **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 221

interface port-channel222
  description ** Link to enc-k002-2-b **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 222

interface port-channel311
  description ** Link to enc-k003-1-a **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 311

interface port-channel312
  description ** Link to enc-k003-1-b **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 312

interface port-channel321
  description ** Link to enc-k003-2-a **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 321

interface port-channel322
  description ** Link to enc-k003-2-b **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  vpc 322

interface port-channel999
  description ** vPC Peer-Link **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100,444,555
  spanning-tree port type network
  speed 40000
  no negotiate auto
  vpc peer-link

interface nve1
  no shutdown
  description ** VTEP/NVE Interface **
  host-reachability protocol bgp
  source-interface loopback1
  member vni 10064
    suppress-arp
    mcast-group 239.1.1.1
  member vni 10065
    suppress-arp
    mcast-group 239.1.1.1
  member vni 10066
    suppress-arp
    mcast-group 239.1.1.1
  member vni 10067
    suppress-arp
    mcast-group 239.1.1.1
  member vni 10068
    suppress-arp
    mcast-group 239.1.1.1
  member vni 10070
    mcast-group 239.1.1.1
  member vni 10100
    suppress-arp
    mcast-group 239.1.1.1
  member vni 10555 associate-vrf

interface Ethernet1/1
  description ** Link to swt-enc-k001-1-a Port E18 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 111 mode active

interface Ethernet1/2
  description ** Link to swt-enc-k001-1-a Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 111 mode active

interface Ethernet1/3
  description ** Link to swt-enc-k001-1-b Port E18 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 112 mode active

interface Ethernet1/4
  description ** Link to swt-enc-k001-1-b Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 112 mode active

interface Ethernet1/5
  description ** Link to swt-enc-k001-2-a Port E18 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 121 mode active

interface Ethernet1/6
  description ** Link to swt-enc-k001-2-a Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 121 mode active

interface Ethernet1/7
  description ** Link to swt-enc-k001-2-b Port E18 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 122 mode active

interface Ethernet1/8
  description ** Link to swt-enc-k001-2-b Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 122 mode active

interface Ethernet1/9
  shutdown
  mtu 9216

interface Ethernet1/10
  description ** Link to VMware Host-1 **
  switchport mode trunk
  switchport trunk allowed vlan 64-65,70,100
  spanning-tree port type edge trunk
  mtu 9216
  speed 10000

interface Ethernet1/17
  description ** Link to swt-enc-k002-1-a Port E18 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 211 mode active

interface Ethernet1/18
  description ** Link to swt-enc-k002-1-a Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 211 mode active

interface Ethernet1/19
  description ** Link to swt-enc-k002-1-b Port E18 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 212 mode active

interface Ethernet1/20
  description ** Link to swt-enc-k002-1-b Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 212 mode active

interface Ethernet1/21
  description ** Link to swt-enc-k002-2-a Port E18 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 221 mode active

interface Ethernet1/22
  description ** Link to swt-enc-k002-2-a Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 221 mode active

interface Ethernet1/23
  description ** Link to swt-enc-k002-2-b Port E18 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 222 mode active

interface Ethernet1/24
  description ** Link to swt-enc-k002-2-b Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 222 mode active

interface Ethernet1/33
  description ** Link to swt-enc-1-a Port E18 **k003
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 311 mode active

interface Ethernet1/34
  description ** Link to swt-enc-k003-1-a Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 311 mode active

interface Ethernet1/35
  description ** Link to swt-enc-k003-1-b Port E18 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 312 mode active

interface Ethernet1/36
  description ** Link to swt-enc-k003-1-b Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 312 mode active

interface Ethernet1/37
  description ** Link to swt-enc-2-a Port E18 **k003
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 321 mode active

interface Ethernet1/38
  description ** Link to swt-enc-k003-2-a Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 321 mode active

interface Ethernet1/39
  description ** Link to swt-enc-k003-2-b Port E18 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 322 mode active

interface Ethernet1/40
  description ** Link to swt-enc-k003-2-b Port E19 **
  no cdp enable
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100
  mtu 9216
  speed 10000
  channel-group 322 mode active

interface Ethernet2/1
  description ** Spine-1 **
  no switchport
  mtu 9216
  medium p2p
  ip address 10.1.11.1/31
  ip ospf authentication-key 3 XXXXXXX
  ip ospf network point-to-point
  ip router ospf UNDERLAY-NET area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet2/2
  description ** Spine-2 **
  no switchport
  mtu 9216
  medium p2p
  ip address 10.2.11.1/31
  ip ospf authentication-key 3 XXXXXXX
  ip ospf network point-to-point
  ip router ospf UNDERLAY-NET area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet2/11
  description ** vPC Peer-Link **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100,444,555
  speed 40000
  no negotiate auto
  channel-group 999 mode active

interface Ethernet2/12
  description ** vPC Peer-Link **
  switchport mode trunk
  switchport trunk allowed vlan 64-68,70,100,444,555
  speed 40000
  no negotiate auto
  channel-group 999 mode active

interface mgmt0
  vrf member management
  no ip redirects
  ip address 172.30.0.31/23

interface loopback0
  description ** RID/BGP Overlay **
  ip address 10.255.1.11/32
  ip router ospf UNDERLAY-NET area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description ** VTEP/Overlay **
  ip address 10.255.255.11/32
  ip address 10.255.255.10/32 secondary
  ip ospf authentication-key 3 fa3ab8e90610229c
  ip router ospf UNDERLAY-NET area 0.0.0.0
  ip pim sparse-mode
cli alias name wr copy running-config startup-config
line console
line vty
boot nxos bootflash:/nxos.9.3.4.bin
router ospf UNDERLAY-NET
  router-id 10.255.1.11
  log-adjacency-changes
  area 0.0.0.0 authentication
router bgp 65001
  router-id 10.255.1.11
  log-neighbor-changes
  template peer VXLAN_SPINE
    remote-as 65001
    update-source loopback0
    address-family ipv4 unicast
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 10.255.0.1
    inherit peer VXLAN_SPINE
    description ** iBGP Peer to Spine-1 **
  neighbor 10.255.0.2
    inherit peer VXLAN_SPINE
    description ** iBGP Peer to Spine-2 **
  vrf CUST1
    log-neighbor-changes
    address-family ipv4 unicast
      redistribute direct route-map DIRECT-PERMIT-ALL
evpn
  vni 10064 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10065 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10066 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10067 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10068 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10070 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 10100 l2
    rd auto
    route-target import auto
    route-target export auto

更新 2：

经过更多测试后，我发现罪魁祸首是 DHCP 数据包，为了测试目的，我在其他 VLAN 上启动了 DHCP 服务器，并注意到 DHCP 发送/接收广播数据包并同时广播泛洪开始。（如果启用了 ARP 抑制）。我将创建 Anycast 网关来证明这个理论，即 arp-suppression 不支持 L2VNI。如果有人有任何历史，请插话。

注意：我没有使用任何花哨的DHCP-Relay功能。

0个回答
没有发现任何回复~
其它你可能感兴趣的问题
上一篇我在哪里可以找到虚拟交换机来了解第 2 层 下一篇P2MP 链路上未编号的 BGP