vPC 与 HSRP 奇怪的 arp 问题

网络工程 思科 路由 转变 网络 高铁
2021-07-13 16:57:52

我有以下 vPC 和 HSRP(版本 1)配置的场景。

在此处输入图片说明

我看到了非常奇怪的问题,我的主机配置为bond + vlan并且我的绑定模式是active-backup,我只为 HSRP 配置了 VLAN 100,我看到我的主机无法 ping HSRP 虚拟 IP,但它可以 ping VLAN 100 上的所有其他主机,这问题刚刚开始,几周前一切正常。

虚拟PC配置

vpc domain 1
  peer-switch
  role priority 10
  peer-keepalive destination 10.5.0.117 source 10.5.0.116
  peer-gateway
  auto-recovery
  ip arp synchronize

HSRP 配置

interface Vlan100
  description *** Public_1 VLAN ***
  no shutdown
  mtu 9216
  no autostate
  no ip redirects
  ip address 74.xx.xx.2/23
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  hsrp 1
    preempt
    priority 110
    ip 74.xx.xx.1

我在主机机器 vlan 10 上配置了两个 VLAN,下面的 vlan 100 是我的主机接口输出。

bond0.10: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.10.1.146  netmask 255.255.0.0  broadcast 10.10.255.255
        inet6 fe80::6e3b:e5ff:feba:84e8  prefixlen 64  scopeid 0x20<link>
        ether 6c:3b:e5:ba:84:e8  txqueuelen 1000  (Ethernet)
        RX packets 18724100  bytes 861377042 (821.4 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1579  bytes 160270 (156.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

bond0.100: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 74.xx.xx.179  netmask 255.255.254.0  broadcast 74.xx.xx.255
        inet6 fe80::6e3b:e5ff:feba:84e8  prefixlen 64  scopeid 0x20<link>
        ether 6c:3b:e5:ba:84:e8  txqueuelen 1000  (Ethernet)
        RX packets 338156  bytes 15584262 (14.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 74  bytes 7230 (7.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

HOSR arp表

[root@host ~]# arp -n
Address                  HWtype  HWaddress           Flags Mask            Iface
74.xx.xx.171           ether   fc:15:b4:13:1e:40   C                     bond0.100
74.xx.xx.2             ether   fc:5b:39:f7:6d:4f   C                     bond0.100
74.xx.xx.170           ether   d8:9d:67:75:2a:98   C                     bond0.100
74.xx.xx.1             ether   00:00:0c:07:ac:01   C                     bond0.100
74.xx.xx.177           ether   6c:3b:e5:b0:f9:f0   C                     bond0.100

SW1 交换机上的 ARP 和 MAC 表

sw1# show ip arp 74.xx.xx.179

Flags: * - Adjacencies learnt on non-active FHRP router
       + - Adjacencies synced via CFSoE
       # - Adjacencies Throttled for Glean
       CP - Added via L2RIB, Control plane Adjacencies       D - Static Adjacencies attached to down interface

IP ARP Table
Total number of entries: 1
Address         Age       MAC Address     Interface
74.xx.xx.179  00:02:35  6c3b.e5ba.84e8  Vlan100

SW1 MAC 表

sw1# show mac address-table address 6c3b.e5ba.84e8
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False, C - ControlPlane MAC
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
*   10     6c3b.e5ba.84e8   dynamic  0         F      F    Po46

FHRP

sw1# show ip arp fhrp-non-active-learn

Flags: D - Static Adjacencies attached to down interface

IP ARP Table for context default
Address         Age       MAC Address     Interface

开关2

sw2# show ip arp 74.xx.xx.179

Flags: * - Adjacencies learnt on non-active FHRP router
       + - Adjacencies synced via CFSoE
       # - Adjacencies Throttled for Glean
       CP - Added via L2RIB, Control plane Adjacencies       D - Static Adjacencies attached to down interface

IP ARP Table
Total number of entries: 1
Address         Age       MAC Address     Interface
74.xx.xx.179  00:05:04  6c3b.e5ba.84e8  Vlan100          *

MAC表

sw2# show mac address-table address 6c3b.e5ba.84e8
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False, C - ControlPlane MAC
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
+   10     6c3b.e5ba.84e8   dynamic  0         F      F    Po46

FHRP

sw2# show ip arp fhrp-non-active-learn

Flags: D - Static Adjacencies attached to down interface

IP ARP Table for context default
Address         Age       MAC Address     Interface
74.xx.xx.179  00:07:32  6c3b.e5ba.84e8  Vlan100

更新 - 1

我们看到很多以下登录 show logging

2018 Jul  5 14:15:05 swt1 %-SLOT1-5-BCM_L2_LEARN_DISABLE: MAC Learning Disabled unit=0
2018 Jul  5 14:15:06 swt1 %-SLOT1-5-BCM_L2_LEARN_ENABLE: MAC Learning Enabled unit=0
2018 Jul  5 14:17:06 swt1 %-SLOT1-5-BCM_L2_LEARN_DISABLE: MAC Learning Disabled unit=0
2018 Jul  5 14:17:06 swt1 %-SLOT1-5-BCM_L2_LEARN_ENABLE: MAC Learning Enabled unit=0
2018 Jul  5 14:17:06 swt1 %-SLOT1-5-BCM_L2_LEARN_DISABLE: MAC Learning Disabled unit=0
2018 Jul  5 14:17:07 swt1 %-SLOT1-5-BCM_L2_LEARN_ENABLE: MAC Learning Enabled unit=0
2018 Jul  5 14:19:06 swt1 %-SLOT1-5-BCM_L2_LEARN_DISABLE: MAC Learning Disabled unit=0
2018 Jul  5 14:19:07 swt1 %-SLOT1-5-BCM_L2_LEARN_ENABLE: MAC Learning Enabled unit=0
2018 Jul  5 14:19:07 swt1 %-SLOT1-5-BCM_L2_LEARN_DISABLE: MAC Learning Disabled unit=0
2018 Jul  5 14:19:08 swt1 %-SLOT1-5-BCM_L2_LEARN_ENABLE: MAC Learning Enabled unit=0

更新-2

Jul  5 14:31:13 10.5.0.116 : 2018 Jul  5 18:24:35 UTC: %L2FM-4-L2FM_MAC_MOVE2: Mac 1458.d05a.f6d8 in vlan 100 has moved between Po43 to Po44
Jul  5 14:31:17 10.5.0.116 : 2018 Jul  5 18:24:39 UTC: %L2FM-4-L2FM_MAC_MOVE2: Mac 1458.d05a.f6d8 in vlan 100 has moved between Po43 to Po44
Jul  5 14:31:25 10.5.0.116 : 2018 Jul  5 18:24:47 UTC: %L2FM-4-L2FM_MAC_MOVE2: Mac fc15.b41f.59e0 in vlan 100 has moved between Po43 to Po44
Jul  5 14:31:29 10.5.0.116 : 2018 Jul  5 18:24:51 UTC: %L2FM-4-L2FM_MAC_MOVE2: Mac 1458.d05a.f6d8 in vlan 100 has moved between Po43 to Po44
Jul  5 14:31:33 10.5.0.116 : 2018 Jul  5 18:24:54 UTC: %L2FM-4-L2FM_MAC_MOVE2: Mac 6c3b.e5b0.c998 in vlan 100 has moved between Po35 to Po36
Jul  5 14:31:35 10.5.0.116 : 2018 Jul  5 18:24:56 UTC: %L2FM-4-L2FM_MAC_MOVE2: Mac 1458.d05a.f6d8 in vlan 100 has moved between Po43 to Po44
Jul  5 14:31:35 10.5.0.116 : 2018 Jul  5 18:24:56 UTC: %L2FM-3-L2FM_MAC_FLAP_DISABLE_LEARN: Disabling learning in vlan 100 for 120s due to too many mac moves
Jul  5 14:31:35 10.5.0.116 : 2018 Jul  5 18:24:56 UTC: %L2FM-4-L2FM_MAC_MOVE2: Mac 1458.d05a.f6d8 in vlan 100 has moved between Po43 to Po44
Jul  5 14:33:35 10.5.0.116 : 2018 Jul  5 18:26:57 UTC: %L2FM-3-L2FM_MAC_FLAP_RE_ENABLE_LEARN: Re-enabling learning in vlan 100
Jul  5 14:34:44 10.5.0.116 : 2018 Jul  5 18:28:06 UTC: %L2FM-4-L2FM_MAC_MOVE2: Mac fc15.b41f.59e0 in vlan 100 has moved between Po43 to Po44
Jul  5 14:34:44 10.5.0.116 : 2018 Jul  5 18:28:06 UTC: %L2FM-3-L2FM_MAC_FLAP_DISABLE_LEARN: Disabling learning in vlan 100 for 120s due to too many mac moves
Jul  5 14:34:44 10.5.0.116 : 2018 Jul  5 18:28:06 UTC: %L2FM-4-L2FM_MAC_MOVE2: Mac fc15.b41f.59e0 in vlan 100 has moved between Po43 to Po44
Jul  5 14:36:43 10.5.0.116 : 2018 Jul  5 18:30:05 UTC: %L2FM-3-L2FM_MAC_FLAP_RE_ENABLE_LEARN: Re-enabling learning in vlan 100

问题:

  1. 为什么即使在主机 arp 表上我也无法从主机 ping HSRP VIP 74.xx.xx.1 ip 地址我可以看到它的 MAC 地址?

  2. 为什么在 MAC 表中我看不到 VLAN 100 mac 地址(但我可以看到 vlan 10 MAC)

  3. fhrp-non-active-learn是什么以及为什么它只出现在 SW2(备用 HSRP 实例)上

1个回答

解决:

问题是主机在round-robin绑定配置上运行,这会导致大量 MAC 抖动并导致锁定 MAC 表并阻止新的 MAC 完整。

我们重新配置了所有服务器以用于active-backup解决此问题。

注意:上图不正确(我们认为它是主动备份但它是循环的)