硬件交换机中OpenFlow流水线的包处理限制

网络工程 sdn 交换机端口 开放流 阿鲁巴岛 切换模式
2022-03-07 17:02:31

我有一个 10Gbps 网络，其中包含 2 个服务器 (DellPowerEdge 4700r) 和一个硬件交换机 (ARUBA 2930F)。两台服务器通过 10G SFP+ 端口连接到交换机。我以 10Gbps 的速度将 TCP 流量从一台服务器发送到另一台服务器。在交换机配置中，我可以通过在交换机上配置 OpenFlow 实例来选择使用 OpenFlow 管道，也可以使用默认的 L2 交换机行为。当我使用默认开关行为（无 OpenFlow）时，我从一台服务器到另一台服务器的吞吐量约为 9.4Gbps。此外，我在入口端口上看不到任何 Rx 丢弃。但是，当我改用 OpenFlow（启用 OpenFlow 实例）并再次发送流量时，我看到交换机上有数千个 Rx 丢弃。结果，吞吐量在 2Gbps 到 8Gbps 之间波动（标准偏差为 3 Gbps）（RTO 命中多次）。

是 OpenFlow 管道中数据包处理速度较慢的问题吗？有没有其他人观察过？这似乎是配置的问题？我已确保控制器代码和 OpenFlow 交换机配置正确。

OpenFlow有标准和自定义管道。我一直在使用 OpenFlow 的标准管道。我已在此处阅读了交换机的文档（在 OpenFlow 自定义管道 -> 自定义管道模型中的性能；还添加了图片参考）关于 OpenFlow自定义的此行为管道。它说在 OpenFlow 自定义管道中：“如果管道忙于处理太多数据包，则数据包会在入口处丢弃。您可以检查交换机上的 Rx 丢弃以获取此类数据”。也许在标准管道中也会出现类似的问题。众所周知，如果数据包在软件中处理，它们将不会以线速处理，即处理它们需要更长的时间并且可能会发生 Rx 丢弃。现在，在 OpenFlow 管道（标准和自定义）中，我们可以选择在软件表 (SRAM) 或硬件表 (TCAM) 中放置流规则的位置。我所有的 OpenFlow 流规则都在硬件表（TCAM）中。现在不应该以线路速率处理它们并且不显示任何 Rx 丢弃吗？

发送流量的服务器连接到交换机的 25 端口，接收流量的服务器连接到 27 端口。交换机上使用 OpenFlow 标准管道设置的配置如下：

要在交换机上显示运行配置：

leaf1(config)# sh running-config

    Running configuration:

    ; JL253A Configuration Editor; Created on release #WC.16.05.0007
    ; Ver #12:08.1d.9b.3f.bf.bb.ef.7c.59.fc.6b.fb.9f.fc.ff.ff.37.ef:ba
    hostname "leaf1"
    module 1 type jl253a
    max-vlans 2048
    snmp-server community "public" unrestricted
    openflow
       controller-id 1 ip 192.168.0.1 controller-interface vlan 2
       instance "l1"
          listen-port
          member vlan 3
          controller-id 1
          version 1.3 only
          limit software-rate 2000
          max-backoff-interval 10
          pipeline-model standard-match
          packet-in vlan-tagging input-form
          enable
          exit
       enable
       exit
    vlan 1
       name "DEFAULT_VLAN"
       no untagged 1-3,11-13,19-28
       untagged 4-10,14-18
       no ip address
       exit
    vlan 2
       name "c-vlan"
       untagged 19-24
       ip address 192.168.1.1 255.255.0.0
       exit
    vlan 3
       name "s-vlan"
       untagged 1-3,11-13,25-28
       ip address 10.1.0.1 255.0.0.0
       exit

管道中有 3 个表：启动表、策略表和 sw-table-1。启动表和策略表在硬件 (TCAM) 中，而 sw-table-1 在软件中。以 1 个流规则启动表，以 5 个流规则启动策略表，然后以 1 个流规则启动 sw-table-1。所有数据包直接进入启动表。起始表中的一个流有一个 go-to-table 规则，它将所有数据包转发到策略表。我的所有规则都在策略表中。然后，sw-table-1 有一个丢弃数据包的规则。这个 sw-table-1 永远不会在管道中调用。

要在标准管道中显示流表：

leaf1(config)# sh openflow instance l1 flow-table

 OpenFlow Instance Flow Table Information

 Table                       Flow     Available Free Miss
 ID    Table Name            Count    Flow Count     Count         Goto Table
 ----- --------------------- -------- -------------- ------------- -------------
 0     Start                 1        NA             0             100
 100   Policy Table          5        NA             722           200
 200   SW Table 1            1        NA             0             *

 * Denotes that the pipeline could end here.

在开始表中显示流规则：

leaf1(of-inst-l1)# sh openflow instance l1 flows flow-table 0

 OpenFlow Flow Table

 Flow 1
 Match
  Incoming Port : Any                   Ethernet Type    : Any
  Source MAC    : Any                   Destination MAC  : Any
  Source MAC Mask      : 000000-000000
  Destination MAC Mask : 000000-000000
  VLAN ID       : Any                   VLAN Priority    : Any
  Source IP Address      : Any
  Destination IP Address : Any
  IP Protocol   : Any
  IP ECN        : Any                   IP DSCP          : Any
  Source Port   : Any                   Destination Port : Any
  Source Port Range      : NA
  Destination Port Range : NA
  TCP Flags     : NA
  Custom Match One         : Any
  Custom Match One Mask    : Any
  Custom Match Two         : Any
  Custom Match Two Mask    : Any
  Custom Match Three       : Any
  Custom Match Three Mask  : Any
  Custom Match Four        : Any
  Custom Match Four Mask   : Any
 Attributes
  Priority      : 0                     Duration         : 16 seconds
  Hard Timeout  : 0 seconds             Idle Timeout     : 0 seconds
  Byte Count    : NA                    Packet Count     : NA
  Flow Table ID : 0                     Controller ID    : 1
  Cookie        : 0x0
  Hardware Index: NA
 Instructions
   Goto Table ID                 : 100

在策略表中显示流规则：

leaf1(config)# sh openflow instance l1 flows flow-table 100

 OpenFlow Flow Table

 Flow 1
 Match
  Incoming Port : 27                    Ethernet Type    : ARP
  Source MAC    : Any                   Destination MAC  : Any
  Source MAC Mask      : 000000-000000
  Destination MAC Mask : 000000-000000
  VLAN ID       : Any                   VLAN Priority    : Any
  Source Protocol Address   : Any
  Target Protocol Address   : Any
  Source Hardware Address      : Any
  Source Hardware Address Mask : 000000-000000
  Target Hardware Address      : Any
  Target Hardware Address Mask : 000000-000000
  ARP Opcode    : Any
  IP ECN        : Any                   IP DSCP          : Any
  Source Port   : Any                   Destination Port : Any
  Source Port Range      : NA
  Destination Port Range : NA
  TCP Flags     : NA
  TCP Mask      : NA
  Custom Match One         : Any
  Custom Match One Mask    : Any
  Custom Match Two         : Any
  Custom Match Two Mask    : Any
  Custom Match Three       : Any
  Custom Match Three Mask  : Any
  Custom Match Four        : Any
  Custom Match Four Mask   : Any
 Attributes
  Priority      : 1                     Duration         : 5239 seconds
  Hard Timeout  : 0 seconds             Idle Timeout     : 0 seconds
  Byte Count    : NA                    Packet Count     : 0
  Flow Table ID : 100                   Controller ID    : 1
  Cookie        : 0x0
  Hardware Index: 0
 Instructions
   Apply Actions
     Output                      : 25

 Flow 2
 Match
  Incoming Port : 25                    Ethernet Type    : ARP
  Source MAC    : Any                   Destination MAC  : Any
  Source MAC Mask      : 000000-000000
  Destination MAC Mask : 000000-000000
  VLAN ID       : Any                   VLAN Priority    : Any
  Source Protocol Address   : Any
  Target Protocol Address   : Any
  Source Hardware Address      : Any
  Source Hardware Address Mask : 000000-000000
  Target Hardware Address      : Any
  Target Hardware Address Mask : 000000-000000
  ARP Opcode    : Any
  IP ECN        : Any                   IP DSCP          : Any
  Source Port   : Any                   Destination Port : Any
  Source Port Range      : NA
  Destination Port Range : NA
  TCP Flags     : NA
  TCP Mask      : NA
  Custom Match One         : Any
  Custom Match One Mask    : Any
  Custom Match Two         : Any
  Custom Match Two Mask    : Any
  Custom Match Three       : Any
  Custom Match Three Mask  : Any
  Custom Match Four        : Any
  Custom Match Four Mask   : Any
 Attributes
  Priority      : 1                     Duration         : 5239 seconds
  Hard Timeout  : 0 seconds             Idle Timeout     : 0 seconds
  Byte Count    : NA                    Packet Count     : 0
  Flow Table ID : 100                   Controller ID    : 1
  Cookie        : 0x0
  Hardware Index: 0
 Instructions
   Apply Actions
     Output                      : 27

 Flow 3
 Match
  Incoming Port : 27                    Ethernet Type    : Any
  Source MAC    : Any                   Destination MAC  : Any
  Source MAC Mask      : 000000-000000
  Destination MAC Mask : 000000-000000
  VLAN ID       : Any                   VLAN Priority    : Any
  Source IP Address      : Any
  Destination IP Address : Any
  IP Protocol   : Any
  IP ECN        : Any                   IP DSCP          : Any
  Source Port   : Any                   Destination Port : Any
  Source Port Range      : NA
  Destination Port Range : NA
  TCP Flags     : NA
  TCP Mask      : NA
  Custom Match One         : Any
  Custom Match One Mask    : Any
  Custom Match Two         : Any
  Custom Match Two Mask    : Any
  Custom Match Three       : Any
  Custom Match Three Mask  : Any
  Custom Match Four        : Any
  Custom Match Four Mask   : Any
 Attributes
  Priority      : 65500                 Duration         : 5239 seconds
  Hard Timeout  : 0 seconds             Idle Timeout     : 0 seconds
  Byte Count    : NA                    Packet Count     : 17202875
  Flow Table ID : 100                   Controller ID    : 1
  Cookie        : 0x0
  Hardware Index: 0
 Instructions
   Apply Actions
     Output                      : 25

 Flow 4
 Match
  Incoming Port : 25                    Ethernet Type    : Any
  Source MAC    : Any                   Destination MAC  : Any
  Source MAC Mask      : 000000-000000
  Destination MAC Mask : 000000-000000
  VLAN ID       : Any                   VLAN Priority    : Any
  Source IP Address      : Any
  Destination IP Address : Any
  IP Protocol   : Any
  IP ECN        : Any                   IP DSCP          : Any
  Source Port   : Any                   Destination Port : Any
  Source Port Range      : NA
  Destination Port Range : NA
  TCP Flags     : NA
  TCP Mask      : NA
  Custom Match One         : Any
  Custom Match One Mask    : Any
  Custom Match Two         : Any
  Custom Match Two Mask    : Any
  Custom Match Three       : Any
  Custom Match Three Mask  : Any
  Custom Match Four        : Any
  Custom Match Four Mask   : Any
 Attributes
  Priority      : 65500                 Duration         : 5239 seconds
  Hard Timeout  : 0 seconds             Idle Timeout     : 0 seconds
  Byte Count    : NA                    Packet Count     : 498479217
  Flow Table ID : 100                   Controller ID    : 1
  Cookie        : 0x0
  Hardware Index: 0
 Instructions
   Apply Actions
     Output                      : 27

 Flow 5
 Match
  Incoming Port : Any                   Ethernet Type    : Any
  Source MAC    : Any                   Destination MAC  : Any
  Source MAC Mask      : 000000-000000
  Destination MAC Mask : 000000-000000
  VLAN ID       : Any                   VLAN Priority    : Any
  Source IP Address      : Any
  Destination IP Address : Any
  IP Protocol   : Any
  IP ECN        : Any                   IP DSCP          : Any
  Source Port   : Any                   Destination Port : Any
  Source Port Range      : NA
  Destination Port Range : NA
  TCP Flags     : NA
  TCP Mask      : NA
  Custom Match One         : Any
  Custom Match One Mask    : Any
  Custom Match Two         : Any
  Custom Match Two Mask    : Any
  Custom Match Three       : Any
  Custom Match Three Mask  : Any
  Custom Match Four        : Any
  Custom Match Four Mask   : Any
 Attributes
  Priority      : 0                     Duration         : 5239 seconds
  Hard Timeout  : 0 seconds             Idle Timeout     : 0 seconds
  Byte Count    : NA                    Packet Count     : 726
  Flow Table ID : 100                   Controller ID    : 1
  Cookie        : 0x0
  Hardware Index: NA
 Instructions
   Apply Actions
    Controller Port

在 sw-table-1 中显示流规则：

leaf1(of-inst-l1)# sh openflow instance l1 flows flow-table 200

 OpenFlow Flow Table

 Flow 1
 Match
  Incoming Port : Any                   Ethernet Type    : Any
  Source MAC    : Any                   Destination MAC  : Any
  Source MAC Mask      : 000000-000000
  Destination MAC Mask : 000000-000000
  VLAN ID       : Any                   VLAN Priority    : Any
  Source IP Address      : Any
  Destination IP Address : Any
  IP Protocol   : Any
  IP ECN        : Any                   IP DSCP          : Any
  Source Port   : Any                   Destination Port : Any
  Source Port Range      : NA
  Destination Port Range : NA
  TCP Flags     : NA
  TCP Mask      : NA
  Custom Match One         : Any
  Custom Match One Mask    : Any
  Custom Match Two         : Any
  Custom Match Two Mask    : Any
  Custom Match Three       : Any
  Custom Match Three Mask  : Any
  Custom Match Four        : Any
  Custom Match Four Mask   : Any
 Attributes
  Priority      : 0                     Duration         : 56 seconds
  Hard Timeout  : 0 seconds             Idle Timeout     : 0 seconds
  Byte Count    : 0                     Packet Count     : 0
  Flow Table ID : 200                   Controller ID    : NA
  Cookie        : 0x0
  Hardware Index: NA
 Instructions
    Drop

现在显示每个端口 25 和 27 的数据包统计信息。当我运行 TCP 流 40 秒时，您将在端口 25 看到数千个 Rx Discards：

leaf1(of-inst-l1)# sh int 25

 Status and Counters - Port Counters for port 25

  Name  :
  MAC Address      : 548028-5c79a7
  Link Status      : Up
  Port Enabled     : Yes
  Totals (Since boot or last clear) :
   Bytes Rx        : 2,260,064,239        Bytes Tx        : 64,489,408
   Unicast Rx      : 26,948,680           Unicast Tx      : 918,509
   Bcast/Mcast Rx  : 46                   Bcast/Mcast Tx  : 2
  Errors (Since boot or last clear) :
   FCS Rx          : 0                    Drops Tx        : 0
   Alignment Rx    : 0                    Collisions Tx   : 0
   Runts Rx        : 0                    Late Colln Tx   : 0
   Giants Rx       : 0                    Excessive Colln : 0
   Total Rx Errors : 0                    Deferred Tx     : 0
  Others (Since boot or last clear) :
   Discard Rx      : 5,704                Out Queue Len   : 0
   Unknown Protos  : 0
  Rates (5 minute weighted average) :
   Total Rx(Kbps) : 2,149,080              Total Tx(Kbps) : 4,320
   Unicast Rx (Pkts/sec) : 174,967        Unicast Tx (Pkts/sec) : 6,433
   B/Mcast Rx (Pkts/sec) : 0              B/Mcast Tx (Pkts/sec) : 0
   Utilization Rx  : 21.49 %              Utilization Tx  : 00.04 %


leaf1(of-inst-l1)# sh int 27

 Status and Counters - Port Counters for port 27

  Name  :
  MAC Address      : 548028-5c79a5
  Link Status      : Up
  Port Enabled     : Yes
  Totals (Since boot or last clear) :
   Bytes Rx        : 64,489,042           Bytes Tx        : 2,251,406,709
   Unicast Rx      : 918,509              Unicast Tx      : 26,948,680
   Bcast/Mcast Rx  : 2                    Bcast/Mcast Tx  : 47
  Errors (Since boot or last clear) :
   FCS Rx          : 0                    Drops Tx        : 0
   Alignment Rx    : 0                    Collisions Tx   : 0
   Runts Rx        : 0                    Late Colln Tx   : 0
   Giants Rx       : 0                    Excessive Colln : 0
   Total Rx Errors : 0                    Deferred Tx     : 0
  Others (Since boot or last clear) :
   Discard Rx      : 0                    Out Queue Len   : 0
   Unknown Protos  : 0
  Rates (5 minute weighted average) :
   Total Rx(Kbps) : 4,256                  Total Tx(Kbps) : 2,118,904
   Unicast Rx (Pkts/sec) : 6,342          Unicast Tx (Pkts/sec) : 172,523
   B/Mcast Rx (Pkts/sec) : 0              B/Mcast Tx (Pkts/sec) : 0
   Utilization Rx  : 00.04 %              Utilization Tx  : 21.18 %

我的流规则都在硬件中，交换机应该能够以线速（即 10Gbps）处理数据包，并且不应该显示任何 Rx 丢弃。任何帮助将不胜感激！

自定义管道模型中的性能

0个回答
没有发现任何回复~
其它你可能感兴趣的问题
上一篇NAT 重叠 下一篇何时在接口上禁用 CEF