acceleration Proceedings of netdev 0.1, Feb 14-17, 2015, Ottawa, On, - - PowerPoint PPT Presentation

IPQ806x Hardware acceleration

Proceedings of netdev 0.1, Feb 14-17, 2015, Ottawa, On, Canada

• Features
• Designed for Home Gateways (CPE)
• Flow detection based “All-or-nothing”
• ffload
• Acceleration supports:
• IPv4, IPv6, NAT, PPPoE, L2TP, VLAN, Qdisc
• Performance gain
• Linux: 640k pps (bridged) – 220kpps (routed)
• NSS: 7200k pps (bridged) – 7200kpps (routed)
• 11x (bridged) – 32x (routed)
• Functional behavior
• 0% cpu load seen in Linux
• Keep Linux counters up to date
• Does not require functional changes at an

upper level (user space)

NSS acceleration model

Proceedings of netdev 0.1, Feb 14-17, 2015, Ottawa, On, Canada

• Inspect all outgoing packets at POSTROUTING chain by registering post routing hooks.
• Inspect conntrack and device events to destroy and regenerate connection.
• Inspect NSS status and stats to update the connection state and statistics info in Linux and ECM DB.

ECM Front End Inspect Packets and Events

FRONT END NETFILTER PF_BRIDGE POST ROUTING L2 PKTs L2 PKTs NETFILTER IPvX POST ROUTING L3 PKTs L3 PKTs INTERFACE NOTIFIER MTU / UP/ DOWN / BOND NOTIFIER Bond Link/ Release/ Enslave NSS Driver NSS Status and Stats Sync NSS Firmware LINUX CONNTRACK IPCT_DESTROY / IPCT_MARK CONNTRACK NOTIFIER

Proceedings of netdev 0.1, Feb 14-17, 2015, Ottawa, On, Canada

Example of IPv4 TCP rule creation

NSS Firmware Linux

TCP Data The Accelerated TCP flow TCP: ACK Or Syn+ACK

Transmit the packet

Add the rule & reply the establish cmd Match Any Accelerated Rule? No

ECM

Send the Packet to Linux Create IPv4 Rule

Packet

Linux net stack

Netfiter Prerouting Routing or Bridging Netfiter Postrouting

NSS Driver

Connection Established

Check TCP Connection can be accel?

Proceedings of netdev 0.1, Feb 14-17, 2015, Ottawa, On, Canada

Example of IPv4 TCP rule destroy

NSS Firmware

Linux Kernel TCP Data The Accelerated TCP flow TCP: FIN

Packet

Send the Packet to Linux Transmit The packet Destroy IPv4 Rule IPv4 Rule Sync Add the rule & reply the establish cmd Is it a connected TCP Data? No

Linux net stack

Netfiter Prerouting Routing / Bridging Netfiter Postrouting

ECM

Remove the connection

NSS Driver

Conntrack Conntrack destroy event Connection Destroy ?

Proceedings of netdev 0.1, Feb 14-17, 2015, Ottawa, On, Canada

Example of IPv4 rule API

• Common messaging interfaces
• Protocol type: IPv4, IPv6, PPP, LAG…
• Rule type: CREATE, DESTROY, CONN_STAT_SYNC , NODE_STAT_SYNC…
• Callback/args pointers: will be passed back in the FW ACK/NACK reply
• IPv4 rule create message structure example

struct nss_ipv4_rule_create_msg { /* Request */ uint16_t valid_flags; /* Indicate which of the parameters below is filled-in Indirectly says which operation to be done on the flow */ uint16_t rule_flags; /* Bit flags associated with the rule */ struct nss_ipv4_5tuple tuple; /* src_ip, dst_ip, src_port, dst_port, proto */ struct nss_ipv4_connection_rule conn_rule; /* src_mac, dst_mac, src_iface, dst_iface, src_mtu, dst_mtu, nat_src_ip, nat_dst_ip, nat_src_port, nat_dst_port */ struct nss_ipv4_protocol_tcp_rule tcp_rule; /* TCP related accleration parameters */ struct nss_ipv4_pppoe_rule pppoe_rule; /* flow_session_id, flow_remote_mac, ret_session_id, ret_remote_mac */ struct nss_ipv4_qos_rule qos_rule; /* flow_qos_tag, ret_qos_tag, */ struct nss_ipv4_dscp_rule dscp_rule; /* flow_dscp, ret_dstp */ struct nss_ipv4_vlan_rule vlan_primary_rule; /* ingress_vlan_tag, egress_vlan_tag */ struct nss_ipv4_vlan_rule vlan_secondary_rule; /* ingress_vlan_tag, egress_vlan_tag – for QinQ */ /* Response */ uint32_t index; /*Slot ID for cache stats to host OS */ };

Proceedings of netdev 0.1, Feb 14-17, 2015, Ottawa, On, Canada

Linux

ECM

Interfaces & Connections statistics update

• Stats updates sent

periodically from the Firmware

• Per-interfaces stats

(update net_devices)

• Per-connections stats

(update conntracks)

• Minor modifications to

ppp/l2tp/ipsec… layers for iface look-up and stats update

NSS Driver NODE_STATS_SYNC message CONN_STATS_SYNC message ppp0 ethN tun0 br0

Look-up interface

NSS Firmware

Look-up conntrack

stats++ conntracks stats++

Proceedings of netdev 0.1, Feb 14-17, 2015, Ottawa, On, Canada

Qdisc acceleration

# tc qdisc add dev eth0 root handle 1: tbf rate 1000Mbit burst 100k limit 100 # tc qdisc add dev eth0 parent 1: handle 10: prio bands 3 # tc qdisc add dev eth0 parent 10:1 handle 100: tbf rate 2Mbit burst 10k limit 100 # tc qdisc add dev eth0 parent 100: handle 1000: pfifo limit 100 # tc qdisc add dev eth0 parent 10:2 handle 200: tbf rate 40Mbit burst 30k limit 100 # tc qdisc add dev eth0 parent 200: handle 2000: red limit 100k min 30k max 80k \ avpkt 1k burst 55 probability 0.20 # tc qdisc add dev eth0 parent 10:3 handle 300: red limit 100k min 30k max 80k \ avpkt 1k burst 55 probability 0.30

tbf tbf tbf r e d prio r e d p f i f

• 1:

10: 100: 200: 300: 1000: 2000: 10:3 10:2 10:1

nsstbf nsstbl nsstbl n s s r e d nssprio n s s r e d n s s p f i f

• 1:

10: 100: 200: 300: 1000: 2000: 10:3 10:2 10:1

# tc qdisc add dev eth0 root handle 1: nsstbl rate 1000Mbit burst 100k # tc qdisc add dev eth0 parent 1: handle 10: nssprio bands 3 # tc qdisc add dev eth0 parent 10:1 handle 100: nsstbl rate 2Mbit burst 10k # tc qdisc add dev eth0 parent 100: handle 1000: nsspfifo limit 100 # tc qdisc add dev eth0 parent 10:2 handle 200: nsstbl rate 40Mbit burst 30k # tc qdisc add dev eth0 parent 200: handle 2000: nssred limit 100k min 30k max 80k \ avpkt 1k burst 55 probability 0.50 # tc qdisc add dev eth0 parent 10:3 handle 300: nssred limit 100k min 30k max 80k \ avpkt 1k burst 55 probability 0.50 set_default

Proceedings of netdev 0.1, Feb 14-17, 2015, Ottawa, On, Canada