Quality of Service on Gigabit Ethernet for Event Builder Y.Yasu (1) - - PowerPoint PPT Presentation

▶
quality of service on gigabit ethernet for event builder
SMART_READER_LITE
LIVE PREVIEW

Quality of Service on Gigabit Ethernet for Event Builder Y.Yasu (1) - - PowerPoint PPT Presentation

Jun 05, 2023 159 likes •399 views

DAQ2000, Lyon ATLAS-Japan DAQ Group Quality of Service on Gigabit Ethernet for Event Builder Y.Yasu (1) , Y.Nagasaka (2) , H.Hasegawa (3) , A.Manabe (1) , M.Nomachi (4) , H.Fujii (1) , Y.Watase (1) (1)High Energy Accelerator Research

slide-1
SLIDE 1

October 20, 2000 Y.Yasu 1

ATLAS-Japan DAQ Group DAQ2000, Lyon

Quality of Service on Gigabit Ethernet for Event Builder

Y.Yasu(1), Y.Nagasaka(2), H.Hasegawa(3), A.Manabe(1), M.Nomachi(4), H.Fujii(1), Y.Watase(1)

(1)High Energy Accelerator Research Organization (KEK) Oho 1-1, Tsukuba, Ibaraki 305 Japan (2)Department of Electrical Engineering Nagasaki Institute of Applied Science (NIAS) 536 Aba-machi, Nagasaki 851-01 Nagasaki Japan (3)Faculty of Science Shinshu University Asahi 3-1-1, Matsumoto, 390-8621 Nagano Japan (4)Graduate School of Science Osaka University Machikaneyama 1-1, Toyanaka-city, 560-0043 Osaka Japan

slide-2
SLIDE 2

October 20, 2000 Y.Yasu 2

ATLAS-Japan DAQ Group DAQ2000, Lyon

Contents

Gigabit Ethernet and Quality of Service(QoS) for Event Builder Basic performance of Gigabit Ethernet QoS in Linux QoS performance on Gigabit Ethernet

slide-3
SLIDE 3

October 20, 2000 Y.Yasu 3

ATLAS-Japan DAQ Group DAQ2000, Lyon

Congestion Avoidance by Global Traffic Control(Fermilab, KEK) Special Hardware with Barrel shifter etc. by traffic shaping for each node(RD31/ CERN) ATM including QoS(Constant Bit Rate)

Gigabit Ethernet and Quality of Service for Event Builder

How can we use Gigabit Ethernet with TCP/IP ?

slide-4
SLIDE 4

October 20, 2000 Y.Yasu 4

ATLAS-Japan DAQ Group DAQ2000, Lyon

Basic performance of Gigabit Ethernet

Hardware configuration and setup Performance measurement of memory copy Tuning network driver Performance measurement of Gigabit Ethernet

slide-5
SLIDE 5

October 20, 2000 Y.Yasu 5

ATLAS-Japan DAQ Group DAQ2000, Lyon

Hardware configuration

v0.47 v0.47 v0.47 Network Driver version AceNIC(1MB) AceNIC(1MB) AceNIC(1MB) NIC egcs-2.91.66 egcs-2.91.66 egcs-2.91.66 gcc version 2.2.14-5 2.2.14-12 2.2.14-12 OS version 64-bit/32-bit 32-bit 32-bit PCI(bus width) 128 256 256 Memory size SDRAM SDRAM SDRAM Memory type 133MHz 100MHz 100MHz Bus speed 440BX SuperWorks ServerSet III LE 440GX Chipset 256KB 256KB 512KB Cache 800MHz 733MHz 500MHz CPU Freq. PentiumIII PentiumIII PentiumIII CPU type PC800 PC733 PC500 System

slide-6
SLIDE 6

October 20, 2000 Y.Yasu 6 50 100 150 200 250 300 350 400 450 500 0.1 1 10 100 PentiumIII- 733MHz/SuperWorks PentiumIII-500MHz/440GX PentiumIII-800MHz/440BX

ATLAS-Japan DAQ Group DAQ2000, Lyon

Performance measurement of memory copy

Memory size in MB Memory-copy speed in MB/s by memcpy system call

slide-7
SLIDE 7

October 20, 2000 Y.Yasu 7

ATLAS-Japan DAQ Group DAQ2000, Lyon

Hardware setup

PC500 PC500 PC500 PC500 Alteon Switch 180 PC733 PC800 PC733 PC800

For measurement of QoS performance and measurement of basic performance For measurement of basic performance

slide-8
SLIDE 8

October 20, 2000 Y.Yasu 8

100 200 300 400 500 600 700 800 900 1000 0.01 1 100 10000 tx=20:rx=30 tx=200:rx=300 tx=400:rx=600 tx=2:rx=3 100 200 300 400 500 600 700 800 900 1000 0.01 1 100 10000 tx=400:rx=120 tx=996:rx=1000 tx=2000:rx=2000 tx=40:rx=12

ATLAS-Japan DAQ Group DAQ2000, Lyon

Tuning network driver

Message size in KB Transfer speed in Mbit/s Message size in KB Transfer speed in Mbit/s Tuned parameters of Network driver : /proc/sys/net/core/rmem_max 4000000 /proc/sys/net/core/wmem_max 4000000 TCP buffer size : 524288 bytes Nagle algorithm is enable. Normal frame : tx_coal=996 : rx_coal=1000 Jumbo frame : tx_coal=200 : rx_coal=300 Other parameters : default Used systems : PC500s Measuring tool : Netperf Normal frame Jumbo frame

slide-9
SLIDE 9

October 20, 2000 Y.Yasu 9

ATLAS-Japan DAQ Group DAQ2000, Lyon

Transfer speed in normal frame Transfer speed in jumbo frame Bus width dependency(64-bit PCI vs. 32-bit PCI)

Performance measurement of Gigabit Ethernet

slide-10
SLIDE 10

October 20, 2000 Y.Yasu 10

100 200 300 400 500 600 700 800 900 1000 0.01 0.1 1 10 100 1000 Pentium500MHz Pentium733MHz Pentium800MHz

ATLAS-Japan DAQ Group DAQ2000, Lyon

Transfer speed in normal frame

Transfer speed does not depend on CPU speed in normal frame

Message size in KB Transfer speed in Mbit/s

slide-11
SLIDE 11

October 20, 2000 Y.Yasu 11

100 200 300 400 500 600 700 800 900 1000 0.01 1 100 10000 Pentium500MHz Pentium733MHz Pentium800MHz

ATLAS-Japan DAQ Group DAQ2000, Lyon

Transfer speed in jumbo frame

Transfer speed depends on Chipset performance in jumbo frame.

Message size in KB Transfer speed in Mbit/s

slide-12
SLIDE 12

October 20, 2000 Y.Yasu 12

100 200 300 400 500 600 700 800 900 1000 0.01 1 100 10000 32-bit PCI(mtu1500) 64-bit PCI(mtu1500) 32-bit PCI(mtu9000) 64-bit PCI(mtu9000)

ATLAS-Japan DAQ Group DAQ2000, Lyon

Bus width dependency(64-bit PCI vs. 32-bit PCI)

Message size in KB Transfer speed in Mbit/s

Bus width does not affect transfer speed in normal frame. 64-bit PCI improves transfer speed in jumbo frame, up to 990Mbit/s.

On PC733s

slide-13
SLIDE 13

October 20, 2000 Y.Yasu 13

ATLAS-Japan DAQ Group DAQ2000, Lyon

QoS performance on Gigabit Ethernet 1x1 system 3x1 system 1x3 system QoS in Linux Class-Based Queueing Token Bucket Filter Traffic control in TCP/IP Queueing discipline for Linux

slide-14
SLIDE 14

October 20, 2000 Y.Yasu 14

ATLAS-Japan DAQ Group DAQ2000, Lyon

CBQ can classify incoming packets into multiple classes. CBQ can share and limit the transfer rate.

Class-Based Queueing (CBQ)

estimator Set overlimit classifier Class 1 Class 2 Class 3 packet Packet scheduler

slide-15
SLIDE 15

October 20, 2000 Y.Yasu 15

ATLAS-Japan DAQ Group DAQ2000, Lyon

TBF can limit the transfer rate according to the token rate.

Token Bucket Filter (TBF)

Data packet Data Buffer Token buffer rate Bucket size

slide-16
SLIDE 16

October 20, 2000 Y.Yasu 16

ATLAS-Japan DAQ Group DAQ2000, Lyon

QoS in Linux

QoS in Linux kernel 2.2.x and Iproute2 for Linux supplies the following functions to control IP packets; For IP packets of inbound and outbound Classifying Prioritizing Sharing Limiting QoS in Linux kernel 2.2.x : in kernel standard distribution Iproute2 : ftp://ftp.inr.ac.ru/ip-routing/iproute2-current.tar.gz Iproute2 supports a typical command called tc. Can we use the QoS such as Class-Based Queueing and Token Bucket Filter for shaping traffic on Event Builder?

slide-17
SLIDE 17

October 20, 2000 Y.Yasu 17

ATLAS-Japan DAQ Group DAQ2000, Lyon

Traffic control in TCP/IP

Input de-multiplexing Forwarding Output queueing Upper layers(TCP,UDP,...) Traffic Control

Traffic control is done only at output queueing.

slide-18
SLIDE 18

October 20, 2000 Y.Yasu 18

ATLAS-Japan DAQ Group DAQ2000, Lyon

Queueing discipline for Linux

Ῡ Qdisc ̰ CBQ

1: bandwidth:1000Mbit Ῡ Class ̰ CBQ 1:1 rate:1000Mbit

Qdisc (TBF) Rate:100Mbit

Classifier u32: Specified IP Address

U32 is a classifier. TBF limits the transfer rate to 100Mbit/s at this example. The configuration was used at 1x1 system for the evaluation.

slide-19
SLIDE 19

October 20, 2000 Y.Yasu 19

100 200 300 400 500 600 100 200 300 400 500 600 Ideal rate Measured rate

ATLAS-Japan DAQ Group DAQ2000, Lyon

QoS performance on 1x1 system

Assigned rate in Mbit/s Measured rate in Mbit/s No QoS

Bandwidth assignment worked well. The overhead of QoS was very small and nearly 1 %.

On PC500s

slide-20
SLIDE 20

October 20, 2000 Y.Yasu 20

50 100 150 200 250 50 100 150 200 250 Ideal rate Measured rate on node1 Measured rate on node2 Measured rate on node3

ATLAS-Japan DAQ Group DAQ2000, Lyon

QoS performance on 3x1 system

Assigned rate in Mbit/s Measured rate in Mbit/s

Bandwidth assignment worked well

No QoS

On PC500s

slide-21
SLIDE 21

October 20, 2000 Y.Yasu 21

50 100 150 200 250 50 100 150 200 250 Ideal rate Measured rate on node1 Measured rate on node2 Measured rate on node3

ATLAS-Japan DAQ Group DAQ2000, Lyon

QoS performance on 1x3 system

Assigned rate in Mbit/s Measured rate in Mbit/s

Bandwidth assignment worked well at small rate, but did not work at full rate. Packet sharing performance without QoS was better than that with QoS.

No QoS On PC500s

slide-22
SLIDE 22

October 20, 2000 Y.Yasu 22

ATLAS-Japan DAQ Group DAQ2000, Lyon

Conclusion

* Traffic management of event data flow is necessary for event builder. * IP-based QoS with Gigabit Ethernet on PC/Linux was investigated. * The transfer speed did not depend on CPU speed in normal frame. * High performance chipset and 64-bit PCI improved the transfer speed in jumbo frame, up to 990Mbit/s. * Assignment of transfer rate was possible by using Linux/TC command, but it is not clear that the QoS on Gigabit Ethernet is effective for event builder. Thus, more investigation is necessary.