Quality of Service on Linux for JPS ATLAS the Atlas TDAQ Event - - PowerPoint PPT Presentation

▶
quality of service on linux for
SMART_READER_LITE
LIVE PREVIEW

Quality of Service on Linux for JPS ATLAS the Atlas TDAQ Event - - PowerPoint PPT Presentation

Feb 07, 2023 439 likes •665 views

Quality of Service on Linux for JPS ATLAS the Atlas TDAQ Event Building Network Okinawa TDAQ Y.Yasu (1) , A.Manabe (1) , Y.Nagasaka (2) , H.Hasegawa (3) , M.Shimojima (4) , M.Nomachi (5) , H.Fujii (1) and Y.Watase (1) on behalf of the Atlas

slide-1
SLIDE 1

September 24, 2001 Y.Yasu 1

ATLAS TDAQ JPS Okinawa

Quality of Service on Linux for the Atlas TDAQ Event Building Network

Y.Yasu(1), A.Manabe(1), Y.Nagasaka(2), H.Hasegawa(3), M.Shimojima (4), M.Nomachi(5), H.Fujii(1) and Y.Watase(1)

  • n behalf of the Atlas Trigger/DAQ group

(1)High Energy Accelerator Research Organization (KEK) Oho 1-1, Tsukuba, Ibaraki 305 Japan (2) Department of Information and Intellectual Systems Engineering Faculty

  • f Engineering Hiroshima Institute of Technology

2-1-1 Miyake, Saeki-ku, Hiroshima 731-5193 Japan (3)Faculty of Science, Shinshu University Asahi 3-1-1, Matsumoto, 390-8621 Nagano Japan (4)Department of Electrical Engineering Nagasaki Institute of Applied Science (NIAS) 536 Aba-machi, Nagasaki 851-01 Japan (5)Graduate School of Science, Osaka University Machikaneyama 1-1, Toyanaka-city, 560-0043 Osaka Japan

slide-2
SLIDE 2

September 24, 2001 Y.Yasu 2

ATLAS TDAQ

Contents

Why QoS is necessary What QoS is Performance Measurements Packet loss on UDP/IP multicast transfer QoS software overhead on TCP/IP transfer Packet distribution on TCP/IP transfer Conclusion

JPS Okinawa

slide-3
SLIDE 3

September 24, 2001 Y.Yasu 3

ATLAS Event Builder System

ATLAS TDAQ JPS Okinawa

slide-4
SLIDE 4

September 24, 2001 Y.Yasu 4

ATLAS TDAQ

Why QoS is necessary

> UDP/IP multicast transfer is attractive for broadcasting messages to Event Builder Sources from scalability point of view, but UDP/IP may lose the messages(packets). Data Flow Manager Event Builder Sources > Constant Bit Rate of ATM network can eliminate the packet loss by managing the bandwidth. > QoS in Linux kernel can manage the bandwidth. Can the QoS eliminate the packet loss?

JPS Okinawa

slide-5
SLIDE 5

September 24, 2001 Y.Yasu 5

ATLAS TDAQ

CBQ can classify incoming packets into multiple classes. CBQ can share and limit the transfer rate.

What QoS is:Class-Based Queueing (CBQ)

estimator Set over-limit classifier Class 1 Class 2 Class 3 packet Packet scheduler

JPS Okinawa

slide-6
SLIDE 6

September 24, 2001 Y.Yasu 6

TBF can limit the transfer rate according to the token rate

What QoS is :Token Bucket Filter (TBF)

Data packet Data Buffer Token buffer rate Bucket size

ATLAS TDAQ JPS Okinawa

slide-7
SLIDE 7

September 24, 2001 Y.Yasu 7

What QoS is :Traffic control in Linux kernel

Input de-multiplexing Forwarding Output queueing Upper layers(TCP,UDP,...) Traffic Control

Traffic control is done only at output queueing. Outgoing packet is only controlled.

ATLAS TDAQ JPS Okinawa

slide-8
SLIDE 8

September 24, 2001 Y.Yasu 8

ATLAS TDAQ

What QoS is :Queueing discipline for Linux

Class(CBQ) 1:1 bandwidth:1000Mbit/s Qdisc (TBF) bandwidth:100Mbit/s Classifier u32: Specified IP Address

U32 can specify IP address to manage the bandwidth. TBF limits the bandwidth to 100Mbit/s on 1000Mbit/s network at this example.

Root Qdisc(CBQ) 1: bandwidth :1000Mbit/s

JPS Okinawa

slide-9
SLIDE 9

September 24, 2001 Y.Yasu 9

ATLAS TDAQ

Performance measurement

Packet loss measurement Measurement of QoS software overhead Packet distribution measurement

JPS Okinawa

slide-10
SLIDE 10

September 24, 2001 Y.Yasu 10

Testbed

PC500 PC500 PC500 PC500 3Com Super StackII Switch 9300

ATLAS TDAQ 12 ports

Linux kernel version : 2.4.5 gcc version: egcs-2.91-66 acenic driver version : 0.8

Configuration of PC500 system

CPU: PentiumIII/500MHz Chipset 440GX Memory: 100MHz/SDRAM/256MB PCIbus: 32-bit/33MHz NIC: AceNIC(1MB)

JPS Okinawa

slide-11
SLIDE 11

September 24, 2001 Y.Yasu 11 PC500 PC500 PC500 PC500 3Com Super StackII Switch 9300

ATLAS TDAQ 12 ports

Packet loss on UDP/IP multicast transfer

Sender Receiver Receiver Receiver

Sender sends message with 10 bytes to receivers in best effort.

JPS Okinawa

slide-12
SLIDE 12

September 24, 2001 Y.Yasu 12

ATLAS TDAQ

200 400 600 800 1000 1200 300 600 900 1200 1500 10 20 30 40 50 60 70 80 90 100

Message size in bytes Transfer speed in Mbit/s Packet loss in percentage

Packet loss on UDP/IP multicast transfer (Cont.)

No QoS packet loss in any size

JPS Okinawa

slide-13
SLIDE 13

September 24, 2001 Y.Yasu 13

ATLAS TDAQ

2 4 6 8 10 12 300 600 900 1200 1500 20 40 60 80 100

Message size in bytes

Assigned bandwidth = 10 Mbit/s

Packet loss on UDP/IP multicast transfer(Cont.)

No packet loss in any size

Transfer speed in Mbit/s Packet loss in percentage

JPS Okinawa

slide-14
SLIDE 14

September 24, 2001 Y.Yasu 14

10 20 30 40 50 20 40 60 80 100 20 40 60 80 100

ATLAS TDAQ

Message size in bytes

Assigned bandwidth = 50 Mbit/s

Packet loss on UDP/IP multicast transfer(Cont.)

packet loss in 60-byte size

Transfer speed in Mbit/s Packet loss in percentage

JPS Okinawa

slide-15
SLIDE 15

September 24, 2001 Y.Yasu 15

20 40 60 80 100 120 300 600 900 1200 1500 20 40 60 80 100

ATLAS TDAQ

Message size in bytes

Assigned bandwidth = 100 Mbit/s

Packet loss on UDP/IP multicast transfer(Cont.)

No packet loss out of 200-byte size

Transfer speed in Mbit/s Packet loss in percentage

JPS Okinawa

slide-16
SLIDE 16

September 24, 2001 Y.Yasu 16

QoS Software Overhead

PC500 PC500 PC500 PC500 3Com Super StackII Switch 9300

ATLAS TDAQ 12 ports

Sender Receiver

Assigned bandwidth is 1000Mbit/s. Sender sends message with 10 bytes to a receiver in best effort. This bandwidth does not limit the transfer, but it consumes CPU time.

JPS Okinawa

slide-17
SLIDE 17

September 24, 2001 Y.Yasu 17

100 200 300 400 500 600 700 10 100 1000 10000 No QoS QoS

ATLAS TDAQ

QoS Software Overhead: TCP/IP transfer speed

Message size in bytes Transfer speed in Mbit/sec.

Peak speed was same.

JPS Okinawa

slide-18
SLIDE 18

September 24, 2001 Y.Yasu 18

60 70 80 90 100 110 10 100 1000 10000 sender cpu usage(No QoS) sender cpu usage(QoS)

ATLAS TDAQ

QoS Software Overhead: the CPU Usage

Message size in bytes CPU Usage (%)

In small size of message, QoS was 10 % worse. In large size of message, the percentage increases.

JPS Okinawa

slide-19
SLIDE 19

September 24, 2001 Y.Yasu 19

TCP/IP Packet Distribution

PC500 PC500 PC500 PC500 3Com Super StackII Switch 9300

ATLAS TDAQ 12 ports

Sender Receiver

Assigned bandwidth is 10Mbit/s. Sender sends message with 10 bytes to a receiver. tcpdump utility is used for capturing packets

JPS Okinawa

slide-20
SLIDE 20

September 24, 2001 Y.Yasu 20

ATLAS TDAQ

TCP/IP Packet Distribution(Cont.)

QoS No QoS Horizontal axis : sending packet time interval With QoS, some packets were sent at 10 msec scheduling time. Without QoS, packets are mostly sent in several 10 usec.

JPS Okinawa

slide-21
SLIDE 21

September 24, 2001 Y.Yasu 21

ATLAS TDAQ

TCP/IP Packet Distribution(Cont.)

With 1kHz trigger, packets are mostly sent around 1msec. There are no difference between QoS and NoQoS. QoS with 1kHz trigger rate No QoS with 1kHz trigger rate

JPS Okinawa

slide-22
SLIDE 22

September 24, 2001 Y.Yasu 22

ATLAS TDAQ

Conclusion

We measured with/without QoS in Linux kernel: 1) Packet loss on UDP/IP multicast transfer 2) QoS software overhead on TCP/IP transfer 3) Packet distribution on TCP/IP transfer We concluded: 1) QoS could eliminate packet loss on UDP/IP multicast transfer. This shows the feasibility to solve the scalability issue

  • f ATLAS TDAQ Event Builder.

2) QoS was efficiently performed on TCP/IP transfer. CPU usage of QoS on the transfer was small. 3) With QoS, some packets were sent at 10 msec scheduling time.

JPS Okinawa