Accel Ac celeration of Er eration of Erasur asure e Cod Codin - - PowerPoint PPT Presentation

To Towar ards ds In In-networ network k Ac Accel celeration of Er eration of Erasur asure e Cod Codin ing

Yi Qiao, Xiao Kong, Menghao Zhang, Yu Zhou, Mingwei Xu, Jun Bi Tsinghua University

Eras rasure ure Coding ing (EC)

• In data centers, machine failures happen very frequently. Facebook

reports up to 50 machine failures per day in their data warehouses.

• EC provides data fault tolerance with much lower storage overheads

(~1.4x) than replication (3x), with similar degree of availability.

• EC reconstructs missing data with remaining data and pre-calculated

parities.

• For example:
• XOR (RAID 5)
• Reed-Solomon Codes

EC C Examples xamples

𝑏 𝑐 𝑑 𝑞 = 𝑏⨁𝑐⨁𝑑 Reconstruct b with 𝑐 = 𝑏⨁𝑑⨁𝑞 XOR (RAID 5) 𝑏 𝑐 𝑑 𝑞1 = 𝑏 + 𝑐 + 𝑑 𝑞2 = 𝑏 + 2𝑐 + 2𝑑 Reconstruct a with 𝑏 = 2𝑞1 − 𝑞2 Reconstruct c with 𝑑 = 𝑞2 − 𝑞1 − 𝑐 Reed Solomon Code (Conceptual) These are Galois Field arithmetics. For simplicity, just comprehend them as integer arithmetics.

• Conclusion: EC reconstruction can

be modelled with

𝑛 = ෍

𝑗=1 𝑙

𝑏𝑗𝑦𝑗 , 𝑛 : reconstructed symbol 𝑦𝑗 : symbols from remaining machines 𝑏𝑗 : pre-computed coefficients

• Addition refers to XOR
• Multiplication is on Galois Field

linear combinations

EC Pro roblems blems

• Low reconstruction rate
• Several hours to reconstruct a disk
• Several seconds for degraded reads
• EC is mostly used for storing “cold” data in data warehouses.
• Why so slow?

Motiva tivation tion

𝑩 𝑪𝟐 𝑪𝟑 𝑪𝟒

ToR

NetEC 𝑩 𝑪𝟐 𝑪𝟑 𝑪𝟒

ToR

Forward Forward DISK CPU Multiplexed NIC DISK CPU NIC

Disk Reconstruction Rate = 1/3

• f available NIC capacity

No NIC Sharing/multiplexing

Near 100% of available NIC capacity

Line width represents throughput

Ne NetE tEC

• We present NetEC that offloads EC reconstruction to

programmable switches.

• It improves reconstruction rates by k times, where k is the

number of the machines to download from.

• It also entirely removes CPU usage.

Bri rief ef Ov Over erview view of f Ne NetE tEC Da Data ta Pla lane ne

𝒚𝟐 𝒛𝟐

Partial XOR Sum Buffer Progress Tracker

𝒃𝟐𝒚𝟐+𝒃𝟑𝒚𝟑+𝒃𝟒𝒚𝟒 𝒃𝟐𝒛𝟐+𝒃𝟑𝒛𝟑+𝒃𝟒𝒛𝟒

𝐶1 𝐶2 𝐶3

…… …… On Switch Decoding Buffer … … Drop Drop

① ② ③ ⑤ ⑥ ⑦ ⑧

A

𝒃𝟐𝒚𝟐+𝒃𝟑𝒚𝟑+𝒃𝟒𝒚𝟒 111 𝒃𝟐𝒚𝟐 100 𝒃𝟐𝒚𝟐+𝒃𝟑𝒚𝟑 110 000

P1 P2 P3 P1 arrives P2 arrives P3 arrives

GF Mult.

… …

𝒚𝟑 𝒛𝟑

…

𝒚𝟒 𝒛𝟒

…

𝒃𝟐𝒚𝟐 𝒃𝟐𝒛𝟐

…

𝒃𝟑𝒚𝟑 𝒃𝟑𝒛𝟑

…

𝒃𝟒𝒚𝟒 𝒃𝟒𝒛𝟒

…

④

Extracted In PHVs Stateful Registers

Cha halle llenges nges an and Des Design ign

• Galois Field Multiplication Offloading
• Rate Synchronization
• Deep Payload Inspection/assembly

Cha halle llenges nges an and Des Design( ign(1) 1)

• Galois Field Multiplication Offloading
• We convert it to addition, logarithm and exponents
• To calculate 𝒃𝟐𝒚𝟐,
• Look up 𝒎𝒑𝒉(𝒚𝟐) in the logarithm table
• Add with a pre-known 𝒎𝒑𝒉(𝒃𝟐) : 𝒎𝒑𝒉(𝒃𝟐𝒚𝟐) = 𝒎𝒑𝒉(𝒃𝟐)+𝒎𝒑𝒉(𝒚𝟐)
• Look up 𝒃𝟐𝒚𝟐in the exponent table: 𝒃𝟐𝒚𝟐 = 𝒇 𝒎𝒑𝒉(𝒃𝟐𝒚𝟐)
• Note that the logarithms and exponents are also on the Galois Field,

where this method is valid.

• Rate Synchronization
• Deep Payload Inspection/assembly

Cha halle llenges nges an and Des Design( ign(2) 2)

• Computation Offloading
• Rate Synchronization
• Switch has to temporarily buffer partial XOR sums since first packet

arrives until last packet leaves.

• One-to-many TCP
• The switch only needs to buffer partial XOR sums whose size is equal

to in-flight packets, bounded by BDP (bandwidth-delay product)

• SSD peak write speed: 1GB/s
• DC RTT : 250 us
• BDP = 250KB
• Deep Payload Inspection/assembly

Cha halle llenges nges an and Des Design( ign(3) 3)

• Computation Offloading
• Rate Synchronization
• Deep Payload Inspection/assembly
• Many switch constraints leads to limited number of processed bytes,

while small-sized packets reduce throughput.

• Use recirculation inspired by PPS (SOSR 19)
• Redesign l4 checkcum updates.

Di Discu scussion ssions s an and li limitation mitations

• Will NetEC cause incast?
• NetEC actually prevents incast.
• Most incoming packets are dropped in the ingress pipeline.
• Outbound PPS ≈ Inbound PPS
• Is NetEC scalable?
• The number of machines to download from: 3, 6, 10
• The number of concurrent tasks
• Problem:
• Currently, a table or register can only be accessed once per packet, so that we

need multiple logarithm/exponent tables.

• Limited number of registers per stage.

Im Implementatio plementation n an and Eva valuat luation ion

• We implement a prototype of NetEC on commodity switches,

and integrate it with HDFS-EC.

Conc nclusion lusion

• EC low reconstruction rate is due to multiplexed NIC capacity
• In-network computation resolves this problem, leading to

great performance improvement.

• We design and implement NetEC, addressing three challenges,

and conduct preliminary evaluations to show effectiveness.

Tha hank nk yo you! u!