ONSEN ONSEN HLT/DatCON Merger and HLT/DatCON Merger and - - PowerPoint PPT Presentation

▶

Mar 22, 2024 337 likes •432 views

ONSEN ONSEN HLT/DatCON Merger and HLT/DatCON Merger and Trigger/ROI Distributor Trigger/ROI Distributor Bjrn Spruck Talk held somewhere Once upon a time after 30.4.2014 30.4.14, p. 1 Belle II PXD DAQ Belle II PXD DAQ (PC Farm) (PC

SLIDE 1

30.4.14, p. 1

ONSEN HLT/DatCON Merger and Trigger/ROI Distributor ONSEN HLT/DatCON Merger and Trigger/ROI Distributor

Björn Spruck Talk held somewhere Once upon a time after 30.4.2014

SLIDE 2

30.4.14, p. 2

Belle II PXD DAQ Belle II PXD DAQ

(PC Farm) (PC Farm) (PC Farm)

SLIDE 3

30.4.14, p. 3

DHH Load Balancing DHH Load Balancing

Full Subevent (5 DHH) to one output. Selected by trigger number!

SLIDE 4

30.4.14, p. 4

Event Wise View Event Wise View

1 5 ..

new DHHC

*4

4*4 DHH every 4th event => 1/2 PXD f.e. upstream part 4*4 DHH every 4th event => 1/2 PXD f.e. downstream part

ONSEN

These are the Event Numbers!

2 6 .. 3 7 .. 4 8 .. 1 5 .. 2 6 .. 3 7 .. 4 8 .. 1 5 .. 2 6 .. 3 7 .. 4 8 .. 1 5 .. 2 6 .. 3 7 .. 4 8 .. 1 5 .. 2 6 .. 3 7 .. 4 8 .. 1 5 .. 2 6 .. 3 7 .. 4 8 .. 1 5 .. 2 6 .. 3 7 .. 4 8 .. 1 5 .. 2 6 .. 3 7 .. 4 8 ..

Each DHHC sees 1/8 of the whole PXD

= Carrier with 4 AMCs = 4 AMCs served by one DHHC

SLIDE 5

30.4.14, p. 5

Load Balancing / Multiplexing by Event Number Load Balancing / Multiplexing by Event Number

Be flexible, don't use “just” the lower two bits as selection. The following was decided by Igor, Dima and Björn: We use a lookup table with 64 entries.

6 lower bits of trigger number as index in table which selects then one of the four outputs Table can be changed “on the fly“ by Slow Control Easily adaptable if we cannot use all four outputs in a test Deactivate one output w/o loosing data Generate lower or higher data rates on one output

Because of the delayed HLT trigger, modifying the table is only possible during a stopped DAQ (e.g. HLT queue/fifos must be completely empty!)

SLIDE 6

30.4.14, p. 6

Closeup of Merger Closeup of Merger

xFP AMC Carrier Board #0

Me r g e r D i s t r i b u t

Carrier Board #1 Carrier Board #2 Carrier Board #3 Carrier Board #4 Carrier Board #5 Carrier Board #6 Carrier Board #7 Carrier Board #8

D a t C

H L T

D i s t r i b u t e s H L T T r i g g e r b y E v e n t N u m b e r i n e x a c t l y t h e s a m e w a y a s t h e D H H C d

s i t . C

fi g u r a b l e ! D u p l i c a t e s t h e e a c h d a t a t

w

t p u t s . ( e . g . f

w a r d / b a c k w a r d ) S e l e c t s f w / b w R O I s w i t h i n d a t a (

t i

a l )

SLIDE 7

30.4.14, p. 7

Results Results

Implemented in VHDL including PLB registers for LUT configuration Tested with 1GEvents (32GB) with different LUT setup

Checksums, events number selection and continuity are O.K.

SITCP in → Distributor → (SITCP out 1, nirvana, nirvana, SITCP out 2) on an AMC board (Virtex 5). But

Not checked with four real outputs (can only use two or three for sitcp) Not checked on Virtex 4, not checked on carrier board (connectivity...) Not checked with backplane (AURORA)

SLIDE 8

30.4.14, p. 8

Forward/Backward ROI Sorting Forward/Backward ROI Sorting

ROIs for forward/backward part of PXD are distinguishable by their DHH-ID

We can reduce the HLT data which is send to each node by sending only ROI data which the node needs Remark: Data rate is already low, just double the output (aka DESY test) will work, too. But: We would save some memory bandwidth on the Selector nodes.

Results:

Works. Checked with 1,6 GEvts Input (=570 GB) → 2*400 MEvts Output

(=2*70 GB), Rate ~112 MB/s Input Checksum is recalculated as HLT/ROI content of package changes by selecting FW/BW ROIs only.

But:

Not checked with eight real outputs (can only use two or three for sitcp) Not checked on Virtex 4, not checked on carrier board (connectivity...) Not checked with backplane (AURORA)

SLIDE 9

30.4.14, p. 9

Closeup of Other Carriers Closeup of Other Carriers

Carrier Board #0

D i s t r i b u t

Carrier Board #1 Carrier Board #2 Carrier Board #3 Carrier Board #4 Carrier Board #5 Carrier Board #6 Carrier Board #7 Carrier Board #8 AMC #1 AMC #2 AMC #3 AMC #4

ONSEN HLT/DatCON Merger and Trigger/ROI Distributor ONSEN HLT/DatCON Merger and Trigger/ROI Distributor

Björn Spruck Talk held somewhere Once upon a time after 30.4.2014

Belle II PXD DAQ Belle II PXD DAQ

(PC Farm) (PC Farm) (PC Farm)

DHH Load Balancing DHH Load Balancing

Full Subevent (5 DHH) to one output. Selected by trigger number!

Event Wise View Event Wise View

new DHHC

*4

ONSEN

These are the Event Numbers!

Each DHHC sees 1/8 of the whole PXD

Load Balancing / Multiplexing by Event Number Load Balancing / Multiplexing by Event Number

Be flexible, don't use “just” the lower two bits as selection. The following was decided by Igor, Dima and Björn: We use a lookup table with 64 entries.

6 lower bits of trigger number as index in table which selects then one of the four outputs Table can be changed “on the fly“ by Slow Control Easily adaptable if we cannot use all four outputs in a test Deactivate one output w/o loosing data Generate lower or higher data rates on one output

Because of the delayed HLT trigger, modifying the table is only possible during a stopped DAQ (e.g. HLT queue/fifos must be completely empty!)

Closeup of Merger Closeup of Merger

Me r g e r D i s t r i b u t

D a t C

H L T

D i s t r i b u t e s H L T T r i g g e r b y E v e n t N u m b e r i n e x a c t l y t h e s a m e w a y a s t h e D H H C d

s i t . C

fi g u r a b l e ! D u p l i c a t e s t h e e a c h d a t a t

w

t p u t s . ( e . g . f

w a r d / b a c k w a r d ) S e l e c t s f w / b w R O I s w i t h i n d a t a (

t i

a l )

Results Results

Implemented in VHDL including PLB registers for LUT configuration Tested with 1GEvents (32GB) with different LUT setup

Checksums, events number selection and continuity are O.K.

SITCP in → Distributor → (SITCP out 1, nirvana, nirvana, SITCP out 2) on an AMC board (Virtex 5). But

Not checked with four real outputs (can only use two or three for sitcp) Not checked on Virtex 4, not checked on carrier board (connectivity...) Not checked with backplane (AURORA)

Forward/Backward ROI Sorting Forward/Backward ROI Sorting

ROIs for forward/backward part of PXD are distinguishable by their DHH-ID

We can reduce the HLT data which is send to each node by sending only ROI data which the node needs Remark: Data rate is already low, just double the output (aka DESY test) will work, too. But: We would save some memory bandwidth on the Selector nodes.

Results:

(=2*70 GB), Rate ~112 MB/s Input Checksum is recalculated as HLT/ROI content of package changes by selecting FW/BW ROIs only.

But:

Not checked with eight real outputs (can only use two or three for sitcp) Not checked on Virtex 4, not checked on carrier board (connectivity...) Not checked with backplane (AURORA)

Closeup of Other Carriers Closeup of Other Carriers

D i s t r i b u t

D

b l e r ^ 2 R e ma r k : A l l A MC i n

e C a r r i e r p r

e s s s u b

v e n t s

t h e s a me E v e n t N u mb e r . T h u s n

u r t h e r s e l e c t i

n e c e s s a r y n e e d e d . T h e d a t a j u s t h a s t

e c l

e s t

r

t p u t s (

d

b l e d t w

i me s ; d

b l e d l i k e i n “ D E S Y t e s t ” ) O n e c

l d s e l e c t f u r t h e r

t h e D H H I D s , b u t t h a t s n

n e e d e d , d a t a r a t e i s a l r e a d y l

. ( n e v e r t h e l e s s , i t w

l d s a v e me mo r y b a n d w i d t h . . . ) D

b l e r D

b l e r D

b l e r