Nikhef plans (and some comments from Jos Vermeulen) Frank - - PowerPoint PPT Presentation

Nikhef plans

(and some comments from Jos Vermeulen)

Frank Filthaut, Paul de Jong, Milo Vermeulen

Proposed hardware setup

1 FELIX PC (SuperMicro)

single CPU (3.6 GHz E5-1650v4 or 3.5 GHz E5-1650v2), 6 cores, hyperthreading; 16 or 32 GB memory (2 hosts available; upgrade from 16 GB to 32 GB possible)

8-lane PCIe Gen3 ➠ measured max. throughput > 6 GB/s

HTG710 board, 6 input links ➠ input rate 57.6 Gb/s (payload: 45 Gb/s)

emulate 6 WIB links using VC709 (optical splitter to create 6 links from 4)

dual port 40 Gb/s Ethernet (or 56 Gb/s Infiniband) NIC (Mellanox), 8-lane PCIe Gen3 (3 available)

1 event builder PC

data sink

BoardReader PC

(if needed)

All hardware available (Nikhef DAQ test lab)

2

custom full
 mode

Block view

ProtoDUNE triggered mode:

house Board Reader on the FELIX PC (suggestion by Giovanna)

handles both trigger (timestamp) selection and subsequent compression to be seen whether this is feasible (especially compression likely to be CPU intensive)

Fall-back options:

1: additional computing power (separate host) for compression (or perhaps — but unlikely — even for timestamp selection) 2: determine maximum number of WIB links / FELIX PC (if not 6)

3

HTG710
 / CPU VC709

WIB link
 emulator FELIX Board
 Reader

CPU FELIX PC TCP/IP
 / … UDP Event
 Builder
 PC trigger

artDAQ

CPU

Block view

DUNE trigger-less mode:

house Board Reader on the FELIX PC and/or offload to additional computing power

compression only; exact setup to be determined from throughput tests (should get a good idea from compression in triggered setup)

4

HTG710
 / CPU VC709

WIB link
 emulator FELIX Board
 Reader

CPU FELIX PC custom full
 mode TCP/IP
 / … UDP Event
 Builder
 PC

artDAQ

CPU Board
 Reader PC

Board
 Reader

CPU

Progress

Minimal hardware setup exists

relying heavily on / using setup of core FELIX firmware team at Nikhef additional hardware can be bought quickly and at modest cost

BoardReader: Milo (new PhD student) to begin ~ now

experience with C++ but needs to get acquainted with the context

Full mode emulator: firmware programming to begin soon

driven also by ATLAS needs René Habraken (new engineer)

5

Comments (from Jos) on Eric’s slides

p4 (throughput): suggest that the issue may not be with the number of lanes but with the use of the DMA controller p10: with ≤ 4 input links, cheaper VC709 should suffice in lieu of HTG710

also a target for FELIX firmware development

p13: suggest cheaper motherboard / CPUs

Supermicro X10SRA-F motherboard with 3.6 GHz E5-1650V4 CPU and 32 GB DDR4 2400 MHz ECC reg memory in 4 U Supermicro rack-mountable chassis (743T

• 665B) (in Amsterdam 1640 Euro ex.

VAT, including 256 GB SSD). The slots can be configured as two 16-lane and one 8-lane Gen3 PCIe slot or as

• ne 16-lane and four 8-lane slots (8-lane cards can be used in 16-lane slots)

used both at Nikhef and at CERN

6

Throughput (from Jos)

7

PCIe&throughput&

! Measurements&where&done&using&a& simple&counter&filling&the&PCIe&FIFO& ! For&test&purposes&only&:&single&shot&DMA& transfers& ! For&normal&operabon:&conbnuous&DMA& transfer&is&foreseen&

18&

One"card" Two"cards"

JV,&Münich&Muon&Week,&15COctC2015&