IDROGEN High speed acquisition board Daniel Charlet 1 ,Cedric Viou 2 - PowerPoint PPT Presentation

IDROGEN High speed acquisition board Daniel Charlet 1 ,Cedric Viou 2 ,Jean-Pierre Cachemiche,Damien Tourres 1 : Laboratoire de l'Accélérateur Linéaire Université Paris XI, 91898 Orsay 2 : Station de Radioastronomie de Nançay, Observatoire de Paris, PSL Research University, CNRS, Université d’Orléans, 18330 Nançay, France 1

Agenda  DAQGEN  NEBULA  IDROGEN  White Rabbit Daniel Charlet Guiyang -09-2018 2

DAQGEN project  Acquisition system for physic  Open system : Hardware, firmware, software for IN2P3 *  Base on the know-how of different IN2P3 laboratories  Hardware : Schematics & brd  Firmware : Low level interface  Software :  Slow control  Configuration  Data acquisition * French National Institute of Nuclear and Particle Physics Daniel Charlet Guiyang -09-2018 3

DAQGEN ARCHITECTURE  Standard : xTCA for Physics  Based on on the shelf component  Crate controler : MCH N.A.T  Backplane data readout : PCIe 4x Gen3 ou Eth10G.  Data transfer :  Eth10G, PCIe-over cable (NAT compagny)  100GEth (in development by NAT)  Configuration : 1GEth & IP bus. Daniel Charlet Guiyang -09-2018 4

NEBULA Board 5

PAON IV Analog chain limitation 1 Antenna board RF Prototype Board 50 Ω Gain=20DB Gain=30DB 50 Ω 10m Low pass Low-pass Pass band Gain=19DB 50m Mixer Filter Filter Ampli Filter LNA IF 1 FPGA IF RF ADC 2 LNA Ampli Gain=20DB RF IF: Intermediate frequency LO=1.2GHZ RFout LO Prototype Board  Local oscillator distribution 1  Phase stability Master LO=1.2GHZ 2 Frequency 3  Temperature stability PLL RF 4 Splitter 1/8 50MHz 5 6 7 Phase-locked loop 8 Daniel Charlet Guiyang -09-2018 6

PAON IV Analog chain limitation 2 OL Amplifier Amplier OL 75Ω 50 Ω 75 Ω 75 Ω 50 Ω 50 Ω 50 Ω 50/75 75 50/75 LNA LNA 8m 50m 10m 1m 2m 0.5m 50 Ω 50m ADC ADC Central frequency 1375Mhz Bandwith 250Mhz Sampling frequency 500Mhz Daniel Charlet Guiyang -09-2018 7

NEBULA board ADC 081020 2x 1GSPS • MTCA 4.0 standard, Double-width, Input bandwidth : 2Ghz full size AMC. • FPGA : 5AGTMC7G3F31 WR 1Gb • Stand alone mode (12v) Eth • ADC 2 channels 1GSPS. Synch & config 10Gb Data • White Rabbit compliant. config Eth 10Gb • On board configuration (µC) Data/config FPGA ctrl/cde • Very low noise synthesizer PLL & conf synthesizer cleaner (LM04828) SPI IPBus SPI • Front panel : WR SFP+ µC ATMEGA EPLD 2x SFP+ 10GbEth IPMI SPI I2C MAXV • Backplane connectivity : PCIE x4 Gbe IP bus,PCI 4x Gen3, IPMB IPMB, CLK & trigger lane. SSRAM Flash x 2 Power CTRL LMK04828 WhiteRabbit DS1014 512K 1Mb Daniel Charlet Guiyang -09-2018 8

NEBULA board PLL Cleaner WR components 9

NEBULA board, DAC test f =490MHz 4℮6 f =1490MHz 4℮6 samples samples f =990MHz 4℮6 f =1990MHz 4℮6 samples samples 10

NEBULA board, WhiteRabbit test sytem •Paris Observatory ( SYRTE) collaboration 11

NEBULA board, PPS test 1 • Paris Observatory ( SYRTE) collaboration. • Future test : • long distance > 100Km • Firmware upgrade (freq DDMTD) •400 fs after 1000s over 100 m 1ps 12

NEBULA board, PPS test 2 •~700 Fs after 10 days & air conditioning problem 1ps 10 days 13

IDROGEN Board 14

Carte IDROGEN Clk synthesiser 8 GX Jitter cleaner FMC+ • MTCA 4.0 standard, Double-width, 80 diff 28 diff full size AMC. R • FPGA : 10GX027H4F34 T M • Stand alone mode (power 12v) • HighPinCount FMC slot. • White Rabbit compliant. 1 Eth-1G ARRIA 10 white-Rabbit 1 Config. & CTRL 1 • Front panel connectivity : WR SFP+ Eth-1G AMC connector SFP+ Eth-1G IPBus QSFP+ 40G, USB 4 4 • Backplane connectivity : Serial link 1G PCIe 4x Gen3/ Eth-40G QSFP+ 1Gbe IPbus,PCI 4x Gen3, ETH 10G Data transfert & Config. & CTRL IPMB, CLK & trigger lane. Data transfert ● RTM connector : J30. ● Low cost Daniel Charlet Guiyang -09-2018 15

IDROGEN board CLK.Ext LMK04828 SI5338 10 FMC+ 7 4 I2C 160 SPI • On board configuration (µC) CLK CLK • Very low noise synthesizer PLL RTM 28 synthesizer cleaner (LM04828) for PPS / trig.Ext •WR clk and derived clkl. PCIEx 4x Gen3 Ethernet 40G ARRIA 10 SX • Dedicated PLL for serial links USB 10AX027H4F34 ● Integrated USBBlaster II. White rabbit Ethernet 1G SFP+ Ethernet 1G ● FPGA configuration : Active serial, IP bus I2C IP bus. Ethernet 1G ● External connectivity : PPS, Trigger, QSFP+ Serial link 40G Ext CLK. AS prog JTAG JTAG I2C I2C spi IPMB MAX10 ATMEGA128 Cypress EPCQ USB FX2LP USB Daniel Charlet Guiyang -09-2018 16

4DSP FMC 125 • Form factor FMC+. • Cooling : air condition • Function : Analog input : • Data rate 5000 (1ch), 2500 (2ch) 1250 (4ch). • Channels : 1 , 2 or 4 •Coupling AC •Resolution 8 bits. 17

IDROGEN board, power tree VCCP, VCC Seq 1 0,9v VCCR_GXB Filter LTM4627 Seq 2 Iout 15A 1,03v Filter VCCT_GXB EN6347 VCCA_FPLL Filter Seq 3 1,8v Filter VCCH_GXB EN6360 VCCPGM VCCBAT Filter VCCIO AMC con. 12v Other LTM4627 Filter 3.3v components Iout 15A 1.8V FMC Seq 5 Power con. 12v 3.3V FMC 12V FMC Max 17545 3.6v 12V RTM Seq 4 MAX8527 WR 3.3v 3.3V Power-management Max 17545 Ena MAX8517 PLL 3.3v 18 0.7a max Stand-alone mode

IDROGEN board, clock tree RTM30 clk0 RTM30_clk1 RTM CLK_SMA 100MHz RTM_ clk0 FMC_LA_CLK LMK_CLK12 AMC_ clk1 CLK3_bidir CLK1_M2C FMC_LA_CLK ref_in0 LMK_CLK8 CLK2_bidir LMK_CLK4 LMK04828 LMK_CLK2 100 MHz CLK0_M2C clk_free 25 MHz LMK_CLK1 ref_in1 LMK_CLK0 GBT_CLK0 CLK_WR GBT_CLK1 LMK_CLK12 Clkref_FMC Clkref CLK clkref GPIO GPIO CLKT CLKB DAC 16b DAC DAC VCTCXO Clkref Triga_0 AD5662 AD5662 AD5662 25MHz XTCA connector Triga_1 DAC SPI Trig_in DAC 16b VCXO AMC_clk2 Trig_out AD5662 25MHz AMC_clk1 PLL_out 10AX027F35 clkref 25 MHz PCIECLK clkref WR DMTD CLKUSR RFCLKL 1Gb RFCLKL 10Gb 100MHz 125 MHz 156.25MHz 100 MHz SI 5338 100 MHz MAX 10 50 MHz 19

Carte IDROGEN, configuration FPGA IP_bus Serial link Eth1G SFP+ Eth1G PCIe PCIe AS prog FPGA Avallon bus Remote Update IP JTAG EPCQ_L r o t c e FMC RTM n n o c C JTAG JTAG M Swich JTAG A MMC USB blaster II USB Cypress MAX 10 20

IDROGEN, Whitte Rabbit implementation DS18S20 Temp & Serial num Flash FPGA Eth 1g Phy Endpoint 125MHz Ref clock generator PLL 125MHz VCXO 50Mhz WR PTP core 25MHz User core PLL 2 PLL 1 Periph LMK04828 Clock jitter cleaner Wishbone crossbar SysCon DAC VCXO DDMT Lattice SoftPLL Micro32 DDMT clock generator 62.5MHz 25MHz DDMT PLL DAC VCXO 62.5MHz 125MHz 125MHz https://www.ohwr.org/projects/white-rabbit PLL 21

White Rabbit firmware WR PTP Core Ethernet 1G Fabric EB master Phy Endpoint WR redirection wrapper UART Wishbone crossbar Mini-NIC Bulid ID Wishbone EEPROM Periph crossbar SysCon Wishbon Wishbone Avalon Tunable SoftPLL crossbar oscillators Avalon Lattice PPS RAM crossbar Micro32 JTAG Master http://www.ohwr.org/projects/white-rabbit/wiki/WRReferenceDesign RAM https://github.com/GSI-CS-CO/bel_projects 22

IDROGEN Firmware WR Eth 1G So Si TBI/serdes Etherbone & Phy Si So Avalon Bridge WR PTP Core Ext WB M WB S oscillator WR clk Avalon MM PPS M Avalon MM S Avalon MM S FMC Data acquisition Avalon ST So IP Bus Avalon ST Si Etht 40G Etht1G Data transfer Avalon MM S PCIe Avalon MM S 23

IDROGEN board, brd Daniel Charlet Guiyang -09-2018 24

WHITE RABBIT Daniel Charlet Guiyang -09-2018 25

An extension to Ethernet which provides: Synchronous mode (Sync-E) – – Deterministic routing latency Sub-nanosecond synchronization in WR is achieved by using the following three ● technologies together: – Precision Time Protocol (IEEE1588). – Synchronous Ethernet. DMTD phase tracking. – Open hardware (CERN ) Daniel Charlet Guiyang -09-2018 26

Precision Time Protocol (IEEE1588) ● Synchronizes local clock with the master clock by measuring and compensating the delay introduced by the link. ● Link delay is measured by exchanging packets with precise hardware transmit/receipt timestamps. Daniel Charlet Guiyang -09-2018 27

Synchronous Ethernet – ● All network nodes use the same physical layer clock, generated by the System Timing Master. ● PTP is used only for compensating clock offset. ● Having the same clock frequency everywhere enables phase detector technology as the means of measuring time. Daniel Charlet Guiyang -09-2018 28

Phase tracking Measure the phase shift between transmit and receive clock on the master side, taking the advantage of Synchronous Ethernet. Monitor phase of bounced-back clock continuously. Phase-locked loop in the slave follows the phase changes measured by the master. Daniel Charlet Guiyang -09-2018 29

Digital Dual Mixer Time Domaine phase detector Daniel Charlet Guiyang -09-2018 30

Conclusion  End of Board routing in October.  Board test in november  Firmware develop on evaluation board (ATILA sce REFLEX).  Installation on PAON IV second quarter of 2019.  Future development :  Compact version of Idrogen (removing FMC+ slot).  Enhance WR stability on very long distance > 100Km (SYRTE) Daniel Charlet Guiyang -09-2018 31