PD front-end electronics Josh Spitz, University of Michigan 30% - - PowerPoint PPT Presentation

▶
pd front end electronics
SMART_READER_LITE
LIVE PREVIEW

PD front-end electronics Josh Spitz, University of Michigan 30% - - PowerPoint PPT Presentation

May 31, 2023 358 likes •655 views

PD front-end electronics Josh Spitz, University of Michigan 30% DUNE Design Review, 11/12/2018 with: Dave Warner, Jon Ameel, Gustavo Cancelo, Rory Fitzpatrick, Chris Barnes, Matt Toups, Sten Hansen, Dante Totani, Joel Mousseau, Alex Himmel, and

slide-1
SLIDE 1

PD front-end electronics

Josh Spitz, University of Michigan 30% DUNE Design Review, 11/12/2018

with: Dave Warner, Jon Ameel, Gustavo Cancelo, Rory Fitzpatrick, Chris Barnes, Matt Toups, Sten Hansen, Dante Totani, Joel Mousseau, Alex Himmel, and others

slide-2
SLIDE 2

Introduction

  • Commercial ADCs used in ultrasonic transducers (Texas

Instruments) are being used for digitization in the SiPM- based mu2e cosmic ray veto scintillator.

  • Low noise, high gain, high dynamic range.
  • 80 Megasamples per second, 12 bit
  • Low cost ($50/channel) and capable of handling the

envisioned DUNE MPPC (SiPM) warm-side PD signals.

!2

slide-3
SLIDE 3
  • Warm-side FEB
  • 64 channels of 80 MS/s, 12 bit ADCs
  • Bias generator (for SiPMs; 80 V max)
  • Current measurement (100pA resolution) for IV curves of SiPMs
  • Power-over-ethernet power (600 mA) for entire board’s power. One Cat6-cable for data and

power.

  • 1 GB DRAM data buffer, divided in 4 places (256 MB each) on the board, corresponding to the

4 FPGAs

  • Parallel flash ROM for fast FPGA re-load (50 ms)
  • Low cost, high bandwidth HDMI used to connect to cold-side
  • Readout controller
  • 24 FEB link ports. Supplies timing, trigger, and power to FEBs
  • Can produce (e.g. TPC) triggers and also accept external (e.g. accelerator) triggers

Warm-side electronics elements

Adapted from Sten Hansen

!3

slide-4
SLIDE 4

20

Local Trigger, Gate

Ethernet Aux +48

Isolated 48 DC-DC Supply

Arm uC Ultrasound Chips LV DC-DC Supplies 64 Meg Flash Bias Generator

2Gb LPDDR Spartan 6 FPGA

HDMI Connecters to CMB boards

TCP/IP Chip

Link to Controller

USB

Front-end board

(64 channels, 80 MS/s, 12 bit)

Courtesy of Sten Hansen

  • 4
slide-5
SLIDE 5

Octal UltraSound Processor Octal UltraSound Processor

FPGA

FR Clk Bias Bus Bias Trim DAC 1 of 4 SiPMs

One of 8 12 bit 80msps ADCs/chip

Cvt Clk Counter Mother Board S Dat

One of 64 Channels

256MB LPDDR RAM Parallel FLASH CFG ROM Chan0..15 Chan16..31 Chan32..47 Chan48..63 Timing/ Trigger LV DC-DC Converters 3.3v 1.8v 1.2v Bias Bus SiPM Bias Generator 48V ARM Microcontroller with ECC RAM USB 2.5v Data Ethernet PHY RJ-45 Controller Link

Courtesy of Sten Hansen

Front-end board

(64 channels, 80 MS/s, 12 bit)

!5

slide-6
SLIDE 6

Commercial ultrasound ADC

!6

Eight channels of: low noise preamp, variable gain amp,

Adapted from Sten Hansen Eight channels of: low noise preamp, variable gain amp, programmable gain amp, programmable low pass filter, 80 MS/s, 12 bit ADC, $8 per channel, 120 mW per channel. Can adjust gain so that 1pe=10 ADC

slide-7
SLIDE 7

48 V Bulk Supply

+5V Local Supply

2 TTL inputs 2 TTL outputs Ethernet in / out Fiber in / out

Controller

Front view Back view

Courtesy of Sten Hansen

!7

slide-8
SLIDE 8

Front End Board and Controller

  • Controller

FEB

Controller w/ one FEB

!8

slide-9
SLIDE 9

Controller FEB FEB

Controller w/ two FEBs

!9

slide-10
SLIDE 10

Readout Controller Block Diagram

Courtesy of Sten Hansen

!10

slide-11
SLIDE 11

Isolation

+ 48VDC

  • Primary
Transformer

120 VAC

Secondary 48V Power Supply 700W

Feedback

Chassis POE Signal Transformer

POE+ PSE Controller POE+ PD Controller

To POL Converters

Transformer

Cat 6 Cable

~20K

Isolated 48V Supply One of 24 Ports

CRV Front End Board

Board Ground

Primary

Secondary

Feedback

POE Signal Transformer POE Signal Transformer POE Signal Transformer POE Signal Transformer POE Signal Transformer POE Signal Transformer POE Signal Transformer

Isolation

One chip Controls four ports

Single 8 pin chip

PSE – power sourcing equipment PD – powered device

+ 48VDC

  • CRV Controller

Power and Data Link Arrangement

POE+ Is IEEE 802.3at standard Courtesy of Sten Hansen

!11

slide-12
SLIDE 12

DAQ concept

  • 12 FEBs referenced to a single chassis, each FEB communicates
  • ver copper to a master controller (DC isolated on both sides).
  • Each controller can take digital inputs from 24 FEBs (64 channels

each->1536 channels, or in the case of 40 channels each->960 channels).

  • Controller provides power (48 V supply) to FEBs.
  • Controller connects to a DAQ PC either over fiber optic or ethernet

(copper).

  • Pulses (timestamp and pulse height) are sent from FEBs to
  • controller. Controller issues global trigger to FEBs.

!12

slide-13
SLIDE 13

Can the FEB work with the active summing board?

(yes)

!13

slide-14
SLIDE 14

MPPC Tests

SiPM Power Supply OpAmp Power Supply Laptop for DAQ

Bench-top tests

(in cold) Bias for op-amp and MPPCs provided by external DC power supply Op-amp output is differentiated, while the FEB has a single-ended op-amp. Need to “undifferentiate” signal with balun

!14

slide-15
SLIDE 15

72 MPPC array test

!15

each of 12 rows has 6 of 6mm2 MPPCs in parallel total capacitance of 7.8 nF per row MPPC=Hamamatsu S13360-6050VE

slide-16
SLIDE 16

Successfully demonstrated single photon resolution!

Comparisons to SSPs

  • Comparable resolution measured as peak to width ratio.

Mu2e FEB S/N = 4 SSP , S/N = 5 courtesy of Gustavo and Dante Comparable resolution measured as peak-to-width ratio

!16

slide-17
SLIDE 17

1 tick = 12.55 ns

  • Rise time: 125 ns
  • Fall time: 350 ns
  • Recovery time: O(2us) — what’s shown on the right is as wide a window

as the FEB allows

Average waveforms

!17

slide-18
SLIDE 18

Zero-suppression

  • Zero suppression: Time tick is saved to the board’s RAM if

the tick is above a pre-set pedestal value.

  • mu2e and SBND will use the FEB+controller to take zero-

suppressed data.

  • FEB+controller zero suppression interface work is
  • ngoing.
  • Work towards fully understanding the zero-suppression

(suppression factor) requirements is ongoing.

  • Dependent on readout window and overall trigger rate.

!18

slide-19
SLIDE 19

Bandwidth and rates

  • Bottleneck is 10MB/s FEB to controller rate (per FEB).
  • Currently considering 40 readout channels per board.
  • 80 MHz, 12 bit ADC; 5.5 us waveforms=5.3 kbit/waveform
  • Consistent with longest waveforms (including late-light) expected
  • DC rate: 250 Hz/channel; 53 Mbps/APA (1 APA=40 channels)
  • 6.6 MB/s FEB to controller DC rate (compare to 10 MB/s FEB-controller bottleneck)
  • Can develop multi-channel coincidence+threshold requirement at the FEB firmware level to

mitigate (study ongoing).

  • DAQ interface spec: 8Gbps per connection. DAQ takes 24 FEB signals (10 Mbps each)=240
  • Mbps. Ok!
  • Maximum instantaneous rate: 6000 channels fire at once
  • 32 Mbits (4 MB) at once.
  • The controller can handle all 24 boards firing at once. Write speed for 24 boards is 150 MB/s

and could likely be increased to 400 MB/s with some work (according to Sten Hansen).

!19

slide-20
SLIDE 20

Firmware development

!20

  • m the

ultrasound chips to the FEB and (2) deliver

  • perly spaced intervals

Synthesis, the first step, checks the syntax es ciently

  • u can run a behavioral simulation, one

that is run without timing information,

Source: “FPGA Design Flow Overview”, Xilinx Website.

Xilinx ISE: This is a design environment used to design firmware written in VHDL for use in Field-Programmable Gate Arrays (FPGAs). The ISE (“Integrated Synthesis Environment”) version, 14.7, is the last available version that works with a Spartan 6 FPGA. ModelSim: This is a simulation environment for VHDL and other hardware description languages, distributed by Mentor Graphics. The version, 10.2c, is the version that works with the ModelSim license at Fermilab.

slide-21
SLIDE 21

Structure of the Firmware

The firmware is written in VHDL, called “Very High Speed Integrated Circuit Hardware Description Language”. The firmware is separated into three components:

  • 1. Main Firmware File (extension “vhd”): contains the logic for piping

data into the FEB and out to the controller.

  • 2. Test Bench (extension “tb”): contains the timing structure for each of

the signals that the logic in #1 handles.

  • 3. Constraints File (extension “ucf”): contains the associations between

the ports on the FEB and the signals within #1.

  • 4. Project Definitions File (extension “vhd”): defines iterators and

constants that are used within #1. The remainder of the firmware consists of the functional models of components that are called within #1. Initially written for mu2e->adapting it for DUNE (ongoing work)

!21

slide-22
SLIDE 22

Power considerations

  • In addition to digitizing the SiPM signals, the FEB is nominally designed to bias

the SiPMS.

  • Can the power be used for the active summing board as well?
  • No, it does not provide a stable enough voltage. The on-board Cockroft-

Walton should not be used to bias the differential amplifier of the summing board.

  • How to handle this?
  • Redesign on-FEB power supply?
  • Include another cable/wire in design?
  • We are working to address this.

!22

slide-23
SLIDE 23

Rack space and power consumptions

  • 6000 channels total; assume 40 channels/FEB
  • 12 FEB/chassis, 13 chassis (6u each) required for FEBs.
  • 7 controllers (controlling 24 FEB each), 1 u each.
  • ~85u required. Assuming 42u/rack, we will need just over 2

racks.

  • Power supply on a controller is 700 W, each FEB takes 20 W.

!23

slide-24
SLIDE 24

Grounding scheme

!24

Feedthrough FEB Chassis (x12) FEB (x12) Cat 6a, twisted pair (shielded) (x192) Controller Chassis (x7) Cat 6, twisted pair (x12) 48 V 700 W Supply Cat 6, twisted pair (x12)

5 V Supply

DAQ PC (x1) LEGEND

Digital Signal Analog Signal LV Power (< 80 V) PCB Power Supply Chassis

POE Card 80 V max, 3 mA trip 48 V max, 700 mA fuse RJ 45 connector: 100 VAC, 1.5 A Cat6a: 26 AWG, 2.2 A Cat6: 26 AWG, 2.2 A Solid lines denote Cu Dashed lines Denote optical Building Ground Detector Ground

1

(DC) Controller

slide-25
SLIDE 25

Other option (optical from controller to DAQ)

!25

Feedthrough FEB Chassis (x12) FEB (x12) Cat 6a, twisted pair (shielded) (x192) Controller Chassis (x7) Cat 6, twisted pair (x12) 48 V 700 W Supply Cat 6, twisted pair (x12)

5 V Supply

DAQ PC (x1) LEGEND

Digital Signal Analog Signal LV Power (< 80 V) PCB Power Supply Chassis

POE Card 80 V max, 3 mA trip 48 V max, 700 mA fuse RJ 45 connector: 100 VAC, 1.5 A Cat6a: 26 AWG, 2.2 A Cat6: 26 AWG, 2.2 A Solid lines denote Cu Dashed lines Denote optical Building Ground Detector Ground

1

Controller (DC)

slide-26
SLIDE 26

Alternative front-end (SSP)

D1 D2 D3

D1 D3 R1 R2 R3 D2

D1 D2 D3

D1 D3 R1 R2 R3 D2

ND.(

  • Argonne SSPs (150 MS/s, 14 bit) [comparing to 80 MS/s, 12 bit]
  • Higher cost with significantly more utility
  • Multiple onboard utilities for online signal characterization
  • Used in ProtoDUNE (288 channels)

JINST 11 P05016 (2016)

!26

(in ProtoDUNE)

slide-27
SLIDE 27

D “ fi

SSP (SiPM Signal Processor) Block Diagram

!27

From Zelimir Djurcic

Alternative front-end (SSP)

slide-28
SLIDE 28

Main issues moving forward

  • Demonstrate full FEB+controller+DAQ chain (and merging with TPC info) in ICEBERG.
  • Develop solution for active summing board’s power.
  • Possible board re-designs
  • 48 channel board re-design (from 64 channels)?
  • 40 channels (1 APA) + 8 spare?
  • Power scheme, including both MPPC and active summing board bias?
  • Cat6 instead of HDMI?
  • Explore the possibility of using DC power input to the controller (perhaps from a Weiner supply), in

consideration of noise issues.

  • Develop firmware and zero-suppression (including multi-channel coincidence+threshold on FEB) scheme.
  • Considerations: DC rate, radiogenics rate, maximum instantaneous rate

!28