The Continuous Readout Stream of the MicroBooNE Liquid Argon Time - - PowerPoint PPT Presentation

The Continuous Readout Stream of the MicroBooNE Liquid Argon Time Projection Chamber for Detection of Supernova Neutrinos

José I. Crespo-Anadón Columbia University Nevis Laboratories 10/21/2019 DUNE DAQ Meeting

Based on MICROBOONE-NOTE-1030-PUB from https://microboone.fnal.gov/public-notes/

José I. Crespo-Anadón 2

The Short-Baseline Neutrino Program at Fermilab

ICARUS L = 600 m 476 t ν data in 2019 Booster Neutrino Beam (BNB) 8 Gev protons on Be target 〈 Eν ~ 700 MeV 〉 MicroBooNE L = 470 m 85 t ν data since Oct 2015 SBND L = 110 m 112 t ν data in 2021 MicroBooNE physics goals:

• Investigate the excess of electron-like events observed in MiniBooNE.
• Perform high-precision measurements of cross-sections of νµ and νe on Ar.
• Develop further the LArTPC detector technology.
• Perform searches for astroparticles and exotic physics exploiting the LArTPC capabilities

(on-beam & off-beam)

José I. Crespo-Anadón 3

The Short-Baseline Neutrino Program at Fermilab

MicroBooNE physics goals:

• Investigate the excess of electron-like events observed in MiniBooNE.
• Perform high-precision measurements of cross-sections of νµ and νe on Ar.
• Develop further the LArTPC detector technology.
• Perform searches for astroparticles and exotic physics exploiting the LArTPC capabilities

(on-beam & off-beam)

85 t Near surface 4 × 10 kt 1475 m underground

José I. Crespo-Anadón 4

• 85 t of liquid argon (active).
• Drift time: 2.3 ms.
• Three wire planes to reconstruct 3D interaction.

3 mm wire pitch. 8256 channels. 2 induction planes with 2400 wires each at ± 60º from vertical. 1 collection plane with 3456 vertical wires. Cold front-end electronics. 2 MHz digitization with warm electronics.

• 32 8” Hamamatsu R5912 Cryogenic PMTs

mounted behind the wire planes. 64 MHz digitization readout electronics. Scintillation light in coincidence with beam gates used for triggering.

MicroBooNE TPC & PMTs

Beam 2.5 m (drift) 1 . 4 m 2.3 m

JINST 12 P02017 (2017)

José I. Crespo-Anadón 5

The MicroBooNE Continuous Readout Stream

José I. Crespo-Anadón 6

MicroBooNE TPC continuous readout

• Enable the acquisition of the neutrino

burst from a nearby core-collapse SN.

• Expectation in MicroBooNE:

–

~ O(10) events for a SN at 10 kpc.

–

CC: νe + 40Ar →e- + 40K* (Eth ~ 5 MeV)

• MicroBooNE challenges:

–

Near-surface detector: large cosmic- ray rate: ~ 5.5 kHz

–

Small number of events.

–

Low energy. → No self-trigger.

• Instead, read out detector continuously

and rely on a delayed external trigger from the SNEWS alert.

• Also, demonstrate processing of TPC

information in real time. Foundation for a TPC-based trigger for DUNE. Complementary to a PMT-based trigger.

Prediction for DUNE [arXiv:1807.10334]

νe

Prediction for DUNE (40 kt) [arXiv:1512.06148]

José I. Crespo-Anadón 7

The MicroBooNE Continuous Readout Stream

• PMT stream not used for this work.
• Talk at May 2019 Collaboration Meeting with more details on electronics:

https://indico.fnal.gov/event/18681/session/4/contribution/21/material/slides/1.pdf and MICROBOONE-NOTE-1030-PUB at https://microboone.fnal.gov/public-notes/ Legend Triggered stream Continuous stream

José I. Crespo-Anadón 8

Data challenge

• Data is stored temporarily on a 15 TB HDD at each DAQ server, awaiting an SNEWS alert to be

transferred to permanent storage.

• Continuous readout of the TPC generates 33 GB/s

→ Distributed between 9 servers: ~ 3.7 GB/s/server

• Bottleneck: disk writing speed of the DAQ servers (conservatively 50 MB/s).
• Need a compression factor ~ 80.
• Lossless compression (Huffman) gives factor ~ 5: not enough.
• Requires additional lossy compression. Feasible since images are sparse.
• Also: writing at 50 MB/s gives us a window of > 48 h before data is deleted.

José I. Crespo-Anadón 9

Data reduction algorithms

• Implemented in the Front End Module

FPGA (Altera Stratix III).

• Zero suppression: only the waveform

passing the amplitude threshold (configurable per channel) with respect to the channel baseline is saved plus presamples and postsamples (configurable per FEM).

• The baseline can be

dynamically computed using preceding samples (if within mean and RMS tolerances)

• r

loaded as static value at the beginning of the run (both have been commissioned and tested).

• Additional compression provided by Huffman

encoding of ADC differences using a fixed table.

José I. Crespo-Anadón 10

Configuration of zero suppression parameters: Physics-driven plane-wide thresholds

• First approach (now deprecated).
• One threshold per plane.
• Physics driven: separate signal (cosmic-

ray muons) from electronics noise using previous Trigger Stream data.

• ADC distributions of max and min values

found after running a zero-suppression emulation with dynamic baseline and a loose threshold. U plane threshold: - 25 ADC V plane threshold: ± 15 ADC Y plane threshold: +30 ADC

José I. Crespo-Anadón 11

Configuration of zero suppression parameters: Bandwidth-driven channel-wise thresholds

• Current method.
• One threshold per channel.
• Bandwidth driven: suppress 98.5% of

ADC distribution using Trigger Stream data.

• Tested with dynamic and static baseline

firmwares. Static baseline is given by the mode of the distribution.

• Average threshold is 3.6, 2.2 and 5.2

times smaller than previous method for planes U, V, Y, respectively → Higher acceptance for low-energy signals.

José I. Crespo-Anadón 12

Data compression results

José I. Crespo-Anadón 13

Data compression results

• Compression factor =

Expected data rate Observed data rate

• SN Run Period 1: physics-driven plane-wide thresholds + dynamic baseline.

Excessive compression. Except for SEB06: too many noisy channels, preventing dynamic baseline estimation. Feasible to lower thresholds.

José I. Crespo-Anadón 14

Data compression results

• Compression factor =

Expected data rate Observed data rate

• SN Run Period 2: bandwidth-driven channel-wise thresholds + dynamic baseline.

Compression closer to target goal, but unstable. Noisy channels still preventing dynamic baseline estimation.

José I. Crespo-Anadón 15

Data compression results

• Compression factor =

Expected data rate Observed data rate

• SN Run Period 3: bandwidth-driven channel-wise thresholds + static baseline.

Target compression factor achieved. Data rates stable. Current running mode

José I. Crespo-Anadón 16

Analysis of Continuous Readout Stream data

José I. Crespo-Anadón 17

• Unlike Trigger Stream, “events” are not built by the DAQ. The DAQ just writes the data to the

local 15 TB disk at each server. Data lifetime is > 48 h.

• Continuous data needs to be first retrieved from the servers, and then built.
• “Event” needs to be defined (done at LArSoft level). For this analysis:

1 event =[last 0.8 ms (1600 samples) from frame N - 1] + [full frame N (1.6 ms, 3200 samples)] + [first 0.8 ms (1600 samples) from frame N + 1] so TPC objects can be reconstructed across frame boundaries.

• Only objects in the central frame are used for analysis.

Event building

José I. Crespo-Anadón 18

Signal processing

• Two data streams:

–

Continuous (SN) stream. Zero suppressed.

–

Trigger Stream. Not zero suppressed. First stages of signal processing are different.

• No noise filter: not effective since most of the

baseline is zero suppressed. No ROI Finder since it is done by the FPGA. A Zero-Suppression Emulation allows us to convert Trigger Stream data into SN Stream- like data.

• Challenge: flipping bits (FB) (bits in ADC

word switching from 0 to 1, or vice versa) affecting 4% of samples→ ADC values shifted by a combination of powers of 2.

• Baseline subtraction: linear interpolation

using presamples and postsamples.

• Flipping-bit correction: linear interpolation

between neighboring samples. If difference is bigger than 32 ADC, replace ADC value with the interpolated one.

José I. Crespo-Anadón 19

Signal processing

• Common stages:
• 1-D deconvolution using the same tool as

in the Trigger Stream.

• Same Hit Finder and Pandora Cosmic

clustering.

• Same Michel Reconstruction as in

MicroBooNE publication. Extended to any of the 3 TPC planes. Michel electrons used as a proxy for low- energy electrons produced by SN neutrinos.

JINST 12 P09014 (2017)

José I. Crespo-Anadón 20

Michel electron reconstruction: total energy spectrum

• 53.31 min of SN stream DATA.
• 58.82 min of Trigger stream DATA, normalized to SN stream exposure.

–

Also processed through Zero-Suppression (ZS) Emulation.

Plane Y Deficit at low energy. Excess at high energy. Flat ratio.

José I. Crespo-Anadón 21

Michel electron reconstruction: total energy spectrum (II)

Plane U Plane V

• SN stream Michel

electrons reconstructed at lower energies.

• Plane V: large rate and

shape disagreement with respect to Trigger Stream. Signals on V plane are smaller, and more likely to be zero suppressed.

• Effects well reproduced

by zero-suppression emulation.

José I. Crespo-Anadón 22

Michel electron reconstruction: rates and shapes

• Statistical uncertainty only.
• ~ 10% discrepancy in rates. Seasonal variation expected to be ~10%.

–

SN Stream data: 53.31 min from 9/21/2018. Average temp: 26 ºC.

–

Trigger Stream data: 58.82 min between 12/1/2017 and 07/07/2018 (218 days). Average temp: 6 ºC.

• Good shape agreement when simulating zero suppression and area normalized.

Plane U Plane V Plane Y

José I. Crespo-Anadón 23

Michel electron reconstruction: ionization and radiative components

Raw Hits

José I. Crespo-Anadón 24

Michel electron reconstruction: ionization spectra

Plane U Plane V Plane Y

• Michel electron ionization component reconstructed at lower energies in SN stream.
• Effect more pronounced for planes U & V (smaller signals and higher thresholds).

Well reproduced by zero-suppression emulation.

José I. Crespo-Anadón 25

Michel electron reconstruction: radiative spectra

Plane U Plane V Plane Y

• Michel electron radiative component reconstructed at higher energies in SN stream.

Well reproduced by zero-suppression emulation.

José I. Crespo-Anadón 26

Michel electron reconstruction: Interpretation

Channels with high zero- suppression thresholds radiative* e ionization*

José I. Crespo-Anadón 27

Michel electron reconstruction: ionization length

Plane U Plane V Plane Y

• Confirmation: Michel electron ionization component is shorter in SN stream.

Well reproduced by zero-suppression emulation.

José I. Crespo-Anadón 28

Michel electron reconstruction: ionization hit multiplicity

Plane U Plane V Plane Y

• Confirmation: Michel electron ionization component has fewer hits in SN stream.

Well reproduced by zero-suppression emulation.

José I. Crespo-Anadón 29

Michel electron reconstruction: radiative hit multiplicity

Plane U Plane V Plane Y

• Confirmation: Michel electron radiative component has an excess of hits in SN stream.
• Ionization and radiative components compensate when integrating over the full Michel electron.

Well reproduced by zero-suppression emulation.

José I. Crespo-Anadón 30

Michel electron reconstruction: Hit-level analysis

José I. Crespo-Anadón 31

Michel electron reconstruction: ionization hit spectra

Flipping bits (more common on plane Y) Plane U Plane V Plane Y

• Simulating flipping bits (not shown), we estimate a 10% additional resolution on hit energy.
• Michel e reconstruction has a dominant 20% energy resolution due to non-reconstruction of low-

energy photons. Additional flipping bit effect is acceptable, but investigation of origin continues.

José I. Crespo-Anadón 32

Michel electron reconstruction: ionization hit spectra

Plane U Plane V Plane Y

• Limited number of presamples in FPGA firmware (7)

does not fully capture some of the pulses on plane U.

Compton- like tail

José I. Crespo-Anadón 33

Michel electron reconstruction: radiative hit spectra

Plane Y

• Flipping bits manifest as a peak at 0.1 – 0.2 MeV.
• Zero suppression also creates low energy radiative hits which are

similar to the flipping-bit ones.

• Induction planes show similar behavior.

José I. Crespo-Anadón 34

The Continuous Readout Stream as a development platform

• Continuous readout of the TPC is the first stage towards a

self-trigger using TPC patterns.

• MicroBooNE is the first LArTPC with continuous readout.
• For designing a TPC-based trigger, the effects reported in

previous slides are important. Michel electrons will appear as lower energy (lower multiplicity, lower length). → Impact on choice of thresholds.

• For a TPC trigger that uses induction planes (for redundancy
• r to extend the trigger to 3D), the suppression of signals
• bserved on plane V can cause inefficiencies.

Plane V signals are smaller (more challenging to separate from noise) and symmetrical (destructive interference for some topologies) due to shielding by first induction plane.

Michel e ionization

• n plane Y

Michel e ionization

• n plane V

José I. Crespo-Anadón 35

• C. E. Hinrichs (NSF REU student) studied

impact of DUNE trigger primitive definition

• n Michel electrons using MicroBooNE

Continuous Stream Data. https://www.nevis.columbia.edu/reu/2019/rep

• rts/Hinrichs_report.pdf

Iris Ponce (Columbia student) working on porting the Michel electron reconstruction for ProtoDUNE triggering.

• SBND will have a Continuous Readout

Stream based on MicroBooNE design.  Platform for testing and demonstrating TPC hit primitive, candidate, trigger generation.

The Continuous Readout Stream as a development platform

Trigger Prim. Gauss Hits

José I. Crespo-Anadón 36

Conclusion

• MicroBooNE Continuous Readout Stream: a new way to look at data from the

MicroBooNE detector. Grants the possibility to acquire the neutrino burst from a core-collapse SN using the SNEWS alert. Selection and reconstruction of SN neutrinos for a future work.

• Achieved target compression factor (~80) using FPGA-based algorithms. Zero

suppression with static baselines and bandwidth-driven thresholds provide best results.

• Using Michel electrons from stopping cosmic-ray muons as proxy for SN neutrinos.

Relative efficiency of (93.0 ± 0.8)% on plane Y, (72.4 ± 0.5)% on plane V, and (94.8 ± 0.7)% on plane U (stat. only). Not including seasonal variation.

• Flipping bits affecting 4% of ADC samples. Mitigated offline. 10% impact on resolution.
• Laying the foundation for a TPC-based trigger. Lessons learned:

Zero suppression causes Michel electron ionization to be reconstructed at lower energies. Zero suppression has stronger impact on middle induction plane (smaller signals due to shielding by the first induction plane). Well reproduced in simulation.

• Expansion of the non-beam physics program of MicroBooNE: SN neutrinos, but also

e.g. prototyping nucleon decay searches (p decay, neutron-antineutron oscillation...) in a LArTPC.

José I. Crespo-Anadón 37

Backup

José I. Crespo-Anadón 38

First fully automated electron reconstruction.

• Input from Pandora pattern recognition..

Eur.Phys.J. C78 (2018) no.1, 82

• Uses muon Bragg peak and decay kink to

select events.

JINST 12 P09014 (2017) 30º 80 cm

Cosmogenic Michel electrons in MicroBooNE

José I. Crespo-Anadón 39 3.2 ms

Two parallel TPC readout streams

Triggered stream

• “Still shots” of the detector.
• Readout upon reception of a trigger (BNB,

NuMI, external...).

• Read an event: 1.6 ms before trigger and 3.2

ms after trigger.

• Waveform of 9600 samples (2 MSps).

Lossless compression (Huffman). Continuous stream

• A “movie” of the detector.
• Continuous readout.
• Read frames of 1.6 ms.
• Region Of Interest (ROI) of variable length.

Lossy compression: zero suppression + Huffman compression.

Time Frame 1.6 ms Frame 1.6 ms Frame 1.6 ms Frame 1.6 ms Frame 1.6 ms Frame 1.6 ms Time 1.6 ms Trigger ROI ROI ROI ROI ROI ROI

Each stream has a DRAM to buffer the processed data until retrieved. Data processing by FPGA (Altera Stratix III). FPGA splits data in two streams: