Some DUNE Perspectives on Event Overlays Tom Junk LArSoft Overlay - - PowerPoint PPT Presentation

Some DUNE Perspectives on Event Overlays

Tom Junk LArSoft Overlay Workshop March 5, 2019

The Extremes of DUNE

• One Far Detector module will be enormously larger than any LArTPC ever
• built. 30x ProtoDUNE-SP in size.
• DUNE will be the farthest underground of any LArTPC
• Physics program includes Supernova burst and Nucleon Decay studies
• ProtoDUNE-SP is the largest monolithic single-phase LArTPC built to date.
• One FD-SP module will have 384,000 channels. If we build four of these, it'll

be 1.5M. Dual-phase will have fewer channels, but longer drift, and so data volume is similar per event.

• The recommended Near Detector will have a pixel readout LArTPC and a pixel

readout Gaseous Argon TPC, and a 3D scintillator tracker. LArTPC and GArTPC each have ~1M channels.

• Intensity of neutrino beam à millions of ND events, many thousands of FD

events (nue appearance is still rare – hundreds).

• DUNE will run multiple decades
• DUNE has over 1000 collaborators from a large number of countries

Mar 4, 2019

• T. Junk | DUNE Overlays

2

Claims to [ Fame | Infamy ]

• 35t was the noisiest large LArTPC.
• ProtoDUNE-SP has the biggest space-charge effects of any

LArTPC.

• ProtoDUNE-DP will have even bigger space-charge effects
• We are still uncertain about FD-DP space-charge effects.
• Electronics effects in all of the detectors are nontrivial
• saturation
• sticky ADC codes
• Timing skews

Mar 4, 2019

• T. Junk | DUNE Overlays

3

Event Overlays in 35-ton

• It's already been done!

Mar 4, 2019

• T. Junk | DUNE Overlays

4

Some online monitor event displays s:n=2:1 in the induction planes on a good day high noise state "low noise" state

35-ton Data Overlays

• Matt Thiesse's thesis: FERMILAB-THESIS-2017-32: --

measuring the electron lifetime in the 35-ton prototype

• Matt had to use the data as the noise model
• Noise was large
• Changed by large amounts over time
• Different channels/APA's had different amounts of noise
• Noise affected his hit-finding efficiency and resolution, biasing purity

mesurements

• MC modeling of the signal wasn't all that great, either. But that's a

smaller set of parameters to vary

Mar 4, 2019

• T. Junk | DUNE Overlays

5

Hit Efficiency and Resolution Models

Mar 4, 2019

• T. Junk | DUNE Overlays

6

Done with overlaid MC signal on 35-ton Data noise

• M. Thiesse

Additional 35-ton Issues

• DAQ was designed for continuous readout
• Hardware bandwidth was insufficient (GBit links, not enough)
• Expected with low noise (really high signal-to-noise), zero suppression and

compression would let us save "all" the data.

• Noise prohibited this. Firmware was also not developed in time, as developers

knew ZS would likely not work (and could always be explored later in software)

• But the readout and triggers were misaligned. Readout runs continuously, but
• nly some blocks can be saved.
• artdaq events had arbitrary boundaries with respect to the triggers, though the

data were there, just sometimes on the next event

• Offline, we had to trim the time windows, and stitch events together

http://indico.fnal.gov/event/12068/session/1/contribution/9/material/slides/1.pdf

• We wrote our own input source (T. Junk, K. Warburton, with help from M. Paterno and K.

Knoepfel)

Mar 4, 2019

• T. Junk | DUNE Overlays

7

Relevant for Supernova Burst Trigger

• LArTPC's produce a continuous stream of data
• We can only read out a subset of it
• Supernova Burst Triggers want to read out a very long time (~30

seconds or more)

• We will need to packetize the data somehow
• Overlay schemes will have to think about this.
• Need a sample of zero-bias data on which to overlay
• Supernova burst triggers (most are fake) will be biased.
• Overlaying a simulated SNB on top of zero-bias data will require

a workflow that has more steps than reading all the data in, adding it, and processing it all in one art event-loop call.

Mar 4, 2019

• T. Junk | DUNE Overlays

8

ProtoDUNE-SP

• Event overlays not yet performed (that I know of), but they are a

natural way to control systematics

• ProtoDUNE-SP MC uses CORSIKA to model cosmic rays.
• Showering, hadronic interactions, and neutron production and

interactions are all highly suspect.

• Measuring interaction rates means we need detection

efficiencies and resolutions.

• Known beam particle species and momentum
• Upstream material model not perfect
• Some particles get lost on their way in to the detector, so the

denominator of the efficiency may not be perfectly predictable.

Mar 4, 2019

• T. Junk | DUNE Overlays

9

ProtoDUNE-SP

• We have plenty of beam-off data (between spills, interleaved

with spill data).

• Between-spill data lack halo muons however
• MC needs to model space charge (and not noise!) before
• verlaying with data.
• ProtoDUNE-SP data is not zero-suppressed, making life easier.

Mar 4, 2019

• T. Junk | DUNE Overlays

10

A Typical Beam Event in Protodune

Mar 4, 2019

• T. Junk | DUNE Overlays

11

https://www.phy.bnl.gov/twister/bee/set/protodune-gallery/event/14/?camera.ortho=true Many thanks to Chao Zhang and collaborators for a very nice event display! 7 GeV electron shower plus cosmic rays plus beam halo

ProtoDUNE-SP Electronics Issues

• Nonlinearity (differential and integral)
• Saturation
• Front-End
• ADC
• Ledge Effect (front-end)
• Undershoot
• Long-range induction effects
• Coherent Noise and removal algorithm
• Sticky ADC codes
• Dead and noisy channels

Mar 4, 2019

• T. Junk | DUNE Overlays

12

Mar 4, 2019

• T. Junk | DUNE Overlays

13

Matt Worcester Talk @ DRA meeting

Sticky Codes

Mar 4, 2019

• T. Junk | DUNE Overlays

14

Example from FEMB 302 David Adams, talk at the ProtoDUNE DRA meeting

Saturation and Ledge Effect

Mar 4, 2019

• T. Junk | DUNE Overlays

15

• D. Adams's

Raw Event Display And Undershoot

Saturation and Ledge

Mar 4, 2019

• T. Junk | DUNE Overlays

16 Pulse with Large Charge over Short time

Still alive here

Exponential, low RMS channel blind to new charge

Recovery

Less common in induction planes but we see it there too

A Big Shower Showing Long-Range Induction Effects

1/31/19 Tom Junk | Interplane Induction 17

Collection V U

Undershoot (and correction)

1000 2000 3000 4000 5000 6000

t [ticks]

60

• 40
• 20
• 20

40 60 80 100

q [ADC] Mar 4, 2019

• T. Junk | DUNE Overlays

18

• T. Junk, DUNE DocDB 11662

This is all linear, so no big deal (except for saturation cases)

Undershoot (and correction)

1000 2000 3000 4000 5000 6000

t [ticks]

60

• 40
• 20
• 20

40 60

q [ADC] Mar 4, 2019

• T. Junk | DUNE Overlays

19

Example where event started during the recovery from undershoot. Correct the data first then overlay? Or overlay and correct? What about adding data to data?

Ajib Paudel's Median dQ/dx Z plane (y,z)

1/31/19 Tom Junk | Interplane Induction 20

APA 1 APA 2 APA 3 And similarly for APA's 4, 6, and 5 Beam Right

Temporal Variation

1/31/19 Tom Junk | Interplane Induction 21

Run 5460 and others Run 5387 Ajib gave me two sets of histograms from different runs, and they all looked similar in shape (some scale offsets however) except for this one. Could be field cage charging.

ProtoDUNE-DP Electronics

• High gain expected at the LEM (up to 180 seen in prototypes)
• Protect against saturation while keeping resolution for small

signals: double-slope gain (deliberately nonlinear response function!)

Mar 4, 2019

• T. Junk | DUNE Overlays

22 WA105 TDR L. Agostino et al., https://arxiv.org/abs/1409.4405 CERN-SPSC-2014-013; SPSC-TDR-004

Nonlinear Response

• Deconvolution is a linear operator on the data
• Need to adjust the input data to a linear scale before

deconvolving or adding other signals to it

• Re-apply amplifier model after signal superposition

Mar 4, 2019

• T. Junk | DUNE Overlays

23

Unipolar Signals on Induction-Plane Wires

Sept 25, 2018 Tom Junk | ProtoDUNE-SP Data 24

Bipolar Unipolar

Wire Tick M Z V U G Bipolar V waveforms Unipolar V waveforms "Backwards" Z drift Signal processing will have to cope with some unusual waveforms as tracks cross through the wire planes.

V

Signal Response Nonuniformities

• A mixture of space charge, field-cage nonuniformity, and APA

construction/transparency issues. Electron diverters

• These are features of the data and not (yet) of the MC. Overlay

MC without these features and a CVN may key on them instead

• f the physics it's supposed to.

Mar 4, 2019

• T. Junk | DUNE Overlays

25

Mar 4, 2019

• T. Junk | DUNE Overlays

26

Alex Radovic, Fermilab Machine Learning Group mtg, Jul 2017

Mar 4, 2019

• T. Junk | DUNE Overlays

27

Alex Radovic, Fermilab Machine Learning Group mtg, Jul 2017

Mar 4, 2019

• T. Junk | DUNE Overlays

28

Alex Radovic, Fermilab Machine Learning Group mtg, Jul 2017

Mar 4, 2019

• T. Junk | DUNE Overlays

29

Alex Radovic, Fermilab Machine Learning Group mtg, Jul 2017

Mar 4, 2019

• T. Junk | DUNE Overlays

30

Zero Suppression: Regions of Interest

Mar 4, 2019

• T. Junk | DUNE Overlays

31

Jonathan Insler, 35ton MC. LBNE DocDB 11053

Front Porch Back Porch: ticks before and after threshold saved to catch sub-threshold behavior and help deconvolution DAQ group balks at "side" porches

Overlays and Zero Suppression

• Don't mix very well!
• Information lost in the samples below threshold.
• Signals that are below threshold may push above threshold onlh

if there is some noise to add to it!

• Fill in missing ticks with a noise model perhaps?
• Need to re-apply threshold after overlay, so that a CVN cannot

latch onto the artificial difference

Mar 4, 2019

• T. Junk | DUNE Overlays

32

Coherent Noise Subtraction

• Not much present in ProtoDUNE, but a big deal on 35-ton.
• We hope this is not a problem for DUNE
• Relies on calculations of medians. Not a strictly linear process
• Medians calculated protecting signals. Definitely not a linear

process.

• If you overlay a MC signal, do you have to re-do the signal

protection and coherent noise subtraction?

• May leave artifacts in the image that a CVN can pick out. ("I can

tell that's been photoshopped!")

Mar 4, 2019

• T. Junk | DUNE Overlays

33

Door vs. Beach

Mar 4, 2019

• T. Junk | DUNE Overlays

34

Practical Issues Overlaying MC on Data

• DUNE's uncompressed events are very big – 3-4 GB apiece.
• We are thinking about how to process them in pieces. Future

computers may be heavy on cores, with less RAM per core.

• Overlays would have to adapt to a processing of less than a full

event if we need to go that route.

• Simpler issues: What are the run and event numbers?
• Event type? How do we store this data so we don't get

confused?

Mar 4, 2019

• T. Junk | DUNE Overlays

35

Photon Detectors and CRT

• We haven't even started overlaying signals here.
• Should have a tool for each subdetector.
• 35-ton event splitter did split all pieces of the event – TPC,

photon detectors, and MC particles/MC truth

• Overlaid events have what for MC truth? Maybe they are "data"

Mar 4, 2019

• T. Junk | DUNE Overlays

36

DUNE's Near Detector Complex

Mar 4, 2019

• T. Junk | DUNE Overlays

37

Pixel Readout LArTPC non-magnetized Magnetized High-pressure gaseous Argon TPC (0.4 T, 10 bar) to S. Dakota

• M. Kordosky

We also propose a 3D Scintillator Tracker

ECAL Liquid Argon Gaseous Argon

ALICE

Jan 29, 2019 Tom Junk | GArSoft Tracking 38

The ALICE TPC at CERN

• 18 Sectors, two sides. Anodes on either end, cathode in the middle.
• Outer radius: 2.58 meters, inner radius 0.788 meters (1.7 meters

active).

• Length: 5.19 meters
• Gas 1 Atm Ne (+CO2+N2)
• B = 0.5 Tesla
• Alice will upgrade the

readout chambers for HL-LHC.

• Pad readout: total

557,568 channels

Jan 29, 2019 Tom Junk | GArSoft Tracking 39

From the ALICE TPC TDR

June 22, 2018 Tom Junk | HPGTPC Simulations 40

CERN-OPEN-2000-183

Zooming in on the edge

Filling the Hole

June 22, 2018 Tom Junk | HPGTPC Simulations 41

• Rectangular array of pixels

in a disk

• Pixel size: 6mm x 6mm

c.f. 4 mm x 7.5mm for inner pad rows.

• Central disk has 62060 pads

per detector side.

• Total channels per side is

now 354200. About 18%

• f channels are in the disk.

Jan 29, 2019 Tom Junk | GArSoft Tracking 42

Pb-Pb collision in ALICE -- up to 8000 tracks per unit rapidity

Collider people "embed" objects of interest in data events to test efficiencies and

• resolutions. e.g. J/psi, photons, leptons....

A "Typical" Raw Event in the HPGTPC

Jan 29, 2019 Tom Junk | GArSoft Tracking 43

This one's near the side. Hits in simulation drifted towards nearest pads

• - distorted near

field cage. Chamber almost empty!

Zoom in on the Primary Vertex

Mar 4, 2019

• T. Junk | DUNE Overlays

44

Limitations on resolution:

• Diffusion
• Pixelation of

readout

• Pulse shapes
• physics (delta

rays, etc).

DUNE ND Future Issues

• We're still putting our software together
• LArPix: edep-sim, probalby moving to LArSoft in the future
• HPGTPC: edep-sim + GArSoft
• 3DST: edep-sim + ???
• Events will exit one detector and particles will enter another

Muons leaving the liquid especially will be measured in the HPGTPC

• Need to overlay liquid, gas, ECAL, 3DST coherently if we're

going that route

Mar 4, 2019

• T. Junk | DUNE Overlays

45

DUNE ND Pileup

• With the intense beam, pileup is an issue: more than one beam-

related interaction will be expected per spill.

• For the gas, the ECAL is 40x heavier and surrounds the gas.

Most pileup events will not originate in the gas.

• We will doubtless want to overlay signal MC on data in order to

get external pileup and backgrounds modeled properly.

Mar 4, 2019

• T. Junk | DUNE Overlays

46

Sometimes the Added Features are an Underlay and an Overlay

Mar 4, 2019

• T. Junk | DUNE Overlays

47

DUNE-Specific Recommandations

• Data are to be corrected and prepared before overlay (if

corrections are significant)

• Need flags for data issues that must survive the correction

process (saturation and ledges)

• Events need to be processed in a segments so as not to run out
• f memory. Overlaid events need to do the same.
• Care about adding signals to zero-suppressed samples.

Mar 4, 2019

• T. Junk | DUNE Overlays

48

Extras

Mar 4, 2019

• T. Junk | DUNE Overlays

49

Generator-Level Overlays

• https://github.com/GENIE-MC-Community/OverlayGenie

Mar 4, 2019

• T. Junk | DUNE Overlays

50