35t FEMB Status and Noise 20160224 - 35t - B. Kirby TPC Readout - - PowerPoint PPT Presentation
35t FEMB Status and Noise 20160224 - 35t - B. Kirby TPC Readout - - PowerPoint PPT Presentation
35t FEMB Status and Noise 20160224 - 35t - B. Kirby TPC Readout Noise - Main Issues Two main problems with TPC readout noise (in order of importance): Noise levels are not stable: all FEMB channels have some chance to jump into a
TPC Readout Noise - Main Issues
- Two main problems with TPC readout noise (in order of importance):
○ Noise levels are not stable: all FEMB channels have some chance to jump into a “high-noise” state that can only be removed by power-cycling the ASICs ○ Noise levels are high: when the readout is not in a bad state the noise levels are still high
2
3
“High Noise” State
RMS >100 ADC Pedestal ENC > 20000 e- Oscillatory signal Affects entire detector, never occurs on single FEMB
Recent Noise Tests
4
Date ELOGs Test Feb 17 5917 Power off everything non-TPC related Feb 18 6004-6030 Change FE ASIC baseline to 200mV, vary shaping time settings Feb 19 6055-6066 Power off everything non-TPC related for extended time Feb 19-20 6067-6112 Overnight stability test with everything off Feb 21 6148 Run with only one FEMB with ASICs on, all other FEMB ASICs off Feb 21 6149-6153 Turn on FEMB ASICs sequentially Feb 22 6207-6273 Run with one FEMB with ASICs on a time, vary shaping time settings Feb 22-23 6275-6308 Alternate wire-bias and drift field on Feb 23 Turn on FEMB ASICs sequentially, correlate power draw with high noise Many tests, main goals were:
- check if high noise caused by an external source
- check if high noise caused by single board
- check if high noise caused by FEMB power on/reconfiguration
Run 11481 Run 11504
- APA3 channels started producing realistic noise following power-cycle
- Clearly see more noise when everything on vs everything off (~1500e- ENC)
- Did not see “high-noise” state when everything off, however test was too short
Run State Collection Noise (ENC) Induction Noise (ENC) Collection Noise 11kHz Subtracted Induction Noise 11kHz Subtracted 11481 Everything Off 2310 e- 4060 e- 1310 e- 2660 e- 11504 Everything On 2940 e- 4780 e- 1990 e- 3570 e-
5
Feb 17 “Everything Off” Test Results
Feb 18 FE-ASIC Configuration Scan
- Changed FEMB channel baseline settings on all channels to 200mV to
evaluate effect on noise
- Varied shaping time settings to see if any setting was more or less
susceptible to high noise state
○ At this point non-TPC related systems were turned back on ○ All FEMB ASICs were kept ON throughout the test
- High-noise state was observed on all configurations tried
○ High-noise occurred randomly, removed via power-cycling or wire-bias trick
- Conclusion: FE ASIC settings do not affect whether “high noise” occurs
6
Feb 19 “Turn Everything Off” Test - Powering Down
7
Run # ELOG# System State 6055 Lakeshore, thermo cables, RTDs unplugged, SSPs off, LCM off Drift HV off, wire bias off, cameras + pumps + purity monitors on 11765 6066 All ASICs power-cycled, everything off, “high noise” state
- Turned off all non-TPC readout related systems to see if “high noise” state
- ccurred in the absence of potential external noise sources
○ “High noise” detected run 11765
- Conclusion: “high noise” can occur when non-TPC related systems are off
Feb 19-20 Overnight Stability Test
- Wanted to measure how long FEMB could run without changing settings until
high-noise issue occurred
○ This was done with the HV and wire-bias OFF
- FEMBs took data without high noise for ~24 hours
8
Run State Collection Noise (ENC) Induction Noise (ENC) Collection Noise 11kHz Subtracted Induction Noise 11kHz Subtracted 11768 Drift field + wire bias off 2310 e- 2700 e- 1530 e- 1980 e-
Feb 21, 22 Single FEMB Runs
- Can control LV power supply to ASICs on each FEMB independently
- Turned off ASIC power on all boards, all other LV power channels left on
○ Emphasis: FPGAs were left on, have not been turned off since detector cooldown
- Sequentially turned on ASICs on one FEMB, took data
○ This was done with drift field + wire-bias off
- High-noise state didn’t occur
○ >15 FEMB ASIC power cycles, >45 ASIC reconfigurations
- Suggests high noise doesn’t occur when single FEMB running
9
Alternate Wire-Bias and Drift Field
- Set of runs taken with wire-bias and drift field alternately on to determine if
ionization charge can cause high-noise state in FEMBs
○ HV off, wire-bias nominal values: Runs 12453-12517 ○ HV drift on (60kV), wire-bias off: Runs 12538-12691
- Still analyzing...
10
Feb 21 Sequentially Turn on FEMB ASICs
11
- Lower than normal power-draw on ASIC
LV supply correlated with “high noise” runs
- Current draw not logged in plot, but
- bservations suggest drop in power
associated with change in current draw
- N. Barros
Feb 23 Sequentially Turn on FEMB ASICs
12
- Continuation of sequential FEMB ASIC turn-on tests
○ Turned on RCE15 FEMB ASICs, observed lower power draw then normal ○ Turned on ASICs for FEMBs on RCE14-12 ○ RCE11 FEMB ASICs turned on, ASIC power draw dropped on FEMB 13-15 ○ “High noise” observed in next data run, removed by turning FEMB 14-15 ASICs off ○ Effect is reproducible
- Conclusion: high noise is correlated with abnormal ASIC current draw and
can be induced by changes in ASIC current draw on single FEMBs
- N. Barros
Feb 23 Sequentially Turn on FEMB ASICs II
- Similar tests were done with FEMBs on different APAs
- Preliminary result:
○ Noise state depended on FEMBs on APA3 ○ If APA3 FEMBs were noisy (ie had abnormal power draw), then FEMBs on APA0 would also be noisy when ASICs were powered on ○ If APA3 FEMBs were normal, FEMB ASICs on APA0 could be powered on and the following run was not noisy
- Suggests one or more FEMBs on APA3 are specifically problematic and
induce noise in all other FEMBs, but needs more tests
○ Reminder: before FEMB ASICs were power-cycled in the cold APA3 channels had essentially turned off, electronics response gain was very small ○ Explains why wire-bias “trick” would work when power-cycling would not, APA3 boards have a high chance of entering “high noise” state and inducing problem in rest of system
13
Summary of Noise Tests
- Understanding high-noise state as been focus of recent tests:
○ Demonstrated high noise can occur when most non-TPC related systems are off ○ Choice of FE ASIC settings does not affect incidence ○ System can be stable on ~24 hour timescale, when only single FEMB is running ○ High-noise state correlated with abnormal FEMB ASIC power draw (clear hardware issue) ○ High-noise can be reliably induced by changes in LV current draw on single board
- Noise levels are still generally high
○ Changing baselines to 200mV seems to have helped at cost of induction signal dynamic range ○ Non-TPC related systems seem to introduce ~1500e- ENC in TPC readout
14
Plans
- Focus on understanding how changes in current draw on single FEMB affects
- ther boards, introduces high noise state:
○ Continuation of Feb 21, 23 sequential power on tests ○ Record exactly the ASIC LV current draw at each stage when FEMBs are turned on ○ Explore changes in power-on procedure ○ Demonstrate if ASIC reconfiguration has similar effect
- Identify stable operating configuration:
○ Can we identify a set of FEMBs/power on sequence/reconfiguration process that largely prevents high-noise state from occurring? ○ Determine to what extent (if any) running with drift field and wire-bias on affects incidence of high-noise state
15
Backup
16
35t Recent FEMB History and Noise I
- Initial post-cooldown noise high compared to room temperature, due to increase in 11kHz correlated noise
○ Run 10115-10123 - pedestal runs taken at all FE-ASIC settings ○ Wire bias + HV off through up to 10151 ○ Run 10119 noise: collection wires ~2030e-, induction wires ~2750e-
- Wire-bias turned on (ELOG 4995, Feb 5)
○ Very high noise observed starting run 10164 (ELOG 5018), noise persists after wire-bias turned off ○ Run 10195 noise: collection wires ~21000e-, induction wires ~22000e- ○ Reduced with wire-bias ramping “trick” (ELOG 5089, 5144, 5146, Feb 6-7)
- Drift field ramped up with wire-bias on and trick used (ELOG 5187)
○ Run 10400 taken with wire-bias + HV ON ○ Run 10400 noise: collection wires ~2860e-, induction wires ~3090e- ○ Pulser runs 10422-10431, wire bias ON, HV off (Feb 8), pulse signals look OK
- Noise levels fairly stable from Feb 8th to 11th
○ Run 10788 taken with wire bias on, drift field off before pumps turned on ○ Run 10788 noise: collection wires ~2770e-, induction wires ~3050e-
17
35t Recent FEMB History and Noise II
- LAr Pumps turned on (ELOG 5376, Feb 11)
- FEMB pedestals set back to “default” values (ELOG 5450, Feb 11), Run 10852
○ Run 10830 noise: collection wires ~2810e-, induction wires ~3220e- ○ Run 10852 noise: collection wires ~3020e-, induction wires ~4450e-
- Drift field ramped up with pumps on (ELOG 5479, Feb 12), Run 10895
○ Run 10895 average noise: collection wires ~3050e-, induction wires ~4560e-
- FEMB noise increased (ELOG 5574, Feb 12 ~6pm)
○ Started ROUGHLY run 10949, might be earlier ○ High noise has persisted, temporarily reduced by wire-bias ramping trick but still very high
- Changed FE ASIC settings to 4.7mV/fC, 3us (NO ELOG, Feb 13)
- High noise persists on all settings, occasionally increases to extremely high noise (>10000e-)
○ Run 10895 (Feb 12) average noise: collection wires ~6000e-, induction wires ~8000e- ○ APA3 channels have gone “quiet”, relatively low noise and very low gain
18
Recent Noise Tests - I
- Powered off “everything” non-TPC related
○ Goal: check for external noise sources in measured noise levels ○ ELOG 5917 - Feb 17 - 15:30 to 18:30
- Changed induction wire channels’ baseline to 200mV, varied FE ASIC configs
○ Goal: Check if choice FE-ASIC configuration causes “high noise ○ ELOG 6004 to 6030 - Feb 18 - 18:45 to 23:11
- Powered off “everything” non-TPC related (again!)
○ Goal: check if “high noise” occurs when all likely external noise sources off ○ ELOG 6055 to 6066 - Feb 19 - 14:45 to 16:30
- Check if noise is stable with everything off
○ Goal: See how long can run with everything off for “high noise” state to occur ○ ELOG 6067 to 6112 - Feb 19 16:37 to Feb 20 16:02 (~24 hours)
19
Recent Noise Tests - II
- Run with just RCE00 FEMB ASICs, turn other FEMB ASICs off
○ Goal: Measure effect on noise and check for “high noise” ○ ELOG 6148 - Feb 21 18:17
- Turn on FEMB ASICs sequentially
○ Goal: Check if “high noise” occurs when additional FEMB ASICs turned on ○ ELOG 6149 to 6153 - Feb 21 - 18:17 to 19:38
- Turn on one FEMB at a time, take data with different configurations
○ Goal: Check if “high noise” occurs with only one FEMB on, if reconfiguring induces it ○ ELOG 6207-6273 - Feb 22 - 11:28 to 17:53
- Alternate data-taking with drift field and wire bias on
○ Goal: see if ionization charge causes “high noise” ○ ELOG 6275-6308 - Feb 22 18:33 to Feb 23 08:35
- Turn on FEMB ASICs sequentially (again!)
○ Goal: Correlate changes in FEMB ASIC power draw with “high noise”
20
Feb 17 “Turn Everything Off” Test - Powering Down
Run # ELOG# System State 11468 5918 Wire-bias, drift field voltage supplies turned off, PC4 lights off 11469 PTB + CCU units also turned off 11471 SSPs, purity monitors and cameras turned off 11473 RTDs unplugged 11475 ASICs on RCE03 and RCE14 FEMBs power-cycled 11477 ASICs on RCE00 FEMB power-cycled 11479 ASICs on RCE00-03 FEMBs power-cycled 11481 5927 All ASICs power-cycled, “everything off”
21
Feb 17 “Turn Everything Off” Test - Powering Up
Run # ELOG# System State 11484 5929 Turned on purity monitor, RTD, PC4 11487 SSPs turned on 11488 CCU and PTB turned on 11492 Wire-bias ramped up, “high noise” state observed 11493 ASICs power-cycled, “high noise” removed 11494 Drift field bias ramped up 5936 Cameras back on 11504 5940 “Everything on”
22