35 Ton Run 2 Hardware and Operational experiences Alan Hahn - - PowerPoint PPT Presentation
35 Ton Run 2 Hardware and Operational experiences Alan Hahn 4/14/16 Dune Friday Phone Meeting 1 Organization Overall Run Cryo Detector Hardware Categories TPC Including HV Cold Electronics Photon Detectors
4/14/16 Dune Friday Phone Meeting 1
4/14/16 Dune Friday Phone Meeting 2
4/14/16 Dune Friday Phone Meeting LAPD Pump trip XFer from LAPD GAr xfer from LAPD (over weekend) ~1.5 tons Top-Off from TRAILER Membrane Temperature sensors 3
4/14/16 Dune Friday Phone Meeting
PrM2 & 3, both long PrMs midway heights in Cryostat(these lifetimes may drop a bit (~10%) using PrM0 as a constraint)
Sitewide Power Outage LN2 Cooling Loss Tubing Break Pump Start
4
3 ms
Bad News Period
4/14/16 Dune Friday Phone Meeting
PrM 1 (near Cryostat bottom) PrM 4 (near surface) O2 Gas Analyzer, ~(~80 cm from bottom)
5
4/14/16 6 Dune Friday Phone Meeting
4 purity monitors in this corner (Geometry Not in CFD model) PrM1 Pump intake Pump outflow
Discussion of Results / Lessons Learned
Phase separator cools inlet flow to approach saturation temperature, which is below the “bulk temperature” of the argon. This cooler, more dense layer of argon tends to stay at the bottom, and not mix well with the rest of the argon. Back pressure control at the phase separator could allow us to flow in warmer argon, which would then mix well with the bulk.
4/14/16 7 Dune Friday Phone Meeting
Pump Suction Pump Discharge Pipe with Phase Separator on Top, LAr flows down, and impinges on cryostat floor.
New constraint
RTD measurements previous slide verifies this temperature stratification
4/14/16 Dune Friday Phone Meeting 8
Pump is off PrM 1&4 show same lifetime 3 RTD offsets made to match Barometric Pressure Pump is on Uppermost RTD Lowest RTD PrM 1&4 lifetimes diverge
12 hrs after Outage
1. Site-wide power outage stopped our recirculation and dropped the LAr Purity (3/4—Friday Night)
2. Loss of LN2 Cooling (broken valve) caused a loss of LAr (~1/2 ton through boil off) (3/12--Saturday)
previous week’s Power Outage—Doh!
– Only noticed on Sunday and fixed in afternoon
3. Unrecoverable Contamination of LAr (3/17-Saturday 2 AM)
transfer GAr between LAPD and 35Ton broke- pulling in air and injecting it into the LAr stream.
– Estimate it would take ~20 weeks of filtering/filter regeneration to remove just the O2, N2 would not be removed.
4/14/16 Dune Friday Phone Meeting 9
4/14/16 Dune Friday Phone Meeting
10
Bellows Pump
– Had problems establishing the 35Ton Internal LAr Pumping – Pumps had been rebuilt after Phase 1, but had never been run in. – Finally putting 3 phase 208 AC on pump motor broke it free.
– Lesson learned—nothing goes into cryostat without a functionality test
– Autodialer not working cost us LAr level when LN2 valve stopped functioning.
– Painful to use the Cryo Interface at time – incredible slow downs.
– I believe there will be a review of exactly what happened and that will suggest several lessons to be learned from
time for people to recognize what is going on.
4/14/16 Dune Friday Phone Meeting 11
– Thanks Jack Fowler, Dan Wenman, Bo Yu, Lee Scott, Bob Kubinski – Demonstrates that careful planning actually works!
– 60 kV (@Cathode) in “clean” LAr
“risking” raising the HV higher and potentially breaking something.
ramping back up after first breakdown).
contamination
– Large disappointment with LAr contamination is that we weren’t able to test Cathode HV at 120 kV in pure LAr – However we did continue the tests in what we had
– Most info is in elog
4/14/16 Dune Friday Phone Meeting 12
4/14/16 Dune Friday Phone Meeting 13
breakdowns 67kV 133kV 150kV 135kV 150kV
PowerOutage contamination Cooling Loss
These are Voltages at Power Supply , ~10% higher than at cathode.
4/14/16 Dune Friday Phone Meeting 14
4/14/16 Dune Friday Phone Meeting 15
Recent data from Cold Run17366 3/16)
Test done by Bo Y. and Matt W. at BNL with 3 Bd
Data from 4 Bd taken at DAB (7/15)
4/14/16 Dune Friday Phone Meeting 16
4/14/16 Dune Friday Phone Meeting 17
“Normal” Noise State Very High Noise State APA Long drift Short drift
recognized as the serious problem it was.
4/14/16 Dune Friday Phone Meeting 18
– All signals/electrical services to APA/PD pass through this Board. – Had serious issues with initial version – Second version much better
voltage
– Slow start, but eventually it wsa providing our triggers
– Allowed us to trigger on tracks that are useful for looking at the APA features.
– Very problematic at DAB and early at PC4. Often had to “reseat” the
– Settled down at some point and were less of a problem for rest of run.
– Once things calmed down, they appear to be working fine.
4/14/16 Dune Friday Phone Meeting 19
– Turned out to be one of the highest risk items for the 35 T run (until Tubing at least). – Contains DAQ Servers, RCE Systems, Network Switches in a closed Air Condition Rack – The power for the Air Conditioning was put on an interlock that was tied into the Rack Smoke Detection Module and was not plugged into the rack power. Also an internal power switch for Air Conditioning could be switched off. – Three“near miss” incidents
disabled, with the rack doors closed. However these were caught before anything happened.
in > 60C range as reported by Network switch in rack
readout wasn’t working at ROC-W. He opened the doors, things cooled off. – After this last episode, we established
Conditioning fails. – Recommendation that one needs to do a careful risk analysis on a system like this.
– For better or worse, the DAQ servers were not setup as Fermi Standard, so we had several cases where we were halted because the experts were not available to get the systems running. – DAQ Running.
4/14/16 Dune Friday Phone Meeting 20
concrete, and that was then welded to the A&B plates (which were welded to the stainless steel membrane).
as to what the impedance between detector and building grounds should be, since the concrete impedance was sensitive to moisture, and that varied a lot from the installation to the running period.
4/14/16 Dune Friday Phone Meeting 21
4/14/16 Dune Friday Phone Meeting 22
Carbon Steel Vapor barrier Inside the Concrete Shell—note the iron
each other
it should be lower, rather than blaming external factors as causing the noise.
equipment is working (or not).
4/14/16 Dune Friday Phone Meeting 23
4/14/16 Dune Friday Phone Meeting 24
4/14/16 Dune Friday Phone Meeting
LAr Level Drop ~ 3.8 cm ~1/2 ton RTD on tip of Corrugation Cryostat Pressure RTD on flat of top Membrane.
25
4/14/16 Dune Friday Phone Meeting 26
“Corrugations” Cathode (up to 135 kV) APA’s
4/14/16 Dune Friday Phone Meeting 27