Commissioning and Operation of the New CMS Phase-1 Pixel Detector - PowerPoint PPT Presentation

Commissioning and Operation of the New CMS Phase-1 Pixel Detector Weinan Si University of California, Riverside On Behalf of CMS Phase1 Pixel upgrade team DPF Meeting 2017 Fermilab, Jul31-Aug04

Motivation & Overview Motivation: Original detector not suited for operation at L ~ 2x10 34 cm -2 s -1 Limited bandwidth readout chip to backend ● Overview: CMS Phase1 pixel detector general introduction ● Upgrade ○ DAQ system ○ Readout and control ○ Detector calibration example ● Detector commissioning ● High (random) trigger rate test ○ Cosmic data-taking ○ Collision data-taking ○ Detector status ● Conclusion ● Weinan Si 2 Commissioning and operation of the new CMS Phase 1 pixel detector

CMS Phase1 Pixel Detector Phase1 1 r e y a L x i P B 4 - 2 r e y a L x i P B x i P F Phase0 Additional layer, ★ New CO 2 cooling New ROC: R ead O ut C hip New μ TCA based ★ ★ ★ 87.8% more pixels system New TBM: T oken B it DAQ → closer to beampipe, → reduce material M anager 4 hit coverage budget → analog → digital → handle higher data rate Weinan Si 3 Commissioning and operation of the new CMS Phase 1 pixel detector

CMS Phase1 Pixel Detector 3 disks FPix 4 layers BPix For constructions, see M. Alyari’s talk earlier ~5.6 m D - BPIX -D C Connector Boards B Optical links A DC-DC converters BPix supply CCU links tube (x4) FPix service cylinder (x4) Figure : Mechanic view of Phase1 pixel detector Weinan Si 4 Commissioning and operation of the new CMS Phase 1 pixel detector

DAQ System SFP+ S-Link Express FEROL 12CH (28 + 80) (FPix + BPix) 12CH 2368 μTCA Backplane POH links FED (CTA) Detector mFEC mDOH (x4 links) (672 + 1184) 12 Ch fiber ribbons (FPix + BPix) C-DAQ MCH Portcard/DOH mFEC motherboard (8 + 8) (x4 links) μTCA Backplane Pix FEC (CTA) I 2 C DCDC 400Mbps read-out “fast I 2 C” pixel link Ethernet CCU control link Ethernet control mFEC DOH TCDS: clock, trigger, AMC13 (x4 links) TTS DOH (4 + 8) CCU (1 + 2) Service Tk FEC (CTA) Cylinders TCDS USC μTCA Crate Figure : Complete overview of the μ TCA DAQ system of Phase1 pixel detector Weinan Si 5 Commissioning and operation of the new CMS Phase 1 pixel detector

DAQ System FED : SFP+ Decodes incoming data stream from S-Link Express FEROL detector front-end, assemble all 24 12CH channels data into event fragments, 12CH then pushed to central DAQ μTCA Backplane POH FED (CTA) Detector mFEC Pixel FEC : mDOH (x4 links) 12 Ch fiber ribbons Distribute clock, trigger and fast C-DAQ MCH signals to pixel modules, program Portcard/DOH mFEC motherboard 400Mbps read-out DAC registers of ROC and TBM (x4 links) “fast I 2 C” pixel link μTCA Backplane Pix FEC (CTA) CCU control link I 2 C DCDC Based on CTA card Ethernet control Ethernet (variant of FC7) which TCDS: clock, trigger holds a Xilinx Kintex 7 Tracker FEC : mFEC DOH FPGA AMC13 (x4 links) program auxiliary components in pixel supply-electronics like DOH CCU opto-hybrids and DC-DC converters via I 2 C iterface and PIA port of a Service Tk FEC (CTA) CCU. Cylinders TCDS USC μTCA Crate Figure : Complete overview of the μ TCA DAQ system of Phase1 pixel detector Weinan Si 6 Commissioning and operation of the new CMS Phase 1 pixel detector

System Control & Readout Control : 1. tkFEC programs auxilary electronics a. DC-DC converters → powering b. delay25 → data/clock alignment for pxFEC c. TPLL, QPLL → decoding trigger, clock Tk FEC 2. pxFEC programs front-end ASICs a. TBM settings b. ROC settings Pix FEC Readout: 1. Pixel hit information cached in buffer waiting for trigger and token acknowledgement FEROL 2. Datastream properly formatted and encoded, converted FED into optical signal and transmitted to FED through portcard 3. FED collects data from all 24 fibers (48 channels) and build event fragments, then pushed to cDAQ (1 + 1) Weinan Si (1 + 1) 7 Commissioning and operation of the new CMS Phase 1 pixel detector (1 + 1) (1 + 1)

Detector Calibrations Compared with the old system, the differences are mainly on readout electronics. ❖ We recycled a lot of the software and extended it since much of the functionality of phase1 was also in ❖ phase0. Two new calibrations: POHbias and TBM phase scan. ❖ CMS pixel preliminary POHbias POH bias controls the ● 3. Just adding amount of light sent offset to the digital signal from detector to 2. Not yet added enough DAQ. bias to get the As POH bias ● whole digital increases, more light signal above is sent, and the RSSI the detection ( R eceived S ignal S trength threshold 1. Digital I ndication) values on signal is the FED also increase. below The bias value of the ● detection laser diode is chosen right after the second slope change as indicated by the blue line. 1 Bias setting corresponds to 0.45mA Weinan Si 8 Commissioning and operation of the new CMS Phase 1 pixel detector

Detector Calibrations Compared with the old system, the differences are mainly on readout electronics. ❖ We recycled a lot of the software and extended it since much of the functionality of phase1 was also in ❖ phase0. Two new calibrations: POHbias and TBM phase scan. ❖ CMS pixel preliminary TBM phase scan 400 MHz TBMPLL setting For each of the 48 ● channels of the FED, this calibration scans over all possible 400MHz and The optimal setting 160MHz TBM delay point is chosen as setting phase space. the most surrounded point For each setting point in ● by efficient phase the 2D plot, a score ( 200 is blocks. the “perfect” score ) is calculated based on data stream structure. 160 MHz TBMPLL setting Weinan Si 9 Commissioning and operation of the new CMS Phase 1 pixel detector

Detector Commissioning Important phases of commissioning: 1. Test under high trigger rates of random triggers 2. Cosmic data taking for gross time alignment of pixel system 3. Data taking with pp collisions for fine time and final alignment FEDTester : Full emulator of pixel module optical output Workbench setup GLIB based μ TCA board connected to CMS trigger system ● Test and validate FED fw before deployment ● Test and validate FED throughput to CMS cDAQ ● Link to CMS cDAQ in a test crate 10% trigger Send 100kHz random triggers (L1 rate in ● throttled at PU=130 CMS) Load FED with emulated hits ● Readout through 10Gbps link of FEROL No trigger ● throttled at PU=70 Trigger rates get throttled according to the ● (expected in 2017) FED status Weinan Si 10 Commissioning and operation of the new CMS Phase 1 pixel detector

Detector Commissioning Important phases of commissioning: 1. Test under high trigger rates of random triggers 2. Cosmic data taking for gross time alignment of pixel system 3. Data taking with pp collisions for fine time and final alignment Pixel DAQ crate at USC Workbench setup FED internal emulator ● 3 emulated hits/ROC ● (Pile Up ~ 105) in all FEDs (108) Experience from pilot blade ● system with μ TCA readout Link to CMS cDAQ in a test crate (commissioned in CMS in 2016) helps a lot! NO problems ➔ Not blocking cDAQ ◆ Weinan Si 11 Commissioning and operation of the new CMS Phase 1 pixel detector

Detector Commissioning Important phases of commissioning: 1. Test under high trigger rates of random triggers 2. Cosmic data taking for gross time alignment of pixel system 3. Data taking with pp collisions for fine time and final alignment FPix join global cosmic run | Apr.07, 2017 ● Saw hits, but timing setting not optimal ○ Timing setting scans ○ Masking noisy pixels ○ Cooling system redundancy test ● BPix join global | Apr.17, 2017 ● Magnet B=3.8T | Apr.19, 2017 ● Private Resync → reducing dead time from pixels ● Pause triggers ○ Pause m 1 orbits ○ Send Pixel only Resync command ○ Pause m 2 orbits ○ Re-enable triggers ○ Beam commissioning | May23, 2017 ● Figure : CMS DAQ status page Weinan Si 12 Commissioning and operation of the new CMS Phase 1 pixel detector

Detector Commissioning Important phases of commissioning: 1. Test under high trigger rates of random triggers 2. Cosmic data taking for gross time alignment of pixel system 3. Data taking with pp collisions for fine time and final alignment First data with the new detector: Figure : Occupancy map of BPix - Time alignment of detector layer1 during cosmic run - Local calibrations to find optimal TBMPLL settings - Very small part of the detector inactive - “Resets” evolve as we learn about preformance and needs Observation accumulated, experience gained ➔ Finer timing scans with first stable beams, optimal timing setting from cosmic runs as starting point. FPix disk -3 Figure : Cluster position map of FPix during cosmic run Weinan Si 13 Commissioning and operation of the new CMS Phase 1 pixel detector