P0 P01:Bar arrel el C Cal alorimeter er Ov Overvie view - - PowerPoint PPT Presentation
P0 P01:Bar arrel el C Cal alorimeter er Ov Overvie view Colin Jessop, University of Notre Dame USCMS L2 Barrel Calorimeter Manager Technical Design Review Aug. 28-29, 2017 Colin Jessop Barrel Calorimeter
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 1
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 2
§ Biographical sketch § Upgrade Scope and Motivation § Requirements
§ Science § Engineering
§ Design
§ U.S. Deliverables
§ WBS Structure
§ L3 Managers § Institutions
§ Interfaces and Partners § Development Plan
§ Milestones § R&D Budget § Design Maturity
§ Risks § Scope Options § Overview of Construction Schedule and Cost § Quality Assurance and ES&H § Summary
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 3
Education: B.A/M.A Cambridge University, UK Ph.D. Harvard University (CDF ECAL/HCAL Top &Higgs Analysis) Training: Post-Doc @ SLAC: R&D on BaBar ECAL readout also CP Violation analysis Positions: Panofsky Fellow @SLAC Managed BaBar ECAL calibration at startup of BaBar experiment Managed BaBar EM radiative decay analysis program (bà sg) Professor: University of Notre Dame LPC electron/photon group leader (2006-2007) US CMS ECAL Institute Board Chair (2008-2012) US CMS L2 project manager for ECAL operations (2012-present) CMS ECAL upgrade manager (2012-present) US CMS L2 phase 2 ECAL/HCAL Barrel upgrade manager (2015-) Higgs analysis (Hàgg, Hàtt), leading group to search for lepton flavor violating decay of Higgs in run 2
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 4
§ Point out where your system is in the CMS detector drawing
~4 Barrel EM Calorimeter: Front and back end electronics. Project Cost: $11.6M Barrel Hadron Calorimeter: Back end
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 5
To use modern high speed digital electronics to enhance the performance and discovery capabilities of the barrel electromagnetic/hadronic calorimeter, particularly Higgs Boson physics.
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 6
muons (isolation).
A high mass dijet event EM Energy Hadronic Energy
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 7
Parameter Barrel h coverage |h| < 1.5 # crystals 61200 Granularity 36 supermodules
A Hàgg candidate event observed in the ECAL
Barrel Endcap
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 8
𝐼 → 𝑎 𝑓𝑓 𝑎(𝑓𝑓)
Hàgg
decay modes with photons and electrons
determine the presence of new physics indirectly
Higgs boson or new Higgs decays with e/g in final state.
e.g H(bb)H(gg) helps determines vacuum potential for the Universe
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 9
Physics Drivers HL-LHC technical Proposal CMS DocDB 12143 NSF project execution plan CMS DocDB 13279 Science Goals and Requirements CMS DocDB 13317 Science-Engineering Requirements CMS DocDB 13337 Engineering Requirements CMS DocDB 13337 Precision Measurement of H->gg Example Trigger at low enough ET thresholds for g from H->gg (sci-req-002) Trigger Level1 accept rate of 750 KHz and 12.5µS (BCAL-sci-engr-005) Rates in on to off detector
(BCAL-engr-018)
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 10
Physics Drivers HL-LHC technical Proposal CMS DocDB NSF project execution plan CMS DocDB Science Goals and Requirements CMS DocDB 13317 Science-Engineering Requirements CMS DocDB 13337 Engineering Requirements CMS DocDB 13337 Precision Measurement of H->gg Example Trigger at low enough ET thresholds for g from H->gg (sci-req-002) Trigger Level1 accept rate of 750 KHz and 12.5µS (BCAL-sci-engr-005) Rates in on to off detector
(BCAL-engr-018)
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 11
Physics Drivers HL-LHC technical Proposal CMS DocDB NSF project execution plan CMS DocDB Science Goals and Requirements CMS DocDB 13317 Science-Engineering Requirements CMS DocDB 13337 Engineering Requirements CMS DocDB 13337 Precision Measurement of H->gg Example Trigger at low enough ET thresholds for g from H->gg (sci-req-002) Trigger Level1 accept rate of 750 KHz and 12.5µS (BCAL-sci-engr-005) Rates in on to off detector
(BCAL-engr-018)
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 12
The physics drivers lead to a set of 15 science requirements documented in CMS DOC-DB-13337 Those directly relevant to ECAL and HCAL in table below
Science Requirements ID descrption. Redundancy Robustness sci-req-1 Detector remains efficient to 3000 fb-1 Electroweak Scale Trigger Thresholds sci-req-2 Efficient triggering of Higgs decays Dijet Resolution sci-req-5 Excellent dijet resolution (H to bb). Diphoton Resolution sci-req-6 H to γγ resolution. Pileup Mitigation sci-req-10 Mitigation pileup so that LHC performance is recovered at HL-LHC. Missing Transverse Energy Resolution sci-req-11 BSM particles Lepton Identification Efficiency and purity sci-req-12 H to ZZ Photon Identification Efficiency and purity sci-req-13 H to γγ , H(γγ)
Example: Precision H to γγ requires large statistics (sci-req-1,sci-req-2), excellent photon resolution (sci-req-6) and identification efficiency (sci-req-13)
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 13
at the HL-LHC as for the legacy LHC (10 x integrated and instantaneous luminosity)
(sci-req-2,5,6,10,11,12,13)
granularity in the calorimeter trigger to be able to trigger efficiently on Higgs bosons
(sci-req-2)
(The calorimeter trigger must be able to accommodate the latency required to trigger
in the ECAL. Recovers resolution performance for Higgs bosons. (sci-req-10)
cooling the ECAL detector and optimizing the electronics noise filtering in the readout Recovers resolution performance for Higgs bosons. (sci-req-2,5,6,10,11,12,13)
by improved timing and finer granularity. Maintains Trigger performance for Higgs Bosons (sci-req-2) Leads to a set of high level engineering requirements documented in CMSDocDB 13317
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 14
ECRequirements Description
Radiation hardness BCAL-sci-engr-001 Rad hard to survive 3000 fb-1 of HL-LHC running. On detector components are exposed to radiation and must be designed to be radiation tolerant sci-req-5, 6, 10,11 ECAL Replacement BCAL-sci-engr-002 HL-LHC ECAL upgrade shall replace all ECAL channels The scintillating crystals and mechanical infrastruture are not being replaced so the channel count and physical footprint of the boards is constrained by the existing system sci-req-6, sci- req- HCAL Replacement BCAL-sci-engr-003 HL-LHC HCAL upgrade shall replace all HCAL channels on the existing detector. The scintillating towers and on-detector readout are not being replaced so the channel count is constrained by the existing system sci-req-5, sci- req-11 ECAL/HCAL Trigger granularity BCAL-sci-engr-004 Single crystal/tower information to L1 Single crystal and tower information is required to match to tracks sci-req-2 ECAL/HCAL Trigger bandwidth and latency BCAL-sci-engr-005 ECAL/HCAL system shall trigger at 750 kHz with 12.5 us latency. Latency requirement to accommodate the processing needed for the track trigger. Rate to allow trigger thresholds low enough to trigger on objects from physics processes sci-req-2 ECAL Timing resolution BCAL-sci-engr-006 ECAL/HCAL system shall achieve 30 ps timing for 50 GeV photons. Required to mitigate the effects of pileup sci-req-6, sci- req-10 Intersystem
BCAL-sci-engr-007 Backend board shall be capable of being used in both ECAL and HCAL with only firmware modifications. Required for long term maintainability of the system. Reliability BCAL-sci-engr-008 FE must operate for 3000fb w/o intervention The on detector electronics will be inaccessible for the duration of the HL-LHC sci-req-5, sci- req-6, sci-req- 10, sci-req-11
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 15
Max Level 1 Accept Rate Max Latency Trigger Phase 2 750 kHz 12.5 µS ECAL Phase 1 150 kHz 6.4 µS
trigger requirements
to trigger instead of present
also better spike rejection
increase due to radiation damage. Also change shaping time of pre-amp for more optimal noise filter and 30ps timing
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 16
APD Crystal HCAL (2)
Reminder: Hadrons interacting with the APD’s causing anomalous high E deposits
Hadrons come from primary interaction and backsplash. Currently 15% of L1 electron/photon
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 17
Timing requirement affects off detector bandwidth and clock precision requirements
30ps timing resolution @ 50 GeV is possible.
Impact on H->gg.
high pileup enhances mass resolution
cross-section measurement
charged track Timing layer
⇒
Adi Bornhei
Leads to requirement of 30ps timing resolution at 50 GeV for photons
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 18
Motherboard L V R V F E V F E V F E V F E V F E FE HV S Trigger Data Crystals with two Avalanche Photodiodes ( Connected so I channel per 2 APD’s) Motherboard is passive – basically just a connector HV and LV to APD and signals to VFE VFE = Very Front End. Preamp, Shaping and ADC FE = Front End. Forms sum of 5 crystals in a strip For trigger primitive and has latency buffer to store data while waiting for L1 accept 800 MB/s
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 19
Motherboard L V R V F E V F E V F E V F E V F E FE HV S Trigger Data Crystals with two Avalanche Photodiodes ( Connected so I channel per 2 APD’s) Motherboard is passive – basically just a connector HV and LV to APD and signals to VFE VFE = Very Front End. Preamp, Shaping and ADC FE = Front End passes single crystal information to the back end. No latency buffer in front end 10000 Mb/s
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 20
Feature extraction Trigger primitive formation Suppress isolated anomalous deposits Motherboard L V R V F E V F E V F E V F E V F E FE HV S 10 Gb/s Data Clock Control L1 Trigger DAQ Trigger primitive, feature extraction done
U.S delivers complete block of readout with No dependencies on other countries
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 21
Motherboard L V R V F E V F E V F E V F E V F E FE HV S 10 GB/s Data Clock Control L1 Trigger DAQ Trigger Concentrator (402.03.03.04) Data concentrator (402.03.03.03) Clock and Control (402.03.03.05) Front end (FE) and Optical links (402.03.03.02)
Single Board called the Barrel Calorimeter Processor (BCP)
System Clock and Detector Control system
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 22
34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72
2448 FE Boards (1 per trigger tower) 13840 Optical links (5 per FE board, 4 up @ 10Gb/s + 1 down @2.4 Gb/s) + η
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 23
GBT4 G8 GBT- SCA VTT x VT Rx
Technical details and development plan in talks by Sasha Singovski
back-end electronics and receives Clock and control signals
tolerant
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 24
34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72
+ η
108 Barrel Calorimeter Processor (BCP) Boards (6 per wedge with18 wedges) Technical Details of ECAL system architecture (Trigger/DAQ/control) in talk from Nikitas Loukas
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 25
▪ ▪ ▪ ▪
FPGA’s
Board development BCP Technical details in talk from Stephen Goadhouse. Clock distribution Issues in talk from Erich Frahm
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 26
NEW or Existing VFE card NEW FE card
AD41240
MGPA
AD41240
MGPA
AD41240
MGPA
AD41240
MGPA
AD41240
MGPA FPGA
VFE- FE adapter card
GBTx GBTx GBTx GBTx GBTx
GBT GBT GBT GBT GBT
GLIB
Versatile link
Multimode
cable
Card to adapt devlopment FE board to legacy
Demonstrator FE card (now fabricated with GBTx) CTP7 is a CMS trigger board that serves as first demonstrator for the aTCA board we will develop (delivered August 2016 and now installing firmware)
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 27
Entire readout chain from FE board onwards and firmware to operate it Item ATCA Quantity Required (including spares) Front End (FE) Board 2768 Optical Links 13840 Barrel Calorimeter Processor Boards 118 ATCA crates 15 Fully qualified boards and links must be delivered Q4 2023 (end of project) Installation is the responsibility of CERN technical co-ordination (so we are not Dependent on LHC schedule) Maintenance and operation is responsibility of USCMS operations program
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 28
17 Layers of Brass/Scintillator sampling calorimeter in 18 wedges Scintillator is arranged in ~10x10cm megatiles With Wavelength shifter readout (WLS) No replacement of scintillator/WLS or front-end readout
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 29
GBT FPGA FE− VTTX
Calorimeter Trigger uHTR AMC13 cDAQ
QIE10
SiPMs FEE Card (12 channels) FE Module (48 channels) Detector
Replace back-end electronics with same boards as developed for ECAL barrel but Different firmware
Max Level 1 Accept Rate Max Latency Trigger Hl-LHC 750 kHz 12.5 µS HCAL LHC 200 kHz 12.5 µS
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 30
16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 1 2 3 4
Technical Details in talk by Alberto Belloni An HCAL Barrel Wedge
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 31
Off detector electronics and firmware Item Quantity Required Barrel Calorimeter Processor Boards 22 ATCA crates 4 Fully qualified boards and links must be delivered Q4 2023 (end of project) Installation is the responsibility of CERN technical co-ordination (so we are not Dependent on LHC schedule) Maintenance and operation is responsibility of USCMS operations program
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 32
A full resource loaded schedule with milestones has been developed in Primavera 6 with accompanying BOE’s following the guidance of FNAL project specialists R&D 2017-2019 Pre-production 2020 Production 2021-2023 End of Project Q4 2023 Details with milestones in talk from Bob Hirosky
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 33
33 NSF CDR -- Barrel Calorimeter/Forward Muon Overview Colin Jessop, March 15 2016
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 34
34
L2 Manager Colin Jessop (Notre Dame) L3 Managers Bob Hirosky (Virginia) Alberto Belloni (Maryland) Systems Engineer Tom Gorski (Wisconsin)
402.03.03.02 FE Board/Optical Links Sasha Singovski (engineer) Sasha Dolgpolov(engineer) (US ECAL)
402.03.03.03 Data Concentrator Stephen GoadHouse (Engineer) Virginia 402.03.03.04 Trigger Concentrator Nikitas Loukas (Engineer) Noyre Dame 402.03.03.05 Clock and Control System Eric Fra
402.03.02.04 Tulio Grassi (engineer) Maryland
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 35
North Eastern Minnesota Notre Dame Maryland Virginia Wisconsin Institution Responsibilities Maryland HCAL BCP Minnesota BCP clock & control North Eastern FE, Software Notre Dame FE, BCP trigger, HCAL Virginia BCP DAQ, HCAL Wisconsin BCP board infrastructure
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 36
36
Motherboard L V R V F E V F E V F E V F E V F E FE HV S 10 GB/s Data Clock Control L1 Trigger Trigger Concentrator (402.03.03.04) Data concentrator (402.03.03.03) Clock and Control (402.03.03.05) Front end (FE) and Optical links (402.03.03.02) DAQ System Clock and Detector control
ICD in CMSDocDB 13337
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 37
37
5 GB/s Data Clock Control L1 Trigger Trigger Concentrator (402.03.03.04) Data concentrator (402.03.03.03) Clock and Control (402.03.03.05) DAQ System Clock and Detector control
ICD in CMSDocDB 13337 HCAL Front-End Indian groups will likely help with commissioning and testing as in phase 1
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 38
Front end Board/Optical links: Specified and demonstrator tested. Planning for first prototype See Talks from Sasha Singovsksi Off detector System architecture specified See Talks from Nikitas Loukas (ECAL) Alberto Belloni (HCAL) System architecture allows preliminary specification of BCP and now moving towards beginning first demonstrator design See talks from Erich Frahm (Timing) and Stephen Goadhouse (BCP) Rough estimate of design maturity (#tasks complete/total) ~ 25%
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 39
We have been able to reduce costs by design optimization: 1.) We moved from 25 Gb/s to 16 Gb/s rate in the optical links in the off-detector electronics BCP to the DAQ and Trigger. This allows us to use cheaper optical link components and FPGA’s (see talk by Nikitas Loukas) 2) We optimized the choice of FPGA to use 2 cheaper FPGA’s per board rather than a more expensive single FPGA (i.e more expensive than two of the cheaper ones). (see talk by Nikitas Loukas) 3) We removed a custom ASIC called the FENIX chip from the FE board by arguing that its functionality could be absorbed in the VFE ASIC (in European scope) (see talk by Sasha Singovski)
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 40
Charge: Q4 Risks are documented in “Risk Registry” CMS-DocDB 12897 Risk: The lpGBT cannot meet the bandwidth requirement Probability: 10% Cost Impact $4M Mitigation: FE engineers work closely with CERN development team in lpGBT development Reaction: Use twice the number of presently available GBT links at half the rate Risk: The lpGBT cannot meet the timing jitter requirement. Probability: 50% Cost Impact $650K Mitigation: FE engineers work closely with CERN lpGBT development team. R&D in alternate clock distribution with additional fiber. Reaction: Add addition dedicated clock fiber Risk: A firmware release does not meet requirements Probability: 30% Cost Impact 15 to 45K Mitigation: Long development cycle in projects with multiple firmware releases Reaction: Additional 2-3 month development requiring extra engineering
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 41
The scoping options are documented in CMS DocDB 13357
Descope: Remove integration task so just deliver burnt-in boards Technical Impact: Difficulty commissioning system causing delays and increased operations costs Cost Impact: $628K Descope: Use student labor instead of technicians for board quality assurance Technical Impact: Potential for lower quality boards delivered causing schedule delays increasing operations cost. Additional engineering supervision required Cost Impact: $250K Desccope: Use less power FPGA’s in back end board. Technical Impact: Less information provided to trigger with potential to increase thresholds and waste luminosity for physics Cost Impact: $641K Descope: Partial instrumentation of prototype wedge Technical Impact: Potential to miss integration and synchronization problems causing schedule delays Cost Impact: $121K
Total descope options $1.64M of total project cost $14.3M = 11.4%
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 42
The BCP can be used in the HO (9 boards), HF (9 boards) and Barrel Timing layer (12) (Cost $21K per board HO ($189K) HF ($189K) BTL ($252K) Total $630K The HF/HO legacy boards could work in principle but they will need adapter boards developed to interface with DAQ/Trigger and they will be obsolete by 2026 – they are close to obsolete already (i.e Xilinx stops supporting the FPGA firmware)
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 43
WBS Descrip2on Construction $K AY Contingency $K AY Total $K AY 402.03.03.01 Management 280 85 365 402.03.03.02 FE/opMcal links 3498 1142 4640 402.03.03.03 Data concentrator 1502 393 1895 402.03.03.04 Trigger concentrator 1500 390 1890 402.03.03.05 Clock and control 1513 406 1920 402.03.03.08 Prototype 288 86 373 Total 8581 2502 11083
Costs are taken from P6 and are actual year (AY) including P6 escalation model
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 44
WBS Description Construction $K AY Contingency $K AY R&D $K AY Total $K AY 402.03.04.01 Management 120 12 89 221 402.03.04.04 BEE 1211 442 395 2048 402.03.04.05 Prototype 315 116 431 Total 1646 570 484 2700
Costs are taken from P6 and are actual year (AY) using P6 escalation model
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 45
§ Responsible institute validates HW with tagged FW+SW prior to final
shipping
§ Validation repeated upon receipt at Electronics Integration Center at
CERN
§ …and similarly for P5 install, local runs, global runs
Charge: Q7
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 46
§ L2 Manager (W.S.) responsible for applying ISM to BCAL upgrade
(Under direction of US CMS Project Management)
§ Modules similar to others built before, of small size and no high voltage § Integrated into existing well-tested and long-term performing safety
system
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 47
Colin Jessop Barrel Calorimeter Technical Design Review Aug. 28-29, 2017 p. 48