B05 4 Global Correlator 402.06.05 R. Cavanaugh U. Illinois Chicago - - PowerPoint PPT Presentation
B05 4 Global Correlator 402.06.05 R. Cavanaugh U. Illinois Chicago and Fermilab, L3 Manager, HL-LHC Global Correlator 402.06.05 Directors Review of US-CMS HL-LHC Upgrades 2-3 February 2016 R. Cavanaugh, 2015 February 2-3 Director's
1
Director's Review – L1 Trigger Overview
2
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
3
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
402.06 Trigger Jeff Berryhill (FNAL) 402.06.03 Calorimeter Trigger Wesley Smith (UW) 402.06.04 Muon Trigger Darin Acosta (UF) 402.06.05 Global Correlator Rick Cavanaugh (UIC/FNAL)
§ R.C. (U. Illinois Chicago and Fermilab) § CMS Particle Flow & Tau ID Convener 2007-2008 § Fermilab LHC Physics Center Coordinator 2010-2013 § Phase-1 Stage-1 Global Calorimeter Trigger § Wesley Smith (U. Wisconsin) – US CMS HL-LHC L3 Calorimeter Trigger Project Manager § CMS Trigger Project Manager 1994-2007, § Trigger Coordinator 2007 – 2012 § Trigger Performance and Strategy Working Group 2012 - 2015 § US CMS L2 Trigger Project Manager (construction and operations) 1998 – present § US CMS Phase 1 Upgrade L2 Trigger Project Manager 2013 – present § Sridhara Dasu (U. Wisconsin) § US CMS L3 Manager Calorimeter Trigger (construction & operations) 1998 – present § US CMS L3 Manager Phase 1 Calorimeter Trigger Upgrade 2013 – present § Author of original and upgrade cal. trig. Algorithms 1994 – present § Darin Acosta (U. Florida) § CMS Trigger (co)Project Manager, 2012-16 § EMU Track-Finder, 1998- present § Ivan Furic (U.Florida) § CSC and EMU Track-Finder projects, 2008-present § Alexi Safonov (TAMU) § CSC Trigger Motherboards § Jeff Berryhill (Fermilab) § CMS & US CMS Phase-1 Stage-1 Calorimeter Trigger Project Manager
4
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 3
§ Tom Gorski (U. Wisconsin) – Electrical Engineer – Lead Engineer § Over a decade of engineering on the CMS Calorimeter Trigger § Delivered final phase of original Regional CMS Calorimeter Trigger § Delivered Phase 1 Layer-1 Calorimeter Trigger Upgrade Electronics § Ales Svetek (U. Wisconsin) – Firmware Engineer § 3 years on Phase 1 Calorimeter Trigger Upgrade Firmware § (4 years ATLAS Beam Conditions Monitor Firmware, DAQ, Commissioning, Detector Operations) § Marcelo Vicente (U. Wisconsin) – Firmware Engineer § 3 years on Phase 1 Calorimeter Trigger Upgrade Firmware + HCAL Firmware § 2 Years on ECAL Phase 1 Upgrade Trigger Primitive Generation Electronics (oSLB, oRM) § Jes Tikalski (U. Wisconsin) – Software Engineer § 3 years on Phase 1 Calorimeter Trigger Upgrade Software and embedded systems § Alex Madorsky (U. Florida) – Electrical Engineer § Over a decade of engineering on CMS Trigger, EMU, Track-Finder (since 1999) § Eric Hazen (Boston U.) – Electrical Engineer § Long history of engineering on CMS hadron calorimeter electronics and trigger § Shouxiang Wu (Boston U.) – Electrical Engineer § Long history of engineering on CMS hadron calorimeter electronics and trigger
5
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 3
6
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
7
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
Sorting/Merging Layer Muon Track-Finder MPC CSC DT LB RPC Global Correlations (Matching, PT, Isolation, vertexing, etc.) Splitters fan-out fan-out ECAL EB HCAL HB HCAL HF
single xtal
Regional Calo Trigger Layer Global Calo Trigger Layer
Calorimeter Trigger Muon Trigger
Tracker Track-Finding
Track Trigger
GEM + iRPC Global Trigger Tracker Stubs HGCAL
HGCAL
U.S. Covers fraction DR Question 1
8
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
From CMS Technical Proposal: CERN-LHCC-2015-10 DR Question 1
9
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
From CMS Technical Proposal: CERN-LHCC-2015-10 DR Question 1
for 20 GeV leading photon
10
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
From CMS Technical Proposal: CERN-LHCC-2015-10 DR Question 1
– match to tracks – apply track-based isolation
– match to EM-cluster
11
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
(GeV)
vis T
E
MC
τ
50 100 150 200
Efficiency
0.2 0.4 0.6 0.8 1 1.2
+
τ → VBF H CaloTaus TkCaloTaus (CaloTaus and Tracks) TkEmTaus (EM and Tracks)
CMS PhaseII Simulation PU=140, 14 TeV
From CMS Technical Proposal: CERN-LHCC-2015-10 DR Question 1
[GeV/c]
T
jet p 50 100 150 200 250 300 Efficiency 0.75 0.8 0.85 0.9 0.95 1
CMS Phase-2 Simulation Efficiency for |z(jet) - z(true)| < 1cm All-hadronic ttbar, <PU>=140 > 20 GeV
T
| < 2.0, p η Jet |
12
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
From CMS Technical Proposal: CERN-LHCC-2015-10 DR Question 1
13
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
From CMS Technical Proposal: CERN-LHCC-2015-10 DR Question 1
e's μ's τ's γ's jets Σ's development e's μ's τ's γ's jets Σ's development e's μ's τ's γ's jets Σ's development e's μ's τ's γ's jets Σ's development
14
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
Global Processing Regional Processing Crate A (“barrel-”) Crate B (“barrel+”)
Crate D (“endcap+”) L1 Calo L1 Muons L1 Tracks L1 Calo L1 Muons L1 Tracks L1 Calo L1 Muons L1 Tracks
Based on successful Phase 1 Architecture for CMS Calorimeter Trigger
Global Trigger
Base processors on existing CMS Virtex7 trigger processor boards
Calculate global sums (MET, SET, etc), sort
Inspired by particle flow (PF already used at HLT in Run-I)
L1 Objects DR Question 1
15
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
preprocess+distribute L1 calo, L1 muons, L1 tracks
Match L1 tracks to L1 Calo-clusters Match L1 tracks to L1 muons Use L1 tracks to find primary vertex
Calculate L1 track corrected
Match L1 tracks to primary vertex Calculate L1 track isolation of objects
0.5 μs 1.0 μs 1.5 μs 2.0 μs 2.5 μs DR Question 1
§ WBS includes all Engineering and Technical activities as well as M&S to produce the Correlator L1 Trigger electronics. § The system takes as its input data on optical fibers from the CMS Track, Calorimeter, and Muon Triggers and provides processed trigger data for the CMS Global Trigger. § WBS includes managing production of the boards, engineering in support of the production of the boards, procurement of the optical components, FPGAs and all other components on the Correlator L1 Trigger electronics. § WBS also includes fabrication of the PCBs and assembly of the finished Correlator L1 Trigger electronics. § Costs associated with production of boards are assumed to be consistent with the costs experienced with the Phase 1 Trigger Upgrade with appropriate economies of scale applied when quotes justify them.
§ Less expensive optical parts and FPGAs lack bandwidth to process trigger data in planned number of cards and crates, resulting in a multiplication of the system in size by factors of 2-4, which costs significantly more in both M&S and labor.
§ Technology extrapolations may potentially reduce costs but not included.
16
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
§ 402.06.05.01 Correlator Trigger Management
§ 402.06.05.01.01 Correlator Trigger Milestones, Interfaces § 402.06.05.01.02 Correlator Trigger Travel
§ 402.06.05.02 Regional Correlator Trigger
§ 402.06.05.02.01 Regional Correlator Trigger M&S (Detail Next Slide) § 402.06.05.02.02 Regional Correlator Trigger Engineering § 402.06.05.02.03 Regional Correlator Trigger Technical Work § 402.06.05.02.04 Regional Correlator Trigger FW § 402.06.05.02.05 Regional Correlator Trigger SW
§ 402.06.05.03 Global Correlator Trigger
§ 402.06.05.03.01 Global Correlator Trigger M&S (Detail Next Slide) § 402.06.05.03.02 Global Correlator Trigger Engineering § 402.06.05.03.03 Global Correlator Trigger Technical Work § 402.06.05.03.04 Global Correlator Trigger FW § 402.06.05.03.05 Global Correlator Trigger SW
§ 402.06.05.04 Correlator Trigger Infrastructure
§ 402.06.05.04.01 Crates and Power Supplies M&S § 402.06.05.04.02 Cables, Fibers and Patch Panel M&S § 402.06.05.04.03 Test Facilities M&S § 402.06.05.04.04 Infrastructure Engineering § 402.06.05.04.05 Infrastructure Technical Work
17
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
18
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
19
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
§ Assume ~350 tracks sent from track trigger at 100 bits per track
§ 35 kb per BX
§ Assume similar amount of information from calorimeter and muon triggers
§ 35 kb + 35 kb = 70kb per BX
§ Total input to Correlator is 105 kb per BX
§ Total Bandwidth at 40 MHz: 4.2 Tbs
§ Assume present day boards with 80x10 Gbps links running 192 bits at 40 MHz with 80% packing efficiency
§ CTP7 ($15.4k) can handle 492 Gbs
§ Number of boards per task (physics object reco/ID)
§ Regional: total number of boards per task: 4.2 Tbs / 0.492 Tbs ≈ 9 § Global: total number of boards per task: 3 (one third of regional-layer)
§ Number of boards:
§ 7 tasks = 6 physics objects {e, γ, μ, τ, jets, sums} + 1 dev (test new algos in situ) § (7x9=63 system, 2 test stand, 10 spare) 75 regional boards at $15.4k each, § (7x3=21 system, 1 test stand, 3 spare) 25 global boards at $15.4k each.
§ Infrastructure
§ Power Supplies, Fibers, Patch Panels summed to $17k/crate of 12 cards
§ Total Cost: $1.8M (US pays for $1M)
20
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
§ 2 prototype cards at $20k apiece in FY17,18,19 each § $13K ($15K) in FY17 (18,19) for crates and other testing infrastructure.
§ (63 system, 2 test stand, 10 spare) 75 regional boards at $15.4k each, § (21 system, 1 test stand, 3 spare) 25 global boards at $15.4k each. § Design one board for both, different FW
§ Based on costs of the Phase 1 cards § FPGA costs (Virtex plus ZYNQ) of $6.6k. § Optical Components $3.6K, § Miscellaneous Components $1.4k, § Printed Circuit Board $2k § Assembly $1.8k.
§ Based on costs of the Phase 1 Infrastructure § Power Supplies, Fibers, Patch Panels summed to $17k/crate of 12 cards
21
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
NB: Assumed Phase 1 μTCA Form Factor smaller than other FF’s considered (e.g. ATCA) means costs are higher since overall card infrastructure is amortized
conservative cost model DR Question 2
§ This includes software to interface with the Trigger Online System
§ Including implementing the trigger algorithms, diagnostics, data acquisition and readout.
22
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
23
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
24
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
CD4 CD1 CD2CD3 CD0
Specification and Technology R&D Trigger TDR
Pre- production
Installation
LS 2 LS 3 Physics Physics
LHC Schedule
CDR PDR CD3A FDR
Prototyping and Demonstrators Production Readiness Review Production and Test Test & Commission
Reality Check:
1st Proto: Q4/CY13 6 Pre. Prod. Proto: Q2-3/CY14 1st Production Board: Q1/CY15 50 Board Production Complete Q2CY15 Installation Complete: Q2/CY15 Commissioned, Operations start: Q3/CY15 ⇒ 21 Months: 1st prototype to pp operations
25
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
Cost = AY $k (No Contingency)
*Includes travel NB: escalation at 3%/year starting in FY17 DR Question 2
26
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
27
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
0.2 0.4 0.6 0.8 1 1.2
FY16 FY17 FY18 FY19 FY20 FY21 FY22 FY23 FY24 FY25
DR Question 2
§ Based on conceptual design documented in CMS HL-LHC Technical Proposal § Items based on existing Phase 1 Trigger upgrade with documented costs
§ Based on LOE required as determined from Phase 1 Trigger Upgrade
§ Based on expert judgment using documented experience of similar work required for the Phase 1 Trigger Upgrade § Development of activities defined at a conceptual level informed by the experience of the Phase 1 Trigger Upgrade § Technical requirements are moderately challenging, but straightforward extrapolation from the Phase 1 Trigger Upgrade
§ Using recent data for labor, M&S in the cost, contingency, risk estimates
28
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
29
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
§ Hire new engineer, subcontract to consulting firm, use FNAL engineer
§ Hire extra expert effort to recover schedule and help personnel
§ Hire extra effort to speed up testing schedule
§ Acquire spending authority to use alternative vendors (while original funds are being unencumbered).
§ Built in capabilities of trigger electronics provide signals for their own inputs &
30
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 2
31
02-Feb-2016 R. Cavanaugh B01-4: HL-LHC Global Correlator
§ L3 Manager (R.C.) responsible for applying ISM to trigger upgrade.
§ Modules similar to others built before, of small size and no high voltage
§ Regularly evaluate achievement relative to performance requirements and appropriately validate or update performance requirements and expectations to ensure quality. § QA: Equipment inspections and verifications; Software code inspections, verifications, and validations; Design reviews; Baseline change reviews; Work planning; and Self-assessments. § All modules have hardware identifiers which are tracked in a database logging QA data through all phases of construction, installation, operation and repair.
§ Apply appropriate level of analysis, controls, and documentation commensurate with the potential to have an environmental, safety, health, radiological, or quality impact. § Four ESH&Q Risk levels are defined and documented in CMS-doc-11584.
32
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 4
§ After full testing at institute, shipped to CERN
§ All tests recorded (of all types) for individual boards in database § Tests use and validate software and firmware test release
§ Acceptance Testing in Electronics Integration Center (EIC) at CERN
§ Individual labs for Muon, Calorimeter, (and now) Correlator Triggers § Boards retested to validate institute test results § Tests use software and firmware test release
§ Integration Testing in EIC
§ Row of racks with DAQ, Trigger, Central Clock, Crates of other subsystem electronics § Operation of a vertical slice with electronics to be tested installed. § Tests use and validate software and firmware commissioning release
§ Integration Testing at P5: Global Runs
§ Test with all CMS with cosmics when beam not running/with beam when running § Electronics installed in final locations with final cables § Full-scale tests with full CMS DAQ/Trigger/Clocking § Tests use software and firmware commissioning release
§ Handover to Operations at P5: Global Runs/Parallel Operation
§ After testing completes, continue with Global Runs/Parallel Operation § Validate software and firmware initial operational release
33
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 3
34
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 1
feasibility of trigger design, leads to:
needed for →
35
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 1
§ Full-function correlator prototypes produced which allow local comparison with emulator (TP.L1.6) – 4QCY18
bandwidth optical links to meet the requirements of the final boards
however, and only local comparisons will be possible between hardware behavior and the emulator.
§ Demonstrator trigger system shows correlator integration and scaling, global/full-chain comparison with emulator (TP.L1.7) – 4QCY19
multiple full-capability prototype boards and the prototype full-capability infrastructure
and testing environment capable of being connected to its front end detector for test-beam validation to follow.
§ HL-LHC Trigger TDR (TP.L1.8) – 1Q2020
36
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 1
§ Supporting HL-LHC Tracking Trigger and Calorimeter Trigger R&D § Comparatively “young” platform (< 2 years old) w/ new technology
§ Two CTP7 boards for regional correlation layer § One CTP7 board for global correlation layer § One crate plus fibers and patch panels
§ Establish baseline trigger algorithms between hardware and emulator § Explore algorithms beyond baseline (inspired by particle flow)
37
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
DR Question 1
38
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger
39
02-Feb-2016 R. Cavanaugh HL-LHC Correlator Trigger