HCAL Trigger Readout HCAL Trigger Readout HCAL Trigger Readout HTR - - PowerPoint PPT Presentation

▶
hcal trigger readout hcal trigger readout hcal trigger
SMART_READER_LITE
LIVE PREVIEW

HCAL Trigger Readout HCAL Trigger Readout HCAL Trigger Readout HTR - - PowerPoint PPT Presentation

Dec 28, 2022 264 likes •494 views

HCAL Trigger Readout HCAL Trigger Readout HCAL Trigger Readout HTR Status and Clocking Issues D. Baden, T. Grassi http://www.physics.umd.edu/hep/esr_dec_2002.pdf 1 CMS ESR December 2002 FE/DAQ Electronics FE/DAQ Electronics FE/DAQ

slide-1
SLIDE 1

CMS ESR December 2002

1

HTR Status and Clocking Issues

  • D. Baden, T. Grassi

http://www.physics.umd.edu/hep/esr_dec_2002.pdf

HCAL Trigger Readout HCAL Trigger Readout HCAL Trigger Readout

slide-2
SLIDE 2

CMS ESR December 2002

2

Shield Wall S B S HPD FE MODULE

12 HTRs per Readout Crate, 2 DCC

FRONT-END RBX Readout Box

(On detector)

READ-OUT Crate

Trigger Primitives

Fibers at 1.6 Gb/s 3 QIE-channels per fiber

QIE QIE QIE QIE QIE QIE

CCA

GOL

D C C

TTC

GOL

CCA

H T R H T R CAL REGIONAL TRIGGER

32 bits @ 40 MHz 16 bits @ 80 MHz

CCA

S-Link: 64 bits @ 25 MHz

Rack CPU

FE/DAQ Electronics FE/DAQ Electronics FE/DAQ Electronics

C L K H T R

slide-3
SLIDE 3

CMS ESR December 2002

3

HTR Principal Functions HTR Principal Functions HTR Principal Functions

  • 1. Receive front-end data for physics running
  • Synchronize optical links
  • Data validation and linearization
  • Form TPG’s and transmit to Level 1 at 40 MHz
  • Pipeline data, wait for Level 1 accept
  • Upon receiving L1A:
  • Zero suppress, format, transmit to the concentrator (no filtering)
  • Handle DAQ synchronization issues (if any)
  • 2. Calibration processing and buffering of:
  • Radioactive source calibration data
  • Laser/LED calibration data
  • 3. Support a VME data spy monitoring
slide-4
SLIDE 4

CMS ESR December 2002

4

Readout VME Crate Readout VME Crate Readout VME Crate

“BIT3” board

– Slow monitoring over VME – Commercial VME/PCI Interface to CPU

FanOut board

– Takes TTC stream in – Clone and Fanout timing signals

HTR (HCAL Trigger and Readout) board

– Spy output over VME – FE-Fiber input – TPG output (SLBs) to CRT – DAQ/TP Data output to DCC

DCC (Data Concentrator Card) board

– Input from HTRs – Spy output – Output to DAQ

D C C

VME CRATE

20m Copper 1.2 Gb/s

DAQ Calorimeter Regional Trigger

B I T 3 Fiber 1.6 Gb/s F a n O u t H T R

Front End Electronics

H T R D C C H T R H T R ... TTC fiber

slide-5
SLIDE 5

CMS ESR December 2002

5

  • Board organized around 2

identical sets of circuitry:

  • Optical inputs
  • 1.6 GHz, 8B/10B frames, 3ch/link
  • Dual LC detectors and drivers
  • TI TLK2501 Deserializers
  • Crystal RefClk
  • TTC 80MHz backup
  • Xilinx Virtex FPGA XCV1000E
  • 24 channels each
  • TPG signals
  • Sent to SLB over backplane,

LVDS

  • SLBs mounted 6 to a transition

board

  • Level 1 accept output to DCC
  • LVDS output
  • VME
  • Altera FPGA and firmware

OLD DESIGN

“Old” HTR Design (Summer 2002) “ “Old” HTR Design Old” HTR Design (Summer 2002) (Summer 2002)

slide-6
SLIDE 6

CMS ESR December 2002

6

HTR Functional Experience HTR Functional Experience HTR Functional Experience

  • What was tested:
  • VME fully tested and working
  • Some changes necessary to conform to CMS VME standards
  • Optical links and synchronization
  • No indication of any problems. Big success here – was a real worry
  • LVDS to DCC
  • Tested, working (Will change cable/connector to Cat 6/RJ45)
  • Fanout of timing signals on two Cat5 cables
  • Plan to change to a single Cat6 or Cat7 cable (very low cross-talk)
  • Firmware – full tests of:
  • Pipeline and L1A triggering capability
  • In-line histogramming for source calibration
  • TTCrx
  • Not working at all (4 bad on 4 tested).
  • What was not tested: Anything to do with TPG
slide-7
SLIDE 7

CMS ESR December 2002

7

HTR Board Experience HTR Board Experience HTR Board Experience

  • Produced ~12 boards
  • Several bare boards were delivered warped
  • Many opens under FPGA after assembly (~9 boards)
  • Some fixed after reflow (a few)
  • Some worse after reflow (shorts)
  • X-rayed a few boards, sometimes inconclusive
  • Some opens on VME side
  • Non BGA FPGA, indicates bad vias
  • Few other various open circuits
  • Finally got ~8 boards to “work”
  • Questionable reliability
slide-8
SLIDE 8

CMS ESR December 2002

8

Modifications Modifications Modifications

  • Change board from using white-tin to gold traces
  • This process was sold to us by the board maker. Our mistake.
  • Used only for very high volume, cost competitive products, very

difficult and expensive to control.

  • Gold is flatter and not very much more expensive (~$50/board),

better for FPGAs

  • Change assembly house
  • Insufficient Quality Control on current assembler – they are fired.
  • We visited 2 high-end assemblers
  • Modern Machines
  • Step up and step down oven temp control.
  • In-line X-ray for BGA QC
  • Manufacturability Review
  • Add stiffeners to HTRS
  • Flexability of 9U VME boards was underestimated
  • Worry: fine-line BGA (FBGA) can pop connections
slide-9
SLIDE 9

CMS ESR December 2002

9

Modifications (cont) Modifications (cont) Modifications (cont)

  • Change from FBGA to BGA
  • FBGA 1.0mm pitch, change to BGA 1.27mm pitch
  • No additional expense, plenty of available real estate, no

need to push

  • We are just being very conservative here
  • JTAG capabilities added
  • Will help with debugging
  • By making these changes…
  • We have profited from the summer
  • We have reduced our production risk considerably
slide-10
SLIDE 10

CMS ESR December 2002

10

HTR Design Changes HTR Design Changes HTR Design Changes

  • SLB transition board issues:
  • Worries about so many LVDS signals over backplane for old design
  • Routing is too complicated
  • Many signals going to same backplane location
  • Requires multi-layer routing with many vias
  • TPG cables very thick
  • Mechanical issues are very worrisome
  • SLB changes needed (e.g. height reduced after ECAL redesign…)
  • Solution: move SLB’s to HTR motherboard
  • Benefits:
  • Mechanically attach SLB’s to HTR front panel for mechanical stability
  • Eases routing requirements, reduces board and assembly risks, cheaper too
  • Change from Xilinx VirtexE to Virtex2
  • More resources, block ram, hardware multipliers
  • Big cost reduction (save $180k)
  • More modern chip for long-term maintenance
  • Clock synchronization
  • Decouple “80MHz” crystal from FPGA system clock
  • Will allow us to use crystal to maintain synchronization of serdes
  • This gives us 2 solutions for our “40ps” jitter requirement issue
slide-11
SLIDE 11

CMS ESR December 2002

11

P1

to DCC

New HTR Conceptual Design New HTR Conceptual Design New HTR Conceptual Design

P2

LVDS

to Level 1 Cal Trigger

LVDS SLB SLB SLB SLB SLB SLB

FPGA Xilinx XC2V

LC LC

TI TI TI TI TI TI TI TI

LC LC

FPGA Xilinx XC2V

LC LC

TI TI TI TI TI TI TI TI

LC LC VME FPGA

Fibers

No P3!

8-way 8-way

from Fanout

RJ45 TTC

slide-12
SLIDE 12

CMS ESR December 2002

12

Clocking Schematic Clocking Schematic Clocking Schematic

TTCrx

TTC 80 MHz LVPECL Crystal

1 to 8 Fanout 1 to 8 Fanout

Single width VME

BC0 80MHz 40MHz

SLB SLB SLB SLB SLB SLB

TI (16)

F P G A

BC0 BC0 40MHz

1 to 8 Fanout

80MHz

TTC mezz

TTC TTC broadcast bus

Cat 6/7 quad cable (allows PECL) TTC Fanout Board QPLL

  • Start with Fanout card
  • TTCrx Maryland mezzanine card or CERN TTCrm daughterboard
  • QPLL
  • Fanout on Cat6/7 quad twisted pair TTC, BC0, 40MHz, 80MHz
  • In HTR:
  • Send TTC signal to TTCrx mezzanine board, access to all TTC signals
  • Send 80MHz clean clock (cleaned by QPLL) to mux
  • Select 80MHz clean clock OR crystal to TI deserializers

80 MHz 40 MHz

slide-13
SLIDE 13

CMS ESR December 2002

13

HCAL TRIDas Clock Scheme HCAL HCAL TRIDas TRIDas Clock Scheme Clock Scheme

TTCrx QPLL

(‘CC’ means Clean Clock)

Cat6/7 RJ45 RJ45 TTCMezz

TTC

SLB

Xilinx

TTC broadcast, L1A, BCR, EVR, CLK40

Fanout Card

4 twisted pair… TTC BC0 CC40 CC80

HTR Board

CC40 CC80 BC0

slide-14
SLIDE 14

CMS ESR December 2002

14

Fanout – HTR scheme Fanout Fanout – – HTR scheme HTR scheme

HTR

TTC fiber

TTC LVDS CLK80 3.3V-PECL RX_BC0 LVDS

Cat6E

  • r Cat7

cable

8 clks to TLKs

DS90LV001

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 MC100LVE310 3.3V PECL CLK40 3.3V-PECL

LVDS Fanout x 8

PCK953 LVPECL- to-LVTTL Fanout (top layer) PCK953 LVPECL- to-LVTTL Fanout (top layer) 8 clks to TLKs + TPs To 6 SLBs

  • Diff. to 2

Xilinx + termin.

  • Diff. to 6 SLBs

Single-end to 2 xilinx

TTC daughter card IN IN_b

Brdcst<7:2>, BrcstStr, L1A, BCntRes to xilinx and SLBs CLK80 LVDS

Fanout Board

Low-jitter Fanout x 15 O/E

Brdcst<7:2>, BrcstStr BC0

Fanout buffer TTC TTC TTC

FPGA

Fanout x 15 Brdcst<7:2>, BrcstStr, BCntRes, L1A

CMOS

LVDS

  • r diff

PECL …….. …….. …….. …….. 15 connectors

  • n bottom

layer ? 15 Cables & Connectors tbd …….. ……..

NB100LVEP221 is LVDS compatible TTCrx (or daughter card)

QPLL

AN1568/D Fig 11 Onsemi.com

RJ45 ~Fifteen RJ45 connectors

PECL fanout e.g. DS90LV110

.. ..

2 Test Points for CLK40 and BC0 CLK40 LVDS PECL fanout

.. .. .. ..

80.0789 MHz 3.3V crystal

  • Diff. PECL

MC100LVEL37 CK CK CK/2 CK/2

.. .. .. ..

9U Front-panel space = 325 mm ; => space per connector ~ 21.5 mm

Notes: SLBs require fanout of CLK40, BC0. FE-link possibly requires CLK80. PECL fanout was tested in TB2002. One Cat6E cable (low x-talk) replaces the 2 Cat5 cables used in TB2002. TTC and BC0 remain LVDS as in Weiming’s board. HTR needs Broadcast bus, BCntRes and L1A: from TTCrx if we get it to work,

  • therwise we have to fan them out.

LVDS Tullio Grassi <tullio@physics.umd.edu>

slide-15
SLIDE 15

CMS ESR December 2002

15

TTCrx Mezzanine card TTCrx TTCrx Mezzanine card Mezzanine card

  • Very simple card:
  • 2 PMC connectors
  • TTCrx chip
  • TTC signal driver
  • n motherboard
  • Will be sent out for

prototype next week

  • Used by HTR,

DCC, and Fanout cards

slide-16
SLIDE 16

CMS ESR December 2002

16

TTC Distribution – Fanout Card TTC Distribution TTC Distribution – – Fanout Fanout Card Card

  • Currently HCAL has 6 TTC partitions:
  • Each partition requires TTCvi and TTCex
  • Each HCAL VME crate will have a single TTCrx

receiving data directly from TTCex in a single VME card (Fanout Card)

  • Fanout TTC signal to HTR mezzanine card with

TTCrx chip

  • Use quad twisted pair CAT6/7 cable allows PECL fanout
  • TTC raw, BC0, 40MHz clean, 80MHz clean fanout
  • Cost savings and simplification
  • TTC monitoring by Fanout card over VME
  • Count resets, etc…
slide-17
SLIDE 17

CMS ESR December 2002

17

Random Latency Issue Random Latency Issue Random Latency Issue

  • Texas Instruments TLK2501 Serdes
  • Run with 80MHz frame clock – 20 bits/frame, 1.6GHz bit clock
  • 625ps bit time
  • Latency from data sheet:
  • ~20ns variation (overall latency between 47 and 67ns)
  • Fiber to fiber alignment could cross a 40MHz bucket boundary.
  • How to fix?
  • SLB “knows” this latency – we will read it out after each reset
  • HCAL LED fast rise time
  • Can pulse during abort gap and align channels
  • Requires LED pulsing alignment
  • FE will send BC0 signal on all fibers
  • Will measure this alleged latency with new HTR boards
slide-18
SLIDE 18

CMS ESR December 2002

18

TPG Alignment TPG Alignment TPG Alignment

  • TPG alignment performed in SLB
  • Necessary: All HTRs will send common BC0 to SLB’s

within each of 16 VME crates

  • Calibration procedure to be performed for crate-crate

alignment

  • Initial alignment with LEDs, laser, etc.
  • Final alignment with LHC first beam data
  • Use “1-bucket” running to check everything
  • This will ensure successful alignment
slide-19
SLIDE 19

CMS ESR December 2002

19

DAQ Alignment DAQ Alignment DAQ Alignment

  • DAQ data must also be aligned
  • Must know L1A bucket for zero suppression
  • Only if we will do ZSP on 1 or 2 HCAL channels centered
  • n L1A bucket
  • If ZSP done with sum over 5 channels, then this alignment is not

critical

  • Solution: discussed in previous slide
  • Read from SLB
  • FE sending known ID after with fixed offset relative to

BC0 during abort gap

  • Comparison of the two for error checking
  • DAQ check on BC0 in DCC for alignment
  • Will send BC0, BCN, and EVN with the data to DAQ
slide-20
SLIDE 20

CMS ESR December 2002

20

MISC Errors MISC Errors MISC Errors

  • What happens if DCC finds mismatch in EVN?
  • DCC will then issue resynch request to aTTS system
  • Details not yet defined but is fully programmable
  • Fiber Link/synchronization errors (GOL/TI)
  • Work out protocols to inform DCC
  • Reset requests to aTTS as well
  • FE Clock/GOL PLL link errors
  • If GOL loses synch, then transmitter will send out IDLE

characters

  • IDLE characters are illegal in a pipelined system!
  • HTR will trap on IDLE as a signal that FE/GOL is having

trouble

slide-21
SLIDE 21

CMS ESR December 2002

21

Schedule Schedule Schedule

O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D Firmware Board layout Fab/assembly

20 boards will be built but not assembled

Pre-production HTR board Checkout Board layout

if needed

Fab/assembly

if needed

Production prototype Checkout Production Testbeam ? Vertical Slice ? 2002 2003 2004 2005