LSE-78 LSST Observatory Network Design Ron Lambert LSST Network - - PowerPoint PPT Presentation

▶
lse 78 lsst observatory network design ron lambert lsst
SMART_READER_LITE
LIVE PREVIEW

LSE-78 LSST Observatory Network Design Ron Lambert LSST Network - - PowerPoint PPT Presentation

May 04, 2023 275 likes •422 views

LSE-78 LSST Observatory Network Design Ron Lambert LSST Network Architect LSST Project and Community Workshop 2018 August 13, 2018 #lsst2018 #lsst2018 LSST Project and Community Workshop 2018 Tucson August 13 - 17 LSST Project and

slide-1
SLIDE 1

1

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

LSE-78 LSST Observatory Network Design Ron Lambert LSST Network Architect LSST Project and Community Workshop 2018 August 13, 2018

slide-2
SLIDE 2

2

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

  • Covers the Long-Haul Network for LSST to enable the mission.
  • Has been dramatically amended over the years of development from 2005

to 2018 to design the flow of traffic from the summit on Cerro Pachon through Santiago to Miami to NCSA.

  • The links and distances include
  • Summit to Base 85Km
  • La Serena to Santiago 700Km
  • Santiago to Miami 7000Km
  • Miami to NCSA 2300Km
  • These are the necessary links to satisfy the data flows required from

summit to NCSA

  • Does not include details of Local Area Networks in Summit and Base

Facilities (refer to LSE-309)

LSE-78 LSST Observatory Network Design

slide-3
SLIDE 3

3

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018 Summit Site

Summit Facility

Telescope and Camera Data Acquisition Crosstalk Correction

Networks and Path Diversity between Sites and Centers

Base Site

Base Center

Long-term storage (copy 1)

Data Access Center

Data Access and User Services

HQ Site

HQ Facility

Observatory Management Science Operations Education and Public Outreach

LSST Data Facility

Archive Center

Alert Production Data Release Production Calibration Products Production EPO Infrastructure Long-term Storage (copy 2)

Data Access Center

Data Access and User Services

French Site

Satellite Processing Center

Data Release Production

slide-4
SLIDE 4

4

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

Nightly International Data Flows

LDF Site Base Site VOEvent Broker Server in La Serena

(Non-DM)

10M Alerts (average) per night

60 second requirement stops here VOEvent Broker Server

(Non-DM)

OCS/TCS Data (up to 1 Gbps) Engineering Facility DB Data Quality Metrics Atmospheric Telescope Images and Spectra DM Data (up to 39 Gbps) Northbound Southbound Crosstalk Images Calibration Images Raw Images DIAObject Catalog Wavefront Images DIASource Catalog Raw Calibration Images

Raw and Crosstalk Corrected Images

24 - 30 TB / night 6.4 GB per image

slide-5
SLIDE 5

5

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

Non-Nightly International Data Flows

LDF Site Base Site US DAC

Calibration products L2 Data Products Eastbound Westbound Raw images 1/2 of L2 Data Products Calibration products Engineering Facility Database 1/2 of L2 Data Products (coadds, catalogs, SDQA)

Backup: Physical Shipments LDF Site to Base Site Hardware New and Replacement Hardware (annually) Data Release loaded (secondary method)

Chilean DAC French Site (satellite processing center) EPO Center

Calibration products L2 Data Products L2 Data Products Data Release: Coadds Object catalog Source catalog Forced Source catalog

slide-6
SLIDE 6

6

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

Tucson La Serena Cerro Pachon Santiago Sao Paolo Chicago

40x10Gb AURA (Science Data) 20x10Gb AURA (OCS/TCS Control)

Panama

1 x 100Gb 1 x 40 –100 Gb REUNA 2 x 100Gb Amlight/ANSP/ RedCLARA

CC-IN2P3 Lyon Miami

100Gb (spectrum) Amlight/Angola Cable

Champaign

2 * 100Gb NCSA/ ICCN

Santiago Boca Raton

10Gb Amlight/internet2 10Gb IN2P3 10x10Gb Non-LSST AURA (Fail-over) 100Gb (spectrum) AmLight/RNP/RedCLARA (via Buenos Aires)

Secondary Path Primary Path SLAC, Other DOE

10Gb ESnet (institutional) 2x 100Gb Esnet/FLR/i2

LSST Baseline Long Haul Network Links

slide-7
SLIDE 7

7

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

Baseline Allocation of Bandwidth

NOTES: [1] Guaranteed reservation [S] Spectrum [2] Expected use, Prioritized/QoS [3] Carried on AURA circuits, not LSST. Fail-over from AURA to LSST is not baselined. [4] Combinations of Cases use numbers from lowest performance case [5] All numbers are stated in line speed reservation, efficiency may lower actual delivered BW "[6] Cases of total network failure"" on a link, i.e. all paths connecting two points are not shown. Those cases rely on data buffering and catch-up."

slide-8
SLIDE 8

8

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

  • Segment 1: “Mountain-Base”
  • From Cerro Pachon to La Serena computing facility, 6 x 100 Gbs

(plus 2x100Gbs AURA shared)

  • Primary circuit on LSST DWDM
  • Secondary “fail-over” backup circuit on AURA DWDM
  • Two fiber sub-segments, single path
  • Summit-Gatehouse (dedicated fibers on AURA property) = 30km
  • Gatehouse-La Serena (dedicated fibers in public cable) = 55km
  • Segment 2: La Serena-Santiago
  • From La Serena computing facility to connection point in

Santiago with international links

  • Level 3 facility, exploring redundancy in Santiago
  • Primary link is dedicated fibers in public cable (100 Gbps)
  • Secondary link is shared leased capacity 100Gbs likely, 40Gbs

worst case

Network Segments Baseline

slide-9
SLIDE 9

9

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

  • Segment 3 : International links
  • Provided by Amlight/RNP/CLARA (100 Gbps each)
  • Santiago – Panama – Miami west coast
  • Santiago – Sao Paolo – Miami east coast
  • Santiago – Boca Raton – Miami
  • Segment 4: US links
  • Utilizing Internet-2 and ESnet
  • Agreement with Esnet for 2 x 100 Gbps Atlanta – Chicago in

work

  • NCSA provides Chicago – Champaign (LDF)

Network Segments Baseline (cont.)

slide-10
SLIDE 10

10

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

AMPATH International Links

slide-11
SLIDE 11

11

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

Bandwidth Timeline: LDM-142

NOTES: NxM notation indicates N connections of M speed. All bandwidths are in gigabits/second (Gb/s). All allow burst up to unused capacity [1] best/lworst case presumes continued IRNC, international REN investments, and industrial partnerships through AMPATH/AMLIGHT likely case presumes only continued IRNC investment, worst case assumes only LSST support. [2] This is proposed. IN2P3 provides this and will match the other links in capacity to the degree possible. [3] Prior baseline funding for 200 Gbps was sufficient to acquire 600 Gbps for same cost This is for LSST side only, AURA side is also available for fail-over 200 Gbps [4] Requires Telefonica to upgrade equipment on this link. This is their main trunk on this route, so likely will be done, but no schedule. [5] AMLight will have 2x100 Gbps fiber links. LSST to buy more 1x100 line and 10x10 client interfaces in the DWDM equipment at REUNA on this link to go beyond 10x10. [6] Gap between Commissioning and Operations is schedule contingency for Construction/Commisssioning. Operations will start sooner if this contingency is not used. [7] Leased/shared capacity [8] Lambda/dedicated capacity [9] Bandwidth via RNP/CLARA [10] LSST DWDM is 6 x 100, AURA DWDM is 2x 100 [11] Via Boca Raton [12] B/W is to clients, users. Wired LAN connections in computer rooms are 10G from switches to servers, 100G between switches. WAPs to access switches are 5G, between acess switches 10G.

slide-12
SLIDE 12

12

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

  • Accepted fibers
  • Summit – AURA Gatehouse
  • AURA Gatehouse – La Serena
  • La Serena - Santiago
  • Installed Dense Wave Division Multiplex (DWDM) equipment
  • n Cerro Pachon, in La Serena, and in Santiago
  • Conducted First Optic Light Demonstration
  • Activated 100 Gbps Ring Santiago – Miami
  • Installed temporary network and phone service on summit

Recent Accomplishments

slide-13
SLIDE 13

13

LSST Project and Community Workshop 2018 • Tucson • August 13 - 17

#lsst2018

  • Fail-over testing of Cisco ACI-based Summit Network
  • Connect Summit Network to DWDM
  • Update obsolete Microwave equipment
  • Voice, web, mail backup, not for data, control
  • DWDM procurement for 100 Gbps Spectrum/activate link
  • Design of Base Data Center in La Serena
  • Agreement with Esnet to provide 2 x 100 Gbps links Atlanta –

Chicago

  • Maintenance agreement for fiber on AURA property

Work in Progress