Building a LAN to Support Multiple Lightpath Projects Ronald van - - PowerPoint PPT Presentation

▶
building a lan to support multiple lightpath projects
SMART_READER_LITE
LIVE PREVIEW

Building a LAN to Support Multiple Lightpath Projects Ronald van - - PowerPoint PPT Presentation

Mar 23, 2024 38 likes •211 views

Building a LAN to Support Multiple Lightpath Projects Ronald van der Pol <rvdp@sara.nl> E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands rvdp@sara.nl About SARA Computing and Networking services Houses and operates national

slide-1
SLIDE 1

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

Building a LAN to Support Multiple Lightpath Projects

Ronald van der Pol

<rvdp@sara.nl>

slide-2
SLIDE 2

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

About SARA

Computing and Networking services Houses and operates national supercomputer Huygens Houses and operates national cluster Lisa LightHouse (joint lab of SARA, UvA and SURFnet for

  • ptical networking experiments and demos)

SURFnet's subcontractor for SURFnet6 NOC SURFnet's subcontractor for NetherLight NOC One of the co-location sites of the AMS-IX CERN LHC Tier-1 site LOFAR Tier-1 site

slide-3
SLIDE 3

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

LHC OPN Tier-1 Site

slide-4
SLIDE 4

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

LOFAR Tier-1 Site

LOw Frequency ARray Radiotelescope Consists of Sensor Fields Data Storage @ SARA

slide-5
SLIDE 5

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

IMAU Climate Model

Rendering at SARA Visualization at IMAU Connected with a SURFnet6 1G lightpath

slide-6
SLIDE 6

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

Traditional ISP Connection

SURFnet SARA router router router router Layer 3 IP interconnect

slide-7
SLIDE 7

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

Introduction of Lightpaths

SURFnet6 Hybrid Network router router router router SARA ? ? ?

slide-8
SLIDE 8

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

Lightpath Challenges

Interconnect sites at L2 or at L3? How to handle security? How to handle addressing? How to protect against configuration errors and accidents at other site?

slide-9
SLIDE 9

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

L2 versus L3

L2 pros

Cheap Ethernet switches

L2 cons

No IP ACLs Mixing of administrative domains  One broadcast domain, one IP subnet

L3 pros

Well-known (we know how to do this between sites) Supports ACLs and firewall Easier fault resolution  Ping, traceroute, router reachability

L3 cons

Routers (and L3 switches) usually more expensive

slide-10
SLIDE 10

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

SARA's Requirements

Keep services separated

Access to one service does not mean access to another service, unless explicitly allowed

No (accidental) connectivity between lightpaths via SARA No (accidental) Internet connectivity via SARA Solution must scale to multiple services and multiple lightpath peer sites Solution must support multiple 10G connections No big routing tables on the servers

Only a default gateway

Segmenting the routing tables

e.g. No LHCOPN prefixes in global routing table

slide-11
SLIDE 11

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

Problems Encountered in LHCOPN

Only storage servers traffic allowed on the LHCOPN Other hosts and servers must reach CERN via Internet Traditional destination based routing does not work We needed to find a good, scalable solution Internet LHCOPN CERN SARA SARA router Data storage

slide-12
SLIDE 12

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

SARA's Choices

Interconnect at L3

L2 only for few very simple cases

BGP routing

BGP detects when peer is unreachable BGP needed when there are multiple paths

Routing segmentation

Put each lightpath project in its own virtual router Good way to keep projects and services separated

slide-13
SLIDE 13

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

Virtual Routing

if1 if2 if3 if4 if5 if6 if7 if8 Storage cluster Render cluster LHCOPN LHCOPN LHCOPN Internet IMAU LOFAR

Global Table: if1, if4, if5 VR1 (LHCOPN): if6, if7, if8 VR2 (IMAU): if2 VR3 (LOFAR): if3

slide-14
SLIDE 14

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

Virtual Router Solution

Virtual routing is a scalable way to keep services and lightpath peers separated Problem with traditional destination based routing + ACLs:

ACLs are difficult to maintain Not a scalable solution Configuration errors mean unwanted access

Problem with policy based routing:

Only 1 next hop, does not work with multiple links Next hop is specified as specific interface Does not use BGP, no route information exchange No link failure detection when switches in path

slide-15
SLIDE 15

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

Problems Encountered

Often little BGP knowledge at peer sites Many peer sites do not have a global AS Most routers have insufficient Virtual Routing capabilities We had to gain knowledge of virtual routing Detecting of link failures often difficult

Link failures do not propagate through Ethernet switches (BGP session, 802.1ag, BFD, ...)

slide-16
SLIDE 16

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

Conclusions

Supporting multiple lightpaths and multiple services is not a trivial task Virtual routing is a relatively simple way to handle the routing and separation requirements Routing requirements often result in the choice for BGP

slide-17
SLIDE 17

rvdp@sara.nl E2E Workshop, 1-2 Dec, Amsterdam, The Netherlands

Thank You

Ronald van der Pol

rvdp@sara.nl