NL-T1 Expectations, findings, and innovation Bas Kreukniet, Sr - - PowerPoint PPT Presentation

▶

Sep 06, 2022 571 likes •795 views

NL-T1 Expectations, findings, and innovation Bas Kreukniet, Sr Network Specialist at SURF SARA Geneva Workshop 10 Februari 2014 Outline 1. Expectations from NL-T1 grid administrators 2. Findings while connecting to the LHCONE 3. Innovation:

SLIDE 1

Bas Kreukniet, Sr Network Specialist at SURFSARA

NL-T1 Expectations, findings, and innovation

Geneva Workshop 10 Februari 2014

SLIDE 2

Outline

1. Expectations from NL-T1 grid administrators
2. Findings while connecting to the LHCONE
3. Innovation: Ethernet OAM and NSI

SLIDE 3

Expectations from NL-T1 administrators Advice from grid administrators NL-T1:

Bulk data with simple applications (grid-FTP).

“Keep it simple”

Network provisioning from application is

considered “complex”

Don’t rush to merge LHCOPN with LHCONE

SLIDE 4

NL-T1 connected to LHCONE since 20 Jan 2014

SLIDE 5

BGP routing starts at the connected site. BGP routing for T1’s: Not only a “NREN thing” – it already starts at your

rganisation!

Focus on some BGP topics for connecting sites to LHC networks

SLIDE 6

LHCOPN connectivity

SLIDE 7

LHCONE connectivity

128.142.0.0/16 *[BGP/170] 1w2d 04:07:47, MED 120, localpref 100 AS path: 20965 20641 513 I > to 62.40.126.161 via xe-4/0/1.2012

NL-T1 AS1162 GÉANT AS20965 LHCONE-RS CERN AS20641 CERN AS513 LHCONE.inet.0: 133 destinations, 133 routes (133 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both

SLIDE 8

Preferred route: LHCOPN, LHCONE or Internet

primary: direct T1–T1 over LHCOPN
secondary: T1–T1 over LHCOPN via another T1
tertiary: LHCONE
quaternary: Internet

SLIDE 9

Preferred route: route (a)symmetry

If everyone makes this choice, all connections are symmetric. But: sites may have different preferences:

10 Gb/s 10 Gb/s 1 Gb/s 100 Gb/s site 1 site 2 ISP-B ISP-A

SLIDE 10

Tie-breaker between LHCOPN and LHCONE CNAF routers at NL-T1

LHCOPN

131.154.128.0/17 *[BGP/170] 14:58:18, localpref 100 AS path: 34878 137 I

LHCONE

131.154.128.0/17 *[BGP/170] 1w2d 04:23:40, MED 120, localpref 100 AS path: 20965 137 I

SLIDE 11

LHCOPN / LHCONE route preference BNL routes at NL-T1

LHCOPN

130.199.185.0/24 *[BGP/170] 2w5d 09:35:43, MED 10, localpref 100 AS path: 513 43 I > to 192.16.166.73 via xe-1/1/0.0 [BGP/170] 3d 01:15:22, MED 51, localpref 100 AS path: 39590 513 43 I > to 109.105.124.17 via xe-2/1/0.0

LHCONE

130.199.185.0/24 *[BGP/170] 1w2d 04:27:43, MED 120, localpref 100 AS path: 20965 293 43 I > to 62.40.126.161 via xe-4/0/1.2012

SLIDE 12

Problems and concerns regarding BGP

A site advertising his routes has no control who to

send a route to. At best they can give hints with BGP communities.

The site receiving a route decides which route to

accept and how to accept.

SLIDE 13

BGP in LHC networks

Only As is not enough. More info needed.  community for origin of a route T1/T2? Or even better:

site connected to LHCOPN
site connected to LHCONE
Site connected to both:

Type A “prefer LHCONE for this route” Type B “prefer LHCOPN for this route”

Specials: dedicated link between two (T1) sites.

(“VPN”or “private link”)

SLIDE 14

Route Preference Solutions

Idea: Tag routes with two types of communities:
One for origin or source
One for destinations
See also: BGP hinting by Martin Sweeny (Indiana U)
BGP Always-compare-MED always on. We sometimes

add metrics on incoming routes.

SLIDE 15

Operational Issues

We received routes over LHC from a site,

but traffic we send is blackholed by that site

The site was still reachable over the Internet.
This happened to us twice recently: on LHCOPN (accidental route

redistribution) and LHCONE (incoming IP filter).

Configuration errors will be made (we’re also just human)
LHCOPN Link NL-T1 – TRIUMF link is still unstable
31 outages last 4 months

 Monitoring remains important!

SLIDE 16

LHCONE and LHCOPN layer 2 monitoring

Ethernet OAM monitoring or Layer 2 monitoring NIKHEF, TRIUMF to participate as Measurement Points Looking for T1’s and T2 to participate Advantages:

Layer2 keep-alive, ping and traceroute
Interdomain, intervendor solution
L2 devices can be made visible
unidirectional fibercuts can be signalled

SLIDE 17

LHCONE Innovation NL-T1 likes to partcipate in NSI for LHCONE NSI experiences so far:

Use-case: Life Science Grid (LSG) in Holland

makes use of NSI since autumn 2013.

Freek contributed to standard
Sander wrote NSI client and implemented it for

“Cloud Bypassing” in the Life Science Grid.

SLIDE 18

Cloud Bypassing on Life Science Grid (LSG)

Regular Internet

RUG campus network SARA data center network

default d e f a u l t

Life Science grid cluster @ SURFsara Life Science grid cluster @ RUG

campus network

d e f a u l t d e f a u l t

Life Science grid cluster @ AMC Life Science grid cluster @ TUD

AMC campus network

Compute clusters at ±10 locations in the Netherlands

SLIDE 19

Cloud Bypassing on Life Science Grid (LSG)

SURFnet dynamic lightpaths

e s p e c i f i c ( B G P ) More specific (BGP) More specific (BGP) More specific (BGP)

Regular Internet

RUG campus network SARA data center network

default d e f a u l t

Life Science grid cluster @ SURFsara Life Science grid cluster @ RUG

campus network

d e f a u l t d e f a u l t

Life Science grid cluster @ AMC Life Science grid cluster @ TUD

AMC campus network

Offloading campus networks (some only have 1 Gb/s Internet)

SLIDE 20

LHCONE Innovation: Offloading is cheaper

Internet full routing (incl backup): € 8k – €10k per month for 10 Gb/s LHCONE or dynamic lightpath: € 2k – 3k per month for 10 Gb/s

SLIDE 21

Bas Kreukniet, Sr Network Specialist at SURFSARA

NL-T1 Expectations, findings, and innovation

Outline

Expectations from NL-T1 administrators Advice from grid administrators NL-T1:

“Keep it simple”

considered “complex”

NL-T1 connected to LHCONE since 20 Jan 2014

BGP routing starts at the connected site. BGP routing for T1’s: Not only a “NREN thing” – it already starts at your

Focus on some BGP topics for connecting sites to LHC networks

LHCOPN connectivity

LHCONE connectivity

NL-T1 AS1162 GÉANT AS20965 LHCONE-RS CERN AS20641 CERN AS513 LHCONE.inet.0: 133 destinations, 133 routes (133 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both

Preferred route: LHCOPN, LHCONE or Internet

Preferred route: route (a)symmetry

If everyone makes this choice, all connections are symmetric. But: sites may have different preferences:

10 Gb/s 10 Gb/s 1 Gb/s 100 Gb/s site 1 site 2 ISP-B ISP-A

Tie-breaker between LHCOPN and LHCONE CNAF routers at NL-T1

LHCOPN

LHCONE

LHCOPN / LHCONE route preference BNL routes at NL-T1

LHCOPN

LHCONE

Problems and concerns regarding BGP

send a route to. At best they can give hints with BGP communities.

accept and how to accept.

BGP in LHC networks

Only As is not enough. More info needed.  community for origin of a route T1/T2? Or even better:

Type A “prefer LHCONE for this route” Type B “prefer LHCOPN for this route”

(“VPN”or “private link”)

Route Preference Solutions

add metrics on incoming routes.

Operational Issues

but traffic we send is blackholed by that site

redistribution) and LHCONE (incoming IP filter).

 Monitoring remains important!

LHCONE and LHCOPN layer 2 monitoring

Ethernet OAM monitoring or Layer 2 monitoring NIKHEF, TRIUMF to participate as Measurement Points Looking for T1’s and T2 to participate Advantages:

LHCONE Innovation NL-T1 likes to partcipate in NSI for LHCONE NSI experiences so far:

makes use of NSI since autumn 2013.

“Cloud Bypassing” in the Life Science Grid.

Cloud Bypassing on Life Science Grid (LSG)

Compute clusters at ±10 locations in the Netherlands

Cloud Bypassing on Life Science Grid (LSG)

Offloading campus networks (some only have 1 Gb/s Internet)

LHCONE Innovation: Offloading is cheaper

Questions?

Erik Ruiter Bas Kreukniet Diederik Vandevenne Sander Boele Farhad Davani Freek Dijkstra